• Introduction and review of basic stochastic processes, CTMCs, queues
• Renewal theory, Wald’s equation, alternating and delayed renewal process
• Renewal reward process, regenerative process, applications in queueing and reliability
• Markov decision processes, applications in congestion control, asset pricing
• Martingales, stopping times
• Optional Sampling Theorem and its Applications
• Midterm exam
• Random walks, exchangeable random variables
• Applications to G/G/1 queue and ruin problem
• Brownian motion, hitting times
• Reflected Brownian motion, geometric, integrated Brownian motion
• Brownian motion with drift, diffusion equations, semi-Markov processes
• Applications
  
  

“ A First Course in Stochastic Processes”, by S. Karlin and H. M. Taylor, 2nd Ed., Academic Press, 1975.

“ A First Course in Stochastic Models”, by H. C. Tijms, , Wiley, 2003.

“ Stochastic Processes”, by Sheldon Ross, 2nd Ed., Wiley, 1996.

30% Midterm Exam, 30% HWs and Project, 40% Final Exam

You will be provided with a cheat sheet.

If you have a disability that influences testing procedures, provide me an official letter from the Office of Disability Services at the start of the semester.

All exams are closed book, closed notes, and no cell phones, no electronic or hard copy cheat sheets are allowed. Please go over the Rutgers Academic Integrity Policy.

I do NOT tolerate cheating. Students are responsible for understanding the RU Academic Integrity Policy. I will strongly enforce this Policy and pursue all violations. On all examinations and assignments, students must sign the RU Honor Pledge, which states, “On my honor, I have neither received nor given any unauthorized assistance on this examination or assignment.” Don’t let cheating destroy your hard-earned opportunity to learn.