• Introduction and review of basic stochastic processes
• Renewal theory, Renewal reward process, regenerative process, applications in queueing and reliability
• Markov decision processes, applications in congestion control, asset pricing
• Martingales, stopping times, optional sampling theorem and its Applications
• Random walks, exchangeable random variables
• Applications to G/G/1 queue and ruin problem
• Brownian motion: reflected, geometric, integrated; diffusion, semi-Markov processes
• Applications
  
  

“ Stochastic Processes”, by R. F. Bass, Cambridge University Press, 2011.

“ A First Course in Stochastic Processes”, by S. Karlin and H. M. Taylor, 2nd Ed., Academic Press, 1975.

“ A Second Course in Stochastic Processes”, by S. Karlin and H. M. Taylor, Academic Press, 1981.

“ Stochastic Processes”, by J. Doob, Wiley, 1953.

“ A First Course in Stochastic Models”, by H. C. Tijms, , Wiley, 2003.

“ Stochastic Processes”, by Sheldon Ross, 2nd Ed., Wiley, 1996.

The course will include class projects related to stochastic modeling and homework assignments. There will be no exams.