COURSE OUTLINE 

Topics to be covered: 
 Review of Probability: Discrete and Continuous Random Variables, Expected Value, Independence, Joint Probability Distribution, Functions of Random Variables, Moment Generating Functions, Limit Theorems, Conditional Probability and Conditional Expectation, Computing Probabilities by Conditioning, Computing Expectations by Conditioning.
 Stochastic Processes; stationarity, ergodicity, Introduction to Martingales, Gaussian and Markovian Processes.
 Markov Chains: ChapmanKolmogorov Equations, Classification of States, Limiting Probabilities.
 Markov Chains: Absorbing Chains, First Step Analysis, Branching Processes.
 Time Reversible Markov Chains, Markov Decision Processes.
 Poisson Process: Interarrival and Waiting Time Distributions, Properties, Nonhomogenous Poisson Process, Compound Poisson Process.
 ContinuousTime Markov Chains: Birth and Death Process, Limiting Probabilities, Time Reversibility, Uniformization, Birth and Death Queues.
 Renewal Theory and Its Applications: Distribution of N(t), Limit Theorems, Renewal Reward Process, Regenerative Process, SemiMarkov Processes, The Inspection Paradox.
 Stochastic Modeling Applications.

Text: 
“Modeling and Analysis of Stochastic Systems", V. G. Kulkarni, CRC, 2nd Edition, 2010

References: 
 “Introduction to Stochastic Modeling”, H. M. Taylor and S. Karlin, AP, 3rd Ed., 1998.
 “Introduction to Probability Models”, Sheldon Ross, AP, 9th Ed., 2007.
 “Stochastic Modeling and the Theory of Queues”, Ronald Wolff, Prentice Hall, 1989.
 “Stochastic Processes”, Sheldon Ross, Wiley, 2nd Ed., 1996.
 “Introduction to Stochastic Processes”, Erhan Cinlar, Prentice Hall, 1975.
 “A First Course in Stochastic Processes”, S. Karlin and H. M. Taylor, AP, 1975.

Grading: 
30% Midterm Exam, 30% HWs and Project, 40% Final Exam

※ All exams are closed book closed notes, no electronic or hard copy cheat sheets are allowed. Please go over the Rutgers Academic Integrity Policy.

