1. Review of Probability: Discrete and Continuous Random Variables, Expected Value, Independence, Joint Probability Distribution, Functions of Random Variables, Moment Generating Functions, Limit Theorems, Conditional Probability and Conditional Expectation, Computing Probabilities by Conditioning, Computing Expectations by Conditioning.
   
   
2. Stochastic Processes; stationarity, ergodicity, Introduction to Martingales, Gaussian and Markovian Processes.
   
   
3. Markov Chains: Chapman-Kolmogorov Equations, Classification of States, Limiting Probabilities.
   
   
4. Markov Chains: Absorbing Chains, First Step Analysis, Branching Processes.
   
   
5. Time Reversible Markov Chains, Markov Decision Processes.
   
   
6. Poisson Process: Interarrival and Waiting Time Distributions, Properties, Nonhomogenous Poisson Process, Compound Poisson Process.
   
   
7. Continuous-Time Markov Chains: Birth and Death Process, Limiting Probabilities, Time Reversibility, Uniformization, Birth and Death Queues.
   
   
8. Renewal Theory and Its Applications: Distribution of N(t), Limit Theorems, Renewal Reward Process, Regenerative Process, Semi-Markov Processes, The Inspection Paradox.
   
   
9. Stochastic Modeling Applications.
   
   

1. “Introduction to Stochastic Modeling”, H. M. Taylor and S. Karlin, AP, 3rd Ed., 1998.
   
   
2. “Introduction to Probability Models”, Sheldon Ross, AP, 9th Ed., 2007.
   
   
3. “Stochastic Modeling and the Theory of Queues”, Ronald Wolff, Prentice Hall, 1989.
   
   
4. “Stochastic Processes”, Sheldon Ross, Wiley, 2nd Ed., 1996.
   
   
5. “Introduction to Stochastic Processes”, Erhan Cinlar, Prentice Hall, 1975.
   
   
6. “A First Course in Stochastic Processes”, S. Karlin and H. M. Taylor, AP, 1975.
   
   

30% Midterm Exam, 30% HWs and Project, 40% Final Exam

You will be provided with a cheat sheet.

If you have a disability that influences testing procedures, provide me an official letter from the Office of Disability Services at the start of the semester.

All exams are closed book, closed notes, and no cell phones, no electronic or hard copy cheat sheets are allowed. Please go over the Rutgers Academic Integrity Policy.

I do NOT tolerate cheating. Students are responsible for understanding the RU Academic Integrity Policy. I will strongly enforce this Policy and pursue all violations. On all examinations and assignments, students must sign the RU Honor Pledge, which states, “On my honor, I have neither received nor given any unauthorized assistance on this examination or assignment.” Don’t let cheating destroy your hard-earned opportunity to learn.