Rutgers University
College of Engineering
Department of Industrial Engineering
540:515 Stochastic Models in ISE
Instructor: Dr. M. B. Gürsoy
Office: CoRE 218
Phone: X-5465
Topics to be covered:
    1. Review of Probability: Discrete and Continuous Random Variables, Expected Value, Independence, Joint Probability Distribution, Functions of Random Variables, Moment Generating Functions, Limit Theorems, Conditional Probability and Conditional Expectation, Computing Probabilities by Conditioning, Computing Expectations by Conditioning.

    2. Stochastic Processes; stationarity, ergodicity, Introduction to Martingales, Gaussian and Markovian Processes.

    3. Markov Chains: Chapman-Kolmogorov Equations, Classification of States, Limiting Probabilities.

    4. Markov Chains: Absorbing Chains, First Step Analysis, Branching Processes.

    5. Time Reversible Markov Chains, Markov Decision Processes.

    6. Poisson Process: Interarrival and Waiting Time Distributions, Properties, Nonhomogenous Poisson Process, Compound Poisson Process.

    7. Continuous-Time Markov Chains: Birth and Death Process, Limiting Probabilities, Time Reversibility, Uniformization, Birth and Death Queues.

    8. Renewal Theory and Its Applications: Distribution of N(t), Limit Theorems, Renewal Reward Process, Regenerative Process, Semi-Markov Processes, The Inspection Paradox.

    9. Stochastic Modeling Applications.


“Modeling and Analysis of Stochastic Systems", V. G. Kulkarni, CRC, 2nd Edition, 2010

    1. “Introduction to Stochastic Modeling”, H. M. Taylor and S. Karlin, AP, 3rd Ed., 1998.

    2. “Introduction to Probability Models”, Sheldon Ross, AP, 9th Ed., 2007.

    3. “Stochastic Modeling and the Theory of Queues”, Ronald Wolff, Prentice Hall, 1989.

    4. “Stochastic Processes”, Sheldon Ross, Wiley, 2nd Ed., 1996.

    5. “Introduction to Stochastic Processes”, Erhan Cinlar, Prentice Hall, 1975.

    6. “A First Course in Stochastic Processes”, S. Karlin and H. M. Taylor, AP, 1975.


30% Midterm Exam, 30% HWs and Project, 40% Final Exam

※ All exams are closed book closed notes, no electronic or hard copy cheat sheets are allowed. Please go over the Rutgers Academic Integrity Policy.