Notice: For the latest information and guidance on Purdue's response to COVID-19 please visit:

Stochastic Models In Operations Research I


Credit Hours:


Learning Objective:

An introduction to techniques for modeling random processes used in operations research. Markov chains, continuous time Markov processes, Markovian queues, reliability, Martingales, and Brownian motion.


An introduction to techniques for modeling random processes used in operations research - Markov chains, continuous time Markov processes, Markovian queues, Martingales, Optimal Stopping/Optional Stopping Theorem, Brownian Motion, Option Pricing.

Topics Covered:

1.Conditional Probability and Expectation: variance; discrete and continuous case
2.Markov chains: stochastic processes; transition probabilities; transient and recurrent states; limiting probabilities; mean time spent; pattern times; branching processes
3. Exponential Distribution and Poisson Process: properties of memoryless distribution; interarrival and waiting time; conditional distribution of arrival times
4.Continuous time Markov chains: transition rates and approximations; Laplace transforms; steady-state behavior and convergence theorem; birth-death process
5.Reliability: k out of n systems; Serial Repair; Parallel Repair
6.Queuing theory: Little's Formula; steady-state and limiting probabilities; Queuing system with bulk service; network of queues; generalizations; closed queuing systems
7.Brownian Motion: Random Walk; Hitting Times; Martingales; Optimal Stopping; Option Pricing


Course prerequisites are MATH 262 AND (IE 230 OR MATH 416) . A strong working knowledge of probability theory (e.g. MATH 416), basic linear algebra (e.g., MATH 265) and calculus (e.g. MATH 266) is highly recommended. Random variables will not be taught (even though there will be a quick revision), and the students will be expected to know them before-hand (Reference book Ross Chapter 1-2).

Applied / Theory:

40 / 60

Web Address:

Web Content:

Syllabus, grades, lecture notes, and solutions.


There will be an assigned homework every two weeks, and is due back in a week. Return homework to


Optional Project: Each student has a flexibility to do a research project. The project will be worth 40%, and the maximum of the marks from the project and the final exam would be chosen as the marks for the final exam. If the student however does exceptionally well on both the exam and the project, up to 5% bonus marks can be added at the discretion of the Instructor. If interested, you must notify the instructor of your intent within the following three weeks. There will be a presentation scheduled for the project.


2 midterm exams and 1 final exam.


Required:Introduction to Probability Models, 11th Edition, Sheldon Ross, Academic Press, ISBN:9780124079489
Introduction to Stochastic Processes with R, 1st Edition, Robert Dobrow, Wiley, ISBN:9781118740651

Computer Requirements:

ProEd Minimum Computer Requirements

ProEd Minimum Requirements: