Stochastic Processes

Course Description

Probability Review and Introduction to Stochastic Processes (SPs): Probability spaces, random variables and probability distributions, expectations, transforms and generating functions, convergence, LLNs, CLT.
Definition, examples and classification of random processes according to state space and parameter space.
Stationary Processes: Weakly stationary and strongly stationary processes, moving average and auto regressive processes
Discrete-time Markov Chains (DTMCs): Transition probability matrix, Chapman-Kolmogorov equations; n-step transition and limiting probabilities, ergodicity, stationary distribution, random walk and gambler’s ruin problem, applications of DTMCs.
Continuous-time Markov Chains (CTMCs): Kolmogorov differential equations for CTMCs, infinitesimal generator, Poisson and birth-death processes, stochastic Petri net, applications to queueing theory and communication networks.
Martingales: Conditional expectations, definition and examples of martingales, applications in finance.
Brownian Motion: Wiener process as a limit of random walk; process derived from Brownian motion, stochastic differential equation, stochastic integral equation, Ito formula, Some important SDEs and their solutions, applications to finance.
Renewal Processes: Renewal function and its properties, renewal theorems, cost/rewards associated with renewals, Markov renewal and regenerative processes, non Markovian queues, applications of Markov regenerative processes.
Branching Processes: Definition and examples branching processes, probability generating function, mean and variance, Galton-Watson branching process, probability of extinction.

Stochastic Processes
A stochastic process is a series of trials the results of which are only probabilistically determined. Examples of stochastic processes include the number of customers in a checkout line, congestion on a highway, and the price of a financial security. (Source:
Not yet rated

Video Lectures & Study Materials

Visit the official course website for more study materials:

# Lecture Play Lecture
I. Review of Probability Theory
1 Introduction to Stochastic Processes (55:11) Play Video
2 Introduction to Stochastic Processes (Contd.) (59:10) Play Video
3 Problems in Random Variables and Distributions (48:40) Play Video
4 Problems in Sequences of Random Variables (41:18) Play Video
II. Definition and Simple Stochastic Processes
5 Definition, Classification and Examples (50:35) Play Video
6 Simple Stochastic Processes (57:02) Play Video
III. Stationary and Auto Regressive Processes
7 Stationary Processes (54:37) Play Video
8 Autoregressive Processes (1:02:14) Play Video
9 Introduction, Definition and Transition Probability Matrix (56:01) Play Video
10 Chapman-Kolmogrov Equations (56:45) Play Video
IV. Discrete-time Markov Chain
11 Classification of States and Limiting Distributions (51:14) Play Video
12 Limiting and Stationary Distributions (59:39) Play Video
13 Limiting Distributions, Ergodicity and Stationary Distributions (48:25) Play Video
14 Time Reversible Markov Chain (56:31) Play Video
15 Reducible Markov Chains (55:41) Play Video
V. Continuous-time Markov Chain
16 Definition, Kolmogrov Differential Equations and Infinitesimal Generator Matrix (55:24) Play Video
17 Limiting and Stationary Distributions, Birth Death Processes (58:36) Play Video
18 Poisson Processes (56:09) Play Video
19 M/M/1 Queueing Model (56:23) Play Video
20 Simple Markovian Queueing Models (58:03) Play Video
21 Queueing Networks (58:43) Play Video
22 Communication Systems (51:18) Play Video
23 Stochastic Petri Nets (58:01) Play Video
VI. Martingales
24 Conditional Expectation and Filtration (48:45) Play Video
25 Definition and Simple Examples (55:51) Play Video
VII. Brownian Motion and its Applications
26 Definition and Properties (46:41) Play Video
27 Processes Derived from Brownian Motion (39:29) Play Video
28 Stochastic Differential Equations (47:38) Play Video
29 Ito Integrals (50:15) Play Video
30 Ito Formula and its Variants (39:53) Play Video
31 Some Important SDE`s and Their Solutions (39:31) Play Video
VIII. Renewal Processes
32 Renewal Function and Renewal Equation (46:48) Play Video
33 Generalized Renewal Processes and Renewal Limit Theorems (37:58) Play Video
34 Markov Renewal and Markov Regenerative Processes (1:01:08) Play Video
35 Non Markovian Queues (39:39) Play Video
36 Non Markovian Queues Cont,, (44:25) Play Video
37 Application of Markov Regenerative Processes (47:43) Play Video
IX. Branching Processes
38 Galton-Watson Process (43:48) Play Video
39 Markovian Branching Process (46:06) Play Video


There are no comments. Be the first to post one.
  Post comment as a guest user.
Click to login or register:
Your name:
Your email:
(will not appear)
Your comment:
(max. 1000 characters)
Are you human? (Sorry)
CosmoLearning is promoting these materials solely for nonprofit educational purposes, and to recognize contributions made by Indian Institute of Technology, Delhi (IIT Delhi) to online education. We do not host or upload any copyrighted materials, including videos hosted on video websites like YouTube*, unless with explicit permission from the author(s). All intellectual property rights are reserved to IIT Delhi and involved parties. CosmoLearning is not endorsed by IIT Delhi, and we are not affiliated with them, unless otherwise specified. Any questions, claims or concerns regarding this content should be directed to their creator(s).

*If any embedded videos constitute copyright infringement, we strictly recommend contacting the website hosts directly to have such videos taken down. In such an event, these videos will no longer be playable on CosmoLearning or other websites.