# Stochastic Processes

### Course Description

Probability Review and Introduction to Stochastic Processes (SPs): Probability spaces, random variables and probability distributions, expectations, transforms and generating functions, convergence, LLNs, CLT.
Definition, examples and classification of random processes according to state space and parameter space.
Stationary Processes: Weakly stationary and strongly stationary processes, moving average and auto regressive processes
Discrete-time Markov Chains (DTMCs): Transition probability matrix, Chapman-Kolmogorov equations; n-step transition and limiting probabilities, ergodicity, stationary distribution, random walk and gambler’s ruin problem, applications of DTMCs.
Continuous-time Markov Chains (CTMCs): Kolmogorov differential equations for CTMCs, infinitesimal generator, Poisson and birth-death processes, stochastic Petri net, applications to queueing theory and communication networks.
Martingales: Conditional expectations, definition and examples of martingales, applications in finance.
Brownian Motion: Wiener process as a limit of random walk; process derived from Brownian motion, stochastic differential equation, stochastic integral equation, Ito formula, Some important SDEs and their solutions, applications to finance.
Renewal Processes: Renewal function and its properties, renewal theorems, cost/rewards associated with renewals, Markov renewal and regenerative processes, non Markovian queues, applications of Markov regenerative processes.
Branching Processes: Definition and examples branching processes, probability generating function, mean and variance, Galton-Watson branching process, probability of extinction.

A stochastic process is a series of trials the results of which are only probabilistically determined. Examples of stochastic processes include the number of customers in a checkout line, congestion on a highway, and the price of a financial security. (Source: http://www.talkativeman.com/definition-stochastic-processes/)
Not yet rated

### Video Lectures & Study Materials

Visit the official course website for more study materials: http://nptel.ac.in/syllabus/111102014/

# Lecture Play Lecture
I. Review of Probability Theory
1 Introduction to Stochastic Processes (55:11) Play Video
2 Introduction to Stochastic Processes (Contd.) (59:10) Play Video
3 Problems in Random Variables and Distributions (48:40) Play Video
4 Problems in Sequences of Random Variables (41:18) Play Video
II. Definition and Simple Stochastic Processes
5 Definition, Classification and Examples (50:35) Play Video
6 Simple Stochastic Processes (57:02) Play Video
III. Stationary and Auto Regressive Processes
7 Stationary Processes (54:37) Play Video
8 Autoregressive Processes (1:02:14) Play Video
9 Introduction, Definition and Transition Probability Matrix (56:01) Play Video
10 Chapman-Kolmogrov Equations (56:45) Play Video
IV. Discrete-time Markov Chain
11 Classification of States and Limiting Distributions (51:14) Play Video
12 Limiting and Stationary Distributions (59:39) Play Video
13 Limiting Distributions, Ergodicity and Stationary Distributions (48:25) Play Video
14 Time Reversible Markov Chain (56:31) Play Video
15 Reducible Markov Chains (55:41) Play Video
V. Continuous-time Markov Chain
16 Definition, Kolmogrov Differential Equations and Infinitesimal Generator Matrix (55:24) Play Video
17 Limiting and Stationary Distributions, Birth Death Processes (58:36) Play Video
18 Poisson Processes (56:09) Play Video
19 M/M/1 Queueing Model (56:23) Play Video
20 Simple Markovian Queueing Models (58:03) Play Video
21 Queueing Networks (58:43) Play Video
22 Communication Systems (51:18) Play Video
23 Stochastic Petri Nets (58:01) Play Video
VI. Martingales
24 Conditional Expectation and Filtration (48:45) Play Video
25 Definition and Simple Examples (55:51) Play Video
VII. Brownian Motion and its Applications
26 Definition and Properties (46:41) Play Video
27 Processes Derived from Brownian Motion (39:29) Play Video
28 Stochastic Differential Equations (47:38) Play Video
29 Ito Integrals (50:15) Play Video
30 Ito Formula and its Variants (39:53) Play Video
31 Some Important SDE`s and Their Solutions (39:31) Play Video
VIII. Renewal Processes
32 Renewal Function and Renewal Equation (46:48) Play Video
33 Generalized Renewal Processes and Renewal Limit Theorems (37:58) Play Video
34 Markov Renewal and Markov Regenerative Processes (1:01:08) Play Video
35 Non Markovian Queues (39:39) Play Video
36 Non Markovian Queues Cont,, (44:25) Play Video
37 Application of Markov Regenerative Processes (47:43) Play Video
IX. Branching Processes
38 Galton-Watson Process (43:48) Play Video
39 Markovian Branching Process (46:06) Play Video