Virtual Learning Environment (VLE) for

Stochastic Processes (F79SP1)
Part 1: Discrete-time Markov chains and simple random walks
Autumn 2009

Click here for multimedia demonstrations.
How to study for the exam
NEW: Some of you want to know EXACTLY what to study for the exam. So I have color-coded the pages of the notes as follows:
BRIGHT YELLOW PARTS: you need to study them
VERY DARK PARTS: you don't need this for the exam
GREEN PARTS: this is background material

Schedule

Campus map
(Ignore entries with small letters}
Tuesday 11:20-12:10 / LT2 / Lecture and examples
.... Tuesday 13:20-14:10 / EM182 / Tutorials on quantitative methods
.... Tuesday 16:20-17:10 / WA211 / Tutorials on quantitative methods
.... .... .... EofP

.... Wednesday 09:20-10:10 / WA208 / Tutorials on quantitative methods
Wednesday 10:20-11:10 / SR214 / Exercises and supplements
.... Wednesday 11:20-12:10 / NS101 / Tutorials on quantitative methods
Wednesday 12:20-13:10 / SR214 / Exercises and supplements
.... .... .... EofP

Thursday 14:20-15:10 / LT2 / Lecture and examples

Friday 11:20-12:10 / LT2 / Lecture and examples

Disclaimer

The disctinction between "tutorials" and "lectures" is rather artificial. I will give lots of "tutorials" (i.e. examples) during lectures and I will also "lecture" during so-called tutorial sessions, especially when I see that there are topics you don't understand. So be sure to attend everything. As I said in class, I have no idea how to teach in the so-called vocational manner that you may have been exposed to, i.e. in telling you recipes and formulae. I only know one way to teach and this is by explaining what I am talking about.

What part 1 of the module covers

Lecture notes

PRIMARY:
Introductory lecture notes on Markov chains and random walks

SECONDARY:
Markov Chains (Grinstead and Snell)
Random Walks (Grinstead and Snell)

Bibliography

C.M. Grinstead and J.L. Snell (1997) Introduction to Probability (Chapters 10, 11 and 12). American Mathematical Society. [The book is available for free.]
P. Bremaud (1999) Markov Chains. Springer.
J.R. Norris (1998) Markov Chains. Cambridge University Press.

Exercises and weekly assignments

You are supposed to solve the assigned exercises before the tutorial sessions. You will be asked to present your solutions to the class.
Homework 1 |Solutions
Homework 2 |Solutions
Homework 3 |Solutions
Homework 4 |Solutions
Homework 5 |Solutions
Homework 6 |Solutions

The tutorial exercises are (mostly) taken from:
A hundred exercises for Stochastic Processes I.

In addition, you can try some of the exercises from the book of Grinstead and Snell (see above). Answers to some of them are here.

Contacts

Instructor: Prof. Takis Konstantopoulos, takis"at"ma.hw.ac.uk

Studying for the exam

Instructions
Past exams (Be aware that this module is examined together with the module on continuous time Markov chains.)



Multimedia demonstrations

Credits: John Bohr, Jim Carlson, Jeff Rosenthal, Charles Stanton

Comments on the organisation, content, omissions, etc., about this web page, are welcome and should be directed to: Prof Takis Konstantopoulos, takis"at"ma.hw.ac.uk