4 Some useful texts
At increasing levels of mathematical sophistication:
Häggström (2002) “Finite Markov chains and algorithmic applications”.
Delightful introduction to finite state-space discrete-time Markov chains, from point of view of computer algorithms.Grimmett and Stirzaker (2001) “Probability and random processes”.
Standard undergraduate text on mathematical probability. If you are going to buy one book on probability, this is a good choice because it contains so much material.Norris (1998) “Markov chains”.
Markov chains at a more graduate level of sophistication, revealing what I have concealed, namely the full gory story about \(Q\)-matrices.Williams (1991) “Probability with martingales”.
Excellent graduate text for theory of martingales: mathematically demanding.
4.1 Free on the web
Doyle and Snell (1984) “Random walks and electric networks”
Available on the web at http://arxiv.org/abs/math/0001057.
Lays out (in simple and accessible terms) an important approach to Markov chains using relationship to resistance in electrical networks.Kindermann and Snell (1980) “Markov random fields and their applications”
Available on the web at https://doi.org/10.1090/conm/001.
Sublimely accessible treatment of Markov random fields (Markov property, but in space not time).Meyn and Tweedie (1993) “Markov chains and stochastic stability”
Available on the web at http://probability.ca/MT/.
The place to go if you need to get informed about theoretical results on rates of convergence for Markov chains (e.g. because you are doing MCMC).Aldous and Fill (2001) “Reversible Markov Chains and Random Walks on Graphs”
Only available on the web at http://www.stat.berkeley.edu/~aldous/RWG/book.html.
The best unfinished book on Markov chains known to us.
4.2 Going deeper
Kingman (1993) “Poisson processes”.
Very good introduction to the wide circle of ideas surrounding the Poisson process.Kelly (1979) “Reversibility and stochastic networks”.
We’ll cover reversibility briefly in the lectures, but this shows just how powerful the technique is.Lindvall (1992) “Lectures on the coupling method”.
We’ll also talk briefly about the beautiful concept of coupling for Markov chains; this book gives a very nice introduction.Steele (2004) “The Cauchy-Schwarz master class”.
The book to read if you decide you need to know more about (mathematical) inequality.Aldous (1989) “Probability approximations via the Poisson clumping heuristic”.
See http://www.stat.berkeley.edu/~aldous/Research/research80.html.
A book full of what ought to be true; hence good for stimulating research problems and also for ways of computing heuristic answers.Øksendal (2003) “Stochastic differential equations”.
An accessible introduction to Brownian motion and stochastic calculus, which we do not cover at all.Stoyan, Kendall, and Mecke (1987) “Stochastic geometry and its applications”.
Discusses a range of techniques used to handle probability in geometric contexts.