4 Some useful texts

At increasing levels of mathematical sophistication:

  • Häggström (2002) “Finite Markov chains and algorithmic applications”.
    Delightful introduction to finite state-space discrete-time Markov chains, from point of view of computer algorithms.

  • Grimmett and Stirzaker (2001) “Probability and random processes”.
    Standard undergraduate text on mathematical probability. If you are going to buy one book on probability, this is a good choice because it contains so much material.

  • Norris (1998) “Markov chains”.
    Markov chains at a more graduate level of sophistication, revealing what I have concealed, namely the full gory story about \(Q\)-matrices.

  • Williams (1991) “Probability with martingales”.
    Excellent graduate text for theory of martingales: mathematically demanding.

4.1 Free on the web

  • Doyle and Snell (1984) “Random walks and electric networks”
    Available on the web at http://arxiv.org/abs/math/0001057.
    Lays out (in simple and accessible terms) an important approach to Markov chains using relationship to resistance in electrical networks.

  • Kindermann and Snell (1980) “Markov random fields and their applications”
    Available on the web at https://doi.org/10.1090/conm/001.
    Sublimely accessible treatment of Markov random fields (Markov property, but in space not time).

  • Meyn and Tweedie (1993) “Markov chains and stochastic stability”
    Available on the web at http://probability.ca/MT/.
    The place to go if you need to get informed about theoretical results on rates of convergence for Markov chains (e.g. because you are doing MCMC).

  • Aldous and Fill (2001) “Reversible Markov Chains and Random Walks on Graphs”
    Only available on the web at http://www.stat.berkeley.edu/~aldous/RWG/book.html.
    The best unfinished book on Markov chains known to us.

4.2 Going deeper

  • Kingman (1993) “Poisson processes”.
    Very good introduction to the wide circle of ideas surrounding the Poisson process.

  • Kelly (1979) “Reversibility and stochastic networks”.
    We’ll cover reversibility briefly in the lectures, but this shows just how powerful the technique is.

  • Lindvall (1992) “Lectures on the coupling method”.
    We’ll also talk briefly about the beautiful concept of coupling for Markov chains; this book gives a very nice introduction.

  • Steele (2004) “The Cauchy-Schwarz master class”.
    The book to read if you decide you need to know more about (mathematical) inequality.

  • Aldous (1989) “Probability approximations via the Poisson clumping heuristic”.
    See http://www.stat.berkeley.edu/~aldous/Research/research80.html.
    A book full of what ought to be true; hence good for stimulating research problems and also for ways of computing heuristic answers.

  • Øksendal (2003) “Stochastic differential equations”.
    An accessible introduction to Brownian motion and stochastic calculus, which we do not cover at all.

  • Stoyan, Kendall, and Mecke (1987) “Stochastic geometry and its applications”.
    Discusses a range of techniques used to handle probability in geometric contexts.

References

Aldous, David J. 1989. Probability Approximations via the Poisson Clumping Heuristic. Vol. 77. Applied Mathematical Sciences. New York: Springer-Verlag.
Aldous, David J., and James A. Fill. 2001. Reversible Markov Chains and Random Walks on Graphs. Unpublished. http://www.stat.berkeley.edu/~aldous/RWG/book.html.
Doyle, Peter G., and J. Laurie Snell. 1984. Random Walks and Electric Networks. Vol. 22. Carus Mathematical Monographs. Washington, DC: Mathematical Association of America.
Grimmett, Geoffrey R., and David R. Stirzaker. 2001. Probability and Random Processes. Third. New York: Oxford University Press.
Häggström, Olle. 2002. Finite Markov Chains and Algorithmic Applications. Vol. 52. London Mathematical Society Student Texts. Cambridge: Cambridge University Press.
Kelly, Frank P. 1979. Reversibility and Stochastic Networks. Chichester: John Wiley & Sons Ltd.
Kindermann, Ross, and J. Laurie Snell. 1980. Markov Random Fields and Their Applications. Vol. 1. Contemporary Mathematics. Providence, R.I.: American Mathematical Society. https://doi.org/10.1090/conm/001.
Kingman, J. F. C. 1993. Poisson Processes. Vol. 3. Oxford Studies in Probability. New York: The Clarendon Press Oxford University Press.
Lindvall, Torgny. 1992. Lectures on the coupling method. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons Inc.
Meyn, S. P., and R. L. Tweedie. 1993. Markov Chains and Stochastic Stability. Communications and Control Engineering Series. London: Springer-Verlag London Ltd. http://probability.ca/MT/.
Norris, J. R. 1998. Markov Chains. Vol. 2. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press.
Øksendal, Bernt. 2003. Stochastic Differential Equations. Sixth. Universitext. Berlin: Springer-Verlag.
Steele, J. Michael. 2004. The Cauchy-Schwarz Master Class. MAA Problem Books Series. Washington, DC: Mathematical Association of America.
Stoyan, D., W. S. Kendall, and J. Mecke. 1987. Stochastic Geometry and Its Applications. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Chichester: John Wiley & Sons Ltd.
Williams, David. 1991. Probability with Martingales. Cambridge Mathematical Textbooks. Cambridge: Cambridge University Press.