Non-homogeneous random walks on a semi-infinite strip

Nicholas Georgiou and Andrew R. Wade

Stochastic Processes and their Applications, 124, no. 10, October 2014, 3179–3205. DOI: 10.1016/j.spa.2014.05.005

Supported by EPSRC award Non-homogeneous random walks (EP/J021784/1).

Abstract

We study the asymptotic behaviour of Markov chains on the product state space given by the non-negative integers times a finite set $S$. Neither coordinate is assumed to be Markov. We assume a moments bound on the jumps of the first coordinate, and that, roughly speaking, the second coordinate is close to being Markov when the first is large. This departure from much of the literature, which assumes that the $S$-coordinate is itself a Markov chain, enables us to probe precisely the recurrence phase transitions by assuming asymptotically zero drift for the first coordinate. We give a recurrence classification in terms of increment moment parameters and the stationary distribution for the limit of the $S$-process when the first coordinate is large. In the null case we also provide a weak convergence result. Our results can be seen as generalizations of Lamperti's results for non-homogeneous random walks on the non-negative integers (the case where $S$ is a singleton). Motivation arises from modulated queues or processes with hidden variables where $S$ describes an internal state of the system.