Special Discrete-Time Markov Chains

Abstract

This chapter has been devoted to some selected special discrete-time Markov chains with wide applications including random walk, simple random walk, simple symmetric random walk, Gambler’s ruin problem as a special simple random walk, branching process, hidden Markov chains, time-reversible discrete-time Markov chains, and Markov decision processes. Some illustrative examples and problems have been provided for each special discrete-time Markov chain.