Continuous-Time Markov Chains

Abstract

The first part of this chapter presented the definition of a continuous-time Markov chain with two properties, and the introduction of a B&D process with some special examples such as homogeneous Poisson process as a pure birth process, and the population model as a B&D process. The second part of the chapter was an extension of a B&D process to the B&D queueing models including the general B&D queueing model, M/M/1 queueing model, and M/M/m queueing model. Finally, the last part of the chapter introduced Kolmogorov’s backward/forward equations, the infinitesimal generator matrix of a CTMC, and the time-reversibility condition for a CTMC. Several examples and problems including those related to the population model, the server systems, and the patient states have been provided and solved.