Continuous-Time Markov Chains

1 Transition Functions and Resolvents.- 1. Markov Chains and Transition Functions: Definitions and Basic Properties.- 2. Differentiability Properties of Transition Functions and Significance of the Q-Matrix.- 3. Resolvent Functions and Their Properties.- 4. The Functional-Analytic Setting for Transition Functions and Resolvents.- 5. Feller Transition Functions.- 6. Kendall’s Representation of Reversible Transition Functions.- 7. Appendix.- 2 Existence and Uniqueness of Q-Functions.- 1. Q-Functions and the Kolmogorov Backward and Forward Equations.- 2. Existence and Uniqueness of Q-Functions.- 3 Examples of Continuous-Time Markov Chains.- 1. Finite Markov Chains.- 2. Birth and Death Processes.- 3. Continuous Time Parameter Markov Branching Processes.- 4 More on the Uniqueness Problem.- 1. Laplace Transform Tools.- 2. Non-uniqueness—Construction of Q-Functions Other Than the Minimal One.- 3. Uniqueness—The Non-Conservative Case.- 5 Classification of States and Invariant Measures.- 1. Classification of States.- 2. Sub-invariant and Invariant Measures.- 3. Classification Based on the Q-Matrix.- 4. Determination of Invariant Measures from the Q-Matrix.- 6 Strong and Exponential Ergodicity.- 1. The Ergodic Coefficient and Hitting Times.- 2. Ordinary Ergodicity.- 3. Strong Ergodicity.- 4. Geometric Ergodicity for Discrete Time Chains.- 5. The Croft-Kingman Lemmas.- 6. Exponential Ergodicity for Continuous-Time Chains.- 7 Reversibility, Monotonicity, and Other Properties.- 1. Symmetry and Reversibility.- 2. Exponential Families of Transition Functions.- 3. Stochastic Monotonicity and Comparability.- 4. Dual Processes.- 5. Coupling.- 8 Birth and Death Processes.- 1. The Potential Coefficients and Feller’s Boundary Conditions.- 2. Karlin and McGregor’s Representation Theorem and Duality.- 3. The Stieltjes Moment Problem.- 4. The Karlin-McGregor Method of Solution.- 5. Total Positivity of the Birth and Death Transition Function.- 9 Population Processes.- 1. Upwardly Skip-Free Processes.- 2. Extinction Times and Probability of Extinction for Upwardly Skip-Free Processes.- 3. Multidimensional Population Processes.- 4. Two-Dimensional Competition Processes.- 5. Birth, Death, and Migration Processes.- Symbol Index.- Author Index.