Markov Chains: Theory and Applications

Preface<br>Arni S.R. Srinivasa Rao<br>1. Markov Chain Estimation, Approximation, and Aggregation for Average Reward Markov Decision Processes and Reinforcement Learning<br>Ronald Ortner<br>2. Ladder processes: symmetric functions and semigroups<br>Philip Feinsilver<br>3. Continuous-time Markov Chains and Models: Study via Forward Kolmogorov System<br>Alexander Zeifman<br>4. Analysis of Data Following Finite-State Continuous-Time Markov Chains<br>Wenyaw Chan<br>5. Computational applications of poverty measurement through Markov model for income classes<br>Guglielmo D&rsquo;Amico<br>6. Estimation and calibration of continuous time Markov chains<br>Manuel L. Esquível<br>7. Additive High-Order Markov Chains<br>Serhii Melnyk, Galyna Prytula and Oleg Victorovich Usatenko<br>8. The role of the random-product technique in the theory of Markov chains on a countable state space.<br>Brian Fralix, Amin Khademi and Farhad Hasankhani<br>9. On estimation problems based on type I Longla copulas<br>Martial Longla<br>10. Long time behaviour of continuous time Markov chains<br>Xueping Huang<br>11. To be Determined<br>Alan Krinik