Markov Chains - Michael K. Ng

. .

Résumé :
This new edition of Markov Chains: Models, Algorithms and Applications has been completely reformatted as a text, complete with end-of-chapter exercises, a new focus on management science, new applications of the models, and new examples with applications in financial risk management and modeling of financial data. This book consists of eight chapters.? Chapter 1 gives a brief introduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also be highlighted. Some classical iterative methods for solving linear systems will be introduced for finding the stationary distribution of a Markov chain. The chapter then covers the basic theories and algorithms for hidden Markov models (HMMs) and Markov decision processes (MDPs). Chapter 2 discusses the applications of continuous time Markov chains to model queueing systems and discrete time Markov chain for computing the PageRank, the ranking of websites on the Internet. Chapter 3 studies Markovian models for manufacturing and re-manufacturing systems and presents closed form solutions and fast numerical algorithms for solving the captured systems. In Chapter 4, the authors present a simple hidden Markov model (HMM) with fast numerical algorithms for estimating the model parameters. An application of the HMM for customer classification is also presented. Chapter 5 discusses Markov decision processes for customer lifetime values. Customer Lifetime Values (CLV) is an important concept and quantity in marketing management. The authors present an approach based on Markov decision processes for the calculation of CLV using real data. Chapter 6 considers higher-order Markov chain models, particularly a class of parsimonious higher-order Markov chain models. Efficient estimation methods for model parameters based on linear programming are presented. Contemporary research results on applications to demand predictions, inventory control and financial risk measurement are also presented. In Chapter 7, a class of parsimonious multivariate Markov models is introduced. Again, efficient estimation methods based on linear programming are presented. Applications to demand predictions, inventory control policy and modeling credit ratings data are discussed. Finally, Chapter 8 re-visits hidden Markov models, and the authors present a new class of hidden Markov models with efficient algorithms for estimating the model parameters. Applications to modeling interest rates, credit ratings and default data are discussed. ? This book is aimed at senior undergraduate students, postgraduate students, professionals, practitioners, and researchers in applied mathematics, computational science, operational research, management science and finance, who are interested in the formulation and computation of queueing networks, Markov chain models and related topics. Readers are expected to have some basic knowledge of probability theory, Markov processes and matrix theory.

Biographie:
Sini Guo is an Assistant Professor in School of Management, Beijing Institute of Technology, and recipient of the Young Elite Scientists Sponsorship Program of the Beijing High Innovation Plan. He received the Ph.D. degree in mathematics from the University of Hong Kong in 2021. His research interests include operations research, optimization and financial engineering, and has published over 20 research papers in journals such as IISE Transactions, European Journal of Operational Research, Quantitative Finance, Omega, IEEE Transactions on Neural Networks and Learning Systems, IEEE Transactions on Fuzzy Systems, and Automatica. He is granted multiple funds, including National Natural Science Foundation of China, China Postdoctoral Science Foundation, International Exchange Program for Postdoctoral Researchers, and serves as Guest Editors of Axioms and Mathematics. Jia-Wen Gu is an Associate Professor in the Department of Mathematics at Southern University of Science and Technology (SUSTech). She obtained her Bachelor's degree in mathematics from the Department of Mathematics at Sun Yat-Sen University in 2010, and received her Ph.D. degree in financial mathematics from the Department of Mathematics at the University of Hong Kong. Her professional experience includes an internship in a quantitative research group at JP Morgan, a postdoctoral researcher position at the University of Copenhagen and at the University of Hong Kong. Her main research areas cover optimal portfolio selection, quantitative trading, and machine learning applications in finance. She has published numerous research papers in prestigious journals including European Journal of Operational Research, Quantitative Finance, SIAM Journal on Control and Optimization, Mathematics of Operations Research, and Omega, etc. Wai-Ki Ching is a full Professor at the Department of Mathematics, University of Hong Kong. He obtained his B. Sc. and M. Phil. in Mathematics from University of Hong Kong and his Ph.D. in Systems Engineering and Engineering Management from Chinese University of Hong Kong. He received 2013 Higher Education Outstanding Scientific Research Output Awards (Second Prize) from the Ministry of Education, China (2014), Distinguished Alumni Award, Faculty of Engineering, Chinese University of Hong Kong (2017), 2019 Higher Education Outstanding Scientific Research Output Awards (Second Prize), Hunan Province, China (2019), Outstanding Research Student Supervisor Award, University of Hong Kong (2020) and he was World's Top 2% Most-cited Scientists (2021) by Stanford University. His research interests are Matrix Computations and Stochastic Modeling for Quantitative Finance and Bioinformatics. He is an author/editor of over 350 publications including over 250 journal papers, 5 edited journal special issues, 6 books and over 110 book chapters and conference proceedings....

Sommaire:
Introduction.- Manufacturing and Re-manufacturing Systems.- A Hidden Markov Model for Customer Classification.- Markov Decision Processes for Customer Lifetime Value.- Higher-order Markov Chains.- Multivariate Markov Chains.- Hidden Markov Chains.

From the reviews of the First Edition: The authors outline recent developments of Markov chain models ... . This book is aimed at students, professionals, practitioners, and researchers in scientific computing and operational research, who are interested in the formulation and computation of queuing and manufacturing systems. It gives a number of useful tools for researchers in real applications ... . (Alexander I. Zejfman, Zentralblatt MATH, Vol. 1089 (15), 2006) In this book's ... essential notions on Markov chains, hidden Markov models, and Markov decision processes are covered, with special emphasis on iterative methods for solving linear systems. ... Each chapter finishes with a short summary and sometimes a selection of open problems. ... This book is intended for students and researchers in applied mathematics, scientific computing, and operations research ... . Overall, this book offers much interesting and up-to-date material on a wide variety of topics, dealing with finite-space Markov processes. (Jozef L. Teugels, Journal of the American Statistical Association, Vol. 103 (483), September, 2008)