Advanced SearchSearch Tips
Numerical Iteration for Stationary Probabilities of Markov Chains
facebook(new window)  Pirnt(new window) E-mail(new window) Excel Download
 Title & Authors
Numerical Iteration for Stationary Probabilities of Markov Chains
Na, Seongryong;
  PDF(new window)
We study numerical methods to obtain the stationary probabilities of continuous-time Markov chains whose embedded chains are periodic. The power method is applied to the balance equations of the periodic embedded Markov chains. The power method can have the convergence speed of exponential rate that is ambiguous in its application to original continuous-time Markov chains since the embedded chains are discrete-time processes. An illustrative example is presented to investigate the numerical iteration of this paper. A numerical study shows that a rapid and stable solution for stationary probabilities can be achieved regardless of periodicity and initial conditions.
Markov chain;embedded chain;periodicity;power method;stationary probability;numerical iteration;balance equation;
 Cited by
Multiclass loss systems with several server allocation methods, Korean Journal of Applied Statistics, 2016, 29, 4, 679  crossref(new windwow)
Na, S. (2010). Markov modeling of multiclass loss systems, The Korean Journal of Applied Statistics, 23, 747-757. crossref(new window)

Nesterov, Y. and Nemirovski, A. (2014). Finding the stationary states of Markov chains by iterative methods, Applied Mathematics and Computation, Available from: http:/, In press.

O'Leary, D. P. (1993). Iterative methods for finding the stationary vector for Markov chains, Linear Algebra, Markov Chains, and Queueing Models (C. D. Meyer and R. J. Plemmons (ed.)), The IMA volumes in mathematics and its applications, 48, 125-136, Springer, New York.

Ross, S. M. (1996). Stochastic Processes, 2nd edition, Wiley, New York.

Stewart, W. J. (2000). Numerical methods for computing stationary distributions of finite irreducible Markov chains, Computational Probability (W. K. Grassmann (ed.)), International series in operations research and management science, 24, 81-111, Springer, New York. crossref(new window)

Zhao, D., Li, H. and Su, D. (2012). A numerical algorithm on the computation of the stationary distribution of a discrete time homogeneous finite Markov chain, Mathematical Problems in Engineering, 2012, Article ID 167453, 10 pages.