preface
1 introduction
1.1 two-time-scale markovian systems
1.2 literature review
1.3 why do we need this book?
1.4 outline of the book
part i asymptotic results:two-time-scale markov chains
2 summary of two-time-scale markov chains:finite state spacecases
2.1 two-time-scale continuous-time markov chains
2.2 properties of two-time-scale markov chains
2.2.1 asymptotic expansions
2.2.2 occupation measures
2.2.3 exponential bounds
2.3 ramifications
2.4 notes
3 switching diffusion limits
3.1 introduction
3.2 problem formulation and preliminaries
3.2.1 formulation
3.2.2 conditions
3.2.3 preliminaries
3.3 asymptotic properties
3.3.1 a mean square estimate
3.3.2 weak convergence of the aggregated process
3.4 inclusion of transient states in the jump process
3.5 notes
4 countable state space i:single-group recurrent states
4.1 introduction
4.2 formulation
4.2.1 basic notation
4.2.2 two-time-scale markov chains
4.3 asymptotic expansions
4.3.1 formal expansions
4.3.2 asymptotic justification
4.3.3 asymptotic expansion of transition probability matrices
4.4 occupation measures
4.4.1 second moment bounds and mixing property
4.4.2 functionals of the two-time-scale markov chain
4.4.3 invariance theorem and limit distribution
4.5 applications to queueing processes
4.6 notes
5 countable state space ii:multi-group recurrent states
5.1 introduction
5.2 formulation
5.2.1 notation
5.2.2 queue length and two-time-scale markov chains
5.3 asymptotic properties of probability distribution
5.3.1 formal expansions
5.3.2 asymptotic justification
5.3.3 asymptotic expansion of transition probability matrices
5.4 aggregation and weak convergence
5.5 switching diffusion limit
5.6 an example
5.7 notes
part ii several application examples to financialengineering,insurance,queueing networks,and filtering
6 financial engineering
6.1 geometric brownian motion model
6.2 stock selling rule
6.2.1 two-point boundary value problems
6.2.2 limit problem and near optimality
6.2.3 expected exit time and related probabilities
6.2.4 numerical examples
6.3 near-optimal asset allocation
6.3.1 optimal asset allocation
6.3.2 convergence of value functions
6.3.3 near-optimal asset allocation
6.4 notes
7 near-optimal dividend policy
7.1 formulation
7.2 limit problem
7.3 convergence of the cost and value functions
7.4 near-optimal dividend policy
7.5 notes
8 queueing networks
8.1 application to m_t/m_t/1/m
8.2 markovian queueing networks
8.3 markov-modulated-rate fluid models
8.4 notes
9 wonham filtering
9.1 introduction
9.1.1 wonham filtering
9.2 two-time scale markov chains
9.2.1 two-time-scale filters
9.3 limit filter and two-time-scale approximation
9.3.1 limit filter
9.3.2 two-time-scale approximation
9.4 a numerical example
9.5 inclusion of transient states
9.6 notes
a background materials
references
index