1 Introduction 1.1 Background 1.2 Motivation and objectives of the book 1.3 Layout plan of the book 1.4 Notations 2 Literature Survey 2.1 Introduction 2.2 Literatures on stochastic optimal control problems 2.3 Literature on Bellman's optimality principle or Dynamic program ming principle 2.4 Works on the Hamilton-Jacobi-Bellman (HJB) equation or Dynamic programming equation 2.5 Brief survey of literature on viscosity and classical solution of HJB equation 2.6 Literatures on the existence and development of optimal policies with reference to cost control 2.7 Concluding remarks 3 Stochastic Differential Equations relating to Stochastic Control The ory 3.1 Introduction 3.2 Preliminaries 3.2.1 Some definitions 3.2.2 Stochastic integrals 3.2.3 Stochastic differential equations (SDEs) 3.3 Linear control systems 3.4 Optimal control problems 3.4.1 Linear regulator problem 3.4.2 Stochastic control problems in standard forms 3.4.3 The linear-quadratic regulator problem 3.5 Concluding remarks 4 Viscosity Solution of the Degenerate Bellman Equation of Linear Regulator Control Problem 4.1 Introduction 4.2 Stochastic linear regulator control problem 4.2.1 Problem formulation 4.2.2 The Hamilton-Jacobi-Bellman Equation 4.2.3 Value function 4.3 Viscosity solutions of the Degenerate Bellman Equation 4.3.1 Definition of viscosity solution 4.3.2 Viscosity properties of the value function 4.3.3 Dymnamic programming princtiple 4.4 Convergence of the value function 4.4.1 The value function is a viscosity solution of degenerate Bell man equation 4.5 Uniqueness of degenerate Bellman equation 4.6 Stability properties of viscosity solutions 4.6.1 The limiting value function is a viscosity solution of degenerate Bellman equation 4.7 Concluding remarks 5 Existence of Classical Solution of the Degenerate Bellman Equation