FOREWORD PREFACE Patrick Billingsley 1925- 2011 Chapter 1 PROBABILITY 1. BOREL'S NORMAL NUMBER THEOREM, 1 The Unit Interval--The Weak Law of Large Numbers--The Strong Law of Large Numbers--Strong Law Versus Weak-- Length--The Measure Theory of Diophantine Approximation* 2. PROBABILITY MEASURES, 18 Spaces --Assigning Probabilities--Classes of Sets--Probability Measures--Lebesgue Measure on the Unit Interval--Sequence Space* - Constructing ��-Fields* 3. EXISTENCE AND EXTENSION, 39 Construction of the Extension--Uniqueness and the ��-�� Theorem--Monotone Classes--Lebesgue Measure on the Unit Interval- Completeness-- Nonmeasurable Sets--Two Impossibility Theorems* 4. DENUMERABLE PROBABILITIES, 53 General Formulas-- Limit Sets-Independent Events--Subfields--The Borel-Cantelelli Lemmas--The Zero-One Law 5. SIMPLE RANDOM VARIABLES, 72 Definition-- Convergence of Random Variables--Independence--Existence of Independent Sequences-- Expected Value--Inequalities 6. THE LAW OF LARGE NUMBERS, 90 The Strong Law--The Weak Law--Bernstein's Theorem--A Refinement of the Second BoreI-Cantelli Lemma 7. GAMBLING SYSTEMS, 98 Gambler's Ruin--Selection Systems--Gambling Policies--Bold Play*--Timid Play* 8. MARKOVCHAINS, 117 Definitions-- Higher-Order Transitions --An Existence Theorem--Transience and Persistence--Another Criterion for Persistence--Stationary Distributions-- Exponential Convergence*--Optimal Stopping* 9. LARGE DEVIATIONS AND THE LAW OF THE ITERATED LOGARITHM, 154 Moment Generating Functions--Large Deviations -- Chernoff's Theorem*--The Law of the Iterated Logarithm Chapter 2 MEASURE 167 10. GENERAL MEASURES, 167 Classes of Sets-- Conventions Involving �� -- Measures-- Uniqueness 11. OUTER MEASURE, 174 Outer Measure--Extension--An Approximation Theorem 12. MEASURES IN EUCLIDEAN SPACE, 181 Lebesgue Measure--Regularity--Specifying Measures on the Line--Specifying Measures in Rk-strange Euclidean Sets* 13. MEASURABLE FUNCTIONS AND MAPPINGS, 192 Measurable Mappings-- Mappings into Rk- Limits and Measurability--Transformations of Measures 14. DISTRIBUTION FUNCTIONS, 198 Distribution Functions--Exponential Distributions--Weak Convergence-- Convergence of Types* -- Extremal Distributions* Chapter 3 INTEGRATION 211 15. THE INTEGRAL, 211 Definition -- Nonnegative Functions-- Uniqueness 16. PROPERTIES OF THE INTEGRAL, 218 Equalities and Inequalities--Integration to the Limit--Integration over Sets-- Densities-- Change of Variable-- Uniform Integrability-- Complex Functions 17. THE INTEGRAL WITH RESPECT TO LEBESGUE MEASURE, 234 The Lebesgue Integral on the Line--The Riemann Integral--The Fundamental Theorem of Calculus--Change of Variable--The Lebesgue Integral in Rk--Stieltjes Integrals 18. PRODUCT MEASURE AND FUBINI'S THEOREM, 245 Product Spaces-- Product Measure-- Fubini's Theorem--Integration by Parts-- Products of Higher Order 19. THE Lp SPACES*, 256 Definitions-- Completeness and Separability-- Conjugate Spaces--Weak Compactness--Some Decision Theory--The Space L2-An Estimation Problem Chapter 4 RANDOM VARIABLES AND EXPECTED VALUES 271 20. RANDOM VARIABLES AND DISTRIBUTIONS, 271 Random Variables and Vectors-- Subfields-- Distributions -- Multidimensional Distributions--Independence--Sequences of Random Variables--Convolution--Convergence in Probability--The Glivenko-Cantelli Theorem* 21. EXPECTED VALUES, 291 Expected Value as Integral--Expected Values and Limits-- Expected Values and Distributions-- Moments--Inequalities--Joint Integrals--Independence and Expected Value-- Moment Generating Functions 22. SUMS OF INDEPENDENT RANDOM VARIABLES, 300 The Strong Law of Large Numbers--The Weak Law and Moment Generating Functions--Kolmogorov's Zero-One Law-- Maximal Inequalities-- Convergence of Random Series--Random Taylor Series* 23. THE POISSON PROCESS, 316 Characterization of the Exponential Distribution--The Poisson Process--The Poisson Approximation--Other Characterizations of the Poisson Process--Stochastic Processes 24. THE ERGODIC THEOREM*, 330 Measure-Preserving Transformations-- Ergodicity-- Ergodicity of Rotations--Proof of the Ergodic Theorem--The Continued-Fraction Transformation-- Diophantine Approximation Chapter 5 CONVERGENCE OF DISTRIBUTIONS 349 25. WEAK CONVERGENCE, 349 Definitions-- Uniform Distribution Modulo 1 * --Convergence in Distribution--Convergence in Probability--Fundamental Theorems--Helly's Theorem--Integration to the Limit 26. CHARACTERISTIC FUNCTIONS, 365 Definition -- Moments and Derivatives-- Independence--Inversion and the Uniqueness Theorem--The Continuity Theorem-- Fourier Series* 27. THE CENTRAL LIMIT THEOREM, 380 Identically Distributed Summands--The Lindeberg and Lyapounov Theorems--Dependent Variables* 28. INFINITELY DIVISIBLE DISTRIBUTIONS*, 394 Vague Convergence--The Possible Limits--Characterizing the Limit 29. LIMIT THEOREMS IN Rk, 402 The Basic Theorems-- Characteristic Functions-- Normal Distributions in Rk--The Central Limit Theorem 30. THE METHOD OF MOMENTS*, 412 The Moment Problem--Moment Generating Functions--Central Limit Theorem by Moments--Application to Sampling Theory--Application to Number Theory Chapter 6 DERIVATIVES AND CONDITIONAL PROBABILITY 425 31. DERIVATIVES ON THE LINE*, 425 The Fundamental Theorem of Calculus--Derivatives of Integrals--Singular Functions--Integrals of Derivatives--Functions of Bounded Variation 32. THE RADON-NIKODYM THEOREM, 446 Additive Set Functions--The Hahn Decomposition--Absolute Continuity and Singularity--The Main Theorem 33. CONDITIONAL PROBABILITY, 454 The Discrete Case--The General Case--Properties of Conditional Probability-- Difficulties and Curiosities-- Conditional Probability Distributions 34. CONDITIONAL EXPECTATION, 472 Definition-- Properties of Conditional Expectation--Conditional Distributions and Expectations-- Sufficient Subfields* -- Minimum-Variance Estimation* 35. MARTINGALES, 487 Definition -- Su bmartingales-- Gambling -- Functions of Martingales-- Stopping Times-- Inequalities-- Convergence Theorems--Applications: Derivatives-- Likelihood Ratios-- Reversed Martingales--Applications: de Finetti's Theorem--Bayes Estimation--A Central Limit Theorem* Chapter 7 STOCHASTIC PROCESSES 513 36. KOLMOGOROV'S EXISTENCE THEOREM, 513 Stochastic Processes-- Finite-Dimensional Distributions-- Product Spaces-- Kolmogorov's Existence Theorem--The Inadequacy of RT-A Return to Ergodic Theory--The Hewitt-Savage Theorem* 37. BROWNIAN MOTION, 530 Definition --Continuity of Paths-- Measurable Processes--Irregularity of Brownian Motion Paths--The Strong Markov Property--The Reflection Principle--Skorohod Embedding --I nvariance* 38. NONDENUMERABLE PROBABILITIES, 558 Introduction -- Definitions-- Existence Theorems--Consequences of Separability* APPENDIX NOTES ON THE PROBLEMS BIBLIOGRAPHY INDEX