# Probability%2C Random Processes%2C and Statistical Analysis

Post on 14-Oct-2014

423 views

TRANSCRIPT

<p>Probability, Random Processes, and Statistical AnalysisTogether with the fundamentals of probability, random processes, and statistical analy-sis, this insightful book also presents a broad range of advanced topics and applicationsnot covered in other textbooks.Advanced topics include: Bayesian inference and conjugate priors Chernoff bound and large deviation approximation Principal component analysis and singular value decomposition Autoregressive moving average (ARMA) time series Maximum likelihood estimation and the Expectation-Maximization (EM) algorithm Brownian motion, geometric Brownian motion, and Ito process BlackScholes differential equation for option pricing Hidden Markov model (HMM) and estimation algorithms Bayesian networks and sum-product algorithm Markov chain Monte Carlo methods Wiener and Kalman lters Queueing and loss networksThe book will be useful to students and researchers in such areas as communications,signal processing, networks, machine learning, bioinformatics, and econometrics andmathematical nance. With a solutions manual, lecture slides, supplementary materials,and MATLAB programs all available online, it is ideal for classroom teaching as wellas a valuable reference for professionals.Hisashi Kobayashi is the Sherman Fairchild University Professor Emeritus at PrincetonUniversity, where he was previously Dean of the School of Engineering and AppliedScience. He also spent 15 years at the IBM Research Center, Yorktown Heights, NY,and was the Founding Director of IBM Tokyo Research Laboratory. He is an IEEE LifeFellow, an IEICE Fellow, was elected to the Engineering Academy of Japan (1992), andreceived the 2005 Eduard Rhein Technology Award.Brian L. Mark is a Professor in the Department of Electrical and Computer Engineer-ing at George Mason University. Prior to this, he was a research staff member at theNEC C&C Research Laboratories in Princeton, New Jersey, and in 2002 he received aNational Science Foundation CAREER award.William Turin is currently a Consultant at AT&T Labs Research. As a Member of Tech-nical Staff at AT&T Bell Laboratories and later a Technology Consultant at AT&TLabs Research for 21 years, he developed methods for quantifying the performanceof communication systems. He is the author of six books and numerous papers.This book provides a very comprehensive, well-written and modern approach to thefundamentals of probability and random processes, together with their applications inthe statistical analysis of data and signals. It provides a one-stop, unied treatment thatgives the reader an understanding of the models, methodologies, and underlying princi-ples behind many of the most important statistical problems arising in engineering andthe sciences today.Dean H. Vincent Poor, Princeton UniversityThis is a well-written, up-to-date graduate text on probabilty and random processes. Itis unique in combining statistical analysis with the probabilistic material. As noted bythe authors, the material, as presented, can be used in a variety of current applicationareas, ranging from communications to bioinformatics. I particularly liked the historicalintroduction, which should make the eld exciting to the student, as well as the intro-ductory chapter on probability, which clearly describes for the student the distinctionbetween the relative frequency and axiomatic approaches to probability. I recommendit unhesitatingly. It deserves to become a leading text in the eld.Professor Emeritus Mischa Schwartz, Columbia UniversityHisashi Kobayashi, Brian L. Mark, and William Turin are highly experienced uni-versity teachers and scientists. Based on this background, their book covers not onlyfundamentals, but also a large range of applications. Some of them are treated in atextbook for the rst time. Without any doubt the book will be extremely valuable tograduate students and to scientists in universities and industry. Congratulations to theauthors!Professor Dr.-Ing. Eberhard Hnsler, Technische Universitt DarmstadtAn up-to-date and comprehensive book with all the fundamentals in probability, ran-dom processes, stochastic analysis, and their interplays and applications, which lays asolid foundation for the students in related areas. It is also an ideal textbook with verelatively independent but logically interconnected parts and the corresponding solutionmanuals and lecture slides. Furthermore, to my best knowledge, similar editing in PartIV and Part V cant be found elsewhere.Zhisheng Niu, Tsinghua UniversityProbability, Random Processes,and Statistical AnalysisHI SASHI KOBAYASHIPrinceton UniversityBRI AN L. MARKGeorge Mason UniversityWI LLI AM TURI NAT&T Labs ResearchC A M B R I D G E U N I V E R S I T Y P R E S SCambridge, New York, Melbourne, Madrid, Cape TownSingapore, So Paulo, Delhi, Tokyo, Mexico CityCambridge University PressThe Edinburgh Building, Cambridge CB2 8RU, UKPublished in the United States of America by Cambridge University Press, New Yorkwww.cambridge.orgInformation on this title: www.cambridge.org/9780521895446c _Cambridge University Press 2012This publication is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,no reproduction of any part may take place without the writtenpermission of Cambridge University Press.First published 2012Printed in the United Kingdom at the University Press, CambridgeA catalogue record for this publication is available from the British LibraryISBN 978-0-521-89544-6 HardbackAdditional resources for this publication at www.cambridge.org/9780521895446Cambridge University Press has no responsibility for the persistence oraccuracy of URLs for external or third-party internet websites referred toin this publication, and does not guarantee that any content on suchwebsites is, or will remain, accurate or appropriate.ToMasae, Karen, and GalinaContentsList of abbreviations and acronyms page xviiiPreface xxiiiAcknowledgments xxx1 Introduction 11.1 Why study probability, random processes, and statistical analysis? 11.1.1 Communications, information, and control systems 21.1.2 Signal processing 21.1.3 Machine learning 31.1.4 Biostatistics, bioinformatics, and related elds 41.1.5 Econometrics and mathematical nance 41.1.6 Queueing and loss systems 61.1.7 Other application domains 61.2 History and overview 71.2.1 Classical probability theory 71.2.2 Modern probability theory 91.2.3 Random processes 101.2.4 Statistical analysis and inference 121.3 Discussion and further reading 14Part I Probability, random variables, and statistics 152 Probability 172.1 Randomness in the real world 172.1.1 Repeated experiments and statistical regularity 172.1.2 Random experiments and relative frequencies 182.2 Axioms of probability 182.2.1 Sample space 192.2.2 Event 192.2.3 Probability measure 202.2.4 Properties of probability measure 21viii Contents2.3 Bernoulli trials and Bernoullis theorem 262.4 Conditional probability, Bayes theorem, and statistical independence 302.4.1 Joint probability and conditional probability 302.4.2 Bayes theorem 312.4.3 Statistical independence of events 342.5 Summary of Chapter 2 362.6 Discussion and further reading 372.7 Problems 373 Discrete random variables 423.1 Random variables 423.1.1 Distribution function 433.1.2 Two random variables and joint distribution function 443.2 Discrete random variables and probability distributions 453.2.1 Joint and conditional probability distributions 463.2.2 Moments, central moments, and variance 503.2.3 Covariance and correlation coefcient 513.3 Important probability distributions 533.3.1 Bernoulli distribution and binomial distribution 543.3.2 Geometric distribution 553.3.3 Poisson distribution 563.3.4 Negative binomial (or Pascal) distribution 593.3.5 Zipfs law and zeta distribution 623.4 Summary of Chapter 3 653.5 Discussion and further reading 663.6 Problems 664 Continuous random variables 724.1 Continuous random variables 724.1.1 Distribution function and probability density function 724.1.2 Expectation, moments, central moments, and variance 734.2 Important continuous random variables and their distributions 754.2.1 Uniform distribution 754.2.2 Exponential distribution 764.2.3 Gamma distribution 784.2.4 Normal (or Gaussian) distribution 804.2.5 Weibull distributions 864.2.6 Pareto distribution 884.3 Joint and conditional probability density functions 904.3.1 Bivariate normal (or Gaussian) distribution 924.3.2 Multivariate normal (or Gaussian) distribution 944.4 Exponential family of distributions 954.5 Bayesian inference and conjugate priors 97Contents ix4.6 Summary of Chapter 4 1034.7 Discussion and further reading 1044.8 Problems 1045 Functions of random variables and their distributions 1125.1 Function of one random variable 1125.2 Function of two random variables 1155.3 Two functions of two random variables and the Jacobian matrix 1195.4 Generation of random variates for Monte Carlo simulation 1235.4.1 Random number generator (RNG) 1245.4.2 Generation of variates from general distributions 1255.4.3 Generation of normal (or Gaussian) variates 1305.5 Summary of Chapter 5 1315.6 Discussion and further reading 1315.7 Problems 1326 Fundamentals of statistical data analysis 1386.1 Sample mean and sample variance 1386.2 Relative frequency and histograms 1406.3 Graphical presentations 1416.3.1 Histogram on probability paper 1426.3.2 Log-survivor function curve 1446.3.3 Hazard function and mean residual life curves 1486.3.4 Dot diagram and correlation coefcient 1496.4 Summary of Chapter 6 1526.5 Discussion and further reading 1526.6 Problems 1537 Distributions derived from the normal distribution 1577.1 Chi-squared distribution 1577.2 Students t-distribution 1617.3 Fishers F-distribution 1637.4 Log-normal distribution 1657.5 Rayleigh and Rice distributions 1677.5.1 Rayleigh distribution 1687.5.2 Rice distribution 1707.6 Complex-valued normal variables 1727.6.1 Complex-valued Gaussian variables and their properties 1727.6.2 Multivariate Gaussian variables 1737.7 Summary of Chapter 7 1767.8 Discussion and further reading 1777.9 Problems 178x ContentsPart II Transform methods, bounds, and limits 1838 Moment-generating function and characteristic function 1858.1 Moment-generating function (MGF) 1858.1.1 Moment-generating function of one random variable 1858.1.2 Moment-generating function of sum of independent randomvariables 1898.1.3 Joint moment-generating function of multivariate randomvariables 1908.2 Characteristic function (CF) 1928.2.1 Characteristic function of one random variable 1928.2.2 Sum of independent random variables and convolution 1968.2.3 Moment generation from characteristic function 1988.2.4 Joint characteristic function of multivariate randomvariables 1998.2.5 Application of the characteristic function: the central limittheorem (CLT) 2018.2.6 Characteristic function of multivariate complex-valuednormal variables 2028.3 Summary of Chapter 8 2048.4 Discussion and further reading 2058.5 Problems 2069 Generating functions and Laplace transform 2119.1 Generating function 2119.1.1 Probability-generating function (PGF) 2129.1.2 Sum of independent variables and convolutions 2159.1.3 Sum of a random number of random variables 2179.1.4 Inverse transform of generating functions 2189.2 Laplace transform method 2269.2.1 Laplace transform and moment generation 2269.2.2 Inverse Laplace transform 2299.3 Summary of Chapter 9 2349.4 Discussion and further reading 2359.5 Problems 23510 Inequalities, bounds, and large deviation approximation 24110.1 Inequalities frequently used in probability theory 24110.1.1 CauchySchwarz inequality 24110.1.2 Jensens inequality 24510.1.3 Shannons lemma and log-sum inequality 24610.1.4 Markovs inequality 248Contents xi10.1.5 Chebyshevs inequality 24910.1.6 Kolmogorovs inequalities for martingales andsubmartingales 25010.2 Chernoffs bounds 25310.2.1 Chernoffs bound for a single random variable 25310.2.2 Chernoffs bound for a sum of i.i.d. random variables 25410.3 Large deviation theory 25710.3.1 Large deviation approximation 25710.3.2 Large deviation rate function 26310.4 Summary of Chapter 10 26710.5 Discussion and further reading 26810.6 Problems 26911 Convergence of a sequence of random variables and the limit theorems 27711.1 Preliminaries: convergence of a sequence of numbers or functions 27711.1.1 Sequence of numbers 27711.1.2 Sequence of functions 27811.2 Types of convergence for sequences of random variables 28011.2.1 Convergence in distribution 28011.2.2 Convergence in probability 28211.2.3 Almost sure convergence 28511.2.4 Convergence in the rth mean 28811.2.5 Relations between the modes of convergence 29211.3 Limit theorems 29311.3.1 Innite sequence of events 29411.3.2 Weak law of large numbers (WLLN) 29811.3.3 Strong laws of large numbers (SLLN) 30011.3.4 The central limit theorem (CLT) revisited 30311.4 Summary of Chapter 11 30611.5 Discussion and further reading 30711.6 Problems 308Part III Random processes 31312 Random processes 31512.1 Random process 31512.2 Classication of random processes 31612.2.1 Discrete-time versus continuous-time processes 31612.2.2 Discrete-state versus continuous-state processes 31712.2.3 Stationary versus nonstationary processes 31712.2.4 Independent versus dependent processes 31812.2.5 Markov chains and Markov processes 31812.2.6 Point processes and renewal processes 321xii Contents12.2.7 Real-valued versus complex-valued processes 32112.2.8 One-dimensional versus vector processes 32212.3 Stationary random process 32212.3.1 Strict stationarity versus wide-sense stationarity 32312.3.2 Gaussian process 32612.3.3 Ergodic processes and ergodic theorems 32712.4 Complex-valued Gaussian process 32912.4.1 Complex-valued Gaussian random variables 32912.4.2 Complex-valued Gaussian process 33012.4.3 Hilbert transform and analytic signal 33312.4.4 Complex envelope 33812.5 Summary of Chapter 12 33912.6 Discussion and further reading 34012.7 Problems 34013 Spectral representation of random processes and time series 34313.1 Spectral representation of random processes and time series 34313.1.1 Fourier series 34313.1.2...</p>

Recommended

View more >