probability%2c random processes%2c and statistical analysis

Download Probability%2C Random Processes%2C and Statistical Analysis

Post on 14-Oct-2014




2 download

Embed Size (px)


Probability, Random Processes, and Statistical AnalysisTogether with the fundamentals of probability, random processes, and statistical analy-sis, this insightful book also presents a broad range of advanced topics and applicationsnot covered in other textbooks.Advanced topics include: Bayesian inference and conjugate priors Chernoff bound and large deviation approximation Principal component analysis and singular value decomposition Autoregressive moving average (ARMA) time series Maximum likelihood estimation and the Expectation-Maximization (EM) algorithm Brownian motion, geometric Brownian motion, and Ito process BlackScholes differential equation for option pricing Hidden Markov model (HMM) and estimation algorithms Bayesian networks and sum-product algorithm Markov chain Monte Carlo methods Wiener and Kalman lters Queueing and loss networksThe book will be useful to students and researchers in such areas as communications,signal processing, networks, machine learning, bioinformatics, and econometrics andmathematical nance. With a solutions manual, lecture slides, supplementary materials,and MATLAB programs all available online, it is ideal for classroom teaching as wellas a valuable reference for professionals.Hisashi Kobayashi is the Sherman Fairchild University Professor Emeritus at PrincetonUniversity, where he was previously Dean of the School of Engineering and AppliedScience. He also spent 15 years at the IBM Research Center, Yorktown Heights, NY,and was the Founding Director of IBM Tokyo Research Laboratory. He is an IEEE LifeFellow, an IEICE Fellow, was elected to the Engineering Academy of Japan (1992), andreceived the 2005 Eduard Rhein Technology Award.Brian L. Mark is a Professor in the Department of Electrical and Computer Engineer-ing at George Mason University. Prior to this, he was a research staff member at theNEC C&C Research Laboratories in Princeton, New Jersey, and in 2002 he received aNational Science Foundation CAREER award.William Turin is currently a Consultant at AT&T Labs Research. As a Member of Tech-nical Staff at AT&T Bell Laboratories and later a Technology Consultant at AT&TLabs Research for 21 years, he developed methods for quantifying the performanceof communication systems. He is the author of six books and numerous papers.This book provides a very comprehensive, well-written and modern approach to thefundamentals of probability and random processes, together with their applications inthe statistical analysis of data and signals. It provides a one-stop, unied treatment thatgives the reader an understanding of the models, methodologies, and underlying princi-ples behind many of the most important statistical problems arising in engineering andthe sciences today.Dean H. Vincent Poor, Princeton UniversityThis is a well-written, up-to-date graduate text on probabilty and random processes. Itis unique in combining statistical analysis with the probabilistic material. As noted bythe authors, the material, as presented, can be used in a variety of current applicationareas, ranging from communications to bioinformatics. I particularly liked the historicalintroduction, which should make the eld exciting to the student, as well as the intro-ductory chapter on probability, which clearly describes for the student the distinctionbetween the relative frequency and axiomatic approaches to probability. I recommendit unhesitatingly. It deserves to become a leading text in the eld.Professor Emeritus Mischa Schwartz, Columbia UniversityHisashi Kobayashi, Brian L. Mark, and William Turin are highly experienced uni-versity teachers and scientists. Based on this background, their book covers not onlyfundamentals, but also a large range of applications. Some of them are treated in atextbook for the rst time. Without any doubt the book will be extremely valuable tograduate students and to scientists in universities and industry. Congratulations to theauthors!Professor Dr.-Ing. Eberhard Hnsler, Technische Universitt DarmstadtAn up-to-date and comprehensive book with all the fundamentals in probability, ran-dom processes, stochastic analysis, and their interplays and applications, which lays asolid foundation for the students in related areas. It is also an ideal textbook with verelatively independent but logically interconnected parts and the corresponding solutionmanuals and lecture slides. Furthermore, to my best knowledge, similar editing in PartIV and Part V cant be found elsewhere.Zhisheng Niu, Tsinghua UniversityProbability, Random Processes,and Statistical AnalysisHI SASHI KOBAYASHIPrinceton UniversityBRI AN L. MARKGeorge Mason UniversityWI LLI AM TURI NAT&T Labs ResearchC A M B R I D G E U N I V E R S I T Y P R E S SCambridge, New York, Melbourne, Madrid, Cape TownSingapore, So Paulo, Delhi, Tokyo, Mexico CityCambridge University PressThe Edinburgh Building, Cambridge CB2 8RU, UKPublished in the United States of America by Cambridge University Press, New Yorkwww.cambridge.orgInformation on this title: _Cambridge University Press 2012This publication is in copyright. Subject to statutory exceptionand to the provisions of relevant collective licensing agreements,no reproduction of any part may take place without the writtenpermission of Cambridge University Press.First published 2012Printed in the United Kingdom at the University Press, CambridgeA catalogue record for this publication is available from the British LibraryISBN 978-0-521-89544-6 HardbackAdditional resources for this publication at University Press has no responsibility for the persistence oraccuracy of URLs for external or third-party internet websites referred toin this publication, and does not guarantee that any content on suchwebsites is, or will remain, accurate or appropriate.ToMasae, Karen, and GalinaContentsList of abbreviations and acronyms page xviiiPreface xxiiiAcknowledgments xxx1 Introduction 11.1 Why study probability, random processes, and statistical analysis? 11.1.1 Communications, information, and control systems 21.1.2 Signal processing 21.1.3 Machine learning 31.1.4 Biostatistics, bioinformatics, and related elds 41.1.5 Econometrics and mathematical nance 41.1.6 Queueing and loss systems 61.1.7 Other application domains 61.2 History and overview 71.2.1 Classical probability theory 71.2.2 Modern probability theory 91.2.3 Random processes 101.2.4 Statistical analysis and inference 121.3 Discussion and further reading 14Part I Probability, random variables, and statistics 152 Probability 172.1 Randomness in the real world 172.1.1 Repeated experiments and statistical regularity 172.1.2 Random experiments and relative frequencies 182.2 Axioms of probability 182.2.1 Sample space 192.2.2 Event 192.2.3 Probability measure 202.2.4 Properties of probability measure 21viii Contents2.3 Bernoulli trials and Bernoullis theorem 262.4 Conditional probability, Bayes theorem, and statistical independence 302.4.1 Joint probability and conditional probability 302.4.2 Bayes theorem 312.4.3 Statistical independence of events 342.5 Summary of Chapter 2 362.6 Discussion and further reading 372.7 Problems 373 Discrete random variables 423.1 Random variables 423.1.1 Distribution function 433.1.2 Two random variables and joint distribution function 443.2 Discrete random variables and probability distributions 453.2.1 Joint and conditional probability distributions 463.2.2 Moments, central moments, and variance 503.2.3 Covariance and correlation coefcient 513.3 Important probability distributions 533.3.1 Bernoulli distribution and binomial distribution 543.3.2 Geometric distribution 553.3.3 Poisson distribution 563.3.4 Negative binomial (or Pascal) distribution 593.3.5 Zipfs law and zeta distribution 623.4 Summary of Chapter 3 653.5 Discussion and further reading 663.6 Problems 664 Continuous random variables 724.1 Continuous random variables 724.1.1 Distribution function and probability density function 724.1.2 Expectation, moments, central moments, and variance 734.2 Important continuous random variables and their distributions 754.2.1 Uniform distribution 754.2.2 Exponential distribution 764.2.3 Gamma distribution 784.2.4 Normal (or Gaussian) distribution 804.2.5 Weibull distributions 864.2.6 Pareto distribution 884.3 Joint and conditional probability density functions 904.3.1 Bivariate normal (or Gaussian) distribution 924.3.2 Multivariate normal (or Gaussian) distribution 944.4 Exponential family of distributions 954.5 Bayesian inference and conjugate priors 97Contents ix4.6 Summary of Chapter 4 1034.7 Discussion and further reading 1044.8 Problems 1045 Functions of random variables and their distributions 1125.1 Function of one random variable 1125.2 Function of two random variables 1155.3 Two functions of two random variables and the Jacobian matrix 1195.4 Generation of random variates for Monte Carlo simulation 1235.4.1 Random number generator (RNG) 1245.4.2 Generation of variates from general distributions 1255.4.3 Generation of normal (or Gaussian) variates 1305.5 Summary of Chapter 5 1315.6 Discussion and further reading 1315.7 Problems 1326 Fundamentals of statistical data analysis 1386.1 Sample mean and sample variance 1386.2 Relative frequency and histograms 1406.3 Graphical presentations 1416.3.1 Histogram on probability paper 1426.3.2 Log-survivor function curve 1446.3.3 Hazard function and mean residual life curves 1486.3.4 Dot diagram and correlation coefcient 1496.4 Summary of C


View more >