Index

Absolute error loss, 86Absolutely continuous

function, 91random variable, 116, 239

Addition of matrices, 29Adjoint, 34Adjugate, 34Admissible estimator, 86Approximation of

density functions, 456functions, 403, 495integrals, 517, 533normal integral, 460, 469quantiles of distributions, 456, 458, 468

Arcsin x, 6, 78Asymptotically equal, 66, 147Asymptotic distributions, 120Average squared bias, 360, 431Average variance, 360, 400

Balanced mixed model, 43Behrens�Fisher test, 324Bernoulli sequence, 14Bernstein polynomial, 406, 410Best linear unbiased estimator, 312Beta function, 256Bias, 359, 360

criterion, 359, 366in the model, 359, 366

Biased estimator, 196Binomial distribution, 194Binomial random variable, 14, 203Bolzano�Weierstrass theorem, 11Borel field, 13Boundary, 262, 297

point, 262Bounded sequence, 132, 262Bounded set, 9, 13, 72, 262

Bounded variation, function of, 210, 212Bounds on eigenvalues, 41Box and Cox transformation, 120Box�Draper determinant criterion, 400Box�Lucas criterion, 368, 422, 424�425

Calibration, 242Canonical correlation coefficient, 56Cartesian product, 3, 21Cauchy criterion, 137, 221Cauchy distribution, 240Cauchy�Schwarz inequality, 25, 229Cauchy sequence, 138, 222Cauchy’s condensation test, 153Cauchy’s mean value theorem, 103, 105, 129Cauchy’s product, 163Cauchy’s test, 149Cell, 294Center-point replications, 400Central limit theorem, 120, 516Chain rule, 97, 116Change of variables, 219, 299Characteristic equation, 37Characteristic function, 505

inversion formula for, 507Characteristic root, 36Characteristic vector, 37Chebyshev polynomials, 415, 444

discrete, 462of the first kind, 444of the second kind, 445zeros of, 415, 444

Chebyshev’s inequality, 19, 184, 251, 259Chi-squared random variable, 246Christoffel’s identity, 443Closed set, 11�13, 72, 262Coded levels, 347Coefficient of variation, 242

665

Advanced Calculus with Applications in Statistics, Second Edition. Andre I. KhuriCopyright 2003 John Wiley & Sons, Inc.

ISBN: 0-471-39104-2

INDEX666

Cofactor, 31, 357Compact set, 13Comparison test, 144Complement of a set, 2

relative, 2Composite function, 6, 97, 265, 276Concave function, 79Condition number, 47Confidence intervals, simultaneous, 382, 384,

388Consistent estimator, 83Constrained maximization, 329Constrained optimization, 288Constraints, equality, 288Continuous distribution, 14, 82Continuous function, 66, 210, 264

absolutely, 91left, 68piecewise, 481, 486, 489, 498right, 68, 83

Continuous random variable, 14, 82, 116, 239Contrast, 384Controlled random search procedure, 336, 426Convergence

almost surely, 192in distribution, 120in probability, 83, 191in quadratic mean, 191

Convex function, 79, 84, 98Convex set, 79Convolution, 499Cornish�Fisher expansion, 460, 468Correlation coefficient, 311Countable set, 6Covering of a set, 12

open, 12Cubic spline regression model, 430Cumulant generating function, 189Cumulants, 189, 459Cumulative distribution function, 14, 82, 304

joint, 14Curvature, in response surface, 342Cylindrical data, 505

d’Alembert’s test, 148Davidon�Fletcher�Powell method, 331Deleted neighborhood, 57, 262Density function, 14, 116, 239, 304

approximation of, 456bivariate, 305marginal, 304

Derivative, 93directional, 273

partial, 267total, 270

DesignA-optimal, 365approximate, 363Box�Behnken, 359center, 344, 356central composite, 347, 353, 358, 400dependence of, 368�369, 422D-optimal, 363, 365, 369efficiency of, 365E-optimal, 365exact, 363first-order, 340, 356, 358G-optimal, 364� -optimal, 3991matrix, 356measure, 362�363, 365moments, 362nonlinear model, 367optimal, 362, 425orthogonal, 358parameter-free, 426Plackett�Burman, 358points, 353, 360response surface, 339, 355, 359rotatable, 350, 356second-order, 344, 3583k factorial, 3582 k factorial, 340, 358

Design measurecontinuous, 362discrete, 363, 366

Determinant, 31Hessian, 276Jacobian, 268Vandermonde’s, 411

DETMAX algorithm, 366Diagonalization of a matrix, 38Differentiable function, 94, 113Differential operator, 277Differentiation under the integral sign, 301Directional derivative, 273Direct product, 30, 44Direct search methods, 332Direct sum

of matrices, 30of vector subspaces, 25

Discontinuity, 67first kind, 67second kind, 67

Discrete distribution, 14Discrete random variable, 14, 182

INDEX 667

Disjoint sets, 2Distribution

binomial, 194Cauchy, 240chi-squared, 245continuous, 14, 82discrete, 14exponential, 20, 91, 131F, 383�384gamma, 251hypergeometric, 463logarithmic series, 193logistic, 241marginal, 304multinomial, 325negative binomial, 182, 195normal, 120, 306, 311, 400Pearson Type IV, 503Poisson, 119, 183, 204power series, 193t, 259truncated Poisson, 373uniform, 83, 92, 116, 250

Domain of a function, 5Dot product, 23

Eigenvalues, 36�37Eigenvectors, 37EM algorithm, 372, 375Empty set, 1Equal in distribution, 15Equality of matrices, 28Equivalence class, 4Equivalence theorem, 365Equivalent sets, 5Error loss

absolute, 86squared, 86

Estimable linear function, 45, 382, 387Estimation of unknown densities, 461Estimator

admissible, 86best linear unbiased, 312biased, 196consistent, 83least absolute value, 327M, 327minimax, 86minimum norm quadratic unbiased, 380minimum variance quadratic unbiased, 382quadratic, 379ridge, 56, 196

Euclidean norm, 24, 42, 179, 261

Euclidean space, 21Euler’s theorem, 317Event, 13Exchangeable random variables, 15Exponential distribution, 20, 91, 131Extrema

absolute, 113local, 113

Factorial moment, 191Failure rate, 130F distribution, 383�384Fisher’s method of scoring, 374Fixed effects, 44Fourier

coefficients, 472integral, 488transform, 497, 507

Fourier series, 471convergence of, 475differentiation of, 483integration of, 483

Fubini’s theorem, 297Function, 5

absolutely continuous, 91bounded, 65, 72bounded variation, 210, 212composite, 6, 97, 265, 276concave, 79continuous, 67, 210, 264convex, 79, 84, 98differentiable, 94, 113homogeneous, 317implicit, 273inverse, 6, 76, 102left continuous, 68limit of, 57Lipschitz continuous, 75, 409loss, 86lower semicontinuous, 90monotone, 76, 101, 116, 210multivariable, 261one-to-one, 5onto, 5periodic, 473, 489, 504Riemann integrable, 206, 210, 215Riemann�Stieltjes integrable, 234right continuous, 68, 83risk, 86uniformly continuous, 68, 74, 82, 265,

407upper semicontinuous, 90

Functionally dependent, 318

INDEX668

Function of a random variable, 116Fundamental theorem of calculus, 219

Gamma distribution, 251Gamma function, 246, 532

approximation for, 532Gaussian quadrature, 524Gauss�Markov theorem, 55Gauss’s test, 156Generalized distance approach, 370Generalized inverse, 36Generalized simulated annealing method, 338Generalized variance, 363Geometric mean, 84, 230Geometric series, 142Gibbs phenomenon, 514Gradient, 275

methods, 329Gram�Charlier series, 457Gram�Schmidt orthonormalization, 24Greatest-integer function, 238Greatest lower bound, 9, 72

Harmonicfrequencies, 501mean, 85series, 145

Hazard rate, 131Heine�Borel theorem, 13Hermite polynomials, 447

applications of, 456, 542normalized, 461

Hessiandeterminant, 276matrix, 275, 285, 374

Heteroscedasticity, 118Holder’s inequality, 230�231¨Homogeneous function, 317

Euler’s theorem for, 317Homoscedasticity, 118Hotelling’s T 2 statistic, 49Householder matrix, 38Hypergeometric

distribution, 463probability, 462series, 157

Idempotent matrix, 38Ill conditioning, 46Image of a set, 5Implicit function, 273Implicit function theorem, 282Importance sampling, 537

Improper integral, 220absolutely convergent, 222, 226conditionally convergent, 222, 226of the first kind, 220, 528of the second kind, 221, 225

Indefinite integral, 217Indeterminate form, 107Inequality

Cauchy�Schwarz, 25, 229Chebyshev’s, 19, 184, 251Holder’s, 230, 231¨Jensen’s, 84, 233Markov’s, 19, 185Minkowski’s, 232

Infimum, 9, 73Inner product, 23Input variables, 339, 351Integral

improper, 220indefinite, 217Riemann, 206, 293Riemann�Stieltjes, 234

Integral test, 153, 224Interaction, 43Interior point, 262Intermediate-value theorem, 71, 217Interpolation, 410

error, 414, 416, 423Lagrange, 411, 419, 422points, 411�412, 414, 423

Intersection, 2Interval of convergence, 174, 190Inverse function, 6, 76, 102Inverse function theorem, 280, 305Inverse of a matrix, 34Inverse regression, 242Irrational number, 9

Jacobiandeterminant, 268matrix, 268

Jacobi polynomials, 443applications of, 462

Jensen’s inequality, 84, 233Jordan content, 298Jordan measurable, 298

Kernel of a linear transformation, 26Khinchine’s theorem, 192Knots, 418, 431Kolmogorov’s theorem, 192Kronecker product, 30Kummer’s test, 154

INDEX 669

Lack of fit, 342, 399Lagrange interpolating polynomial, 411, 416,

423, 518, 521Lagrange interpolation, 413, 419, 422

accuracy of, 413Lagrange multipliers, 288, 312, 329, 341, 344,

352, 383Laguerre polynomials, 451

applications of, 462Laplace

approximation, 531, 546, 548method of, 531, 534, 546

Law of large numbers, 84Bernoulli’s, 204strong, 192weak, 192

Least-squares estimate, 46, 344Least-squares polynomial approximation,

453�455Least upper bound, 9, 73Legendre polynomials, 440, 442Leibniz’s formula, 127, 443, 449, 452Level of significance, 15L’Hospital’s rule, 103, 120, 254Likelihood

equations, 308, 373function, 308, 373

Limit, 57left sided, 59lower, 136of a multivariable function, 262one sided, 58right sided, 59of a sequence, 133subsequential, 136two sided, 58upper, 136

Limit point, 11, 262Linearly dependent, 22Linearly independent, 23Linear model, 43, 46, 121, 195, 355, 367,

382balanced, 43first order, 340second order, 343

Linear span, 23Linear transformation, 25

kernel of, 26one-to-one, 27

Lipschitz condition, 75Lipschitz continuous function, 75, 409Logarithmic series distribution, 193Logistic distribution, 241

Loss function, 86Lower limit, 136Lower semicontinuous function, 90Lower sum, 206, 294

Maclaurin’s integral test, 153Maclaurin’s series, 111Mapping, 5Markov chain Monte Carlo, 549Markov’s inequality, 19, 185Matrix, 28

diagonal, 28full column rank, 34full rank, 34full row rank, 34Hessian, 275, 285, 374Householder, 38idempotent, 38identity, 28inverse of, 34Jacobian, 268moment, 363orthogonal, 38, 351, 357partitioned, 30rank of, 33singular, 32, 37skew symmetric, 29square, 28symmetric, 29trace of, 29transpose of, 29

Maximumabsolute, 113, 283, 346local, 113, 283relative, 113

Maximum likelihood estimate, 308, 372Mean, 117, 239Mean response, 343, 359, 367, 423

maximum of, 343Mean squared error, 360, 431

integrated, 360, 431Mean value theorem, 99�100, 271

for integrals, 217Median, 124Mertens’s theorem, 163Method of least squares, 121Minimax estimator, 86Minimization of prediction variance, 356Minimum

absolute, 113, 283, 336, 346local, 113, 283relative, 113

Minimum bias estimation, 398

INDEX670

Minimum norm quadratic unbiased estimation,327, 378

Minkowski’s inequality, 232Minor, 31

leading principal, 32, 40, 285principal, 32, 292

Modal approximation, 549Modulus of continuity, 407Moment generating function, 186, 250, 506Moment matrix, 363Moments, 182, 505

central, 182, 240, 457design, 362factorial, 191first negative, 242noncentral, 182, 240region, 361

Monotone function, 76, 101, 210Monotone sequence, 133Monte Carlo method, 535, 540

error bound for, 537variance reduction, 537

Monte Carlo simulation, 83Multicollinearity, 46, 196, 353, 387Multimodal function, 336, 338Multinomial distribution, 325Multiresponse

experiment, 370function, 370optimization, 370

Multivariable function, 261composite, 265, 276continuous, 264inverse of, 280limit of, 262optima of, 283partial derivative of, 267Riemann integral of, 293Taylor’s theorem for, 277uniformly continuous, 265

Negative binomial, 182, 195Neighborhood, 10, 262

deleted, 57, 262Newton�Cotes methods, 523Newton�Raphson method, 331, 373Noncentral chi-squared, 44Noncentrality parameter, 44Nonlinear model, 367

approximate linearization of, 422design for, 367

Nonlinear parameter, 367Norm, 179

Euclidean, 24, 42, 179, 261minimum, 380of a partition, 205spectral, 179

Normal distribution, 120, 306bivariate, 311p-variate, 400

Normal integral, 460, 469approximation of, 460, 469

Normal random variable, 245Null space, 26

One-to-one correspondence, 5One-to-one function, 5One-way classification model, 387o notation, 66, 117O notation, 65, 156Open set, 10, 262Optimality of design

A, 365D, 364, 367, 431E, 365G, 364

Optimization techniques, 339Optimum

absolute, 113compromise, 371conditions, 370ideal, 371local, 113, 284

Ordered n-tuple, 4Ordered pair, 3Orthogonal complement, 25Orthogonal design, 358Orthogonal functions, 437Orthogonal matrix, 38, 351, 357

parameterization, 49Orthogonal polynomials, 437, 453

Chebyshev, 444of the first kind, 444of the second kind, 445zeros of, 444

on a finite set, 455Hermite, 447

applications of, 456Jacobi, 443Laguerre, 451least-squares approximation with, 453Legendre, 440, 442sequence of, 437zeros of, 440

Orthogonal vectors, 24Orthonormal basis, 24

INDEX 671

Parameterization of orthogonal matrices, 49Parseval’s theorem, 496Partial derivative, 267Partially nonlinear model, 369Partial sum, 140Partition, 205, 294

of a set, 5Periodic

extension, 482function, 473, 489, 504

Periodogram, 501Piecewise continuous function, 481, 486, 489,

498Poisson approximation, 194Poisson distribution, 119, 183, 204

truncated, 373Poisson process, 122, 131Poisson random variable, 14, 124, 183Polynomial

approximation, 403Bernstein, 406, 410Chebyshev, 444Hermite, 447, 456, 542interpolation, 410Jacobi, 443, 462Lagrange interpolating, 411, 416, 423Laguerre, 451, 462Legendre, 440, 442piecewise, 418trigonometric, 495, 500, 504

Power series, 174distribution, 194

Prediction equation, 348Prediction variance, 347, 350, 352, 356

minimization of, 356standardized, 363�364

Principal components, 328Probability of an event, 13Probability generating function, 190

continuity theorem for, 192Probability space, 13Product of matrices, 29Projection of a vector, 25Proper subset, 1

Quadratic form, 39extrema of, 48nonnegative definite, 40positive definite, 40positive semidefinite, 40

Quadrature, 524Gauss�Chebyshev, 526Gauss�Hermite, 528, 542

Gauss�Jacobi, 526Gauss�Laguerre, 528Gauss�Legendre, 526

Raabe’s test, 155Radius of convergence, 174Random effects, 44Random variable, 13

absolutely continuous, 116, 239Bernoulli, 194binomial, 14, 203chi-squared, 246continuous, 14, 82, 116, 239discrete, 14, 182function of, 116normal, 245Poisson, 14, 124, 183

Random vector, 304transformations of, 305

Range of a function, 5Rank of a matrix, 33Rational number, 8Ratio test, 148Rayleigh’s quotient, 41

infimum of, 41supremum of, 41

Rearrangement of series, 159Refinement, 206, 214, 294Region of convergence, 174Relation, 4

congruence, 5equivalence, 4reflexive, 4symmetric, 4transitive, 4

Remainder, 110, 257, 279Response

constrained, 352maximum, 340minimum, 340optimum, 350predicted, 341, 344, 355, 504primary, 352surface, 340

Response surface design, 339, 355, 359Response surface methodology, 339, 355, 367,

504Response variable, 121Restricted least squares, 429Ridge analysis, 343, 349, 352

modified, 350, 354standard, 354

Ridge estimator, 56, 196

INDEX672

Ridge plots, 346Ridge regression, 195Riemann integral, 206, 293

double, 295iterated, 295n-tuple, 295

Riemann�Stieltjes integral, 234Risk function, 86Rodrigues formula, 440, 443, 447, 451Rolle’s theorem, 99, 110Root test, 149Rotatability, 347

Saddle point, 114, 284, 344, 349Saddlepoint approximation, 549Sample, 14Sample space, 13Sample variance, 14Satterthwaite’s approximation, 324Scalar, 21

product, 23Scheffe’s confidence intervals, 382, 384´Schur’s theorem, 42Sequence, 132, 262

bounded, 132, 262Cauchy, 138, 222convergent, 133, 262divergent, 133of functions, 165, 227limit of, 133limit infimum of, 136, 148, 151limit supremum of, 136, 148, 151of matrices, 178monotone, 133of orthogonal polynomials, 437subsequential limit of, 136uniformly convergent, 166, 169, 228

Series, 140absolutely convergent, 143, 158, 174alternating, 158conditionally convergent, 158convergent, 141divergent, 141Fourier, 471of functions, 165geometric, 142Gram�Charlier, 457harmonic, 145hypergeometric, 157Maclaurin’s, 111of matrices, 178, 180multiplication of, 162of positive terms, 144power, 174

rearrangement of, 159sum of, 140Taylor’s, 111, 121trigonometric, 471uniformly convergent, 166, 169Ž .Set s , 1

bounded, 9, 72, 262closed, 11, 72, 262compact, 13complement of, 2connected, 12convex, 79countable, 6covering of, 12disconnected, 12empty, 1finite, 6image of, 5open, 10, 262partition of, 5subcovering of, 13uncountable, 7universal, 2

Sherman�Morrison formula, 54Sherman�Morrison�Woodbury formula, 54,

389Simplex method, 332Simpson’s method, 521Simulated annealing, 338Simultaneous diagonalization, 40Singularity of a function, 225Singular-value decomposition, 39, 45Singular values of a matrix, 46Size of test, 15Spectral decomposition theorem, 38, 181, 351Spherical polar coordinates, 322Spline functions, 418, 428

approximation, 418, 421cubic, 419, 428designs for, 430error of approximation by, 421linear, 418properties of, 418

Squared error loss, 86Stationary point, 284, 345Stationary value, 113Statistic, 14

first-order, 20Hotelling’s T 2, 49nth-order, 20

Steepest ascentmethod of, 330, 340path of, 396

Steepest descent

INDEX 673

method of, 329path of, 343

Step function, 237Stieltjes moment problem, 185Stirling’s formula, 532Stratified sampling, 313Stratum, 313Submatrix, 29

leading principal, 29principal, 29

Subsequence, 135, 262Subsequential limit, 136Subset, 1Supremum, 9, 73Symmetric difference, 17

Taylor’s formula, 110Taylor’s series, 111, 121

remainder of, 110, 257, 279Taylor’s theorem, 108, 114, 277t distribution, 259Time series, 500Topological space, 10Topology, 10

basis for, 10Total derivative, 270Transformation

Box and Cox, 120of continuous random variables, 246of random vectors, 305variance stabilizing, 118

Translation invariance, 379Trapezoidal method, 517Triangle inequality, 25

TrigonometricŽ .polynomial s , 495, 500, 504

series, 471Two-way crossed classification model, 43, 401

Uncountable set, 7Uniform convergence

of sequences, 166, 169, 228of series, 167, 169

Uniform distribution, 83, 92, 116, 250Uniformly continuous function, 68, 74, 82, 265,

407Union, 2Universal set, 2Upper limit, 136Upper semicontinuous function, 90Upper sum, 206, 294

Variance, 117, 239Variance components, 44, 378Variance criterion, 359, 366Variance stabilizing transformation, 118Vector, 21

column, 28row, 28zero, 21

Vector space, 21basis for, 23dimension of, 23

Vector subspace, 22Voronovsky’s theorem, 410

Weierstrass approximation theorem, 403Weierstrass’s M-test, 167, 172