lecture 2 probability and measurement error, part 1
DESCRIPTION
Lecture 2 Probability and Measurement Error, Part 1. Syllabus. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/1.jpg)
Lecture 2
Probability and Measurement Error, Part 1
![Page 2: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/2.jpg)
SyllabusLecture 01 Describing Inverse ProblemsLecture 02 Probability and Measurement Error, Part 1Lecture 03 Probability and Measurement Error, Part 2 Lecture 04 The L2 Norm and Simple Least SquaresLecture 05 A Priori Information and Weighted Least SquaredLecture 06 Resolution and Generalized InversesLecture 07 Backus-Gilbert Inverse and the Trade Off of Resolution and VarianceLecture 08 The Principle of Maximum LikelihoodLecture 09 Inexact TheoriesLecture 10 Nonuniqueness and Localized AveragesLecture 11 Vector Spaces and Singular Value DecompositionLecture 12 Equality and Inequality ConstraintsLecture 13 L1 , L∞ Norm Problems and Linear ProgrammingLecture 14 Nonlinear Problems: Grid and Monte Carlo Searches Lecture 15 Nonlinear Problems: Newton’s Method Lecture 16 Nonlinear Problems: Simulated Annealing and Bootstrap Confidence Intervals Lecture 17 Factor AnalysisLecture 18 Varimax Factors, Empircal Orthogonal FunctionsLecture 19 Backus-Gilbert Theory for Continuous Problems; Radon’s ProblemLecture 20 Linear Operators and Their AdjointsLecture 21 Fréchet DerivativesLecture 22 Exemplary Inverse Problems, incl. Filter DesignLecture 23 Exemplary Inverse Problems, incl. Earthquake LocationLecture 24 Exemplary Inverse Problems, incl. Vibrational Problems
![Page 3: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/3.jpg)
Purpose of the Lecture
review random variables and their probability density functions
introduce correlation and the multivariate Gaussian distribution
relate error propagation to functions of random variables
![Page 4: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/4.jpg)
Part 1
random variables and their probability density functions
![Page 5: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/5.jpg)
d=?
random variable, dno fixed value until it is realized
d=?indeterminate
d=1.04indeterminate
d=0.98
![Page 6: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/6.jpg)
random variables have systematics
tendency to takes on some values more often than others
![Page 7: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/7.jpg)
0 20 40 60 80 100 120 140 160 180 2001
2
3
4
5
6
7
8
N realization of data
index, i
d i 50
10
![Page 8: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/8.jpg)
(C) (B) (A)
0 5 100
0.1
0.2
0.3
0.4
0.5
d
p(d)
0 5 100
10
20
30
d
coun
ts
0 5 100
0.1
0.2
0.3
0.4
0.5
d
p(d)
![Page 9: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/9.jpg)
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
d
p(d)p(d)
Δd d
probability of variable being between d and d+Δd is p(d) Δd
![Page 10: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/10.jpg)
in general probability is the integral
probability that d is between d1 and d2
![Page 11: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/11.jpg)
the probability that d has some value is 100% or unity
probability that d is between its minimum and
maximum bounds, dmin and dmax
![Page 12: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/12.jpg)
How do these two p.d.f.’s differ?
dp(d)
dp(d)00
55
![Page 13: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/13.jpg)
Summarizing a probability density function
typical value“center of the p.d.f.”
amount of scatter around the typical value“width of the p.d.f.”
![Page 14: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/14.jpg)
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
d
p(d)p(d)
ddML
Several possibilities for a typical value
point beneath the peak or “maximum likelihood point” or
“mode”
![Page 15: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/15.jpg)
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
d
p(d)p(d)
ddmedianpoint dividing area in half or “median”
50% 50%
![Page 16: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/16.jpg)
0 2 4 6 8 100
0.1
0.2
0.3
0.4
0.5
d
p(d)p(d)
d<d>balancing point or
“mean” or “expectation”
![Page 17: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/17.jpg)
can all be different
dML ≠ dmedian ≠ <d>
![Page 18: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/18.jpg)
formula for“mean” or “expected value”
<d>
![Page 19: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/19.jpg)
≈ sd
ds
≈ s NsN
data
histogram
Ns
dsp≈ s P(ds)
probability distribution
step 1: usual formula for mean
step 2: replace data with its histogram
step 3: replace histogram with probability distribution.
<d><d><d>
![Page 20: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/20.jpg)
If the data are continuous, use analogous formula containing an
integral:
≈ s P(ds)<d>
<d>
![Page 21: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/21.jpg)
quantifying width
![Page 22: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/22.jpg)
This function grows away from the typical value:
q(d) = (d-<d>)2so the function q(d)p(d) is
small if most of the area is near <d> , that is, a narrow p(d)large if most of the area is far from <d> , that is, a wide p(d)
so quantify width as the area under q(d)p(d)
![Page 23: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/23.jpg)
0 5 100
5
10
d
q(d)
0 5 100
0.5
1
dp(
d)
0 5 100
0.5
1
d
q(d)
*p(d
)
0 5 100
5
10
d
q(d)
0 5 100
0.5
1
d
p(d)
0 5 100
0.5
1
dq(
d)*p
(d)
(C)(B)(A)
(F)(E)(D)
![Page 24: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/24.jpg)
variance
width is actually square root of variance, that is, σmean
![Page 25: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/25.jpg)
estimating mean and variance from data
![Page 26: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/26.jpg)
estimating mean and variance from data
usual formula for “sample
mean”
usual formula for square of
“sample standard
deviation”
![Page 27: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/27.jpg)
MabLab scripts for mean and variance
dbar = Dd*sum(d.*p);q = (d-dbar).^2; sigma2 = Dd*sum(q.*p); sigma = sqrt(sigma2);
from tabulated p.d.f. p
from realizations of data
dbar = mean(dr);sigma = std(dr);sigma2 = sigma^2;
![Page 28: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/28.jpg)
two important probability density functions:
uniform
Gaussian (or Normal)
![Page 29: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/29.jpg)
uniform p.d.f.
ddmin dmax
p(d)1/(dmax- dmin)
probability is the same everywhere in the range of possible values
box-shaped function
![Page 30: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/30.jpg)
Large probability near the mean, d. Variance is σ2.
0 10 20 30 40 50 60 70 80 90 1000
0.02
0.04
0.06
0.08
d2σ
Gaussian (or “Normal”) p.d.f.
bell-shaped function
![Page 31: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/31.jpg)
probability between <d>±nσGaussian p.d.f.
![Page 32: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/32.jpg)
Part 2
correlated errors
![Page 33: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/33.jpg)
uncorrelated random variables
no pattern of between values of one variable and values of another
when d1 is higher than its meand2 is higher or lower than its mean with equal probability
![Page 34: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/34.jpg)
0
0.2
0.4
0.6
0.8
1
d1
d20 100
10
0
0.2
0.4
0.6
0.8
1
0.0
0.2pjoint probability density function
uncorrelated case
<d1 >
<d2 >
![Page 35: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/35.jpg)
0
0.2
0.4
0.6
0.8
1
d1
d20 100
10
0
0.2
0.4
0.6
0.8
1
0.0
0.2p
no tendency for d2 to be either high or low when d1 is high
<d1 >
<d2 >
![Page 36: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/36.jpg)
in uncorrelated case
joint p.d.f.is just the product of individual p.d.f.’s
![Page 37: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/37.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.2
0.4
0.6
0.8
1
d1
d20 100
10 0.00
0.25p
2σ1
<d1 >
<d2 >
![Page 38: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/38.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.2
0.4
0.6
0.8
1
d1
d20 100
10 0.00
0.25p
2σ1
<d1 >
<d2 >
tendency for d2 to be high when d1 is high
![Page 39: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/39.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
0.2
0.4
0.6
0.8
1
d1
d20 100
10 0.00
0.25p
2σ1
2σ12σ2
θ<d1 >
<d2 >
![Page 40: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/40.jpg)
d1
d2
2σ1
<d2><d 1>
![Page 41: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/41.jpg)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
d1
d2<d2>
<d 1>
d1
d2<d2><d 1>
d1
d2<d2>
<d 1>
(A) (B) (C)
![Page 42: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/42.jpg)
formula for covariance
+ positive correlation high d1 high d2
- negative correlation high d1 low d2
![Page 43: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/43.jpg)
joint p.d.f.mean is a vector
covariance is a symmetric matrix
diagonal elements: variancesoff-diagonal elements: covariances
![Page 44: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/44.jpg)
estimating covariance from a table D of data
Dki: realization k of data-type i in MatLab, C=cov(D)
![Page 45: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/45.jpg)
univariate p.d.f. formed from joint p.d.f.
p(d) → p(di)behavior of di irrespective of the other ds
integrate over everything but di
![Page 46: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/46.jpg)
d1
d2
d1
integrate over d2
integrate over d1
d2
p(d1,d2)
p(d2)
p(d1)
![Page 47: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/47.jpg)
multivariate Gaussian (or Normal) p.d.f.
![Page 48: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/48.jpg)
covariance matrix
mean
![Page 49: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/49.jpg)
Part 3
functions of random variables
![Page 50: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/50.jpg)
functions of random variables
data with measurement
error
data analysis process
inferences with
uncertainty
![Page 51: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/51.jpg)
functions of random variables
given p(d)with m=f(d)
what is p(m) ?
![Page 52: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/52.jpg)
univariate p.d.f.use the chair rule
![Page 53: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/53.jpg)
univariate p.d.f.use the chair rule
rule for transforming a univariate p.d.f.
![Page 54: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/54.jpg)
multivariate p.d.f.
determinant of matrix with elements ∂di/∂mj
Jacobiandeterminant
![Page 55: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/55.jpg)
multivariate p.d.f.
rule for transforming a multivariate p.d.f.
Jacobian determinant
![Page 56: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/56.jpg)
simple example
data with measurement
error
data analysis process
inferences with
uncertainty
one datum, duniform p.d.f.
0<d<1
m = 2d one model parameter, m
![Page 57: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/57.jpg)
m=2d and d= m½d=0 → m=0d=1 → m=2dd/dm = ½p(d ) =1p(m ) = ½
![Page 58: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/58.jpg)
p(d)
0 1 2 d01
p(m)
0 1 2 m01
![Page 59: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/59.jpg)
another simple example
data with measurement
error
data analysis process
inferences with
uncertainty
one datum, duniform p.d.f.
0<d<1
m = d2 one model parameter, m
![Page 60: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/60.jpg)
m=d2 and d=m½d=0 → m=0d=1 → m=1dd/dm = m½ -½p(d ) =1p(m ) = m½ -½
![Page 61: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/61.jpg)
A)
B)
0.0 0.2 0.4 0.6 0.8 1.01.02.0
0.0 0.2 0.4 0.6 0.8 1.01.02.0
p(d)p(m)
d
m
![Page 62: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/62.jpg)
multivariate example
data with measurement
error
data analysis process
inferences with
uncertainty
2 data, d1 , d2uniform p.d.f.
0<d<1for both
m1 = d1+d2m2 = d2-d2
2 model parameters, m1 and m2
![Page 63: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/63.jpg)
d2
d1
1-1
-11
m1=d1+d1
m2=d1-d1(A)
![Page 64: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/64.jpg)
p(d ) = 1
m=Md with
d=M-1m so ∂d/ ∂ m = M-1 and J= |det(M-1)|= ½and |det(M)|=2
p(m ) = ½
![Page 65: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/65.jpg)
d2
d1
1-1
-11
m1=d1+d1
m2=d1-d1
m2
m1
1-1
-112
11
d1
p(d1)
0.5 p(m1)
0-1
m1
12
-10
(A)
(B)
(C)
(D)
Note that the shape in m is different than the shape in d
![Page 66: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/66.jpg)
d2
d1
1-1
-11
m1=d1+d1
m2=d1-d1
m2
m1
1-1
-112
11
d1
p(d1)
0.5 p(m1)
0-1
m1
12
-10
(A)
(B)
(C)
(D)
The shape is a square with sides of length √2. The amplitude is ½. So the area is ½⨉√2⨉√2 = 1
![Page 67: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/67.jpg)
d2
d1
1-1
-11
m1=d1+d1
m2=d1-d1
m2
m1
1-1
-112
11
d1
p(d1)
0.5 p(m1)
0-1
m1
12
-10
(A)
(B)
(C)
(D)
![Page 68: Lecture 2 Probability and Measurement Error, Part 1](https://reader036.vdocuments.mx/reader036/viewer/2022062305/5681610d550346895dd06131/html5/thumbnails/68.jpg)
moral
p(m) can behavior quite differently from p(d)