overlapping expert information: learning about...
TRANSCRIPT
![Page 1: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/1.jpg)
Overlapping Expert Information: Learning about Dependencies in
Expert Judgment
Jason R. W. Merrick
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the
National Science Foundation.
1
![Page 2: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/2.jpg)
Extending Winkler’s Multivariate Normal
Aggregation
2
![Page 3: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/3.jpg)
P(Event | X1,…,Xn)
P(Event)
3
![Page 4: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/4.jpg)
P(Event | X1,…,Xn)
Event = Collision between two vesselsX1 = Type of other vesselX2 = Proximity of other vesselX3 = Wind speedX4 = Wind directionX5 = Current speedX6 = Current directionX7 = Visibility
4
![Page 5: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/5.jpg)
P(Event | X1,…,Xn)
Event = Incoming vessel contains RDDX1 = Last country dockedX2 = 2nd to last country dockedX3 = 3rd to last country dockedX4 = Frequency of US callsX5 = Vessel ownershipX6 = Type of vesselX7 = Type of crew
5
![Page 6: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/6.jpg)
Issaquah class ferry On the Bremerton to Seattle route
Crossing situation within 15 minutesOther vessel is a navy vessel
No other vessels aroundGood visibility
Negligible wind
What is the probability of a collision?
6
![Page 7: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/7.jpg)
Issaquah class ferry On the Bremerton to Seattle route
Crossing situation within 15 minutesOther vessel is a navy vessel
No other vessels aroundGood visibility
Negligible wind
Issaquah class ferry On the Bremerton to Seattle route
Crossing situation within 15 minutesOther vessel is a product tanker
No other vessels aroundGood visibility
Negligible wind
7
![Page 8: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/8.jpg)
Issaquah Ferry Class -SEA-BRE(A) Ferry Route -
Navy 1st Interacting Vessel Product TankerCrossing Traffic Scenario 1st Vessel -
< 1 mile Traffic Proximity 1st Vessel -No Vessel 2nd Interacting Vessel -No Vessel Traffic Scenario 2nd Vessel -No Vessel Traffic Proximity 2nd Vessel -> 0.5 Miles Visibility -Along Ferry Wind Direction -
0 Wind Speed -Likelihood of Collision -
9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 99 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 99 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9
8
![Page 9: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/9.jpg)
P(Event | X , p0 ,β ) = p0 exp X Tβ( )
P(Event | R,β )P(Event | L,β )
yi, j = ln(zi, j ) = XiTβ + ui, j
=
p0 exp(RTβ )p0 exp(LTβ )
= exp (R − L)T β( )
9
![Page 10: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/10.jpg)
9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9Likelihood of Collision
yi, j = ln(zi, j ) = XiTβ + ui, j
10
![Page 11: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/11.jpg)
u =u1up
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟~ MVNormal 0,Σ( )
θ
ui = µi −θ
11
![Page 12: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/12.jpg)
π (θ;µ,Σ)∝ exp − θ − µ*( )2 / 2σ *2( )
µ* = 1TΣ−1µ1TΣ−11
σ *2 = 11TΣ−11
12
![Page 13: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/13.jpg)
u =u1up
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟~ MVNormal 0,Σ( )
θ
ui = µi −θ
XiTβ
ui, j = yi, j − XiTβ
ui =ui,1ui,p
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟~ MVNormal 0,Σ( )
13
![Page 14: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/14.jpg)
y1,1 y1,p yN ,1 yN ,p
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟=
x1,1 x1,q xN ,1 xN ,q
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟
β1 β1 βq βq
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟+
u1,1 u1,p uN ,1 uN ,p
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟
Y = Xβ1T +U
14
![Page 15: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/15.jpg)
p Y |X,β,Σ( )∝ Σ −N
2 exp − 12 tr VΣ
−1( ){ } exp − 12 tr
B− β1T( )T XTX
B− β1T( )Σ−1( ){ }
Σ*
β =X T X( )−1
1TΣ−11 µ*
β = B̂Σ−111TΣ−11
15
![Page 16: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/16.jpg)
Σ( ) ~ Inv −Wishart G,m( )
β |Y,X,Σ( ) ~ MVNormal ϕ, A1TΣ−11
⎛⎝⎜
⎞⎠⎟
Σ |Y,X( ) ~ Inv −Wishart G +V,m + N( )
β |Y,X,Σ( ) ~ MVNormal A−1 +XTX( )−1 XTX B̂Σ−11
1TΣ−11+A−1ϕ
⎛⎝⎜
⎞⎠⎟,A−1 +XTX( )−11TΣ−11
⎛
⎝⎜⎜
⎞
⎠⎟⎟
V = Y −XB̂( )T Y −XB̂( )
16
![Page 17: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/17.jpg)
Description Notation Values
Ferry route and class FR_FC 26
Type of 1st interacting vessel TT_1 13
Scenario of 1st interacting vessel TS_1 4
Proximity of 1st interacting vessel TP_1 Binary
Type of 2nd interacting vessel TT_2 5
Scenario of 2nd interacting vessel TS_2 4
Proximity of 2nd interacting vessel TP_2 Binary
Visibility VIS Binary
Wind direction WD Binary
Wind speed WS Continuous
17
![Page 18: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/18.jpg)
Assume independence between the experts a priori
18
![Page 19: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/19.jpg)
Comparing the two
scenarios we pictured earlier a priori
19
![Page 20: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/20.jpg)
Analysis with dependenceDoesn’t dependence
between experts increase posterior variance?
Analysis with independence
20
![Page 21: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/21.jpg)
Experts 1, 3 and 7 are correlatedExperts 2, 4 and 6 are correlatedExperts 5 and 8 are negatively or uncorrelated with other experts
Remember we assumed independence a priori, but we learnt
about Σ!
21
![Page 22: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/22.jpg)
Comparing the two
scenarios we pictured earlier Text
Prior [1.88*10-35, 5.32*1034]Dependent [4.38,5.84] ½ width = 0.73Independent [4.43,7.04] ½ width = 1.3
Text
90% Credibility
Interval
22
![Page 23: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/23.jpg)
Getting the Right Mix of Experts
23
![Page 24: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/24.jpg)
z1,..., zp( )
r1,...,rp( )
24
![Page 25: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/25.jpg)
25
![Page 26: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/26.jpg)
26
![Page 27: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/27.jpg)
27
![Page 28: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/28.jpg)
28
![Page 29: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/29.jpg)
29
![Page 30: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/30.jpg)
30
![Page 31: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/31.jpg)
31
![Page 32: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/32.jpg)
32
![Page 33: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/33.jpg)
33
![Page 34: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/34.jpg)
34
![Page 35: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/35.jpg)
35
![Page 36: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/36.jpg)
36
![Page 37: Overlapping Expert Information: Learning about ...dimacs.rutgers.edu/Workshops/ExpertOpinion/Slides/Merrick.pdf · Overlapping Expert Information: Learning about Dependencies in Expert](https://reader031.vdocuments.mx/reader031/viewer/2022040422/5e1390aaf94f0a0c593e842f/html5/thumbnails/37.jpg)
37