appendix e: roc datapoints at far of 0.01% and 1%, tar 98%

24
Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98% for each SDK. The exact values at these operating points of FAR 0.01% and 1%, TAR 98% were linearly interpolated (log scale on the x-axis) from the ROC curve data values. The tables are in three groups. The first group of SDK tables are for datasets DOS-C, DHS2-C, VISIT_POE, VISIT_POE_BVA, BEN, DHS10, and TXDPS; the next group of tables is for OHIO data; followed by tables for the SD29 data. VTB DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI TAR@FAR .01% 83.20 75.68 79.45 77.63 83.66 81.18 82.95 78.10 TAR@FAR 1% 92.09 87.27 91.03 90.21 92.87 91.20 92.87 90.30 FAR@TAR 98% 33.88 50.34 35.92 42.54 28.23 39.22 23.22 40.62 DHS10 TXDPS RT LT RI LI RT LT RI LI TAR@FAR .01% 83.98 83.99 73.70 73.51 89.96 88.59 86.01 84.64 TAR@FAR 1% 91.18 90.89 84.66 84.08 95.29 94.13 93.17 92.13 FAR@TAR 98% 45.81 47.78 68.40 71.70 17.81 25.15 35.14 37.36 BEN RT LT RI LI TAR@FAR .01% 96.20 94.67 88.54 83.21 TAR@FAR 1% 98.13 97.28 94.31 91.73 FAR@TAR 98% 0.74 3.66 21.74 30.66 A DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI TAR@FAR .01% 87.63 78.09 52.39 50.05 87.80 82.18 89.44 82.28 TAR@FAR 1% 94.95 89.81 69.56 70.54 95.92 93.31 95.96 93.29 FAR@TAR 98% 14.19 51.97 NA NA 10.46 39.30 6.62 26.79 DHS10 TXDPS RT LT RI LI RT LT RI LI TAR@FAR .01% 77.19 74.80 77.49 74.07 81.64 77.88 87.48 81.16 TAR@FAR 1% 87.73 86.21 87.50 85.95 89.79 88.09 94.06 90.77 FAR@TAR 98% NA NA NA NA NA NA 48.40 NA BEN RT LT RI LI TAR@FAR .01% 96.08 92.04 90.86 84.76 TAR@FAR 1% 97.92 95.91 95.41 91.96 FAR@TAR 98% 1.31 15.64 24.28 NA E-1

Upload: others

Post on 21-Jul-2022

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98% for each SDK.

The exact values at these operating points of FAR 0.01% and 1%, TAR 98% were linearly interpolated (log scale on the x-axis) from the ROC curve data values. The tables are in three groups. The first group of SDK tables are for datasets DOS-C, DHS2-C, VISIT_POE, VISIT_POE_BVA, BEN, DHS10, and TXDPS; the next group of tables is for OHIO data; followed by tables for the SD29 data.

VTB DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 83.20 75.68 79.45 77.63 83.66 81.18 82.95 78.10 TAR@FAR 1% 92.09 87.27 91.03 90.21 92.87 91.20 92.87 90.30 FAR@TAR 98% 33.88 50.34 35.92 42.54 28.23 39.22 23.22 40.62

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 83.98 83.99 73.70 73.51 89.96 88.59 86.01 84.64 TAR@FAR 1% 91.18 90.89 84.66 84.08 95.29 94.13 93.17 92.13 FAR@TAR 98% 45.81 47.78 68.40 71.70 17.81 25.15 35.14 37.36

BEN RT LT RI LI

TAR@FAR .01% 96.20 94.67 88.54 83.21 TAR@FAR 1% 98.13 97.28 94.31 91.73 FAR@TAR 98% 0.74 3.66 21.74 30.66

A DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 87.63 78.09 52.39 50.05 87.80 82.18 89.44 82.28 TAR@FAR 1% 94.95 89.81 69.56 70.54 95.92 93.31 95.96 93.29 FAR@TAR 98% 14.19 51.97 NA NA 10.46 39.30 6.62 26.79

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 77.19 74.80 77.49 74.07 81.64 77.88 87.48 81.16 TAR@FAR 1% 87.73 86.21 87.50 85.95 89.79 88.09 94.06 90.77 FAR@TAR 98% NA NA NA NA NA NA 48.40 NA

BEN RT LT RI LI

TAR@FAR .01% 96.08 92.04 90.86 84.76 TAR@FAR 1% 97.92 95.91 95.41 91.96 FAR@TAR 98% 1.31 15.64 24.28 NA

E-1

Page 2: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

B DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 91.03 84.27 74.99 74.07 NA 84.61 88.17 83.02 TAR@FAR 1% 96.44 92.65 90.83 90.77 NA 94.11 95.75 93.07 FAR@TAR 98% 7.59 NA NA NA NA NA 9.16 NA

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 77.19 74.80 77.49 74.07 81.64 77.88 87.48 81.16 TAR@FAR 1% 87.73 86.21 87.50 85.95 89.79 88.09 94.06 90.77 FAR@TAR 98% NA NA NA NA NA NA 48.40 NA

BEN RT LT RI LI

TAR@FAR .01% 93.30 90.00 86.20 79.85 TAR@FAR 1% 95.88 93.83 92.23 87.78 FAR@TAR 98% 27.66 NA NA NA

C DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 96.39 93.13 90.86 91.03 97.39 96.16 97.63 95.87 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% NA NA NA NA 0.06 NA 0.03 NA

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 92.44 90.05 87.91 86.34 83.00 83.84 95.10 93.89 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% NA NA NA NA NA NA NA NA

BEN RT LT RI LI

TAR@FAR .01% 98.81 97.79 96.36 93.31 TAR@FAR 1% NA NA NA NA FAR@TAR 98% 0.00 0.02 0.20 NA

D DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 96.01 93.53 95.30 95.72 97.46 96.12 97.99 96.25 TAR@FAR 1% 98.52 96.91 98.25 98.30 98.65 98.23 99.47 98.52 FAR@TAR 98% 0.27 6.38 0.60 0.53 0.06 0.59 0.01 0.27

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 95.27 94.55 89.65 88.90 95.89 93.85 95.11 93.75 TAR@FAR 1% 97.22 96.96 93.97 93.35 97.61 96.57 97.40 96.46 FAR@TAR 98% 8.41 8.01 38.29 35.71 2.99 16.10 5.61 11.07

BEN RT LT RI LI

TAR@FAR .01% 98.75 97.98 96.45 94.24 TAR@FAR 1% 99.25 98.89 98.27 97.00 FAR@TAR 98% 0.00 0.01 0.53 6.26

E-2

Page 3: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

E DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 79.18 67.81 40.33 41.04 73.75 70.57 76.56 69.56 TAR@FAR 1% 86.87 78.99 48.50 52.31 82.28 81.14 84.83 79.95 FAR@TAR 98% 73.37 87.30 97.60 97.67 77.69 84.27 77.22 85.56

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 78.89 75.33 59.33 57.15 83.60 79.62 76.32 71.32 TAR@FAR 1% 83.55 80.74 68.75 65.08 87.33 84.33 81.59 77.96 FAR@TAR 98% 87.57 93.79 94.24 95.63 80.71 91.89 86.54 92.83

BEN RT LT RI LI

TAR@FAR .01% 91.48 87.63 72.72 63.36 TAR@FAR 1% 93.37 90.82 78.79 70.58 FAR@TAR 98% 70.45 89.02 84.95 93.17

F DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 97.62 95.32 97.06 97.45 98.30 97.53 98.89 97.66 TAR@FAR 1% 99.27 98.03 98.87 99.10 99.12 98.86 99.62 99.04 FAR@TAR 98% 0.03 0.95 0.08 0.04 0.00 0.04 0.00 0.04

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 96.44 96.20 92.30 90.86 97.07 95.53 96.43 95.07 TAR@FAR 1% 98.51 98.26 95.25 95.15 98.49 97.74 98.09 97.37 FAR@TAR 98% 0.42 0.53 22.81 16.71 0.27 1.73 0.77 2.92

BEN RT LT RI LI

TAR@FAR .01% 99.23 98.90 97.93 96.07 TAR@FAR 1% 99.65 99.48 99.16 98.04 FAR@TAR 98% 0.00 0.00 0.01 0.94

G DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 96.91 93.91 96.56 96.16 96.91 96.27 98.18 96.31 TAR@FAR 1% 98.59 96.93 98.34 98.63 98.47 98.02 99.17 98.29 FAR@TAR 98% 0.15 8.89 0.17 0.25 0.16 0.95 0.01 0.30

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 96.09 95.55 90.67 89.91 98.11 97.28 96.70 95.85 TAR@FAR 1% 97.57 97.27 94.05 93.50 98.82 98.43 98.04 97.42 FAR@TAR 98% 3.52 4.53 47.35 43.32 0.01 0.12 0.85 8.53

BEN RT LT RI LI

TAR@FAR .01% 98.93 98.86 97.36 95.28 TAR@FAR 1% 99.36 99.26 98.44 97.21 FAR@TAR 98% 0.00 0.00 0.12 4.11

E-3

Page 4: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

H DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 98.89 97.41 97.37 97.31 98.92 98.69 99.40 98.70 TAR@FAR 1% NA NA 98.84 98.97 NA 99.35 NA NA FAR@TAR 98% 0.00 0.05 0.04 0.06 0.00 0.00 0.00 0.00

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 97.73 97.75 95.60 95.80 99.11 98.82 98.40 98.02 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% 0.04 0.02 NA 0.72 0.00 0.00 0.00 0.01

BEN RT LT RI LI

TAR@FAR .01% 99.65 99.58 99.07 98.19 TAR@FAR 1% NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.01

I DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 98.91 97.17 98.17 98.49 98.68 98.55 99.50 98.62 TAR@FAR 1% 99.70 98.74 99.24 99.40 99.17 99.21 99.70 99.38 FAR@TAR 98% 0.00 0.11 0.01 0.00 0.00 0.00 0.00 0.00

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 97.78 97.89 94.63 94.29 98.90 98.46 98.14 97.36 TAR@FAR 1% 98.97 98.92 97.18 97.53 99.40 99.24 99.11 98.83 FAR@TAR 98% 0.02 0.02 4.24 1.90 0.00 0.00 0.00 0.09

BEN RT LT RI LI

TAR@FAR .01% 99.52 99.34 98.62 97.29 TAR@FAR 1% 99.80 99.69 99.55 98.98 FAR@TAR 98% 0.00 0.00 0.00 0.06

J DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 97.05 94.40 96.56 96.16 97.48 96.81 98.22 97.12 TAR@FAR 1% 98.45 97.28 98.34 98.63 98.67 98.65 99.21 98.74 FAR@TAR 98% 0.15 4.56 0.17 0.25 0.05 0.12 0.00 0.07

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 96.67 96.52 92.93 91.95 98.56 98.04 97.75 96.95 TAR@FAR 1% 98.00 97.98 95.32 95.02 98.95 98.65 98.59 98.06 FAR@TAR 98% 1.00 1.11 39.12 40.41 0.00 0.01 0.05 0.72

BEN RT LT RI LI

TAR@FAR .01% 99.36 99.16 97.84 96.32 TAR@FAR 1% 99.59 99.51 98.88 98.03 FAR@TAR 98% 0.00 0.00 0.02 0.89

E-4

Page 5: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

K DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 94.24 89.86 88.67 88.17 94.92 92.87 95.64 92.63 TAR@FAR 1% 97.58 95.16 94.95 95.31 97.76 96.87 98.38 97.16 FAR@TAR 98% 2.16 22.68 24.07 20.12 1.86 5.20 0.38 3.80

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 89.59 90.51 87.36 86.25 94.92 93.44 94.64 93.68 TAR@FAR 1% 95.19 95.00 92.92 92.51 97.44 96.68 97.36 96.54 FAR@TAR 98% 25.21 29.34 49.98 50.98 5.40 14.91 4.71 12.02

BEN RT LT RI LI

TAR@FAR .01% 98.18 97.00 94.79 91.15 TAR@FAR 1% 98.94 98.23 97.37 95.15 FAR@TAR 98% 0.00 0.40 5.42 30.36

L DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 87.80 81.43 80.98 80.13 89.88 87.58 91.22 87.26 TAR@FAR 1% 95.59 92.64 94.16 93.69 95.94 95.09 97.07 95.37 FAR@TAR 98% 7.21 24.60 11.60 13.55 7.46 12.99 2.27 6.40

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 91.50 90.53 79.88 79.18 94.98 93.45 89.40 87.36 TAR@FAR 1% 95.63 94.77 89.35 88.44 97.72 96.76 95.14 94.19 FAR@TAR 98% 18.82 23.07 51.65 55.59 1.68 8.19 16.12 18.20

BEN RT LT RI LI

TAR@FAR .01% 97.81 96.78 91.82 88.54 TAR@FAR 1% 98.82 98.11 95.70 93.18 FAR@TAR 98% 0.02 0.72 12.54 33.35

M DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 71.06 51.55 23.77 19.84 78.68 69.78 59.60 45.00 TAR@FAR 1% 80.37 64.74 28.30 27.21 86.42 80.24 69.94 58.36 FAR@TAR 98% NA NA NA NA NA NA NA NA

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 67.22 64.89 33.79 33.65 66.61 67.58 46.63 46.50 TAR@FAR 1% 76.13 75.28 40.70 40.97 76.81 78.11 50.54 52.26 FAR@TAR 98% NA NA NA NA NA NA NA NA

BEN RT LT RI LI

TAR@FAR .01% 98.19 97.39 96.32 94.13 TAR@FAR 1% 99.12 98.72 98.22 96.97 FAR@TAR 98% 0.00 0.07 0.49 7.83

E-5

Page 6: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

N DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 94.01 88.96 92.71 90.78 95.46 94.37 96.18 94.34 TAR@FAR 1% 97.79 95.35 97.63 97.09 98.31 97.85 98.78 98.15 FAR@TAR 98% 1.37 12.28 1.84 3.03 0.42 1.39 0.16 0.80

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 91.70 90.96 89.32 87.97 95.65 94.38 95.92 94.67 TAR@FAR 1% 95.94 95.65 94.31 93.53 98.07 97.34 97.83 97.33 FAR@TAR 98% 22.00 17.12 45.39 43.09 0.86 2.95 1.67 3.84

BEN RT LT RI LI

TAR@FAR .01% 98.19 97.39 96.32 94.13 TAR@FAR 1% 99.12 98.72 98.22 96.97 FAR@TAR 98% 0.00 0.07 0.49 7.83

O DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 98.80 97.02 97.11 96.80 98.72 98.30 99.21 98.42 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% 0.00 NA 0.12 0.13 0.00 0.00 0.00 0.00

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 97.12 96.67 94.84 94.99 98.65 98.17 98.16 97.56 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% NA NA NA NA 0.00 0.00 0.00 0.06

BEN RT LT RI LI

TAR@FAR .01% 99.40 99.03 98.70 97.46 TAR@FAR 1% NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.05

P DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 98.04 95.79 95.60 95.37 98.21 97.74 98.69 97.66 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% 0.01 NA NA NA 0.00 0.05 0.00 NA

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 95.23 94.49 92.25 91.75 97.75 96.75 97.10 95.85 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% NA NA NA NA NA NA NA NA

BEN RT LT RI LI

TAR@FAR .01% 99.16 98.55 98.17 96.27 TAR@FAR 1% NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 NA

E-6

Page 7: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

Q DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 98.61 96.69 96.61 97.17 97.76 97.22 98.68 97.94 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% 0.00 NA NA 0.09 0.03 0.06 0.00 0.02

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 96.61 96.57 91.97 91.86 97.26 98.32 97.26 96.32 TAR@FAR 1% NA NA NA NA NA NA NA NA FAR@TAR 98% NA NA NA NA 0.38 0.00 0.38 NA

BEN RT LT RI LI

TAR@FAR .01% 99.21 99.04 97.78 96.29 TAR@FAR 1% NA NA NA NA FAR@TAR 98% 0.00 0.00 0.02 NA

R DOS-C DHS2-C VISIT_POE VISIT_BVA RI LI RI LI RI LI RI LI

TAR@FAR .01% 99.13 97.64 97.68 97.97 98.94 98.78 99.63 98.79 TAR@FAR 1% 99.77 98.92 98.47 98.78 99.29 99.29 99.75 99.40 FAR@TAR 98% 0.00 0.02 0.03 0.01 0.00 0.00 0.00 0.00

DHS10 TXDPS RT LT RI LI RT LT RI LI

TAR@FAR .01% 98.27 97.98 95.68 95.55 99.21 99.03 98.67 98.25 TAR@FAR 1% 99.16 98.95 97.44 97.85 99.42 99.31 99.12 98.93 FAR@TAR 98% 0.00 0.01 5.85 1.48 0.00 0.00 0.00 0.00

BEN RT LT RI LI

TAR@FAR .01% 99.54 99.39 98.97 97.78 TAR@FAR 1% 99.79 99.72 99.57 99.00 FAR@TAR 98% 0.00 0.00 0.00 0.02

E-7

Page 8: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

VTB OHIO P2P RT RI RM RR RL

TAR@FAR .01% 95.99 94.81 91.16 89.14 80.77 TAR@FAR 1% 98.34 98.38 96.38 94.75 90.78 FAR@TAR 98% 0.59 0.55 6.33 16.58 37.68

LT LI LM LR LL TAR@FAR .01% 95.02 95.17 90.49 88.27 75.90 TAR@FAR 1% 98.37 98.04 95.91 94.19 87.90 FAR@TAR 98% 0.61 0.86 10.50 20.61 46.11

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 93.44 90.21 87.76 82.52 73.25 TAR@FAR 1% 97.25 96.54 95.18 92.12 87.29 FAR@TAR 98% 3.28 7.00 13.26 40.92 59.31

LT LI LM LR LL TAR@FAR .01% 91.90 90.28 86.88 77.41 66.76 TAR@FAR 1% 96.66 95.76 94.77 88.98 84.89 FAR@TAR 98% 7.55 11.54 14.12 53.01 63.89

A OHIO P2P RT RI RM RR RL

TAR@FAR .01% 97.73 97.57 94.03 88.27 81.11 TAR@FAR 1% 98.97 99.27 97.72 94.52 93.14 FAR@TAR 98% 0.04 0.02 1.88 19.04 44.23

LT LI LM LR LL TAR@FAR .01% 94.00 95.99 92.54 87.38 76.04 TAR@FAR 1% 97.55 99.09 96.21 93.05 90.27 FAR@TAR 98% 1.50 0.06 12.70 33.51 66.05

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 93.02 95.98 93.48 86.01 80.01 TAR@FAR 1% 97.21 98.34 96.98 93.64 92.21 FAR@TAR 98% 4.74 0.39 4.86 22.09 43.45

LT LI LM LR LL TAR@FAR .01% 90.70 95.46 91.41 83.51 76.25 TAR@FAR 1% 96.03 98.06 96.70 92.58 90.51 FAR@TAR 98% 9.50 0.93 8.25 54.98 52.92

E-8

Page 9: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

B OHIO P2P RT RI RM RR RL

TAR@FAR .01% 94.68 91.12 90.12 82.19 TAR@FAR 1% 97.32 96.06 95.77 91.47 FAR@TAR 98% 2.84 13.52 NA NA

LT LI LM LR LL TAR@FAR .01% 91.62 93.48 88.22 86.56 78.05 TAR@FAR 1% 96.40 97.87 94.48 94.16 89.84 FAR@TAR 98% 9.45 1.36 NA NA NA

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 88.82 87.40 83.19 77.24 TAR@FAR 1% 94.64 93.29 91.52 87.48 FAR@TAR 98% NA NA NA NA

LT LI LM LR LL TAR@FAR .01% 84.99 88.91 84.09 77.91 73.72 TAR@FAR 1% 92.63 94.76 92.05 88.29 85.49 FAR@TAR 98% NA NA NA NA NA

C OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.43 99.09 98.28 97.35 94.49 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.08 NA

LT LI LM LR LL TAR@FAR .01% 99.14 99.14 97.88 96.79 90.95 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.01 0.10 NA

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 98.29 99.07 98.42 97.04 94.90 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.08 NA

LT LI LM LR LL TAR@FAR .01% 98.71 98.63 98.33 95.85 93.92 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 NA NA

E-9

Page 10: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

D OHIO P2P RT RI RM RR RL

TAR@FAR .01% 98.29 99.07 98.42 97.04 94.90 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.08 NA

LT LI LM LR LL TAR@FAR .01% 98.71 98.63 98.33 95.85 93.92 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 NA NA

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.39 99.12 98.48 97.80 95.55 TAR@FAR 1% 99.83 99.69 99.34 99.08 97.76 FAR@TAR 98% 0.00 0.00 0.00 0.02 1.79

LT LI LM LR LL TAR@FAR .01% 99.19 98.74 98.40 96.70 94.39 TAR@FAR 1% 99.73 99.42 99.13 98.61 97.27 FAR@TAR 98% 0.00 0.00 0.00 0.21 3.57

E OHIO P2P RT RI RM RR RL

TAR@FAR .01% 93.82 87.53 86.08 82.76 68.78 TAR@FAR 1% 96.15 93.41 90.84 88.99 80.63 FAR@TAR 98% 30.00 50.56 64.55 66.09 82.05

LT LI LM LR LL TAR@FAR .01% 91.05 87.94 82.62 81.47 64.32 TAR@FAR 1% 94.23 92.38 88.08 88.55 74.83 FAR@TAR 98% 57.31 67.27 84.71 72.60 84.63

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 91.39 85.88 86.12 81.27 70.90 TAR@FAR 1% 94.58 90.19 90.69 87.97 80.52 FAR@TAR 98% 54.65 69.44 67.99 73.08 81.90

LT LI LM LR LL TAR@FAR .01% 88.94 85.37 84.38 78.89 68.72 TAR@FAR 1% 92.62 90.17 88.94 86.48 78.13 FAR@TAR 98% 64.06 79.33 81.74 80.53 87.07

E-10

Page 11: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

F OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.83 99.66 99.54 98.80 98.31 TAR@FAR 1% 99.89 99.83 99.83 99.44 99.59 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.89 99.37 99.09 98.58 96.59 TAR@FAR 1% 99.89 99.71 99.71 99.43 98.40 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.46

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.70 99.37 99.18 98.36 97.05 TAR@FAR 1% 99.89 99.79 99.62 99.35 98.70 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.25

LT LI LM LR LL TAR@FAR .01% 99.64 99.12 98.84 97.59 96.03 TAR@FAR 1% 99.81 99.64 99.60 99.08 98.41 FAR@TAR 98% 0.00 0.00 0.00 0.05 0.45

G OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.57 99.49 98.55 97.27 94.69 TAR@FAR 1% 99.77 99.71 99.31 98.49 98.14 FAR@TAR 98% 0.00 0.00 0.00 0.07 0.87

LT LI LM LR LL TAR@FAR .01% 99.66 99.08 97.94 96.77 92.61 TAR@FAR 1% 99.83 99.54 98.94 98.37 96.19 FAR@TAR 98% 0.00 0.00 0.01 0.20 21.79

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.30 98.99 98.56 96.99 95.15 TAR@FAR 1% 99.58 99.55 99.00 98.54 97.37 FAR@TAR 98% 0.00 0.00 0.00 0.16 4.90

LT LI LM LR LL TAR@FAR .01% 99.39 98.81 97.97 95.82 93.61 TAR@FAR 1% 99.69 99.28 98.90 97.76 97.15 FAR@TAR 98% 0.00 0.00 0.01 1.59 5.16

E-11

Page 12: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

H OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.89 99.83 99.77 99.03 99.09 TAR@FAR 1% NA NA NA NA 99.54 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.89 99.71 99.38 99.07 98.68 TAR@FAR 1% NA NA NA NA 99.09 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.73 99.81 99.70 99.16 98.79 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.70 99.66 99.62 98.75 98.71 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

I OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.89 99.71 99.67 99.17 98.91 TAR@FAR 1% 99.89 99.83 99.83 99.71 99.53 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.89 99.66 99.37 99.03 97.71 TAR@FAR 1% 99.89 99.77 99.77 99.45 99.20 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.02

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.85 99.77 99.58 99.08 98.38 TAR@FAR 1% 99.92 99.86 99.73 99.59 99.33 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.81 99.54 99.48 98.79 98.10 TAR@FAR 1% 99.81 99.70 99.71 99.47 99.14 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.01

E-12

Page 13: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

J OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.59 99.31 98.96 97.81 95.25 TAR@FAR 1% 99.77 99.58 99.43 98.74 98.29 FAR@TAR 98% 0.00 0.00 0.00 0.02 0.52

LT LI LM LR LL TAR@FAR .01% 99.60 98.88 98.28 97.34 94.25 TAR@FAR 1% 99.89 99.49 98.95 98.45 97.23 FAR@TAR 98% 0.00 0.00 0.00 0.17 5.76

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.22 98.79 98.54 97.53 96.15 TAR@FAR 1% 99.53 99.39 99.10 98.48 97.78 FAR@TAR 98% 0.00 0.00 0.00 0.14 1.78

LT LI LM LR LL TAR@FAR .01% 99.30 99.00 98.23 96.58 94.42 TAR@FAR 1% 99.70 99.35 98.91 98.18 97.60 FAR@TAR 98% 0.00 0.00 0.00 0.62 2.95

K OHIO P2P RT RI RM RR RL

TAR@FAR .01% 98.25 98.59 96.41 94.87 93.21 TAR@FAR 1% 99.37 99.50 98.63 97.87 97.29 FAR@TAR 98% 0.01 0.00 0.08 1.24 2.86

LT LI LM LR LL TAR@FAR .01% 98.47 98.25 95.96 93.64 90.72 TAR@FAR 1% 99.43 99.31 98.04 97.30 95.70 FAR@TAR 98% 0.00 0.01 0.89 3.64 19.96

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 97.88 98.09 96.10 94.19 90.07 TAR@FAR 1% 98.85 99.46 98.50 97.35 95.93 FAR@TAR 98% 0.01 0.01 0.21 3.69 13.52

LT LI LM LR LL TAR@FAR .01% 97.42 97.88 96.21 92.00 89.25 TAR@FAR 1% 98.82 98.89 98.63 96.07 95.52 FAR@TAR 98% 0.07 0.01 0.23 13.50 22.24

E-13

Page 14: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

L OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.19 97.47 96.21 94.44 85.12 TAR@FAR 1% 99.49 99.37 98.40 97.59 93.47 FAR@TAR 98% 0.00 0.05 0.22 2.03 36.33

LT LI LM LR LL TAR@FAR .01% 99.04 96.55 95.09 92.96 79.81 TAR@FAR 1% 99.73 98.77 97.34 97.17 91.08 FAR@TAR 98% 0.00 0.09 4.52 3.05 40.74

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 98.82 97.46 97.84 95.28 87.97 TAR@FAR 1% 99.28 98.91 98.84 98.09 94.33 FAR@TAR 98% 0.00 0.03 0.02 0.91 13.98

LT LI LM LR LL TAR@FAR .01% 98.53 96.81 96.19 92.19 84.42 TAR@FAR 1% 99.25 98.75 98.48 96.88 93.50 FAR@TAR 98% 0.00 0.15 0.23 4.81 21.10

M OHIO P2P RT RI RM RR RL

TAR@FAR .01% 92.78 37.23 42.13 35.73 21.38 TAR@FAR 1% 96.73 41.86 46.37 43.00 27.74 FAR@TAR 98% 9.33 NA NA NA NA

LT LI LM LR LL TAR@FAR .01% 89.23 42.89 36.76 34.37 12.48 TAR@FAR 1% 94.24 49.67 44.45 41.83 18.59 FAR@TAR 98% NA NA NA NA NA

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 88.45 63.61 64.84 50.89 30.07 TAR@FAR 1% 93.13 74.69 79.86 67.64 43.64 FAR@TAR 98% NA NA NA NA NA

LT LI LM LR LL TAR@FAR .01% 83.54 59.55 53.78 49.18 28.93 TAR@FAR 1% 91.91 75.23 72.17 67.62 44.24 FAR@TAR 98% NA NA NA NA NA

E-14

Page 15: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

N OHIO P2P RT RI RM RR RL

TAR@FAR .01% 97.99 97.96 96.68 94.95 93.13 TAR@FAR 1% 99.40 99.30 98.57 97.56 97.35 FAR@TAR 98% 0.01 0.01 0.14 2.84 2.82

LT LI LM LR LL TAR@FAR .01% 97.89 98.26 95.79 93.51 89.38 TAR@FAR 1% 99.66 99.20 97.88 97.25 96.04 FAR@TAR 98% 0.01 0.00 1.10 3.89 19.70

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 97.42 98.56 97.32 95.09 92.43 TAR@FAR 1% 99.04 99.43 98.94 97.93 97.22 FAR@TAR 98% 0.04 0.00 0.04 1.40 3.31

LT LI LM LR LL TAR@FAR .01% 97.37 97.98 96.84 93.07 91.32 TAR@FAR 1% 98.90 99.00 98.83 96.87 96.31 FAR@TAR 98% 0.06 0.01 0.12 4.12 9.95

O OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.77 99.77 99.65 98.63 98.87 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.89 99.66 99.14 98.74 98.40 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.49 99.61 99.43 98.74 98.57 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.14 99.42 99.14 97.97 98.29 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.01 0.01

E-15

Page 16: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

P OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.49 99.60 99.49 98.23 98.51 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.60 99.49 98.86 98.16 97.19 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 NA

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 98.86 99.37 99.26 97.89 97.92 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 NA 0.02

LT LI LM LR LL TAR@FAR .01% 98.69 99.26 99.14 96.87 97.39 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 NA NA

Q OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.71 99.71 99.77 99.30 98.23 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.77 99.60 99.31 98.97 97.88 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.05

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.71 99.57 99.54 99.14 98.12 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.01

LT LI LM LR LL TAR@FAR .01% 99.54 99.26 99.49 98.86 98.45 TAR@FAR 1% NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

E-16

Page 17: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

R OHIO P2P RT RI RM RR RL

TAR@FAR .01% 99.89 99.83 99.72 99.23 99.06 TAR@FAR 1% 99.89 99.83 99.83 99.43 99.71 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.89 99.71 99.53 99.13 98.33 TAR@FAR 1% 99.89 99.83 99.77 99.60 99.19 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

OHIO P2R RT RI RM RR RL

TAR@FAR .01% 99.89 99.71 99.71 99.18 98.44 TAR@FAR 1% 99.89 99.78 99.77 99.62 99.26 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

LT LI LM LR LL TAR@FAR .01% 99.77 99.60 99.54 98.88 98.86 TAR@FAR 1% 99.77 99.67 99.60 99.29 99.38 FAR@TAR 98% 0.00 0.00 0.00 0.00 0.00

E-17

Page 18: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

VTB SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 76.67 78.97 75.04 80.08 79.49 77.61 TAR@FAR 1% 95.10 91.98 89.22 90.25 88.21 91.43 FAR@TAR 98% 12.95 39.61 37.13 71.04 30.01 28.11

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 89.21 87.73 78.83 78.91 83.84 81.48 TAR@FAR 1% 94.46 93.41 90.24 88.84 93.88 92.20 FAR@TAR 98% 15.59 25.59 48.49 60.82 16.52 33.44

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 94.72 91.27 89.45 88.20 95.58 88.48 TAR@FAR 1% 96.77 94.77 95.85 96.85 96.77 94.98 FAR@TAR 98% 5.79 10.15 19.85 3.42 3.63 23.67

A SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 92.22 90.28 79.76 80.28 85.00 81.15 TAR@FAR 1% 95.56 95.58 92.97 93.52 91.65 95.00 FAR@TAR 98% 29.18 24.79 22.00 14.29 31.82 9.43

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 87.10 85.28 86.94 82.50 90.69 85.64 TAR@FAR 1% 95.21 94.78 95.27 93.62 95.48 95.26 FAR@TAR 98% 31.31 29.08 27.25 28.12 11.02 14.93

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 88.89 85.32 93.95 90.84 93.89 90.56 TAR@FAR 1% 97.65 94.44 96.67 95.13 97.78 97.78 FAR@TAR 98% 2.94 12.74 6.19 30.32 1.85 2.81

B SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 86.11 83.33 78.33 72.92 77.36 76.25 TAR@FAR 1% 92.78 91.11 91.11 87.95 89.44 87.43 FAR@TAR 98% NA 39.93 NA NA NA NA

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 76.53 76.20 77.42 74.03 80.83 75.14 TAR@FAR 1% 84.10 87.31 86.24 84.72 88.75 87.92 FAR@TAR 98% NA NA NA NA NA NA

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 77.22 72.22 82.22 80.70 83.97 82.78 TAR@FAR 1% 86.67 86.67 90.00 88.42 90.46 88.96 FAR@TAR 98% NA NA NA NA NA NA

E-18

Page 19: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

C SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 98.33 95.00 93.33 94.44 93.89 92.22 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 NA NA NA 0.44 NA SD29 P2R

RT LT RI LI RM LM TAR@FAR .01% 96.39 94.58 92.78 93.06 95.97 94.03 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% NA NA NA NA NA NA SD29 R2R

RT LT RI LI RM LM TAR@FAR .01% 97.78 93.33 97.22 95.56 98.33 98.33 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.05 NA NA NA 0.01 0.01

D SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 95.84 95.60 94.03 92.99 90.84 92.82 TAR@FAR 1% 98.33 98.89 97.64 97.39 97.78 98.33 FAR@TAR 98% 0.59 0.32 12.46 1.33 1.36 0.79

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 98.46 96.68 94.29 93.32 95.89 95.42 TAR@FAR 1% 99.44 98.26 98.04 96.98 98.66 98.60 FAR@TAR 98% 0.01 0.40 0.94 5.14 0.09 0.08

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 98.89 98.33 97.78 98.89 97.22 TAR@FAR 1% 100.00 100.00 99.44 99.01 99.99 99.61

FAR@TAR 98% 0.01 0.01 0.01 0.04 0.01 0.10

E SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 81.32 77.78 61.67 64.17 61.81 61.11 TAR@FAR 1% 86.92 84.24 70.32 74.28 69.19 69.08 FAR@TAR 98% 84.09 93.23 86.84 90.64 91.05 94.90

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 82.50 79.60 66.03 66.87 71.77 68.44 TAR@FAR 1% 87.87 86.10 74.24 74.74 76.54 76.68 FAR@TAR 98% 80.58 90.43 88.60 91.90 88.57 93.63

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 86.67 81.67 83.33 79.03 85.00 82.78 TAR@FAR 1% 90.00 84.93 84.44 82.78 88.13 87.25 FAR@TAR 98% 76.97 89.94 90.69 92.49 83.52 91.45

E-19

Page 20: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

F SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 96.39 97.78 94.72 94.44 93.89 95.42 TAR@FAR 1% 99.44 99.44 97.98 97.22 98.33 98.33 FAR@TAR 98% 0.04 0.04 1.02 1.59 0.41 0.21

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 98.75 97.52 96.26 95.70 97.81 97.53 TAR@FAR 1% 99.66 99.00 98.82 98.24 99.49 99.30 FAR@TAR 98% 0.01 0.07 0.14 0.84 0.04 0.03

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 98.89 99.44 97.78 99.44 98.41 TAR@FAR 1% 99.44 100.00 99.44 99.46 100.00 100.00 FAR@TAR 98% 0.01 0.01 0.01 0.03 0.01 0.01

G SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 98.33 97.78 92.30 91.67 94.03 96.11 TAR@FAR 1% 99.44 99.44 97.22 98.89 97.47 98.33 FAR@TAR 98% 0.01 0.01 3.08 0.38 1.21 0.27

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 98.98 97.85 96.25 94.44 97.50 96.85 TAR@FAR 1% 99.17 98.72 98.33 96.94 98.70 98.77 FAR@TAR 98% 0.01 0.01 0.17 10.37 0.05 0.16

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 99.44 99.73 99.44 99.49 99.44 99.44 TAR@FAR 1% 99.44 100.00 100.00 100.00 99.44 99.44 FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

H SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 99.44 99.44 98.33 98.33 98.33 98.33 TAR@FAR 1% NA 99.44 NA NA 100.00 100.00 FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 99.86 98.75 99.31 98.61 99.58 99.44 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01 SD29 R2R

RT LT RI LI RM LM TAR@FAR .01% 100.00 100.00 100.00 100.00 100.00 100.00 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

E-20

Page 21: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

I SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 98.89 99.44 96.25 97.27 97.22 98.33 TAR@FAR 1% 100.00 100.00 98.89 99.94 98.70 98.89

FAR@TAR 98% 0.01 0.01 0.20 0.06 0.02 0.01 SD29 P2R

RT LT RI LI RM LM TAR@FAR .01% 99.72 98.65 98.73 98.28 98.89 99.03 TAR@FAR 1% 100.00 99.44 99.69 99.28 99.72 99.58

FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01 SD29 R2R

RT LT RI LI RM LM TAR@FAR .01% 100.00 99.44 100.00 100.00 100.00 100.00 TAR@FAR 1% 100.00 100.00 100.00 100.00 100.00 100.00 FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

J SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 99.44 98.33 97.78 94.03 95.84 97.22 TAR@FAR 1% 99.44 99.44 98.89 97.22 97.22 98.89 FAR@TAR 98% 0.01 0.01 0.10 12.97 1.82 0.38

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 98.19 97.29 95.75 97.92 98.50 TAR@FAR 1% 99.17 99.14 99.10 98.18 98.61 99.32 FAR@TAR 98% 0.01 0.01 0.05 0.40 0.01 0.01

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 99.44 99.44 98.89 100.00 98.89 98.89 TAR@FAR 1% 99.44 100.00 98.89 100.00 99.44 99.44 FAR@TAR 98% 0.01 0.01 0.01 0.00 0.01 0.01

K SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 90.56 89.37 90.56 94.03 95.06 92.78 TAR@FAR 1% 96.40 97.78 97.28 98.33 99.44 98.33 FAR@TAR 98% 4.68 3.34 1.43 0.17 0.08 0.10

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 95.42 94.28 93.04 94.62 96.03 94.58 TAR@FAR 1% 98.74 96.97 98.49 98.48 98.83 98.89 FAR@TAR 98% 0.26 11.49 0.55 0.47 0.17 0.12

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 100.00 96.67 95.56 96.82 98.33 97.78 TAR@FAR 1% 100.00 98.89 98.89 99.44 98.89 100.0 FAR@TAR 98% 0.01 0.06 0.06 0.03 0.01 0.05

E-21

Page 22: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

L SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 92.22 92.36 85.70 84.58 82.78 85.87 TAR@FAR 1% 97.78 97.78 93.54 93.33 93.89 93.66 FAR@TAR 98% 1.28 2.11 4.61 53.66 79.45 31.17

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 94.17 93.19 86.54 83.09 88.28 88.99 TAR@FAR 1% 98.41 97.22 95.39 93.04 96.74 96.15 FAR@TAR 98% 0.67 3.17 11.23 16.38 5.28 2.33

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 97.22 95.08 95.56 99.44 98.33 TAR@FAR 1% 100.00 98.94 98.89 98.89 100.00 99.64

FAR@TAR 98% 0.01 0.07 0.17 0.31 0.01 0.01

M SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 89.44 89.44 24.44 28.47 33.89 23.89 TAR@FAR 1% 92.16 92.22 29.44 33.89 37.40 27.78 FAR@TAR 98% NA NA NA NA NA NA

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 83.69 82.76 46.26 42.69 49.03 43.66 TAR@FAR 1% 89.03 90.15 52.85 54.53 56.44 49.42 FAR@TAR 98% NA NA NA NA NA NA

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 86.90 81.81 87.22 79.86 90.56 85.00 TAR@FAR 1% 94.44 86.52 92.78 90.50 93.98 89.44 FAR@TAR 98% 48.52 NA NA NA 39.14 NA

N SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 95.00 97.22 93.41 92.22 93.33 89.83 TAR@FAR 1% 98.89 99.44 97.22 97.22 96.82 97.78 FAR@TAR 98% 0.64 0.06 3.87 2.01 3.92 1.37

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 96.60 95.28 94.63 93.75 95.83 94.17 TAR@FAR 1% 99.03 97.56 97.78 97.78 98.38 98.08 FAR@TAR 98% 0.04 3.15 1.59 1.97 0.36 0.92

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 96.11 95.56 95.56 97.61 95.56 TAR@FAR 1% 98.89 99.09 99.44 99.44 99.44 99.44 FAR@TAR 98% 0.01 0.37 0.09 0.03 0.02 0.07

E-22

Page 23: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

O SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 98.89 98.33 99.44 97.22 97.22 98.37 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 0.01 0.08 0.02 0.01 SD29 P2R

RT LT RI LI RM LM TAR@FAR .01% 99.31 98.61 99.03 98.61 98.75 98.96 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01 SD29 R2R

RT LT RI LI RM LM TAR@FAR .01% 100.00 98.89 100.00 100.00 100.00 100.00 TAR@FAR 1% NA NA NA NA NA NA FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

P SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 98.33 98.33 96.67 97.22 97.78 96.67 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 NA NA NA NA SD29 P2R

RT LT RI LI RM LM TAR@FAR .01% 98.19 97.78 97.87 97.36 97.36 96.67 TAR@FAR 1% NA NA NA NA NA NA FAR@TAR 98% 0.01 0.05 NA NA NA NA

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 98.89 97.78 98.89 98.89 98.89 100.00 TAR@FAR 1% NA NA NA NA NA NA FAR@TAR 98% 0.01 NA 0.01 0.01 0.01 0.01

Q SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 100.00 100.00 93.97 96.25 96.11 96.81 TAR@FAR 1% NA NA NA NA 99.25 NA FAR@TAR 98% 0.01 0.01 NA 0.04 0.21 0.02

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 99.44 98.61 97.78 96.67 98.33 99.17 TAR@FAR 1% NA NA NA NA NA NA

FAR@TAR 98% 0.01 0.01 0.02 0.02 0.01 0.01 SD29 R2R

RT LT RI LI RM LM TAR@FAR .01% 100.00 100.00 99.44 99.44 100.00 99.44 TAR@FAR 1% NA NA NA NA NA NA FAR@TAR 98% 0.00 0.00 0.01 0.01 0.00 0.01

E-23

Page 24: Appendix E: ROC datapoints at FAR of 0.01% and 1%, TAR 98%

R SD29 P2P RT LT RI LI RM LM

TAR@FAR .01% 100.00 99.44 98.33 97.22 98.89 99.44 TAR@FAR 1% 100.00 99.44 99.44 100.00 99.44 99.84 FAR@TAR 98% 0.01 0.01 0.01 0.02 0.01 0.01

SD29 P2R RT LT RI LI RM LM

TAR@FAR .01% 100.00 99.17 99.35 97.92 99.31 99.31 TAR@FAR 1% 100.00 99.66 100.00 99.66 99.72 99.72 FAR@TAR 98% 0.01 0.01 0.01 0.01 0.00 0.01

SD29 R2R RT LT RI LI RM LM

TAR@FAR .01% 100.00 100.00 100.00 99.44 100.00 100.00 TAR@FAR 1% 100.00 100.00 100.00 100.00 100.00 100.00 FAR@TAR 98% 0.01 0.01 0.01 0.01 0.01 0.01

E-24