static and dynamic sign sheeting testing using in-vehicle observers
TRANSCRIPT
Microsoft PowerPoint - Session 8B - Static and Dynamic Sign
Sheeting Testing Using In-Vehicle Observers.ppt [Compatibility
ModeJuly 17th, 2013
Static and Dynamic Sign Sheeting Testing Using InVehicle Observers
Edward J. Smaglik, Associate Professor Craig Roberts, Associate Professor
Northern Arizona University AZTrans:
• Limited resources available to DOT for control signage. • Focus on best combination of safety and price. • Currently using a “superior” product, but from sole source.
• Two manufactures offering recent products as “superior”, but are essentially untested.
• If claims true, then competition should result in lower cost to ADOT to provide signs with “superior” sheeting.
Motivation for Research
Research Question
One(1) new sign sheeting material is offered by its manufacturer as “equivalent” to the current superior product being used by ADOT.
Can this equivalency be supported by evidence from a double blind observational experiment under actual roadway conditions?
(1)Originally two new sheetings were to be tested but at the last minute, one of the manufactures was unable to provide product. So instead of using two new competing materials, only one could be tested against the “known” superior product. Therefore, a third sheeting was substituted that was known to be very good, but not superior.
Northern Arizona University AZTrans:
• Test method is an observational study using volunteer observers (no physical testing done).
• Test Site allows two types of testing: • Static testing: Evaluations at fixed distances from sign. • Dynamic testing: Evaluation while moving toward sign.
• Investigator uses “double blind” methods: • Test subjects only identified by a number. • Test materials only identified by a letter.
Testing Methods
• Material C: Avery Dennison Omni View, Series T9500
Notes: (1)This is the material that had demonstrated its superior qualities
and was currently in use by ADOT at the time of this test. (2)This material was substituted after a manufacture dropped out
of the test. It was the predecessor “superior” product that ADOT formerly used.
Three Materials Tested
All Testing done under nighttime conditions without supplemental freeway lighting
Northern Arizona University AZTrans:
Sign 1:
Sign 2:
Sign 3:
Double Blind Test (Neither Observers nor Analyst knows which material is assigned to which line on the signs)
KEY: Material by Line Sign 1: C, A, B Sign 2: A, B, C Sign 3: B, C, A
Northern Arizona University AZTrans:
Northern Arizona University AZTrans:
Two Tests: Static and Dynamic
Static Test • Observer stops at three different distances in front of each sign.
• Rates sign at each stop: • 1 = Unsatisfactory • 2 = Marginal • 3 = Satisfactory • 4 = Good • 5 = Excellent.
• Three Distances and Legibility Indices (13.33 in capital letter height for Clearview text)
• 600 ft yields 40.4 ft/in • 450 ft yields 33.8 ft/in • 300 ft yields 22.5 ft/in
Dynamic Test • Observer states when s/he can first read sign while traveling toward sign at 55 mph.
• Observer in passenger seat, Recorder in back seat, and Driver who maintains speed and lane.
• Distance recorded with a DMI (Distance Measuring Device) installed in vehicle. Start DMI when observer can read it and then stop DMI when pass under sign.
Northern Arizona University AZTrans:
1007550250
100
80
60
40
20
0
Mean StDev N
Metro-age Dynamic-age Static-age
Variable
CDF Age Distributions Phoenix Metro vs. Static Test vs. Dynamic Test
With fitted Normal Distributions
• B and C are essentially equal
Analysis of Variance for drv_rated, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P mat 2 1267.067 1267.067 633.534 1141.62 0.000 distance 2 888.423 888.423 444.211 800.47 0.000 mat*distance 4 63.659 63.659 15.915 28.68 0.000 driver 99 718.527 718.527 7.258 13.08 0.000 Error 2592 1438.406 1438.406 0.555 Total 2699 4376.083 Bonferroni Simultaneous Tests Response Variable drv_rated All Pairwise Comparisons among Levels of mat NOTE: If P-Value < 0.05, then means are different (95% CI) mat = A subtracted from: Difference SE of Adjusted mat of Means Difference T-Value P-Value B -1.400 0.03512 -39.87 0.0000 C -1.501 0.03512 -42.75 0.0000 mat = B subtracted from: Difference SE of Adjusted mat of Means Difference T-Value P-Value C -0.1011 0.03512 -2.879 0.0121 Bonferroni Simultaneous Tests Response Variable drv_rated All Pairwise Comparisons among Levels of distance NOTE: If P-Value < 0.05, then means are different (95% CI) distance = far subtracted from: Difference SE of Adjusted distance of Means Difference T-Value P-Value mid 0.9233 0.03512 26.29 0.0000 near 1.3789 0.03512 39.27 0.0000 distance = mid subtracted from: Difference SE of Adjusted distance of Means Difference T-Value P-Value near 0.4556 0.03512 12.97 0.0000
Northern Arizona University AZTrans:
C
B
A
54321
l
Rating
2.88222
2.98333
4.38333
2.88222
2.98333
4.38333
Comparison of Materials - Ratings means Static Test - 95% Bonferroni CI for the Mean
Unsatisfactory ExcellentSatisfactory
Results: • A significantly
• B and C are essentially equal.
Analysis of Variance for Dynamic-Distance, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Material 2 321910 321910 160955 15.62 0.000 Driver 21 2645595 2645595 125981 12.22 0.000 Error 42 432830 432830 10305 Total 65 3400335 Bonferroni Simultaneous Tests Response Variable Dynamic-Distance All Pairwise Comparisons among Levels of Material NOTE: If P-Value < 0.05, then means are different (95% CI) Material = A subtracted from: Difference SE of Adjusted Material of Means Difference T-Value P-Value B -145.2 30.61 -4.745 0.0001 C -150.9 30.61 -4.930 0.0000
Material = B subtracted from: Difference SE of Adjusted Material of Means Difference T-Value P-Value C -5.682 30.61 -0.1856 1.000
Northern Arizona University AZTrans:
C
B
A
10008006004002000
l
Dynamic-Distance
609.545
615.227
760.455
Dynamic Test - Comparison of Material Means 95% Bonferroni CI for the Mean
Note: This Confidence Interval (C.I.) graph has more noise (error) in it so C.I.s overlap (blue brackets).
Actual ANOVA test removed noise by blocking, revealing A has a greater dynamic distance than either B or C (red brackets).
Note: Red brackets for visual purposes only; Not calculated.
Northern Arizona University AZTrans:
• Statistically significant differences shown between the three materials • Dynamic testing correlated static testing results
• ADOT continued use of its current “superior” product • Research showed that dynamic testing may be an acceptable surrogate for static testing • More work needed in this area
Conclusions
AZTrans: The Arizona Laboratory for Applied Transportation Research
Questions?
nearmidfar
6
5
4
3
2
1
distance
Static and Dynamic Sign Sheeting Testing Using InVehicle Observers
Edward J. Smaglik, Associate Professor Craig Roberts, Associate Professor
Northern Arizona University AZTrans:
• Limited resources available to DOT for control signage. • Focus on best combination of safety and price. • Currently using a “superior” product, but from sole source.
• Two manufactures offering recent products as “superior”, but are essentially untested.
• If claims true, then competition should result in lower cost to ADOT to provide signs with “superior” sheeting.
Motivation for Research
Research Question
One(1) new sign sheeting material is offered by its manufacturer as “equivalent” to the current superior product being used by ADOT.
Can this equivalency be supported by evidence from a double blind observational experiment under actual roadway conditions?
(1)Originally two new sheetings were to be tested but at the last minute, one of the manufactures was unable to provide product. So instead of using two new competing materials, only one could be tested against the “known” superior product. Therefore, a third sheeting was substituted that was known to be very good, but not superior.
Northern Arizona University AZTrans:
• Test method is an observational study using volunteer observers (no physical testing done).
• Test Site allows two types of testing: • Static testing: Evaluations at fixed distances from sign. • Dynamic testing: Evaluation while moving toward sign.
• Investigator uses “double blind” methods: • Test subjects only identified by a number. • Test materials only identified by a letter.
Testing Methods
• Material C: Avery Dennison Omni View, Series T9500
Notes: (1)This is the material that had demonstrated its superior qualities
and was currently in use by ADOT at the time of this test. (2)This material was substituted after a manufacture dropped out
of the test. It was the predecessor “superior” product that ADOT formerly used.
Three Materials Tested
All Testing done under nighttime conditions without supplemental freeway lighting
Northern Arizona University AZTrans:
Sign 1:
Sign 2:
Sign 3:
Double Blind Test (Neither Observers nor Analyst knows which material is assigned to which line on the signs)
KEY: Material by Line Sign 1: C, A, B Sign 2: A, B, C Sign 3: B, C, A
Northern Arizona University AZTrans:
Northern Arizona University AZTrans:
Two Tests: Static and Dynamic
Static Test • Observer stops at three different distances in front of each sign.
• Rates sign at each stop: • 1 = Unsatisfactory • 2 = Marginal • 3 = Satisfactory • 4 = Good • 5 = Excellent.
• Three Distances and Legibility Indices (13.33 in capital letter height for Clearview text)
• 600 ft yields 40.4 ft/in • 450 ft yields 33.8 ft/in • 300 ft yields 22.5 ft/in
Dynamic Test • Observer states when s/he can first read sign while traveling toward sign at 55 mph.
• Observer in passenger seat, Recorder in back seat, and Driver who maintains speed and lane.
• Distance recorded with a DMI (Distance Measuring Device) installed in vehicle. Start DMI when observer can read it and then stop DMI when pass under sign.
Northern Arizona University AZTrans:
1007550250
100
80
60
40
20
0
Mean StDev N
Metro-age Dynamic-age Static-age
Variable
CDF Age Distributions Phoenix Metro vs. Static Test vs. Dynamic Test
With fitted Normal Distributions
• B and C are essentially equal
Analysis of Variance for drv_rated, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P mat 2 1267.067 1267.067 633.534 1141.62 0.000 distance 2 888.423 888.423 444.211 800.47 0.000 mat*distance 4 63.659 63.659 15.915 28.68 0.000 driver 99 718.527 718.527 7.258 13.08 0.000 Error 2592 1438.406 1438.406 0.555 Total 2699 4376.083 Bonferroni Simultaneous Tests Response Variable drv_rated All Pairwise Comparisons among Levels of mat NOTE: If P-Value < 0.05, then means are different (95% CI) mat = A subtracted from: Difference SE of Adjusted mat of Means Difference T-Value P-Value B -1.400 0.03512 -39.87 0.0000 C -1.501 0.03512 -42.75 0.0000 mat = B subtracted from: Difference SE of Adjusted mat of Means Difference T-Value P-Value C -0.1011 0.03512 -2.879 0.0121 Bonferroni Simultaneous Tests Response Variable drv_rated All Pairwise Comparisons among Levels of distance NOTE: If P-Value < 0.05, then means are different (95% CI) distance = far subtracted from: Difference SE of Adjusted distance of Means Difference T-Value P-Value mid 0.9233 0.03512 26.29 0.0000 near 1.3789 0.03512 39.27 0.0000 distance = mid subtracted from: Difference SE of Adjusted distance of Means Difference T-Value P-Value near 0.4556 0.03512 12.97 0.0000
Northern Arizona University AZTrans:
C
B
A
54321
l
Rating
2.88222
2.98333
4.38333
2.88222
2.98333
4.38333
Comparison of Materials - Ratings means Static Test - 95% Bonferroni CI for the Mean
Unsatisfactory ExcellentSatisfactory
Results: • A significantly
• B and C are essentially equal.
Analysis of Variance for Dynamic-Distance, using Adjusted SS for Tests Source DF Seq SS Adj SS Adj MS F P Material 2 321910 321910 160955 15.62 0.000 Driver 21 2645595 2645595 125981 12.22 0.000 Error 42 432830 432830 10305 Total 65 3400335 Bonferroni Simultaneous Tests Response Variable Dynamic-Distance All Pairwise Comparisons among Levels of Material NOTE: If P-Value < 0.05, then means are different (95% CI) Material = A subtracted from: Difference SE of Adjusted Material of Means Difference T-Value P-Value B -145.2 30.61 -4.745 0.0001 C -150.9 30.61 -4.930 0.0000
Material = B subtracted from: Difference SE of Adjusted Material of Means Difference T-Value P-Value C -5.682 30.61 -0.1856 1.000
Northern Arizona University AZTrans:
C
B
A
10008006004002000
l
Dynamic-Distance
609.545
615.227
760.455
Dynamic Test - Comparison of Material Means 95% Bonferroni CI for the Mean
Note: This Confidence Interval (C.I.) graph has more noise (error) in it so C.I.s overlap (blue brackets).
Actual ANOVA test removed noise by blocking, revealing A has a greater dynamic distance than either B or C (red brackets).
Note: Red brackets for visual purposes only; Not calculated.
Northern Arizona University AZTrans:
• Statistically significant differences shown between the three materials • Dynamic testing correlated static testing results
• ADOT continued use of its current “superior” product • Research showed that dynamic testing may be an acceptable surrogate for static testing • More work needed in this area
Conclusions
AZTrans: The Arizona Laboratory for Applied Transportation Research
Questions?
nearmidfar
6
5
4
3
2
1
distance