smart city – innovation, connectivity and integration of ...smart city – innovation,...
TRANSCRIPT
Smart City – Innovation, Connectivity and Integration of
Road Infrastructure
Bill ButtlarUniversity of Missouri-Columbia
Buttlar Research Group (esp. H. Majidifard, B. Jahangiri, K. Barri), Yaw Adu-
Gyamfi, Amir Alavi, Shantanu Chakraburtty, Henrique Reis, Missouri
Department of Transportation, Illinois Tollway, Missouri Asphalt Pavement
Association, National Science Foundation, American Society of Civil
Engineers, Virgin Hyperloop One
Acknowledgements
What is a ‘Smart City?’
3
“A smart city is an urban development vision to integrate multiple information and communication technology (ICT) solutions in a secure fashion to manage a city’s asset, including: schools, libraries, transportation systems, hospitals, power plants, water supply networks, waste management, law enforcement, and other community services.
The goal of building a smart city is to improve quality of life by using technology to improve the efficiency of services and meet residents’ needs. ICT allows city officials to interact directly with the community and the city infrastructure and to monitor what is happening in the city, how the city is evolving, and how to enable a better quality of life.”
en.wikipedia.org/wiki/Smart_city
http://www.barcinno.com/smart-city-barcelona/
Smart Infrastructure Applications
4
Energy-Harvesting Sidewalk (UK)
Infrastructure HealthDiagnosis and Self-Repair
Vehicle Charging
3D Scanning/Paving
Intelligent Infrastructure Research at MAPIL
Active Digital Asset Management System
Mobile Molecular Scanner
Smart / Nano-Modified Asphalt (Windgo) Partners:
Smartphone Pavement Assessment
Machine Learning
Infrastructure HealthDiagnosis and Self-Repair: Acoustic Emission
Source Location
Behnia,B., Dave, E.V., Buttlar, W.G., Reis. H., “Acoustic Emissions (AE) Technique for Evaluation of Embrittlement Temperature of Asphalt Binders: Development and Field Calibration”, International Journal of Road Materials and Pavement Design, Vol. 14, pp. 57-78, 2013.Sun, Z., Behnia, B., Buttlar, W. G., and Reis, H. "Assessment of Low-Temperature Cracking in Asphalt Materials Using an Acoustic Emission Approach," Journal of Testing and Evaluation, Vol. 45, No. 6, 2017, pp. 1948-1958.
Multifunctional Pavement Using RFID Sensing Technology
• Wireless sensor networks
• Radio frequency identification technology
• Battery free
scheme of the sensor tag
Realized prototype of the sensor(credit: Chakrabartty research group, Washington University in St. Louis
Attaching the tags on DC(T) samples and monitoring the crack propagation
Smart Materials
InputProcess
Definition:A material that exhibits one or more properties that can be significantly changed in a controlled fashion by external stimuli.
Output
Key point: Truly smart materials will have ‘intelligence’ built in at the material scale: Examples: smart roofing shingle, self-healing concrete
𝟏𝟏𝟏𝟏𝟑𝟑𝟑𝟑
Machine Learning Is Changing The World
Machine Learning in Transportation
• Transportation
Elimination of human error in the driving process should make our journeys faster and safer.
Optimization of shipping routes speeds delivery, and lowers cost/environmental footprint.
Example: Development of a model using an innovative machine
learning technique ~ Genetic Programming (GP), to predict the
fracture energy of asphalt mixture specimens at low temperatures
0
1
2
3
4
0 5 10Load
(kN
)
CMOD (mm)
Gf = Fracture energyGP
Graph GP
Linear GP
Tree GP
Mix Quality Characteristics:(voids, aggr. structure, asphalt content, recycling type/amt….)
Machine Learning in Materials
Majidifard, H., Jahangiri, B., Alavi, A., Buttlar, W., (2018), “New Machine Learning-based Prediction Models for Fracture Energy of Asphalt Mixture,” Measurement, 135 (2019) 438–451.
GP Method
In the case of GP, prior knowledge about the underlying
physical process based on engineering judgement can be
incorporated into the learning formulation, which greatly
enhances the usefulness of GP over other ML techniques,
such as Artificial Neural Networks (ANNs).
ANNs are black box models…you cannot recover the inside of the black box, whereas in GP, you get the highly nonlinear reln’s, plus the ability to put the solution in the form of a straight-forward equation. This is a breakthrough for practical engineering.
Gene Expression Programming
Invented by Koza (1992), GP is an extension to the genetic
algorithm (GA) approach, which can automatically
generate mathematical models, akin to Darwinian
evolutionary theory (millions of models generated, only the
best survive evolution).
GP
Graph GP
Linear GPTree GP
GP Approaches (Alavi and Gandomi 2011)
Our initial GEP-based prediction model
𝐺𝐺𝑓𝑓𝑗𝑗𝑚𝑚2 = 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 − 9 𝑁𝑁𝐴𝐴 − 𝑅𝑅𝑁𝑁𝑅𝑅 5.36 + 𝐴𝐴 − 𝐿𝐿𝐴𝐴𝑅𝑅𝐺𝐺 − 𝐿𝐿𝐴𝐴𝑅𝑅𝐺𝐺
+ 𝑁𝑁𝐴𝐴 𝐺𝐺4 − 1.7 + 𝑈𝑈𝐴𝐴𝑈𝑈 +𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 − 𝑈𝑈𝐴𝐴𝑈𝑈
𝐴𝐴 + 𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑁𝑁𝐴𝐴 − 𝑁𝑁𝐴𝐴 + 6.45− 𝑅𝑅𝑁𝑁𝑁𝑁 + 𝑅𝑅𝑁𝑁𝑅𝑅
+𝐿𝐿𝐴𝐴𝑅𝑅𝐺𝐺𝑁𝑁𝐴𝐴 + 𝐿𝐿𝐴𝐴𝑅𝑅𝐺𝐺2 + 3.49𝐴𝐴 + 𝑁𝑁𝐴𝐴 × 𝑅𝑅𝑁𝑁𝑅𝑅
+ 𝐴𝐴 𝐴𝐴𝑅𝑅𝐴𝐴3 − 10𝐴𝐴 − 𝑅𝑅𝑁𝑁𝑅𝑅 × 𝑁𝑁𝐴𝐴 − 6.461 + 𝑈𝑈𝐴𝐴𝑈𝑈
* Derived after controlling millions of highly nonlinear models, which is not feasible via other nonlinear regression approaches
Training data Testing data
Measured vs. predicted Gf
Actually, three portions – Learning (70%), Validation (10%), and Testing (20%). Here we combined Learning and Validation in the figure (both are involved in model selection process).
Hamburg Machine Learning Project @ MAPIL
Table 1. Statistical parameters of the dependent and independent variables. Mix
type UTI (℃)
HTPG (℃)
AC (%)
ABR (%)
NMAS (mm)
RAP (%)
RAS (%) G AT CRC
(%) T
(℃) Passes R (mm)
Mean 1.1 87.8 58.9 5.9 28.8 11.3 20.9 8.0 1.2 1.2 2.3 52.2 13131 3.8 Median 1.0 86.0 58.0 5.7 32.5 12.5 20.4 0.0 1.0 1.0 0.0 50 10000 2.7 Range 1.0 18.0 24.0 2.8 48.4 14.3 35.3 33.0 1.0 1.0 10.0 24 15000 19.2 Max 2.0 98.0 70.0 7.9 48.4 19.0 35.3 33.0 2.0 2.0 10.0 64 20000 19.7 Min 1.0 80.0 46.0 5.1 0.0 4.8 0.0 0.0 1.0 1.0 0.0 40 5000 0.6
Sensitivity of Variables
Trends Predicted by Machine Learning Model
0.0
5.0
10.0
15.0
20.0
25.0
0 10000 20000 30000
Rut
Dep
th (m
m)
Number of Passes
4.5% AC5% AC5.5% AC
(b)
Pavement Management: You Know the Underlying Principle…
• Accurate pavement condition monitoring is important, Why?If the cracks are detected sooner, the maintenance process will be
less expensive.
Increased costdue to delayed
monitoring
Pavement Monitoring
• Traditional method
ARAN, laser-based sensing,
GPR
Cost :
A vehicle equipped with modern sensor and computing systems was purchased by the Ohio Department of Transportation for US$1,179,000 with an annual operating cost US$70,000 (Vavrik et al., 2013).
Deep Learning Approach for Automatic Pavement Distresses Detection using Google Street view images
Why Google images? Free and available for every road section
Why Automated system?
1) Significant cost benefit
2) Begins to remove human error and judgment from the calculation process
Annotating 7500 images with nine classes
Distress Type Distress IDReflective Crack D0Transvers Crack D1
Block Crack D2Longitudinal Crack D3
Alligator Crack D4Sealed Reflective Crack D5Lane Longitudinal Crack D6
Sealed Longitudinal Crack D7pothole D8
Table 1. Distress types versus their corresponding distress ID
* To our knowledge, this is the most comprehensive dataset annotated by pavement experts in the literature
Model accuracy
YOLO V2 D0 D1 D2 D3 D4 D5 D6 D7 D8D0 0.99 0.01 0.00 0.00 0.00 0.01 0.00 0.00 0.00D1 0.02 0.97 0.01 0.00 0.00 0.00 0.00 0.00 0.00D2 0.00 0.00 0.99 0.00 0.00 0.00 0.00 0.00 0.00D3 0.00 0.00 0.01 0.98 0.00 0.00 0.01 0.00 0.00D4 0.00 0.00 0.00 0.00 0.99 0.00 0.00 0.00 0.00D5 0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00D6 0.00 0.00 0.00 0.00 0.00 0.00 0.99 0.00 0.00D7 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00 0.00D8 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00
Crack class name Precision Recall F1Reflective crack 0.93 0.76 0.84Transverse crack 0.9 0.83 0.86
Block crack 0.93 0.79 0.85Longitudinal crack 0.91 0.83 0.87
Alligator crack 0.91 0.74 0.82Sealed transverse crack 0.93 0.83 0.87
Sealed longitudinal crack 0.93 0.79 0.85Lane longitudinal crack 0.94 0.57 0.71
Pothole 0.96 0.78 0.86Average 0.93 0.77 0.84
Detection and classification results for nine distress types
Confusion matrices obtained on the classification dataset using YOLO v2
Development of a model using an innovative machine learning
technique ~ Genetic Programming (GP), to predict the Pavement
condition as PASER rating using the detected distresses from
developed crack detection model
Predicted PASERRating
GP
Graph GP
Linear GP
Tree GP
Number and types of detected
distresses per section
Pavement Condition Prediction Model Development based on YOLO Model outputs
𝑅𝑅𝑁𝑁𝑁𝑁𝐸𝐸𝑅𝑅= 𝑓𝑓[𝑑𝑑(1),𝑑𝑑(2),𝑑𝑑(3),𝑑𝑑(4),𝑑𝑑(5),𝑑𝑑(6),𝑑𝑑(7),𝑑𝑑(8),𝑑𝑑(9)]
where, d(1)= Reflective Crack, d(2)= Transverse Crack, d(3)= Block Crack, d(4)= Longitudinal Crack, d(5)= Alligator Crack, d(6) = Sealed Reflective Crack, d(7)= Lane Longitudinal Crack, d(8)= Sealed Longitudinal Crack,d(9)= Pothole. All the variable including d(1) to d(9) are the average number of distresses per each section.
Training dataset
Validation dataset
Testing dataset
Results from GEP prediction Model
Developing a U-Net based Model for Distress Quantification
a) Raw Image b) pre-trained U-Net output
a) Raw Image
c) Re-trained U-Net output
b) pre-trained U-Net output
c) Re-trained U-Net output
Benefits of Integrated YOLO and U-Net Model
• Considers both type and density of distresses
• Neither Yolo nor U-net modeling alone can assure accurate pavement condition assessment because:
• YOLO can detect types of cracks by drawing frames around them, but cannot differentiate the severity or density of the detected cracks.
• U-Net can quantify the density of cracks, but it can not discriminate crack types. As an example, longitudinal joint cracks and sealed cracks are not as detrimental to pavement condition as compared to wheel path longitudinal cracks. The U-net model is not able to address this issue.
30
https://fox2now.com/2019/10/03/hyperloop-test-pod-on-display-at-mizzou/
Is Hyperloop for Real? How Will it Integrate with Current Surface Transportation?
With Smart Cities of the Future?
Hyperloop One Pod at University of Missouri-Columbia, October 3-4, 2019
Hyperloop One Master Class, held at University of Missouri-Columbia, October 3, 2019
National Coverage on CNN
Hyperloop Smart Infrastructure Monitoring
• Integrating self-powered smart sensors in pylons and tube structures to ensure consistent response and to identify anomalies (i.e., micro-cracks)…wirelessly and economically
• Developing UAV monitoring system + solar-powered data transmission modules for collecting data and validating observations
• Developing machine learning algorithms for early prediction of structural fatigue, damage, cracking, and other anomalies
A network of low-cost, self-powered sensing nodes
Unveiled at ASCE National Convention, Oct. 10-13, 2019(to be shared with CAPSA, with permission from ASCE)
https://www.futureworldvision.org
Alternative Energy
Change on this scale can drive
confusion and dysfunction unless
industries, organizations, and
individuals are prepared to tackle
new realities
Autonomous Vehicles
Climate Change Smart Cities
High-Tech Construction/ Advanced Materials Policy & Funding
35
Floating City Example1. Concept Art for Flooded Cityscape
2. Systems View of Interactive Prototype
3. Floating City Detail Concept Art
4. Concept Art for Floating City Overview
5. Computer Model of City
1
2 3
4