articulo pav
Post on 03-Jun-2018
233 Views
Preview:
TRANSCRIPT
-
8/11/2019 Articulo Pav
1/12
Review
Chronological development history ofXYtable based pavement crack sealers and
research ndings for practical use in the eld
Young S. Kim a, Hyun S. Yoo a, Jeong H. Lee b,, Seung W. Han a
a Department of Architectural Engineering, Inha University, South Koreab Center for Cost Engineering Research, Inha University, South Korea
a b s t r a c ta r t i c l e i n f o
Article history:
Accepted 19 February 2009
Keywords:
Automation
Pavement
Crack sealing
Machine vision
Robotics
Teleoperation
During the last two decades, several teleoperated and machine-vision assisted systems have been developed
to automate the overall process of routing and sealing pavement cracks. Productivity improvement, improved
safety and quality, and reduced road user costs have motivated these developments. This paper presents the
chronological development history ofxy table based pavement crack sealers, which have been developed
and demonstrated since the early 1990s, and compares their technical advances. This paper also discusses
primary research ndings in machine vision software and hardware designs of an automated pavement crack
sealer to be newly developed for practical use in the eld. Finally, conclusions and recommendations are
made concerning the value of implementing and practically using the automated pavement crack sealer.
2009 Elsevier B.V. All rights reserved.
Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
2. Automation needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5143. Automated pavement crack sealing systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
3.1. Chronological development history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
3.2. Evolution of the control paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514
3.3. Evolution of machine vision software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515
3.4. Evolution of hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516
4. Primary researchndings for practical application in the eld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
4.1. Software design requirement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
4.1.1. Path planning (full automation) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
4.1.2. Graphical user interface design for crack sealer control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518
4.2. Hardware design requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
4.2.1. Types of crack to be sealed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
4.2.2. Crack image collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
4.2.3. Entire crack sealing system architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
4.2.4. Manipulator and end-effector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
5. Conclusions and recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
1. Introduction
Crack sealing, a routine and necessary part of pavement main-
tenance, is a dangerous,costly,and labor-intensiveoperation.During the
last two decades, several systems based onxytable for automatically
Automation in Construction 18 (2009) 513524
Corresponding author. Center for Cost Engineering Research, Inha University, 253
Yonghyun-dong, Nam-gu, Incheon, 402-751,South Korea. Tel.:+82 32 8729757; fax: +82 32
866 4624.
E-mail address:inhacmr@hotmail.com(J.H. Lee).
0926-5805/$ see front matter 2009 Elsevier B.V. All rights reserved.
doi:10.1016/j.autcon.2009.02.007
Contents lists available at ScienceDirect
Automation in Construction
j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / a u t c o n
mailto:inhacmr@hotmail.comhttp://dx.doi.org/10.1016/j.autcon.2009.02.007http://www.sciencedirect.com/science/journal/09265805http://www.sciencedirect.com/science/journal/09265805http://dx.doi.org/10.1016/j.autcon.2009.02.007mailto:inhacmr@hotmail.com -
8/11/2019 Articulo Pav
2/12
routing and sealing pavement cracks have been developed. Examples
include: 1) CMU laboratory prototype (1990) [1,2], 2) CMU-UT eld
prototype (1992)[2,3], 3) UT Automated Road Maintenance Machine
(ARMM) (1997)[36], and4) AutomatedPavement Crack Sealer (APCS)
(2004) [7,8]. Since automating pavement crack sealing can improve
safety, productivity, and quality, and also reduce road user costs, there
has been extremely large demand for practical use of automated
pavement crack sealers in the areas of road construction and
maintenance. While early works sought to completely automate crackmapping and sealing activities, experience and the resulting deeper
understanding of the enabling technologies have highlighted the
importance ofnding a desirable balance between human and machine
functions in the control of automated pavement crack sealers.
Through trial and error and about 20 years of perseverance, the
APCS, the most recent deliverable research, has achieved a desirable
balance between manual and automated functions for automated
pavement crack sealing. Recent eld trials of the full scale APCS have
also indicated that automated pavement crack sealing is now
technically, economically, and nancially feasible. Despite such
numerous efforts to automate conventional crack sealing operations,
lessons learned from previous system developments and eld trials
have indicated that several improvements in both software and
hardware designs are still required for their practical application in
the eld. The primary objective of this paper is to present overall
software and hardware design requirements for practical use of an
automated pavement crack sealer in an effort to fulll the aforemen-
tioned demand in road construction and maintenance. This paper
presents the chronological development history of xy table based
pavement crack sealers, which have been developed and demon-
strated since the early 1990s, and compares their technological
advances. This paper then proposes primary research ndings in
machine vision software and hardware designs of an automated
pavement crack sealer to be newly developed for practical use in the
eld. A conceptual hardware design for a new model is proposed in
this paper as well. Finally, conclusions and recommendations are
made concerning the value of implementing and practically using the
automated pavement crack sealer.
2. Automation needs
Crack sealing is a maintenance technique commonly used to
prevent water and debris penetration and reduce future pavement
degradation. The conventional crack sealing operations are, however,
dangerous, costly, and labor-intensive. With respect to crack sealing
crews, labor turnover and training are also increasing problems.
Automation of the crack sealing process would improve productivity
and quality, and offers safety benets by getting workers off the road.
The reduction in crew size and the increase in productivity of the
automated sealing process are expected to be translated directly into
signicant cost savings.
For example, APCS eld test results indicated that the daily
productivity would be 1.59 km/day. Compared with the productivityof a conventional crack sealing method (1.21 km/day), that of the
APCS was as much as 0.39 km/day higher. On-site tests and a
performance analysis of the APCS demonstrated that its use would
allow a 50% reduction of the labor force and 32.5% enhanced
productivity [8]. Furthermore, when considering nighttimeoperations
and possible hardware improvements of the developed APCS, the
productivity of the APCS would be even higher.
The results of an economic feasibility analysis of the APCS also
revealed that automating the conventional crack sealing operation is
highly feasible and would potentially bring enormous cost savings.
Information for the analysis was gathered to estimate costs and
benets, analysis perspectives were chosen, and the market was
studied. Rate of return, benetcost ratio, break-even point, and
sensitivity analyses were used to verify the economic feasibility of
implementing the automated method in place of the conventional
method. Under assumptions such as an APCS purchase cost of US
$72,000, 100 working days per year, use of 1 APCS, 10% MARR, a
10 year planning horizon, a 50% reduction in labor force, etc., it was
anticipated that a contractor would be able to cut conventional
maintenance costs by 43.6%. With the above assumptions, the
economic analysis results of the APCS also showed a value of 122.5%,
5.5, a 15 month in rate of return period, benetcost ratio, break-even
point, respectively, thus making the use of APCS highly attractive. Theresults of a sensitivity analysis and predictions pertaining to reduction
of road user costs obtained using Paramics simulation software have
been presented elsewhere[8].
3. Automated pavement crack sealing systems
In this chapter,xytable based systems developed since the early
1990s for automatically routing and sealing pavement cracks are
briey described. Technological details of the systems are presented,
and related research accomplishments, concerns, and technical
advances are identied. Visual appearances of each prototype system
are illustrated as well.
3.1. Chronological development history
Table 1 briey describes the accomplishments and major limita-
tions of previous research works. The hardware of early xy table
based pavement crack sealing systems was incomplete in the early
stage. At the same time, the software for mapping and modeling the
crack network to be sealed and path planning were not efcient in
terms of productivity, quality, and accuracy for practical use in the
eld. In addition, the hardware and software were not integrated
properly, causing inaccurate and inefcient movement of the auto-
mated pavement crack sealing systems. The experience acquired from
such early attempts and recent advances in relevant robotic
technologies motivated authors to develop more advanced pavement
crack sealer (APCS). Although the APCS employs a unique man-
machine interfaced control process and provides innovative technical
advances compared to previous research works [16,10], they havenot been practically used on sites due to limitations in their hardware
and software. Detailed comparisons and several limitations regarding
the control paradigm, machine vision software, and hardware in the
xy table based pavement crack sealers developed in previous
research works are presented inSections 3.23.4.
3.2. Evolution of the control paradigm
Complete autonomy[1,3]could be achieved for the whole process
(image acquisition, crack detection and mapping, path planning,
blowing, sealing, and squeegeeing) of automated pavement crack
sealing, but usually at a cost, speed, and accuracy that is impractical
and unacceptable. Complex evolution of the control paradigm has
resulted in a functional production prototype system [46,9,10]thatachieves a good balance between manual functions and automated
functions by taking advantage of the respective strengths of man and
machine in the whole process. Teleoperation based on remote video,
man-machine interfaces, machine vision, and graphical programming
alone can achieve benets of automation in the unstructured
pavement crack sealing work environment. Lessons learned from
system developments and eld trials have also indicated that
computer assistance in the form of man-machine interfaced graphical
programming and machine vision is essential and can cost-effectively
help to achieve improvements in the productivity and quality of
automated methods. Considering recent successful developments in
teleoperated construction eld robots, it is thought that evolution
toward teleoperation as a control paradigm of the automated
pavement crack sealer is highly desirable. The ARMM and the APCS
514 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
3/12
employ some forms of man-machine interfaced control process for
automatically sealing pavement cracks.
3.3. Evolution of machine vision software
In general, machine vision software in automated pavement crack
sealers can be classied into the following four categories:
Image acquisition
In previouslydevelopedpavementcrack sealers, a computer imaging
system is typically used to view cracks to be sealed on the surface. The
use of remote video cameras is widely employed to provide visual
feedback in the machine. To capture and digitize video images of the
cracks in the machine's work space, a commercial image processing
board is installed in the PC that controls the automated crack sealers.
One or two security cameras mounted on a super-structure of the XY-
table manipulator ((1)(4) inTable 1) acquire live pavement surface
images displayed on the PC's monitor.
Initial crack detection, mapping, and representation
To automatically seal cracks, exact crack locations are the most
important information.In developinga new image processingalgorithm
for automatically sensing, mapping, and representing cracks to be sealed
in pavements, the quality of the segmentation must take the highest
priority. The overall efciency of the algorithm is also important for
project success,if thecracks areautomatically detected andmapped bya
computer.
In CMU laboratory and CMU-UT eld prototypes, a video camera is
used to acquire images. These images are then digitized and subse-
quently combined with laser range data of surface contours using a
specially designed multi-layer quadtree model and image analysis
algorithms, via the process of sensor fusion [1
3]. However, it wasconcluded that detection and mapping pavement cracks to be sealed by
computer functions alone may not be desirable. This is due to the fact
that most pavement cracks are highly noisy and unstructured in their
digitizedform at the pixel level. Thelow contrast betweenthe distresses
and the background also further complicate the pavement analysis. As a
result, the lessons learned revealed that a human operator should
interact directly with the computer in thecrackdetection, mapping, and
representation processes, as in the ARMM and the APCS.
As an example, such a man-machine interface was rst tested to
detect, map, and represent pavement cracks to be sealed in the ARMM
[46,9,10]. Under proper conditions of illumination, humans can see
color, brightness, and form, thus easily distinguishing real cracks from
pavement background noise such as sealed cracks or oil marks or skid
marks. As such, allowing the operator to point out the existence and
location of pavementcracks using a styluson a touchsensitive monitoris
an effective approach. A graphical program can be used to generate a
computer-based model of the surface cracks to be sealed and to provide
visual feedback to the system operator. A good model can be achieved
since the work environment of the automated pavement crack sealer is
fairly static. Despite such advantages, this approach has a drawback
that can signicantly degrade the quality of the resultant seal. The
imperfection of human hand-eye coordination causes errors even in a
good work environment. Also, operator arm fatigue can increase such
errorswhen tracing cracks on themonitor. In theARMM,machine vision
based line snapping or manual editing wasthus required to compensate
human hand-eye coordination errors [5,6]. In a subsequent research
effort, a similar man-machine interface was applied to the ARMM for
detecting, mapping, and representing the pavement cracks to be sealed.
However, in the APCS, crack network detection, mapping, andrepresentation are operated in two modes: full automation and
semi-automation. A unique and innovative machine vision algorithm
has been developed for the automation of pavement crack sealing.
Detailed descriptions of both fully and semi-automated crack network
detection, mapping, and modeling algorithm have been presented
elsewhere[9].
Path planning
Compared to conventional crack sealing operations, the efciency of
movement (pathplan) of an automated crack sealer is a key factor with
regard to performance. Efciency of movement is governed by the
length of the paththat the end-effectors follow to cover the whole crack
network. Other performance factors include manipulator speed and
accuracy, and the duration of crack detection, mapping, and representa-tion. If cracks are fully monitored and mapped by thehuman naked eye,
a path planis notrequired. However,if cracksare mappedmanually on a
computer monitor, an implicit pathplan is generated by the sequence of
strokes made by the operator, including the beginning and end points.
Meanwhile, automated path planning is necessary when cracks are
automatically mapped by the computer. In the CMU-UT eld prototype
and the ARMM, a series of greedy path planning algorithms using a
double linked data structure and an array data structure were proposed
in an effort to automatically generate a path to effectively traverse the
cracknetwork to be sealed[10]. Thetest results showedthat,on average,
the time taken to compute a greedy path was much less than the time
saved by executing the greedy path. Recently, optimal and more
advanced greedy path planning algorithms were successfully developed
and applied to the APCS [7]. In an effort to minimizethe idlepath inthe
Table 1
Previous research works related to thexytable based pavement crack sealing systems.
Previous research works Accomplishments and major concerns
(1) CMU laboratory
prototype (1990)
Conceptual design of S/W and H/W for
full automation
Crack detection, mapping and
representation using digital image
processing
Incomplete and unstable fabrication of
H/W(X
Ymanipulator) Excessive processing time in the crack
detection, mapping and modeling
(2) CMU-UT eld
prototype (1992)
Development of a machine vision
algorithm using sensor (vision and
laser range scanner) fusion
Development of path planning algorithm
using linked data structure
Fabrication of more robust H/W
(XY-manipulator) and partial integration of
S/W and H/W
Partial verication of fully automated
crack sealing process
Excessive processing time due to slow
range scanning
(3) ARMM (1997) Suggestion of a man-machine balanced
control process for effectively sealing
pavement cracks
Development of a machine vision
algorithm (manual mapping, line snapping
and manual editing) using graphical
programming and user-friendly control
software
Development of greedy path planning
algorithm using array data structure
Need to improve the design of turret and
machine vision algorithm[2]
(4) APCS (2004) Suggestion of a man-machine balanced
control process for effectively sealing
pavement cracks
Development of an innovative machine
vision algorithm which can both
automatically and semi-automatically seal
pavement crack network using user-friendly
graphical control software
Development of greedy and optimal pathplanning algorithms
Nighttime operation using lighting
system mounted on the super-structure of
the machine
515Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
4/12
crack network to be traversed, as well as to maximize the efciency
(productivity) of the APCS, the developed optimal and greedy path
planning algorithms are intelligently and selectively used, based on the
complexity of crack networks to be traversed in a given workspace.
Table 2 briey summarizes the technical advances related to the control
paradigm and machine vision software made in previous research
works.
3.4. Evolution of hardware
In previous research works, the system architectures for the xy
table based pavement crack sealing consisted of multi-units (tow
truck, sealant melter, and crack sealer). The CMU laboratory proto-
type, CMU-UT eld prototype, the ARMM, and the APCS use multi-
units as shown in Table 1. XY-table manipulators with a turret
structure as an end-effector and multi-DOF manipulator arms were
employed as the main bodies of the crack sealing hardware. Table 2
compares the technical aspects of the previous automated pavement
crack sealing hardware.
The APCS is an extended version of the ARMM with innovative
improvements to both hardware and software. The hardware of the
APCS is composed of thefollowing three units:1) a towtruck, inwhich
a systemoperator controlsthe APCS using a PC;2) a sealant melter that
supplies 170180C sealant tothe APCS; and 3) a crack sealing unit for
blowing, sealing, and squeegeeing cracks in pavements. The major
components of the cracksealing unit of the APCS areas follows: 1)XY-
manipulator with cart, gantry, turret, and CCD camera, 2) control box,
3) industrial PC with a frame grabber and a touch sensitive monitor,
and 4) power supplies. In designing and fabricating the hardware of
the APCS, the following factors were considered:
(1) Designing anXY-table manipulator that does not twist against
the pavement level and various working situations;
(2) Adding rotating and telescoping functions to the end-effector
(turret structure) to effectively blow, seal, and squeegee the
crack network to be sealed in rutted pavement;
(3) Adding a heating function to theinside of the turret structure to
inject sealant with a certain degree of viscosity without rapid
solidication;
(4) Designing a turret structure to effectively squeegee the sealantinjected into the pavement cracks (turret with a squeezing
device having a Vor Ushape); and
(5) Mounting a lighting system for night work to minimize road-
user-costs and increase productivity.
Based on theidentied major considerations, theAPCS wasdesigned
and fabricated as shown in Fig.1. The APCS usesanXY-manipulator with
a rotating turret to blow, seal, and squeegee pavement cracks in one
sealing travel. While themanipulatoris movingwithin itsworkspace, its
frame is stationary. Sealing cracks in one work area and then moving to
the next work area is considered one work cycle. To control the APCS
through a work cycle, the following six steps are required:
(1) Acquire a pavement image including the crack network using a
digital CCD camera installed on the superstructure of the APCS
and a frame grabber board installed in the PC;
(2) Stop the APCS if any crack network is detected on the operator's
touch sensitive monitor;
(3) Automatically eliminate noises for mapping and modeling the
cracknetwork(This process is required only forthe case of fully
automated crack network mapping and modeling, to be
explained in Section 3.3);
Table 2
Comparison result of control paradigm, machine vision software, and crack sealing hardware developed in previous research works.
Control paradigm (1) (2) (3) (4)
Full automation
Teleoperation
Machine vision software (1) (2) (3) (4)
Vision Image acquisition Fully automated (vision sensor)
Fully manual (naked eyes)
Initial crack identication on video image Fully automated
Fully manual (naked eyes)
Crack mapping and representation Fully automated
Semi-automated (MMI)
Manual editing
Fully manual (naked eyes)
Graphical user interface
Path planning Fully automated Optimal
Greedy
Not considered
Hardware (1) (2) (3) (4)
Entire system architecture Single unitMulti-unit
Manipulator and end-effector XY-table manipulator with turret
Multi-DOF manipulator arm
Independent sealing apparatus
Functions equipped Blow
Seal
Squeegee
Telescope
Heat
Crack image collection Remote video
Laser range scanner
Human driver(sys. operator)
Lighting system for nighttime operation
Types of cracks sealed Transverse cracking
Longitudinal cracking
Block cracking
(1) CMU laboratory prototype, (2) CMU-UT eld prototype, (3) ARMM, (4) APCS.
516 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
5/12
(4) Extract the exact spine of the crack network using the
developed crack network mapping and modeling algorithm
(In the APCS, the system operator can map and model the crack
network to be sealed in a given work space by both automated
and semi-automated methods selectively);
(5) Automatically perform optimal path planning, based on the
results of mapping and modeling for the crack network; and
(6) Automatically blow air, inject sealant into the crack network,
and squeeze the sealant based on the calculated path plan.
The eld test results showed that the APCS was able to more
intelligently detect cracks through video images, and subsequently
automatically blow dust, inject sealant materials into cracks, and moreefciently squeegee the sealed cracks than the previous version (the
ARMM). Longitudinal, transverse, and block (random) cracks could be
effectively repaired by this machine. Although the current prototype
showed signicant improvements in hardware and software system
development, too much time was required to set up the whole system
including the tow truck, sealant melter, and crack sealing unit. Also, it
was difcult for the driver to precisely control the directional change
of three different units connected sequentially. For practical use under
a more controlled work environment, a new design is thus required to
reduce the preparation time and improve the mobility.Table 3briey
summarizes the advantages and disadvantages of hardware design
used for the APCS. This state-of-the art automated pavement crack
sealing research prototype is the most advanced research deliverable
in the form ofxytable manipulator.
4. Primary research ndings for practical application in the eld
It has taken approximately 20 years to achieve a balance between
human and machine functions for automated pavement crack sealing,
instead of pursuing an all-in-one full automation system. Over the
course of developing four physical crack sealing prototype systems
usingxytables (Table 1), a series of unique man-machine balanced
control loops for successful automation of crack sealing have been
developed for computer assisted teleoperation (Fig. 1). A complex
evolution in both software and hardware design has resulted in
functional production prototype system (APCS) that veries the
Fig. 1.Automated pavement crack sealer (APCS).
Table 3
Advantages and disadvantages in hardware design of the APCS.
APCS
Advantages
_ More effective design of end-effector (turret)
_ High accuracy for crack sealing
_ Cracks in pavement with rutting can be effectively sealed.
_ Potential productivity improvement and cost savings due to night
time operation
Disadvantages
_ Poor mobility: three units a tow truck, a sealant melter, and the
XY-table manipulator with turret
_ Only cover the area within theXYtable. It may require multi-paths
to cover one full lane, if the workspace ofXY-table is small.
_
Only used for technology feasibility demonstration
517Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
6/12
prevalence of the automated crack sealing method. The following are
lessons learned and primary research ndings in software and
hardware designs obtained from previous works over the last two
decades.
4.1. Software design requirement
(1) Image acquisition (full automation): As aforementioned, tele-
operation was recommended as a preferred control option forcrack sealing over complete automation. Quite often, the
teleoperation user is in a location that provides either poor or
no visual feedback. A remote camera is thereforegenerally used
to provide visual feedback. A security or infra-red camera can
be used for automatically acquiring live pavement images. The
images acquired by means of the remote camera are then
digitizedby a framegrabber, and theresultscan be displayed on
the monitor.
(2) Initial crack identication on video images (manual): Since
humans can instantly distinguish real cracks from pavement
background noise such as sealed cracks or oil or skid marks
under proper conditions of illumination, a human operator
should interact in the process of initial crack identication. As
shown in the test results of the most recent xy table based
pavement crack sealer (APCS), lessons learned also show that
initially identicationof a crackis betterperformed by a human
than by a computer in a very time-effective manner.
(3) Crack detection, mapping, and modeling (full or semi-automa-
tion): Automatically detecting, mapping, and modeling cracks in
pavement are desirable and technically feasible. However, there
is considerable concern about the accuracy and time component
of the processed results, because most pavement cracks are
highly noisy and unstructured in their digitized form at the pixel
level. Allowing the operator to point out the existence and
location of a pavement crack using a mouse or a stylus on a touch
sensitive monitor is economical and benecial. A graphical
program can be used to generate a computer-based model of
the surface cracks to be sealed and to provide visual feedback to
the operator. A good model can be achieved since the workenvironment of the automated pavement crack sealer is fairly
static. Despite such advantages, this approach has a drawback
that can signicantly degrade the quality in the resultant seal.
That is, the imperfection of human hand-eye coordination and
arm fatigue when tracing cracks on the monitor can cause errors,
even in a good work environment.
A good model for crack network detection, mapping, and represen-
tation was proposed for the APCS. In the APCS, crack network detection,
mapping, and representation are operated in two modes: full automa-
tion and semi-automation. A unique and innovative machine vision
algorithm has been developed for automation of pavement crack
sealing. Fully automated crack network detection, mapping, and
representation process successively includes: binarizing, noise elimina-tion, dilation, edge thinning and linking, and complete crack network
modeling, as shown inFig. 2. Sixty pavement crack images with shades
and various intensity of pavement surface were experimented to
measure the accuracy (86.6%) and efciency (0.46sec./image) of the
developed algorithm. Fig. 2 illustrates the semi-automated crack
network detection, mapping, and representation process of the APCS
in a very simplied form. Detailed descriptions of both fully and semi-
automated crack network mapping and modeling algorithms have been
presented elsewhere[7,8].
The test results showed that the semi-automated crack network
mapping andmodelingalgorithm wassuperior in terms of accuracyover
the fully automated crack network mapping and modeling algorithm,
but inferior in terms of productivity. However, both methods could
eventually guarantee 100% accuracy, because a manual editing function
witha rubber banding capability(Fig.2b)could beusedin boththe fully
automated and semi-automated methods. Complex and innovative
technical advances have been made in machine vision-based crack
network mapping and modeling algorithms compared to previous
researchefforts. Duringeldtrials, it wasfoundthat thesystem operator
usually uses the automated crack network and modeling method, as its
accuracy was sufcient for practical use. Furthermore, if there were any
errors in the extracted model, the resultant crack network model could
be easilyadjustedusing themanualediting function. Finally,the authorsestimate that the automated and semi-automated crack mapping and
modeling algorithms and processes developed for the APCS can be
directly applied to the automated pavement crack sealer to be newly
developed.
4.1.1. Path planning (full automation)
If there are several lines in the crack network extracted from a
pavement image, the number of paths is 2nn!. In the greedy path
planning algorithm, the nearest nodepoint from the home point among
the end node points of each crack network is the start node point for
sealing. The end-effector of the automated pavementcrack sealer injects
sealant along thecracknetworkuntilit reaches theotherend nodepoint
in the greedy path planning algorithm (Fig. 3a). Previous studies
proposed some greedy path planning algorithms for automated
pavement crack sealing. However, the paths planned by such greedy
algorithms often were not the optimal path.
An optimal path planning algorithm that can always provide the
shortest path in a given crack network was developed for the APCS. In
the optimal path planning algorithm, a computer calculates the lengths
of all paths so that the shortest path can be selected ( Fig. 3b). Ex-
perimentaltest results showed that the optimal pathplanning algorithm
is effective if the number of crack lines in a network is 6 or less, and the
greedy path planning algorithm is useful if the number of crack lines in
a network is more than 6, from a computational time perspective. Once
the algorithm automatically identies the number of crack lines in a
given crack network, the path planning method (optimal or greedy
path plan) is then selectively chosen and their xand ycoordinates in
2D workspace, which are ultimately required for control of the auto-
mated pavement crack sealer, are generated. Therefore, the pathplanning algorithm proposed for the APCS can be directly applied to
the automated pavement crack sealer to be newly developed as well.
4.1.2. Graphical user interface design for crack sealer control
In any man-machine system, an effective and user-friendly designof
the graphical user interface for controllinghardware is veryimportant in
terms ofnal acceptance in the market. Poor user interface design or
difcult system operation signicantly degrades the system perfor-
manceand productivity.Mostworkers in theeldare seldomexposedto
complicated computer control systems for road construction and
maintenance tasks and labor turnover rate is relatively high. As such,
ease of operation based on a user-friendly graphical interface design
would be a critical factorin attracting prospective end-usersas well as in
practically using the man-machine system in the eld. It is alsoestimated that overall efciency of the automated pavement crack
sealer will depend as much on the user-friendliness of software and
hardware controls as on the capabilitiesof themachine visionalgorithm
and crack sealing speed.
Findings in previous research efforts have indicated that the method
of clicking graphical menu buttons on a touch sensitive monitor is the
most appropriate design option for effective control of an automated
pavement cracksealer.The graphicaluser interfaceof theAPCS shown in
Fig. 4is a good example of this. Under such a user-friendly graphical
interface, the system operator should be able to easily understand the
man-machineinterfaced pavement crack sealing process and effectively
control the machine. The system operator can also observe the current
working status via the monitor in real time and instantly command
actions to be performed by clicking the menu buttons on the touch
518 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
7/12
-
8/11/2019 Articulo Pav
8/12
sensitive monitor. This user-friendly graphical interface design would
also signicantlyreduce possible problems related to labor turnover and
training of crack sealing crews. Therefore, it is anticipated that the
graphicaluser interfaceof theAPCS shown in Fig.4 can beadopted inthe
present form, or used as a reference in developing a new model of the
automated pavement crack sealer.
4.2. Hardware design requirements
4.2.1. Types of crack to be sealed
In general, crack patterns existing on pavements are classied into
the following four categories: 1) longitudinal, 2) transverse, 3) block,
and 4) alligator cracks. Longitudinal, transverse, and block cracks are
mainly repaired by sealing, while overlay or patching is used for
repairing alligator cracks. In order to appeal to private contractors and
government departments, as well as for practical use in the eld, it
must be guaranteed that the automated pavement crack sealer can
seal any type of longitudinal, transverse, and block cracks on normal
and/or rutted pavement surfaces.
4.2.2. Crack image collection
As aforementioned, the system operator of the automated pavement
crack sealer is in a location (e.g., the tow vehicle's cab) where visual
feedback of the workspace is not available. In this situation a typical
method for providing visual feedback is through the use of remote video
cameras. The video cameras provide several different views of the
equipment in its workspace, as well as the crack sealing status in real
time.This operatingenvironment provides the system operator with the
necessary visual feedback to control the automated pavement crack
sealer. The remote video signal is also processed by a computer using a
machine visionalgorithmto aidin thecompletionof theimage collection
task for crack identication. Single or multi-CCD (or infrared) camerascan be used based on the hardware architecture and the size of the
workspace designed. A series of lightingsystemsmust alsobe considered
for nighttime operation of the automated pavement crack sealer.
4.2.3. Entire crack sealing system architecture
Since a multi-unit crack sealer (e.g., APCS), which consists of a tow
truck, sealant melter, and crack sealer, would require excessive
mobilization and demobilization time, a single self-contained vehicle
[11,12]would be a better alternative for the entire system architecture
(appearance) of an automated pavement crack sealer. The reduction in
time to connect/disconnect the three independent units and to hook/
unhook all the cables for mobilization and demobilization would be
directly translated into signicant potential improvements in daily
productivity of the crack sealer.
Fig. 3.Comparison of greedy and optimal path plan results in the APCS.
520 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
9/12
4.2.4. Manipulator and end-effector
An automated pavement crack sealer to be newly developed needs
a minimum of 4 degrees of freedom to effectively move and seal along
thespineof cracknetworks.X-axis and Y-axis movements are required
to move in any direction on a 2D surface. Z-axis movement
(telescoping function) is also required to guarantee that cracks in
rutted pavement are sealed properly. In addition, the turret needs to
rotate to effectively blow, seal, and squeegee along the spine of the
crack network. Therefore, a motion control system is required to direct
the crack sealer along the X-, Y-, and Z-axes and to rotate the end-
effector of the automated pavement crack sealer. From previous
studies, ow ability of the melted sealant has been identied as an
important factor. To maintain the sealant in an appropriately viscous
state without any internal congestion in the hose, the length of the
Fig. 5.Software and hardware design requirements for enhancing the eld applicability.
Fig. 4.User-friendly graphical interface design of the APCS [7].
521Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
-
8/11/2019 Articulo Pav
10/12
Fig. 6.Conceptual hardware design of automated proposed pavement crack sealer.
-
8/11/2019 Articulo Pav
11/12
-
8/11/2019 Articulo Pav
12/12
[7] J.H. Lee, H.S. Yoo, Y.S. Kim, M.Y. Cho, J.B. Lee, The development of a machine visionassisted, teleoperated pavement crack sealer, Proc. of the 21st ISARC, Jeju, Korea,September 2004, pp. 149158.
[8] J.H.Lee, Y.S. Kim, J.B. Lee, M.H. Jeong,Developmentof an automatedpavement cracksealingmachine and its economic feasibility analysis, Korea Institute of ConstructionEngineering and Management(KICEM), Seoul, Korea 7 (6) (2006) 151164.
[9] Y. Kim, H. Yoo, J. Seo, Machine vision algorithm for automation of pavement cracksealing, Submitted to Journal of Computer-Aided Civil and InfrastructureEngineering on February (2007).
[10] Y.Kim, C. Haas, R. Greer,Path planning for a machine vision assisted, tele-operatedpavement crack sealer, ASCE Journal ofTransportation Engineering 124(2) (1998)137143.
[11] S.A. Velinsky, Fabrication and testing of an automated crack sealing machine,SHRP-H-659, National Research Council, Washington, D.C., 1993.
[12] S. Velinsky, B. Rabani, Longitudinal crack sealing and random crack sealing inintroduction to advanced highway maintenance and construction technology onhttp://www.ahmct.ucdavis.edu(2008).
524 Y.S. Kim et al. / Automation in Construction 18 (2009) 513524
top related