ssrr2015 lbr sarmiento duncan murphy

Preliminary Analysis of Reconstructions from Aerial Images of DisasterProps

Traci A. Sarmiento,1 Brittany A. Duncan,2 and Robin R. Murphy3

Abstract— This paper provides a preliminary analysis ofreconstructions of seven disaster props from datasets collectedover five flights with a small unmanned aerial vehicle andfinds 181 anomalies present in four categories. Commercialand free software are available to create reconstructions fromdigital images, but their accuracy and limitations have not beenformally explored for disaster assessment. This work analyzesthe output of two packages, Agisoft Photoscan, and MicrosoftICE for five image datasets of seven Disaster City R©props.The 181 anomalous results are divided into four categories:ghosting, misalignment, misproportions, and the Dali effect.These metrics are valuable to Urban Search and Rescueteams, structural specialists, and insurance claims adjusterswho need to reliably assess whether it is safe to enter a building,or accurately estimate the severity of damage. The analysissuggests the reconstructions might be misleading and furtherwork must be done to determine why and suggest methods ofimprovement.

I. INTRODUCTION

Each of the 28 FEMA Urban Search and Rescue (US&R)Task Forces are required to conduct physical search and res-cue operations in damaged, or collapsed structures, providestructural and hazard evaluations of buildings needed forimmmediate occupancy to support disaster relief operations,and assess damage and needs to provide feedback to local,state, and federal officials among other responsibilities [1].The performance of these responsibilities could be enhancedthrough the correct application of unmanned aerial systemstechnologies combined with digital image processing tech-niques however, the following analysis shows inaccuraciesin each of the fifteen reconstructions completed across fivedatasets with two different software packages.

II. RELATED WORK

Still digital images and videos taken by small unmannedaerial vehicles (UAV) can be reconstructed into panoramasthrough stitching, or orthomosaics through photogrammetrytechniques. In [2], the authors noted problems with misalign-ment, ghosting, and exposure differences when creating 2Dpanoramic reconstructions from still images. They showedhaving the ability to overcome these errors to create fast andaccurate reconstructions in 2D for disaster scenarios, likeflooding, is important. Using photogrammetric techniques,

1Traci A. Sarmiento is a Phd Student in the Department ofComputer Science and Engineering at Texas A&M Universitytasarmiento@tamu.edu

2Brittany A. Duncan is an Assistant Professor with the Departmentof Computer Science and Engineering at University of Nebraska-Lincolnbduncan@cse.unl.edu

3Robin R. Murphy is Director, Center for Robot-Assisted Search andRescue at Texas A&M University robin.r.murphy@tamu.edu

Irschara created reconstructions from UAV imagery, butnoted reprojection errors in both applications tested [3].

III. IMAGE DATASETS

The Texas A&M Engineering Extension Service maintainsa complex of props desgined to represent physical conditionsexperienced by US&R teams in actual disasters. Seven propsat the complex, known as Disaster City R©, were flown overa total of five flights. The digital images were captured bythe AirRobot AR180 quadrotor using a 14 megapixel Sigmacamera during pre-planned, autonomous missions.

IV. ANALYSIS

Two software packages were utilized to create the re-constructions for analysis. The first was Agisoft PhotoScan,which is a commercial software product, available for pur-chase, that creates orthomosaics through photogrammetricprocessing of digital images. The second was Microsft ImageComposite Editor (ICE), which is a free software packagethat creates panoramas by stitching sets of overlappingdigital images. Three reconstructions, identified in Figure1 as AGISOFT–No EXIF, AGISOFT–EXIF, and ICE, werecreated for each of the five datasets.

A. Method and Metrics

To identify inconsistencies in the reconstructions, the threeauthors separately viewed the three reconstructions for eachof the datasets. The reconstructions were marked when oneof four types of anomalies was found: ghosting, misalign-ment, misproportions, and Dali effect. Different colors wereused to help identify the effects. Green was used for ghosting,red for misalignment, blue for misproportions, and yellow forthe Dali effect. These marked images were compared and thenumber of consensus anomalies for each category pertainingto each reconstruction are shown in Figure 1.

Ghosting, where an object, or portion of an object, appearsblurry in the image, is seen in Figure 2, the AGISOFT-NoEXIF reconstruction of Flight 2. Misalignment, from theMicrosoft ICE reconstruction of Flight 2, appears in Figure 3because the digital images are not matched correctly causingouter edges of objects to appear jagged, or disjointed whenthey should be aligned. Misproportioning in a reconstructionoccurs when an object in the scene does not reflect its truesize, as seen in Figure 4 from the Microsoft ICE stitch ofFlight 2. Finally, the Dali effect is evident when swirlingand melting is seen in a reconstruction where there shouldbe clear lines. This is shown in Figure 5 from the AGISOFT–EXIF reconstruction of Flight 3.

Fig. 1. Number of consensus anomalies identified in each of four categories for the three types of reconstructions over five datasets.

Fig. 2. An example of the anomaly described as ghosting from theAGISOFT-No EXIF stitch of Flight 2

Fig. 3. An example of the anomaly described as misalignment from theMicrosoft ICE stitch of Flight 2

Fig. 4. An example of the anomaly described as misproportion from theMicrosoft ICE stitch of Flight 2

Fig. 5. An example of the anomaly described as the Dali effect from theAGISOFT–EXIF reconstruction of Flight 3

Fig. 6. An example of a reconstruction with all four anomalies identifiedfrom AGISOFT-No EXIF Flight 5

V. CONCLUSIONS AND FUTURE WORKAs shown in Figure 1, the reconstructions created from

the five image datasets collected by an UAV do not alwaysaccurately represent the structures surveyed. Figure 6 is anorthomosaic from AGISOFT–No EXIF of Flight 5 where allfour types of anomalies are present with two ghostings, tenmisalignments, two misproportions, and seven Dali effects.Within the fifteen reconstructions created there was a con-sensus of 181 anomalies seen across four categories. Theghosting, misalignment, misproportioning, and Dali effectmight be caused by insufficient overlap of the still images,lighting effects due to the time of day, weather conditionslike wind, the angle of image capture, the altitude flown,or any combination of these. Determining the cause of theanomolies and providing input on improvements to decrease,or eliminate the issues will require more in-depth analysis ofthe data and future work with more controlled studies.

ACKNOWLEDGMENTThe authors would like to thank the Texas A&M Engi-

neering Extension Service. This work was supported in partby a NSF Graduate Research Fellowship.

REFERENCES

[1] http://www.fema.gov/urban-search-rescue-participants[2] Li, Ming, Li, Deren, and Fan, Dengke, ”A Study on Automatic UAV

Imge Mosaic Method for Paroxysmal Distater ”, International Archivesof the Photogrammetry, Remote Sensing, and Information Sciences,Volume XXXIX-B6, 2012.

[3] Irschara, A., Kaufmann, V., Klopschitz, M., Bischof, H., and Leferl,F. ”Towards Fully Automatic Photogrammetric Reconstruction UsingDigital Images Taken from UAVs”, ISPRS TC VII Symposium - 100Years ISPRS, Vienna, Austria, Vol XXXVIII-7A, 2010.