the ohio state university center for mapping charles k. toth 1 & dorota a. grejner-brzezinska 2...

The Ohio State University Center for Mapping

Charles K. TothCharles K. Toth1 1 & Dorota A. Grejner-Brzezinska& Dorota A. Grejner-Brzezinska22

Center for MappingCenter for Mapping11

andandDepartment of Civil and Environmental Engineering and Geodetic ScienceDepartment of Civil and Environmental Engineering and Geodetic Science22

The Ohio State UniversityThe Ohio State University

e-mail: [email protected]://www.cfm.ohio-state.edu/

CLEM 2001 CLEM 2001

Specialist Meeting onSpecialist Meeting on

Centerline Extraction and MaintenanceCenterline Extraction and Maintenance

August 6-7, 2001August 6-7, 2001

Santa BarbaraSanta Barbara

OSU

Integrated GPS/INS/CCD System for Integrated GPS/INS/CCD System for High-Precision Centerline ExtractionHigh-Precision Centerline ExtractionIntegrated GPS/INS/CCD System for Integrated GPS/INS/CCD System for High-Precision Centerline ExtractionHigh-Precision Centerline Extraction


National Consortium for Remote Sensing in Transportation - Flows

Develop, demonstrate, and disseminate cost-effective remote sensing techniques for application to transportation flows

NCRST - FNCRST - FNCRST - FNCRST - F


• Traffic Monitoring

• Traffic Management

• Freight and Intermodal Analysis

From Satellite or Airborne Platforms

NCRST - F Application AreasNCRST - F Application AreasNCRST - F Application AreasNCRST - F Application Areas


• Multi-sensor and multi-platform fusion for flow acquisition and interpretation

• Findings to date– Positioning quality at flight altitude: 2-5cm per

coordinate– Positioning quality on the ground: 20-30cm

from 300m altitude– Strong dependence on system calibration

Progress to DateProgress to DateProgress to DateProgress to Date


Tower ImagesTower ImagesTower ImagesTower Images


Presentation Outline Presentation Outline Presentation Outline Presentation Outline

Problem description – background

Solution/System design

Hardware configuration

Prototype imaging software suite

Performance experiences

Imaging component

(GPS/IMU positioning system)

Summary


cable location

Precision Centerline Mapping Precision Centerline Mapping Precision Centerline Mapping Precision Centerline Mapping


Mapping road lane markers at decimeter-level accuracy while maintaining traffic flow and ensuring safety of survey personnel

Finding the connection between the high-accuracy vehicle navigation and the road surface

Mechanical solutions (early system concept) Accident hazard Strong dependence on driver

Non-contact measurements

Problem Description Problem Description Problem Description Problem Description


Multisensor data fusionHigh speed image acquisition

2D/3D image processing (photogrammetry) Computer vision Real-time processing

Tightly-coupled GPS/INS system Post-processing Real-time relative motion support for the real-time image

processing system

Proposed SolutionProposed SolutionProposed SolutionProposed Solution


Digital Camera

Y

H

ZINS

-YINS

GPS Antenna

INS

Mapping ConceptMapping ConceptMapping ConceptMapping Concept


System Design ConceptSystem Design ConceptSystem Design ConceptSystem Design Concept

Image

Acquisition

Control and

Storage

Image

Acquisition

Control and

Storage

StrapdownNavigation

Solution

TightlyCoupledGPS/INS

Kalman Filter

StrapdownNavigation

Solution

TightlyCoupledGPS/INS

Kalman Filter

User

Interface,

Control &

Display Unit

User

Interface,

Control &

Display Unit

PulnixTMC 6700

Digital Color

Camera

PulnixTMC 6700

Digital Color

Camera

RoverGPS Station

RoverGPS Station

LN-100LN-100

BaseGPS Station

BaseGPS Station

L1 and L2 phase observable

Delta V

Delta

L1 and

L2

Image Data

ExposureControl

Time Tag/Sync

EO Data

Host/SlaveCommunication

ControlSignal

OptimalPosition,Velocity,AttitudeEstimates

Real-timeImage

Processing Block

Real-timeImage

Processing Block

Image Post-processing

Block

Image Post-processing

Block


Digital frame camera used in tests presented here based on Lockheed Martin Failrchild 4K 4K CCD

sensor 60mm by 60mm imaging area (15-micron pixel size) Hasselblad 553 ELX camera body with 50 mm Zeiss

lens 6 second image acquisition rate (0.16 Hz)

Target sensor Pulnix color digital camera (TMC 6700) 9 mm by 6 mm imaging area 30 images per second max acquisition rate (30 Hz)

Imaging ComponentImaging ComponentImaging ComponentImaging Component


RGB to STransformation

Median Filter Binary Conversion

Boundary PointsCenterline Extraction

RGB to STransformation

Median Filter Binary Conversion

Boundary PointsCenterline Extraction

Feature Point Extraction

Feature Point Matching

Affine Model Formation

Centerline Strip Formation

Image n

Centerline PositionRefinement

Export to GIS/CAD

Post-processing

Stereo Image Processing

Data Acquisition

ImagePreprocessing

Feature Point Extraction

Image n-1

Final Navigation Data

Relative Motion Estimates

Image Processing ConceptImage Processing ConceptImage Processing ConceptImage Processing Concept


Typical Road/Line PatternsTypical Road/Line PatternsTypical Road/Line PatternsTypical Road/Line Patterns


B/W Image of Low-quality Road SurfaceB/W Image of Low-quality Road SurfaceB/W Image of Low-quality Road SurfaceB/W Image of Low-quality Road Surface


Sobel Edge OperatorSobel Edge OperatorSobel Edge OperatorSobel Edge Operator


Color SeparationColor SeparationColor SeparationColor Separation

RGB to IHS transformation


Color TransformationColor TransformationColor TransformationColor Transformation


Scanline

Centerline

Intersection

noise

Centerline Extraction (1)Centerline Extraction (1)Centerline Extraction (1)Centerline Extraction (1)

SL

RL

CL

P


Centerline Extraction (2)Centerline Extraction (2)Centerline Extraction (2)Centerline Extraction (2)

Monoscopic solution moderate accuracy

High accuracy solution 3D processing


Feature Point ExtractionFeature Point ExtractionFeature Point ExtractionFeature Point Extraction

R(x, y) = det[C] – k trace2[C]

^

2^

^^2

yyx

yxx

III

IIIC

Corner detector:


Feature Point MatchingFeature Point MatchingFeature Point MatchingFeature Point Matching

s1

s2

s3

m2m1

1

m21

m13

m23

n

i

m

j

vuIjviuIvuIjviuImsscore1 1

22222211111111 ]),(),([]),(),([),(

)()(),:,()()(

1

iji

ik

kn

l

klk

klk

iji

iji mspmspmsmscmsp


Strip FormedStrip FormedStrip FormedStrip Formed

Pairwise affine transformation (6-parameter model)


StatusStatusStatusStatus

Prototype system implemented in C++

Experiences Two sets of images (~15-20) with various

road/centerline conditions Computation-intensive algorithms with

convincing performance (post-processing)

Remaining tasks Real-time data acquisition (camera interface

and navigation data from GPS/IMU system) Software reengineering (multithreading) Performance tuning (dual P-IV @ 1.7 GHz)


System Calibration ComponentsSystem Calibration ComponentsSystem Calibration ComponentsSystem Calibration Components

GPS

CameraCalibration

Lever ArmCalibration

INS OTFCalibration

GPSBase

INS

Sensor Mount

CCD

BoresightMisalignment


Digital camera

GPS antenna

LN 100

Test Hardware ConfigurationTest Hardware ConfigurationTest Hardware ConfigurationTest Hardware Configuration


Performance Analysis TestsPerformance Analysis TestsPerformance Analysis TestsPerformance Analysis Tests

Single side-looking camera, tilted downwards by 5°, 4K by 4K B/W imagery

50-mm focal length

Imagery collected along the surveyed road (edge/center line location)

Stereo-pairs formed by subsequent images

7-8 m object distance for boresighting

8-20 m object distance for ground control points (check points)


Vehicle TrajectoryVehicle TrajectoryVehicle TrajectoryVehicle Trajectory

-83.0418 -83.0416 -83.0414 -83.0412 -83.041 -83.0408 -83.040639.997

39.9975

39.998

39.9985

39.999

39.9995

40

longitude [deg]

latit

ude

[deg]

Start

-83.052 -83.05 -83.048 -83.046 -83.044 -83.042 -83.0439.997

39.998

39.999

40

40.001

40.002

40.003

longitude [deg]

latit

ude

[d

eg]

Start/end

Calibration range

Centerline test range


Positioning AccuracyPositioning AccuracyPositioning AccuracyPositioning Accuracy

4.165 4.17 4.175 4.18 4.185 4.19 4.195 4.2 4.205x 105

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

GPS time [sec]

RM

S [

m]


Attitude AccuracyAttitude AccuracyAttitude AccuracyAttitude Accuracy

4.165 4.17 4.175 4.18 4.185 4.19 4.195 4.2 4.205

x 105

0

5

10

15

20

25

30

35

GPS time [sec]

RM

S [

arc

sec]


Boresight Calibration Test RangeBoresight Calibration Test RangeBoresight Calibration Test RangeBoresight Calibration Test Range


Boresight Misalignment PerformanceBoresight Misalignment PerformanceBoresight Misalignment PerformanceBoresight Misalignment Performance

PointID

E [m] N [m] H [m] E residual[m]

N residual[m]

H residual[m]

14 553789.008 221908.411 212.473 -0.003 -0.001 0.00215 553789.426 221909.427 212.448 -0.006 0.002 0.00024 553788.985 221908.418 211.366 0.010 0.000 -0.00225 553789.407 221909.438 211.356 0.009 -0.002 -0.00134 553788.974 221908.415 210.272 -0.006 0.002 0.00135 553789.390 221909.437 210.248 -0.004 0.000 0.000

0.007 0.001

Boresight performed by comparison of GPS/INS results with AT solution

Accuracy of the boresight components

1-2 cm for offsets

17-22 arcsec for attitude


Centerline Test RangeCenterline Test RangeCenterline Test RangeCenterline Test Range


Ground coordinate difference for check points measured from different stereo-pairs

Statistic Easting [m] Northing [m] Height [m] Mean 0.015 0.004 0.008

Median 0.006 0.003 0.006 Maximum 0.050 0.025 0.035

RMS 0.019 0.007 0.010 Ground coordinate difference for check points measured

on stereo-pairs from different passes

Statistic Easting [m] Northing [m] Height [m] Mean 0.015 0.014 0.044

Median 0.013 0.011 0.045 Maximum 0.050 0.034 0.130

RMS 0.020 0.018 0.052

Testing Positioning PerformanceTesting Positioning PerformanceTesting Positioning PerformanceTesting Positioning Performance


GPS Losses of LockGPS Losses of LockGPS Losses of LockGPS Losses of Lock


Units Gap 1 Gap 2 Gap 3 Gap 4East [m] 0.19 0.12 19.0 131.0North [m] 0.40 0.06 33.4 209.2Total errorHeight [m] 0.58 0.18 7.2 162.4

Gap duration [sec] 84 140 320 1200INS calibration [sec] 236 723 860 1050

Positional Error Growth Positional Error Growth during GPS Outageduring GPS Outage

Positional Error Growth Positional Error Growth during GPS Outageduring GPS Outage


Summary Summary Summary Summary

Multisensor all-digital mapping systemReal-time image processing needed

Automatic extraction of centerlines is a reality Real-time implementation is feasible

Direct orientation can be achieved with high accuracy Strong GPS-dependency in urban areas INS support needed (attitude less critical) Proper system calibration is crucial


LIDAR/Digital Image DataCourtesy of

The EarthData Group

Baltimore, MDBaltimore, MDBaltimore, MDBaltimore, MD


Panchromatic Image and LIDAR Panchromatic Image and LIDAR Elevations Plotted as IntensityElevations Plotted as Intensity

Panchromatic Image and LIDAR Panchromatic Image and LIDAR Elevations Plotted as IntensityElevations Plotted as Intensity


+ + + + + + + +

+ + + + + + +

+ + + + + + + +

+ + + + + + + +


Image 1 pixel tessellation

Image 2 pixel tessellation

LIDAR footprint

Image/LIDAR Sampling PatternImage/LIDAR Sampling PatternImage/LIDAR Sampling PatternImage/LIDAR Sampling Pattern

0.4-0.8 1.064


LIDAR Spots Overlaid in the Nadir AreaLIDAR Spots Overlaid in the Nadir Area LIDAR Spots Overlaid in the Nadir AreaLIDAR Spots Overlaid in the Nadir Area


Misalignment of LIDAR Spots and ImageryMisalignment of LIDAR Spots and Imagery Misalignment of LIDAR Spots and ImageryMisalignment of LIDAR Spots and Imagery


What Could Be the Problem?What Could Be the Problem?What Could Be the Problem?What Could Be the Problem?

Sensor modeling of the camera (interior orientation)

LIDAR sensor modeling (scan angle error)

Digital camera boresight misalignment

LIDAR boresight misalignment

Varying navigation performance

Mechanical flex of the sensor mount

Etc.

the ohio state university center for mapping charles k. toth 1 & dorota a. grejner-brzezinska 2...

Documents

ohio state university

ohio state university

mapping charles

mapping tower images

date slide

mapping national consortium

flow acquisition

system calibration progress