1

A Whole New World of Mapping and Sensing: Uses from Asset to Management to In‐‐‐‐Vehicle

Sensing for Collision Avoidance

Charles Toth, Dorota A Grejner-Brzezinska, Carla Bailo and Joanna Pinkerton

Satellite Positioning and Inertial Navigation (SPIN) Laboratory

Department of Civil, Environmental and Geodetic Engineering

The Ohio State University

Email: toth.2@osu.edu

2017 OTEC Symposium Columbus

October 10-11, 2017

2

� As vehicle technology is moving towards higher autonomy, the

demand for highly accurate map data is rapidly increasing, as

accurate maps have a huge potential to increasing safety. In

particular, engineering scale 3D maps, including road topography

and infrastructure as well as city models along the transportation

corridor represent tremendous support for driverless vehicles.

� The quality of the 3D data, measured in accuracy and currency

must be clearly far superior to any conventional map data,

traditionally provided by federal and local governments, and also

significantly richer in information than the typical 2D map data used

in car navigation systems.

� In a Smart City environment, integrated data exchange can

provide the link between the geospatial/GIS database and the

vehicles (V2I/V2X). Access to this database, which includes high definition 3D maps and the corresponding metadata, is essential

for autonomous vehicles as it facilitates the sensor systems to

accurately relate the vehicle’s location/trajectory to the surrounding

environment in any situation.

Driverless Vehicle Technology

3

Driverless Vehicle Technology

� Driverless technology is rapidly evolving

� High-definition geospatial/GIS data is an enabling component to

improve localization and, subsequently, safety

� Huge amount of GIS data is already available, the question is how

to access it, and then the communication

� Crowdsourcing will be the dominant data acquisition technology

(Big Data, Big Geo Data)

� Full autonomy is still a long way

4

� Driving by human beings is found to be dangerous and has

led to countless deaths over the years. Worldwide, per the

Global Road Crash Data [1], traffic crashes are the major

cause of death and injuries, specifically estimated at 1.3 million fatalities each year, on average 3,287 deaths per day.

� In the United States, there are over 37,000 deaths and an

additional 2.35 million injuries in road crashes each year. Of

these, 94% are caused by human error [4], reported by

USA’s National Highway Traffic Safety Administration

(NHTSA) research.

� The cost of traffic crashes is incredibly high, reaching USD $518 billion globally and $230.6 billion in United States.

Unless action is taken, traffic crashes are predicted to be the

fifth leading cause of death by 2030.

Motivation

5

� Most of the accident happen close to our homes (urban areas)

� An average American driver spends nearly 300 hours on road each year

� Traffic congestion and parking are painful

Picture credit: pixabay

Traffic in Cities

6

Smart Columbus

7

Olli at CAR

8

Dynamic object detection on road

� Object detection is the process of finding instances of real-world

objects such as vehicles, bicycles in images or videos.

� Traditionally, Support Vector Machine (SVM) combined with the

Histogram of Orientation (HOG) features have become the most

efficient algorithm

Collision Avoidance

9

Tracking

10

Tracking

11

OSU GPSVan

� General sensor platform with highly accurate georeferencing system

� Data acquisition capabilities to support� Creating accurate high-definition maps� Provide sensor data stream to support

driverless vehicle technology research

12

Sensors and FOV

13

Navigating without Maps

OSU data, SPIN Lab CDD/IMU/SLAM solution

14

Various Viewpoints

Many people involved in driverless technologies approach the

problem from different angles

� Navigation/positioning people focus on GPS/GNSS trying

to push the envelop; accuracy matters but confidence in the

data is more important

� Sensor manufacturers are divided by technologies

� Optical sensing

� Laserscanning

� Radar

� Navigation and mapping fields are converging

� Car manufacturers are learning, just recognizing the

importance of geospatial/GIS data

� Many companies are already acquiring high-definition maps, hoping to sell it manufacturers

15

Localization accuracy required by autonomous driving

� High accuracy: 3-10 cm

� Single frequency GPS is not enough accurate (2-5 m, no GPS)

� More complex GPS processing requires communication and

special infrastructure (RTK, GNSS)

� “Urban canyon effect” is still a problem

� IMU has large drift error and navigation-grade IMUs are

expensive

Solution:

� Map matching algorithms provide reliable and accurate

localization solution

� Map matching requires precise map a prior the drive

Vehicle Localization

16

Navigating with Maps

KITTI data, widely used benchmark, SPIN Lab CDD/IMU/SLAM solution

17

How to obtain precise map?

� Mobile mapping (e.g. Google Street View; not accurate)

� First, the autonomous vehicle has to be driven manually

along the route to create a map (SLAM); e.g., shuttles

Crowdsourcing:

Vehicles/platforms share information (vehicle to vehicle, V2V

distributed, or vehicle to infrastructure, V2I, centralized),

including

� Precise location information

� Redundant video and other sensor streams (bandwidth is

an issue)

� Detailed representation of the scene (requires the

processing of the sensor streams)

High Definition Maps

18

What is Mapped?

iPhone

Android

19

MobileEye’s Road Experience Management (REM)

The solution is comprised of three layers: harvesting agents (any camera-equipped vehicle), map aggregating server (cloud), and map-consuming agents (autonomous vehicle):

� The harvesting agents collect and transmit data about the driving path’s geometry and stationary landmarks around it

� The cloud server aggregates and reconciles the continuous stream of “Road Segment Data” – a process resulting in a highly accurate and low level “Time to Reflect Reality” map, called “Roadbook” (MobilEye GIS)

� The last link in the mapping chain is localization, in order for any map to be used by an autonomous vehicle, the vehicle must be able to localize itself within it, the Mobileye software running within the map-consuming agent (the autonomous vehicle) automatically localizes the vehicle within the Roadbook by real-time detection of all landmarks stored in it

MobileEye Implementation

20

Power of crowdsourcing: Building Rome in a day

� Researchers create 3D model of cities using images shared on Flickr

(2009)

� Website: https://grail.cs.washington.edu/rome

� Videos:

https://www.youtube.com/watch?v=qYaU1GeEiR8&list=PLDFDB5B8C

80DB3AD6

City Model Demonstration

21

Pilot Project Objectives

Sensing Data for Autonomous Mobility: OSU Campus Benchmark Dataset

Collection to Aid Autonomous Vehicle Testing in Smart Columbus

� The main objective is to acquire data streams from mobile platforms,

including vehicles, bicycles, pedestrians, etc. These data are essential

to testing vehicle sensing and maneuvering capabilities, and directly

support research and development of autonomous vehicle

technologies.

� A second objective is to create a high definition map of the test area

that includes the mobile data collection and additional surveying of the

area. The availability of such maps has a significant effect on

autonomous driving by providing a detailed description of the object

space of and around the transportation corridor, greatly improving the

reliability of the vehicle’s self‐localization and path planning capabilities.

22

High-precision Mapping – OSU Sensors, Data

23

High-precision Mapping Capabilities

Data acquisition components:

� Local reference GPS network (densified, if needed, by the OSU team)

� GPSVan, mobile mapping platform to collect data

� High-accuracy georeferencing system (permanently installed)

� GPS/GNSS data for platform positioning

� IMU data for platform positioning and orientation data

� Imaging sensors (configuration depends on project objectives)

� Various digital cameras; video and still imagery from various

perspectives

� Mobile laserscanner data to generate dense point clouds

Phase 1 data processing steps:

� Static georeferencing data processing

� Vehicle trajectory computation

� LiDAR and image point cloud processing

24

Sensor Configuration

Sensor# of

sensorsFOV [°°°°] Sampling rate Data rate

Storage

Gbyte/hour

Velodyne VLP-16 4 + 4 360 x 30 20 Hz 2 MB/s 7.2

Velodyne HDL-32 1 360 x 40 20 Hz 4 MB/s 14.4

GoPro 3 45-180 4k @ 30 Hz 60 MB/s 216

Nikon 1 45/65 1 Hz 15 MB/s 65

Point Grey 4 45 15 Hz 120 MB/s 400

Sony Nex7 2 45 0.5 Hz 48 MB/s 172

Canon 2 45 Video mode 15 MB/s 54

Casio 2 45 Video mode 15 MB/s 54

MobileEye 1

MicroStrain IMU 1 N/A 200 Hz 100 kB/s 0.36

EPSON IMU 1 N/A 300 Hz 150 kB/s 0.54

H764G 2 N/A 256 Hz 200 MB/h 0.20

GPS 2 N/A 5Hz 0.02

Total: ~ 1 TB

25

Observation Space, Sensor FOV

26

Schedule

1. System configuration

� Sensor selection

� FOV optimization

� Sampling rate

2. System shake-up

� Laboratory testing

� CAR/road testing

3. System calibration, sensor orientation with respect to vehicle

reference frame

� Target site construction at CAR (point and planar controls)

� Calibration measurements

� Processing, estimating calibration

4. Main data collections

1. Set 1, data collection around CAR

2. Set 2, data collection on Campus

3. Set 3, data collection including reference moving objects, including

bicycles and pedestrians, equipped with GPS/IMU/UWB positioning

and imaging sensors

27

SUMMER DATA ACQUISITION

28

Laser Data

29

Multiple Vehicle Configuration

30

Summary

� Driverless vehicle technologies greatly benefit from high

resolution city model level geospatial data

� Mapping, data acquisition methods for transportation GIS

systems are rapidly changing

� Need for high-definition, and highly accurate geospatial data

is quickly growing

� Crowdsourcing is becoming mainstream; vehicles are

becoming the prime source of geospatial data

� Communication is essential; Smart City environment provides

the larger framework, integrated data exchange can provide

the link between the geospatial/GIS database and the

vehicles (V2I/V2X)