Washington D.C. Dataset:

GIS Building OutlinesGeo-tagged images from Panoramio

CRCV

Acknowledgments: This work was completed as part of the REU (Research Experience for Undergraduates) at The University of Central Florida, under the supervision of Dr. Mubarak Shah and Dr. Neils de Vitoria Lobo.• Entropy Rate Superpixel Segmentation Ming-Yu Liu, Oncel Tuzel, Srikumar Ramalingam, and Rama Chellappa

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'11) , Colorado Spring, Colorado, June 2011.

• Shervin Ardeshir, Amir Roshan Zamir, Alejandro Torroella, Mubarak Shah. GIS-Assisted Object Detection and Geospatial Localization. ECCV 2014

• Barinova O, Lempitsky V, Tretiak E, Kohli P. Geometric Image Parsing in Man‐Made Environments. European Conference on Computer Vision, 2010

Segmentation and GIS Projection Fusion

Problem & Motivation:

Use GIS information to extract semantically meaningful segments from the image.

Automatically annotate image segments with their addresses and absolute geo-locations.

Applications in mobile navigation such as Google Glass

GIS Projections:

Results:

Building Addresses and Street Names Google Maps API Street Names:

A grid of points is generated covering the area not covered by the buildings.

The street names associated to each point on the grid is extracted using Maps API

Building Addresses:

The Google Maps API is used to convert GIS information into street names and addresses

Building addresses as well as street names are shown overlaid here

Geo-Location Aware Semantic Segmentation

Malcolm Collins-Sibleycollins-sibley.k@husky.neu.edu

Northeastern University

Shervin ArdeshirShervin@knights.ucf.edu

University of Central Florida

Initial Super-pixel Segmentation

Single Segment Optimization:

Building Segment after ThresholdingInitial Super-Pixel Scores Final Scores after Markov Iterations

Multi-Segment Joint Optimization:

Street Segment Building SegmentBuilding & Street Segments

Initial Super-Pixel Segmentation (Randomly Colored)

How to find initial segment scores?

Example image post segmentation

Example binary map

The score for each super-pixel is defined by the percentage of it’s pixels which are covered by the binary map

= Set of pixels within super-pixeli

= super-pixeli covered by the binary map

Waterbury CT Dataset:

Geo-tagged images from Panoramio GIS Building Outlines

Framework Outline:

Street Grid Points After name classification

Street Grid points after occlusion handling

GIS Map Building Projection Binary Map of Building

Input Image

Alignment using

Horizon

Markov Chain Propagation

Building & Street

Binary Maps & Addresses

Super-pixel Segmentation

GIS Projections and Occlusion

Handling

Forcing each super-pixel to belong to a projection