November 1, 2009 1

Presented by,Pradeep Thomas ThundiyilM.Tech 1st Sem DCN

Contents Objective Introduction Concept System Overview Working Feasibility Analysis Limitations System Extensions Conclusion References

November 1, 2009 2

Objective

To help individuals with Cognitive Impairments navigate in Indoor Spaces.

Accessing directory information related to the current location.

November 1, 2009 3

Cognitive Impairment ?

Concentration Difficulty Learning Disability Hallucinations Forgetfulness

November 1, 2009 4

Introduction The act of pointing the camera forms the query into

information about the building and sends them to building server.

The server processes the cell phone camera image and matches detected landmarks from the image to a building.

The system calculates camera location and

dynamically overlays information directly on the cell phone image.

November 1, 2009 5

November 1, 2009 6

Concept

Determining landmarks in the cell phone image and matching them to previously cached landmarks in the environmental space.

By matching enough landmarks the camera pose can be precisely computed and thereby accurately overlay information onto the display.

November 1, 2009 7

System Overview

Image processing System: Locates landmarks and compares them to the buildings floor plan.

Building Server: Holds floor plan data and provides the computation cycles for extracting the landmarks and performs the matching.

WI-FI Connection: Client and Server communicates through a WI-FI connection.

November 1, 2009 8

Working

Extracts the relevant features in the image. Uses the location estimate to select the

search area in the floor plan. Finds the correspondence between the

image features and the building’s floor plan. Computes the Camera pose. Returns information overlay for display on

the phone client. November 1, 2009 9

Working

Steps

System Diagram

System diagram for calculating camera pose and overlayinginformation on a camera-phone image. Three input sources generatethe augmented image in a five-step process.

November 1, 2009 10

Implementation

November 1, 2009 11

Feature Detection

Locating the features in the camera phone image.

Mean Shift Method for segmenting the image.

System traces edge of the segment and identifies the corners using a “Cornerity Metric”.

These points are ready to be matched with the floor plan.

November 1, 2009 12

The mean shift method is a popular method for a wide variety of applications: video tracking, image filtering, clustering and image segmentation.

November 1, 2009 13

Mean Shift Method

A geometrical feature detection system which is to be used for data acquisition and pre-processing, segmentation and landmark extraction and characterisation.

Cornerity Metric

Segmentation & Corner finding results

November 1, 2009 14

a). System first segments

the hallway image, then traces the floor’s edge and locates the corners. Corners marked with yellow circles are candidates for matching to floor plan features.

b). The X marks the cell phone’s estimated location on the floor plan and the red dots marks the feature points.

Feature Matching

Image features must be matched to the floor plan features.

A potential set of points in the floor plan are chosen to match.

Location System provides a rough location estimate that becomes the centre for the region to be tested.

RANSAC approach is used to determine correspondence from the two sets of features and two points along the lines are chosen.

November 1, 2009 15

RANSAC Algorithm

RANSAC (Random Sample Consensus) is a re-sampling technique that generates candidate solutions by using the minimum number observations (data points) required to estimate the underlying model parameters.

Algorithm: 1: Select randomly the minimum number of points required to

determine the model parameters.2: Solve for the parameters of the model.3: Determine how many points from the set of all points fit with a

predefined tolerance.4: If the fraction of the number of inliers over the total number

points in the set exceeds a predefined threshold re-estimate the model parameters using all the identified inliers and terminate.

5: Otherwise, repeat steps 1 through 4 (maximum of N times).

November 1, 2009 16

Results of the point-correspondence algorithm in the Computer Science Building. (a) Numbers label the detected points on the image and the corresponding matched points on the floor plan. (b) The floor plan is warped into the image space and overlaid on the original image. This example has 10 image points and 32 map points and completes matching in about 4 seconds.

November 1, 2009 17

Augmenting Images

While the system finds the correspondence between the image floor plan landmarks, it simultaneously solves for both the camera location and its orientation.

Application can leverage the increased location accuracy and camera orientation when determining what information to display.

They can use the mapping between the image and floor plan to overlay information on the camera image.

November 1, 2009 18

Feasibility Analysis

Speed and Accuracy

System can accurately complete an entire cycle from taking a picture on the phone to displaying an augmented image in approximately 10 seconds using a standard 2.8 GHz computer and processing in Java.

Accuracy of 30 cms.

Image transfer over WI-FI - 1 sec System Processing - 9 secs

Segmentation - 1.5 secsFeature Correspondence - 2 to 6 secsImage Augmentation - 0.5 sec

November 1, 2009 19

Changes in floor material, reflective floors and structures like catwalks causes problems.

People or clutter in the hallways causes problems in detecting features.

Environments with low contrast difference between floors and walls.

November 1, 2009 20

LimitationsExamples that pose problems for Segmentation.

System Extensions

System is fast enough, but further improvements can be done.

Better Feature Detection would reduce the no: of matches and speed up the algorithm.

Performing some processes on the phone reduces amount of data and the time needed for the transfer.

November 1, 2009 21

Conclusion & Future Works The current system’s limitations in handling the full

range of environment can be solved by improving the feature detection and matching.

Results can be improved by using more sophisticated image comparison algorithms.

Accelerometers can also help in improving accuracy.

The system is a simple, low cost navigation assistant to provide a low cognitive load interface on a user’s standard camera phone.

November 1, 2009 22

References

Hile. H, Borriello. G. “Positioning and Orientation in Indoor Environments Using Camera Phones” Computer Graphics and Applications, IEEE, Volume 28, Issue 4, July-Aug. 2008, Pages: 32-39.

G. Fritz, C. Seifert, and L. Paletta, “A Mobile Vision System for Urban Detection with Informative Local Descriptors,” Proc. 4th IEEE Int’l Conf Computer Vision Systems (ICVS 06), IEEE CS Press, 2006, pp. 30

William A. Hoff, Khoi Nguyen , “Computer vision-based registration techniques for augmented reality” Proceedings of Intelligent Robots and Computer Vision XV, SPIE Vol. 2904, Nov 18-22, 1996, Boston, MA, pp. 538-548.

Nishkam Ravi, Pravin Shankar, Andrew Frankel, Ahmed Elgammal and Liviu Iftode, “Indoor Localization Using Camera Phones”.

November 1, 2009 23

Thank ..!You

November 1, 2009 24