grocery shopping assistant carolina galleguillos pixel-café / june 2 2006

of 20/20
Grocery Shopping Assistant Carolina Galleguillos Pixel-café / June 2 2006

Post on 27-Mar-2015




0 download

Embed Size (px)


  • Slide 1

Grocery Shopping Assistant Carolina Galleguillos Pixel-caf / June 2 2006 Slide 2 Description GroZi project (grocery shopping assistant) Increase independence of people with low vision (specially blind) to perform grocery shopping in a supermarket or store. Help to plan shopping list, walking path to the store and grocery shopping. Slide 3 Motivation 1.3 million legally blind people in the U.S Grocery store are underselling to this market. Blind people are high cost customers. Advance research on object recognition for mobile robotics with constrained computing resources. Slide 4 Motivation Characteristics Grocery Store: Structured Environment (+). Controlled Lightening (+). Maintained by staff (+). Well indexed (+). People moving around aisles (-). Huge amount of products (30K) (-). Slide 5 Motivation Possible existing solutions: 1.Seeing-eye dog trained. 2.RFID tags (aisle, shelf, product). 3.Barcode scanning (shelf). 4.Help of sighted guide/customer service. 5.Memorize store layout. 6.Home delivery. Slide 6 Why computer vision? 1.Limited ability of dogs. 2.RFID tags bring privacy concerns and heavy infrastructure. 3.Eye safety and mislabeling. 4.Independence. 5.Store layout changes constantly. 6.Autonomy. Slide 7 Our Solution Develop a handheld device that performs visual object recognition with haptic feedback. Avail of complementary resources (RFID, Barcode scan, sighted guide) We are focusing on the computer vision aspects of this problem. Slide 8 MoZi Box General purpose low-cost mobile system geared for computer vision applications. MoZi is a combination of the Mobile Vision System (MoVs) and ZigZag Finite memory : Compact Flash (CF) cards ranging from 256 MB to 4 GB. Processor speed: in the neighborhood of 60-400MHz Frame rate: enough snapshots to cover the shelf with some overlap (as in panoramic stitching) (15fps instead of 30fps?). Color Calibration: Macbeth color chart to calibrate the color space. Slide 9 Use of the System Creating a Shopping List. Getting to the Grocery Store. Navigating the Store. Slide 10 Shopping List Online Website: Website stores data and images of different products. Feedback from users. Provides walking path. Prepare shopping list. Download information into Mozi Box. Slide 11 On the way Separate project. Mozi Box with GPS. Visual waypoints. Traffic/Street sign reading. Use in addition to cane and asking sighted bystanders. Slide 12 Inside the Store Finding aisle (OCR, RFID, ask). Avoiding obstacles (cane). Finding products (sweep of aisle, spot product, barcode check). Checking out (coupon and cash). Slide 13 Obtaining training data Online Images (Web). Collecting from MoZi box (in situ). Collecting from embedded camera near the barcode scanner (in situ). Known databases (COIL-100,ETH- 80, etc.)(more research oriented) Synthetic examples. Active learning. Slide 14 Obtaining training data Active learning problem: Find UPC for the corresponding image (labeling). Semi-supervised. Weakly labeled. Slide 15 Obtaining training data Sunshine Store @ UCSD. Venue for pilot study. 4K items in stock. 1749 sq. ft. (assignable) We want to scale to a bigger number of products (30K). No bakery or vegetables. Slide 16 Object Recognition 2 types of recognition (m:n, m