Whiteboard Shape Recognition

using

Deformable Templates and Loopy Belief Propogation

Noah SnavelyDavid Bargeron

April 2004

Introduction

• Want to recognize shapes– What constitutes a shape?– Where is the shape in an arbitrary image?– What if the shape has deformed?

• Applications– Whiteboard reco– Lifting annotations– Object reco and tracking in video

• Appoach: BP with lots of optimizations

Application

Implementation Issues

• Paper suggests computing the highest belief location for each node independently (max marginals)– This tends to fail for objects with rotational symmetry– Using max-product algorithm can help

Sum-product (max marginals) Max-product (MAP)

Implementation Issues

• Optimization: after each round of message passing, prune states with low beliefs– Sometimes the correct states get pruned in

early iterations of BP– Solution: always keep a minimum number of

states (we used 50)

Issues & Future Work

• Problem: Hallucinating large shapes in a jumble of smaller ones

– Solution: Labelled CC image

Issues & Future Work

• Problem: Scaling.– Current: Make sure template is appropriate size– Future: Cluster CCs on size, scale template to each of

the cluster means

• Problem: Deformable Templates for reco– Future: Need infrastructure for finding multiple hits,

distinguishing between competing models