an algorithm for in-place vision-based skewed 1d barcode scanning in the cloud
TRANSCRIPT
An Algorithm for In-Place Vision-Based Skewed 1D Barcode Scanning
in the Cloud
Vladimir Kulyukin Tanwir ZamanDepartment of Computer Science
Utah State UniversityLogan, UT, USA
vkedco.blogspot.com
Introduction● A common requirement of many smartphone barcode
scanners, both commercial and open source, is camera alignment: the camera must be aligned with a target barcode to obtain at least one complete scanline
● This requirement presents difficulties for visually impaired smartphone users or for users who do not have adequate control of their hands
● Another challenge is lack of coupling of barcode scanning to comprehensive nutrition label databases
vkedco.blogspot.com
Introduction● In this paper, an algorithm is presented for in-place
vision-based camera alignment that requires no camera alignment
● The algorithm is in-place in that it performs no rotation of input images to align localized barcodes for scanning
● The algorithm is cloud-based because in that most image processing is done in the cloud
● The algorithm is coupled to a database of 200,000 nutrition labels
vkedco.blogspot.com
Outline
● Background● In-Place 1D Skewed Barcode Scanning with Relaxed
Pitch, Roll, and Yaw Constraints● Linux Cluster for Image Processing & Data Storage● Experiments & Results
vkedco.blogspot.com
Dominant Orientation of Gradients● In our previous work (Kulyukin et al., IPCV 2013), we
developed a vision-based algorithm for localization of skewed barcodes on smartphones (pdf)
● The algorithm localizes skewed barcodes in captured frames by computing dominant orientation of gradients (DOGs) of image segments and collecting smaller segments with similar DOGs into larger connected components
● The algorithm was successfully evaluated on 1,066 images
vkedco.blogspot.com
Dominant Orientation of GradientsDominant Orientation of GradientsDominant Orientation of GradientsDominant Orientation of Gradients
Regions with similar DOGs are combined into clusters on the basis of their geometric proximity
vkedco.blogspot.com
In-Place 1D Barcode Scanningwith Relaxed Pitch, Roll, and Yaw
Constaints
vkedco.blogspot.com
Algorithm in Action
Online demo videos:- lab video 1- lab video 2- supermarket video 1- supermarket video 2
vkedco.blogspot.com
Growth of Scanlines
Scanlines are grown orthogonally to the skew line detected by thethe DOG algorithm
vkedco.blogspot.com
Growth of Scanlines
The number of grown scanlines is a parameter currently set to 10
vkedco.blogspot.com
Hardware
● A Linux cluster was built out of four Dell computers ● Each node computer has an Intel Core i5-6503.2 GHz
dual-core processor that supports 64-bit computing● The node's processor has 3MB of cache memory● Each node has 6GB DDR3 SDRAM and 320GB of
hard disk space
vkedco.blogspot.com
Hardware
● A Linux cluster was built out of four Dell computers ● Each node computer has an Intel Core i5-6503.2 GHz
dual-core processor that supports 64-bit computing● The node's processor has 3MB of cache memory● Each node has 6GB DDR3 SDRAM and 320GB of
hard disk space
vkedco.blogspot.com
Software
● Ubuntu 12.04 LTS was installed on each machine● JBoss was used to build and configure the cluster● JBoss AS 7.1.1 was installed on every node● Load balancing is done with the Apache mod_cluster
module● The cluster has one master and three slaves● All four machines are part of a local area network
vkedco.blogspot.com
Software ● The system's front end was implemented as a
smartphone application that runs on Android 4.3 or higher
● The system's back end was deployed on the Linux cluster
● A Java servlet for image recognition was deployed on the master node
● The servlet receives images from smartphones through HTTP POST requests and sends nutrition labels back to the smartphones
vkedco.blogspot.com
Experiment Design ● The system was evaluated on a set of 506 video
recordings of common grocery products● The videos have a 1280x720 resolution, an average
duration of 15 seconds and were recorded on an Android 4.2.2 Galaxy Nexus smartphone in a supermarket
● The videos covered four product categories: bags, boxes, bottles, and cans
● The videos are publicly available here
vkedco.blogspot.com
Experiment Design ● There are 130 box videos, 127 bag videos, 125 box
videos, and 124 can videos● Frames were extracted at a rate of 1 frames per
second, which resulted in 7,545 frames● Frames were manually classified into 1950 box
frames, 1905 bag frames, 1875 bottle frames, and 1860 can frames
● Manually classified frames were automatically classified as blurred or sharp with an algorithm based on 2D Haar filter
vkedco.blogspot.com
Experiment Design ● There are 130 box videos, 127 bag videos, 125 box
videos, and 124 can videos● Frames were extracted at a rate of 1 frames per
second, which resulted in 7,545 frames● Frames were manually classified into 1950 box
frames, 1905 bag frames, 1875 bottle frames, and 1860 can frames
● Manually classified frames were automatically classified as blurred or sharp with an algorithm based on 2D Haar filter
vkedco.blogspot.com
Average Request-Response Times (ms)
16 sessions of 100 frames per session; tests were done over a Wi-Fi network with a download speed of 72.31 Mpbs and an upload speed of 24.64 Mbs vkedco.blogspot.com
Conclusions● In all four categories of products, the true negative and
false positive percentages are 0● The false negative percentages are relatively high: the
algorithm rejects frames on the slightest chance that it does not contain a barcode
● The algorithm scans barcodes of any orientation in the yaw plane, of up to 73 degrees in the pitch plane, and of up to 55 degrees in the roll plane
vkedco.blogspot.com
NutriGlass 1.0 on Google Play
The free front end application is available on Google Play; (here) search “NutriGlass on Google Play” to download
vkedco.blogspot.com