Download - Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister
![Page 1: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/1.jpg)
1Department of Electrical and Computer Engineering
Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz
Ian McAlisterColin Smith
Chris NeylandErick Drummond
TEAM GANZ Final Project Review
![Page 2: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/2.jpg)
2Department of Electrical and Computer Engineering
OUTLINE Motivation Product Requirements Product Design Manufacturing Image Processing Experimental Results Conclusions SDP Demonstration Day Tasks Future Enhancements
![Page 3: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/3.jpg)
3Department of Electrical and Computer Engineering
Motivation Population• 1.8 million legally blind individuals in the U.S. • 21.2 million reported experiencing vision loss
US Currency Problem• No current identification methods• Current technology is very expensive
Goals Develop low cost, easy-to-use, portable
electronic bill reader for the blind community
![Page 4: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/4.jpg)
4Department of Electrical and Computer Engineering
Product Requirements Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 5: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/5.jpg)
5Department of Electrical and Computer Engineering
Final Product Design – HardwareSystem Block Diagram: Production Model:
Beagleboard
Audio Device
Camera
SD Card
Audio Library
Image
Capture
Image Library
Batteryw/ 5V Regulator
LEDs
Image Recognition Algorithm
Push Buttons AVR Microcontroller
![Page 6: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/6.jpg)
6Department of Electrical and Computer Engineering
Cost list here with total Costs
Beagleboard: $160 Enclosure: $14.60 Microcontroller and Misc Hardware: $15 PS3 Eye: $35 Battery/Charger $25 SD Memory Card: $12 Total: 246.60
![Page 7: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/7.jpg)
7Department of Electrical and Computer Engineering
Product Requirements Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 8: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/8.jpg)
8Department of Electrical and Computer Engineering
Manufacturing of PCB Custom PCB needed for User Interface Designed Schematic Using Eagle
Photo Paper used for layout relative to Beagleboard Top/Bottom layers printed separately on glossy paper
Bottom layer taped to copper board and holes drilled Top layer mirrored, placed on reverse side, and lined up
with holes Hot iron used to transfer toner to copper board
Paper rinsed in water bath PCB place in Ferric Chloride bath to dissolve unwanted copper Toner washed off with acetone and through holes drilled PCB fit and tested
![Page 9: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/9.jpg)
9Department of Electrical and Computer Engineering
Product Requirements Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 10: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/10.jpg)
10Department of Electrical and Computer Engineering
Manufacturing of Project Enclosure Polycarbonate Case – Best for BATBRD Project Dimensions in inches
4.53L X 3.54W X 2.17H = 34.8 in3
Meets size requirements Painted for lighting/cosmetic purposes Holes drilled for switches and power connections Rails installed for user interface
![Page 11: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/11.jpg)
11Department of Electrical and Computer Engineering
Product Requirements Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 12: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/12.jpg)
12Department of Electrical and Computer Engineering
CAMERAIMAGEDATA
CAPTURE IMAGE COMMAND
BUTTON DEPRESSED?
CHECK IMAGE AGAINST $1 TEMPLATES
HIGEST CORRELATION >
X?
TEMPLATE IMAGEDATA(ONE)
RETRIEVE AND OUTPUT ERROR
MESSAGE
START(AWAITING INPUT)
YES
NO
TIMEOUT
CHECK IMAGE AGAINST $10 TEMPLATES
TEMPLATE IMAGEDATA(TEN)
RETRIEVE AND OUTPUT MESSAGE CORRESPONDING
TO TEN
YES
![Page 13: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/13.jpg)
13Department of Electrical and Computer Engineering
Image Processing Find Best Match by Using Normalized Cross-Correlation
Coefficients Function (NCC)
C = Correlation Matrix I = Captured Image, = Sample Image Mean at location: u,v T = Template Image, = Template Mean u,v = Indices at Point of Correlation
Best Match Retrieved by OpenCV Function: cvMinMaxLoc( image, &minval, &maxval, &minloc, &maxloc, 0 );
Where maxval is a number from -1 to +1 corresponding to best match. This Value Can then Be Used to Determine if There is a Match
yx yxvu
vuyx
TyxTIyvxuI
TyxTIyvxuIvuC
, ,22
,
,,
]),([]),([
]),(][),([),(
![Page 14: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/14.jpg)
14Department of Electrical and Computer Engineering
Experimental Design Experiment 1
Timing/Accuracy with Ideal Images Experiment 2
Thresholds with Ideal Images and Second ID Experiment 3
Thresholds/Accuracy with degraded images Experiment 4
Thresholds/Accuracy with further degraded images Experiment 5
Project Enclosure Threshold/Accuracy worst case degraded image.
![Page 15: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/15.jpg)
15Department of Electrical and Computer Engineering
Experiment 1 – Timing/Accuracy with Ideal Images Pixel Size
360 x 240 Templates under complete darkness (no backlighting) Results
Significant time delay Reduced Accuracy
Thresholds too low New generation bill Rail Structure
Bill Type Time (s)Maximum
Threshold ValueOverall
Accuracy
Reg $1 1.902 0.936 100.0001gen $5 12.793 0.965 100.0002gen $5 6.610 0.911 87.5003gen $5 4.963 0.908 75.000
1gen $10 9.601 0.905 100.0002gen $10 3.890 0.879 83.3333gen $10 3.794 0.888 75.0001gen $20 10.253 0.950 100.0002gen $20 4.534 0.933 100.0003gen $20 2.860 0.957 100.000
Overall 6.120 0.923 92.083
Data includes averages of 12 tests conducted for each bill*
![Page 16: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/16.jpg)
16Department of Electrical and Computer Engineering
Exp. 2 – Thresholds with Ideal Images and 2nd ID Changes from Experiment 1
Pixels reduced to 160x120 Templates retaken with tinted backlighting Camera moved Rails more securely fixed Max Threshold .85
Results Worst case time < 4 seconds All corners identifying well Backlighting no longer a problem
Averages of 12 tests conducted for each bill
Bill TypeMax Threshold
Value2nd Highest Threshold
Reg $1 0.917 0.6351gen $5 0.898 0.5622gen $5 0.947 0.7893gen $5 0.916 0.829
1gen $10 0.945 0.6762gen $10 0.942 0.8583gen $10 0.936 0.8611gen $20 0.924 0.5902gen $20 0.931 0.7573gen $20 0.939 0.842
![Page 17: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/17.jpg)
17Department of Electrical and Computer Engineering
Exp 3 Threshold/Accuracy non-ideal images
Bill TypeMaximum Threshold
Value
2nd Highest Threshold different
denominationOverall Accuracy
Reg $1 0.814 0.532 100.0001gen $5 0.817 0.548 100.0002gen $5 0.858 0.584 100.0003gen $5 0.809 0.574 100.0001gen $10 0.855 0.540 100.0002gen $10 0.881 0.534 100.0003gen $10 0.842 0.515 100.0001gen $20 0.841 0.507 100.0002gen $20 0.830 0.495 100.0003gen $20 0.845 0.531 100.000
Test 3 Accuracy with non-ideal images (1 diagonal Stripe)Data includes averages of 8 tests conducted for each bil l
1 Diagonal Stripe added to image capture area to simulated degraded/damaged bills
![Page 18: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/18.jpg)
18Department of Electrical and Computer Engineering
Exp. 4 – Continued Bill Degradation Tests Simulated Degradation 2 Stripes
3 Stripes
4 Stripes
Bill Type Max Threshold 2 Stripes
Max Threshold 3 Stripes
Max Threshold 4 Stripes
Reg $1 0.762 0.746 0.6501gen $5 0.747 0.782 0.6972gen $5 0.805 0.803 0.7253gen $5 0.795 0.774 0.708
1gen $10 0.754 0.709 0.6412gen $10 0.845 0.834 0.7443gen $10 0.790 0.804 0.7151gen $20 0.763 0.744 0.6782gen $20 0.800 0.782 0.7213gen $20 0.808 0.809 0.735
Data includes averages of 8 tests conducted for each bil l
![Page 19: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/19.jpg)
19Department of Electrical and Computer Engineering
Exp. 5 – Project Enclosure, Non Ideal Images Maximum Threshold Identification over 11% wrong identification Using Greater than .65 for threshold Zero Wrong IDs
High rescan rate no incorrect identification
% ID % Rescan Request
% Wrong ID
Max Threshold Only 88.75 0 11.25Greater than .6 53.75 43.75 2.5
Greater than .625 40 58.75 1.25Greater than .65 32.5 67.5 0
Statistics all Bills under "worst case" degredation"
![Page 20: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/20.jpg)
20Department of Electrical and Computer Engineering
Product Requirements Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 21: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/21.jpg)
21Department of Electrical and Computer Engineering
Power Usage Battery Power Supply:
The Battery Provides 7.4V and 2.2Ahr 7.4V * 2.2A hr = 16.28W hr
BATBRD Power Usage Beagleboard and supporting hardware
operates at 5V and draws 600 – 800mA The BATBRD draws a maximum of 4W. We can safely operate the board for at
least ~4hrs
![Page 22: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/22.jpg)
22Department of Electrical and Computer Engineering
Product Requirements - Conclusion Product Requirements
Low Cost Intuitive Interface Design – 2 to 3
buttons max Small Form Factor - <50in3
Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use
![Page 23: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/23.jpg)
23Department of Electrical and Computer Engineering
TBD for SDP Demonstration Day
Work out minor issues with user interface, power-up sequence Meet with Jenny again: let her use it, get her input Audio adjustments: improve playback, eliminate ‘pops’ Possibly remake templates, modify lighting Continue testing for accuracy as changes are made
![Page 24: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/24.jpg)
24Department of Electrical and Computer Engineering
Future Outlook Jenny’s requests
Audio notification of battery life/charging status Prerecorded audio user’s manual Physically unique charging plug shape
Beyond SDP 2010 Tailored processing hardware (FPGA, ASIC) Application-specific DSP techniques in IP algorithm Smaller, low-profile imaging device & battery
![Page 25: Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister](https://reader033.vdocuments.mx/reader033/viewer/2022051518/56816181550346895dd112f4/html5/thumbnails/25.jpg)
25Department of Electrical and Computer Engineering
Questions ?