![Page 1: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/1.jpg)
LYU0203Smart Traveller with Visual Translatorfor OCR and Face Recognition
Supervised by Prof. LYU, Rung Tsong Michael
Prepared by: Wong Chi Hang
Tsang Siu Fung
Department of Computer Science & Engineering
The Chinese University of Hong Kong
![Page 2: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/2.jpg)
Outline
Introduction System Architecture Korean OCR Friend Reminder Conclusion Acknowledgement
![Page 3: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/3.jpg)
Introduction – What is VTT?
Smart Traveller with Visual Translator (VTT)Mobile Device which is convenient for a
traveller to carry Mobile Phone, Pocket PC, Palm, etc.
Recognize and translate the foreign text into native language
Detect and recognize the face into name
![Page 4: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/4.jpg)
Introduction – Objective
Two main features: Korean to English
Visual Translation
Remind Somebody’s Information with Face Image
![Page 5: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/5.jpg)
Introduction – Objective (Cont.)
Real Life ExamplesSometimes we lose the way, we need to know
where we are.Sometimes we forget somebody we met
before.
![Page 6: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/6.jpg)
System Architecture
GUI
Camera API
Camera
Korean OCR Face Recognizer
Face DatabaseStroke Database
&Dictionary
Request
RequestData
Data
RequestOutput
User
QueryResult Query UpdateResult
Request Response
Request Response Request Response
![Page 7: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/7.jpg)
Korean OCR (KOCR) Usage
Visual Translator from Korean to English Procedure for using KOCR
Text Area DetectionCharacter IdentificationTranslation
![Page 8: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/8.jpg)
KOCR – Program FlowInitialization
Capture Image
Text Segmentation
Recognition
Translation
![Page 9: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/9.jpg)
KOCR – Text Area Detection
Edge Detection using Sobel Filter Horizontal Projection and Vertical Projection Find Potential Text Area by threshold
-1 -2 -1
0 0 0
1 2 1
-1 0 1
-2 0 2
-1 0 1
Hor
izon
tal
Pro
jec
tion
Threshold
Vertical Projection
![Page 10: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/10.jpg)
KOCR – Text Area Detection (Cont.)
![Page 11: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/11.jpg)
KOCR – Character Identification
Features on Stroke Extracted by Labeling Connected Component
algorithm
Proposed Feature Extraction Five rays each side Difference of adjacent rays (-1 or 0 or 1) Has holes (0 or 1) Dimension ratio of Stroke (width/height) (-1 or 0 or 1)
![Page 12: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/12.jpg)
KOCR – Character Identification (Cont.)
![Page 13: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/13.jpg)
KOCR – Translation
Dictionary Korean to English About 1000 Korean Words
Matching Longest Match from left to right
![Page 14: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/14.jpg)
KOCR – Translation (Cont.)
![Page 15: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/15.jpg)
KOCR – Evaluations
OCR CorrectnessTraining Set (3327 – 30% of all Character)Testing Set (7845 – Others)Result (64%)Suggestion
Train all Korean characters
![Page 16: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/16.jpg)
KOCR – Evaluations (Cont.)
Text Segmentation Correctness45 Captured Images99 CharactersResult
Segment 83% characters correctly Segment 71% image correctly
Acceptable Result
![Page 17: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/17.jpg)
KOCR – Evaluations (Cont.)
OCR Correctness45 Captured Images99 CharactersResult
79% Characters correctly Recognized 69% Images correctly Recognized
![Page 18: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/18.jpg)
Friend Reminder – Program FlowInitialization
Capture Image
Face Segmentation
Recognition
Show Profile
![Page 19: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/19.jpg)
Friend Reminder (FR)
UsageShow the Profile of Friend by capturing a
photo Procedure for using FR
Face SegmentationFace IdentificationFriend’s Profile
![Page 20: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/20.jpg)
FR – Face Segmentation
Eye DetectionAlgorithm
Gabor Wavelet Feature Log-Polar Sampling
Manual Selected (Suggest) Selected Eyes and Mouth Positions
![Page 21: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/21.jpg)
FR – Face Segmentation
![Page 22: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/22.jpg)
FR – Face Identification
EigenFaceBy using Principal Component Analysis (PCA)Project the input face into the eigenvectors th
at pre-learnedFind the difference between the projection an
d the faces in databaseFace determined to be ‘NEW’ if the difference
is larger than a threshold
![Page 23: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/23.jpg)
FR – Friend’s Profile
![Page 24: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/24.jpg)
FR – Evaluations
Eye Detection Correctness40 ImagesResult
22.5% Image Successfully Detected
Non-acceptableSuggestion
Manually Select Eyes and Mouth Positions
![Page 25: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/25.jpg)
FR – Evaluations Face Identification
Evaluation Information 26 Test Persons’ Faces
16 faces is in database 10 faces is not in database
3 faces Trained per person 8 persons in face database
Result 77% Successfully Identified
63% Successfully Identified as person in database 100% Successfully Identified as person not in database
![Page 26: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/26.jpg)
Conclusion
Combined Modern Equipments Digital camera Personal Data Assistant (PDA)
Techniques Learned Image Processing Optical Character Recognition Face Recognition Techniques
VTT Integrated VTT for Korean to English OCR VTT for Friend Reminder
![Page 27: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/27.jpg)
Acknowledgement
Thanks Professor Michael Lyu,Project SupervisorGive us valuable adviceProvide us necessary equipments
Thanks Edward Yau,Technical Manager of VIEW projectGive us many ideas
![Page 28: LYU0203 Smart Traveller with Visual Translator for OCR and Face Recognition](https://reader036.vdocuments.mx/reader036/viewer/2022070401/5681375e550346895d9eec5e/html5/thumbnails/28.jpg)
~The End~