tour guide robot project face detection and face orientation on the mobile robot robotino gökhan...
TRANSCRIPT
![Page 1: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/1.jpg)
Tour Guide Robot Project
Face Detection and Face Orientation on The Mobile Robot Robotino
Gökhan Remzi YavuzAyşenur Bilgin
![Page 2: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/2.jpg)
Outline
IntroductionHardwareProject RequirementsProject ProgressQuestions
![Page 3: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/3.jpg)
Introduction
Human Computer Interaction - HCIDetecting human facesDetecting interested people
Face detection Face orientation and face rotation
Keeping people interested by interacting with them
Getting closer when a certain distance is exceeded
![Page 4: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/4.jpg)
Hardware
Robotino: a mobile robot platform from Festo
To satisfy project's requirements, we are planning some extensions on Robotino: Cylindirical body, total height: 1.60m including camera A laptop computer, attached to body at 1.20m Second color camera on top of the laptop computer dedicated to HCI
9 infrared sensors3 omnidirectional drive unitsBump sensorColor camera
Control Unit: a PC with a 300 Mhz Processor and 128 MB SDRam
![Page 5: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/5.jpg)
Hardware con’t
• Sketch of hardware modifications on the robot base Robotino
• 1 additional laptop computer, 2 additional color cameras, skeleton with steel sticks and not yet sketched cylindirical cover for better appearance
![Page 6: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/6.jpg)
Project Requirements
A tour guide robot should:
be able to navigate autonomously and purposefully inside the building
operate in a dynamic environment
attract visitors' attention by interacting with them
be able to monitor its charge level and move to its charging dock on its own
be able to dynamically arrange tours by combining selected destinations
deliver information in forms of visual, audio and text messages
be able to synchronize with the CMPE web site, in order to get up to date information.
![Page 7: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/7.jpg)
Project Progress
What we did? Architecture Pseudocode Examples
What we did not?What we encountered?What we achieved?
![Page 8: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/8.jpg)
What we did?
Combining Robotino API and OpenCVModifying the open source code to satisfy our needsRunning the modified algorithm on the mobile robotTesting the behavior
![Page 9: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/9.jpg)
Architecture
Robotino
RobotinoAPI
Application OpenCV RobotinoAPI
Watchdog Signal Action Requests Camera Requests Camera Data
Application
Action Requests Camera Data
OpenCV
Image Data Face Information
Wireless
![Page 10: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/10.jpg)
Pseudocode
Connect to Robotino over Wireless
While bumpsensorvalue = false do
Retrieve camera image from RobotinoAPI
Process image
. OpenCV face detection using haarcascades, frontal and profile faces
. Compute approximate distances of faces and their angle to the robot
Decide action; rotate, move forward, attract attention, stop
Act
end While
![Page 11: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/11.jpg)
Examples
![Page 12: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/12.jpg)
Examples cont’d
![Page 13: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/13.jpg)
What we did not?
Pose estimationTrackingFace recognitionCompletion of hardware modifications
![Page 14: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/14.jpg)
What we encountered?
Major challenge : Erroneous detection of faces
Processing power Minor problems :
Relatively low (10 fps) performance when frontal and profile face detection algorithms are combined
Position of the camera
![Page 15: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/15.jpg)
What we achieved?
Experiments held in laboratoryMoving towards the human face until the
distance is reduced to 1 meter approximately
Turning with respect to the weighted average of areas of detected faces
![Page 16: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/16.jpg)
Questions???
![Page 17: Tour Guide Robot Project Face Detection and Face Orientation on The Mobile Robot Robotino Gökhan Remzi Yavuz Ayşenur Bilgin](https://reader036.vdocuments.mx/reader036/viewer/2022062422/56649e865503460f94b89b87/html5/thumbnails/17.jpg)
Thank you!