the nao robot, a case of study - politecnico di...

The NAO Robot, a case of studyRobotics 2013-2014

Franchi Alessio [email protected]

Cesare AlippiGiuseppe Andreoni

Who am I?

Franchi Alessio MauroMaster Degree in Computer Science Engineer at Politecnico of Milan

First year Ph.D. student, collaborating with prof. Gini

If you need to contact me:mail: [email protected]: AirLab – DEIB, building 20, ground floortelephone: 3565

Research areas:Cognitive roboticsSymbolic reasoningGrasping


This lesson

Introducing the NAO robot (~20 mins)• Structure• Sensors• Capabilities• …

Computer vision - OpenCV(~60 mins)Loading/reading images;Images manipulationImages filteringVideos streaming

Two sections


Computer Vision/OpenCVWhat is OpenCV (Open Source Computer Vision)?

A set of libraries focused on real-time images processing.A project started by Intel Russia in 1998Now are maintened by Itseez and Willow Garage2500 optimized algorithmsA full-featured CUDA and OpenCL interfaces are being actively developed right now

http://opencv.org/

Built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products


Computer Vision/OpenCVSome application:

Detect and recognize faces,Identify objects, Classify human actions in videos,

Track camera movements and moving objects,Extract 3D models of objects and produce 3D point clouds,Recognize scenery and establish markers to overlay it with augmented reality,Etc…


Augmented Reality/OpenCVThe technology functions by enhancingone’s current perception of reality

Live direct or indirect view of a physical real-world environment whose elements are augmented/extended


Let’s start with OpenCV

1. Go to http://opencv.org/2. Download auto-extracting or binary

files to be compiled3. Set environment variables

a) Open command prompt and type “setx -m OPENCV_DIR D:\OpenCV\Build\x86\vc11”

b) Right click on “Computer” ->”Properties”->”Advanced-System settings”->”Advanced”tab->”Environment variable”->”System variables”

c) Add to PATH %OPENCV_DIR%\bin

d) Reboot the system

How to install OpenCV on Windows + VisualStudio 2012


Let’s start with OpenCV

1. Open Visual Studio 20122. Select “File”->”New”->”Project”3. Once created right click on the “project name”->”Property”4. ”Configuration properties”->”c/c++”->”General” add to Additional

“Include Directories” the following “$(OPENCV_DIR)\..\..\include”5. “Linker”->”General” add to «Additional libraries Directories” the

following $(OPENCV_DIR)\lib”6. ”Linker”->”Input” add to ”Additional Dependencies” rhe following list

“opencv_core249d.lib;opencv_imgproc249d.lib;opencv_highgui249d.lib;opencv_ml249d.lib;opencv_video249d.lib;opencv_features2d249d.lib;opencv_calib3d249d.lib;opencv_objdetect249d.lib;opencv_contrib249d.lib;opencv_legacy249d.lib;opencv_flann249d.lib”

Now create a new Visual Studio project


Let’s start with OpenCVThe «Hello world» example


Some theoryHow images are composed – Black and white

Any digital image is a matrix of pixels. Any pixel have a value.The value of pixels are between 0 (=black) and a maximum depending on the number of bits of the image (8 bits = 255 = white) .

The image-depth refers to the bits allocated for each pixel.

When the value of the pixel is increased, the intensity of that pixel is also increased.

ExampleImage-depth 8 bit1 channel, grayscale imageHeight is 4 pixel, width is 5 pixelsResolution = 4x5.


Some theoryHow images are composed – Color

Color image is composed by 3/4 planes;RedGreenBlueAlpha (optional)

Any pixel’s final color is the combination of the corresponding values of pixels in the three/four planes

(255, 0, 0) = red.(0, 255, 0) = green(255, 0, 255) = violate


Some theoryIntroducing the concept of feature

An "interesting" part of an imageHuman beings are naturally able to find these «features», it is something innate.As consequences we are able to find two similar images, to compose images or to play with puzzle

A simple game: are you able to tell me the exact location of each patche?


Some theory


Let’s go on with OpenCVDilation The value of the output pixel is the maximum value of all the

pixels in the input pixel's neighborhood. Pixels beyond the image border are assigned the minimum

Erosion The value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood.Pixels beyond the image border are assigned the maximum


Let’s go on with OpenCVCanny Edge detector1. Filter out any noise. The Gaussian filter is

used for this purpose

2. Four filters to detect horizontal, vertical and diagonal edges in the blurred image are used

3. Search for local maxima in the gradient map with non-maximum suppression

4. Thresholding with hysteresis:Lower than low threshold: pixel is not considered;Higher than high threshold: pixel is on an edge;Between the two thresholds: pixel is considered if nearby an already considered pixel.


Let’s go on with OpenCVHarris corner detector

1. Corners represents a variation in the gradientin the image, we will look for this “variation”.

Maximize the equation above to find windows with a large variation

So sweep a window in both direction and compute the variation of intensity

A window with a score R greater than a certain value is considered a “corner”


Let’s go on with OpenCVThe Hough transformA technique for detecting significant configurations of points in the image.

Lines, segments, curves,…

The shapes can be expressed by a function in a new parameters space


Let’s go on with OpenCVThe Hough transform

How points are transformed? Points are the intersection of two or more linesSo points is a curve in the parameters space


Let’s go on with OpenCVThe Hough transform

-How lines are transformed?Curves corresponding the three points on the lines intersect in a given point in the parameters spaceThis point is the image of the lines.


The NAO robot

25 degrees of freedom (DOF)2 cameras, 4 directional microphones, sonar, 2 IR receiver/emitter, 1 inertial board, 9 tactile sensors and 8 pressure sensorsvoice synthesizer, LED lights, and 2 speakers

NAO is a programmable, 58cm tall humanoid robot


Links and axisAxis definition

The X axis is positive toward NAO’s front, the Y from right to left and the Z is vertical.

Links’ length


Joints


Sensors

Sensor network:2 cameras,4 directional microphones,sonar rangefinder, 2 IR emitters and receivers,1 inertial board,9 tactile sensors8 pressure sensorsvoice synthesizer, LED lights,2 speakers


Some examples…Some basic movements…Standing up;Sitting down;Walking

…and some more complex:Dancing;Tracking obects;Climbing stairs;Robocup.

Videos!

the nao robot, a case of study - politecnico di...

Documents