opencv tutorial
DESCRIPTION
OpenCV Tutorial. Omri Perez Adapted from: Gary Bradski Senior Scientist, Willow Garage Consulting Professor: Stanford CS Dept. http://opencv.willowgarage.com www.willowgarage.com. 1. Vision is Hard Camera Model, Lens, Problems and Corrections OpenCV OpenCV Tour. Vision is Hard. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/1.jpg)
11
OpenCV Tutorial
Omri Perez
Adapted from: Gary Bradski
Senior Scientist, Willow GarageConsulting Professor: Stanford CS Dept.
http://opencv.willowgarage.com www.willowgarage.com
![Page 2: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/2.jpg)
• Vision is Hard• Camera Model, Lens, Problems and Corrections• OpenCV• OpenCV Tour
2CS324
![Page 3: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/3.jpg)
• What is it?– Turning sensor readings into perception.
• Why is it hard?– It’s just numbers.
Maybe try gradients to find edges?
Vision is Hard
3CS324
![Page 4: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/4.jpg)
4
• Depth discontinuity• Surface orientation
discontinuity• Reflectance
discontinuity (i.e., change in surface material properties)
• Illumination discontinuity (e.g., shadow) Slide credit: Christopher Rasmussen
Use Edges? … It’s not so simple
CS324
![Page 5: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/5.jpg)
Must deal with Lighting Changes …
5CS324
![Page 6: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/6.jpg)
Lighting is also a Strong Cue
6Gary Bradski (c) 2008 6
![Page 7: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/7.jpg)
The Brain Assumes 3D Geometry
7
Perception is ambiguous … depending on your point of view!
7
![Page 8: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/8.jpg)
8
Geometrical aberrations
spherical distortion
astigmatism
tangential distortion
coma
aberrations are reduced by combining lenses
Marc Pollefeys
Non-Geometrical aberrations
Chromatic
Vignetting
These are typically what are corrected for in camera Calibration
![Page 9: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/9.jpg)
Distortion Correction so that Lenscan Approximate a Pinhole Camera
• Distortions are corrected mathematically– We use a calibration pattern
• We find where the points ended up• We know where the points hould be
• OpenCV 2.2 Function:double calibrateCamera(
const vector<vector<Point3f> >& objectPoints, const vector<vector<Point2f> >& imagePoints, Size imageSize, Mat& cameraMatrix, Mat& distCoeffs, vector<Mat>& rvecs, vector<Mat>& tvecs, int flags=0);
CS324 9
![Page 10: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/10.jpg)
• Vision is Hard• Camera Model, Lens, Problems and Corrections• OpenCV• OpenCV Tour
10CS324
![Page 11: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/11.jpg)
OpenCV Overview: OpenCV Overview: General Image Processing Functions
Machine Learning:• Detection,• Recognition
Segmentation
Tracking
Matrix Math
Utilities and Data Structures
Fitting
Image Pyramids
Camera calibration,Stereo, 3D
Transforms
Features
Geometric descriptors
Robot support
opencv.willowgarage.com > 2000 algorithms
11Gary Bradski
![Page 12: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/12.jpg)
OpenCV Tends Towards Real Time
http://opencv.willowgarage.com
![Page 13: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/13.jpg)
Where is OpenCV Used?
2M downloads
• Well over 2M downloads
Screen shots by Gary Bradski, 2005
• Google Maps, Google street view, Google Earth, Books• Academic and Industry Research• Safety monitoring (Dam sites, mines, swimming pools)
• Security systems• Image retrieval• Video search• Structure from motion in movies• Machine vision factory production inspection systems• Robotics
![Page 14: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/14.jpg)
OpenCV Modules• Calib3d
– Calibration, stereo, homography, rectify, projection, solvePNP
• Contrib– Octree, self-similar feature, sparse L-M, bundle adj, chamfer match
• Core– Data structures, access, matrix ops, basic image operations
• features2D– Feature detectors, descriptors and matchers in one architecture
• Flann (Fast library for approximate nearest neighbors)• Gpu – CUDA speedups • Highgui
– Gui to read, write, draw, print and interact with images
• Imgproc – image processing functions• Ml – statistical machine learning, boosting, clustering• Objdetect – PASCAL VOC latent SVM and data reading• Traincascade – boosted rejection cascade
CS324 14
![Page 15: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/15.jpg)
Software Engineering• Works on:
– Linux, Windows, Mac OS (+ Android since open CV 2.2)
• Languages: – C++, Python, C
• Online documentation:– Online reference manuals: C++, C and Python.
![Page 16: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/16.jpg)
• Vision is Hard• Camera Model, Lens, Problems and Corrections• OpenCV• OpenCV Tour
16CS324
![Page 17: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/17.jpg)
Gradients: Scharr instead of Sobel
• Sobel has been the traditional 3x3 gradient finder.
• Use the 3x3 Scharr operator instead since it is just as fast but has more accurate response on diagonals.
CS324 17
void Scharr(const Mat& src, Mat& dst, int ddepth, int xorder, int yorder, double scale=1, double delta=0, int borderType=BORDER_DEFAULT)
![Page 18: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/18.jpg)
Canny Edge DetectorCanny Edge Detector
18OpenCV team, Gary Bradski
Canny()
![Page 19: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/19.jpg)
Hough Transform
Gary Bradski (c) 2008 19
HoughCircles(), HoughLines(), HoughLinesP() (probabilistic Hough)
![Page 20: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/20.jpg)
Scale Space
void cvPyrDown(IplImage* src, IplImage* dst, IplFilter filter =
IPL_GAUSSIAN_5x5);
void cvPyrUp(IplImage* src, IplImage* dst, IplFilter filter =
IPL_GAUSSIAN_5x5);20Gary Bradski (c) 2008
![Page 21: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/21.jpg)
Space Variant vision: Log-Polar Transform
21Gary Bradski (c) 2008
cvLogPolar(src,dst,center,size, CV_INTER_LINEAR+CV_WARP_FILL_OUTLIERS)
![Page 22: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/22.jpg)
Delaunay Triangulation, Voronoi Tessellation
Gary Bradski (c) 2008 22
CvSubdiv2D* cvCreateSubdivDelaunay2D(CvRect rect, CvMemStorage* storage)
![Page 23: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/23.jpg)
Contours
Gary Bradski (c) 2008 23
void findContours()
![Page 24: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/24.jpg)
Histogram Equalization
Gary Bradski (c) 2008 24
void equalizeHist(const Mat& src, Mat& dst)
![Page 25: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/25.jpg)
Image textures• Inpainting:• Removes damage to images, in this case, it removes the text.
25Gary Bradski (c) 2008
void inpaint(const Mat& src, const Mat& inpaintMask, Mat& dst, double inpaintRadius, int flags);
![Page 26: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/26.jpg)
Morphological Operations ExamplesMorphological Operations Examples• Morphology - applying Min-Max. Filters and its combinations
Opening IoB= (IB)BDilatation IBErosion IBImage I
Closing I•B= (IB)B TopHat(I)= I - (IB) BlackHat(I)= (IB) - IGrad(I)= (IB)-(IB)
26Gary Bradski (c) 2008
Void morphologyEx() createMorphologyFilter()
erode() dilate()
![Page 27: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/27.jpg)
Distance TransformDistance Transform• Distance field from edges of objects
Flood FillingFlood Filling
27Gary Bradski (c) 2008
void distanceTransform(const Mat& src, Mat& dst, int distanceType, int maskSize)
int floodFill(Mat& image, Point seed, Scalar newVal, Rect* rect=0, Scalar loDiff=Scalar(), Scalar upDiff=Scalar(), int flags=4)
![Page 28: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/28.jpg)
Thresholds
Gary Bradski (c) 2008 28
void adaptiveThreshold()
double threshold()
![Page 29: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/29.jpg)
Segmentation
• Pyramid, mean-shift, graph-cut
• Here: Watershed
Gary Bradski (c) 2008 29
void watershed(const Mat& image, Mat& markers)
![Page 30: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/30.jpg)
Background SubtractionBackground Subtraction
30Gary Bradski (c) 2008
BackgroundSubtractorMOG2(), see samples/cpp/bgfg_segm.cpp
![Page 31: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/31.jpg)
31
Image Segmentation & Minimum Cut
ImagePixels
Pixel Neighborhood
w
SimilarityMeasure
MinimumCut
* From Khurram Hassan-Shafique CAP5415 Computer Vision 2003
![Page 32: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/32.jpg)
• Graph Cut based segmentation
Gary Bradski 32
GrabCutvoid grabCut(const Mat& image, Mat& mask, Rect rect, Mat& bgdModel, Mat& fgdModel, int iterCount, int mode)
![Page 33: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/33.jpg)
Motion Templates Motion Templates (My work with James Davies)
• Object silhouette• Motion history images• Motion history gradients• Motion segmentation algorithm
silhouette MHI MHG
33Gary Bradski (c) 2008
![Page 34: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/34.jpg)
Segmentation, Motion Tracking
PoseRecognition
MotionSegmentation
GestureRecognition
MotionSegmentation
34Gary Bradski (c) 2008
void updateMotionHistory(); void calcMotionGradient(); double calcGlobalOrientation();
James Davies, Gary Bradski
![Page 35: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/35.jpg)
Tracking with CAMSHIFT• Control game with head
RotatedRect CamShift(const Mat& probImage, Rect& window, TermCriteria criteria)
![Page 36: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/36.jpg)
3D tracking3D tracking• Camera Calibration• View Morphing• POSIT
void POSIT()A more general technique for solving pose issolving the Percpective N Point problem:
void solvePnP(…)
![Page 37: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/37.jpg)
Mean-Shift for Tracking
Gary Bradski (c) 2008 37
CamShift();MeanShift();
![Page 38: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/38.jpg)
Optical Flow// opencv/samples/c/lkdemo.cint main(…){…CvCapture* capture = <…> ?
cvCaptureFromCAM(camera_id) : cvCaptureFromFile(path);
if( !capture ) return -1;for(;;) { IplImage* frame=cvQueryFrame(capture); if(!frame) break; // … copy and process imagecvCalcOpticalFlowPyrLK( …) …) cvShowImage( “LkDemo”, result ); c=cvWaitKey(30); // run at ~20-30fps
speed if(c >= 0) { // process key }}cvReleaseCapture(&capture);}
y
xt
yyx
yxx
I
IIb
III
IIIGyxX
bXG
dtdyyIdtdxxItI
tyxIdttdyydxxI
,,
,),,(
,
);/(/)/(//
);,,(),,(
2
2
calcOpticalFlowPyrLK()Also see dense optical flow:calcOpticalFlowFarneback()
![Page 39: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/39.jpg)
Features 2D
CS324 39
Read two input images:
Mat img1 = imread(argv[1], CV_LOAD_IMAGE_GRAYSCALE);
Detect keypoints in both images:
// detecting keypointsFastFeatureDetector detector(15);vector<KeyPoint> keypoints1;detector.detect(img1, keypoints1);
Compute descriptors for each of the keypoints:
// computing descriptorsSurfDescriptorExtractor extractor;Mat descriptors1;extractor.compute(img1, keypoints1, descriptors1);
Now, find the closest matches between descriptors from the first image to the second:
// matching descriptorsBruteForceMatcher<L2<float> > matcher;vector<DMatch> matches;matcher.match(descriptors1, descriptors2, matches);
![Page 40: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/40.jpg)
Features 2D continued …
CS324 40
Viusalize the results
namedWindow("matches", 1);Mat img_matches;drawMatches(img1, keypoints1, img2, keypoints2, matches, img_matches);imshow("matches", img_matches);waitKey(0);
Find the homography transformation between two sets of points:
vector<Point2f> points1, points2;// fill the arrays with the points....Mat H = findHomography(Mat(points1), Mat(points2), CV_RANSAC, ransacReprojThreshold);
Create a set of inlier matches and draw them. Use perspectiveTransform function to map points with homography:
Mat points1Projected;perspectiveTransform(Mat(points1), points1Projected, H);
Use drawMatches() again for drawing inliers.
![Page 41: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/41.jpg)
Features2d contents
Detectors available• SIFT• SURF• FAST• STAR• MSER• GFTT (Good Features To Track)
Descriptors available• SIFT• SURF• One way• Calonder (under construction)
• FERNS
Detection: Description:
![Page 42: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/42.jpg)
Kalman Filter, Partical Filter for Tracking
Gary Bradski (c) 200842
Kalman Condensation or Particle Filter
::KalmanFilter class ConDensation
![Page 43: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/43.jpg)
Projections
Mat getAffineTransform()
Mat getPerspectiveTransform()
void warpAffine()
void warpPerspective()
Find: Warp:
![Page 44: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/44.jpg)
Homography• Maps one plane to another
– In our case: A plane in the world to the camera plane– Great notes on this: Robert Collins CSE486
• http://www.cse.psu.edu/~rcollins/CSE486/lecture16.pdf– Derivation details: Learning OpenCV 384-387
Gary Bradski, CS223A, Into to Robotics 44
Gary Bradski and Adrian Kaehler: Learning OpenCV
10
0
0
100
00
00
'
'
'
ZY
X
f
f
z
y
xZ
Xfy
Z
Xfx
Perspective Matrix Equation(camera coords Pt in world to pt on image)
CPMp int
![Page 45: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/45.jpg)
Homography
• We often use the chessboard detector to find 4 non-colinear points – (X,Y * 4 = 8 constraints)– To solve for the 8 homography parmeters.
• Code: Once again, OpenCV makes this easy– findHomography(…) or:– getPerspectiveTransform(…)
Gary Bradski, CS223A, Into to Robotics 45
![Page 46: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/46.jpg)
Single Camera CalibrationSingle Camera Calibration
Now, camera calibration can be done by holding checkerboard in front of the camera for a few seconds.
And after that you’ll get:
3D view of checkerboard3D view of checkerboard Un-distorted imageUn-distorted image
46Gary Bradski (c) 2008
See samples/cpp/calibration.cpp
![Page 47: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/47.jpg)
Stereo … Depth from Triangulation
• Involved topic, here we will just skim the basic geometry.
• Imagine two perfectly aligned image planes:
47
Depth “Z” and disparity “d” are inversly related:
![Page 48: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/48.jpg)
Stereo• In aligned stereo, depth is from similar triangles:
• Problem: Cameras are almost impossible to align• Solution: Mathematically align them:
48
rl
rl
xx
fTZ
Z
T
fZ
xxT
)(
All: Gary Bradski and Adrian Kaehler: Learning OpenCV
![Page 49: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/49.jpg)
Stereo Rectification• Algorithm steps are shown at right:• Goal:
– Each row of the image contains the same world points– “Epipolar constraint”
49
Result: Epipolar alignment of features:
All: Gary Bradski and Adrian Kaehler: Learning OpenCV
![Page 50: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/50.jpg)
samples/c
50
In ...\opencv_incomp\samples\c
bgfg_codebook.cpp - Use of a image value codebook for background detection for collecting objects
bgfg_segm.cpp - Use of a background learning engineblobtrack.cpp - Engine for blob tracking in imagescalibration.cpp - Camera Calibrationcamshiftdemo.c - Use of meanshift in simple color trackingcontours.c - Demonstrates how to compute and use
object contoursconvert_cascade.c - Change the window size in a recognition cascadeconvexhull.c - Find the convex hull of an objectdelaunay.c - Triangulate a 2D point clouddemhist.c - Show how to use histograms for recognitiondft.c - Discrete fourier transformdistrans.c - distance map from edges in an imagedrawing.c - Various drawing functionsedge.c - Edge detectionfacedetect.c - Face detection by classifier cascadeffilldemo.c - Flood filling demofind_obj.cpp - Demo use of SURF featuresfitellipse.c - Robust elipse fittinghoughlines.c - Line detectionimage.cpp - Shows use of new image class, CvImage();inpaint.cpp - Texture infill to repair imagerykalman.c - Kalman filter for trackignkmeans.c - K-Meanslaplace.c - Convolve image with laplacian.
letter_recog.cpp - Example of using machine learning Boosting, Backpropagation (MLP) and Random forestslkdemo.c - Lukas-Canada optical flowminarea.c - For a cloud of points in 2D, find min
bounding box and circle. Shows use of Cv_SEQ
morphology.c - Demonstrates Erode, Dilate, Open, Closemotempl.c - Demonstrates motion templates
(orthogonal optical flow given silhouettes)mushroom.cpp - Demonstrates use of decision trees (CART)
for recognitionpyramid_segmentation.c - Color segmentation in pyramidsquares.c - Uses contour processing to find squares
in an imagestereo_calib.cpp - Stereo calibration, recognition and disparity map computationwatershed.cpp - Watershed transform demo.
![Page 51: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/51.jpg)
samples/cpp Code of Possible use for Projects• Brief_match_test
– Use of fast det., brief descrp. ORB will replace. See video_homography.cpp
• Calibration (single camera)• Chamfer (2D edge matching)• Connected_components
– Using contours to clean up regions in images.
• Contours2 (finding and drawing)• Convexhull (finding in 2D)• Cout_mat – (print out Mat)• Demhist using calcHist()
– histograms and histogram normalization
• Descriptor_extractor_matcher– Use of features 2D detector descriptor– Also see matcher_simple.cpp
• Distrans – Use of the distanceTransform on edge
images and voroni tessel.
• Edge (Canny edge detection) CS324 51
• Ffilldemo (flood fill methods)• Filestorage (I/O of data structs)• Fitellipse (find contours, fit ellispe)• Grabcut (energy based segmentation)• Imagelist_creator (yaml or xml lists)
• Read using: starter_imagelist.cpp
• Kalman (Using the kalman filter)• Kinect_maps (using kinect in OpenCV) • Kmeans (using kmeans clustering)• Laplace (finding points/edges)• Letter_recog (machine learning)
• Use of Random trees, boosting, MLP
• Lkdemo (Lukas Kanada optical flow)• Morphology2 (erosion, dilation etc)• Multicascadeclassifier (rejection cascade)• Peopledetect (use of HOG)• Select3dobj (calc R and t from calib)• Stereo_* (stereo calib. and matching)• Watersed (segmentation algorithm)
![Page 52: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/52.jpg)
ML for Recognition
52Gary Bradski (c) 2008
![Page 53: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/53.jpg)
CLASSIFICATION / REGRESSIONCLASSIFICATION / REGRESSION(new) Fast Approximate NN (FLANN)(new) Extremely Random Trees(coming) LSHCARTNaïve BayesMLP (Back propagation)Statistical Boosting, 4 flavorsRandom ForestsSVMFace Detector(Histogram matching)(Correlation)
CLUSTERINGCLUSTERINGK-MeansEM(Mahalanobis distance)
TUNING/VALIDATIONTUNING/VALIDATIONCross validationBootstrappingVariable importanceSampling methods
Machine Learning Library (MLL)
AACBAABBCBCC
AAA
AACACB CBABBC
CCB
BCC
ABBCCB
BC A BBC
CBB
5353http://opencv.willowgarage.com
![Page 54: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/54.jpg)
K-Means, Mahalanobis
Gary Bradski (c) 2008 54
double kmeans() double Mahalanobis()
K-Means: •Choose K data points as cluster centers• While cluster centers change:
• Assign each data point to the closest center• If a cluster has no points, chose a random point from points far away from other cluster centers• Move the centers to the mean position of points in their cluster
![Page 55: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/55.jpg)
Patch Matching
Gary Bradski (c) 2008 55
void matchTemplate()
![Page 56: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/56.jpg)
Gesture Recognition
Up R L Stop OKGestures:
Meanshift Algorithm used to track, histogram intersection with gradient used to recognize.
Gesture via:Gesture via: Gradient h histogram* based gesture recognition with Tracking.
56Gary Bradski (c) 2008
*Bill Freeman
double compareHist()
![Page 57: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/57.jpg)
Boosting: Face Detection withBoosting: Face Detection withViola-Jones Rejection CascadeViola-Jones Rejection Cascade
57Gary Bradski (c) 2008
In samples/cpp, see:Multicascadeclassifier.cpp
![Page 58: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/58.jpg)
Machine learning• Good features often beat good algorithms
• Choose an operating point that trades off accuracy vs. cost
Gary Bradski (c) 2008 58
TP FN
FP TN
100%
100%
![Page 59: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/59.jpg)
59
Some project ideas: (feel free to steal, modify or ignore)1. Identify faces in (cellphone) pictures using facebook as database.
2. Use the (cellphone) camera to detect dangerous road events and or detect when someone is awake or sleeping (even with sunglasses on?) also in low light conditions.
3. Use webcam/cellphone to take pictures or videos of a room and then generate the floor plan.
4. Photograph or video a Jenga tower, and advise the player which is the safest block to remove.
5. Make a multiplayer game (if possible more than one computers/ cameras) based on CV.
6. Make an intuitive two handed UI for the OS (extra points for adding the use of facial gestures).
7. Do something with kinect (e.g. a golf game)
8. For engineers: make a paintball turret (e.g. http://www.paintballsentry.com/Videos.htm).
9. Make a security system with multiple cameras that records high quality portrait images and low quality video and alerts the presence suspicious people in real time (e.g. covered faces).
10. Use the camera to cheat/gain an advantage in real life interactions (sports, gambling)
11. Make a system (on the cellphone) that identifies/ classifies photographed objects (for example mushrooms)
![Page 60: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/60.jpg)
Questions?
![Page 61: OpenCV Tutorial](https://reader035.vdocuments.mx/reader035/viewer/2022062221/56813ae1550346895da33377/html5/thumbnails/61.jpg)
Useful OpenCV Links
6161
OpenCV Wiki:http://opencv.willowgarage.com/wiki
OpenCV Code Repository:svn co https://code.ros.org/svn/opencv/trunk/opencv
New Book on OpenCV:http://oreilly.com/catalog/9780596516130/
Or, direct from Amazon:http://www.amazon.com/Learning-OpenCV-Computer-Vision-Library/dp/0596516134
Code examples from the book:http://examples.oreilly.com/9780596516130/
Documentationhttp://opencv.willowgarage.com/documentation/index.html
User Group (44700 members 4/2011):http://tech.groups.yahoo.com/group/OpenCV/join