lab 2 - forsiden - universitetet i · pdf file๐= 27.3,−20.9, ... • samregistrer...
TRANSCRIPT
Lab 2
04.02.2016
Ressurser
โข OpenCV documentation: โ http://opencv.org/documentation.html
โข Eigen documentation : โ http://eigen.tuxfamily.org/dox/ โ Quick reference quide: http://eigen.tuxfamily.org/dox/group__QuickRefPage.html
โข C++:
โ http://en.cppreference.com/w/
โข Image Watch: An image debugger plug-in for Visual Studio โ Download directly from Visual Studio:
Tools Extensions and Updatesโฆ Online Search for โImage Watchโ
2
Litt om cv::Mat
3
Huskeliste
โข Lage prosjekt
โข Konstruere en cv::Mat โ Datatyper, lage ยซvinduerยป โ MatCommaInitializer
โข Hente og endre piksler
โ at<โฆ>() โ forEach
โข Regne pรฅ matriser
โ Operasjoner, MatExpr, โฆ
4
Perspektivkameramodellen
5
The perspective camera model
6
๐ถ
Vehicle
๐
๐
๐
Perspective Camera
World
๐๐๐
๐๐ถ
๐๐ถ๐
Point observed by the camera
๐ง๐ ๐ฆ๐
๐ฅ๐
๐ฅ๐
๐ฆ๐ ๐ง๐
๐ง๐ถ
๐ฅ๐ถ ๐ฆ๐ถ
๐ is a local NED coordinate system (North East Down)
๐ถ is a standard coordinate system for a camera ๐ฅ๐ถ - Right ๐ฆ๐ถ - Down ๐ง๐ถ - Forward
๐ is a standard coordinate system for a vehicle ๐ฅ๐ - Forward ๐ฆ๐ - Right ๐ง๐ - Down
The perspective camera model
โข The point ๐ โ Position ๐๐ = 27.3,โ20.9,โ11.5 ๐
โข The vehicle
โ 3m wide, 6m long โ The origin of ๐ is chosen to be at
the center of the vehicle, 1m above the ground
โ Pose relative to the world: ๐ฅ = 6.7๐ ๐ฆ = โ2.4๐ ๐ง = โ14.0๐ ๐๐๐๐ = 3.7ยฐ ๐๐๐๐๐ = โ9.3ยฐ ๐๐๐๐๐๐๐ = 307.6ยฐ
โข The camera โ The optical center is 2m in front of
and 1m to the left of the vehicles center
โ The optical center is 4m above ground
โ The y-axis of ๐ถ is perpendicular to the xy-plane of ๐
โ The optical axis, i.e. the z-axis of ๐ถ , is rotated 4.7ยฐ to the right of the x-axis of ๐
โ The camera calibration matrix is
๐พ =1028 0 400
0 1028 3000 0 1
7
The perspective camera model
Problem In which pixel of the image will we observe the point ๐? Sub-problems 1. Represent ๐๐๐ as a SE(3) object ๐๐๐ 2. Represent ๐๐ถ๐ as a SE(3) object ๐๐ถ๐ 3. Represent ๐๐ถ๐ as a SE(3) object ๐๐ถ๐ 4. Determine the camera matrix ๐ = ๐พ ๐ ๐ 5. Determine the pixel ๐ = ๐ข,๐ฃ that the
point ๐ projects to according to the perspective camera model
8
Sub-problem 1
Represent ๐๐๐ as a SE(3) object ๐๐๐ โข Sophus::SE3d objects can be initialized with a Eigen::Matrix3d rotation matrix ๐ and a
Eigen::Vector3d translation vector ๐ โข Roll, pitch, heading relates to the zyx-rotation sequence, so ๐ = ๐ ๐ง๐ ๐ฆ๐ ๐ฅ โข A basic rotation matrix like ๐ ๐ฅ ๐ can be created by
Eigen::AngleAxisd(theta * M_PI / 180, Eigen::Vector3d::UnitX())
9
// Visualization of world and vehicle cv::Mat cv_t_W_V, cv_R_W_V; cv::eigen2cv(t_W_V, cv_t_W_V); cv::eigen2cv(R_W_V, cv_R_W_V); cv::Affine3d cv_T_W_V(cv_R_W_V, cv_t_W_V); cv::viz::Viz3d my_window("window 1"); my_window.showWidget("World-axes", cv::viz::WCoordinateSystem(3.0)); my_window.showWidget("vehicle-axes", cv::viz::WCoordinateSystem(3.0), cv_T_W_V); my_window.showWidget("vehicle", cv::viz::WCube(cv::Vec3d(-3.0, -1.5, -1), cv::Vec3d(3.0, 1.5, 1.0)), cv_T_W_V); my_window.spin();
Sub-problem 2 and 3
Represent ๐๐ถ๐ as a SE(3) object ๐๐ถ๐ โข Which basic rotations must ๐ undergo in order to coincide with ๐ถ ? โข Two basic rotations is enough Represent ๐๐ถ๐ as a SE(3) object ๐๐ถ๐ โข Recall that ๐๐ถ๐ = ๐๐๐ ๐๐ถ๐
10
// Visualization of camera frustum cv::Mat cv_t_W_C, cv_R_W_C, cv_K; cv::eigen2cv(K, cv_K); cv::eigen2cv(T_W_C.translation(), cv_t_W_C); cv::eigen2cv(T_W_C.rotationMatrix(), cv_R_W_C); cv::Affine3d cv_T_W_C(cv_R_W_C, cv_t_W_C); my_window.showWidget("camera_frustum", cv::viz::WCameraPosition(cv_K, 1.0, cv::viz::Color::red()), cv_T_W_C);
Sub-problem 4 and 5
Determine the camera matrix ๐ = ๐พ ๐ ๐ โข Recall that in the perspective camera model ๐ = ๐ ๐๐ถ and ๐ = ๐๐๐ถ , so we can not read ๐ and
๐ directly from ๐๐ถ๐ Determine the pixel ๐ = ๐ข,๐ฃ that the point ๐ projects to according to the perspective camera model โข Recall that ๐๏ฟฝ = ๐๐ฟ๏ฟฝ
11
// Visualization of point (as a small sphere) my_window.showWidget("Q", cv::viz::WSphere({ Q_W(0), Q_W(1), Q_W(2) }, 0.1));
Laplace blending
12
Steg 1: Lag nytt prosjekt og vis frem bilder
โข Kopier ยซopencv_project_templateยป og gi nytt navn โ Husk รฅ endre ยซPROJECT_NAMEยป i CMakeLists.txt
โข Lag nytt prosjekt med Cmake
โข Skriv et program som leser to bilder
โ img_1: free_cat.jpg โ img_2: free_tiger.jpg โ Bildene bรธr konverteres til flyttallsbilder:
cv::imread(โฆ).convertTo(img_1, CV_32F, 1.0/255.0);
โข Vis bildene frem โ cv::namedWindow() โ cv::imshow()
13
Steg 2: Enkel blanding av bilder
โข Samregistrer bildene ved รฅ angi tre punktkorrespondanser
โข Lag maske med rampe
14
cv::Point2f pts_1[] = {{321, 200}, {647, 200}, {476, 509}}; cv::Point2f pts_2[] = {{441, 726}, {780, 711}, {615, 1142}}; cv::Mat trans_mat = cv::getAffineTransform(pts_2, pts_1); cv::warpAffine(img_2, img_2, trans_mat, img_1.size());
cv::Mat mask = cv::Mat::zeros(img_1.size(), CV_32FC1); cv::rectangle(mask, cv::Rect{img_1.cols/2, 0, img_1.cols/2 + 1, img_1.rows}, 1, CV_FILLED); cv::blur(mask, mask, cv::Size{3, 3});
Steg 2: Enkel blanding av bilder
โข Lag funksjon som gjรธr enkel blanding av to bilder vektet med masken
โ Tips: cv::blendLinear()
โข Bruk funksjonen og vis frem resultatet โ Prรธv med forskjellige masker
โข Andre sรธmmer, sirkler, stรธrre glattefilter
15
cv::Mat linear_blend(cv::Mat& img_1, cv::Mat& img_2, cv::Mat& mask)
Steg 3: Laplaceblanding
โข Lag funksjonen โ cv::pyrDown()
16
std::vector<cv::Mat> construct_gaussian_pyramid(cv::Mat& img)
Steg 3: Laplaceblanding
โข Lag funksjonen โ Bruk construct_gaussian_pyramid โ cv::pyrUp()
17
std::vector<cv::Mat> construct_laplacian_pyramid(cv::Mat& img)
Steg 3: Laplaceblanding
โข Lag funksjonen โ cv::pyrUp()
18
cv::Mat collapse_pyramid(std::vector<cv::Mat>& pyr)
Steg 3: Laplaceblanding
โข Lag funksjonen โ For eksempel slik:
19
cv::Mat laplace_blending(cv::Mat& img_1, cv::Mat& img_2, cv::Mat& mask)
std::vector<cv::Mat> mask_pyr = construct_gaussian_pyramid(mask); std::vector<cv::Mat> img_1_pyr = construct_laplacian_pyramid(img_1); std::vector<cv::Mat> img_2_pyr = construct_laplacian_pyramid(img_2); std::vector<cv::Mat> blend_pyr(img_1_pyr.size()); for (int i = 0; i < img_1_pyr.size(); i++) { // TODO: Perform linear blend on each level. } return collapse_pyramid(blend_pyr);
Steg 3: Laplaceblanding
โข Bruk laplace_blending(img_1, img_2, mask) โ Vis frem resultat โ Sammenlign med enkel blending
20
Steg 4: Moroplukk
โข Prรธv andre bilder โ Ta bilder med kameraet โ Finn bilder pรฅ nett โ Angi nye punkter for samregistrering
โข Andre masker
โ Last ned GIMP for รฅ tegne finere masker
21
Steg 5: Dypdykk
โข Implementer pyramiden selv โ Ikke bruk cv::pyrDown() eller cv::pyrUp()
โข Ta en titt pรฅ cv::seamlessClone()
โข Prรธv รฅ implementere warpingen selv
22