cross-calibration of rgb and thermal cameras with … of rgb and thermal cameras with a lidar...
TRANSCRIPT
Cross-Calibration of RGB and Thermal cameras with a LIDAR
Aravindhan K KrishnanArizona State University
Benjamin StinnettArizona State University
Srikanth SaripalliArizona State University
Abstract— In this paper we present a method for combiningdata from an RGB camera, a long-wave-infrared (LWIR) ther-mal camera, and a LIDAR to obtain a registered RGB-depth-thermal (RGBDT) point cloud. Our method assumes that theintrinsic parameters of the cameras are known. For geometriccalibration of the thermal camera, we discuss the calibrationtarget used. For cross-calibration, our method requires that thecalibration targets are placed in the field-of-view of the camerasand the LIDAR. The edges of the calibration target are detectedin the images and the point clouds. The extrinsic paramters (i.e.rotation and translation) between the camera and the LIDARare determined by finding the transformation that aligns theprojected 3D edge points to the edges in the image. We presentresults of the method on one indoor and one outdoor dataset wecollected. We illustrate the convexity of the objective functionand discuss the convergence of the algorithm.
I. INTRODUCTION & RELATED WORK
Metric maps constructed with LIDAR have been theprimary sensing module in many outdoor robot applica-tions [17], [6]. Alternatively, vision only systems have re-sulted in applications such as appearance based naviga-tion [3]. However, metric maps from LIDAR lack texture andvision only systems suffer from illumination changes. Witheach approach having its own weaknesses, it is evident thatmulti-modal sensing is essential for robot autonomy underall weather conditions.
Merging data from RGB cameras and LIDAR to createRGBD maps is not new. But there has not been muchwork on fusing thermal data with LIDAR. It presents newapplications like navigation and scene understanding in thedark. For instance consider the Thermal-LIDAR data shownin Figure 1. A standalone RGB camera does not enable theidentification of a human being in the dark, but the thermalgradient produced by a thermal camera allows such detection.
To merge data from the LIDAR and the camera, theextrinsic parameters (i.e. rotation and translation parameters)between the LIDAR and the camera should be known. Typi-cally, finding the extrinsics is formulated as an optimizationproblem that computes the transformation (T) between thecamera co-ordinate frame and the LIDAR co-ordinate frame.The first approach to camera-LIDAR calibration uses cali-bration targets similar to camera calibration. The calibrationtarget is placed in the field-of-view (FOV) of the cameraand the LIDAR. The equation of the plane (in the LIDARco-ordinate frame) containing the planar calibration target iscomputed by segmenting the LIDAR point cloud [18]. Fora 2D LIDAR a line is used instead of a plane [11]. The
Fig. 1. Thermal data merged with LIDAR. The person has a differenttexture in the dark compared to the surroundings due to the thermal gradientproduced by the camera. Yellow regions have high intensity and red regionshave low intensity in this colormap.
equation of the plane in the camera co-ordinate frame iscomputed from the extrinsics between the camera frame andthe world frame (which is defined on the calibration target).The required transformation T is obtained by aligning theplane (or the line in [11]) in the LIDAR frame to the planein the camera frame. Recently, a similar approach ‘singleshot calibration’ has been presented in [5] . The toolboxthat implements ‘single shot calibration’ also provides anapproach to detect checkerboards corners automatically. Thetoolbox requires that many checkerboards are placed in thescene at once, rather than having to collect multiple scans /images of the checkerboards placed in different orientations.The above approaches rely on the plane equations of thecalibration target obtained from the camera calibration tocompute the cross-calibration parameters; hence are notsuitable for online calibration (i.e. calibration from naturalscenes and not using a calibration target).
Another approach is to find the pose of the camerain the LIDAR frame using the perspective-from-n-points(PnP) algorithm as described in [14]. Corresponding pointsbetween the camera image and the LIDAR point cloud areselected by the user using an intermediate Bearing AngleImage (derived from the point cloud).
Few other recent approaches to camera-LIDAR calibrationassume that the camera is calibrated and the intrinsicsof the camera are known. The LIDAR point cloud (withintensity values) is projected to the camera frame using thecamera intrinsics resulting in an intensity image. The cross-calibration problem is then formulated as an optimizationproblem that finds the transformation (T) which aligns the
LIDAR intensity image with the camera image. Two differentoptimization functions found in the literature are (i) Mutualinformation between the intensity distributions of the twoimages (ii) Mismatch in the edge alignment of the twoimages.
Mutual information methods require multiple scans to geta convex optimization function [10]. A related approachusing Normalized Mutual Information is presented in [15].These methods are targetless and are suitable for onlinecalibration. However, the mutual information metric is notvery useful when a high resolution image (e.g. 3 MP image)is used with a sparse point cloud containing a few thousandpoints. The few thousand points, when projected back to theimage, contribute only to a fraction of pixels in the intensityimage and the other pixels have to be interpolated [9]. So,an averaging of information occurs before the mutual infor-mation is maximized. This can result in poor convergence.Many recent papers use a Velodyne LIDAR where the pointcloud is dense making this approach useful. However, itbecomes impractical while using 2D LIDARs such as theRIEGL Q240i (by the current authors) and Hokuyo.
An edge alignment approach is presented in [8] wherethe edges in the LIDAR point cloud are projected on to theimage. The optimal parameter (T
opt
) is determined from aninitial estimate (T) by a brute force search in a discrete setof points P 2 [T–�, T+�] that projects the edges in theLIDAR data to the edges in the image. The assumption hereis that the edges in the point cloud which arise out of depthdiscontinuities have corresponding edges in the image. Avery good initial estimate of the parameters is required forthis solution to work as one can easily get stuck in a localminima with a local brute force search.
A new metric for optimization called Gradient OrientationMeasure (GOM) is presented in [16]. The point cloud data isprojected on to a unit sphere and the gradient at each point iscomputed. The error metric is defined as the dot product ofthe gradients in the projected image and the camera image.The authors present this work as a targetless calibrationtechnique.
Thermal cameras are used in various applications suchas surveillance, building inspection, gas detection, humandetection etc, but are rarely used in the context of 3Dmapping. A survey of the applications of thermal camerascan be found in [4]. 3D Mapping using stereo thermalcameras is described in [12] and monocular SLAM usinga thermal camera is presented in [19]. Nuchter et al presenttheir work on integrating thermal camera and a LIDAR in[2]. RGB-depth-thermal (RGBDT) data provides a rich setof applications to autonomous systems such as pedestriandetection, change detection, localization etc. And with thecost of thermal cameras going down in recent years, itpresents a new mode of sensing for future robotic systems.
Mutual information is not useful in calibrating a thermalcamera with a LIDAR. Thermal cameras measure the emittedradiation in long wave infrared spectrum (7µ – 14µ) anda LIDAR intensity image is a measure of the reflectedradiation in near infra red (0.7µ – 1µ); these can be vastly
different. Whereas, in the case of color cameras the measuredreflectivity in the visible spectrum (0.4µ – 0.7µ) is closer tothe near infra red spectrum (0.7µ – 1µ) and the reflectivityvalues are assumed to have a strong correlation, makingmutual information metrics useful.
Techniques like GOM are not applicable for the cross-calibration of a thermal camera and a LIDAR unless commonedges are induced in the spectrum of the LIDAR andthe camera. So a targetless cross-calibration of a thermalcamera is highly scene dependent, making online calibrationa difficult task.
The goal of this paper is to present a unified approachfor cross-calibrating an RGB camera, a thermal camera anda LIDAR. We present a solution that treats this problemas two separate instances of camera-LIDAR calibration. Wealso discuss convergence when the initial estimate for cross-calibration is not accurate (which is a common assumptionin literature). Initially we describe the approach to find theextrinsics between the RGB camera and the LIDAR. Laterwe extend this approach to find the extrinsics between thethermal camera and the LIDAR. We describe the design ofthe calibration targets and the required setup. Finally, wediscuss the results.
RGB$camera$
Thermal$camera$ LIDAR$
Fig. 2. Setup containing a LIDAR, an RGB camera and a thermal camera.
We use a 3MP Proscillica GT 1920 color camera, FLIRTau2 (long-wave-infrared, uncooled core, 640 ⇥ 480 reso-lution) thermal camera and a RIEGL Q240i LIDAR. TheRIEGL Q240i is a 2D scanner giving 10000 points persecond. To get 3D point clouds the scanner is rotated usinga Robotis DXLPRO motor which gives ⇠1400 encoderreadings per degree. We assume that there are no errors inthe encoder readings and the scene remains static while onepoint cloud is obtained. The cameras are rigidly mounted tothe frame containing the LIDAR. The system is shown inFigure 2.
II. METHOD
We integrate the encoder readings from the motor and the2D scan from the LIDAR to obtain a 3D point cloud. Theinputs to the cross-calibration algorithm are the 3D pointclouds and the camera images of the calibration target. In thissection, we present cross-calibration of an RGB camera anda LIDAR initially. Later we describe the intrinsic calibration
of the thermal camera, followed by the cross-calibration ofthe thermal camera and the LIDAR.
Cross-calibration of the RGB camera and the LIDAR:We assume that the camera is calibrated already and the
intrinsics are known. A checkerboard of size 4 ft⇥ 3ft isused as the calibration target. The checkerboard is placed indifferent orientations in the common FOV of the LIDAR andthe RGB camera. The extrinsics are computed by finding atransformation that aligns the edges of the calibration targetin the LIDAR point cloud to the edges of the calibrationtarget in the image. The optimization function is shownbelow.
T
optimal
= argminT
X
j
X
i
||Icj
�K ⇤ T ⇤ Ei
j
|| (1)
Here, the index j refers to the ‘image $ point cloud’ paircontaining the same view of the calibration target. Ei
j
are theedge points of the calibration target in the j
th point cloud.K is the camera matrix containing the intrinsic parameters.I
c
j
is the closest edge point in the j
th image, when E
i
j
isprojected on to the camera.
Fig. 3. Calibration target used for Thermal and RGB Camera Calibration.The target was produced by cutting squares of black and white melaminein a laser cutter and gluing them alternatively on a board. First, a blackrectangular frame is cut and glued to the board – which acts as a border.The squares are then glued inside, starting from one of the corners, ensuringthat they are placed tightly without any gaps in between. Because of thecolor contrast, the same target is used for RGB camera calibration too.
Fig. 4. Image of the calibration target in the thermal camera. The intensitycontrast was good enough to detect the checkerboard corners.
1) Detecting edges of the calibration target in the image:Initially, all edges in the image are computed using a Sobelfilter. Then, the user selects the corners of the calibrationtarget in the image. One example of user selection in shown
in Figure 5(a); the green points are selected by the user. Theequation of the lines (shown in red) connecting the cornersare then computed. Edge pixels which lie within a 5⇥5window of any point on these lines are taken as the edgepixels of the calibration target. The user involvement here isvery minimal; it is easy, doesn’t take time and the approachworks very well.
2) Detecting edges of the calibration target in the pointcloud: The plane containing the calibration target is seg-mented from the point cloud and the boundary of thissegment is computed. These boundary points form theedge points of the calibration target. The detailed algorithmpipeline is described step-by-step below.
a. A region growing segmentation from an initial seedis used to segment the calibration target. Initially thesegment contains only the initial seed. The neighbouringpoints whose normals satisfy an angle threshold areadded to the segment. The neighbours of neighboursare then added recursively until no more points are leftto add. The resulting set of points correspond to thesegmented calibration target.
b. To compute the initial seed for segmentation, we makeuse of the checkerboard corners in the correspondingRGB image. The point cloud is projected on to theimage and the points falling within a 20⇥20 windowof any checkerboard corner is considered as a potentialpoint on the calibration target. A plane is fit on thesepotential points using RANSAC, discarding the outliers.The point in the point cloud that is closest to thecentroid of the inliers gives the seed for region growingsegmentation.
c. A boundary estimation algorithm is used to find theboundary of the segment computed above. To improvethe accuracy of the boundary estimation, the segmentis smoothed initially using the ‘Moving Least Squares’algorithm. We know that the calibration target is aplanar object and any point not on the plane is due tosensor noise. We observed that the smoothing reducessensor noise and improves boundary estimation.
d. A few outliers (points in the interior of the segment)still remain after step c. So we compute the concavehull of the points estimated in step c. This removes theall the outliers and we get the boundary points of thecalibration target.We used the PCL [13] implementations for boundaryestimation, moving least squares, and concave hull.
3) Optimization: The optimization function in Equation 1is implemented as a distance transform of the edges com-puted in Section II-A.1. For an edge image, the distancetransform of a pixel gives the distance to the closest edgepixel. When the 3D point is projected to a pixel (x,y) in theimage, the corresponding distance transform at (x,y) givesthe distance to the closest edge pixel in the image. Whenthe distance transform is computed only on the edges of thecalibration target, it is precisely the optimization functionin Equation 1. An example distance transform is shown inFigure 5(b).
(a) User selecting the corners of the calibrationtarget. The selected points are shown in green.
(b) Distance transform of the edges of the calibra-tion target. The image gets brighter towards theboundary.
Fig. 5. Calibration setup and the distance transform.
One can notice that the distance transform looks convexfor most part as the distance to an edge pixel grows graduallytowards the boundary of the image. This ensures a widerconvergence basin and increases the error tolerance of theinitial estimate (T).
The distance transform is computed using OpenCV andthe Leverberg-Marquardt algorithm (from Ceres solver [1])is used for optimization.
Since the jacobian for the optimization function in Equa-tion 1 cannot be computed analytically, we used numericaldifferentiation to compute the gradients. The central differ-ence method for computing the gradient at a point T isgiven by f(T+�)�f(T��)
2� , where � is usually chosen tobe ⇠1e-6. But the optimization function in Equation 1 is adistance transform and both f(T ) and f(T + �) result inthe same pixel x, when � =1e-6, producing a zero gradient.This causes the optimization algorithm to terminate early. Toovercome this, we fit a continuous surface, parameterizedas a second degree polynomial on the distance transformvalues for a 10⇥10 window centered around the pixel x. Withthis surface approximation, the central difference method forcomputing gradients is possible and the optimizer converges.The surface is computed only on pixels onto which theLIDAR points are projected. The surface parameters arestored in a look-up table to avoid re-computation on futureiterations of the optimization.
A. Geometric calibration of the Thermal camera
We made a calibration target that appears like a checker-board pattern in the thermal image. The calibration target wasproduced using black and white melamine, a resin-infusedpaper that is laminated to a medium density fiberboard. Aborder for the target was laser cut from the black melamine,and squares were laser cut from both the black and whitemelamine to fill the border with a checkerboard pattern ofalternating black and white squares. When placed in thesun, the different absorptions of the white and black squarescreates the thermal gradient needed for camera calibration.
We observed that placing the target in the sun for 60minutes gives sufficient thermal gradient to detect corners.The thermal image of the calibration target shown in Figure 4was obtained when the temperature was ⇠77 F (25 C).
For detecting the corners and finding the intrinsic param-eters of the thermal camera, we used the calibration toolboxdescribed in [7].
B. Cross-Calibration of the Thermal camera and the LIDAR
We adopt the same procedure used for calibrating the RGBcamera. We used the target described in the previous section.Again, the user selects the corners of the target from thethermal images and the edges are computed from the givencorners. A distance transform of this edge image is computedand the optimization described in Section II-A.3 gives theextrinsics.
III. RESULTS & DISCUSSION
Detecting the edges of the calibration target in each pointcloud takes ⇠15 seconds. Computing the distance transformof the edges in the image takes time on the order ofmilliseconds. The optimization takes 4–5 minutes with initialestimates that are off by around 30�.
The convergence improves as we increase the numberof images and scan pairs. Figures 8(a) and 8(b) show thevariation of the objective function as we vary one parameter(roll, pitch, yaw, tx, ty, and tz) while keeping others constant.One can notice that the function becomes smooth as thenumber of scans are increased. The smooth function helpsconvergence even with bad initial estimates. For instance,the results in Figure 6(b) and 6(d) were obtained after acalibration which was initialized with an error of 30� inroll and 40� in yaw. Another textured point cloud obtainedoutdoors in shown in Figure 7. We also plotted the objectivefunction by varying two parameters and keeping othersconstant. The surfaces look convex for most part as can beseen from Figures 9(a) through 9(i).
The error tolerance to the initial estimate depends on thenumber of 3D points that are projected within the boundsof the distance transform image. We observed convergencewhen atleast 50% of the points are projected on the distancetransform image. This limitation can be overcome by inter-polating the distance transform values for points that falloutside the bounds of the image. But, we assume that theinitial estimates cannot be that bad.
(a) An RGB image. (b) Pointcloud textured with the RGBimage.
(c) Thermal image with a colormap.Yellow - high intensity, red - lowerintensity.
(d) Pointcloud textured with thermalimage.
Fig. 6. A point cloud of the same scene textured with RGB and thermal data.
The quality of the calibration is given by the reprojectionerror – which is the mean of the distance transform valuesof the projected 3D points. We choose a subset of theimage $ pointcloud pairs for calibration and compute thereprojection error on the remaining pairs. The reprojectionerror for the thermal camera (resolution 640 ⇥ 480) is 0.9pixels. The reprojection error for the RGB camera (resolution1936 ⇥ 1456) is 0.23 pixels. The reprojection error for thethermal camera was higher because some 2D edges of thecalibration target in the thermal image were indiscernibledue to similar thermal intensity of some objects in thebackground. However, the 3D edge points in the pointcloudwere detected and some of the projected 3D edge pointscontributed a lot to the reprojection error. This problem canbe overcome by carefully choosing the calibration site infuture trials.
The method presented in this paper can also be used in thecontext of online calibration. The edges in the entire scenehave to be computed instead of the edges of the calibrationtarget. We intend to use the edge detection method describedin [8] for detecting edges in the LIDAR point cloud. Usingour method directly on natural scenes will depend on theaccuracy of the initial guess (as it is for any online calibra-tion method). Nevertheless, this method presents a unifiedsolution for both initial and online calibration.
Fig. 7. An RGB textured point cloud taken outdoors.
The extrinsics between the thermal camera and the RGBcamera can be computed from the thermal-LIDAR extrinsicsand the RGB-LIDAR extrinsics. The extrinsics can alsobe computed by calibrating the thermal-RGB stereo pair(since we are using the same calibration target for both). Acomparison of the extrinsics will demonstrate the accuracyof the cross-calibration. We plan to do this in the future.
IV. FUTURE WORKOur immediate future work is to test this method for online
calibration. Later we plan to build real time maps usingRGBDT data and look at terrain traversability for field robotsand detect landing sites for Unmanned Aerial Vehicles.
V. CONCLUSIONSWe presented an approach to cross-calibrate RGB and
thermal cameras with a LIDAR. To the best of the authors’knowledge this is the first work that integrates RGB andthermal cameras with a LIDAR. We presented a methodusing calibration targets and describe how it can be applied toonline calibration in natural scenes. Results for the methodare shown on datasets we collected. In the future we areinterested in building real time RGBDT maps from themerged data.
VI. ACKNOWLEDGEMENTSWe would like to thank Sai Vemprala and Yucong Lin
for their help with collecting data. We thank Sai Vemprala,Shatadal Mishra, Andres Mora, and Adrian Carrio for theirfruitful discussions on the calibration target for the thermalcamera. We also thank Zohaib Mian, Dennis Ding, andAndrzej Banaszuk at United Technologies Research Centerfor their support and valuable feedback.
0 10 20 30 40 50 600
1
2
3
4
5
6
7
8 x 104
Varying parameters roll, pitch, yaw, tx, ty, tz
Obj
ectiv
e fu
nctio
n Va
lue
RollPitchYawTxTyTz
(a) 2 scans were used to create the above plot. The function is notsmooth.
0 10 20 30 40 50 600
0.5
1
1.5
2
2.5
3
3.5
4
4.5 x 105
Varying parameters roll, pitch, yaw, tx, ty, tz
Obj
ectiv
e fu
nctio
n Va
lue
RollPitchYawTxTyTz
(b) 10 scans were used to create the above plot. The function becomessmooth as the number of scans are increased. This ensures that theoptimizer does not get stuck in local minimas.
Fig. 8. The variation of parameters roll, pitch, yaw, tx, ty, and tz around the global minimum. Roll, pitch, and Yaw were varied by -30 degrees to +30degrees around the global minimum. Tx, Ty, and Tz were varied by -3m to +3m around the global minimum. The Y axis shows the objective functionvalues. One can notice that the function seems to be largely convex and there are no steep local minimas.
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
8
x 104
RollPitch
(a) Roll vs Pitch
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
x 104
PitchYaw
(b) Pitch vs Yaw
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
8
x 104
RollYaw
(c) Roll vs Yaw
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
8
x 104
TxTy
(d) Tx vs Ty
0
10
20
30
40
50
60 010
2030
4050
60
0
2
4
6
8
x 104
TzTy
(e) Ty vs Tz
0
10
20
30
40
50
60 010
2030
4050
60
0
2
4
6
8
x 104
TzTx
(f) Tx vs Tz
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
8
x 104
RollTx
(g) Roll vs Tx
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
8
x 104
PitchTy
(h) Pitch vs Ty
010
2030
4050
60
010
2030
4050
600
1
2
3
4
5
6
7
x 104
PitchTz
(i) Yaw vs Tz
Fig. 9. Plotting the objective function by varying two parameters simultaneously. In each combination the objective function doesn’t have any steep localminimas and looks convex for the most part. For all the plots, the rotation parameters are varied by -30 to +30 degrees around the global minimum andthe translation parameters by -3m to +3m around the global minimum.
REFERENCES
[1] Sameer Agarwal, Keir Mierle, et al. Ceres solver, 2012.[2] Dorit Borrmann, Hassan Afzal, Jan Elseberg, and Andreas Nuchter.
Mutual calibration for 3d thermal mapping. In Proceedings of the10th International IFAC Symposium on Robot Control (SYROCO12),volume 10, 2012.
[3] Mark Cummins and Paul Newman. Probabilistic appearance basednavigation and loop closing. In Robotics and automation, 2007 IEEEinternational conference on, pages 2042–2048. IEEE, 2007.
[4] Rikke Gade and Thomas B Moeslund. Thermal cameras and appli-cations: a survey. Machine vision and applications, 25(1):245–262,2014.
[5] Andreas Geiger, Frank Moosmann, Omer Car, and Bernhard Schuster.
Automatic camera and range sensor calibration using a single shot. InRobotics and Automation (ICRA), 2012 IEEE International Conferenceon, pages 3936–3943. IEEE, 2012.
[6] Erico Guizzo. How googles self-driving car works. IEEE SpectrumOnline, October, 18, 2011.
[7] Abdallah Kassir and Thierry Peynot. Reliable automatic camera-lasercalibration. In Proceedings of the 2010 Australasian Conference onRobotics & Automation. ARAA, 2010.
[8] Jesse Levinson and Sebastian Thrun. Automatic online calibration ofcameras and lasers. In Robotics: Science and Systems, 2013.
[9] Ashley Napier, Peter Corke, and Paul Newman. Cross-calibration ofpush-broom 2d lidars and cameras in natural scenes. In Robotics andAutomation (ICRA), 2013 IEEE International Conference on, pages3679–3684. IEEE, 2013.
[10] Gaurav Pandey, James R McBride, Silvio Savarese, and Ryan MEustice. Automatic targetless extrinsic calibration of a 3d lidar andcamera by maximizing mutual information. Proceedings of the AAAINational Conference on Artifical Intelligence, 2012.
[11] Robert Pless and Qilong Zhang. Extrinsic calibration of a camera andlaser range finder. In IEEE International Conference on IntelligentRobots and Systems (IROS), volume 13, pages 150–154, 2004.
[12] Arturo Rankin, Andres Huertas, Larry Matthies, Max Bajracharya,Christopher Assad, Shane Brennan, Paolo Bellutta, and Gary WSherwin. Unmanned ground vehicle perception using thermal infraredcameras. In SPIE Defense, Security, and Sensing, pages 804503–
804503. International Society for Optics and Photonics, 2011.[13] Radu Bogdan Rusu and Steve Cousins. 3d is here: Point cloud library
(pcl). In Robotics and Automation (ICRA), 2011 IEEE InternationalConference on, pages 1–4. IEEE, 2011.
[14] Davide Scaramuzza, Ahad Harati, and Roland Siegwart. Extrinsicself calibration of a camera and a 3d laser range finder from naturalscenes. In Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJInternational Conference on, pages 4164–4169. IEEE, 2007.
[15] Zachary Taylor and Juan Nieto. Automatic calibration of lidar andcamera images using normalized mutual information. In Robotics andAutomation (ICRA), 2013 IEEE International Conference on, 2013.
[16] Zachary Taylor, Juan Nieto, and David Johnson. Automatic calibrationof multi-modal sensor systems using a gradient orientation measure. InIntelligent Robots and Systems (IROS), 2013 IEEE/RSJ InternationalConference on, pages 1293–1300. IEEE, 2013.
[17] Sebastian Thrun, Mike Montemerlo, Hendrik Dahlkamp, DavidStavens, Andrei Aron, James Diebel, Philip Fong, John Gale, MorganHalpenny, Gabriel Hoffmann, et al. Stanley: The robot that won thedarpa grand challenge. Journal of field Robotics, 23(9):661–692, 2006.
[18] Ranjith Unnikrishnan and Martial Hebert. Fast extrinsic calibration ofa laser rangefinder to a camera. 2005.
[19] Stephen Vidas and Sridha Sridharan. Hand-held monocular slam inthermal-infrared. In Control Automation Robotics & Vision (ICARCV),2012 12th International Conference on, pages 859–864. IEEE, 2012.