fitting lines to data with outliers and errors in the variables nahum kiryatifreddy bruckstein tau...
Post on 21-Dec-2015
222 views
TRANSCRIPT
Fitting Lines to Data with Outliers and Errors in the Variables
Nahum Kiryati Freddy Bruckstein
TAU Technion
© Copyrighted material. All rights reserved.
Least Squares
• Maximum Likelihood• No errors in x• Errors in y• Errors are
– independent
– identically distributed
– Gaussian
• Analytic solution
From Maximum-Likelihood to Least-Squares (reminder)
),(|)},({ Probmaxarg),(
bmyx iibm
2
2
),( 2
)( expmaxarg
i
ii
ibm
bmxy
i i
ii
bm
bmxy2
2
),( 2
)(minarg
i
iibm
bmxy 2
),()(minarg
Maximum Likelihood:
Independent Gaussian errors:
log is a monotonic function:
Identically distributed errors:
... Least Squares
• Maximum Likelihood• Errors in x and y• Errors are
– independent & identically distributed
> between x and y (isotropic)
> between points– Gaussian
• Analytic solution
Heteroscedastic Errors
• Errors are independent
- between points
- between x and y
• Errors are not identically distributed
- between points
- between x and y
• No analytic solution
Possible reason: A different measurement device is used for each point.
Heteroscedastic & Correlated Errors
• Errors are independent between points
• Errors are correlated between x and y
• Errors are not identically distributed
• No analytic solution
Correlation can appear if the measurement instruments for x and y are both sensitive to a common environmental factor.
OutliersEven a single outlier can pull the line arbitrarily away from its “true” position
MATLAB “robustdemo”
Robust Approaches
• Least Median of Squares
• Robust M-estimation
• Reweighted Least-Squares
• etc...
Robust line fitting methods are far from ideal:
global optimization / local minima / computational complexity
Failure of Reweighted Least Squares (Matlab’s robustdemo)
(and forget about heteroscedasticity and correlation).
Detecting Lines in Edge Images
Unique characteristics:
• Many data points (~10,000)
Algorithm must be fast!
• Majority of outliers
Robustness is everything!
• Small location errors
Fitting the “good” points
is not a big issue.
Lines Tangent to a Circle
It is easy to assign a cost (in parameter space) to lines with isotropic fitting errors!
M-Estimation
• A classical approach to robust regression
• Limits the influence of outliers
• Leads to a nasty global optimization problem
• Usually: local optimization from “a good initial guess”
• Look for a good initial guess…
M-Estimation via Parameter Space
… and look for minimum in paramater space
… and look for maximum in paramater space
save votings
Formally, once we have shown that
2
22
),(2 sincos
sincos s.t.
)()( min
,
ii
ii
iiiiii
yx
YX
yYxXYX
ir
any (nasty) line fitting problem with isotropic cost can be (nicely) solved in parameter space:
sincosminarg)],([minarg),(),(
iiii
yxiii CrC
e.g., TLS & robust TLS, via selection of Ci (.) :
(Distance between data point i and a given line)
Surprise! The (very ugly) correlated heteroscedastic case can also be beautified!
sincos s.t.
),(min
,
1
2
2
),(
2
ii
ii
ii
yxy
xyxiiii
ii
YX
yY
xXyYxX
ii
ii
YX
id
)(
),(
cossin2sincos
sincos
2
2222
2
i
i
xyyx
ii
v
ryx
iii
Nominator: standard TLS term! Denominator: Rho-independent term!
(effective distancebetween data point iand a given line)
Robust, Heteroscedastic, Correlated Line Fitting
Robust TLS (isotropic)
Robust, heteroscedastic(uncorrelated)
Robust, heteroscedastic,correlated
•Ellipses/circles represent saturation (truncation) levels.
•The same algorithm/program was used for all examples.
Robust, Heteroscedastic, Correlated Line Fitting
•Ellipses/circles represent saturation (truncation) levels.
•The same algorithm/program was used for all examples.
ReferencesN. Kiryati and A.M. Brucktein, “Heteroscedastic Hough Transform (HtHT): An Efficient Method for Robust line Fitting in the `Errors in the Variables’ Problem, Computer Vision and Image Understanding, Vol. 78, pp. 69-83, 2000.
N. Kiryati and A.M. Bruckstein, “What’s in a Set of Points?”, IEEE Trans. Pattern Analysis Macine Intelligence, Vol. 14, pp. 496-500, 1992.
N. Kiryati and A.M. Bruckstein, “On Navigating between Friends and Foes”, IEEE Trans. Pattern Analysis Macine Intelligence, Vol. 13, pp. 602-606, 1991.