컴퓨터 과학부 김명재. introduction data preprocessing model selection experiments
TRANSCRIPT
![Page 1: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/1.jpg)
A Practical Guide to Sup-port Vector Classification
컴퓨터 과학부김명재
![Page 2: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/2.jpg)
Introduction
Data Preprocessing
Model Selection
Experiments
Table of Contents
![Page 3: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/3.jpg)
Support Vector Machine
Introduction
![Page 4: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/4.jpg)
SVM (Support vector machine)◦ Training set of instance-label pairs◦ where
◦ Objective function
subject to
Introduction
( , )i ix y 1, ,i l n
ix R
1
1
2
lT
ii
w w C
, ,minw b
( ( ) ) 1Ti i iy w x b
0i
{1, 1}ly
![Page 5: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/5.jpg)
Dual space form◦ Objective function
maximize
subject to
Introduction
1
0l
i ii
t
0 i C
1 1 1
1( )
2
l l lT
i i j i j i ji i j
L t t x x
![Page 6: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/6.jpg)
Nonlinear SVM◦ Kernel method
Training vectors Mapped into a higher dimensional space Maybe infinite Mapping function
Objective function
Introduction
ix
: L H
1 1 1
1( ) ( , )
2
l l l
i i j i j i ji i j
L t t K x x
![Page 7: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/7.jpg)
◦ Kernel function
Linear
Polynomial
Radial basis function
Sigmoid
are kernel parameter
Introduction
( , ) ( ) ( )Ti j i jK x x x x
( , )i jK x x
( , )i jK x x
( , )i jK x x
( , )i jK x x
Ti jx x
( ) , 0T di jx x r
2exp( ), 0i jx x
tanh( )Ti jx x r
, ,r d
![Page 8: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/8.jpg)
Example◦ Data url
http://www.csie.ntu.edu.tw/~cjlin/papers/guide/data/
Introduction
Application #training data
#testingdata
#features #classes
Astroparticle 3, 089 4,000 4 2
Bioinfomat-ics
391 0 20 3
Vehicle 1,243 41 21 2
![Page 9: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/9.jpg)
Proposed Procedure◦ Transform data to format of an SVM package◦ Conduct simple scaling on the data◦ Consider the RBF kernel ◦ Use cross-validation to find the best parameter
and◦ Use the best parameter and to train the whole
training set◦ Test
Introduction
( , )i jK x x2
exp( )i jx x
C C
![Page 10: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/10.jpg)
Categorical Feature◦ Example
Three-category such as {red, green, blue} can be represented as (0, 0, 1), (0, 1, 0), and (1, 0, 0)
Scaling◦ Scaling before applying SVM is very important.◦ Linearly scaling each attribute to the range
[-1, +1] or [0, 1].
Data Preprocessing
![Page 11: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/11.jpg)
RBF kernel◦ RBF kernel is a reasonable first choice◦ Nonlinearly maps samples into a higher dimen-
sional space
◦ The number of hyperparameters which influences the complexity of model selection.
◦ Fewer numerical difficulties
Model Selection
![Page 12: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/12.jpg)
Cross-validation
Model Selection
![Page 13: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/13.jpg)
Cross-validation◦ Find the good
◦ Avoid the overfitting problem
◦ v-fold cross-validation Divide the training set into v subsets of equal size Sequentially, on subset is tested using the classifier
trained on the remaining v-1 subsets.
Model Selection
( , )C
![Page 14: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/14.jpg)
Grid-search◦ Various pairs of◦ Find a good parameter
for example
Model Selection
( , )C
5 3 152 ,2 , , 2C 15 13 32 ,2 , , 2
![Page 15: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/15.jpg)
Grid-search
Model Selection
![Page 16: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/16.jpg)
Grid-search
Model Selection
![Page 17: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/17.jpg)
Astroparticle Physics◦ original accuracy
66.925 %◦ after scaling
96.15 %◦ after grid-search
96.875 % (3875/4000)
Experiments
![Page 18: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/18.jpg)
Bioinformatics◦ original cross validation accuracy
56.5217 %
◦ after scaling cross validation accuracy 78.5166 %
◦ after grid-search 85.1662 %
Experiments
![Page 19: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/19.jpg)
Vehicle◦ original accuracy
2.433902 %
◦ after scaling 12.1951 %
◦ after grid-searching 87.8049 % (36/41)
Experiments
![Page 20: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/20.jpg)
libSVM◦ http://www.csie.ntu.edu.tw/~cjlin/libsvm/
A Training Algorithm for optimal Margin classifiers◦ Bernhard E. Boser, Isabelle M. Guyon, Vladimir N.
Vapnik
수업교재
REFERENCES
![Page 21: 컴퓨터 과학부 김명재. Introduction Data Preprocessing Model Selection Experiments](https://reader034.vdocuments.mx/reader034/viewer/2022051516/56649f575503460f94c7cc97/html5/thumbnails/21.jpg)
A Practical Guide to Sup-port Vector Classification
end of pages