applying support vector learning to stem cells classification
DESCRIPTION
TRANSCRIPT
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying Support Vector Learningto Stem Cells Classification
Ofer M. [email protected]
Natural Computing GroupLeiden University
LUMC, MCB Seminar, 25-09-2006
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Outline1 Introduction
The Problem: Stem Cells ClassificationNucleus Imaging
2 Online Machine LearningThe Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
3 The ApplicationApplying PerceptronApplying SVM
4 DiscussionConclusionsProspectsTake-Home Message
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Outline1 Introduction
The Problem: Stem Cells ClassificationNucleus Imaging
2 Online Machine LearningThe Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
3 The ApplicationApplying PerceptronApplying SVM
4 DiscussionConclusionsProspectsTake-Home Message
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Biological Motivation
The nuclear lamina envelops the nucleus. Intact lamina isvital for cell survival, knowckdown of lamin B results inlethal embryos in mice, and mutations in Lamin A causepremature aging syndromes in human.In human mesenchemyal stem cells (hMSCs) the laminashows a round and flat shape after 3D reconstruction. InhMSCs undergoing cell death the lamina shapedramatically changed and precedes the wholemarks ofapoptosis, such as nuclear breakdown and chromatinfragmentation.Soon after caspase-8 activation, which ultimately leads tocell death, intranuclear organization of the lamina areformed and the depth of the nucleus increased. Similarchanges in lamina organization are found in hMSCsundergoing replicative senescence.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Biological Motivation
Thus, it is possible that changes in the spatial organizationof the lamina are correlated with the functional state of thecell. The spatial organization of the lamina can be used asan early marker to sort between healthy and not-healthycells, as changes in lamina organization are visible beforechanges in cell morphology are detected.Here we tested this hypothesis using a machine learningapproach.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Nucleus Imaging
The lamina of hMSCs was detected after transduction ofthe Lamin B-GFP lentivirus vector.Image stacks of the lamin B-GFP were aquired with aconfocal microscope, and 3D reconstruction was obtainedwith TeloView.In control cells the XY and the XZ orientations revealed around and flat shape of the lamina.After activation of caspase-8, the shape of the lamina issignificantly changed.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Control vs. Apoptotic
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Problem: Stem Cells ClassificationNucleus Imaging
Nucleus Imaging
Serial slicing along the XZ axis taken from an individualnucleus with DIPimage toolbox revealed little changes inthe spatial organization of the lamina in a control cell.High variations were found in serial slicing taken from anapoptotic cell.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Outline1 Introduction
The Problem: Stem Cells ClassificationNucleus Imaging
2 Online Machine LearningThe Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
3 The ApplicationApplying PerceptronApplying SVM
4 DiscussionConclusionsProspectsTake-Home Message
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Machine Learning: TRAINING
Online learning considers a situation in which instancesare presented one at a time, where the learner’s task is tolearn a hypothesis which classifies the data correctly.Training phase: instances {xi}l
i=1 in Rn, and their labelsset Y = {−1,+1} are presented to the machine. Thealgorithm aims to update its hypothesis h : Rn → {±1} inorder to minimize the prediction error.Various algorithms have different update rules.Analogy: teacher, learner, corrections.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Machine Learning: TESTING
This training phase is followed by the testing phase, wheremore data is given to the learned hypothesis.Ideally unseen data. (Why...?)The correct labels are not presented to the machine!The accuracy rate is considered - how did the machineperform?
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Simple Perceptron
The Perceptron algorithm (Rosenblatt, 1957) is an onlinelearning algorithm for finding a consistent hypothesis within theclass of hyperplanes:
C ={h(~x) = sign
(~wT · ~x + b
)~wt ∈ Rn, b ∈ R
}The optimal hyperplane is defined as the one with the maximalmargin of separation between the two instances classes.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Perceptron: Optimal Hyperplane
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Non-Realizable for Hyperplanes Separation
But what if the data is not linearly-separable...?There is no hyperplane separator hypothesis for the problem!
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Mapping...
We would like then to map the instances to a higherdimensional space, where linear separation is feasible:
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Desirable Mapping
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
The Algorithm
The Support Vector Machines (SVM) algorithm (Boser, Guyonand Vapnik, 1992) is a linear method in a high-dimensionalfeature space, which is non-linearly interlinked to the instancespace. It allows learning a hypothesis for data which is notlinearly-separable.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
The Kernel Function
The function φ : Rn → F maps the instance vectors onto a
higher dimensional space F, and then the SVM aims to find a
hyperplane separator with the maximal margin in this space.
k (~xi, ~xj) ≡ φ(~xi)Tφ(~xj)
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Kernels
In particular, we consider the following kernel functions:The polynomial kernel:
k (~xi, ~xj) =(γ
(~xT
i · ~xj
)+ r
)d(1)
Radial basis function (RBF) kernel:
k (~xi, ~xj) = exp{− 1
2σ2‖~xi − ~xj‖2
}(2)
The sigmoid kernel:
k (~xi, ~xj) = tanh(κ
(~xT
i · ~xj
)+ Θ
)(3)
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Images as Instances
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Grayscale Images
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
The Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
Intermediate Conclusions
Grayscale images are simply matrices with normalizedelements in [0, 1].In particular, as instance vectors in Rn!Essentially, an image could be introduced directly tothe learning algorithm, without further processing.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying PerceptronApplying SVM
Outline1 Introduction
The Problem: Stem Cells ClassificationNucleus Imaging
2 Online Machine LearningThe Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
3 The ApplicationApplying PerceptronApplying SVM
4 DiscussionConclusionsProspectsTake-Home Message
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying PerceptronApplying SVM
Experimental Procedure: Modus Operandi
Training phase: provide the machine with shuffled 2000slices and their correct labels.Testing phase: test the machine with shuffled 1040 sliceswithout their labels - and check its accuracy.Correct classification means that the output of the machineper given instance is its correct label as in our database.Wrong classification (error rate) - vice versa.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying PerceptronApplying SVM
Applying Perceptron
Applying the Perceptron was straightforward, with respectto parameter settings, and did not require any preliminarytuning.However, the algorithm obtained, after training, a testaccuracy of 70.38% (732/1040 images were classifiedcorrectly).This result led us to the conclusion that the data was notlinearly-separable, and a stronger approach was muchneeded.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying PerceptronApplying SVM
Applying SVM - Preliminary
Applying SVM (libsvm package) to the classificationproblem with default settings yielded test accuracy of 55%on average.Thus, tuning the kernel parameters was essential - severalparameters as well as the profile of the kernel (Eq. 1, 2, 3)and its various appropriate parameters ({γ, r, d}, {σ} and{κ, Θ}).The Covariance Matrix Adaptation Evolution Strategy(CMA-ES) [Hansen et al., 2001] was selected as theoptimization tool: the cross-validation accuracy rate wasthe objective function to be optimized.Each objective function evaluation takes 11 minutes on asingle processor: runs were limited.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
Applying PerceptronApplying SVM
SVM - Numerical Results
CMA-ES found an RBF kernel with 98.90%cross-validation.Testing phase:Accuracy of 97.02% - 1009/1040 images were classifiedcorrectly!Highly satisfying! Beyond any expectation!
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
ConclusionsProspectsTake-Home Message
Outline1 Introduction
The Problem: Stem Cells ClassificationNucleus Imaging
2 Online Machine LearningThe Teacher-Learner ModelSimple PerceptronThe SVM AlgorithmImages as Instances
3 The ApplicationApplying PerceptronApplying SVM
4 DiscussionConclusionsProspectsTake-Home Message
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
ConclusionsProspectsTake-Home Message
Conclusions
Machine learning as a way of life.Machine classification of stem cells is feasible!Numerical results are remarkably excellent.No further image analysis, after the image acquisition, isrequired.Behind everything in life there is a matrix...
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
ConclusionsProspectsTake-Home Message
Prospects
Classification of other ”colors”.Classification of 3D images!Analysis of time-dependent 3D movies.
Ofer M. Shir SVM to Stem-Cells Classification
IntroductionOnline Machine Learning
The ApplicationDiscussion
ConclusionsProspectsTake-Home Message
Take-Home Message
Natural computing, machine learning and data mining arerich fields with a lot to offer!Find yourself a nice computer-scientist, and invest in yourrelationship.You may prefer to consider those tools as a black-boxes.BUT then apply and boost medicine...
Ofer M. Shir SVM to Stem-Cells Classification