cs 6140: machine learning - college of computer and ... · cs 6140: machine learning spring 2017...
TRANSCRIPT
![Page 1: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/1.jpg)
CS6140:MachineLearningSpring2017
Instructor:LuWangCollegeofComputerandInforma@onScience
NortheasternUniversityWebpage:www.ccs.neu.edu/home/luwang
Email:[email protected]
![Page 2: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/2.jpg)
TimeandLoca@on
• Time:Thursdaysfrom6:00pm–9:00pm
• Loca)on:ForsythBuilding129
![Page 3: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/3.jpg)
CourseWebpage
• hPp://www.ccs.neu.edu/home/luwang/courses/cs6140_sp2017.html
![Page 4: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/4.jpg)
Prerequisites
• Programming– Beingabletowritecodeinsomeprogramminglanguages(e.g.Python,Java,C/C++,Matlab)proficiently
• Courses– Algorithms– Probabilityandsta@s@cs– Linearalgebra
![Page 5: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/5.jpg)
Prerequisites
• Courses– Algorithms– Probabilityandsta@s@cs– Linearalgebra
• Aquiz:– 22simpleques@ons,20ofthemasTrueorFalseques@ons(relevanttoprobability,sta@s@cs,andlinearalgebra)
– Thepurposeofthisquizistoindicatetheexpectedbackgroundofstudents.
– 80%oftheques@onsshouldbeeasytoanswer.– Notcountedinyourfinalscore!
![Page 6: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/6.jpg)
TextbookandReferences
• MainTextbook– KevinMurphy,"MachineLearning-aProbabilis@cPerspec@ve",MITPress,2012.
– ChristopherM.Bishop,"PaPernRecogni@onandMachineLearning",Springer,2006.
• Othertextbooks– TomMitchell,"MachineLearning",McGrawHill,1997.
• Machinelearninglectures
![Page 7: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/7.jpg)
ContentoftheCourse• Regression:linearregression,logis@cregression• DimensionalityReduc)on:PrincipalComponentAnalysis(PCA),Independent
ComponentAnalysis(ICA),LinearDiscriminantAnalysis• Probabilis)cModels:NaiveBayes,maximumlikelihoodes@ma@on• Sta)s)calLearningTheory:VCdimension• Kernels:SupportVectorMachines(SVMs),kerneltricks,duality• Sequen)alModelsandStructuralModels:HiddenMarkovModel(HMM),
Condi@onalRandomFields(CRFs)• Clustering:spectralclustering,hierarchicalclustering• LatentVariableModels:K-means,mixturemodels,expecta@on-maximiza@on
(EM)algorithms,LatentDirichletAlloca@on(LDA),representa@onlearning• DeepLearning:feedforwardneuralnetwork,restrictedBoltzmannmachine,
autoencoders,recurrentneuralnetwork,convolu@onalneuralnetwork• ReinforcementLearning:Markovdecisionprocesses,Q-learning• andothers,includingadvancedtopicsformachinelearninginnaturallanguage
processingandtextanalysis
![Page 8: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/8.jpg)
TheGoal
• Scien@ficunderstandingofmachinelearningmodels
• Howtoapplyanddesignlearningmethodsfornovelproblems
![Page 9: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/9.jpg)
TheGoal
• Notonlywhat,butalsowhy!
![Page 10: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/10.jpg)
Grading• Assignment
– 3assignments,10%foreach
• Quiz– 10in-classtests,1%foreach
• Exam– 1exam,30%
• Project– 1project,27%
• Par@cipa@on– 3%– Classes– Piazza
![Page 11: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/11.jpg)
Exam
• Openbook• April20,2017
![Page 12: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/12.jpg)
CourseProject
• Amachinelearningrelevantresearchproject
• 2-3studentsasateam
![Page 13: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/13.jpg)
Topics
• Machinelearningrelevant– Naturallanguageprocessing– Computervision– Robo@cs– Bioinforma@cs– Healthinforma@cs– …
![Page 14: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/14.jpg)
CourseProjectGrading
• Wewanttoseenovelandinteres@ngprojects!– Theproblemneedstobewell-defined,novel,useful,andprac@cal
– machinelearningtechniques
– Reasonableresultsandobserva@ons
![Page 15: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/15.jpg)
ProjectfromLastYear
![Page 16: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/16.jpg)
ProjectfromLastYear
• Predic@ngFollow-backBehaviorinInstagramUsers
![Page 17: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/17.jpg)
ProjectfromLastYear
• Predic@ngGraspPointsUsingConvolu@onalNeuralNetworks
![Page 18: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/18.jpg)
ProjectfromLastYear
• Ar@ficialNeuralNetworksforDrugResponsePredic@oninTailoredTherapy
![Page 19: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/19.jpg)
ProjectfromLastYear
• ThreatDetec@onfromTwiPer
![Page 20: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/20.jpg)
ProjectfromLastYear
• PlayerRankinginPopularGames
![Page 21: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/21.jpg)
CourseProjectGrading
• Threereports– Proposal(2%)– Progress,withcode(10%)– Final,withcode(10%)
• Onepresenta@on– Inclass(5%)
![Page 22: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/22.jpg)
SubmissionandLatePolicy• Eachassignmentorreport,bothelectroniccopyandhardcopy,isdueatthebeginningofclassonthecorrespondingduedate.
• Programminglanguage– Python,Java,C/C++,Matlab
• Electronicversion– Onblackboard
• Hardcopy– Inclass
![Page 23: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/23.jpg)
SubmissionandLatePolicy
• Assignmentorreportturnedinlatewillbecharged10points(outof100points)offforeachlateday(i.e.24hours).
• Eachstudenthasabudgetof5daysthroughoutthesemesterbeforealatepenaltyisapplied.
![Page 24: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/24.jpg)
Howtofindus?• Coursewebpage:– hPp://www.ccs.neu.edu/home/luwang/courses/cs6140_sp2017.html
• Officehours– LuWang:Thursdaysfrom4:30pmto5:30pm,orbyappointment,448WVH
– RuiDong(TA),Tuesdaysfrom4:00pmto5:00pm,orbyappointment,466BWVH
• Piazza– hPp://piazza.com/northeastern/spring2017/cs614002– Allcourserelevantques@onsgohere
![Page 25: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/25.jpg)
WhatisMachineLearning?
• “Asetofmethodsthatcanautoma@callydetectpaPernsindata,andthenusetheuncoveredpaPernstopredictfuturedata,ortoperformotherkindsofdecisionsmakingundercertainty.”
![Page 26: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/26.jpg)
RealWorldApplica@ons
![Page 27: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/27.jpg)
RealWorldApplica@ons
![Page 28: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/28.jpg)
RealWorldApplica@ons
![Page 29: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/29.jpg)
RealWorldApplica@ons
![Page 30: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/30.jpg)
RealWorldApplica@ons
![Page 31: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/31.jpg)
RealWorldApplica@ons
![Page 32: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/32.jpg)
RealWorldApplica@ons
![Page 33: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/33.jpg)
RealWorldApplica@ons
![Page 34: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/34.jpg)
RealWorldApplica@ons
![Page 35: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/35.jpg)
RealWorldApplica@ons
![Page 36: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/36.jpg)
Rela@onswithOtherAreas
• NaturalLanguageProcessing
• ComputerVision
• Robo@cs
• Alotofotherareas…
![Page 37: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/37.jpg)
Today’sOutline
• Basicconceptsinmachinelearning
• K-nearestneighbors
• Linearregression
• Ridgeregression
![Page 38: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/38.jpg)
Supervisedvs.UnsupervisedLearning
![Page 39: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/39.jpg)
SupervisedLearning
![Page 40: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/40.jpg)
Supervisedvs.UnsupervisedLearning
• Supervisedlearning
Trainingset Trainingsample Gold-standardlabel- Classifica)on,ifcategorical- Regression,ifnumerical
![Page 41: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/41.jpg)
SupervisedLearning
![Page 42: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/42.jpg)
SupervisedLearning
![Page 43: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/43.jpg)
SupervisedLearning
• Goal:– Generalizabletonewinputsamples– Overfivngvs.underfivng– Onesolu@on:weuseprobabilis@cmodels
• Typicalsetup:– Step1:Features– Step2:Trainingset,testset,developmentset– Step3:Evalua@on
![Page 44: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/44.jpg)
SupervisedLearning
![Page 45: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/45.jpg)
SupervisedLearning
![Page 46: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/46.jpg)
SupervisedLearning
![Page 47: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/47.jpg)
SupervisedLearning
• Regression– Predic@ngstockprice– Predic@ngtemperature– Predic@ngrevenue…
![Page 48: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/48.jpg)
Supervisedvs.UnsupervisedLearning
• UnsupervisedLearning
• Moreabout“knowledgediscovery”
![Page 49: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/49.jpg)
UnsupervisedLearning
• Dimensionreduc@on– Principalcomponentanalysis
![Page 50: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/50.jpg)
UnsupervisedLearning
• Clustering(e.g.graphmining)
RolX:RoleExtrac.onandMininginLargeNetworks,byHendersonetal,2011
![Page 51: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/51.jpg)
UnsupervisedLearning
• Topicmodeling
![Page 52: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/52.jpg)
Parametricvs.Non-parametricmodel
• Fixednumberofparameters?– Ifyes,parametricmodel
• Numberofparametersgrowwiththeamountoftrainingdata?– Ifyes,non-parametricmodel
• Computa@onaltractability
![Page 53: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/53.jpg)
Today’sOutline
• Basicconceptsinmachinelearning
• K-nearestneighbors– Supervisedlearning– Anon-parametricclassifier
• Linearregression
• Ridgeregression
![Page 54: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/54.jpg)
Anon-parametricclassifier:K-nearestneighbors(KNN)
![Page 55: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/55.jpg)
Anon-parametricclassifier:K-nearestneighbors(KNN)
• Basicidea:memorizeallthetrainingsamples– Themoreyouhaveintrainingdata,themorethemodelhastoremember
![Page 56: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/56.jpg)
Anon-parametricclassifier:K-nearestneighbors(KNN)
• Basicidea:memorizeallthetrainingsamples– Themoreyouhaveintrainingdata,themorethemodelhastoremember
• Nearestneighbor(or1-nearestneighbor):– Tes@ngphase:findclosetsample,andreturncorrespondinglabel
![Page 57: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/57.jpg)
![Page 58: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/58.jpg)
![Page 59: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/59.jpg)
Anon-parametricclassifier:K-nearestneighbors(KNN)
• Basicidea:memorizeallthetrainingsamples– Themoreyouhaveintrainingdata,themorethemodelhastoremember
• K-Nearestneighbor:– Tes@ngphase:findtheKnearestneighbors,andreturnthemajorityvoteoftheirlabels
![Page 60: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/60.jpg)
![Page 61: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/61.jpg)
AboutK
• K=1:justpiecewiseconstantlabeling• K=N:globalmajorityvote(class)
![Page 62: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/62.jpg)
ProblemsofkNN
• Canbeslowwhentrainingdataisbig– Searchingfortheneighborstakes@me
• Needslotsofmemorytostoretrainingdata
• Needstotunekanddistancefunc@on
• Notaprobabilitydistribu@on
![Page 63: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/63.jpg)
ProblemsofkNN
• Distancefunc@on– Euclideandistance
![Page 64: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/64.jpg)
ProblemsofkNN
• Distancefunc@on– Mahalanobisdistance:weightsoncomponents
![Page 65: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/65.jpg)
Probabilis@ckNN
• Wepreferaprobabilis@coutputbecausesome@meswemaygetan“uncertain”result– 1samplesas“yes”,199samplesas“no”à?– 99samplesas“yes”,101samplesas“no”à?
• Probabilis@ckNN:
![Page 66: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/66.jpg)
Probabilis@ckNN
3-classsynthe@ctrainingdata
![Page 67: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/67.jpg)
Smoothing
• Class1:3,class2:0,class3:1• Originalprobability:– P(y=1)=3/4,p(y=2)=0/4,p(y=3)=1/4
![Page 68: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/68.jpg)
Smoothing
• Class1:3,class2:0,class3:1• Originalprobability:– P(y=1)=3/4,p(y=2)=0/4,p(y=3)=1/4
• Add-1smoothing:– Class1:3+1,class2:0+1,class3:1+1– P(y=1)=4/7,p(y=2)=1/7,p(y=3)=2/7
![Page 69: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/69.jpg)
Soxmax
• Class1:3,class2:0,class3:1• Originalprobability:– P(y=1)=3/4,p(y=2)=0/4,p(y=3)=1/4
• Redistributeprobabilitymassintodifferentclasses– Defineasoxmaxas
![Page 70: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/70.jpg)
Today’sOutline
• Basicconceptsinmachinelearning
• K-nearestneighbors
• Linearregression– Supervisedlearning– Aparametricclassifier
• Ridgeregression
![Page 71: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/71.jpg)
Aparametricclassifier:linearregression
• Assump@on:theresponseisalinearfunc@onoftheinputs
InnerproductbetweeninputsampleXandweightvectorW
Residualerror:differencebetweenpredic@onandtruelabel
![Page 72: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/72.jpg)
Aparametricclassifier:linearregression
• Assumeresidualerrorhasanormaldistribu@on
InnerproductbetweeninputsampleXandweightvectorW
Residualerror:differencebetweenpredic@onandtruelabel
![Page 73: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/73.jpg)
Aparametricclassifier:linearregression
• Wecanfurtherassume
• Basicfunc@onexpansion
![Page 74: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/74.jpg)
Aparametricclassifier:linearregression
Ver@cal:temperatureHorizontal:loca@onwithinaroom
![Page 75: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/75.jpg)
Aparametricclassifier:linearregression
![Page 76: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/76.jpg)
LearningwithMaximumLikelihoodEs@ma@on(MLE)
• MaximumLikelihoodEs@ma@on(MLE)
![Page 77: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/77.jpg)
LearningwithMaximumLikelihoodEs@ma@on(MLE)
• Log-likelihood
• Maximizelog-likelihoodisequivalenttominimizenega@velog-likelihood(NLL)
![Page 78: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/78.jpg)
LearningwithMaximumLikelihoodEs@ma@on(MLE)
• Withournormaldistribu@onassump@on
Residualsumofsquares(RSS)àWewanttominimizeit!
![Page 79: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/79.jpg)
Deriva@onofMLEforLinearRegression
• Rewriteourobjec@vefunc@onas
![Page 80: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/80.jpg)
Deriva@onofMLEforLinearRegression
• Rewriteourobjec@vefunc@onas
• Getthederiva@ve(orgradient)
![Page 81: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/81.jpg)
Deriva@onofMLEforLinearRegression
• Rewriteourobjec@vefunc@onas
• Getthederiva@ve(orgradient)
• Setourderiva@veto0
Ordinaryleastsquaressolu)on
![Page 82: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/82.jpg)
![Page 83: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/83.jpg)
Overfivng
Featureweightsw:
![Page 84: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/84.jpg)
APriorontheWeight
• Zero-meanGaussianprior
![Page 85: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/85.jpg)
APriorontheWeight
• Zero-meanGaussianprior
• Newobjec@vefunc@on
![Page 86: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/86.jpg)
APriorontheWeight
• Zero-meanGaussianprior
• Newobjec@vefunc@on
![Page 87: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/87.jpg)
Today’sOutline
• Basicconceptsinmachinelearning
• K-nearestneighbors
• Linearregression
• Ridgeregression
![Page 88: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/88.jpg)
RidgeRegression
• Wewanttominimize
![Page 89: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/89.jpg)
RidgeRegression
• Wewanttominimize
• Newes@ma@onfortheweight
![Page 90: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/90.jpg)
RidgeRegression
• Wewanttominimize
• Newes@ma@onfortheweight
L2regulariza)on
![Page 91: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/91.jpg)
RidgeRegression
• Wewanttominimize
• Newes@ma@onfortheweight
L2regulariza)on
LeavetheproofinAssignment1!
![Page 92: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/92.jpg)
Whatwelearned
• Basicconceptsinmachinelearning
• K-nearestneighbors
• Linearregression
• Ridgeregression
![Page 93: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/93.jpg)
Homework
• ReadingMurphych1,ch2,andch7(onlythesec@onscoveredinthelecture)
• SignupatPiazza• hPp://piazza.com/northeastern/spring2017/cs614002
• Startthinkingaboutcourseprojectandfindateam!– ProjectproposaldueJan26
![Page 94: CS 6140: Machine Learning - College of Computer and ... · CS 6140: Machine Learning Spring 2017 ... – The purpose of this quiz is to indicate the expected ... • Deep Learning:](https://reader030.vdocuments.mx/reader030/viewer/2022012314/5b5adf117f8b9a24038d691f/html5/thumbnails/94.jpg)
Homework
• ReadingMurphych1,ch2,andch7• SignupatPiazza• hPp://piazza.com/northeastern/spring2017/cs614002
• Startthinkingaboutcourseprojectandfindateam!– ProjectproposaldueJan26
• NextTime:Logis@cRegression,DecisionTree,Genera@veModels(NaiveBayes)– Reading:MurphyCh3,8.1-8.3,8.6,16.2