Research Design in Modelling & Simulation Instructor Contact: Instructor: PK Douglas, PhD Office: Partnership II, Room 322 Email: [email protected] Office Hours/Web Hours: After class, or by appointment Course Description: Thiscourseprovidesanoverviewtomodellingandsimulationresearchtechniques.ThisisacorecourseforgraduatestudentsintheModellingandSimulationdepartment,thoughitisopentostudentsfromotherdisciplines.Itisdesignedtosupportdevelopmentofskillsrelatedtomachinelearningandpatternclassificationof4dimensionaldata(x,y,z,time),aswellasdynamicsystemsmodelingapproachestostudythebrain.LearningObjectives:Developtheabilityto1)identifyimportantandnovellinesofinquiryinthefieldofModelingandSimulationresearch,2)appreciatehowtoframearesearchproblemwiththeappropriatemethodologyandexperimentaldesign,3)understandhowtoimplementbothhypothesis-drivenandmodelfreeanalyticapproachestostudydata,4)recognizethelimitationsofdifferentresearchapproaches,anddetermineappropriateconclusionswithintheseconstraints.Akeythemeofthecoursewillfocusonidentifyingalternatemodelhypotheses,andmethodsformodelselection.Prerequisites:Althoughtherearenoformalprerequisites,abackgroundknowledgeoflinearalgebrawillbeuseful,andtheabilitytocode(e.g.,Matlab,R,Python)wouldbehelpfulforthesecondhalfofthecourse.IamawarethatstudentsintheM&Sdepartmentcomefromavarietyofbackgrounds.Iwillthereforeprovidetutorials,links,andsufficienttimetogetuptospeedonmathsandrelevantcodingpriortohomeworkassignments.
RequiredText:Thereisnorequiredtextforthiscourse.Instead,readingswillbeassignedfromtheliteratureandselectionsfromseveraltexts,whichwillbemadeavailableonline.StudentsmayfindtheintroductoryMLbook“IntroductiontoMachineLearning,”3rd.Ed”byEthemAlpaydintobeausefulreferenceforthatportionofthecourse.
Course Outline: Thecourseisdividedintofourmainparts.Inpart1,youwilllearnthefundamentalsofresearchincludingtopicformulation,experimentaldesign,andclassicdatatypes.Inpart2(Model-->Data),youwilllearnmethodsforgoingfromamodel(hypothesis/hypotheses)totestinghowwellthatmodelfitsthedata,comparedtoalternativemodels.Inpart3,(Data-->Model),youwilllearnmethodsforallowingthedatathemselvestonominatethemodelusinganumberofunsupervisedandclusteringtechniques.Inpart4,wewilldiscussmethodsforcriticallyevaluatingthesemodels.PartI:ExperimentalDesignModule1:IntroductiontoResearchDesign1 HW1:PleasetakethePretest—Thiswillfulfillthestudentengagementrequirementfortheuniversity.
Note–completionofthisquizwillreceivefullcredit(5pts).Thepurposeofthisquizisformetobetterunderstandthestudents’backgroundsinthecourse.
Module2:Formulatingandtestinghypotheses1 Stepsinconstructingtestablehypotheses2 CorrelationandCausation(WhatinferencescanImake?)3 HW2:TopicformationandIntroduction.Inafewsentencesinadiscussionthread,pleasedescribe3
topicsthatinterestyouinmodelingandsimulation.Pleasetakethistimetointroduceyourselftotheclassanddescribeyoureducational/workbackground.(5pts)
Module3:Datasources&DataTypes1 Datatypes(descriptive,correlational,etc)2 Potentialthreatstovalidity,generalizability,andreplicability3 PilotTesting&Identifyingpopulationimplicationsaprioriandtheirinfluenceonstatisticalchoices4 HW3:(GroupAssignment)FormingResearchHypotheses—Discussionpostof2researchquestions,
eachwith3testablehypotheses.Responsewith2recommendations,requestsforclarification,and/oradditionalpotentialhypotheses.*Youmustsubmitanemailofwhoyourgroupmembersarebyweek4(groupsofupto3people)(5pts)
Module4:Datacollection&Design1 Quantitative(e.g.datatypes,computationaltechniques,physiological)2 SamplingandStatistics3 HW4:(GroupAssignment)—Discussionpost.Postadiagramofyourresearchdesign.IncludeDVsand
IVs,interventions/manipulations.Useyourdiagramtoshowhowinformationcollectedwillhelpansweryourresearchquestionbytestingyourhypotheses.Doyouplantouseanexistingdatasetorasimulateddatasetforyourgroupproject?(10pts)
PartII:ModelHypothesis-->DataModule5:ClassicalInference1 ClassicalHypothesisTesting2 GeneralLinearModel3 HW5:GLMexercise(5pts)Module6:CorrectionforMultipleComparisons1 Falsediscoveryrate(e.g.,Benjamini-Hockberg)2 Othermethodsforcorrectionformultiplecomparison3 HW6:WatchthevideoonGaussianRandomFieldtheoryandcompletethequiz.(5pts)Module7:InductiveBias&CorrectionforMultipleComparison1 InductiveBiasInDataModeling&Simulation2 FeatureSelection,Cross-validation,permutationtests3 HW7:Machinelearningexercise(5pts)
Module8:MachineLearning-PartI1 FeatureSelection-Isthisstepimportant?2 DimensionReduction(CurseofDimensionality,KernelTrick)3 HW8:DownloadWeka,andcompletethefeatureselectionexercises(5pts)Module9:MachineLearning-PartII1 InterpretingClassifiers
2 Hownoiseandredundancycaneffectinterpretation3 HW9:CompleteMatlabLDAassignment(10pts)Module10:GenerativeModels1 DynamicSystemsModeling2 ModelInversion(ParameterEstimationTechniques)3 HW10:CompleteMatlabODEassignment(10pts)Module11:FrequencyDomainAnalysis1 LaplaceTransform2 StabilityconstraintswhenmodelingintheFrequencyDomain3 HW11:CompleteFrequencyDomainAssignment(5pts)PartIII:Data-->ModelModule12:UnsupervisedMethods1 Whyrunanunsupervisedanalysis?2 ReviewofLinearAlgebraTechniques3 HW12:CompleteLinearAlgebraReview(5pts)Module13:EigenvaluesandMore1 SingularValueDecomposition(SVD,PCA,ICA,CCA)2 HW13:CompleteMatlabAssignment(10pts)Module14:ModelSelection4 Sparsevs.ComplexModels5 AIC,BIC,andFreeEnergy6 HW11:CompleteModelSelectionExercise(5pts)PartIV:EvaluatingModelModule15:CritiquingModels1 Understandinglimitationsofmodels2 HW15:Readpostedmanuscriptandpostdiscussionoflimitationsofthemethodinthediscussion
section.(10pts)Module16:GroupWeek1 Workinyourgroupsonyourproject
Course Requirements: Attendance,participation,completionofhomeworkandfinalexamarerequiredforcoursecompletion.Accordingtotheuniversity,thisclassis"mixedmode".Hereistheofficialdescription:M–MixedMode/ReducedSeatTimecoursesincludebothrequiredclassroomattendanceandonlineinstruction.Classeshavesubstantialactivityconductedovertheweb,whichsubstitutesforsomeclassroommeetings.
Evaluation & Grading: Homeworkandparticipationinthediscussiontopicswillaccountfor100ptsofyourgrade.Thefinalprojectwillalsobeworth100pts(oralpresenation,50pts,andfinalpaper50pts).Theclasswillfollow
atraditionalgradingscaleshownbelow:LetterGrade PointsA 90-100B 80-89C 70-79D 60-69F 59orbelow
Make up Exams: Homeworks,groupprojects,andexamswillbepostedonline,andstudentsmaytakethemanytimeupuntiltheirfinalduedate.Allgroupmembersareexpectedtocontributetothebestoftheirability,andareexpectedtoparticipateinthefinalgrouppresentationandcontributetothefinalpaper.Groupsshouldsubmitfinalpapersinresearchmanuscriptformat,andindicatethecontributionofeachgroupmember.Ifforsomereason,youcannotmakeitin-persontothefinalgrouppresentation,pleasecontactmetodiscussalternatives.Failuretodosowillresultinalowergradeforthecourse. Attendance Policy: Accordingtotheuniversity,thisclassis"mixedmode".Hereistheofficialdescription:M–MixedMode/ReducedSeatTimecoursesincludebothrequiredclassroomattendanceandonlineinstruction.Classeshavesubstantialactivityconductedovertheweb,whichsubstitutesforsomeclassroommeetings.WhileIdonotdirectlypenalizeformissedclasses,attendanceitisextremelyimportantforsuccessandclassparticipation,andshouldbeconsideredmandatory. Academic Engagement: Allinstructors/facultyarerequiredtodocumentstudents’academicactivityatthebeginningofeachcourse.Inordertodocumentthatyoubeganthiscourse,pleasecompletethepretestassoonaspossible.Failuretodosomayresultinadelayinthedisbursementofyourfinancialaid.Todocumentyouracademicengagement,youwillcompleteaquizbytheendofthefirstweekofclass. Final Paper: ThefinalpaperwillbedueatthetimeassignedbyUCF.ThistimewillbepostedonWebcourseswithinthefirstweek.
Changes/Announcements: Itispossiblethatadjustmentstotheschedulemaybemadeduringthecourse.Intheeventthatanythinginthissyllabuschanges(e.g.classroommoves,changesinduedates,contactinformation),Iwilluseabroadannouncementsothatallstudentswillbeinformedimmediately.Itiscriticallyimportantthatyousetyourwebcourseannouncementssothatyoureceiveallnotificationsandbesuretochecktheclasswebsiteregularly. Academic Integrity: Studentsareencouragedtodiscussproblemswithcolleagues,butthefinalassignmenthandedinshouldbethestudent’sownwork.Plagiarismandcheatingofanykindonanexamination,quiz,orassignmentwillresultatleastinan"F"forthatassignment(andmay,dependingontheseverityofthecase,leadto
an"F"fortheentirecourse)andmaybesubjecttoappropriatereferraltotheOfficeofStudentConductforfurtheraction.SeetheUCFGoldenRuleforfurtherinformation.Iwillalsoadheretothehigheststandardsofacademicintegrity,sopleasedonotaskmetochangeyourgrade. Accessibility Statement: TheUniversityiscommittedtoprovidingreasonableaccommodationsforallpersonswithdisabilities.Thissyllabusisavailableinalternateformatsuponrequest.Studentswithdisabilitieswhoneedaccommodationsinthiscoursemustcontacttheprofessoratthebeginningofthesemestertodiscussneededaccommodations.Noaccommodationswillbeprovideduntilthestudenthasmetwiththeprofessortorequestaccommodations.StudentswhoneedaccommodationsmustberegisteredwithStudentAccessibilityServices,FerrellCommons,7F,Room185,phone(407)823-2371,TTY/TDDonlyphone(407)823-2116,beforerequestingaccommodationsfromtheprofessor. Copyright: Thiscoursemaycontaincopyrightprotectedmaterialssuchasaudioorvideoclips,images,textmaterials,etc.TheseitemsarebeingusedwithregardtotheFairUsedoctrineinordertoenhancethelearningenvironment.Pleasedonotcopy,duplicate,downloadordistributetheseitems.Theuseofthesematerialsisstrictlyreservedforthisonlineclassroomenvironmentandyouruseonly.Allcopyrightmaterialsarecreditedtothecopyrightholder.
