research methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · research...
TRANSCRIPT
![Page 1: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/1.jpg)
Research Methodsin computer science
Spring 2019
Lecture 14
Omprakash GnawaliMarch 4, 2019
![Page 2: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/2.jpg)
Agenda
HW6 Live GradingConference OrganizationExperiments and metricsReading PapersHW7
![Page 3: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/3.jpg)
Types of Experiments
Model / AnalysisSimulationsTestbed (Real word lite)“Real world”
Which one to use when?
![Page 4: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/4.jpg)
Metric
Why do we want to measure?
What to measure?
![Page 5: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/5.jpg)
Most of the time we measure improvements
![Page 6: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/6.jpg)
Eigenfaces for Recognition
[Turk ’91]
“We have developed a near-real-time computer system that can locate and track a subject’s head, and then recognize the person by comparing the characteristics of the face to those of known individuals.”
![Page 7: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/7.jpg)
Scenarios and metrics from [Turk ‘91]
![Page 8: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/8.jpg)
The Anatomy of a Large-Scale Hypertextual Web Search Engine
[Brin and Page ’98]
What hypothesis, scenarios, and metrics should we expect to see in this paper?
![Page 9: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/9.jpg)
[Brin and Page ’98]
![Page 10: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/10.jpg)
[Brin and Page ’98]
Why did the authors decide to report these measurements?
![Page 11: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/11.jpg)
Metrics/Experiments?Accurately Initializing Real Time Clocks to Provide Synchronized Time in Sensor Networks
CTP: An Efficient, Robust, and ReliableCollection Tree Protocol for Wireless Sensor Networks
On the Effectiveness of Energy Metering on Every Node
Surviving Sensor Network Software Faults
![Page 12: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/12.jpg)
Metrics from Classification ResearchClassification AccuracyLogarithmic LossArea Under ROC CurveConfusion MatrixClassification ReportPrecisionRecallF1-Score
Partly from https://machinelearningmastery.com/metrics-evaluate-machine-learning-algorithms-python/
![Page 13: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/13.jpg)
Metrics from Regression Research
Mean Absolute ErrorMean Squared ErrorR^2
Partly from https://machinelearningmastery.com/metrics-evaluate-machine-learning-algorithms-python/
![Page 14: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/14.jpg)
Metrics from Systems Research
ReliabilityLatencyCoverageEnergy
![Page 15: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/15.jpg)
Group Activity
Experiment DesignMetric Selection
![Page 16: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/16.jpg)
Experiments
What experiments are useful?Critical for the main arguments of the paper
What experiments are not useful?Pointless experiments that generate pointless numbers, graphs, and tables
![Page 17: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/17.jpg)
Types of Experiments
From the “context” perspectiveControlledUncontrolled
There are other perspectives to be covered in future lectures
![Page 18: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/18.jpg)
A new algorithm that translates English text to Spanish.
![Page 19: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/19.jpg)
A new wireless networking technology.
![Page 20: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/20.jpg)
A new algorithm that can identify the person in an image.
![Page 21: Research Methods in computer science › ... › cosc6321-s19 › cosc6321-s19-l14.pdf · Research Methods in computer science Spring 2019 Lecture 14 Omprakash Gnawali March 4, 2019](https://reader033.vdocuments.mx/reader033/viewer/2022060210/5f0484f77e708231d40e5f7b/html5/thumbnails/21.jpg)
HW7 – Intro & Related WorkWrite a draft Introduction and Related Work for your paper. It should be about 2 pages in length. Use the template to be used in the conference.