An Undergraduate Course on Software Bug Detection Tools and Techniques

Download An Undergraduate Course on Software Bug Detection Tools and Techniques

Post on 07-Jan-2016

18 views

Category:

Documents

1 download

Embed Size (px)

DESCRIPTION

An Undergraduate Course on Software Bug Detection Tools and Techniques. Eric Larson Seattle University March 3, 2006. Introduction. Course was taught at Seattle University in Winter 2005 quarter. 9 senior undergraduate students senior elective Key contributions: - PowerPoint PPT Presentation

TRANSCRIPT

<ul><li><p>An Undergraduate Course on Software Bug Detection Tools and Techniques </p><p>Eric LarsonSeattle UniversityMarch 3, 2006</p><p>*</p><p>IntroductionCourse was taught at Seattle University in Winter 2005 quarter.9 senior undergraduate studentssenior elective</p><p>Key contributions:First course on software bug detection course geared toward undergraduate students.Assignment infrastructure where students can create their own software bug detection tools. </p><p>*</p><p>Talk OutlineGoals and BackgroundCourse ContentProgramming AssignmentsResultsFuture Work</p><p>*</p><p>Goals of the CourseLearn and analyze algorithms that can be used to find bugs in software.Understand why software bug detection is hard.Gain experience developing software bug detection tools.Become better programmers by thinking about software bug detection. </p><p>*</p><p>Non Goals of the CourseTestingOnly briefly mentioned in the course (types of testing, coverage criteria)A course on testing would make a nice complement to this course.Material in this course would be relevant in a testing course.</p><p>DebuggingOnce a bug is found, what is the source of the bug?Occasionally, we would talk about enhancements to a tool that would make debugging easier.Students mentioned they would like to have learned more.</p><p>*</p><p>Key ChallengesGeared toward undergraduates.Toned down the theory but did not eliminate it.More practical, give students experience writing software tools.</p><p>No textbook.Used relatively easy-to-read papers.</p><p>Few prerequisites.Only prerequisite was a course in algorithms.No pre-reqs in compilers, software engineering, automata theory, or software testing.Focused primarily on C programs.</p><p>*</p><p>Format of the CourseTraditional lecture formatSome in-class activities and discussionsAssignmentsThree programming assignments (in pairs)Daily homework exercise (individual)GradingClass participation5 %Homework20 %Exams (midterm and final)30 %Programming assignments45 %</p><p>*</p><p>Course ContentCourse was broken down into four units:Program Analysis and Terminologysimilar to back-end compiler analysisspecial attention on interprocedural and pointer analysisDynamic Bug Detectionstart of unit focused on testingadding instrumentation to programs to:manage additional state about variables or memory used in the programcheck additional state to detect bugs</p><p>*</p><p>Course ContentStatic Bug Detectionsymbolic path simulationconstraint analysismodel checking</p><p>Other Topicsconcurrent programssafe languagespreventing security attacks (ex: StackGuard)</p><p>*</p><p>Assignment InfrastructureAssignments were completed using a source to source converter called SUDSE.Contains static analyses for static bug detection.Converts a subset of C to instrumented C.No enums, floating point values, or unions. Since students were familiar with C++, the cin and cout I/O statements were added. Internal representation is an AST of simplified C statements for easier analysis.Side effects and short circuited operations are removed.Complex expressions broken down into two or more simpler expressions.Also suitable for other types of assignments.code coverage toolcompilersprofiler</p><p>*</p><p>Example: Simplification// Original codeint bar(int x){ int a[5]; int i; for (i = 0; i &lt; 5; i++) { a[i] = i * i; } return a[x];}</p><p>// Simplified code int bar(int x){ int a[5]; int i; int T1, T2; i = 0; T1 = i &lt; 5; while (T1) { a[i] = i * i; i = i + 1; T1 = i &lt; 5; } T2 = a[x]; return T2;}</p><p>*</p><p>AssignmentsProgram analysiscontrol-flow graph and data flow analysisuses of uninitialized variables.Dynamic bug detectionmore on this laterStatic bug detectionnull deference checker.open-ended (students can use any technique they wanted)</p><p>*</p><p>Dynamic Array CheckerIn this assignment, students had to create a tool that detects array out-of-bounds errors.In part 1, students had to add instrumentation calls to interesting statements:Array declarationsArray references / pointer dereferencesPointer assignmentsArrays/pointers go out of scopeIn part 2, students had to write the instrumentation routines to manage the array state and check for errors.</p><p>*</p><p>Example: Instrumented Code#include "ptr_table.h"int bar(int x){ int a[5]; create_array_entry((void *) a, 5); int i; int T1, T2; i = 0; T1 = i &lt; 5; while (T1) { check_array_bounds((void *) a, i, __FILE__, __LINE__); a[i] = i * i; i = i + 1; T1 = i &lt; 5; } check_array_bounds((void *) a, x, __FILE__, __LINE__); T2 = a[x]; delete_array_entry((void *) a); return T2;}</p><p>*</p><p>Instrumentation Functionsvoid create_array_entry(void *addr, int size);Adds an entry to the array table.Entry is indexed by addr and stores the size of the array.</p><p>void check_array_bounds(void *addr, int index, const char *file, int line);Search for the entry in the table using addr.Check to make sure that the index is not negative and less than the size of the array.If the check fails, print out an error that contains the file name and line number.</p><p>*</p><p>ResultsCourse was a success!Based on student feedback and personal observations.Assignments represented a mix of several key areas in CS: theory, OO-development, algorithmsThe second assignment was the most successful.First assignment was not interesting or challenging to the students who had background in compilers.Not enough time for the open-ended third assignment.Mixed reactions on favorite course topics.Program analysis was either favorite or least favorite. Some students wanted to learn more about debugging.</p><p>*</p><p>Future Modifications Allow more time for the third assignment.Reduce the first assignment to a warm-up exercise.Add section on debugging techniques.Use existing bug detection tools (valgrind).Explore software engineering practices.Where do software bug detection tools fit in different software engineering processes?Coding standards.</p></li><li><p>Questions and Answers</p><p>*</p><p>TerminologyCorrectness Property: A property that describes correct behavior. A bug is defined as a program that violates the property.False Alarm: A bug report produced by a tool that is not actually a bug.Completeness: Every bug report is a bug. No false alarms.Soundness: Every bug is detected. No bugs are missed. </p></li></ul>