An Undergraduate Course on Software An Undergraduate Course on Software Bug Detection Tools and Techniques Bug Detection Tools and Techniques

Eric LarsonSeattle University

March 3, 2006

2

IntroductionIntroduction• Course was taught at Seattle University in Winter

2005 quarter.– 9 senior undergraduate students– senior elective

• Key contributions:– First course on software bug detection course geared

toward undergraduate students.– Assignment infrastructure where students can create their

own software bug detection tools.

3

Talk OutlineTalk Outline• Goals and Background• Course Content• Programming Assignments• Results• Future Work

4

Goals of the CourseGoals of the Course• Learn and analyze algorithms that can be

used to find bugs in software.• Understand why software bug detection is

hard.• Gain experience developing software bug

detection tools.• Become better programmers by thinking

about software bug detection.

5

Non Goals of the CourseNon Goals of the Course• Testing

– Only briefly mentioned in the course (types of testing, coverage criteria)

– A course on testing would make a nice complement to this course.

– Material in this course would be relevant in a testing course.

• Debugging– Once a bug is found, what is the source of the bug?– Occasionally, we would talk about enhancements to a tool

that would make debugging easier.– Students mentioned they would like to have learned more.

6

Key ChallengesKey Challenges• Geared toward undergraduates.

– Toned down the theory but did not eliminate it.– More practical, give students experience writing

software tools.

• No textbook.– Used relatively “easy-to-read” papers.

• Few prerequisites.– Only prerequisite was a course in algorithms.– No pre-reqs in compilers, software engineering,

automata theory, or software testing.– Focused primarily on C programs.

7

Format of the CourseFormat of the Course• Traditional lecture format

– Some in-class activities and discussions

• Assignments– Three programming assignments (in pairs)– Daily homework exercise (individual)

• Grading– Class participation 5 %– Homework 20 %– Exams (midterm and final) 30 %– Programming assignments 45 %

8

Course ContentCourse ContentCourse was broken down into four units:1. Program Analysis and Terminology

– similar to back-end compiler analysis– special attention on interprocedural and pointer

analysis

2. Dynamic Bug Detection– start of unit focused on testing– adding instrumentation to programs to:

• manage additional state about variables or memory used in the program

• check additional state to detect bugs

9

Course ContentCourse Content3. Static Bug Detection

– symbolic path simulation– constraint analysis– model checking

4. Other Topics– concurrent programs– safe languages– preventing security attacks (ex: StackGuard)

10

Assignment InfrastructureAssignment Infrastructure• Assignments were completed using a source to source

converter called SUDSE.– Contains static analyses for static bug detection.

• Converts a subset of C to instrumented C.– No enums, floating point values, or unions. – Since students were familiar with C++, the cin and cout I/O

statements were added. • Internal representation is an AST of simplified C statements for

easier analysis.– Side effects and short circuited operations are removed.– Complex expressions broken down into two or more simpler

expressions.• Also suitable for other types of assignments.

– code coverage tool– compilers– profiler

11

Example: SimplificationExample: Simplification// Original codeint bar(int x){ int a[5]; int i; for (i = 0; i < 5; i++)

{ a[i] = i * i; } return a[x];}

// Simplified code int bar(int x){ int a[5]; int i; int T1, T2; i = 0; T1 = i < 5; while (T1) { a[i] = i * i; i = i + 1; T1 = i < 5; } T2 = a[x]; return T2;}

12

AssignmentsAssignments1. Program analysis

– control-flow graph and data flow analysis– uses of uninitialized variables.

2. Dynamic bug detection– more on this later…

3. Static bug detection– null deference checker.– open-ended (students can use any technique

they wanted)

13

Dynamic Array CheckerDynamic Array Checker• In this assignment, students had to create a

tool that detects array out-of-bounds errors.• In part 1, students had to add instrumentation

calls to interesting statements:– Array declarations– Array references / pointer dereferences– Pointer assignments– Arrays/pointers go out of scope

• In part 2, students had to write the instrumentation routines to manage the array state and check for errors.

14

Example: Instrumented CodeExample: Instrumented Code#include "ptr_table.h"int bar(int x){ int a[5]; create_array_entry((void *) a, 5); int i; int T1, T2; i = 0; T1 = i < 5; while (T1) { check_array_bounds((void *) a, i, __FILE__, __LINE__); a[i] = i * i; i = i + 1; T1 = i < 5; } check_array_bounds((void *) a, x, __FILE__, __LINE__); T2 = a[x]; delete_array_entry((void *) a); return T2;}

15

Instrumentation FunctionsInstrumentation Functionsvoid create_array_entry(void *addr, int size);1. Adds an entry to the array table.2. Entry is indexed by addr and stores the size of the

array.

void check_array_bounds(void *addr, int index, const char *file, int line);

1. Search for the entry in the table using addr.2. Check to make sure that the index is not negative

and less than the size of the array.3. If the check fails, print out an error that contains the

file name and line number.

16

ResultsResults• Course was a success!

– Based on student feedback and personal observations.– Assignments represented a mix of several key areas in CS:

theory, OO-development, algorithms

• The second assignment was the most successful.– First assignment was not interesting or challenging to the

students who had background in compilers.– Not enough time for the open-ended third assignment.

• Mixed reactions on favorite course topics.– Program analysis was either favorite or least favorite. – Some students wanted to learn more about debugging.

17

Future Modifications Future Modifications • Allow more time for the third assignment.

– Reduce the first assignment to a warm-up exercise.

• Add section on debugging techniques.• Use existing bug detection tools (valgrind).• Explore software engineering practices.

– Where do software bug detection tools fit in different software engineering processes?

– Coding standards.

Questions and AnswersQuestions and Answers