• Black box (Finite State Machine) testing• Design for testability• Coverage measures• Random testing• Constraint-based testing• Debugging and test case minimization• Using model checkers for testing• Coverage revisited (“small model property”)

Topics in Testing We’ll Cover

Testing Project• Start with: interface and some definitions• You– Implement (with intentional bugs)– Provide some (possibly VERY minimal) unit tests– Exchange programs with two other students– Submit a test plan for colleagues’ code– Present & defend test plan (bonus points)– Build a tester for your colleagues’ code

• Must include some random testing or SPIN testing by final checkin– Test! (and submit bug tickets)– Submit bug fixes, evaluations, mini-reports at milestones– Turn in final test report

Testing Project• Your submitted code should contain two “subtle” bugs– Make sure your unit tests do not find the bug– Mail me: test cases and descriptions for the bugs• Subject MUST be: CS362W11 BUGS [onidid]

– You get to debug programs in lots of classes and in real life – in this class you get to bug a program intentionally, for once in your life

• I will see which testers find which bugs (over a sample of submitted programs)

Testing Project• Grading criteria–Design/implementation project/tester– Effectiveness of the tester–Presentation of test plan (extra credit)–Quality of test/bug reports• Can I figure out how you tested the system?• Can I figure out what wasn’t tested?

– Coverage, plus information on input restrictions• Can I figure out how reliable you think the code

is? What are key strengths/weaknessess?–How “interesting” and hard-to-find your

intentional bugs are

Testing Project

Expectations• You can program in C• You can figure out possibly poorly

written specifications (a key softwareengineering skill)

• Or know when to ask someone who can!• You can (learn to) use makefiles /

build system

“he who learns to play the harp learns to play by playing it”- Aristotle, Metaphysics, Book IX

Testing ProjectOh, right.What are we implementing and testing?A card game

Well defined notion of correctness“Specifications” not intended for implementation,

but intended to be unambiguousGames are a good example of formal specifications:

no one wants to have to “make up” the rule or interpret ambiguity in the middle of playing a game

Testing ProjectWhat are we implementing and testing?

Dominion (+ expansions)Card game published

by Rio Grande gamesPlayers start with

their own draw decksBuy cards to add to

decks (using cards)Some cards worth

points if in deck

Play thiscards &you candraw acard andgain twoactions(playing acard is anaction)

Schedule• April 7: First code checkin– Initial dominion implementation, expected to perhaps

be buggy/incomplete; not frozen• April 21: Test plan due• April 28: BUGS DUE• May 5: First test report (brief) due• May 26: Second test report (brief) due• June 2: “Late Midterm”• June 7: FINAL TEST REPORT DUE, PROJECT

COMPLETED, FINAL CODE FREEZE, TESTER CODE DUE

Grading & Other Admin StuffProject: 75%Late midterm: 15%(Possibly-in-class) quizzes/exercises/homework: 10%

My office hours: Tuesday, 10-11KEC 3067

TAs: Sheela Surisettysurisets @onid.oregonstate.eduOffice Hours: W 3-4

Assignment 1• Get on beaversource (use your onid account)– http://beaversource.oregonstate.edu– Become my “friend”– Ask to join the class project (CS362S11)• I’ll show you how, now

Assignment 1• Using svn, check out the code in the dominion

directory– Compile it using make all– Look over the code, and the specifications (the rules

in the rules subdirectory)

• Due as of first class next week