Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Chief Chefs of Z to Alloy: Using A KitchenExample to Teach Alloy with Z

Sureyya Tarkan Vibha Sazawal

University of Maryland

November 6, 2009

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Outline

Introduction

Z to Alloy Tutorial

Case Studies with Novice Students

Discussion

Conclusion

Related Work

ReferencesTarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

The Problem

I Z is a well-defined and well-known specification language.

I Model checkers enable automatic checking of specifications.

I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.

I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

The Problem

I Z is a well-defined and well-known specification language.

I Model checkers enable automatic checking of specifications.

I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.

I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

The Problem

I Z is a well-defined and well-known specification language.

I Model checkers enable automatic checking of specifications.

I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.

I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

The Problem

I Z is a well-defined and well-known specification language.

I Model checkers enable automatic checking of specifications.

I The lack of student interest in Formal Methods causeseducators to focus only on one aspect.

I Educators need a technique to master tool usage besides Zwithout burdening the students to learn a new language.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Contribution

I Alloy is a model checking tool that provides a languagesubstantially similar to Z.

I There is a lack of examples and educational material for thetransition from Z to Alloy.

I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.

I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Contribution

I Alloy is a model checking tool that provides a languagesubstantially similar to Z.

I There is a lack of examples and educational material for thetransition from Z to Alloy.

I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.

I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Contribution

I Alloy is a model checking tool that provides a languagesubstantially similar to Z.

I There is a lack of examples and educational material for thetransition from Z to Alloy.

I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.

I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Contribution

I Alloy is a model checking tool that provides a languagesubstantially similar to Z.

I There is a lack of examples and educational material for thetransition from Z to Alloy.

I To populate the materials, we suggest the use of an easy andyet comprehensive and self-explanatory example.

I We share our experiences with novice students using ouronline tutorial designed around a Kitchen Environmentreal-life example.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Birthday Book

I It records people’s birthdays and places a reminder when thatday comes.

I It can be populated and depopulated, and there is a searchoption.

I Its Z specification reviews concepts from set theory andteaches the notation.

I An Alloy model is used to automatically check someassertions.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Birthday Book

I It records people’s birthdays and places a reminder when thatday comes.

I It can be populated and depopulated, and there is a searchoption.

I Its Z specification reviews concepts from set theory andteaches the notation.

I An Alloy model is used to automatically check someassertions.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Birthday Book

I It records people’s birthdays and places a reminder when thatday comes.

I It can be populated and depopulated, and there is a searchoption.

I Its Z specification reviews concepts from set theory andteaches the notation.

I An Alloy model is used to automatically check someassertions.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Birthday Book

I It records people’s birthdays and places a reminder when thatday comes.

I It can be populated and depopulated, and there is a searchoption.

I Its Z specification reviews concepts from set theory andteaches the notation.

I An Alloy model is used to automatically check someassertions.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.

I The primitive object types and built-in functions are given.

I We define a list of assertions that are crucial for our system.

I An event-driven programming approach is suggested.

I We develop an approximate Z specification to later convert itinto a full Alloy model.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.

I The primitive object types and built-in functions are given.

I We define a list of assertions that are crucial for our system.

I An event-driven programming approach is suggested.

I We develop an approximate Z specification to later convert itinto a full Alloy model.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.

I The primitive object types and built-in functions are given.

I We define a list of assertions that are crucial for our system.

I An event-driven programming approach is suggested.

I We develop an approximate Z specification to later convert itinto a full Alloy model.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.

I The primitive object types and built-in functions are given.

I We define a list of assertions that are crucial for our system.

I An event-driven programming approach is suggested.

I We develop an approximate Z specification to later convert itinto a full Alloy model.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I It simulates the actions of a kitchen chef to direct othercook(s) in the preparation of a dish.

I The primitive object types and built-in functions are given.

I We define a list of assertions that are crucial for our system.

I An event-driven programming approach is suggested.

I We develop an approximate Z specification to later convert itinto a full Alloy model.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Schema and Signature

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I At each step, we point out the differences and similaritiesbetween the two versions.

I We explain what the limitations of model checking are.

I We formalize a guarded event implementation.

I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I At each step, we point out the differences and similaritiesbetween the two versions.

I We explain what the limitations of model checking are.

I We formalize a guarded event implementation.

I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I At each step, we point out the differences and similaritiesbetween the two versions.

I We explain what the limitations of model checking are.

I We formalize a guarded event implementation.

I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Tutorial Content

Kitchen Environment

I At each step, we point out the differences and similaritiesbetween the two versions.

I We explain what the limitations of model checking are.

I We formalize a guarded event implementation.

I Our final Alloy model is long but once a rough Z specificationis at hand, Alloy development is not cumbersome.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Implementation

Design

I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.

I The questions ask the student to think beyond what is givenrather than passively reading the text.

I The students reveal the correct answer once they are satisfiedwith their answer.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Implementation

Design

I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.

I The questions ask the student to think beyond what is givenrather than passively reading the text.

I The students reveal the correct answer once they are satisfiedwith their answer.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Implementation

Design

I With Kolb’s theory [4], we decided to have multiple-choicequestions within the tutorial.

I The questions ask the student to think beyond what is givenrather than passively reading the text.

I The students reveal the correct answer once they are satisfiedwith their answer.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Implementation

The Online Interactive Tutorial

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Basics

I Task & ProcedureI The students were asked to answer all the questions in the

tutorial and fill out the surveys.I Each student spent 1-3 hours working on the tutorial.

I Student BackgroundI 8 students took our tutorial.I Students strongly agreed that they have experience with small-

and medium-sized software projects.I Students strongly agreed that they have not had much

experience with formal methods.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Basics

I Task & ProcedureI The students were asked to answer all the questions in the

tutorial and fill out the surveys.I Each student spent 1-3 hours working on the tutorial.

I Student BackgroundI 8 students took our tutorial.I Students strongly agreed that they have experience with small-

and medium-sized software projects.I Students strongly agreed that they have not had much

experience with formal methods.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Tutorial Questions

I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.

I Both the Birthday Book specification and model werewell-received.

I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Tutorial Questions

I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.

I Both the Birthday Book specification and model werewell-received.

I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Tutorial Questions

I Initial questions reviewed the math background but later, morequestions became related to the specifics of the examples.

I Both the Birthday Book specification and model werewell-received.

I Excluding 2 students who dropped out of the KitchenEnvironment, there were 45/58 correct answers on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Assessment

I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.

I The questions were evaluated based on legitimate rationalerather than exact syntax.

I Students received 2/4 on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Assessment

I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.

I The questions were evaluated based on legitimate rationalerather than exact syntax.

I Students received 2/4 on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Assessment

I The quiz questions asked the student to type one line of Alloycode to complete and correct an Address Book model.

I The questions were evaluated based on legitimate rationalerather than exact syntax.

I Students received 2/4 on average.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Student Feedback

I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.

I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.

I Students agreed that the teaching material and the exampleswere good and that they were satisfied.

I Students wanted visualizations (pictures, diagrams, tables).

I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Student Feedback

I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.

I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.

I Students agreed that the teaching material and the exampleswere good and that they were satisfied.

I Students wanted visualizations (pictures, diagrams, tables).

I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Student Feedback

I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.

I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.

I Students agreed that the teaching material and the exampleswere good and that they were satisfied.

I Students wanted visualizations (pictures, diagrams, tables).

I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Student Feedback

I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.

I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.

I Students agreed that the teaching material and the exampleswere good and that they were satisfied.

I Students wanted visualizations (pictures, diagrams, tables).

I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Results

Student Feedback

I Students’ appreciation of formal methods, formalspecification, and model checking were moderate.

I Students strongly agreed with our statements on the ease oftransitioning from Z to Alloy with the tutorial.

I Students agreed that the teaching material and the exampleswere good and that they were satisfied.

I Students wanted visualizations (pictures, diagrams, tables).

I Students needed examples that are between the BirthdayBook and the Kitchen Environment in complexity.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Findings

I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).

I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.

I Students criticized the length of the tutorial but not thecontent.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Findings

I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).

I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.

I Students criticized the length of the tutorial but not thecontent.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Findings

I The student performance did not show a huge drop betweenKitchen Environment (77.5%) and Birthday Book (88%).

I Students believe that Z and Alloy are similar and can betaught together but that textbooks using toy examples for Zor Alloy are insufficient.

I Students criticized the length of the tutorial but not thecontent.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Final Remarks

I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.

I We recommend educators to focus on the Alloy tool whileteaching the Z language.

I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Final Remarks

I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.

I We recommend educators to focus on the Alloy tool whileteaching the Z language.

I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Final Remarks

I Novice students are able to learn Z and Alloy and answerquestions about an intermediate-sized example after only acouple hours of study.

I We recommend educators to focus on the Alloy tool whileteaching the Z language.

I It is necessary to provide well-explained and interestingintermediate-level examples for the Z to Alloy transition.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

I Interactive application of Formal MethodsDean’s interactive case e-study [3], Pandora’s e-tutor [2]

I Similar Educational MaterialsPiaget’s genetic epistemology theory [8], Z to SPIN module [9]

I Content-wiseReal-life examples such as a Bluetooth communicationprotocol, a Eurovision Song Contest [1, 5, 6, 7]

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

Brakman, H., Driessen, V., Kavuma, J., Bijvank, L.N., Vermolen, S.Supporting formal method teaching with real-life protocols.In Formal Methods in the Teaching Lab: Examples, Cases, Assignments andProjects Enhancing Formal Methods Education, pages 59–68, McMasterUniversity, Canada, 2006.

Broda, K., Ma, J., Sinnadurai, G., Summers, A.Friendly e-tutor for natural deduction.In TFM: Practice and Experience, London, 2006. BCS-FACS.

N. Dean.Development of an interactive case e-study.In TFM: Practice and Experience Workshop, pages 13–20, Oxford BrookesUniversity, 2003. BCS-FACS.

D. Kolb.Experiential learning: Experience as the source of learning and development.Prentice-Hall, Englewood Cliffs, NJ, 1984.

P. Larsen.Two courses on vdm++ for embedded systems: Learning by doing.In Formal Methods in the Teaching Lab: Examples, Cases, Assignments andProjects Enhancing Formal Methods Education, pages 21–26, Canada, 2006.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy

Introduction Z to Alloy Tutorial Case Studies with Novice Students Discussion Conclusion Related Work References

D. Lightfoot.‘voici les votes!’ – formal specification as light entertainment: An example ofaudience participation in developing a specification.In TFM: Practice and Experience Workshop, pages 71–75, Oxford BrookesUniversity, 2003. BCS-FACS.

Reed, J.N., Sinclair, J.E.Motivating study of formal methods in the classroom.LNCS, 3294:32–46, 2004.

S. Rosa.Designing algorithms in high school mathematics.LNCS, 3294:17–31, 2004.

J. Rudall.From z to spin in one module.In TFM: Practice and Experience Workshop, pages 71–75, Oxford BrookesUniversity, 2003. BCS-FACS.

Tarkan & Sazawal UMD

Chief Chefs of Z to Alloy