NAVIGATING A PATHWAY BETWEEN ACADEMIC STANDARDS AND

OUTCOMES-FOCUSED THINKINGIN ENGINEERING

Wageeh BolesNational Teaching Fellow

ALTF Forum Presentation – June 2013

Support for the fellowship has been provided by the Australian Government Office for Learning and Teaching. The views in this presentation do not necessarily reflect the views of the Australian Government Office for Learning and Teaching .

www.nationalteachingfellowshipboles.com

2

OUTLINE

• Context and Background

• Methodology

• Fellowship activities

• Fellowship impact and outcomes

• Concluding remarks

Wageeh Boles

3

• The International Scene• National scene • The Local scene

Context and Background

Wageeh Boles

4

The International Scene

International Accreditation agreements

Focus on what is learned rather than what is taught

WorkerMobility

Context and Background (cont.)

Wageeh Boles

5

The National Scene

Excellence in Research Australia, ERA

Academic Standards and Threshold Learning Outcomes

Tertiary Education Quality and Standards Agency,

TEQSA

Focus on QualityAustralian Qualifications Framework, AQF

Context and Background (cont.)

Wageeh Boles

6

The Local Scene

Challenges facing early- and mid-career Academics

Demographics of the Academic workforce

Responsibilities of Heads of Schools or Departments

Context and Background (cont.)

Wageeh Boles

7

Fellowship Objectives• Identify and evaluate international and Australian initiatives

for effective teacher engagement and curriculum reform.• Assist in mapping program outcomes onto appropriately

aligned assessment tasks.• Establish a developmental model for designing assessment

tasks capable of providing evidence of learning.• Facilitate the development of early- and mid-career

academics in teaching as well as research.

Context and Background (cont.)

Wageeh Boles

8

FELLOWSHIP PROGRAM OVERVIEW

Wageeh Boles

9

Research … Why?Teaching … Why?Mentoring … Why?

Focus on people and outcomes

Wageeh Boles

10

Professional Practice Research* is used with investigations characterised as:

• “Critically-informed,” situated within global and national trends in engineering education,

• “Politically-activist,” with the researcher acting as an agent of curriculum and cultural change, and

• “Action-oriented,” involving university leadership, and working with Heads of Schools as mentors.

Methodology

* Macpherson, I., Brooker, R., Aspland, T., & Cuskelly, E. (2010). Constructing a territory for professional practice research: Some introductory considerations. In Action research in education: Fundamentals of applied research, Vol. 1, Sage Publications Ltd. [ISBN 9781848606838]

Wageeh Boles

11

Locations visited in the USAOverseas Study

Wageeh Boles

12

People interviewed in the USA1. Dr. Norman Fortenberry, Director, American Society of Engineering Education, ASEE, Washington D.C., USA2. Dr. Michael Milligan, Executive Director, Accreditation Board for Engineering and Technology, ABET, Baltimore, MD, USA3. Dr. Ashley Ater Kranow, Managing Director, Accreditation Board for Engineering and Technology, ABET, Baltimore, MD, USA4. Professor Milton Cox, Miami University of Ohio, USA5. Assistant Professor Peter Jamieson, Miami University of Ohio, OH, USA6. Professor Anna Dollar, Miami University of Ohio, OH, USA7. Professor Osama Ettouney, Miami University of Ohio, OH, USA8. Dr. Donald Ucci, Miami University of Ohio, OH, USA9. Dr. James Moller, Miami University of Ohio, OH, USA10. Mr. Peter Jamieson, Miami University of Ohio, OH, USA11. Dr. Dmitriy Garmatyuk, Miami University of Ohio, OH, USA12. Professor Jeffrey Froyd, Director of Faculty and Organisational Development, Texas A & M University, TX, USA13. Associate Professor Susan Lord, Director, Optoelectronics Laboratory, University of San Diego, CA, USA14. Professor Geoffrey Orsak, Dean of the SMU Lyle School of Engineering, Southern Methodist University, TX, USA15. Associate Professor James Dunham, Associate Dean, Academic Affairs, Lyle School of Engineering, Southern Methodist University, TX, USA.16. Professor Richard Felder, North Carolina State University, NC, USA17. Associate Professor Mary Besterfield-Sacre, Swanson School of Engineering, Director, Engineering Education Research Center, University of Pittsburgh, PA, USA.18. Professor Sam Spiegel, Associate Director, Engineering Education Research Center, University of Pittsburgh, PA, USA.19. Professor Larry Schuman, Senior Associate Dean for Academic Affairs, Swanson School of Engineering, University of Pittsburgh, PA, USA20. Assistant Professor Melissa Bilec, Director, Construction Management Program and Green Construction Program, University of Pittsburgh, PA, USA.21. Professor Joe McCarthy, William Kepler Whiteford, Department of Chemical and Petroleum Engineering, University of Pittsburgh, PA, USA. 22. Assistant Professor Karen Bursic, Undergraduate Program Director, University of Pittsburgh, PA, USA23. Assistant Professor Natasa Vidic, research interests in assessment and evaluation), University of Pittsburgh, PA, USA24. Professor Annette Jacobson, Teaching Professor of Chemical Engineering, A/ Dean for Undergraduate Education, Director, Carnegie Mellon University, PA, USA25. Professor Paul Steif, , Mechanical Engineering, Engineering education leader, Carnegie Mellon University, PA, USA26. Associate Professor Marsha Lovett, Associate Teaching Professor, Director, Learning and Problem Solving Lab, Carnegie Mellon University, PA, USA27. Professor Michael Prince, Professor of chemical engineering, engineering education leader, Bucknell University, PA, USA28. Associate Professor Candace Stefanou, Associate professor of education, Bucknell University, PA, USA29. Associate professor Margot Vigeant, Associate professor of Chemical engineering, Bucknell University, PA, USA30. Associate Professor David Kelley, Chair, Electrical Engineering Dept. Bucknell University, PA, USA31. Professor Keith Buffinton, Dean of Engineering and Professor of Mechanical Engineering, Bucknell University, PA, USA32. Associate professor Geoffrey Schneider, Associate professor of economics Director of the Teaching and Learning Center, Bucknell University, PA, USA33. Associate professor Timothy Raymond, Associate Professor of chemical engineering, Bucknell University, PA, USA.34. Professor Maurice Aburdene, Professor of Electrical Engineering (in charge of accreditation), Bucknell University, PA, USA.35. Rosemary Aguilar, Director, Professional Development and Curriculum, The Infinity Project, Southern Methodist University, TX, USA.36. Diana McAtee, Director, Academic Relations, The Infinity Project, Southern Methodist University, TX, USA.37. Professor Sam Spiegel, Associate Director of Outreach & Development, EERC, Pittsburgh, PA.

Wageeh Boles

13

Miami University, Oxford, Ohio Faculty Learning Communities

Accreditation Board for Engineering and Technology, ABET, Baltimore, Maryland

Focus on OutcomesAmerican Society for Engineering Education, ASEE,

Washington DC Global Issues Southern Methodist University, Dallas, Texas

Industry

collaboration

Places and foci - 1

Wageeh Boles

14

Bucknell University, Lewisburg, PA Addressing misconceptions

Pittsburgh University, Pittsburgh, PA Model Eliciting Activities, MEAs, and

inverted classroomCarnegie Mellon University, Pittsburgh, PA

Concept InventoriesFrontiers in Education, FIE, conference, Rapid City,

South Dakota Celebrating innovation

Places and foci - 2

Wageeh Boles

15

• WHAT, WHY, WHO, and HOW? – What is the FIRE program? – What were the objectives? – How was the program implemented? – Which Australian universities participated? – Who was involved at the participating

universities?– How was it managed?

FIRE program

FIRE = Fellow-In-Residence Engagement

Wageeh Boles

16

FIRE program activities with:

University and faculty leadership:• Meetings with the deputy vice-chancellor

(academic) or equivalent; the executive dean; the assistant dean (learning and teaching) or equivalent.

• An open seminar (university wide) on the issues of assessment and mentoring.

• A workshop (co-facilitated by an international scholar, when possible).

Wageeh Boles

17

FIRE program activities with:

Head of school: • Explore available mentoring programs and

how they are achieving their objectives.• Work as a shadow mentor with the head

of school for academic staff.• Discuss relevant issues, as may be

determined during the visits.

Wageeh Boles

18

FIRE program activities with:

Academics:Individual and small group meetings to explore:

– How can assessment be used for learning?– Designing specific assessment tasks.– How can classroom activities encourage student learning?– What are possible resources (websites, papers, books,

packages, etc.)?– What are teaching for learning needs and possible ways to

meet them?– Networking opportunities, being part of a learning community.– How to achieve steady enhancement without too much

overhead.– Ideas on balancing teaching.– Research, etc. Wageeh Boles

19

A sample discussion starter

• Which of the following three scenarios best describes your assessment practices?

• Your answer must reflect what actually happens, rather than what you think should happen.

• Ask yourself, if someone else examined my assessment, will they arrive at the same conclusion? What about evidence of learning?

Wageeh Boles

20

Learning Objectives

Assessment task

Scenario #1

Wageeh Boles

21

Learning Objectives

Scenario #2

Assessment task

Wageeh Boles

22

Learning Objectives

Scenario #3

Assessment task

Wageeh Boles

23

Evidence of learning

• Why is evidence of learning important?– For students so that they know how they are

progressing.– For academics so that they take appropriate

actions to support learning, modify teaching methods, examine or provide resources, link with course objectives, etc.

– For accreditation bodies so that they can make an informed assessment of how institutions are meeting their obligations to deliver certified and competent engineers

Wageeh Boles

24

• Impact on participating Australian universities• Impact on individuals and groups• Impact on US participants• Impact through the reference group• Insights gained • Impact on the fellow• Evidence of change

Fellowship impact

Wageeh Boles

25

• Seminars, workshops and papers • Assessment design process and exemplars • Assessment resources • Mentoring guides • The FIRE program as a model for academic staff

development and leadership support• Data from the US interviews • Website:

www.nationalteachingfellowshipboles.com

Fellowship outcomes

Wageeh Boles

26

• The fellowship was motivated by the changes in the higher education environment, including the demographics of academics.

• The program had a focus on academic standards and evidence of learning, and also on mentoring.

• Two components; overseas study, and a Fellow-In-Residence Engagement, FIRE, program.

Concluding remarks

Wageeh Boles

27

• The links between course objectives, unit objectives and how specific assessment items can address those objectives systematically, are often missing or not made explicit.

• When the evidence required to demonstrate the achievement of specific learning outcomes is not clearly identified, it will be difficult to choose the most appropriate assessment instrument.

Concluding remarks (cont.)

Wageeh Boles

28

• The lack of understanding the role of assessment or the ability to design it appropriately, makes it difficult to defend claims of attainment of objectives, academic standards, or student learning outcomes.

• This presentation gave a very brief overview of the fellowship activities and impact. Challenges faced and how they have been addressed or managed are highlighted in the fellowship final report.

Concluding remarks (cont.)

Wageeh Boles

29

“One might argue that we already have sufficient

research on student success (at university) ... What

is missing in our view is the ability to transform the

knowledge that we have into practical knowledge.”

Vincent Tinto

Professor and chair of the higher education program, Syracuse University

Speaking at the US National Symposium on Student Success at College & University

November 2006

A quote ...

Wageeh Boles

30

I believe that our role is not only to help students become better learners, but to also contribute to building their characters; enabling them to become valuable members of society.

... and a thought...

Wageeh Boles

31

Questions?

Wageeh Boles