preparing students to incorporate stakeholder requirements in aerospace vehicle design

67
Preparing Students to Incorporate Stakeholder Requirements in Aerospace Vehicle Design Alexandra E. Coso Dissertation Defense Presentation Friday, March 28 th , 2014 Advisor: Dr. Amy Pritchett Committee: Dr. Brian German, Dr. Wendy Newstetter, Dr. Jerry Seitzman, Dr. Robin Adams, and Dr. Alan Jacobsen

Upload: brigit

Post on 25-Feb-2016

27 views

Category:

Documents


1 download

DESCRIPTION

Preparing Students to Incorporate Stakeholder Requirements in Aerospace Vehicle Design. Alexandra E. Coso Dissertation Defense Presentation Friday, March 28 th , 2014 Advisor: Dr. Amy Pritchett - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Preparing Students to Incorporate Stakeholder Requirements in

Aerospace Vehicle DesignAlexandra E. Coso

Dissertation Defense Presentation Friday, March 28th, 2014

Advisor: Dr. Amy PritchettCommittee: Dr. Brian German,

Dr. Wendy Newstetter, Dr. Jerry Seitzman, Dr. Robin Adams, and Dr. Alan Jacobsen

Page 2: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

2Image references at end of presentation

Page 3: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 3

Stakeholders in Aircraft Design

Stakeholders: All those affected by the system. They may be users or non-users. They need not be clients or decision-makers. Stakeholders may be major or minor, and the

ways in which they interact with a large-scale system are myriad.- Gibson, Scherer, Gibson 2007, p.315

Non-Users

Design Firms

AirlinesFlight Attendants & Passengers

Manufacturers & Maintainers

Ground Personnel

Pilots

Page 4: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

4

Stakeholders in Complex System Design

1Maguire, 2001; 2Green & Jordan, 1999; 3Hupe, 2011; 4Chua & Feigh, 2011; 5Pew, 2008; 6Landsburg et al., 2008

Stakeholders are critical to the successful design of products in many fields1,2

International Civil Aviation Organization (ICAO) has called for more integrated consideration of stakeholders, through social sustainability efforts3

Human-related design criteria are part of certification

requirements established by regulators

Additional efforts are needed to examine how to better incorporate stakeholder requirements consistently

within the design process4-6

AE industry members have explored a variety of tools and

resources for better incorporating stakeholders

Page 5: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

5

Engineering Design Education

Accreditation Board for Engineering and Technology (ABET) defines program

outcomes that include need to graduate engineers who understand impact of

design solutions on environment, economy, and society1

1ABET, 2011; 2ASEE, 2013; 3Hinkle, 2013; 4Dym et al., 2001; 5Ro et al., 2012; 6Jordan & Lande 2012; 7Zoltowski 2010; 8McMasters, 2003

Movement within some fields of engineering to design learning environments which place a higher value on the needs and limitations of stakeholders6-7

Recently, ASEE and NSF report based on a survey of industry representatives,

highlights a need to improve how we prepare students to synthesize

engineering, business, and societal perspectives in the design process2

Studies of design and engineering students’ pathways support the need to determine how to better prepare students to incorporate stakeholder considerations into design3-5

Page 6: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

6

Engineering Design Education

Accreditation Board for Engineering and Technology (ABET) defines program

outcomes that include need to graduate engineers who understand impact of

design solutions on environment, economy, and society1

1ABET, 2011; 2ASEE, 2013; 3Hinkle, 2013; 4Dym et al., 2001; 5Ro et al., 2012; 6Jordan & Lande 2012; 7Zoltowski 2010; 8McMasters, 2003

There is a movement within some fields of engineering to design learning environments which place a higher value on the needs and limitations of stakeholders.6-7

Recently, ASEE and NSF report based on a survey of industry representatives,

highlights a need to improve how we prepare students to synthesize

engineering, business and societal perspectives in the design process2

Studies of design and explorations of engineering students’ pathways in industry support the need to determine how to better prepare students to incorporate stakeholder

considerations into design.3-5

Research is limited in terms of how to improve students’ understanding of the broader context of design and ability to balance the performance

and stakeholder-related considerations within a vehicle design8

Research Goals and Contributions(1) Examine how stakeholder considerations are currently

integrated into aerospace vehicle systems design(2) Examine how stakeholder considerations are currently

integrated into aerospace engineering design education(3) Design empirically-informed and theoretically-grounded

educational interventions and assessment tools that support students’ integration of stakeholder considerations

(4) Isolate the characteristics of the interventions and learning environment that support students’ integration of stakeholder considerations

Page 7: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 7

RQ#4 - What educational interventions can enhance students’ understanding of and ability to integrate

stakeholder considerations into the design of an aerospace vehicle?

Phase I: Examination of Current Practice in Industry and Education

Phase II: Design of Educational Interventions and Assessment Tool

RQ#1 – What conditions enhance or inhibit the integration of

stakeholder considerations into the practice of aerospace vehicle

systems design?

RQ#3 – To what extent do aerospace engineering

students understand and take into consideration the effects

of design decisions on stakeholders?

RQ#2 – To what extent and how does the aerospace engineering

design curricula take into account stakeholder considerations?

Research Overview

Page 8: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

8

Research & Presentation Overview

Phase I: Examination

of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Practice of Aerospace

Engineering Design

Contributions and Future

Work

Aerospace Engineering

Design Curricula

Aerospace Engineering

Students’ Perceptions

Educational Intervention

Specifications

Page 9: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Examination of Practice of AE Vehicle Design

RQ#1 – What conditions enhance or inhibit the integration of stakeholder considerations into the practice of aerospace vehicle systems design?

9

Human factors & related researchAddressed need and provided recommendations for how to

integrate stakeholder considerations into complex system design

Aircraft design researchExamined stakeholder considerations

within multidisciplinary design, aerodynamic analyses, and studies of

the operational environment

However, this literature has not empirically studied how stakeholder considerations are integrated within aerospace vehicle design industry’s

multidisciplinary design teams

Page 10: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 10

Units of Analysis

Functional Groups

Embedded Multiple Case Study

Disciplinary Backgrounds Individuals

Research Setting

Aerospace Vehicle Design FirmTime Frame: 3 Months

Data Sources

First Round InterviewsSemi-Structured Protocol

Second Round Interviews Remaining Questions

Topics included: 1. Overview of the design process at the firm2. The challenges affecting the integration of stakeholder

considerations3. Previously used interventions to improve integration

Page 11: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 11

Results: Group/Team-Level

Contributing Factors

Group StructureThat supports addressing stakeholder

considerations

Common GoalThat addresses stakeholder considerations

Human Factors

SpecialistsDesign

Engineers

Pilot Compartment Group

In-House Pilots

Design Engineers

HF Specialists

Other Functional Groups

Page 12: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Image References included at end of presentation 12

Results: Individual-Level & Interaction-Level

Contributing Factors

Differing Perspectives About collaborations with

individuals from other disciplines

Shared Language & Boundary Objects

To support the exchange of ideas

Page 13: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 13

Educational Intervention Specifications

From Industry Case Study

Group/Team-LevelID1. Stakeholder-centric learning environment

Interaction-LevelID2. Language and vocabulary for discussing stakeholder considerationsID3. Tools and resources to support integration of stakeholder considerations

Individual-LevelID4. Cross-disciplinary problem-solving experiencesID5. Value of designs created from multiple perspectives

Page 14: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

14

Research & Presentation Overview

Phase I: Examination

of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Practice of Aerospace

Engineering Design

Contributions and Future

Work

Aerospace Engineering

Design Curricula

Aerospace Engineering

Students’ Perceptions

Educational Intervention

Specifications

Page 15: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 15

Examination of AE Design Education

RQ#2 – To what extent and how does the aerospace engineering design curricula take into account stakeholder considerations?

A review of publically-available syllabi from top aerospace

engineering programs and relevant published work

Review of the design processes presented in classical aerospace design textbooks (including Roskam; Raymer; Nicolai & Carichner) commonly used

within aircraft design capstone course

Page 16: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

16

Components of AE Design Curricula

1Peet & Mulder, 2004

Elective course offerings & systems engineering coursesIsolated course may be insufficient in training students to integrate

these topics into in their capstone courses and in the future1

Isolated Courses

Design Textbooks

Design Projects

Quantitative approximations (i.e. surrogates) are used to model some stakeholder characteristics

Mention of customer needs and certification at beginning of text, but not emphasized within the design processes themselves

[with exception of Nicolai and Carichner, 2010]

Distinctive from product design due to life span, purpose, and size of traditional aerospace vehicles

Impacts extent to which students experience entire design cycleContact with clients or stakeholders is limited

Page 17: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014

Educational Intervention Specifications

From Industry Case Study

ED1. Awareness of contextual and stakeholder considerations

ED2. Importance of considering broad context and wide variety of stakeholders

ED3. Methods for incorporating novel or qualitative stakeholder considerations

ED4. Interactions with clients or other stakeholders

17

From Examination of Design Curricula and Students’ Prior Knowledge

Group/Team-LevelID1. Stakeholder-centric learning environment

Interaction-LevelID2. Language and vocabulary for discussing stakeholder considerationsID3. Tools and resources to support integration of stakeholder considerations

Individual-LevelID4. Cross-disciplinary problem-solving experiencesID5. Value of designs created from multiple perspectives

Page 18: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

18

Research & Presentation Overview

Phase I: Examination

of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Practice of Aerospace

Engineering Design

Contributions and Future

Work

Aerospace Engineering

Design Curricula

Aerospace Engineering

Students’ Perceptions

Educational Intervention

Specifications

Page 19: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

19

Engineering Students’ Prior KnowledgeRQ#3 – To what extent do aerospace engineering students understand and

take into consideration the effects of design decisions on stakeholders?

1Carberry et al., 2010; 2Adams & Fralick, 2010; 3Ro et al., 2012

Design Understanding and Self-EfficacySelf-Efficacy in Design Scale1

andDesign Ranking Test2

Understanding of Contextual ConsiderationsContextual Competence Scale3

andOpen-Ended Follow-Up Question

Perceptions of the Role of Stakeholders in Design Scenario-Based Design Task

Page 20: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 20

Senior Aerospace Engineering StudentsSample: 80 senior AE students enrolled in an aircraft design capstone course

• 8 women (~10%)• 5 international students (~6%)•30 students (~38%) completed a previous design course •52 students (~65%) described having industry experience•50 students (~63%) took a problem-based learning course within the department

Page 21: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 21

Design UnderstandingImportant Design

ActivitiesProblem-ScopingCommunication

Not Considered “Most Important”

Iteration Making Trade-offs

Understand the Problem

Identify Constraints

Communicate

Evaluate

Test

Goal Set

Make Decisions

Making Trade-offs

Brainstorming

Planning

Generating Alternatives

Using Creativity

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Most Important Design Activities

AE Professionals (n = 20) VT Seniors (n = 53) Study Students (n = 80)

Portion of Sample

Desig

n Ac

tiviti

es

Page 22: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 22

Design UnderstandingImportant Design

ActivitiesProblem-ScopingCommunication

Not Considered “Most Important”

Iteration Making Trade-offs

Sketching

Abstracting

Building

Imagining

Decomposing

Synthesizing

Iterating

Prototyping

Visualizing

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Least Important Design ActivitiesAE Professionals (n = 20) VT Seniors (n = 53) Study Students (n = 80)

Percentage of Sample

Desig

n Ac

tiviti

es

Consistent with aircraft design students from

other aerospace engineering departments

Page 23: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 23

Understanding of Contextual & Stakeholder Considerations

Consistent with aircraft design students from other

aerospace engineering departments

Important Design ActivitiesProblem-ScopingCommunication

Not Considered “Most Important”

Iteration Making Trade-offs

Students consider…Technical and logistical

considerationsDesign’s effects on customer

Economic and mission constraints

Students do not see impact of…

Broader context (i.e. political, historical

considerations)Stakeholders other than

customer(i.e. pilots, maintainers,

passengers)

Page 24: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 24

Educational Intervention Specifications

From Industry Case Study

ED1. Awareness of contextual and stakeholder considerations

ED2. Importance of considering broad context and wide variety of stakeholders

ED3. Methods for incorporating novel or qualitative stakeholder considerations

ED4. Interactions with clients or other stakeholders

ED5. Importance of iteration and trade-offs

From Examination of Design Curricula and Students’ Prior Knowledge

Group/Team-LevelID1. Stakeholder-centric learning environment

Interaction-LevelID2. Language and vocabulary for discussing stakeholder considerationsID3. Tools and resources to support integration of stakeholder considerations

Individual-LevelID4. Cross-disciplinary problem-solving experiencesID5. Value of designs created from multiple perspectives

Page 25: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

25

Phase I Contributions

Phase I: Examination of Current Practice in

Industry and Education

Practice of Aerospace

Engineering Design

Aerospace Engineering

Design Curricula

Aerospace Engineering

Students’ Perceptions

Educational Intervention

Specifications

Empirical evidence of conditions which enhance or inhibit the integration of human factors and stakeholder considerations into

the aircraft design process.

Evaluation method to explore engineering students’ perceptions of how stakeholder

considerations are integrated into a complex system design

Page 26: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

26

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Educational Intervention

Specifications

Requirements Lab

Stakeholders in Design Labs

Page 27: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 27

Requirements Lab

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

GOAL: Introduce students to requirements, their importance, and how and why stakeholder-requirements should be incorporated into the requirements definition phase of the design process

Design Process

Page 28: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 28

Requirements Lab

Student Characteristics (from In-Class Evaluation)Wide spectrum of design knowledge and industry

experience

Nature of the SubjectDesign should be flexible for use within other aerospace

courses and programs

Context of Teaching/Learning Situation (Study-Specific)Two 50-minute lectures per week (80 students),

One three-hour lab session (40 students per lab session)

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 29: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 29

Requirements Lab

LO#3 - Students will be able to identify different stakeholders affected by a design problem

Based on ID3 & ED3

LO#4 - Students will be able to discuss how requirements affect design and the design process

Based on ED1

LO#6 - Students will be able to breakdown a problem and write requirements for performance and stakeholder

considerations Based on ID1

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 30: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 30

Requirements Lab

Opportunities for Direct & Immediate FeedbackClass-wide discussions about requirements and stakeholders

“Forward-looking” Assessment/Check for UnderstandingDesign case provides an open-ended, real-life context for

students to practice developing requirements and selecting verification and validation procedures

Further Opportunities to Assess Students’ UnderstandingAnonymous Student Feedback Forms

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 31: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Requirements Lab

1Kolb & Kolb, 2009 31

Concrete Experience(doing/having an experience)• Reflection on previous

design experiences • Related experience activity• Group work• Interactions with a client

Reflective Observation(reflecting on the experience)• Class-wide discussion about

requirements and stakeholders

Abstract Conceptualisation

(learning from the experience)• Class-wide discussion to

define characteristics of well-written requirements

Active Experimentation(trying out what you have

learned)• Design case• Group work• Interactions with stakeholders

Adaptation of Kolb’s Learning Cycle1

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 32: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Requirements Lab

1Kolb & Kolb, 2009 32

Concrete Experience(doing/having an experience)• Reflection on previous

design experiences • Related experience activity• Group work• Interactions with a client

Reflective Observation(reflecting on the experience)• Class-wide discussion about

requirements and stakeholders

Abstract Conceptualisation

(learning from the experience)• Class-wide discussion to

define characteristics of well-written requirements

Active Experimentation(trying out what you have

learned)• Design case• Group work• Interactions with stakeholders

Adaptation of Kolb’s Learning Cycle1

Designed such that students will create knowledge about stakeholders and requirements through activities that require them to experience and reflect on the relationships between stakeholders,

requirements, and design

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 33: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

33

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Educational Intervention

Specifications

Requirements Lab

Stakeholders in Design Labs

Page 34: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 34

Stakeholders in Design Labs GOALS: • Have students consider how stakeholder requirements and

concerns can be integrated throughout the aircraft design process

• Have students define how stakeholder requirements will be incorporated into the design process they will follow during their team design project

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 35: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 35

Stakeholders in Design Labs

Student Characteristics (from In-Class Evaluation)Wide spectrum of design knowledge and industry

experience

Nature of the SubjectDesign should flexible for use within other aerospace

courses and programs

Context of Teaching/Learning Situation (Study-Specific)Two 50-minute lectures per week (80 students),

One three-hour lab session (40 students per lab session)

Implemented several weeks into the semester

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 36: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 36

LO#1 - Students will be able to identify relevant stakeholders of a fixed wing design and explain how their

concerns affect (or affected) the design solution

Based on specifications ID1, ED1, & ED2

LO#2 - Students will be able to assess methods for integrating stakeholder considerations into the design

process

Based on ID1, ID3, & ED3

Stakeholders in Design Labs

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 37: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 37

Stakeholders in Design Labs

Opportunities for Direct & Immediate FeedbackStudent-generated design processes

Case study evaluationsClass-wide discussions

Additional Checks for UnderstandingCase study evaluations

Student reflectionsAnonymous student feedback forms

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 38: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

38

Stakeholders in Design Labs

1Kim, 2001; 2Bucciarelli, 1988

Social Constructivist Theory1

Knowledge is constructed through social interaction

Based on ID4 and ID5

Design as Reflective Practice2

Design requires negotiation among disciplines and perspectives

Based on ED5

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 39: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 39

Stakeholders in Design Labs

Lab #1

Framing the Lessons

Design Process Negotiation #1

• Pre-Lab Reflection • Individual Design Process

DrawingSituational

Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 40: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 40

Stakeholders in Design Labs

1. Understand the Problem

3. Seek Information

PlanningEvaluating

Generate Alternatives

Making Trade-offs

Make Initial Decisions

Testing Building

2. Visualize &

Brainstorm

Identify Constraints

Communicate Throughout!

Iterating

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 41: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 41

Stakeholders in Design Labs

Lab #1

Framing the Lessons

Design Process Negotiation #1

Introduction to Stakeholder Mapping

• Pre-Lab Reflection • Individual Design Process

Drawing

Tool for isolating stakeholder considerations that can affect a design

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 42: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 42

Stakeholders in Design Labs

Government Regulators(e.g. FAA, EASA)

Passengers

Commercial Aircraft

Pilots

Passengers

Maintainers/ Ground Personnel

Flight Attendants

Non-Users (e.g. people who live near an airport)

Design Firm/ Manufacturers

Suppliers

Wider seat widths

Improved Passenger Experience

Larger fuselage provides greater headroom when walking up and down the

aisle

Airlines

Stakeholder Mapping Technique (adapted from a technique used in Civil and

Environmental Engineering)

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 43: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 43

Stakeholders in Design Labs

Lab #1

Framing the Lessons

Design Process Negotiation #1

Introduction to Stakeholder Mapping

Introduction to Case Study Assessment

Case Study Assessment

Wrap-Up

• Pre-Lab Reflection • Individual Design Process

Drawing

Tool for isolating stakeholder considerations that can affect a design

Real-world examples of aircraft design process and impact of stakeholders on design

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 44: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 44

Stakeholders in Design Labs

Lab #2

Conclusion

Design Process Negotiation #2

Human-Centered Design Methods

Case Study Presentations

Framing the Lessons

• Student groups deconstruct an aircraft case study

• Outcomes include a stakeholder mapping and a design process

• Introduction to 5 HCD methods• Discussion of advantages and

disadvantages of methods and applicability to aircraft design

• Modification of initial design processes based on lessons learned in labs

• Indications of where stakeholders would be accounted for in the process

• Final reflections• Anonymous Student Feedback

Situational Factors

Learning Objectives

Assessment & Feedback

Learning Theory & Activities

Design Process

Page 45: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

45

Contribution of Intervention Design

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Requirements Lab

Stakeholders in Design Labs

Empirically-informed and theoretically-grounded

instructional designs that can be easily integrated into an

aircraft design capstone course or adapted for complex systems design course

Page 46: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

46

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Requirements Lab

Stakeholders in Design Labs

Implementation and Evaluation of Educational Interventions

Page 47: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 47

Evaluation of Educational Interventions: Research Questions

RQ#4 - What educational interventions can enhance students’ understanding of and ability to integrate stakeholder considerations into the design of an aerospace vehicle?

RQ4.1 - What characteristics of the educational interventions support students’ abilities to integrate stakeholder considerations into the design of an aerospace

vehicle?

RQ4.2 - To what extent can the interventions help students integrate stakeholder considerations into the design of an aerospace vehicle?

RQ4.3 - What characteristics of the engineering design learning experience and learning environment support or hinder students’ integration of stakeholder

considerations into the design of an aerospace vehicle?

Page 48: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

48

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Requirements Lab

Stakeholders in Design Labs

Evaluation of Educational

Interventions

Evaluation of the Learning

Environment

Stakeholders in Design

Rubric

Page 49: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

491Adapted from Moon et al 2011 & Kaufman et al 1995

Resources & Processes

• Students’ satisfaction with interventions

Acquisition

• Achievement of learning objectives

Application

• Adoption of intervention content in class

Organization & Course

• Factors within the learning environment

Societal

• Impact after and outside the course

Evaluation of Educational Interventions:Evaluation Framework1

RQ4.1 - What characteristics of the educational interventions support students’ abilities to integrate stakeholder considerations into the design of an aerospace vehicle?

RQ4.2 - To what extent can the interventions help students integrate stakeholder considerations into the design of an aerospace vehicle?

Page 50: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 50

Results Summary: Data from Interventions

Requirements Lab [data from design case & feedback forms]

• Students formulated well-constructed high-level requirements and many students were able to identify a diverse group of stakeholders

• Students considered both performance and stakeholder considerations and most groups included at least three different verification and validation procedures

• Stakeholder-centric focus of the lab was not clear to students

Stakeholders in Design Labs [data from all activities & feedback forms]

• Stakeholder-centric focus supported students’ integration of stakeholder considerations in a variety of different ways in final design processes

• Case studies illustrated challenges of cross-disciplinary design• Misconceptions about HCD methods may have also developed over the

course of these lab sessions

Page 51: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 51

Results Summary: Post-Intervention Evaluation

Design Understanding at Post-Evaluation

• Iteration: 23.8% of the students in the pre-evaluation selected it, as compared with 69.6% of students in the post-evaluation

• Making Trade-offs: 33.8% of the students in the pre-evaluation selected it, as compared with 48.2% of students in the post-evaluation

Stakeholder Considerations

• Students reported significantly more stakeholder-related considerations in the post-evaluations as compared with the pre-evaluations

Page 52: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

52

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Requirements Lab

Stakeholders in Design Labs

Evaluation of Educational

Interventions

Evaluation of the Learning

Environment

Stakeholders in Design

Rubric

Page 53: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

53

Stakeholders in Design Rubric

REFERENCES: Zoltowkski, 2010; Crismond and Adams, 2012

Stakeholder Integration by Design Stage Design Understanding Stakeholder Integration

Examines students’ integration of stakeholder considerations into 3 stages of design

Specifically, students’• intent, • application of

methods, and • overall success

Scores students based on their application of the design process

0pts: Lack of knowledge about the process 4pts: abstract the process and be a reflective designer

Scores students based on the extent to which they integrated stakeholder considerations

0pts: Lack of appreciation

4pts: Leveraging multiple perspectives for a more innovative solution

Page 54: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 54

Examination of Students’ Team Design Projects

Problem Definition Stage

Technology Integration Stage

Overall Design

Most teams did not state an intention to incorporate stakeholder considerations [as noted by 67.6% of judging panels’ scores] Or apply a design process that supported integration of stakeholder considerations [as noted by 58.8% of judging panels’ scores]

Most teams did not apply a design process that supported the integration of stakeholder considerations [as noted by 60.6% of judging panels’ scores]

68.6% of judges’ scores suggested that students were somewhat successful at integrating stakeholder considerations into their final design solutions

Page 55: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 55

Examination of Students’ Team Design Projects

Example of Variations in Scores Across TeamsTeam E received 0’s from all 3 judges

Team C received two 3’s and one 2 from their judges

Median: 2Applies design process as presented in course and demonstrates how design

solution improved through an iterative design process

Median: 2Identifies stakeholder considerations

at isolated points in the design process, but overall these

considerations are not the basis for design decisions and are not

addressed consistently through design

Design Understanding[0 to 4pt Scale]

Stakeholder Integration[0 to 4pt Scale]

0 1 2 3 402468

1012141618

Frequency of Scores for Students' Team Design Project Reports

Design Understanding Stakeholder Integration

Score

Scor

e Fr

eque

ncy

Page 56: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

56

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Requirements Lab

Stakeholders in Design Labs

Evaluation of Educational

Interventions

Evaluation of the Learning

Environment

Stakeholders in Design

Rubric

Page 57: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

571Adapted from Moon et al 2011 & Kaufman et al 1995

Resources & Processes

• Students’ satisfaction with interventions

Acquisition

• Achievement of learning objectives

Application

• Adoption of intervention content in class

Organization & Course

• Factors within the learning environment

Societal

• Impact after and outside the course

Evaluation of Educational Interventions:Evaluation Framework1

RQ4.3 - What characteristics of the engineering design learning experience and learning environment support or hinder students’ integration of stakeholder

considerations into the design of an aerospace vehicle?

Page 58: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 58

Examination of the Learning Environment

TASK

[Ames, 1992]

Team Design Project Requirements

Characteristics of Educational

Interventions

Reliance on Design Process Structure

Available Tools and Resources

Page 59: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 59

Examination of the Learning Environment

TASK

[Ames, 1992]

EVALUATIONPRACTICES

Team Design Project Requirements

Design of an Assessment Rubric

Characteristics of Educational

Interventions

Reliance on Design Process Structure

Available Tools and Resources

Use of Project Requirements as

Evaluation Strategy

Evaluation through Interactions with

External Audience

Page 60: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 60

Examination of the Learning Environment

TASK

[Ames, 1992]

EVALUATIONPRACTICES

DISTRIBUTION OF AUTHORITY

Team Design Project Requirements

Design of an Assessment Rubric

Characteristics of Educational

Interventions

Reliance on Design Process Structure

Available Tools and Resources

Use of Project Requirements as

Evaluation Strategy

Evaluation through Interactions with

External Audience

Transition from Instructor-centered,

Individual-based Learning

Environment to

Student-centered, Team-based

Learning Environment

Page 61: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

61

Additional Phase II Contributions

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Requirements Lab

Stakeholders in Design Labs

Evaluation of Educational

Interventions

Evaluation of the Learning Environment

Stakeholders in Design

Rubric

Empirical evidence of conditions

within the learning environment that impact students’

integration of stakeholder

considerations into a complex systems

design process.

Assessment tool to support students’ understanding of how to design with stakeholders in mind, and to assess students’ abilities to consider the impact of complex system design

decisions on stakeholders.

Page 62: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

62

Research & Presentation Overview

Phase I: Examination of Current Practice in

Industry and Education

Phase II: Design of

Educational Interventions

and Assessment

Tool

Contributions and Future

Work

Page 63: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 63

Research Contributions & Future WorkIndustry Case Study – empirical evidence of conditions that enhance or inhibit the

integration of human factors & stakeholder considerations into the aircraft design process

In-Class Evaluation & Stakeholders in Design Rubric – evaluation methods to (1) explore engineering students’ perceptions of how stakeholder considerations are integrated into the vehicle design process, (2) support students’ understanding of how to design with stakeholders in mind, and (3) assess students’ abilities to consider the impact of complex system design decisions on stakeholders

Requirements Lab & Stakeholders in Design Labs - empirically-informed and theoretically-grounded instructional designs that can be easily integrated into an aircraft design

capstone course or adapted for complex systems design course.

Examination of Learning Environment - empirical evidence of conditions within the learning environment that impact students’ integration of stakeholder considerations into

a complex systems design process

Page 64: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Coso - Ph.D. Thesis Defense - March 28th, 2014 64

Research Contributions & Future WorkDiscipline-based education research that informs teaching and learning within

aerospace engineering and supports continued research into this interdisciplinary field

Starting point for future research into pedagogical techniques which support students’ integration of stakeholder considerations into the design of complex systems

Page 65: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Questions?

Acknowledgments

NSF Graduate Research Fellowship under Grant No. DGE-0644493.

NSF under Grant #EEC-1059472 to the American Society for Engineering

Education.

Thank you to all of the interview participants, the instructors, and the students for their assistance with this

study.

Thank you to my amazing peer debriefers, Subject Matter Experts,

and the former and current members of the Cognitive Engineering Center

at Georgia Tech!

Page 66: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Related PublicationsJournal Papers+ (Submitted) Coso, A. and Pritchett, A. (2013) “The Role of Design Teams in the

Integration of Stakeholder Considerations.” Journal of AircraftConference Proceedings+ (Accepted) Coso, A. and Pritchett, A. (2014) “The Development of a Rubric to

Evaluate and Promote Students’ Integration of Stakeholder Considerations into the Engineering Design Process.” American Society for Engineering Education: Proceedings of the 2014 ASEE Annual Conference and Exposition. Indianapolis, IN

+ Coso, A. and Pritchett, A. (2014) “Incorporating Stakeholder Considerations in the Aircraft Design Process: A Focus on Aircraft Design Education.” Proceedings of AIAA SciTech. National Harbor, MD.

+ Coso, A. and Pritchett, A. (2013) “The Integration of Stakeholder Requirements within Aerospace Engineering Design Education.” American Society for Engineering Education: Proceedings of the 2013 ASEE Annual Conference and Exposition. Atlanta, GA.

Coso - Ph.D. Thesis Defense - March 28th, 2014 66

Page 67: Preparing Students to Incorporate Stakeholder Requirements in  Aerospace Vehicle Design

Image Referenceshttp://claesjohnson.blogspot.com/2012_07_01_archive.htmlhttp://www.amazon.com/Airplane-Design-Part-Preliminary-Airplanes/dp/188488542Xhttp://www.amazon.com/Mission-Analysis-Edition-TECHNOLOGY-LIBRARY/dp/0792359011https://www.aiaa.org/PubDetail.aspx?id=1796http://www.aviationweek.com/Blogs.aspx?plckBlogId=Blog:27ec4a53-dcc8-42d0-bd3a-01329aef79a7&plckPostId=Blog:27ec4a53-dcc8-42d0-bd3a-01329aef79a7Post:6e334c9f-0810-4874-a0be-d7e023b58135http://orbiterchspacenews.blogspot.com/2010/11/nasa-selects-companies-for-advanced.htmlhttp://www.disinfo.com/tag/spacex/http://www.open.ac.uk/about/campus/jennie-lee-research-labs/files/jennieleeresearchlabs/imagecache/node_standard/HCI%202%20Lab-1.jpghttp://www.manvitas.com/images/Corporate.jpghttp://www.inc.com/uploaded_files/image/speed-up-your-prototyping-process-pop_9632.jpghttp://www.kelvinyoung.com/v6/work/storyboard_01.jpghttp://www.nngroup.com/reports/paper-prototyping-training-video/

For literature references, view dissertation text

Coso - Ph.D. Thesis Defense - March 28th, 2014 67