Existing Academic Activity in Support of Systems

Engineering

Dennis M. Buede

Academic Forum2001 INCOSE Symposium

2 July 2001

2 July 2001 2

Education – What is it?

You can observe a lot by watching.

- Yogi Berra

When asked what single event was most helpful in developing

the Theory of Relativity, Albert Einstein replied,

"Figuring out how to think about the problem".- W. Edwards Deming

Education is that which remains when one has

forgotten everything he learned in school.

- Albert Einstein

2 July 2001 3

Topics

• Define the Problem– Concept of Operations for SE Practitioners– What Do Employers Want

• SE Practitioners To Do• How Well

• Describe the Current Situation• Outline an Improvement

– Courses To Be Taught– Teaching Methods To Be Used– Testing Methods To Be Used

• Summary

2 July 2001 4

Concept of Operations for SE Practitioners (SEPs) -

1

Define Problem

Candidate

enters

Aca

dem

ic

Sta

nd

ard

s

Pro

f . S

oc

Su

pp

liers

in s

itu

SE Education Environment

produces

Value Adder

SEP

designs andoperates

ValueCarrier

System

Sponsor

creates

Project

Employer

On-going SE

Non-SE

work

may operate

ValueRealizer

fundsstaffs

after Ring & Wymore

2 July 2001 5

Concept of Operations for SE Practitioners - 2

Define Problem

Candidate

enters

Aca

dem

ic

Sta

nd

ard

s

Pro

f . Soc

Su

pp

liers

in s

itu

SE Education Environment

produces

Value Adder

SEP

designs andoperates

ValueCarrier

System

Sponsor

createsProject

Employer

On-going SE

Non-SE

work

may operate

ValueRealizer

fundsstaffs

after Ring & Wymore

SEP Course Stage SEEE SEP SE Activity ProjectMOE MOE MOE MOEs MOEs MOEs MOEs

SE Artifact System MOEs MOEs

2 July 2001 6

What Do Employers Want:What Are the SEP MOEs?

• SE Practitioners To Do– Many Sources and Data

• Surveys– Might and Foster (NCOSE 1993)– Watts and Mar (INCOSE 1997)– ABET Survey of Industry (Lang et al., 1999)

• Presentations – – Boeing Recommendation (1994)– WMA Chapter Meeting, May 2000

• Historical Perspective (Krick, 1969)

• How Well– No Sources and Data!!

2 July 2001 7

Practitioner Reports• Bob McCaig

– Ability to define and solve problems

– Ability to communicate– Ability to learn on one’s

own– Ability to do trade studies– Ability to develop cost

estimates

• Jude Franklin– Ability to communicate

(write and speak)– Ability to work on a team– Ability to learn on one’s

own

• Bob Tufts– Ability to solve problems– Ability to recognize

problems– General understanding of

SE– Detailed training in 1+

technical areas– Ability to do trade studies– Ability to write– Ability to speak to an

audience– Understand the need for

the big picture

• Art Pyster– Ability to architect a

systemPresent in 2+ lists

Employer Wants

2 July 2001 8

INCOSE Reported Surveys• Might and Foster

– Requirements development– General SE process– Requirements management– Technical writing– System design methods– System architecture methods– Risk assessment– Concurrent engineering– Project management– SE tools– Test and evaluation– Simulation– SW engineering– Optimization techniques– Ethics– Communication networks– Probability– Computer architecture– Statistics– Total quality management– Database management– Reliability– Costing methods– Maintainability– Safety– Logistics– Manufacturing processes– Quality assurance– Finance– Marketing– Contract administration

• Watts and Mar– Basic problem solving– Development and management of

requirements– Teamwork and communication– System optimization (trade studies and

decision making)– System interface design– Mission analysis and design– System and component integration– Architecture development– Risk analysis and management– Systems engineering processes– Breadth of experience with different

systems– System simulation and modeling skills– Design techniques– Test and verification design and

management– Capture of the design data base– Commercial and military standards– Depth of knowledge in a specific system– Present and predicted technology– Project management processes– Engineering specialties (logistics,

maintainability, safety, etc.)– Tools and automation– Engineering economics– Human to machine and human to human

interface design

Employer Wants

2 July 2001 9

ABET Survey• Ability to apply knowledge of mathematics, science, and

engineering• Ability to design and conduct experiments, as well as to

analyze and interpret data• Ability to design a system, component, or process to meet

desired needs• Ability to function on multi-disciplinary teams• Ability to identify, formulate, and solve engineering

problems• Understanding of professional and ethical responsibility• Ability to communicate effectively• Broad education necessary to understand the impact of

engineering solutions in a global/societal context• Recognition of the need for, and an ability to engage in

life-long learning• Knowledge of contemporary issues• Ability to use the techniques, skills, and modern

engineering tools necessary for engineering practice

Employer Wants

2 July 2001 10

Summary of What Employers Want

• Ability to define and solve problems

• Ability to do trade studies

• Ability to communicate

• Ability to learn on one’s own

2 July 2001 11

Thoughts on Metrics for How Well

• Quality of ability …– Theoretical constructs that must preserved– Approximations and when they make sense– Indicators of inappropriate

approximations/application

• Cycle time– Key characteristic for systems engineering

• Recent industrial and academic recognition

– Therefore key characteristic for SE practitioners

• Relates to quality• Requires repetition in real world situations

2 July 2001 12

Current Situation• United States

– 34 Graduate Programs• Part time and full time students• Varying Degree Titles

– M.S.,M.E., Ph.D. Systems Engineering– M.S. Industrial and Systems Engineering – most common– M.S. Industrial Engineering with concentration in Systems Engineering

– 22 Undergraduate Programs• Most accredited by ABET (Accreditation Board of Engineering Technology)• Varying Degree Titles

– B.S. in Systems Engineering– B.S. in Industrial and Systems Engineering– B.S. in Systems Science and Engineering

• Europe– 6 Graduate Programs; all in England– 2 Undergraduate Programs; all in England

• Australia: 2 Graduate Programs• Asia/Mid-East: 3 Graduate Programs (Israel, South Korea, Viet

Nam) • America (non-US): 2 Graduate Programs (Brazil & Canada)• SE Embedded in Other Engineering Programs

(US – MIT; Europe – three)

2 July 2001 13

34 U.S. SE Graduate Programs• Air Force Institute of

Technology• University of Alabama,

Huntsville• University of Arizona• Auburn University• Case Western Reserve

University• Cornell University• University of Florida• George Mason University• George Washington University• University of Idaho• Iowa State University• Johns Hopkins University• Louisiana Tech University• Univ. of Maryland, College

Park• University of Memphis• University of Missouri-Rolla• National Technological

University

• New Jersey Inst. of Technology

• Oakland University• Ohio State University• Ohio University• University of Pennsylvania• Portland State University• Rensselear Polytechnic

University• University of Pittsburgh• Rutgers, The State University• San Jose State University• University of Southern

California• University of Southern

Colorado• Southern Methodist

University• University of South Florida• Stevens Institute of

Technology• University of Virginia• Virginia Polytechnic & State

Univ.

Current Situation

2 July 2001 14

Characterization of 34 U.S. Programs

Current Situation

Are the right courses being taught?

Core Courses | # Univ.

8 10 5 6 2 1 2

SE Design & Mgmt X X X

OR Methods X X X

Engineering Topics X X

Manufacturing X X

Special Topics X X X

Math X

2 July 2001 15

Further CharacterizationSE

Design & Mgmt

OR Methods

Manufac’g& SE +

SE Design & Mgmt

+

Current Situation

Avg = 3.1

Man

’gM

ath

2 July 2001 16

Outline of an Improvement

• Courses To Be Taught– Core Courses

• Key Systems Concepts

• Design and Architecture

• Management• Decision & Risk

Analysis

– Specialization Courses

– Project/Thesis Course

• Teaching Methods – Lecture/Test– Coached Project– On the Job Training

(OJT)– Electronic-distance

• Testing Methods – Project Solution– Problem Solution

2 July 2001 17

Teaching Methods• Lecture/Test

– Good for simple concepts and methods

– Not reasonable for teaching “how to” model

• Coached Project– Reinforces key

modeling concepts– Keeps students on

main course– Puts burden for

learning on students– Works great with

groups

• On the Job Training (OJT)– Experiential learning

is best – But

• Takes much longer• Few “educated” SEs

to serve as coaches• Just-in-time; Just-

enough => Too Shallow

• Electronic-distance– Jury is still out– Face-to-face

communication and feedback is critical

Improvement

2 July 2001 18

Teaching Systems Thinkingto College Freshman & IT

Minors• First year’s attempt was a failure

– Unable to keep students motivated– Project attempt (build web page)

• Succeeded in many great web pages• Failed to get concepts across

• Wanted project that would excite 18-22 year olds– DSMC has had success with Lego Mindstorms– Mindstorms robots require

• Use of hardware and software• Planning and experimentation/testing• Employment of SE concepts (whether good or bad)

2 July 2001 19

Course Concept• Teach key systems engineering concepts

with Lego robots as design lab– Initial lectures and experimentation with Lego

robot• Concepts: objectives, scenarios, inputs/outputs,

architectures• 3 week experimentation with ungraded trial runs• None of the robots completed either course

– Additional lectures and case studies• More on concepts• Case studies: Hubble failure, Black & Decker success• Morphological box developed of Lego robot

alternatives• 3 page paper on 3 alternate Lego robot designs

– 4 week design period with unlimited testing• More on concepts, especially objectives and

architectures

2 July 2001 20

Design Project1. % of distance on short course

– Best: 100; Worst: 0%– Score: percent traveled [ 0

100 ]– 3 minute time period.

If your robot gets stuck• distance will be measured • restart the obstacle course• repeated until end of 3

minutes• use the longest distance

2. % of distance on long course– Best: 100%; Worst: 0%– Score: percent traveled [ 0

100 ]– Note: same as for part 1.

3. Unit Cost of parts– Best: $0– Worst: $12,500– Score: 100 ($11,109 – Unit

Cost) / $11,109

• Relative weights of objectives:– % distance on long course:

0.4– % distance on short course:

0.4– Unit cost of parts: 0.2

• No modifications allowed to design for 2 courses except change of software program

2 July 2001 21

Obstacle Course• Short Course

– 2 Obstacles– Lights along outside offered

• Long Course– Two large turns– Light rope along center

offered

Lighted Path

Light

Barriers

Course 1

S F

Course 2

16.5 ft

22 ft9 ft

3 ft

1 ft

1 ft

Drawingnot to scale

2 July 2001 22

Summary of Results• Diverse set of designs

– Tracks and Wheels (2 & 3)

– Very limited use of sensors

• 2 of 10 groups used touch sensors

• No other sensors used– Reliance on software to

traverse courses open loop

– Very cheap to moderate cost

• Significant effort to reduce cost of designs– Dozen parts, $2278– Highest cost: $3839– Max Possible: $11,109

• Great variation in testing– 2 groups: none– 2 groups: 1 time, 10-75

min.– 3 groups: ~ 200 min.– 2 groups: ~ 350 min.– 1 group: 540 min.

• Results– Short course

• 6 groups max • 2 groups less than 95%

– 75%– 80%

– Long course• 6 groups max • 1 group less than 95%

(30%)

2 July 2001 23

2 Wheels

Bumpers

3 Wheels

Alternate Designs

Tracks

Wheels

Stabilizers

Front WheelDrive

2 July 2001 24

Testing Methods

• Project Solution– Critical for

representing real world complexity

– Builds tremendous confidence

– Great for group learning

– But one replication not sufficient for

• Mastering concepts

• Generalizing across diverse systems

• Problem Solution– Works for basic

concepts and methods

– Provides quick, uniform feedback to build modeling skills

– Useful less than 50% of the time

Improvement

2 July 2001 25

Why Racing Cars & SE? • Difficulty in Communicating What SE Is

– Communication needs to be specific, not abstract• Few domains exist with wide understanding• Race cars

– Too complex to be well understood– But all understand cars, or have access to someone who

does

• Difficulty in Learning about and Researching SE– “Success” of SE is difficult to define

• Success has many, varied dimension in most domains• Success in racing is very clear cut

– Definition of “good SE” is difficult• Long time constant between SE actions and success in

general• Race cars provide a domain with a very short time

constant

2 July 2001 26

Key Products Desired

• High Quality Video– Uses race cars as domain– Describes SE processes– Illustrates the value of SE

• SE Education Laboratory– Part of research facility– Provide site for educational field trips (high school &

college)– Provide case material for use in the classroom

• SE Research Facility– Established in conjunction with a racing team– Conducts research to improve

• SE knowledge across all domains using race car domain• Racing team knowledge using state-of-the-art SE

knowledge

2 July 2001 27

Education

is not the filling of a pail,

but the lighting of a fire.

William Butler Yeats

2 July 2001 28

Summary

• Education Problem Defined – Concept of Operations for SE Practitioners– What Do Employers Want (small sample)

• Ability to define and solve problems• Ability to communicate• Ability to learn on one’s own• Ability to do trade studies

• Current Situation – Chaos but Improving• Outline of an Improvement

– Courses To Be Taught – Focus on Engineering a System

– Teaching Methods – Lecture/Test & Project/Coaching– Testing Methods To Be Used – Emphasis on Projects

2 July 2001 29

References• Asbjornsen, O.A. and Hamann, R.J. (2000). “Toward a Unified Systems Engineering Education”, IEEE

Transactions on SMC (Part C), Vol. 30, No. 2, pp. 175-182.• Bots, P.W.G. and Thissen, W.A.H. (2000). “Negotiating Knowledge in Systems Engineering Curriculum

Design: Shaping the Present While Struggling with the Past”, IEEE Transactions on SMC (Part C), Vol. 30, No. 2, pp. 197-203.

• Brown, D.E. and Scherer, W.T. (2000). “A Comparison of Systems Engineering Programs in the United States”, IEEE Transactions on SMC (Part C), Vol. 30, No. 2, pp. 204-212.

• Franklin, J. (2000). “Systems Engineering Education Requirements” Presentation to INCOSE WMA Chapter.

• Krick, E.V. (1965). Engineering and Engineering Design, Wiley, NY.• Lang, J.D., Cruse, S., McVey, F.D. and McMasters, J. (1999). “Industry Expectations of New Engineers: A

Survey to Assist Curriculum Designers”, Journal of Engineering Education, January, pp. 43-51.• McCaig, R. (2000). “Engineering and Academia” Presentation to INCOSE WMA Chapter.• Might, R. and Foster, R. (1993). “Educating System Engineers: What Industry Needs and Expects

Universities or Training Programs to Teach” in the 1993 NCOSE Symposium Proceedings, Arlington, VA, July, 1993.

• Prados, J.W. (1996). “Educating Engineers for the 21st Century: New Challenges, New Models, New Partnerships”, Academic Forum at 1996 INCOSE Symposium.

• Pyster, A. (2000). “Systems Engineering Education Requirements” Presentation to INCOSE WMA Chapter.

• Ring, J. and Wymore, A.W. (2000). “Overview of a CONOPS for an SE Education Community”. in the 2000 INCOSE Symposium Proceedings, Minneapolis, July 2000.

• Sage, A.P. (2000). “Systems Engineering Education”, IEEE Transactions on SMC (Part C), Vol. 30, No. 2, pp. 164-174.

• Sage, A.P. and Armstrong, J.E. Jr. (2000). Introduction to Systems Engineering, Wiley, NY.• Tufts, R. (2000). “What Kind of Skills Is Industry Looking for from Academia?” Presentation to INCOSE

WMA Chapter.• Van Peppen, A. and Ruijgh-van der Ploeg, M. (2000). “Practicing What We Teach: Quality Management

of Systems-Engineering Education”, IEEE Transactions on SMC (Part C), Vol. 30, No. 2, pp. 189-196.• Watts, J.G. and Mar, B.W. (1997). “Important Skills and Knowledge to Include in Corporate Systems

Engineering Training Programs”. in the 1997 INCOSE Symposium Proceedings, Los Angeles, CA, Aug. 1997.