teamwork in engineering as a socio-technical system - mit...

Bryan R. Moser, 2016

system design andmanagement

MITsdmSystems Thinking Conference

2016

Teamwork in Engineering as a Socio-technical SystemExperiments and Analytics for Performance in Complex Projects

Bryan R. Moser28 September 2016

Leadership, Innovation, Systems Thinking

Overall PM Learning Objective

• Students are able to strategically plan, budget and manage product and service development projects and programs in a larger organizational and business context.

2


Do the Job RightDo the Right Job

High level Framework

PrepareSet targets & request

Plan Perform

Deliver

Monitor

Corporate Strategy & Standards

3

Adapt




2016

Classic Methods

4


Classic Methods Rooted a Century in the Past

“The theory of the proper execution of work is that it should be planned completely before a single move is made, that a route-sheet which will show the names and order of all the operations which are to be performed should be made out and that instruction cards should be clearly written for each operation….

By this means the order and assignment of all work, or routing as it is called, should be conducted by the central planning or routing department. This brings the control of all operations in the plant, the progress and order of the work, back to the central point.”

Henry P. Kendall [Tuck 1912]

5


PM “Standards”

1. PMI (1969) and PMBOK (1983+)

2. PRINCE, PRINCE2 (1989+)

3. IPMA (1965) and ICB (1999)

4. PMAJ (1998) and P2M (2001)

5. ISO 10006:2003 Quality in Project Management

Similar range of standards from systems engineering organizations

6

1. Starting up

2. Directing

3. Initiating

4. Controlling a stage

5. Managing stage boundaries

6. Closing

1

2

4


Project Management & Systems Engineering

Industrial Machinery Project

• 60,000 line Gantt Chart

– Complete system re-design of a critical global platform

– 8 person office (PMO) to create and manage the Gantt chart

– Awards given to PM team

Yet, Execs and Team Leaders

– Little awareness of architecture

– Unable to see own role in context

– Didn’t believe the schedule forecast

Spaceflight System Project

• Project Management

– Plans integrated, Checklists followed

– Earned Value calculated

– Gateways formalized

• Systems Engineering

– Requirements mapped, Scope defined

– Dependencies and Work Packages defined

Even with all boxes checked…

– $9B over 6 years

– NASA shuttle replacement cancelled

7

Chart and Reality Diverge“No Feeling for the

Dependencies”




2016

Engineering Projects as

Sociotechnical Systems

8

Bryan R. Moser, 2016MITsdm

Trend: From Practices to Dynamics

• Classic project management was born through practices

and standards, evolved over decades, and often reflective

of significant embedded know-how.

• The underlying dynamics – the drivers of performance -- are

often assumed or hidden.

.

9


System Definition

• A System is a set of physical or virtual objects whose

interrelationships enable desired function(s).

– more than the sum of its parts

– Undesired (emergent) functions often exist

– System complexity scales with the number of objects as well

as the type and number of interconnections between them

– …

10


Projects are Socio-Technical Systems

Socio = Project Teams in Organizations with values, behavior,

skills, structure, priorities, capacities, skills, and costs

Technical = Projects Outcomes are product systems, with

architecture, interfaces, materials, information, services, …

• Teams and Products are tied together by roles on activities

and complex dependence

• Behaviors of teams and the demand for outcomes combine,

constrained by limits and dependencies,

in often surprising ways

11


Humans in the Loop

Engineering viewed as a socio-technical system if we

include “Humans in the Loop”

• People do work, process information, and interact as part

of an organization

• Individuals allocate attention based on behaviors within

limited capacity. People make mistakes. People learn.

• Organizations with architecture exhibit emergent behavior

(e.g. exception handling, quality…)

Slide 12


A new class of project methods/ tools

for systems thinking

• Integrates systems view of product, process, and organization

• Forecasts surprising, likely, and emergent outcomes– Shows potential outcomes even if undesirable

• Allows participants to explore the trade space. – Exploration takes time.

• Acts as a Social Instrument: Promotes convergence by a cross-functional team on a shared, desirable and feasible plan

13




2016

Research Mechanisms

July

2016

14


New in 2015: Global Teamwork Lab w/ U Tokyo, MIT

15

• Global Teamwork Lab (GTL) is a new group to promote

global capability and multidisciplinary research results for

students, faculty and industry

Kashiwa-no-ha Smart City 柏の葉

GTL Research

• Our research focuses on the underlying mechanisms and dynamics of performance under complexity.

• Teams, their problems, and their environment are instrumented to reveal phenomena in real-time: demands, behaviors, activities, interactions, and outcomes across social and technical boundaries.

• Data-driven experiments are matched with modeling, simulation, systemic analytics, and interactive visualization.

• These methods are developed, tested, and deployed for practical use by our joint industry-university teams.

• 私共は、複雑化した環境において、チームとしての根本的な振舞いの理解し、ダイナミックな能力を発揮する方法を探求します。

• チームがどの様に行動するのか、その環境がどの様に変化するのかをリアルタイムで計測します。例えば、要求、作業、相互作用および成果などが随時記録されます。そして社会的かつ技術面の境界を越えた成果を生み出します。

• データを中心した研究は、モデリング、シミュレーション、系統的分析と視覚化された相互関係図によって統合されます。

• 開発、テストされたこれら方法論は、私共の産学協同チームの今後の実践に活かすために用いられます。

16

Workshop-based Experiments

• What are the currently accepted dynamics of teamwork under complexity? Pick one of these “physics”.

• What do common standard and best practices suggest? Do they make sense?

• Design an experiment to observe these teamwork physics in real time.

• How to collect? Analyze? Visualize?

• Can performance be predicted? Tuned?

• What new mechanisms can we create to allow teams to self-organize and design their own projects?

July 2016 17

18July 2016

Infrastructure

Facilitators

Research

Stakeholders

Academic

Industry

Institutional

Physical Virtual

Participants




2016

Project Design Experiments

July

2016

19


Fundamentals: A Plan is a Design

• The plan, or design, of a

project…

– … integrates a system of

product, process, and

organization

– …is search and choice

towards a desirable and

feasible project

– …predicts the likely Cost,

Schedule, and Scope at

some Risk

20

Feasibility

Desirability

Project Design Tradespace & the “Design Walk”

21

Cost

Duration

Baseline project design scenarioEstimate of likely, realistic outcomes

A new design for the project:Architecture, scope, roles, behaviors,…

Starting Point: City Car Project

22July 2016

Experiment in Process

23

24

What drives teams to better project design?

July 2016 20 teams common starting point, 2 hours

1,482 simulations 316 Scenarios in 2 hours

25

• Each dot is a

feasible project

scenario, yet

perhaps not valuable

• Common starting

point for 20 teams:

$10.1M, 872 days

• Each change is a

test and insight:

which design

changes to the

project as system

are acceptable and

valuable?CPM forecasts

for baseline scenario

Baseline common start for 20 teams

?

July 2016

26July 2016

27

Selected Scenarios & Pathway Patterns

July 2016




2016

Teamwork during Concept

Development

U Tokyo i.School,

Prof. Hideyuki Horii

28


Uniqueness of i.school②

• Innovation science

• Innovation workshop itself is the

subject to study

• Cognitive science, Organizational

behaviors, Knowledge engineering,

Pedagogy

• Results of studies are utilized to

design better innovation workshop 29


2016/3/2


Ideation & Knowledge Structuring

31

Courtesy U Tokyo

i.School


Our partners from

the U Tokyo i.School

run workshops for

upstream concept

generation

Bryan R. Moser, 2016-32-

Investigation of the thinking process in idea

generation

Thinking process in the idea generation task can be identified with

analysis of APISNOTE record and interview survey.

– Ideation process shown in APISNOTE Eunyoung (2015)


Mechanisms for Novelty

1. Understanding others

2. Foresight

3. Clarifying concepts

4. Shifting cognitive pattern

5. Shifting value system

6. Finding new combination

7. Analogical thinking

8. New objective from unexpected use

9. Table flipping33

Courtesy U Tokyo

i.School


Bryan R. Moser, 2016-34-

In the interview, each participant indicated the note that makes creative leap.

Based on the time record in the APISNOTE, each process is coded as follows:

Participants who generated an appropriate idea had deliberation before

reaching the creative leap.

Deliberation before reaching the creative leap stage

1D

1A

1B

1C

1E

Appropriate idea generation

OthersSource retrieval Domain setting Domain refining Mechanism Creative leap Title

Courtesy U Tokyo

i.School



2016/3/2

Courtesy U Tokyo

i.School



Happiness Counter

36

Courtesy U Tokyo i.School





2016

Current Experiment: Attention

and Awareness of Dependencies

37


Dependency Management (“Coordination”)

Causes(Sources of need)

Systemic

Effects

Dependence Satisfaction(None… partial…. Complete)

Pool: Shared Resources

Flow: Results & Info

Product

Process

Organization

Demand(s) to interact

InteractionAwareness& attention

Local

Effect

Timeliness

Quality

Cost

Volume

38

Mechanisms for Satisfaction of Dependence

Bryan Moser, William Grossmann and Phillip Starke, “Mechanisms of Dependence in Engineering

Projects as Sociotechnical Systems”, Advances in Transdisciplinary Engineering, 2015

Experiments in TeamPort | Carl Fruehling | September, 28th 2016© 2016 Prof. Lindemann 39

Product Development Technical University of Munich

We want to track the participants’ attention allocation during

an project design exercise

Research Execution

First, we find patterns in the groups’ behavior by analyzing the sequences of their actions and their model evolution - then, we refer those patterns to their performance

Data Analysis

Action Sequences

12

3

45

6

Model Evolution

12

3

4 5 6

Pre-Survey Demographics

Post-Survey

Fingerprints

Design Walk

Change Log

Pre-Briefing

Post-Briefing

2

4

3

Comprehension5

Tradespace

t

$

t

$

4

High performance

Low performance

Design of Experiment

1

Tutorial

Sample Case

Result Comparison

Feedback

Dependencies

T = Treatment group

C = Control group

2

3Group 1

Group n

…

T

C

The treatment group first has to work on the

structure of the model before it can start

Exercise

T C

Changes

Start

Multiple variables are

analyzed for correlation

Experiments in TeamPort | Carl Fruehling | September, 28th 2016© 2016 Prof. Lindemann 40

Product Development Technical University of Munich

The purpose of our experiment is to focus on the participants

attention allocation towards activity dependencies

Research Questions

To which elements do high performing project design groups allocate

their attention?

Does attention allocation towards key dependencies lead to a higher

designing performance?

Does the focus on the project model structure improve the designing

performance of a project design group?

Through which events do project design groups become aware of

activity dependencies?

Which other action patterns do high performing project design groups

follow during the design process?




2016

Research Strategy & Agenda

Meso-Scale: 7 to 7x7x7 people

• Much research exists at the “micro-scale” for teams, examining the interplay of individuals, their skills, personalities, and biases as part of a small team.

• Emerging research at the “macro-scale” is using “big data” to draw conclusions at the population level.

• Our work at GTL focusses at the meso-scale, “middle out”, the team of teams.

• These size teams correspond to the most common scale and scope of influence by working teams in the field.

• We are open to phenomena not yet characterized

July 2016 42

Sub-atomic Physics of Tasks

• If “tasks” are the atomic particle of project management…

• Let’s reveal the underlying characteristics of tasks, and how they interact dynamically with the sociotechnical environment

• To better understand and predict likely performance.

• the nature of work (the task)

• the nature of behaviors (teams and resources) and

• how they interact (project architecture and dynamics)

43July 2016

Scale and Pace of Research

• Traditional work has proceeded at the pace of a social science PhD

• Deep ethnographic case studies

• Survey-based self-report

• “toy problems”

• Rigorous or not, by form often limited in repeatabilityand scalability.

• Themes and heuristics echo across papers with very little original empirical study.

• GTL looks to build research as platform, so that we can connect to teams, roll-out experiments, observe, towards 10x rapid and 100x scalable experiments.

July 2016 44

Premise of Commonality across Domains

• Successful work cultures evolve to align architectures and behaviors for effective, sustainable performance.

• We respect traditions of teamwork that perform well, even if the basis for outcomes is not explicit.

• We can learn from these traditions, across domains.

• By seeking the underlying mechanisms to meso-scale sociotechnical systems:

• We should see commonality across types of teams and domains

• The shadows from research at the micro and macro scales should make sense, if not inform, the meso-scale mechanisms.

• We will be able to predict and provide teams with real-time adaptive tools and thinking leading to great performance.

July 2016 45




2016

Conclusion

Thank you!

[email protected]

46


AbstractTeamwork in Engineering as a Socio-technical System: Experiments and Analytics for Performance in Complex Projects

• This presentation will introduce recent research on the underlying mechanisms and dynamics of performance under conditions of complexity. Teams and their environments are instrumented to reveal phenomena in real time: demands, behaviors, activities, interactions, and outcomes across social and technical boundaries. Data-driven experiments are matched with modeling, simulation, systemic analytics, and interactive visualization. These methods are developed, tested, and deployed for practical use by joint industry-university teams.

• Engineering projects are viewed as socio-technical systems if we include not only the delivered systems but also the developers as “humans in the loop.” People do work, make mistakes, process information, learn, and interact as part of organizations. They also allocate attention based on embedded behaviors within their individual capacities. Organizations with architecture and culture exhibit emergent behaviors (e.g. exception handling, quality, etc.).

• Much team research exists at the micro scale—examining the interplay of individuals, their skills, personalities, and biases as part of small teams. Emerging research at the macro scale is using big data to draw conclusions at the population level. This presentation details work focused on the meso scale—the team of teams working on systems of systems. This layer of focus corresponds to the most common scale and scope of influence encountered by teams working on complex engineering projects.

47

teamwork in engineering as a socio-technical system - mit...

Documents