varieties of system dynamics practice alan k. graham, phd presented at the international system...

Varieties of System Dynamics PracticeAlan K. Graham, PhD

Presented at the International System Dynamics Conference in Washington, D.C.

July 25 – 29, 2011

What is system dynamics? What is not system dynamics?What is it that we have in common?

• Defined by foundations? The theory of information-feedback systems A knowledge of decision-making processes The experimental model approach to complex systems The digital computer as a means to simulate realistic mathematical models

• -> Broad and indecisive• Defined by current practices?

10 steps 25 validation categories 33 questions model users should ask 982 pages of just one of many textbooks

• -> Whose practices? Complex and unchangeable • Shouldn’t the definition include purpose—the uses for which SD models are intended?

· © Greenwood Strategic Advisors AG

A simple dynamic hypothesis to account for non-explosive growth in a field


Complexity of SD Presentation: 10 Steps 25 Validation categories 33 Questions model users should ask 982 pages of textbook

Pressure on instructorsto cover material quickly

Effectiveness inand throughnessin teaching SD

- -

Number of qualifiedSD learners

Number and capabilitylevel of SD practitioners

+

+

Exposure of potentialclient users to SD at

university+

Execution of appliedSD studies +

+

Ability to explain anddifferentiate SD to potential

client users

-

+

Body of realapplications

+

+

+

Known precedentsrelevant to potential

client users

+

+

+

A model of SD modeling processes to show variations


Step:

1 2 3 4. Validation of 5 6 7

Validation of Problem System System Recommendations Recommendations

Problem System Recommendations Structure Behavior (by modelers) (by experts)

Analysis and Use Requirements Diagram, model “will and won’t”

Block diagram,

Causal diagram

Causal tracing and scoring, interpretation

Equations and parameters

Modelers’ rough expectations in testingComparison of simulated to observed behavior*

Rough expectations for behavior in policy testing

Expert expectations for behavior in policy testing

Kickoff meetings with stakeholders validate purpose of modeling*

Boundary adequacy Structure assessment: Consistency with known facts Level of aggregation consistent w purpose & facts Conservation laws represented Decisions mappable to specific actors or groups? Expert review of structure & key assumptions

Structure assessmentDimensional consistencyParameters have real-world counterparts & valuesResponse to extreme conditions

CalibrationInput / outputExtreme conditionsBehavior sensitivityChallenge by modelers and experts* of behavioral hypotheses (“model of the model”)

System improvementPolicy combinationPolicy sensitivityChallenge improvement hypotheses (“model of policy impact”)*Fit-constrained parameter Monte Carlo test of improvement*

Expert review of analysis summary and “model of the model”*

Getti

ng

fact

sCr

eatin

ghy

poth

eses

Form

at o

f hy

poth

eses

Client issues and needs Client issues and needs

Modeling purpose and scope


Policy impactscoringby experts


Preliminary recommendations, scope & focus


Quantitative (and additional qualitative) information-gathering Quantitative (and additional qualitative) information-gathering

Quantitative model structure


Quantitative model behavior


Quantitative technical impact analysis


Analysis, review and challenge by experts


Recommend- ations

Recommend- ations

Typi

cal v

alid

ation

te

sts

Qualitative Information- gathering


Causal diagram (qual. model)


Suppose we don’t focus on specific hypothesis formats and tests…


Step:




Getti

ng

fact

sCr

eatin

ghy

poth

eses

















Recommend- ations

Recommend- ations





… and don’t worry too much about specific steps, and moreover aggregate system (structure + behavior) and recommendations (modelers + experts). Then…


Step:




Getti

ng

fact

sCr

eatin

ghy

poth

eses

))

…even complex modeling can be described as testing only three kinds of hypothesis:

Step:

1 2 3 4. Validation of

5 6 7

Validation of Problem System System Recommenda-tions

Recommenda-tions

Problem System Recommenda-tions


In system dynamics, we test hypotheses about only three things:• The problem to be addressed• The system it happens in• The recommendations to address the problem

One format for testing the first type of hypothesis: Our understanding of the problems to be addressed


4. …to find policy levers that are more robust w.r.t. outside conditions and deliver better 5-year margin contribution—higher average and less variation

ArbCo wants a better-peforming marketing allocation. We will use a simulator to:

Page 8 · © Greenwood Strategic Advisors AG

2. Simulate the direct and indirect impacts in the multiple marketplaces

Complexity of SD Presentation:10 Steps25 Validation categories33 Questions model users should ask982 pages of textbook

Pressure on instructorsto cover material quickly

Effectiveness inand throughnessin teaching SD

- -

Number of qualifiedSD learners

Number and experiencelevel of SD practitioners

+

+

Exposure of potentialclient users to SD at

university+

Volume of appliedSD studies +

+

Ability to explain anddifferentiate SD to potential

client users

-

+

Body of realapplications

+

+

Known precedentsrelevant to potential

client users

+

+

1. Experiment with ArbCo’s “policy levers”:

• Allocation among activities A, B, C and D

• Allocation to early / mid / late product cycle products

• Different forms of joint selling arrangements

3. Under each combination of events outside of Arbco’s control, for

• Economic downturn

• Reregulation of markets

• Others?

Why, given that our products are demonstrably more cost-effective than competitors, we don’t have a higher market share?

The time horizon for the study is 5 years

The study will cover product lines E, F, G and H

The study will be complete in 8 months, and require the specified collaboration of ArbCo experts

We test our understanding of the second hypothesis, the relevant system, mostly by standard SD tests

• For example:– Boundary adequacy– Extreme conditions– Behavior reproduction– Behavior sensitivity

• For quantitative systems thinking (more about this shortly), the model is a causal diagram with qualitative or quantitative characterizations of each link. Tests are review and discussion with authoritatively-experienced informants:– Does the “voice-over” description of a given link adequately describe what goes on in real life?– Do the characterizations (scoring, time delays) make sense alone and in comparison to

characterizations of other links?– (These are ALWAYS done on buildups—not the whole diagram at once)


The third hypothesis is “we understand what recommendations are needed to improve the system’s performance”

• Some testing of policies is done by modelers as part of policy design:– Policy sensitivity analysis– Extreme conditions / scenario analysis

• Some testing is sometimes done of the modeler’s understanding of why a given set of policies is effective– Starts with a simpler diagram of how policies create their good effects– Implication: severing links on that diagram should reduce effectiveness of policies– Simulation test: Does severing those links in the model reduce effectiveness of policies?

• Some testing of policies is done by subject matter experts– Is the explanation of policy impact consistent with their knowledge of cause and effect in the

system?– Are the changes in system behavior plausible?


I sometimes hear the argument that separate testing of policy outcomes is unecessary; that if the model structure and behavior have been validated, the policy conclusions should be correct. But there’s no such thing as perfect validation. When there are the inevitable time and resource constraints, doing policy testing is a way of focusing time and resources on what matters most to the usefulness of the effort.

If we want to address in the where, when, and how aspects of system dynamics, we can add one or two “bookends” to the three-hypotheses description:


System dynamics is especially useful in complex problems where causes and effects are intertwined, the implications of actions are unclear, and the concern is behavior over time, often aiming to improve future behavior.

To do that reliably, we reality-check that we understand three things:

•The problem(s) to be addressed•The system they happen in•The recommendations to address the problem(s)

We usually use computer simulation to check the second and third hypotheses against both numerical data and expert knowledge of the system.

Our hypotheses draw on a wide body of research about dynamic behavior (from feedback control theory) and about decision-making in many spheres of human

activity, and in particular, on the extant body of system dynamics research.

.

With that definition in hand, let us examine varieties of system dynamics practice—and some activities we’d probably say aren’t “real” SD


Single relationship

Internal detail, value chain, other organizations, etc.

(nar

row

) Sc

ope

of p

urpo

se (

broa

d)

(low) Scope of validation (high)

Little infor-mation used

Follows scientific method extensively

Just

usi

ng th

e so

ftw

are

Exercise models

“Industrial Strength”

SD“Legal

strength” SD“Classic”

SDQuanti-tative

STUns

uppo

rted

syst

ems

thin

king

Varieties of SD practice differ in prominent ways. These are typical:


ProblemDynamicHypothesis Information

System & Recommendations Testing

Quantitative systems thinking (QST)

Explicit Many, with uncertainty

Expert cause & effect knowledge with scoring

Sensitivity testing, focused expert review

Classic SD Implicit One Mostly cause & effect – little quantitative info.

Within modeler

Industrial strength SD

Explicit Multiple competing

Quantitative & expert cause & effect knowledge

Focused, with experts

Legal strength SD Impact quanti-fication

Multiple (adversarial) theories of the case

Quantitative & expert cause & effect knowledge

Extensive, with experts.Confidence bounding.Third party review.

There are two contributions here. First, for modelers, there is a taxonomy of SD practice that describes what is otherwise quite complex and detailed…


Step:




Analysis and Use Requirements Diagram, model “will and won’t”

Block diagram,

Causal diagram

Causal tracing and scoring, interpretation

Equations and parameters

Modelers’ rough expectations in testingComparison of simulated to observed behavior*

Rough expectations for behavior in policy testing

Expert expectations for behavior in policy testing

Kickoff meetings with stakeholders validate purpose of modeling*

Boundary adequacy Structure assessment: Consistency with known facts Level of aggregation consistent w purpose & facts Conservation laws represented Decisions mappable to specific actors or groups? Expert review of structure & key assumptions

Structure assessmentDimensional consistencyParameters have real-world counterparts & valuesResponse to extreme conditions

CalibrationInput / outputExtreme conditionsBehavior sensitivityChallenge by modelers and experts* of behavioral hypotheses (“model of the model”)

System improvementPolicy combinationPolicy sensitivityChallenge improvement hypotheses (“model of policy impact”)*Fit-constrained parameter Monte Carlo test of improvement*

Expert review of analysis summary and “model of the model”*

Getti

ng

fact

sCr

eatin

ghy

poth

eses

Form

at o

f hy

poth

eses

















Recommend- ations

Recommend- ations

Typi

cal v

alid

ation

te

sts





…in terms of a simpler menu of varieties of system dynamics


Single relationship

Internal detail, value chain, other organizations, etc.

(nar

row

) Sc

ope

of p

urpo

se (

broa

d)

(low) Scope of validation (high)

Little infor-mation used

Follows scientific method extensively

“Industrial Strength”

SD“Legal

strength” SD“Classic”

SDQuanti-tative

ST

System dynamics addresses complex, intertwined issues. To do that reliably, we reality-check that we understand three things:

•The problem(s) to be addressed•The system they happen in•The recommendations to address the problem(s)

We use computer simulation and lots of information and data about how people and organizations interact to do the reality-checks.

Second, for both modelers and nonmodelers, some simple yet properly encompassing definitions of system dynamics, e.g.:


This is a framework that is useful with nearly anyone. To name a few we’d like to communicate more effectively with: fellow academics, potential clients, students, parents and spouses.

Questions, comments, expansions?

Alan GrahamGreenwood Strategic Advisors AGZugerstrasse 40CH-6314 Unterägeri, ZGSwitzerland Alan.Graham@ Greenwood-AG.comOffice Tel.: +41 41 754 7447Office Fax: +41 41 754 7448Home Office: +1 781 862 0866mobile: +1 617 803 6757www.greenwood-ag.com


http://www.greenwood-ag.com/

varieties of system dynamics practice alan k. graham, phd presented at the international system...

Documents