2003 Fall SIW

The Process for Coercing Simulations

Sarah Waziruddin, University of VirginiaDavid Brogan, University of VirginiaPaul Reynolds, University of Virginia

2003 Fall SIW

Motivation

• Facilitate the reuse process

• Simulation reuse is highly desirable

• Reuse may occur:

• Modify a simulation to achieve a different set of objectives

• Perhaps change resolutions

• Compose simulations to meet an arbitrary objective

2003 Fall SIW

User Composability Concept of Operations

CONSTRUCTION ANDEVALUATION

Options for Composed Models

Module Repositories

USER SEARCH

Issue:1) Interoperability2) Constraints

“I have a

need

“I have a solution!”

Selected modules

REQUIREMENTSCAPTURE

Issue:What should these modules contain

Issue:How do requirements relate to module functionality

(Translate user requirements into

model requirements)

Ernest Page, “Observations on the Complexity of Composable Simulation” In Proceedings of Winter Simulation Conference, December 1999 in Phoenix, AZ

If interoperability of selected modulesis an issue, coercionoffers a potentialsolution.

2003 Fall SIW

The Goals of Coercion

• Increase efficiency of the reuse process

• Provide tools that lead to new insight about the simulated phenomenon

• Increase the practitioners' confidence in the model

2003 Fall SIW

P: The Coercion Process

P

Requirements

CoercedSimulation

Simulation to reuse

Practitioners

2003 Fall SIW

An Example

Bicyclist: Slower and More Accurate Model

P Coerced Particle: Relatively Fast and

Accurate Model

Particle: Faster and Less Accurate Model

TrajectoryBicyclistParticle

2003 Fall SIW

P: The Coercion Process

P = [m*, o*]*

Some Example Sequences:• m, o, m, o• o, o, m, m, o, m• m • o

2003 Fall SIW

P: The Coercion Process Explained

Identify Flexible Elements

Set Constraints

Define ObjectiveFunction

Select Optimization Technique

Run Optimizer

Identify Functionality to Modify

Make Modification

Done

NoYes

Successful Coercion?

Run optimizer or modify?

Evaluate

Modify

Optimize

Identify Flexible Elements

Set Constraints

Define ObjectiveFunction

Select Optimization Technique

Run Optimizer

Identify Functionality to Modify

Make Modification

Done

NoYes

om

Successful Coercion?

Run optimizer or modify?

2003 Fall SIW

O: Optimization Step

Need a human in the loop for the initial tasks

Utilize optimization technology to automate coercion

Identify Flexible Elements

Set Constraints

Define ObjectiveFunction

Select Optimization Technique

Run Optimizer

2003 Fall SIW

M: Modification Step

• Modify or augment the existing algorithm of the simulation

Identify Functionality to Modify

Make Modification

2003 Fall SIW

Evaluation

• The practitioner determines if coerced simulation meets requirement

• The initial simulation’s output is encoded as a behavior: S0B0

• The coerced simulation’s output is encoded as a behavior: SnBn

• The requirement is encoded as a behavior, Btarget

Successful Coercion?

2003 Fall SIW

Evaluation Function

• Btarget Bn

• An infinite Bn can satisfy this relationship

Btarget

BnBn

Btarget

BnBn

2003 Fall SIW

Paths to Coercion

S0

S0

.

.

S0

SnBn

Sn

Sn

.

.

2003 Fall SIW

Possible Scenarios

O O• The optimizer fails to

converge

• The optimizer gets stuck in a local minima

• Discontinuity, non-linear, large search space

O M• When successive

optimizations fails to yield successful coercion

• Modification may be needed to eliminate non-linearities, discontinuities, etc.

2003 Fall SIW

Possible Scenarios

M O• Modification may lead to

discovery of new flexible elements

• The optimal values for the flexible elements may change due to the modification

M M• Optimization is too

costly/impossible

2003 Fall SIW

An Example

• S0 = f(x) = y

= -0.25x + 0.5

• Requirement = g(x) = y

= sine(x)

• Define Btarget by specifying 16 sample points:

15,,1,0;

4nnx

2003 Fall SIW

An Example

2003 Fall SIW

O O M O M O

• First Iteration: Optimization

• Motivation: We want as much automation as possible

• Initial tasks:

• Identify flexible elements: The slope and y-intercept

• Set constraints: None

• Define the objective function:

• Choose an optimization technique: Linear regression

errors

2003 Fall SIW

O O M O M O

2003 Fall SIW

O O M O M O

• Flexible elements: Slope and y-intercept have values of 0

• Objective function: Value of 0 (no error) but Btarget has not been met

• Insight: Individual errors sum to 0

four have an error of zero, four have an error of , four have an error of , two have an error of one, and two have an error of negative one

• Next Iteration: Modify objective function

22

22

2003 Fall SIW

O O M O M O

• Second Iteration: Optimization

• Motivation: Modify objective function to leverage automation

• Initial tasks:

• Identify flexible elements: The slope and y-intercept

• Set constraints: None

• Redefine the objective function:

• Choose an optimization technique: Linear regression

error

2003 Fall SIW

O O M O M O

2003 Fall SIW

O O M O M O

• Flexible elements: Slope and y-intercept have values of 0

• Objective function: Value of

• Insight: Line fragments track sine curve better than a line

• Next Iteration: Modify the simulation to produce line fragments

424

2003 Fall SIW

O O M O M O

• Third Iteration: Modification

• Functionality to modify: Produce line fragments that span a parameter space of π

• Fourth Iteration: Optimization

• Motivation: Find optimal values for flexible elements in modified algorithm

• Initial tasks:

• Same as the second iteration of optimization

2003 Fall SIW

O O M O M O

• Line fragments do not match sample points defined by Btarget

• Objective function: Error value decreases from previous iteration

• Insight: Smaller line fragments track sine curve better

• Next Iteration: Modify the simulation to produce twice as many line fragments

2003 Fall SIW

O O M O M O

• Fifth Iteration: Modification

• Functionality to modify: Produce line fragments that span a parameter space of π/2

• Sixth Iteration: Optimization

• Motivation: Find optimal values for flexible elements in modified algorithm

• Initial tasks:

• Same as the second iteration of optimization

2003 Fall SIW

O O M O M O

2003 Fall SIW

O O M O M O

• Line fragments match sample points defined by Btarget

• Objective function: Error value is 0

• Practitioner verifies results using visualization tools

• Btarget

Bn; Coercion is successful!

• Multiple sequences can succesfully coerce this simulation to B

target

2003 Fall SIW

Tools available

• SimEx: a tool in production to aid practitioners to coerce a simulation

• Includes visualization, optimization and constraint specification modules

2003 Fall SIW

Ongoing Work

• Is there a best coercion sequence?

• Is there a best initial simulation?

• Is there a best coerced simulation?

• Tools:

• Maximize human interaction in the process

• Identify flexible elements

• Visualization tools

• Optimization tools

2003 Fall SIW

Conclusion

Coercion• Formally specified a semi-automated reuse process

• Greater insight on simulation code

• Greater insight on simulated phenomenon

• Insight leads to greater confidence in model

2003 Fall SIW

Acknowledgment

• Defense Modeling and Simulation Office

• Multi-Resolution Modeling group at the University of Virginia

2003 Fall SIW

Questions

?