![Page 1: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/1.jpg)
1
SIGMA WorkshopPart 1: Pattern-oriented Testing and AnalysisGönenç Yücel, Can SücüllüSESDYN Research GroupBoğaziçi University, Istanbul
![Page 2: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/2.jpg)
Similarity of dynamic behaviors• Comparing/evaluating model outcomes
• Calibration• Validation• Behavior analysis• Scenario and policy analysis
• Similarity in terms of what?• Pattern characteristics
![Page 3: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/3.jpg)
Problem identification/description◦ Oscillating inventory◦ Exponentially growing resource consumption◦ Declining sales
Model calibration and behavior validation
Sensitivity analysis◦ Numeric sensitivity vs. behavior sensitivity
Policy analysis
Thinking in patterns
Page 10.00 25.00 50.00 75.00 100.00
Time
1: Noname 4
Page 10.00 25.00 50.00 75.00 100.00
Time
1: Noname 4
Page 10.00 25.00 50.00 75.00 100.00
Time
1: Noname 4
![Page 4: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/4.jpg)
4
![Page 5: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/5.jpg)
Patternwise Comparison
23.07.2014 BATS - Can Sücüllü & Gönenç Yücel - ISDC 2014 Delft
5
![Page 6: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/6.jpg)
Behavior Analysis and Testing Software (BATS)
![Page 7: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/7.jpg)
Overview of BATS with External Parties
7
BATS
USER
Excel
View theResults
Control theSoftware
Parse Commands
ReceiveSimulation results
ImportData-series
ExportData-series
![Page 8: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/8.jpg)
8
• Excel• txt, csv, tab• Drawn by the user• Vensim results files (.vdf)• Vensim models through Model Docking Window
Data Importing Methods
![Page 9: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/9.jpg)
Set parameter values
Model Docking Window of BATS
Send simulation results to Workspace in order to further analyze the model output using data analysis features
Send commands; Run
Exporting of simulation results from Vensim models
![Page 10: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/10.jpg)
Overview of Features of BATS
Data ImportingLoad From FileModel Docking WindowDraw
Data PreparationSplitSelectExponential SmoothingMoving Average< Trend >
Data AnalysisClassifyTrendAutocorrelationAutocorrelation TestSpectral DensityAmplitude EstimationCrosscorrelationSummary StatsGraphical Comparison
Model AnalysisHypothesis TesterBehavior Space ClassifierBehavior Class Mapper
Data VisualizationPlot
*
**
**
![Page 11: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/11.jpg)
Graphical Comparison
11
![Page 12: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/12.jpg)
BATS Hypothesis Tester
Extreme conditions are imposed
parameter values
Model is run and output is obtained
ISTS Algorithm is called
IF hypothesized behavior <-3 FAILED
IF hypothesized behavior >-3 PASSED
IF multiple >-3 INCONCLUSIVE
![Page 13: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/13.jpg)
Behavior Space Classifier
13
Settings
![Page 14: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/14.jpg)
BATS Behavior Space Classifier
Parameter 1 Range 1
Parameter 2 Range 2
Outcome Variable
For each cell, a
simulation run is made
Model output is
taken from Vensim ISTS
Algorithm is called
25 likelihoods
are obtained
The highest value is
entered in the
Classifier Grid cell
Param1=1Param2=9
oscct
![Page 15: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/15.jpg)
Behavior Class Mapper
15
Settings
![Page 16: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/16.jpg)
BATS Behavior Class Mapper
For each point, a
simulation run is made
Model output is
taken from Vensim ISTS
Algorithm is called
25 likelihoods
are obtained
The likelihood value for pre-specified class is plotted on the
graph
Pre-specified class is sshgr
(s-shaped-growth)
-3.99Param1=8Param2=16
![Page 17: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/17.jpg)
Identified Usage Modes of BATS• Structure-oriented behavior validity testing
• Extreme condition tests
• Behavior pattern validity testing• Barlas’ Multi-Step procedure and Graphical Inspection
• Sensitivity analysis• Behavior pattern sensitivity with respect to parameter changes
• Model calibration• Parameter calibration based on behavior patterns
• Policy design• Policy parameter specification based on behavior patterns
• Policy analysis• Policy structure specification based on pattern characteristics of model behaviors
![Page 18: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/18.jpg)
18
• Current version of BATS can be downloaded from the following link (Only for Windows for now. Mac version will be coming soon)• Website:
http://www.ie.boun.edu.tr/labs/sesdyn/projects/bats/index.html• Download links:
http://www.ie.boun.edu.tr/labs/sesdyn/projects/bats/bats_v1_11_win.ziphttp://www.ie.boun.edu.tr/labs/sesdyn/projects/bats/bats_v1_11_win.rar
• Please, keep in mind that this is a beta version.
• And if you give it a try, do not forget to send us some feedback that can help us to improve it.
![Page 19: SIGMA Workshop Part 1: Pattern-oriented Testing and Analysis](https://reader035.vdocuments.mx/reader035/viewer/2022081507/56815f05550346895dcdc423/html5/thumbnails/19.jpg)
Gönenç Yücel, Assist. Prof. Boğaziçi UniversityIndustrial Engineering [email protected]
For further questions
Thank you…