a core course on modeling
DESCRIPTION
A Core Course on Modeling. Week 7-A working model – and then?. Contents . The Need for Interpretation Approach Criteria for Modeling Genericity Scalability Specialization Audience Convincingness Distinctiveness Surprise Impact Criteria for modeling and purposes - PowerPoint PPT PresentationTRANSCRIPT
1
A Core Course on Modeling
Contents • The Need for Interpretation• Approach • Criteria for Modeling
•Genericity•Scalability•Specialization•Audience•Convincingness•Distinctiveness •Surprise •Impact
• Criteria for modeling and purposes • Summary
• References to lecture notes + book
• References to quiz-questions and homework assignments (lecture notes)
Week 7-A working model – and then?
2
A Core Course on Modeling
The Need for Interpretation
Week 7-A working model – and then?
define
conceptualize
conclude
execute
formalize
context initial problem
initial problem conceptual model
conceptual model formal model
formal model result
result resolve initial problem?
3
A Core Course on Modeling
The Need for Interpretation
Week 7-A working model – and then?
Model makers in 19th century London predicted 3 meter rising of horse manure levels by 1950
Model makers in 21st century predict 0.5 meter rising of sea levels by 2050
A model outcome in itself cannot solve a problem.
Interpretation is necessary final step.
How to achieve a good-quality interpretation?
4
A Core Course on Modeling
Approach
Week 7-A working model – and then?
We want to say that one model (including the interpretation of the outcomes) is better than another.
‘Better’ means: with respect to fulfilling the model’s purpose.
Therefore we need criteria to compare.
The first step to quality assessment is the ability to compare things that are utterly different. Comparison invariably needs criteria.
5
A Core Course on Modeling
Approach
Week 7-A working model – and then?
So: we will try to formulate criteria for comparing problem solutions.
Problem solution =
model outcome + interpretation
can be assessed (compared) with the tools of Chapter 6 (sensitivity analysis, uncertainty analysis)
As yet, we have no means to compare ‘quality’ of interpretations and assumptions
6
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
When formulating criteria for comparing problem solutions, the main problem is the immense variety of problems asking for modeling.
Our criteria need to apply to all of them.
The large variety of possible modeling processes is a complication for finding a single generic set of criteria.
7
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
The standard approach to finding a complete set of ‘things’, is to construct a complete taxonomy.
A complete taxonomy requires enumerable properties: properties where all possible values can be enumerated.
The quatro-game is based on 4 enumerable properties: color(black,white), size(tall,short), shape(round,square), interior(hollow,solid). Rows of 4 must be formed of pieces which all have a common value for one property
8
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
Property 1:
Does the criterion regard the begin (definition stage of the modeling process) or the end (conclusion stage)?
The input-side of a problem relates to stage 1 (problem definition); the output-side relates to stage 5 (conclusion: presentation and interpretation). This photo seems to suggest that input and output are relative notions …
•Example of criterion related to definition: what is the scale of the problem? (scalability)
•Example of criterion related to conclusion: how much depends on it? (impact)
9
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
Property 2:
Does the criterion regard the ‘inside’ (model, modeled system) or the ‘outside’ (stakeholders + context) ?
In this famous Velasquez painting, the stakeholders and the modeled system are deliberately confused. In modeling practice, the distinction between subject (the modeled system) and object (the stakeholder(s)) are clearly distinct. Hopefully.
stakeholder2: the modeler stakeholder1:
the customer
the model: the present painting
the modeled system: part of the world, including the model and some stakeholders
•Example of criterion related to the inside (modeled system): what is the scale of the problem? (scalability)
•Example of criterion related to the outside (stakeholders): how much knowledge do the stakeholders need? (specialization)
10
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
Property 3:
Does the criterion regard qualitative or quantitative aspects ?
The distinction quantitative – qualitative is sometimes associated to the closed – open distinction. ‘Closed’ means that all possible outcomes can be foreseen or enumerated; ‘open’ means that this is not possible.
•Example of criterion related to quantitative aspect: what is the scale of the problem? (scalability)
•Example of criterion related to qualitative aspect: how different can modeled systems be? (genericity)
11
A Core Course on Modeling
Criteria for Modeling
Week 7-A working model – and then?
Combinations
The 3 distinctions together form an imaginary cube, each corner corresponds to one of the criteria.
impactquantitativeoutsideconclude
surprisequalitativeoutsideconclude
distinctive-ness
quantitativeinsideconclude
convin-cingness
qualitativeinsideconclude
quantitativeoutsidedefine
speciali-zation
qualitativeoutsidedefine
scalabilityquantitativeinsidedefine
genericityqualitativeinsidedefine
criterionqualitative / quantitative
inside / outside
define/ conclude define
conclude
inside
outside
qualitative
quantitativeaudience
distinctiveness
scalability
specialization
audiencegenericity
convincingness
surprise
impact
12
A Core Course on Modeling
Genericity (define,inside,qualitative)
Week 7-A working model – and then?
Genericity: to which extent is the approach capable to handle various types of modeled systems / purposes?
Examples:•Dynamical systems: buffers & flows
•Process modeling: state charts
•Data modeling: statistics
•Modeling from scratch: 4 categories
13
A Core Course on ModelingWeek 7-A working model – and then?
Genericity: example
Given an object of some shape and density ; we want to know its mass.
1.If it is a cube with side p: m=a3
2.if it is a rectangular block with height h: m=h*Areatop
3.if it is a truncated pyramid: m= h*(Areatop + (Areatop*Areabottom) + Areabottom)/3 (works for all these shapes!)
Above models are increasingly generic; each can deal with the simpler cases as well.
How heavy is the Borobodur? (E.g., in the context of risks for landslides etc.: the Borobodur is one of the heaviest temples in the world, restauration in the 1980-ies required partial reinforcement of the hill on which it stands to prevent landslides). Only formula (‘model’) 3 is sufficiently generic
Genericity (define,inside,qualitative)
14
A Core Course on Modeling
Scalability (define,inside,quantitative)
Week 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
Many problems contain some characteristic dimension (e.g., n=number of boxes). If n grows, there will come a point where the model fails when trying to solving the problem. The larger n can be, the better the scalability of the solution.
15
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
The size of Google’s data repository, according to http://www. worldwide websize.com/ , varied between 15 and 55 billion webpages in less than 4 months. Still, the response time stays near constant due to sophisticated indexing strategies
nO(1)
Scalability (define,inside,quantitative)
16
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
To tell if there is a needle in the haystack is easy, provided the hay is sorted: check the halfway element. If this is larger than the needle, discard the upper half, otherwise discard the lower half – and checking the halfway element in the remaining part until either found or not found.
nO(1log n)
Scalability (define,inside,quantitative)
17
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
For many tasks, the effort of doing n things is proportional to n. If one chicken lays three eggs per week, 5 chicken lay 15 eggs per week. This does not hold for everything, though … climbing n flights of stairs may take more effort than n the effort of one. But computers don’t get tired, and many computer tasks scale similarly to chicken laying eggs.
nO(n)
Scalability (define,inside,quantitative)
18
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomialSorting an array of unsorted things can be done in O(n log n). Military go faster, but they work in parallel.
nO(nlogn)
Scalability (define,inside,quantitative)
19
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
Straightfowardly sweeping a matrix takes O(n3), but faster methods exist that bring this down to something like O(n2.5). The difference is noticable only for n sufficiently large.
nO(np)
Scalability (define,inside,quantitative)
20
A Core Course on ModelingWeek 7-A working model – and then?
Scalability: to which extent can some characteristic dimensions of the problem increase, where the model still functions?Scale: n (=number of elements, states, records, time steps, …)
Performance:
O(1) – solving effort (SE) does not depend on n
O(log n) – SE hardly depends on n
O(n) – SE proportional to n
O(n log n) – SE slightly worse than proportional
O(np) – SE polynomial
O(2n), O(n!), … – SE worse than polynomial
The travelling salesman problem asks for the shortest route connecting n places. Exact solution is believed impossible in O(np) but good approximations sometimes take only O(np)
nO(n!)
Scalability (define,inside,quantitative)
21
A Core Course on Modeling
Specialization (define,outside,qualitative)
Week 7-A working model – and then?
Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner?
How much control should the modeler give the problem owner over
•validity of assumptions,
•regime of application,
•use of results?
A model, intended for specialists has to face the challenge of withstanding expert scrutiny. On the other hand, a model intended for laymen has to be robust against unintended misuse.
22
A Core Course on ModelingWeek 7-A working model – and then?
Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner?
1.Invest in presentation•graph style (line plot, scatterplot, polar plot, barchart, pie chart, 3D surface, …)
•labels, captions, scales (zero, lin/log, units, major / minor scale lines …)
•how to represent uncertainty?
•which plots to combine in a single graph?
The appropriate visualization style is the answer to the question:
what message should this graph convey?
Specialization (define,outside,qualitative)
23
A Core Course on ModelingWeek 7-A working model – and then?
Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner?
2.When in doubt, be conservative
A modeler resembles a blindfolded torreador: without full knowledge, consequences of the modeled system being mischievous may be quite harmful. For good reasons, the jackass adagium reads: ‘don’t try this at home!’
Specialization (define,outside,qualitative)
24
A Core Course on ModelingWeek 7-A working model – and then?
Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner?
3.Be warned for biased users• give a full and detailed account of as
much as possible assumptions and estimates
• insist on the report being indivisible
• refrain from easy-to-misapprehend results
• Get a second opinion Users having personal interests in model outcomes may not always take a balanced stand.
Specialization (define,outside,qualitative)
25
A Core Course on ModelingWeek 7-A working model – and then?
Specialization: to which extent does the model / model outcomes requires specialized knowledge on behalf of the problem owner?
4.Take care for self-fulfilling and self-denying prophecies
• wicked problems: the problem solution post-hoc modifies the problem statement
Both legendary prophet Michael Nostradamus (glass bowl?) and present-day opinion investigators (black ballot box?) relate to the working principle of self-fulfilling and self-denying prophecies.
Specialization (define,outside,qualitative)
26
A Core Course on Modeling
Audience (define,outside,quantiative)
Week 7-A working model – and then?
Audience: what size of intended audience does the model address?
•Large size: low level of specialization
•Large size: no option for bi-directional communication
•Large size: consider interactive model (example: ‘stemwijzer’)
In the 1999, Wachowski-brothers’ movie ‘the Matrix’, a simulation’s intended audience size is stretched so as to encompass the entire world’s population – with the extra that nearly everybody is ignorant of the fact.
27
A Core Course on Modeling
Convincingness (conclude,inside,qualitative)
Week 7-A working model – and then?
Convincingness: a model is more convincing if it contains fewer and/or less implausible assumptions.
•Are assumptions logically deducible from other, less problematic assumptions?
•If not, are there any first-principle ‘laws’ to back up assumptions (example: if this component behaves as a lever, we can use
the physical laws of torque in a lever)?
•If not, can we construct a simplified formal model system (example: tidal motions as a lumped system) with outcomes that can be empirically verified?
• If not, can we get support from comparison to observations on an empirical model system (example:
behavior of visitors in shopping malls to predict buying behavior)?
•If not, is there at least any argument for consistency of the assumptions? (example: price elasticity, ‘fun’ in the taxi model)
In the early days of photography, a photo (as prototypical empirical evidence) counted as proof. As this no longer holds, we need to re-think what constitutes ‘plausible evidence’.
28
A Core Course on ModelingWeek 7-A working model – and then?
Convincingness: a model is more convincing if it contains fewer implausible assumptions.•Convincingness in the absence of ‘laws’:
•conceive a plausible formal model system to which laws do apply (e.g., replace the world + oceans by a homogenous ball covered by a sheet of water). The measure of correspondence between these two is left out of the discussion;
•convincingness relies on the agreement with empirical data.
Tides are caused by subtle interplay between gravity, water (viscosity vs. inertia), and the complex geometry of the earth’s sea floors. A (glass box) model to predict tidal motions contains none of these, but a hypothetical ‘sheet of water on a perfect sphere’ instead.
Convincingness (conclude,inside,qualitative)
29
A Core Course on ModelingWeek 7-A working model – and then?
Convincingness: a model is more convincing if it contains fewer implausible assumptions.•Convincingness in the absence of a formal model system and/or no empirical validation:
•look for an empirical model system
• physical: water tank, wind tunnel
• economical, social: comparative populations, historical survey data
• biological: the guinea pig!
Scale models are physical replica of the modeled system, usually at a smaller scale. Applications include windtunnel (flow, drag) and hydromechanical investigations, but also scale models as used in architecture and urban studies. These are examples of empirical model systems. BTW: J. Swift’s story seems to relate to a scale model; it is actually a social model.
Convincingness (conclude,inside,qualitative)
30
A Core Course on Modeling
Distinctiveness (conclude,inside,quantitative)
Week 7-A working model – and then?
Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar.•Most purposes ask for some form of distinctiveness. Examples:
•prediction 1: something is correctly predicted to happen at time T1 rather than T2; how close can T1 and T2 be?
•explanation: Q1 causes P1 and not P2; Q2 causes P2 and not P1. How close can Q1 vs. Q2, or P1 vs. P2 be so that the explanation holds?
•optimization: input P1 yields output Q1, and input P2 yields output Q2, where Q1 correctly is found to dominate Q2. How close can P1 and P2 be?
Many modeling purposes boil down to the ability of distinguishing between two alternatives that are as similar as possible. This relates to accuracy and precision, but also to non-numerical purposes.
31
A Core Course on ModelingWeek 7-A working model – and then?
Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar.
This hinges on the notion of ‘distance’, explained in week 6.
The quantitative measure for distinctiveness is usually a similarity measure: the more similar A and A’ whereas the model can successfully distinguish them, the better its distinctiveness.
Distinctiveness (conclude,inside,quantitative)
32
A Core Course on ModelingWeek 7-A working model – and then?
Distinctiveness: a model has higher distinctiveness if it allows distinction between alternatives that are more similar.
Distinctiveness relates to two common types of errors:
•false positive (conclude X where there is no X)
•false negative (don’t conclude X where there is X)
Successful spreading of false money is an example of false negative: the counterfeit goes unnoticed. Coin operated vending machines have to safeguard against both false positives and false negatives; the consequences of the two types of errors being very different.
This is true for most models where distinctiveness plays a role.
Distinctiveness (conclude,inside,quantitative)
33
A Core Course on Modeling
Surprise (conclude,outside,qualitative)
Week 7-A working model – and then?
The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas.
There are Open and closed spaces of outcomes.
Examples of closed outcomes:
•A model computing the probability of X can only produce a number between 0 and 1.
•A model verifying Y can only produce ´true´ or ´false´.
A widespread opinion is, that computations cannot ‘output more than you put in’. This underestimates computers’ scope of computation, or overestimates humans’ scope of anticipation.
34
A Core Course on ModelingWeek 7-A working model – and then?
The potential for surprise of a model outcome is the extent to which it may bring unforeseen new ideas.
Examples of open outcomes:
•result of using ontologies
•outcome of evolutionary algorithms
•outcome of PCA, abstraction, …
Common feature: output space has size O(2n) whereas input space has size O(n)
Evolutionary programming, applied to shape design. Given a comprehensible genotype representation, the space of phenotypes is virtually infinity.
Xai Xu et. al., Special Issue of SIGGRAPH, Vol. 31, No. 4, pp. 57:1-57:10, 2012
Surprise (conclude,outside,qualitative)
35
A Core Course on Modeling
Impact (conclusion,outside,quantitative)
Week 7-A working model – and then?
The potential for impact of a model is the extent to which the model outcome can affect the stakeholders (either beneficially or adversely).
Two perspectives:
•prestige and profit: the more impact the better
•risk and responsability: the less impact the better
‘Impact’ is a double-edged sword: some modelers attempt to increase the impact of their models whereas other modellers prefer to play safe and seek modeling with less impact.
36
A Core Course on ModelingWeek 7-A working model – and then?
The potential for impact of a model is the extent to which the model outcome can affect the stakeholders.Intended impact can be capitalized:
r1=revenues in absence of model outcome;
r2=revenues with model outcome present;
c1=cost of ownership in absence of model outcome;
c2=cost of ownership with model outcome present;
=((r2-r1)-(c2-c1))/(|r2-r1|+|c2-c1|)
To assign a quantitative value to a model outcome we must assume some market mechanism where we identify revenues and costs of ownership, both in the absence of the model outcomes and in their presence.
• 0 ||1: magnitude of impact
• If 0 1: positive impact (e.g., models for optimization);
• if -1 0 negative impact (e.g., risk prediction or analysis)
Impact (conclusion,outside,quantitative)
37
A Core Course on ModelingWeek 7-A working model – and then?
The potential for impact of a model is the extent to which the model outcome can affect the stakeholders.Adverted impact can be quantified or capitalized:
C=the estimated chance (per time interval) of an incident (=an erroneous model outcome, e.g. a type-I or type-II error)
V=the estimated value loss per incident
Both C and V can serve to quantify the negative impact of a model. Their product CV has dimension of money / time and can be added to c2 in the formula for .
Impact (conclusion,outside,quantitative)
38
A Core Course on ModelingWeek 7-A working model – and then?
The potential for impact of a model is the extent to which the model outcome can affect the stakeholders.
The impact of a model should balance with its reliability.
Impact (conclusion,outside,quantitative)
39
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
Given the purpose of the model, the modeler should identify the most relevant criteria to seek improvement when necessary.
For instance: … (see next sheets)
40
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
prediction (1, 2): convincingness, distinctiveness, impact
An almanac is a model to serve the purpose of predictions (both 1 and 2). Its impact is rather large (given the long historical record of almanac-use), but the distinctiveness is small, and its convincingness decreases over time.
41
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
compression: scalability, audience, distinctiveness
Printing is a form of compression (transferring written text into printed text) with huge population and distinctiveness; by present day’s standards, the scalability is not impressive
42
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
inspiration: surprise
All the world’s religious traditions acknowledge holy scriptures of some sort as primary source of inspiration. Their potential for surprise is sometimes rather limited.
43
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
unification: genericity, convincingness, surprise
In alchemy, the pre-scientific forerunner of both chemistry and psychology, unification of conflicting properties was one of the leading purposes. In modern times, scores on all three criteria are low – when seen through modern eyes.
44
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
abstraction: distinctiveness, convincingness, surprise
The term ‘abstraction’ in modern art no longer defines a relation between a modeled system and a model; rather, the absence of any modeled system is the defining feature for true abstract art. The potential for surprise, however, sometimes still applies.
45
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
verification: scalability, convincingness, impact
Biting a golden coin is the proverbial way to verify its authenticity. Scalability and convincingness are probably low, but if the coin contains chocolate, or if the biter has metal inlays, the impact may be quite considerable.
46
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
exploration: genericity, surprise
Few models for exploration will have a larger potential of surprise than the Mars exploration robot.
47
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
decision: convincingness, distinctiveness, impact
The Delphi oracle was a decision-making model that derived its convincingness from its deliberately low distinctiveness – thereby being known for its major impact.
48
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
optimization: genericity, scalability, convincingness, impact
Of all generic optimization models for continuous systems, hill climbing is perhaps one of the most generic ones. It often suffers from scalability, though – even when implemented by modern, automated devices.
49
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
specification: genericity, distinctiveness
Sometimes the distinction between a specification model (blueprint) and a realization is somewhat blurred.
50
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
realization: genericity, distinctiveness, impact
Realization …
51
A Core Course on Modeling
Criteria for modeling and purposes
Week 7-A working model – and then?
Not each of the eight criteria is equally relevant for all purposes.
steering & control: distinctiveness
The prototypical stick-and-carrot steering, with the stick tied to the donkey’s back is an example of open-loop control without a set-point.
52
A Core Course on Modeling
Summary
Week 7-A working model – and then?
•Leading question: to what extent has the initial problem been solved?
•Approach: criteria to assess the quality of the modeling process
•Taxonomy: definition or conclusion stage? inside or outside? Qualitative or quantitative?
•Resulting criteria:
• Genericity: how many different modeled systems can we handle?
• Scalability: how large can the size of the problem be?
• Specialization: how much should the intended audience know?
• Audience: how large can the intended audience be?
• Convincingness: how plausible are the assumptions?
• Distinctiveness: e.g., how accurate, how certain, how decisive is the model outcome?
• Surprise: to what extent can the model outcome give new insight?
• Impact: how big can the consequences of the model outcome be?
•Criteria for modeling quality are related to purposes.