Sensitivity Analysis of anAgent-Based Model of

Culture’s Consequences for Trade

Saskia Burgers, Gert Jan Hofstede,Catholijn Jonker, Tim Verwaart

September 9-10, 2010 - Treviso (Italy)

Sensitivity analysis

• Generally considered “good modelingpractice”

• Of the essence i actual parameter values areuncertain

• A powerful tool in the process of modelverification and validation

• Specific problems arise when performingsensitivity analysis for agent-based models

Sensitivity analysis for ABM• Agent-based models may be very

sensitive to parameter changes inparticular parts of parameter space:– Nothing may happen in large areas in the joint

parameter space– Areas may exist where the system responds

dramatically to slight changes• Parameters may significantly interact• Sensitivity may be studied for aggregated

individual level outputs

The model

• Agent-based model of a gamingsimulation

• Models effects of deceit and trust in trade• Concepts of transaction cost economics(contract incompleteness and opportunism)

Simulation gaming

Agent roles in the game

producermiddleman

retailer

consumer

tracing agency

Strategies a buyer can chose

• trust• trace random samples• in addition to random tracing, negotiate

some refund in case quality of non-compliance

• require certification• buy low quality (no risk)

Process Model (TCE)

Partnerselection

DeliverNegotiation

Trust or Monitorand enforce

F e e d b a c k : b e l i e f u p d a t e

Information asymmetry:opportunity to defect

Purpose of the Sensitivity Analysis

• To explore the behaviour of the model allover the parameter space

Decision models

Endowment factorNegative update factorBelief updateTrust (initial value)Honesty decay factorMinimal honestyDeceit and trustRisk aversionQuality preferenceImpatienceNegotiation speedConcession factorNegotiationPreference (initial value)LearningLoyaltyPartner selectionParameter or variableActivity

Influence of culture

• Culture modifies parameter values in thedecision functions

• Describe culture based on Hofstede’s fivedimensions of national cultures

• Relational attributes have differentsignificance in different cultures:– Group distance– Status difference– Interpersonal trust

Total of 21 parameters

• Which areas in parameter space result inrealistic behavior?

• In which areas of parameter space cantipping points occur?

• Which parameters have significant effectsfor which outputs?

• Which interactions between culture andother parameters are important?

• Are the answers different betweenaggregate and individual level?

Approach is based on Jansen et al.

• meta modeling of results of parameter setsdrawn at random from the joint distribution(Monte Carlo analysis), by GLM;

• analysis of contributions of Top MarginalVariance (TMV) and Bottom MarginalVariance (BMV) of individual parametersor groups of parameters to the varianceexplained by the meta model

Step 1

• Generate random parameter sets, drawnfrom the joint distribution:

• Values of parameters were drawnindepentently, from uniform distributions,over the maximal realistic range

• Note: uniform distribution, because thepurpose is to explore model behavior, notto assess uncertainty

• Check for correlations

Step 2

• Run simulations for each parameter set• Observe relevant outputs• Replications with different values of the

random seed, to estimate the influence ofinternal stochastic effects on outputvariance

Observed outputs

• number of transactions;• number of failed negotiations;• average duration of negotiations;• number of high quality transactions;• number of deceitful transactions;• number of traces requested;• number of fines issued by the tracing agency;• average loyalty

Step 3

• The standard approach is to fit a linearmodel for each output yk

yk = β0 + Σ βi xi

• And analyse the contributions of theparameters xi to output variance

• However: system behavior is far from linear

Standard solution

• If linear model does not fit, try higher orderterms, e.g.

yk = β0 + Σ βi xi + Σ βiʹ′xi2

• Or interaction terms, e.g.

yk = β0 + Σ βi xi + Σ γij xi xj

• But this didn’t result in good fit (R2 > 80%)

First finding of further exploration

• For many of the parameter sets drawn atrandom, no transactions occur

• No obvious regions in parameter spacecould be discovered where transactionsoccur / no transations occur, nor by visualinspection, no by PCA

• Logistic regression could be applied todiscover the parts of parameter spacewhere transactions occur

transactions are unlikely to occur if

• group distance and LTO are both high;• status difference and concession factor are both low;• MAS, LTO, and negotiation speed are all high or all low;• status difference and intitial trust are both low;• IDV, LTO, and quality preference are all high;• status difference is low and initial partner preference is high;• initial partner preference is low and quality preference is high;• IDV and minimal honesty are both high;• honesty decay factor and endowment factor are both high

Conclusion of this first exercise

• For a model with complex interactions, it isvirtually impossible to understand why itbehaves like it behaves

• For further sensitivity analysis, focus onthe regions in parameter space wheretransactions occur, by resampling from theoriginal sample

Metamodel with quadratic terms,and 30 forwardly selected two-factor

interactions (R2 = 80 %)

0.00.0Honesty decay factor3.01.2Initial partner preference

0.00.0Minimal honesty6.32.7Initial trust

0.00.1Endowment factor1.90.0Status difference

0.30.9Negative update factor5.10.3Mean status

3.00.0Risk avoidance6.82.7Group distance

0.71.5Quality preference7.72.0-- MAS

2.50.6Impatience6.80.7-- LTO

39.331.8Negotiation speed3.40.8-- UAI

25.09.1Concession factor5.30.2-- IDV

0.10.0Loyalty decay factor0.60.0-- PDI

0.00.0Loyalty parameterIndex of culture

BMV(%)

TMV (%)ParameterBMV(%)

TMV(%)

Parameter

Sensitivity of other outputs• negotiation failure:

– negotiation speed, concession factor, impatience• negotiation duration:

– negotiation speed, UAI, MAS• quality:

– intitial trust, LTO, quality preference• deceit:

– initial trust, MAS• tracing:

– MAS• fines:

– MAS• loyalty:

– initial partner preference, negotiation speed

Conclusions

• Number of transactions is most sensitiveto negotiation speed and concessionfactor

• Cultural and relational factors have theireffects only in interaction with other factors

• Analyse sensitivity culture by culture,using a sample of 62 actual nationalcultures

Sensitivity differences (TMV)

15.80.10.930.81.68.30.00.5Japan

6.94.811.527.12.63.80.00.0Austria

0.00.656.210.30.40.93.11.3Iran

2.10.028.846.20.31.60.00.7Netherlands

0.00.024.123.45.58.51.74.9Uruguay

11.52.41.437.61.911.50.00.0Hungary

0.00.037.917.54.73.38.70.7Morocco

0.00.040.911.60.00.20.116.9Indonesia

1.60.530.224.81.83.41.04.1mean

riskavoid.

qualitypref.

negot.speed

conces.factor

partnerpref.

initialtrust

meanstatus

groupdistance

nationalculture

Conclusion

• Sensitivity analysis and the interpretationof its results are much easier if parametershaving many interactions are fixed

• For specific situations the results of thispartial analysis may be more useful thanglobal results

Overall conclusion• Logistic regression can be used to predict

regions where realistic behavior occurs.• Parameters that have significant effects can be

identified through metamodeling, even forcomplex systems. However, the analysis is notstraightforward.

• When keeping culture constant, straightforwardmethods for sensitivity analysis can be applied.Results differ considerably across cultures.

• Sensitivity of individual agents can differconsiderably from aggregate level sensitivity.