is simulating forgetting its history?

http://www.simian.ac.uk

Edmund Chattoe-Brown (ecb18@le.ac.uk)Department of Sociology, University of Leicester, UK

Is Simulating Forgetting Its History?

2 http://www.simian.ac.uk

Thanks

• This research funded by the Economic and Social Research Council (UK) as part of the National Centre for Research Methods.

• The usual disclaimer applies.

3 http://www.simian.ac.uk

Simulation as an innovation

• Simulation has had a very long “lead time” outside the mainstream.

• Now that it appears to be reaching “take off” (Conte, Janssen), those encountering it (particularly critics) may start not from ignorance but from mistaken beliefs (gaming, system dynamics).

• We can still see the side effects of long lead time (obscure publications, lack of a “taught canon”).

4 http://www.simian.ac.uk

The “culture” of simulation

• I propose this cautiously. All science has its “culture” which is rarely documented and hard to measure.

• The culture of social simulation appears to me to be that (in its details) work before 1990 (?) is too different to be of much use. It is too vague or too “clunky”. (Citation age distributions.)

• In a nutshell, should we now look backwards a little more systematically?

5 http://www.simian.ac.uk

How?

• A case study (the full paper has two) of a paper that is undeservedly forgotten.

• It was published in a “good” journal (and is not so hard to find).

• It follows the “present day” methodology of simulation with use of data and attempts at calibration and validation. (Significant numbers of modern papers still do not.)

• It was published in 1965: 45 years ago!• It is not cited in JASSS and barely in SCOPUS or

SSCI.

6 http://www.simian.ac.uk

Why?

• We should feel a much greater urgency in our aspirations for “good quality” simulation if it is made clear to us that this standard was achievable 45 years ago.

• We should worry much more that our critics might have longer memories than we do.

• We can no longer claim that the difficulties of meeting “scientific standards” are due to the novelty of the topic.

• We can remind ourselves what we have forgotten and avoid “reinventing the wheel”.

7 http://www.simian.ac.uk

Murder Rates I

UK Murder Rate per million

0

2

4

6

8

10

12

14

16

1880 1900 1920 1940 1960 1980 2000 2020

Date

Number of Murders

Source: http://www.parliament.uk/documents/commons/lib/research/rp99/rp99-111.pdf

8 http://www.simian.ac.uk

Murder Rates II

Source: Eisner (1995).

9 http://www.simian.ac.uk

Spatial diffusion

• Hägerstrand, Torsten (1965) ‘A Monte Carlo Approach to Diffusion’, Archives Européennes de Sociologie, 6, pp. 43-67.

• Doesn’t mention “the s word”: Searches have to be done broadly. Unless people “get dusty” in the library, things that cease to be cited don’t “recover”.

• Cited 22 times in SCOPUS (often not relevantly to the method) and not at all in JASSS (to be fair there is also a book). Compare 98 hits in JASSS for Schelling.

10 http://www.simian.ac.uk

The model I• Spatial distribution of an innovation (farming subsidy

in Sweden).• Knows where farms are, who was eligible and when

they adopted (latter aggregated).• Farms adopt as soon as they hear from an adopter

and they hear probabilistically as a function of distance.

• The distance function is calibrated on phone use and migration data.

• Do we, because of the powers of our technique, tend to assume more complication than we need?

• Real spatial pattern of diffusion can be compared with simulated outcomes.

11 http://www.simian.ac.uk

The model II• More advanced version presented at limits of

computer power.• Number of agents in each cell based on real data.• Attempt to mimic geographical barriers as “bars”

blocking messages with certainty or 50% probability.• Measures fraction of eligible adopters in each cell

adopting over time.• Measures spatial distribution of adoption ranges (0-

20%, 21-40% and so on) and extent of coincidence.• Draws attention to absence of formal quantification

approach for comparison. Has this changed? Null model approach with cheap computing?

12 http://www.simian.ac.uk

NetLogo Grand Canyon

13 http://www.simian.ac.uk

Farms

14 http://www.simian.ac.uk

Initial Adopter

15 http://www.simian.ac.uk

Year 1

16 http://www.simian.ac.uk

Year 2

17 http://www.simian.ac.uk

Year 3

18 http://www.simian.ac.uk

Geographical barriers

19 http://www.simian.ac.uk

Points of interest• Example of KISS. Looks like it shouldn’t work but does.• Apparently neat abstraction of “geography” for particular

domain. (Lost opportunity between “flat planes” and GIS?)• Follows the G+T “box” methodology “instinctively”.• Collects no data but makes intelligent use of it. Very clear

what “more data” (i. e. exact adoption dates) would be good for: Reducing likelihood of spurious fit.

• Pretty clear programme of future work offered.• Cries out for replication and sensitivity (or baseline)

analysis with cheaper computer power.• Raises interesting measurement issue of comparing

contour maps. (Matching real and simulated data too often glossed over or discussed using simple cases.)

20 http://www.simian.ac.uk

Hazards

• Difficulties of reconstructing data from maps for example. Different kinds of data will need to be “archived” in different ways. Is there any danger that in 20 years, ARC data will be like floppy discs?

• Surprisingly “relaxed” about sources of data and details of data collection. (For example, don’t know variance in telephone range data and don’t know if data set chosen is for subsidy area or “typical”.) This may be an idiosyncrasy of TH.

21 http://www.simian.ac.uk

A general lesson• Real models cut through abstract methodological

“worries”.• What should a “tick” mean in our simulation?• Here, it has to mean the data collection period.• But if we can match the data then we have abstracted

from the “real” time period just as we might abstract from any other “real” process or variable. Of course, it may make our other parameters somewhat more abstract in interpretation.

• Don’t worry until you need to (provided you reach the testing stage).

22 http://www.simian.ac.uk

Conclusions I• If we are still publishing models that are not calibrated

or validated, it is not because the method is too new to know yet how this is to be done. (Point by Marco Janssen that this standard may not emerge spontaneously.)

• We clearly can forget really rather good work. Even if this is forgivable (references obscure, no common canon), how much else have we forgotten that is important? Other off the cuff examples: Loehlin, Albin. We need to remember the good work because our critics will remember the bad! Like the murder example, given a different horizon, perhaps we are not progressing but regressing!

23 http://www.simian.ac.uk

Conclusions II• KISS models may help us worry less about

methodology in the abstract. The right answer to the “tick length” problem may be (within reason) what works.

• With a good knowledge of what exists and thoughtful simulation design, we can achieve things with “off the shelf” survey data. The other case study in the full paper does this too. (This will make us a lot more popular with sociologists at least.)

• Good old models are a nice resource for replication (which also “re-promotes” them) given how much computer power has increased. Get dusty in the library!

24 http://www.simian.ac.uk

Conclusions III

• Ironically, even the principled attempt to assess “how we are doing” goes back to the (again virtually forgotten) Dutton and Starbuck survey of 1970.

• Is the reclaiming of our history (both raising what is good to visibility and saving the need for everyone to dredge the bad - far from a “golden age” argument - for themselves) a brilliant use for collaborative content generation (wiki and so on?)