1

Authoring an ontology of place semantics using volunteered

geographic information

Alistair Edwardes and Ross Purves

Department of GeographyUniversity of Zurich

2

Overview

• Motivation for considering place• Why this is useful in the context of image

retrieval• Where can we find place descriptions• How might we build semantic resources

from these

3

Motivation

• GI bias towards spatial representations of Geography– BUT

• Not all geographic information is spatial

4

Motivation

5

Motivation

• GI bias towards spatial representations of Geography– BUT

• Not all geographic information is spatial• Doesn’t reflect how people experience, remember

and talk about geography

• What else is there?

6

Place

• Space-Place Continuum– Objective-Subjective– Universal-Personal– Machine-Human

7

Where is place important?• Location-Based Services

– Movement towards technologies closer to everyday, direct experience, activity based.

“The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004)

• Web 2.0– Social interaction, user generated information, personal memories

8

Where is place important?

[Baostar] documents place in a way that embodies neogeography, where human perspective and social interaction supercede latitude and longitude.

9

Where is place important?• Location-Based Services

– Movement towards technologies closer to everyday, direct experience, activity based.

“The historical demarcation in psychological and behavioural geography between direct and indirect experience blurs when handheld devices are used as an adjunct to reality in the field.” (Longley, 2004)

• Web 2.0– Social interaction, user generated information, personal memories

• Geographic Information Retrieval– Vernacular geography, organising activities

• Photographs

“GI theory articulates the idea of absolute Euclidean spaces quite well, but the socially-produced and continuously changing notion of place has to date proved elusive to digital description except, perhaps, through photography and film.” (Fisher and Unwin, 2006)

10

Place and photographs

• Observer/Viewpoint– Different from universal perspective of maps

• Information is perceptual– Closer to direct experience– Pre-cognitive– Many ways to interpret

• Highly ephemeral– moment in time

11

Problems

• Many such moments in space and time– How do we sort through them?– Image Retrieval

1. How do we access a description of the contents of an image?

2. What do we describe about an image?

12

Image Retrieval Approaches (CBIR)

• Content Based Image Retrieval

• “Natural” for format– Use primitive

features like colour, shape and texture

Smeulders et al, 2000

13

The Semantic Gap

“The semantic gap is the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation.” (Smeulders et al, 2000)

• Concept based image retrieval– Define high-level semantic concepts

• Defined in loosely structured word lists (LSCOM)– Detect using low-level feature vectors

14

Image Retrieval Approaches (TBIR)

• Text-based Image Retrieval– Describe contents in text

1. How do you access this description?2. What should be described?

15

How do you access a description?

• Manual annotate– Expensive and time consuming

• Definitively won’t scale• Need to automate

– Inconsistency amongst annotators (Markey, 1984)

• Inter-annotator agreement (e.g. Ahn and Dabbish, 2004)

• Controlled Vocabulary– Getty Images 12,000 keywords with 45,000 synonyms

(Bjarnestam, 1998)

– Specialist knowledge

16

Tripod Approach

• Describe location instead– Spatial data– Geographic knowledge– Web resources

17

Obligatory Project Slide• European Commission Sixth Framework Programme Project• 3 years (started January 2007)• €3,150,000• Partners

– University of Sheffield, United Kingdom– University of Zurich, Switzerland– Dublin City University, Ireland– Otto-Friedrich-Universität, Bamberg, Germany– Cardiff University, United Kingdom– Ordnance Survey, United Kingdom– Centrica, Italy– Geodan, The Netherlands– Fratelli Alinari Istituto Edizioni Artistiche, Italy– Tilde, Latvia

• Focus on image retrieval by users of professional stock photo libraries• Focus on particularly geographic images

– e.g. Natural landscapes

18

What should be described? ModesFacets

Specific Of Generic Of About

Who?

Individually named persons, animals, things

Kinds of persons, animals, things

Mythical beings, abstraction manifested or symbolised by objects or beings

What?Individually named events

Actions, conditions Emotions, Abstractions manifested by actions

Where?Individually named geographic locations

Kind of place geographic or architectural

Places symbolised, abstractions manifest by locale

When?Linear time; dates or periods

Cyclical time; seasons, time of day

Emotions or abstraction symbolised by or manifest by

Panofsky-Shatford facet matrix – Shatford (1986)

19

What should be described?GenericOf: EngineeringAbout: Innovation, technical brilliance, complexitySpecificOf: [Da Vinci Chambord staircase]

20

What should be described?

• Advertising– Mystery– Isolation– Chocolate

21

Dimensions of Place

• Theoretical Dimensions of Place– Physical setting, activities, meanings Relph (1976)

– location, material form, investment in meaning Gieryn (2000)

– location (spatial distribution activities), locale (the setting), sense of place Agnew (1987)

• Similar to Shatford – SpecificOf – Location/Identity– GenericOf – Setting– About – Sense of place, meanings, activities

22

Organisation in Tripod

• Concept Ontology– GenericOf

• Scene types• Elements

– About• Sense of Place

– Affective, Cognitive, Conative• Qualities and Activities

• Toponym ontology– SpecificOf

• Identity, location

23

How can we elicit place descriptions?

• Inductively– Ask people

• Adjective Check List• Category Norms / Basic Levels

– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts

– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources

• Deductively– Look at structured semantic resources

• Use a combination of these approaches

24

Empirical Elicitation

• Online interactive experiments • Database of 150 landscape photographs from Switzerland, Germany, Holland, Italy, Portugal and the UK.

Free description

Controlled vocabulary Sort and describe

25

landscape village valley countryside wall field top sunlight hills cliffs castle peak fortress narrow birch

town plain land hillside lawn farm creek orchard harbour fall rocks fields flowers buildings ridge

bushes woods track stone ruins meadows undergrowth dunes deciduous cross coast brush wooden weather walls villa monument lane city waterfall vineyard valleys tundra trail terraced

temple sunrise station ship shelter rolling riverbank pylons peninsula pathway park outcrop munros loch lightly historical hilltop ground gardens foot

flood dock cycle croft covered catwalk cargo canopy canal burned brook boulder bay arm angler

altitude acre

wooded country surrounded scene mountainous low dense coastline young world wonderland wintry waterside variation uniform two twilight tuscany

tide three sunlit suburban structure streaming snowland sicily sedimented seaside seacoast savanna ruined route romantic rollign roadside riverside riverbed rising rigi region reach ranch prospect populated pond plateau photographed peeking

pastures passage overgrown outback november mountaintops moor montains mist mediterran lonley lodge leafy lakeside junction heaven hanging gurgling greek

france foreground following floor featureless falling dive development dawn countour copse cliif babbling area alice

mediterranean summer quiet winter hilly calm isolated rural steep open mediteranian cold hot

blurry windy free spring remote lonely clean dangerous cool tranquil arid well rugged rough

picturesque dark vista urban snowy sandy rotting northern medieval intervention home

hike fertile end dusk back

running rowing touring biking looking enjoying bycicle parking live estate

Overall

Overall ∩ Qualities

Overall ∩ Elements

Overall ∩ Activities

27

Subjectivity

28

How can we elicit place descriptions?

• Inductively– Ask people

• Adjective Check List• Category Norms / Basic Levels

– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts

– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources

• Deductively– Look at structured semantic resources

• Use a combination of these approaches

29

Volunteered Resources

Gatliff Trust Hostel on Berneray: Picture taken from the beach on Berneray of the historic Gatliff Trust Hostel. Visited in the 1990s, shortly before the causeway linking Berneray to North Uist was built.

30

Word lists(WordNet)

Scene types

Examine frequency and co-occurrence of scene types and terms with respect to a database of image captions

www.geograph.org.uk

Validation of terms

31

Activities Elements Adjectives Activities Elements Adjectives Beach n=2824 Village n=12707

Surfing Shingle Sandy Conservation Pub Deserted Bathing Sand Deserted Reading Shop Pretty

Defence Cliff Eroded Fishing Inn Green Swimming Headland Soft Playground Church Quiet

Tourism Bay Rocky Defence Housing Lovely Wading Sea Warm Bowling Edge Pleasant

Protection Rock Glacial Tourism Cottage Beautiful Sport Coast Low Football Main Road Remote

Shipping Shore Beautiful Entertainment Village green Unusual Golf Island Lovely Sitting Stone Large

Hill n=16232 Mountain n=1256 Climbing Fort Steep Biking Peak Distant

Skiing Top Distant Kayaking Summit Black Holidays Summit Wooded Outing Ridge Remote

Observation Horizon Black Mountaineering Moorland Rocky Sitting Ridge Rough Escape Quarry Grassy

Walking Sheep Grassy Walks Stream Steep Running Valley Round Fun Sheep Natural Cycling Side Big Racing Forest Dark

Preservation Trees White Climbing Top Broad Escape Track Broad Cycling Path Running

Scene Type Descriptions

32

How can we elicit place descriptions?

• Inductively– Ask people

• Adjective Check List• Category Norms / Basic Levels

– e.g. Mountains, Parks, Beaches, Cities– Attributes, Activities, Parts

– Pick terms from a dictionary and validate– Code unstructured domain knowledge– Data mine web resources

• Deductively– Look at structured semantic resources

• Use a combination of these approaches

33

Data Mining • Analysed Nouns (those used >100 in captions)

– Aim• Identify similar element concepts (equivalence relationships)

– Analyse noun co-occurrence with the landscape adjectives of Craik• Identify related element concepts (associative relationships)

– Analyse noun-noun co-occurrence– Methodology (vector space analysis)

• Identify a list of nouns (inter-annotator agreement)• Form co-occurrence vectors for each noun with nouns or adjectives• Remove insignificant occurrences (tested with chi-squared p>0.01) • Filter out vectors with few occurrences (<3)• Analyse (cosine) similarity between idf-weighted co-occurrence

vectors • Visualise using hierarchical clustering and multi-dimensional

scaling– Throw out largest cluster (noise)

34

Element SimilaritiesHeadland, outcrop, coastline, knoll, shoreline, promontory, outcrops, foreshore

Valley(s), ravine, gorge

Gully, channel(s), cuttingditch(es), holes, pool, fordshaft

hill, bank, slopes, hillside, slope, cliffs, banks, crag, crags, incline, descent, fall, ascent, coombe, gradient

Landforms

land, forest, fields, farmland, moor, moorland, countryside, heathland, grassland, downland

Areas, block, granite, blocks, shed, boulder(s), expanse, slab(s), pieces

Land cover

35

Related ElementsChurch, tower, clock, nave, porch, font, aisle

House, hall, stone, wall, home, manor, brick, grounds, roof, walls, door, structure, floor, parts, window(s), period, glass, mansion, lady, storey, gable(s), architect, doorway, moat, material, façade, wing, doors, bricks, materials rubble, columns, foundations, keys, wings, village, entrance, castle, pub, cottages, inn, avenue

Built Environment

Trees, edge, wood, forest, woodland, plantation, forestry, oak, beech, inclosure, birch, pine, heathland, ponies, conifers, pines, plantations, holly, oaks, lawn, conifer, pony, spruce, sitka, larch, commoners

view, farm, lane, hill, footpath, valley, farmland, bridleway, hedge, heath, hillside, horse, stile, walkers, copse, leaves, chalk, picnic, warren, cyclists, spinney, trails

Nature

railway(s), line, station, train(s), branch, viaduct, cutting, embankment, rail(s), stations, trackbed, goods, passenger(s), terminus, mainline, gauge, platform(s), locomotive, overbridge, sidings, freight, diesel

Road, way, track, junction(s), route(s), section, access, crossing, course, traffic, direction, mile, pass, roundabout, motorway, camera, links, yards, bypass, carriageway, network, lights, pedestrian(s), loop, barrier, flyover

Transport

Bridge, river(s), water(s), bank, burn, stream(s), brook, footbridge, dam, ford, pool, flood, banks, drain, plain, weir, tributary, fish, source, waterfall, bed, meadow(s), levels, gorge, fen, aqueduct, sewage confluence, riverside, reservoir(s), pipe, sluice, salmon, pools, meanders, trout, floods, waterfalls, springs, anglers, channels, table, fishery, outflow, watercourse, wharfe, otter, dike, floodplain, watershed,

Water

36

Flickr

cityart, graffiti, graffitiart, graffitti, grafitti, graphiti, stencil, streetart, urbanart, paint, spray, stickers, street, wall(s), mural

Street Artcity, cityscape, downtown, court, skyline, skyscraper, urban, capitalcity, innercity innerlondon, capitol

countycourthouse, county courthouse, courthouses, cityhall, capitolbuilding, texascountycourthouses, texascourthouses, court,building(s), architecture

alley, billboard, brick(s), castle, cathedral, centre, centreville, chimney, church, clock, county, door, entrance, façade, glass, houses, interior, metro, roof schloss, shop, sign(s), stairs, steps, store, streets, suburb, subway, tower(s), town, underground, ville, wall, window(s), wires

Built environment

butterfly, insect(s), insectes, landscape, landschaft, natur, nature, scenery, bloom, blossom, cherry, flora, flower, flowers, garden, gardens, orchid, plant, plants, rose, wildflowers

barn, bench, countryside, environment, farm, fence, field, flood, fog, grass, meadow, mist, moss, mud, parks, path, pine, rain, rural, storm, weather, wiese, wood, baum, fall, forest, leaf, leaves, tree(s), woodland, woods

Nature

atlantic, beach, coast, ocean, pacific, pier, plage, playa, sand, sea, seagull, seaside, shore, surf, wave(s), water

Sun, bluesky, cloud, clouds, dusk, horizon, sky, sunshine, himmel insel, meer, sonne, strand sonnenuntergang, wasser, wolken

Beaches

Weather

aeroplane , aircraft, airline, airplane, airport, aviation, boeing, flughafen, flugzeug, plane, air, apache, flight, helicopter

Aviation

Work with Christian Matyas of Bamberg University

37

Concepts

Land cover Landforms

Agricultural land crops farmland agriculture

Forest plantation wood woods …

Arable land field fields wheat …

Topographic eminences

Mountains beinn mountain sgurr

....

Hills hill down cnoc

....

Develop Taxonomies

Taxonomy groups

ActualNouns

Conceptcategories

38

Conclusions• Place

– Facets– Importance to geographical semantics

• Eliciting place• Volunteer sources

– Usefulness– Potential biases

• Left open– Infrastructure– Concept detection

39

Acknowledgements

• We would like to gratefully acknowledge contributors to Geograph British Isles, see http://www.geograph.org.uk/credits/2007-02-24, whose work is made available under the following Creative Commons Attribution-ShareAlike 2.5 Licence (http://creativecommons.org/licenses/by-sa/2.5/).

• This research reported in this paper is part of the project TRIPOD supported by the European Commission under contract 045335.