wind force & direction from ships captains logs
DESCRIPTION
CLIWOC 1662 - 1855 track & climatological records for British, Dutch, French & Spanish ships collected by respective maritime institutionsTRANSCRIPT
Andrew ZolnaiCambridge UKwww.zolnai.ca
Shipping and weather data CLIWOC ships captains logs 1662-1855Data points for climate research
Image: Royal Museum Greenwich
Climatological data
• Ships captains logs data from 1662 to 1855• Ships and routing details and location• Weather recording of over 120 parameters• Release 1.x (2004) and Release 2.x (2007)
Shipping data• First added to Esri file geodatabse – The complete (weather and location) dataset has
almost 290,000 points, and is almost 250Mb database– The locations were extracted into a geodatabase (70
Mb), then into a compressed file geodatabase (10 Mb)• Posted for ArcGIS Explorer upload on
original website 1750 – 1850 data from Release 1.5– George Dailey further developed this for education– and posted it on ESRI's Education Blog
• And map package shares Release 2.0– layer files showing them in 25-year time-slices– and for Captains Cook and de la Perouse
Time-enable shipping
• Daniel Schober reposted this as map package– Here layers are time-enabled through the field "Time"
which is the UTC time in the format YYYYMMDD• Re-reposted here and as a YouTube video– data are no longer cast into arbitrarily time-sliced feature-
classes– Time Slider allows to step through all 252,917 points from
1662 to 1855– it beautifully depicts the various colonial activities driven
by, for example, spice trade in the East and West Indies for the British, Dutch, French and Spanish
– in other words Time Slider greatly facilitates the investigation of this large data set
Static vs. dynamic web maps• Amazon Web Service ‘small' elastic service• Mapcentia Geocloud2 stack• Leaflet frontend w Stamen toner, postGIS backend• unlimited data, no performance hits• CLIWOC all nations by year (1750-1775)
• Use time stamps to aggregate layers• Posted as ArcGIS Online map service• But stock server has performance issues• So Filter data by nationality to limit fetches
Dynamic maps II
• Workflow– of real interest was to compare nations shipping over
time, rather then animation per se– thus create a pair of layers to step through
• Context– add Rumsey 1812 map for backdrop from ArcGIS Online– add current wind data from NOAA METAR with
continents mask to show offshore wind patterns• Posted as CLIWOC Compare – move slider to 1802-1812 to view with Rumsey backdrop– turn on Continent Mask + NOAA METAR for current wind
Restore climate data
• CLIWOC posted all shipboard data it had – from British, Dutch, French and Spanish sources– with look-up tables for each different source
• Merged wind force and direction data– table joins with lookup tables gives direction and
speed data consistent numeric values, but…– wind (class, Beaufort and speed) keeps 277,581 points– wind direction however boosts count to 471,214
points– and joining wind direction and force to graduate and
orient the symbols drops the count slightly to 431,651
Lessons learned
• Join and aggregate data in a lossless manner• Use File Geodatabases to allow mega datasets• Filter data by parameters of interest:– nationalities in pairs to compare– step-thru animation to see evolution
• Share on-line to foster re-use & collaboration:– feature template CLIWOCwinds– web map CLIWOC Wind Force+Direction– and layer package WindDutch1812
Take-aways
• Very rich dataset from 160 - 350 years ago
• Lots of data strung out along ship tracks
• Navies had standards • But issues w data
recording and capture
• Compare Old Weather NOAA ICOADS, &c.
• Many visualisations • Few share &/or re-use
• Foster online collaboration
• Further climate research