Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Markus NetelerFEM - Centre for Alpine Ecology, Italy

neteler@cealp.it

OSGeo Foundationneteler@osgeo.org

CASCADOSS International Symposium16 June 2008 in Warsaw, Poland

www.cascadoss.eu

Trends in development of Free and Open Source Software for Geomatics (FOSS4G)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Welcome to FOSS4G!

MODIS: Land surface Temperatures (LST)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Proprietary vs Open Source/Free Software

Component Proprietary Open Source

Dedicated Viewer

Cartography GMT, GRASS, JGRASS

Analysis

Web Viewer

WMS, WFS

Spatial Database

Virtual Globe

Metadata Catalog CubeWerx WRS GeoNetwork

ArcReader gvSIG, QGIS, uDig

ArcMap/ArcGIS

ArcINFO/ArcGIS, ERDAS, ENVI

GRASS, OSSIM, SAGA, ILWIS

Handheld ArcPad gvSIG

ArcIMS Openlayers, Mapbuilder, Mapbender

ArcIMS Mapserver, Geoserver, Mapguide OS

Oracle Spatial, ArcSDE

PostGIS, (MySQL, SQLite)

Google Earth, Virtual Earth

OSSIM Planet, NASA WorldWind

For comparison, see http://www.spatialserver.net/osgis/

Welcome to FOSS4G!

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Interoperability Trends (1)● GDAL/OGR started in 1998 with few formats, used by GRASS and few others

● today many raster (>80) and vector (>30) formats, used by many software packages (FOSS4G and proprietary)

● FDO goes open source in 2006

● OGC standards are implemented in various OSGeo software packages

● Example for community driven specification:GeoRSS becomes OGC whitepaper in 2006

GRASS InteragencySteering Commitee

Open GIS Open Geospatial Consortium (OGC) Consortium (OGC)

1990 1992 2004

Open GRASS Foundation (OGF)

1994 1997

Software Using GDAL/OGR3D DEM Viewer from MS MacroSystem Cadcorp SIS: A Windows GIS with a GDAL plugin. CatchmentSIM: A Windows terrain analysis model for hydrologic applications. Demeter: A another OpenGL based terrain engine somewhat similar to VTP. ESRI ArcGIS 9.2+: A popular GIS platform. Feature Data Objects (FDO): Open source spatial data access libraries. flighttrack: GPS track viewing and downloading software for Mac. FME: A GIS translator package includes a GDAL plugin. GdalToTiles: C# Program (open source) for make image tiles for Google Earth with KML Superoverlay. GeoDjango: A framework for building geographic web applications. GeoMatrix Toolkit, and GeoPlayerPro from GeoFusion: 3D visualization. Google Earth: A 3D world viewer. GRASS GIS: A raster/vector open source GIS uses GDAL for raster/vector import and exportgstat: a geostatistical modelling package. gvSIG: Desktop GIS Client. ILWIS: Remote Sensing and GIS Desktop Package. IONIC Red Spider: an OGC Web Services platform includes a GDAL plugin. Leica TITAN: a geospatial data sharing and visualization environment. libral: A raster algebra implementation and an experimental GUI+CLI GIS with Perl and GTK+. MapGuide: Open source web mapping server. UMN MapServer: A popular web mapping application with GDAL support. MapWindow: open source ActiveX control with GIS functionality. MicroImages TNT products: advanced software for geospatial analysisMirone: Matlab based package for geospatial, oceanographic and geophysical analysis of grids OpenEV: An OpenGL/GTK/Python based graphical viewer which exclusively uses GDAL for raster access. OSSIM: Another geospatial viewing and analysis environment which uses GDAL as one of several plugins. the PYXIS innovation: An application for viewing performing analysis and modeling on user's geo-located data.Quantum GIS (QGIS): A cross platform desktop GIS. ScanMagic: Win32 application for visualization, analysis and processing of remote sensing data Scenomics: Software for building terrain databases uses GDAL for projection and data import/export. SpaceEyes3D: 3D visualization software for cartographic data. SpatialAce: A GIS Rapid Application Development environment. StarSpan: raster/vector analysis. TerraGo Technologies: The GeoPDF file format is using GDAL for data import/export. Thuban: A multi-platform interactive geographic data viewer. vGeo: Interactive data fusion and visual analysis tool. Virtual Terrain Project: fostering tools for easy construction of the real world in interactive, 3D digital form.

FOSS4G and Standardization?

● OSGeo's policy is to not standardize

● Some OSGeo members are active in OGC

● ... but see the OGC history:

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Interoperability Trends (2)

Open Standards are preferred in the geospatial industry. Why?

They facilitate

● Interoperability between agencies which essentially increases availability of data (in Europe, see INSPIRE initiative) – easier data sharing

● Cost reduction through easy data exchange and transparent procedures

● Value added in heterogeneous software environments:both proprietary and FOSS4G benefit from that

Question: But Open Data in Europe??

Without vendor lock-in traps, Open Source/Free Software successfullycompetes with proprietary software: Quality and costs decide!

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Portability, Interoperability

GRASS

View InteractTeach

SpatialAnalysisModeling

GeostatisticsPredictive modeling

Databaseengine:Tables,

attributes DBF

vector

rasterExternal

data

Visualize

OGC Web Services

(WMS, WFS, WPS, ...)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Trends in GIS Analysis with FOSS4G● GRASS and others as free GIS backbone

● distributed and parallel computing (GIS number crunching)

● Parallelized numerical library in GRASS

● also (Desktop) GIS wants to be online: OGC Web Processing Service (WPS) is gaining interest (pyWPS, 52°N implementations)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G WebGIS Trends

Web 1.0 (1993): static pages ...

Web(GIS) 1.5 (1996): a bit dynamic pages

● grasslinks (first WPS style system)

Web(GIS) 2.0 (2006): the "I am GIS" Web

● collective intelligence, crowdsourcing, collective mapping (OSM, ...)

● you and me with GPS: mapping party (new fashion in Italy)

● GIS meets social Web

● Sensor web (SOS) and ubiquitous GIS

● NASA WorldWind, Mapserver (p.mapper, ka-maps, ...), OpenLayers, FeatureServer, Geoserver, MapGuide OS, pyWPS...

● it's all about protocols and mashups

Web(GIS) 3.0 (tomorrow?): the semantic Web or the Geoweb (?)

● our online data get (spatially) organized?

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Community Trends

Communication technology evolves...

● '80s: phone, letter, magnetic tape...

● 1989 (?): GRASS FTP server online - the first FOSS4G GIS server in civil Internet?

● 1991: GRASS Mailing lists started: grass-user and grass-dev

● 1991: WWW is invented - First GRASS Web site (US CERL) in 1993 (?)

● 1999: remotesensing.org established, hosting PROJ, GDAL, OSSIM, libgeotiff, ...

● 2000: FreeGIS.org mailing list and portal launched - fundamental for community building

● 200x: Blogs, Wikis, emails (OSGeo lists: > 14000, just in GRASS lists >4000 subscriptions), VoIP, ... Communication is easy!

National FOSS4G corporations:

2000: GRASS Anwendervereinigung e.V. - 2008: becoming OSGeo-DACH?

2007: Associazione italiana per l'informazione geografica libera (GFOSS.it/OSGeo.it)

2007: many OSGeo local chapters

Meritocracy...!

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

GRASS User map

QGIS Usermap

Mapbender User map

http://planet.osgeo.org/

User Communities

Note: not necessarilyrepresentative dueto technical andcultural differences

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Documentation Trends

80's started with tutorials

2002: Open Source GIS: A GRASS GIS Approach - first FOSS4G book ever

2004 (?): first FOSS4G Wikis come up: public participation

2006: OSGeo Education and Curriculum Committee is formed

2007: FOSS4G primer and other book-like initiatives

2008: Many FOSS4G books now available: bookshelf

www.mappinghacks.com

www.spatialguru.com

wiki.osgeo.org/wiki/Library

www.grassbook.org

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G ConferencesInternational conferences:

● Many GRASS conferences since 80s in USA and Europe, drop in 1995, relaunch Trento 2002, then Bangkok 2004

● Mapserver conference 2005

● Lausanne 2006: Merge of Mapserver, GRASS, EOGEO conferences to FOSS4G conference series, with Canada 2007, now South Africa 2008, next Sydney 2009, ...

Annual national conferences:

● in Italy annual conf. since 1999 (first GRASSoriented, now GFOSS.it)

● in Germany annual conf. since 2005 (first Mapserver oriented, now FOSSGIS.de)

● in Spain: gvSIG conf. Since 2005

● ....

FOSS4G formal training● Companies and universities offering courses; online courseware

● ECDL (European Computer Driving License): efforts to prepare a FOSS4G version, especially in the Italian community: http://wiki.gfoss.it/index.php/ECDL_GIS_FOSS

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

GIS development and economical value:GRASS SLOC Analysis

http://www.ohloh.net/projects/3666

Or do this analysis yourself - Download and run: http://www.dwheeler.com/sloccount/

SLOC Totals grouped by progr. language:(Status GRASS 16 April 2007)

ansi-C: 473155 (84.30%)tcl: 44256 (7.88%)sh: 19821 (3.53%)python: 10517 (1.87%)cpp: 10142 (1.81%)perl: 1608 (0.29%)...

Total Physical SourceLines of Code (SLOC) = 561,286Person-Years = 154.05...Total Estimated Cost to Develop = $ 20,810,621

(average salary = $56,286/year, overhead = 2.40)

Generated using David A. Wheeler's 'SLOCCount'

Basic COCOMO model,but slightly different parametershttp://en.wikipedia.org/wiki/COCOMO#Basic_COCOMOGRASS 6.3.cvs, 16 Apr 2007

GRASS 6.4.svn, 23 May 2008

+1 million USDin one year

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Release Management TrendsCode Management

● 80's: GRASS source code is managed manually by the core developers (hey, no civil internet yet!)

● 12/1999: one day before Year 2000, GRASS CVS is started (GDAL, Mapserver as well some years later)

● 2007: integrated OSGeo development platform: SVN, trac, wiki, mailing lists, IRC bots, buildbots

Release Management

● till 2006 (?): separated, ad-hoc releases by the individual teams

● since 2006: attempts to synchronize releases for common feature support etc.

● since 200x: thorough test cycles with alpha, beta, release candidates (milestones in trac)

cross platform tests before releasing (code portable: GNU/Linux, MacOSX, MS-Windows, ...)

aim: deliver end user product (as requested by today large user communities and distro packagers)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Quality Assessment: the GRASS case(In)formal Request For Comments (RFCs)

Implementation of code submission standards for C, Shell scripts, Tcl/Tk, and Python

GRASS Code Quality Control System started in 2006 (currently under rewrite to open up for other OSGeo projects), various scientific papers on this

● supports clone detection, and

● software engineering measures on code style

GRASSQA

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

GRASS Quality Assessment

Improvement of source code base

Ref.: S. Bouktif, G. Antoniol, E. Merlo, and M. Neteler, ICSM 2006. A feedback based quality assessment to support open source software evolution: the GRASS case study. M. Di Penta, M. Neteler, G. Antoniol, E. Merlo, 2005: A Language-Independent SoftwareRenovation Framework. Journal of Systems and Software (JSS), 77(3), pp. 225-240

S. Bouktif, G. Antoniol, M. Neteler and E. Merlo, 2006: A Novel approach to Optimize Clone Refactoring Activity. Proc. GECCO 2006, Seattle, WA, USA, July 8-12 2006

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Changing GRASS source code: what happens? (1/2) tflag->description = _("Print topology information only");

if (G_parser(argc,argv)) exit(EXIT_FAILURE);

/* open input vector */ if ((mapset = G_find_vector2 (in_opt->answer, "")) == NULL) { G_fatal_error (_("Could not find input map <%s>"), in_opt->answer); }

Developer (e.g., Italy) changes and enters:

svn ci -m”i18N macro added” main.c

SVNsourcecode

repository

USA (OSGeo)

Code differences email isauto-generated and sent to“grass-commit” mailing list

USA

Email notification triggersupdated of GRASS QualityAssessment System

Canada

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Changing GRASS source code: what happens? (2/2)

Clone detection is run as well as other quality measures, results sent out

Email notification triggersupdate of GRASS QualityAssessment System

Canada

USA – worldwide subscribers

Email about code quality is sent to “grass-qa” mailing list

CIA open source monitor receivessimplified QA message

USA?

SIGQ & CIA-IRC robot feed

#grass IRC channel on freenode.net

Total timeelapsed for QA: < 60sec

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

OSGeo: Incubator(following Apache Foundation model) http://www.osgeo.org/incubator

The OSGeo Incubator is run by the Incubation Committee. It is responsible for:

(1) Preparing and recommending incubation process and criteria to the board.

(2) Reviewing applications for incubation and making recommendations to the board on projects that should be admitted into incubation.

(3) Appointing a mentors for incubation projects.

(4) Reviewing progress of projects against incubation requirements, and recommending projects for full project status to the board, when deemed appropriate.

(5) Reporting on the status of projects in incubation and the incubation process to the board periodically (via Chair).

(6) Suggest to graduate or reject

Incubation Process Procedures● Incubator Application Questionnaire ● Incubation Process (code vetting, intellectual property etc.)● Project Evaluation Criteria ● Incubation Mentor Guidelines ● Code Provenance Review Process ● Project Status monitoring (Wiki)● Project Graduation Checklist

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

http://code.google.com/p/google-summer-of-code-2007-osgeo/

http://wiki.osgeo.org/wiki/Google_Summer_of_Code_2008

Sponsored development

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

FOSS4G Business Trends● OSGeo Service Providers directory

● Voluntary registration (~100 registered)

● 30 companies with >= 10 employees

● FOSS4G business model: service oriented and customization

● Value is the solution, not the technology

● Solutions drive the technology - not the other way around

● Commercial (don't mix this up!) Proprietary GI solutions versus FOSS4G

● FOSS4G can be commercial used!

● Ongoing discussion... Provide commercial training directly as OSGeo brand?

1 Israel

1 India 2 Thailand 2 Japan

1 South Africa

4 Australia 4 New Zealand

3 Brazil

9 Canada30 USA

Europe: 57 1 Hungary 1 Norway 2 Belgium 2 Finland 2 Ireland 2 Poland 2 Sweden 3 Netherlands 3 Switzerland 4 Denmark 4 Spain 5 Italy 6 England 6 France14 Germany

OSGeo ServiceProviders (6/2008)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Free Software

Proprietary Software

Tota

l Cos

t of

Ow

ners

hip

(TC

O)

Time

B. Reiter 2004after D. Wheeler 2004

Software operating costs (for customer)

Overhead due to training and rewrite of missing

software

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

Conclusions

FOSS4G delivers solutions:● Software at end user level, even with multi-language user interfaces● Documentation and training

What's inside:● Communities large and yet growing● Quality control procedures established● Organizations at national and international level (OSGeo)

The impact of FOSS4G:● Geospatial industry starts to adopt concepts

and even directly FOSS4G software (in case of permissive licenses)

Mark

us

Nete

ler

CA

SC

AD

OSS.e

u M

eeti

ng

16

June 2

008

, W

ars

aw

License of this document

This work is licensed under a Creative Commons License.http://creativecommons.org/licenses/by-sa/2.5/deed.en

“Trends in development of Free and Open Source Software for Geomatics (FOSS4G)”, © 2008 Markus Neteler, Italy http://www.grassbook.org/neteler/conferences.html [ OpenDocument file available upon request: neteler at cealp it ]

License details: Attribution-ShareAlike 2.5

You are free:- to copy, distribute, display, and perform the work,- to make derivative works,- to make commercial use of the work,under the following conditions:

Attribution. You must give the original author credit.Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one.

For any reuse or distribution, you must make clear to others the license terms of this work.Any of these conditions can be waived if you get permission from the copyright holder.Your fair use and other rights are in no way affected by the above.