figure 1.doc

Natural Resources Canada GeoConnections

Report on Google KML Interoperability for CGDIGovernment Partners

Version 1.0

Submitted byCAE Professional Services, Canada

300–1135 Innovation DriveOttawa, Ontario

Canada

Proprietary ÓCopyright. Her Majesty the Queen in Right of Canada, , 2023

Natural Resources Canada GeoConnections Version 1.0Report on Google KML Interoperability for CGDI Government Partners

9 April 2023

Table of Contents

1 Introduction............................................................................................................41.1 Purpose...........................................................................................................41.2 Scope..............................................................................................................41.3 References......................................................................................................4

2 Background............................................................................................................63 Architecture............................................................................................................6

3.1 Google Maps, Google Earth Architecture........................................................93.1.1 KML..........................................................................................................93.1.2 Google Maps and KML..........................................................................103.1.3 Google Earth and KML..........................................................................10

3.2 GeoRSS Architecture....................................................................................104 Government use of Google KML..........................................................................115 CGDI and KML.....................................................................................................13

5.1 CGDI Standards Comparison and KML........................................................135.2 Additional Comments....................................................................................14

6 Future Considerations..........................................................................................157 Next Steps............................................................................................................15



9 April 2023

SummarySince 2002, Google’s Keyhole Markup Language (KML) has become a new standard in the encoding of data and services and as a distributable mechanism for public map composition. This report provides a high-level review of Google KML and its relevance to the Canadian Geospatial Data Infrastructure (CGDI) partners and to GeoConnections.

KML and its primary client applications, Google Maps and Google Earth, have leveraged the existing services architecture paradigm embodied in the CGDI Architecture Description version 1.0. With much of the geospatial community aligning with the Open Geospatial Consortium (OGC) standards over the past five years, Google has created a format that can encode vector data, imagery data, Web services (WMS), style layer descriptors and external references all in one file. KML is converging capabilities just as the geospatial community was isolating components.

The net result of this divergent approach was that Google’s applications and data format were easily adopted by the public to view and create data. The OGC/CGDI approach enabled large data providers and application developers to implement services for base data required for government as well as public use.

With GeoConnections and CGDI partners having established the technology standards and core services of the CGDI, a new focus on users and communities is turning attention to KML and its eventual endorsement by CGDI.

KML is significant for the following reasons:

o The architecture required to support KML will have performance implications for data service providers;

o The KML encoding of vector data is different from that endorsed and nurtured by CGDI in Geography Markup Language;

o KML may impact future standards development where the standards are encapsulated within KML to some degree. These current standards include Web Map Service, Web Feature Service, Geography Markup Language, and Web Map Context Document;

o Google may impact standards or implementations by not working in catalogues or registries for services, and in a lack of metadata within KML;

o ESRI, MapInfo, Autodesk and others are beginning to consume and publish KML format; and

o CGDI partners are already establishing services using KML to service the Google user base.

In understanding how KML is currently being used, it is hoped that the hard-won lessons in the first five years of the CGDI will assist in the smooth transition of KML, together with its users and developer base, into the core of the CGDI.



9 April 2023

1 Introduction

In April 2007, at a meeting of the Open Geospatial Consortium (OGC) in Ottawa,1 Google confirmed its intention to develop its KML data encoding standard for adoption at version KML 3.0. As a result, what was once considered an open yet proprietary encoding standard will become fully open and managed by the interoperability community. With the adoption of KML as a standard, the endorsed format will make KML a viable option for data and service providers that prioritize interoperability over proprietary formats. It is therefore important to consider the implications of KML within the current Canadian Geospatial Data Infrastructure (CGDI).

1.1 Purpose

This document outlines the potential considerations on the use of KML within the CGDI. It has been prepared for senior technical authorities and senior management within organizations providing services within the CGDI.

1.2 Scope

This document simply provides an overview of the implications of using KML as a data encoding standard. It does not offer implementation recommendations related to unapproved CGDI standards. The scope of the document is confined to KML as an encoding standard for interoperability and to a description of Google Earth as an application platform for visualization with respect to its consumption of KML and KMZ (the zip version of KML) and OGC Web services, namely, Web Map Service (WMS).

1.3 References

http://bbs.keyhole.com/ubb/showflat.php/Cat/0/Number/735788/an/0/page/1#735788

http://code.google.com/apis/kml/schema/kml21.xsd

http://msdn2.microsoft.com/en-us/library/ms978603.aspx

http://www.gartner.com/DisplayDocument?doc_cd=119477

http://www.w3.org/2006/02/WS-ECA.pdf

http://www.allpointsblog.com/archives/2561-OGC-On-Google-KML-Submission.html

http://www.opengeospatial.org/standards/wfs

http://www.opengeospatial.org/standards/wms

http://www.opengeospatial.org/standards/wmc

http://www.geoconnections.org/developersCorner/devCorner_devNetwork/keyDocs/WFS_Considerations_for_CGDI_Government_Partners_v1.0.pdf

http://code.google.com/apis/kml/documentation/kml_tags_21.html#coordinates

http://volcano.wr.usgs.gov/cap/rss/vhpcaprss.xml

http://www.rajsingh.org/blog/?p=24

http://www.oasis-open.org/committees/download.php/19679/soa-rm-cs.pdf

http://www.adobe.com/enterprise/pdfs/Services_Oriented_Architecture_from_Adobe.pdf

1 This meeting was held April 16–20, 2007 and sponsored by ESRI Canada and GeoConnections.


http://www.adobe.com/enterprise/pdfs/Services_Oriented_Architecture_from_Adobe.pdf

http://www.oasis-open.org/committees/download.php/19679/soa-rm-cs.pdf

http://www.rajsingh.org/blog/?p=24

http://volcano.wr.usgs.gov/cap/rss/vhpcaprss.xml

http://code.google.com/apis/kml/documentation/kml_tags_21.html#coordinates



http://www.opengeospatial.org/standards/wmc

http://www.opengeospatial.org/standards/wms

http://www.opengeospatial.org/standards/wfs

http://www.allpointsblog.com/archives/2561-OGC-On-Google-KML-Submission.html

http://www.w3.org/2006/02/WS-ECA.pdf

http://www.gartner.com/DisplayDocument?doc_cd=119477

http://msdn2.microsoft.com/en-us/library/ms978603.aspx

http://code.google.com/apis/kml/schema/kml21.xsd

http://bbs.keyhole.com/ubb/showflat.php/Cat/0/Number/735788/an/0/page/1#735788


9 April 2023

http://www.geoconnections.org/publications/tvip/arch_E/CGDI_Architecture_final_E.pdf

http://www.tbs-sct.gc.ca/fap-paf/documents/iteration/iteration_e.asp

http://www.geobase.ca/geobase/en/data/nrn/demo.html;jsessionid=10966A225D0484AEBE09F31A5B1F4613

http://www.google.com/apis/maps/documentation/#XML_Overlays

http://www.google.com/apis/maps/documentation/kmlOverlay.html

http://www.google.com/apis/maps/documentation/geoRSSOverlay.html

http://www.ogleearth.com/2005/08/the_atlas_of_ca_1.html


http://www.ogleearth.com/2005/08/the_atlas_of_ca_1.html

http://www.google.com/apis/maps/documentation/geoRSSOverlay.html

http://www.google.com/apis/maps/documentation/kmlOverlay.html

http://www.google.com/apis/maps/documentation/#XML_Overlays



http://www.tbs-sct.gc.ca/fap-paf/documents/iteration/iteration_e.asp

http://www.geoconnections.org/publications/tvip/arch_E/CGDI_Architecture_final_E.pdf


9 April 2023

2 Background

In 2002, Google bought Keyhole, a Spatial Data viewing company. In doing so, it introduced the greater computer public to what we now know as Google Earth, Google Maps and a new interchange format, KML. Keyhole Markup Language is a simple XML-based scripting language that is easily readable and allows any data to become a viewable source within Google Maps or Google Earth.

Since its introduction, many users have used the KML format to create their own data for sharing. The pervasiveness of KML in a World Wide Web community has made it a de-facto data interchange standard, as well as a map definition format. Leading GIS companies have incorporated KML map context data source for enterprise GIS applications. ESRI, MapInfo, Autodesk, and other GIS software companies are using KML data services and also allowing users to publish enterprise content in KML. In addition, translation support for traditional GIS formats such as “shape” and “MID/MIF” to KML is available from geospatial industries such as Safe Software.

The focus of OGC on standards originating outside of itself speaks to the true convergence of data content and service capabilities, where location is now part of the core data schema of the new emergent Web 2.0. Through CGDI tracking and understanding these new emergent standards, CGDI partners can adapt and meet the challenge.

3 Architecture

The CGDI has facilitated technical maturity of partner service providers, data providers and application developers and is now focusing more on the service-oriented architecture integration required to align with the Government of Canada’s Federated Architecture, Iteration One.

The Publish/Find/Bind model (Figure 1) articulated in the CGDI Architecture Description Version 2.0 ensures all services meet the technical architecture requirements of the CGDI.

Figure 1: CGDI Publish/Find/Bind Model



9 April 2023

Within the context of a Service-Orientated Architecture (SOA), each service has a request and a response between a consumer and a provider (Figure 2)

Figure 2: Service Consumer, Service Provider

The traceability of finding and binding can be seen in the following diagram (Figure 3). The Service Consumer finds the service through Advertising/Discovery and then binds the service through the Service Client.

Figure 3: Service Consumer Access to Services

Within the SOA, the Service Client controls the timing and access to services. The SOA model is different from the Event-Driven Architecture (EDA) articulated in the Publish/Subscribe model (Figure 4).



9 April 2023

Figure 4: Publish/Subscribe simple model

The resulting Web services publish/subscribe model is executed using three different types: List-Based, Broadcast-Based, and Content-Based.

List-Based Publish/Subscribe is where a subscriber service registers with a publish service. When the content changes, the publish service issues notification to all subscriber services.

Broadcast-Based Publish/Subscribe is where the publisher sends out a message on a network and a listener service determines if the message is of importance to the subscriber and processes it. Sometimes publishers send out a request for response to listeners to determine if they are interested in the publishers’ topic. If the listeners respond, the publisher then notifies the respondents and the subscriber processes the message.

Both Broadcast-Based Publish/Subscribe implementations and List-Based Publish/Subscribe implementations can be broadly categorized as topic-based, since they both use predefined subjects as many-to-many channels. Publish/Subscribe implementations have recently evolved to include a new form — Content-Based Publish/Subscribe. The difference between topic-based and content-based approaches is as follows:

In a topic-based system, processes exchange information through a set of predefined subjects (topics) that represent many-to-many distinct (and fixed) logical channels. Content-based systems are more flexible, as subscriptions are related to specific information content, with the result that each combination of information items can be seen as a single dynamic logical channel. This exponential enlargement of potential logical channels has changed the way of implementing a publish/subscribe system.

These two architectures can exist together within a federated environment. Google Maps and Google Earth using KML/KMZ use portions of both architectures to accomplish the data distribution and map composition capabilities that millions of users access through their applications.



9 April 2023

1.4 Google Maps, Google Earth Architecture

The architecture that allows Google mapping applications, Maps and Earth, to read data sources involves a service-oriented architecture (SOA) approach, in which discrete service URIs or files are added, and the application or user determines the service reacquisition rate. This approach means that, while the service may not have new content, the application makes a request anyway to see if updating is needed.

In addition, Google Maps provides a service-client capability to requery and to cache map layer services (KML/KMZ) within a user-defined time. This timed service update mimics EDA capabilities, but not from the data provider service to the service client as prescribed in the Publish/Subscribe model.

The base map images are acquired from a central server and cached locally while the thematic data is acquired using the KML encoding.

1.4.1 KML

Keyhole Markup Language (KML) serves as an encoding language for the access, organization and symbolization of data available to Google map applications.

Figure 5: KML Schema Schematic

KML is simple and allows data providers a simple way to structure data for visualization. In addition, KML allows for the temporal attributes of objects, linking to external href’s and determining symbolization.

The architecture of KML as an encoding language mirrors what is currently being done with Geography Markup Language2 files but in a more simplified and pervasive

2 GML or Geography Markup Language is a syntax for encoding geographic content in XML.



9 April 2023

manner. KML’s power comes from the core key elements that users want to use rather than those defined by industry.

In addition to KML, there is a compression specification that creates a smaller binary version, KMZ, which allows larger amounts of data to be consumed by applications without inhibiting bandwidth. KMZ operates transparently to the user and therefore is the same as KML, except it is not editable in the zipped form. Most users want access to data and do not worry about the format. Since data providers want to reduce bandwidth per request, KMZ is a common distributed file format.

An example of KML/KMZ file distribution is the GeoBase National Road Network Version 2, Demo Dataset. This data set is available in “Shape,” “GML” and “KML” formats. To illustrate their difference in size, the zipped GML file is twice as large as the binary KML file (KMZ). The difference in practical terms is that, since the client applications can read the zipped KML file, they do not have to manage very large files: a 5.2Mb zipped GML file unzips to 95Mb for use.

1.4.2 Google Maps and KML

Google Maps can use KML files to create map layers. With the KML Overlay in the April 2006 update to the Google Maps API, client or third party resources can be submitted to the Google Maps server and a composite map is returned to the client URL. This means that all information is submitted to the Google Maps server for each client. Each request also requires an additional call to the KML resource, even though no data has been updated.

1.4.3 Google Earth and KML

Google Earth’s use of KML is different due to the caching and timed update components of the thick client application. Upon adding a KML resource to a map, Google Earth caches the file on the local machine. If the KML file is URL based and not a local file, then the user can set an update timing where the application re-acquires the KML resource and replaces the cached file. Since most KML data is static, the timed updates are unnecessary service requests and responses to the data provider.

Cached data for Google Earth is persistent unless updated. The base imagery in Google Earth is managed within a separate cache and, therefore, managed and updated by a different mechanism than thematic maps.

In addition, some KML users are also embedding Web services within KML. Thus, for example, Web Map Service is a common image service used within KML to overlay third party image data in Google Earth.

1.5 GeoRSS Architecture

GeoRSS or RSS represent a new paradigm for net-centric applications. These services are very light, meaning they are very simple and normally do not contain images or complex encodings that slow transmission of the content. Since RSS was developed for quick access to rapidly changing data, the representation of the content is an action of the subscribing application and is therefore not controlled by the data source publisher.

GeoRSS is the first spatially encoded service that follows the publish/subscribe architecture. More specifically, GeoRSS enables the content-based publish/subscribe model more effectively than RSS, simply because spatial context can now be used to determine more appropriate publish content for users.



9 April 2023

Google Maps has an API for GeoRSS display (http://maps.google.com/maps?q=http://slashgeo.org/index.rss).

Figure 6: Slashdot GeoRSS in Google Maps

4 Government use of Google KML

At the present time, KML is not yet an endorsed standard within the Canadian Geospatial Data Infrastructure (CGDI). Nevertheless, there is considerable use of KML within the user community and industry. The CGDI has enabled users and industry to access and share considerable amounts of base data layers such as road networks, digital terrain models and base maps. What is now being seen is the emergence of social thematic data directed to the mass user. Since Google and Microsoft Web mapping software are well established in the mass user domain, it is only natural that the data sources are aligned to these products.

Government has been keenly observing this trend, and some departments are providing services on the yet to be endorsed standards as a way to better understand the technicalities of providing this service and to determine the use of the new services by end users. The ultimate goal of all government data providers is for more Canadians to use geospatial data services.

While few departments are using KML to release government data, Natural Resources Canada and Agriculture and Agri-Food Canada have been testing the use of KML since Google Earth was released.


http://maps.google.com/maps?q=http://slashgeo.org/index.rss

http://maps.google.com/maps?q=http://slashgeo.org/index.rss


9 April 2023

Figure 7: National Road Network V2, Demo Data KML

Figure 8: Canadian Soils Classification KML Image Overlay from Geogratis.ca



9 April 2023

At the time this report was prepared, GeoRSS was not an OGC or CGDI standard. Nevertheless, this application is creating excitement within the geospatial community because of its simple implementation and its great potential to make location-based data reusable and pervasive in a global manner. Currently, GeoRSS is undergoing experimentation and implementation in the areas of public safety and security alerting, as well as in public warnings of natural hazards. Potentially, it would replace the Event Registry Service under investigation by the CGDI.

5 CGDI and KML

1.6 CGDI Standards Comparison and KML

The CGDI has already endorsed many standards that address the data access and portrayal or display elements found in the KML structure. The following provides a brief comparison of the current relevant CGDI standards and the KML structure.

Web Map Service/KML

A Web Map Service (WMS) standard produces dynamically maps (generally rendered in a pictorial format such as PNG, GIF or JPEG) of spatially referenced geospatial data. Geospatial data is defined as “data with implicit or explicit reference to a location relative to the Earth.”

The standard defines an interface for querying and accessing map layers from a mapping server. Clients and servers that adhere to the WMS standard can communicate with one another regardless of their underlying implementation. Software complying with the standard enables automatic overlay, in ordinary web browsers, of map images obtained from multiple map servers, regardless of map scale, projection, earth-coordinate system, storage format, or vendor solution. The WMS standard supports various operations, in particular the GetFeatureInfo operation that retrieves discrete feature information through a call to the data provider’s service.

KML can encode images to map coordinates. KML has been shown to overlay WMS map images. In KML 2.1, a SuperRegion tag allows for nested region specific image services and for service changes upon zooming in and out. Since KML allows for the encoding of features instead of images within the KML file, there may be no need to send subsequent service requests to satisfy a user’s interrogation of the data.

Web Feature Service/KML

Web Feature Services (WFS) allow for the requesting of feature data with the service response being GML encoded. WFS cannot allow only for query and response, but they can allow for features to be created or updated within the source data repository.

KML, as a service, does not allow for the passing of feature information back to the source.

Geography Markup Language/KML

Geography Markup Language (GML) began as a simple encoding of geometry and data and has grown significantly through version 3.1. GML files are verbose due to the schema encoding within the file. In an effort to make GML more useable by a wider client application base, the Simple Feature profile of GML was made available in 2006. GML-SF provides for 2D features and is being more widely adopted by government data providers as an interchange format along with Shapefiles.

GML compression in binary XML format has proven that the bandwidth required for the exchange of GML data can be significantly reduced. Client applications that read GML



9 April 2023

may not at this time be capable to consume the BXML format. As a result, data providers are encoding to GML-SF and serving GML as files instead of through WFS interfaces.

KML does not cover many of the encoding capabilities of GML 3.1. KML was originally more common in the XML readable form, but now the KMZ is more common not human-readable KML, the content data is minimally exposed.

Web Map Context/KML

Web Map Context (WMC) documents provide for the saving map composition information. The WMC document has service link information, style information and spatial reference information for the users map, as well as a metadata section in which the user can describe the map. WMC documents are most like KML files since they consist of a list of resources with visualization encoding. The current WMC version, 1.0.0, uses WMS as binding layers in the map composition encompassed within. While the WMC schema can be extended to encompass other layer formats, the current standard encourages implementers to take the extended types and render them as WMS layers. The OWS Context specification, currently in development at the OGC, allows for other content types — WFS, WCS and static GML documents — but does not explicitly support annotations. There have been requests for the Context RWG to support an even wider range of content types (GeoRSS feeds, annotations, tiled map layers, etc.), and the convergence of KML and OWS Context is likely to be the focus of experimentation in the OGC’s OWS5 testbed, scheduled to take place in summer 2007.

KML files allow for layers to be images or 2D or 3D geometric objects. This function makes KML a more universal exchange format and holistic map composition document.

Considerations

Even though KML has the core capabilities of WMS, WFS, GML and WMC, it lacks the key flexibility of reference coordinate systems. KML locations are a standard latitude/longitude WGS84 with a simple cylindrical projection (Plate Carrée). For client applications with other map projections, as is the case with Canada and other northern countries, KML datasets will need to be transformed on the client machine instead of the server, as is done when using WMS and WFS.

1.7 Additional Comments

KML does use elements of the schema for contact information under what would be considered a very minimum Dublin Core schema. To meet CGDI metadata specifications, the encoding of references to a metadata document is possible within KML.

To ensure consistent adoption of KML, a profile should be adopted so that specific general elements such as <document> can be encoded consistently for use by custom applications.

Compressed KML files (KMZ) represent a lower bandwidth cost over a network. But this performance advantage needs to be taken into consideration when KML is accessed as a network resource and has a timed update greater than the potential update of the data source. Reducing the size of delivery only to reacquire the resource may potentially create more network traffic for high-frequency data providers.

6 Future Considerations



9 April 2023

As KML 3.0 is put through the works of OGC, the real possibility arises that KML will change. This new emergent standard points to several key behaviours along the future path of the CGDI:

New media convergence will produce more holistic specifications, such as KML, instead of many small ones as we now see with WMS, WFS and WMC.

Although Service Oriented Architectures will continue to extend and support a net-centric approach to inter-enterprise collaboration, the greatest growth will be in Event Driven Architectures, specifically publish/subscribe services and the ubiquitous Web and Web 2.0.

Government data providers will in the short term choose between keeping to the pure standards as they are endorsed by CGDI or adopting new service protocols to reach a broader user base. The CGDI architecture needs to be responsive to emerging standards.

Although many GIS vendors have applied KML to their data source capabilities, similar to GML, the version of KML and the limited implementations do not fully take advantage of the source data.

Digital rights and data representation need to be fully thought out because of the rapid dissemination of data.

Because adoption of KML will increase the number of peak users, additional resources may need to be applied by data providers to ensure availability.

Industry and private sector organizations need to be more closely watched by government and standards bodies to detect new emergent standards, such as social network localization, and to understand the implications of new standards in order to align them with the existing technology.

KML represents a new way in which CGDI partners need to extend current services. In doing so, CGDI partners will begin to rationalize their contribution to the ever more complex data and service interoperability architecture. This response may over time see different CGDI partners providing and consuming services through different mechanisms, protocols and schemas, with some concentrating on core base data and others embracing the event-driven architecture model. Both approaches need to exist.

7 Next Steps

CGDI partners need to continually try new service mechanisms to meet clients’ changing technological and data requirements. KML is a good candidate to extend CGDI services, although it is not an endorsed specification.

GeoConnections and the CGDI need to test specific implementations of KML, both as a map composition and data encoding standard, as well as a container for aggregate services such as WMS. By testing these implementations, the CGDI and GeoConnections can support any future endorsement of KML.

KML needs to be followed closely by GeoConnections as KML 3 passes through the OGC process, with careful consideration not to pull it apart into its constituent capabilities. One of the reasons KML is being widely adopted is because of its flexibility and its ability to encode vector and raster resources and styles.

Finally, GeoConnections should actively document a CGDI-accepted implementation strategy for KML within CGDI partners before KML 3 is endorsed. Many implementations of KML-embodied services and files on the Web have been made



9 April 2023

with little care given to the extended service considerations that make current CGDI services successful.


figure 1.doc

Documents