copernicus global land operations · 2019-12-04 · copernicus global land operations – lot 1...

Copernicus Global Land Operations – Lot 1 Date Issued: 29.08.2019 Issue: I1.00

Copernicus Global Land Operations

“Vegetation and Energy” ”CGLOPS-1”

Framework Service Contract N° 199494 (JRC)

VALIDATION REPORT

MODERATE DYNAMIC LAND COVER

COLLECTION 100M

VERSION 2.0

Issue I1.00

Organization name of lead contractor for this deliverable: WUR

Book Captain: Nandika Tsendbazar (WUR)

Contributing Authors: Martin Herold (WUR)

Agnieszka Tarko (WUR)

Linlin Li (WUR)

Myroslava Lesiv (IIASA)

Steffen Fritz (IIASA)


Document-No. CGLOPS1_VR_LC100m-V2.0 © C-GLOPS Lot1 consortium

Issue: I1.00 Date: 29.08.2019 Page: 2 of 53

Dissemination Level PU Public X

PP Restricted to other programme participants (including the Commission Services)

RE Restricted to a group specified by the consortium (including the Commission Services)

CO Confidential, only for members of the consortium (including the Commission Services)




Document Release Sheet

Book captain: Nandika Tsendbazar Sign Date 29.08.2019

Approval: Roselyne Lacaze Sign Date 28.10.2019

Endorsement: Michael Cherlet Sign Date

Distribution: Public




Change Record

Issue/Rev Date Page(s) Description of Change Release

29.08.2019 All First issue I2.00




TABLE OF CONTENTS

Executive summary .................................................................................................................. 10

1 Background of the document ............................................................................................. 12

1.1 Scope and Objectives............................................................................................................. 12

1.2 Content of the document....................................................................................................... 12

1.3 Related documents ............................................................................................................... 12

1.3.1 Applicable documents ................................................................................................................................ 12

1.3.2 Input ............................................................................................................................................................ 12

1.3.3 Output ......................................................................................................................................................... 13

2 Review of Users Requirements ........................................................................................... 14

3 Quality Assessment Method .............................................................................................. 19

3.1 Overvall procedure ................................................................................................................ 19

3.2 CGLS_LC100m 2015 products ................................................................................................. 19

3.3 In-situ Reference Products ..................................................................................................... 22

3.4 Map accuracy assessment...................................................................................................... 28

3.5 Comparison with other global land cover maps ...................................................................... 30

3.5.1 Other global land cover products used for comparison ............................................................................. 30

3.5.2 Qualitative comparison ............................................................................................................................... 32

3.5.3 Quantitative comparison ............................................................................................................................ 32

4 Results .............................................................................................................................. 34

4.1 Accuracy of the CGLS_LC100m V2.0 product ........................................................................... 34

4.2 Comparison with other global land cover maps ...................................................................... 42

4.2.1 Qualitative comparison ............................................................................................................................... 42

4.2.2 Quantitative comparison ............................................................................................................................ 47

5 Conclusions ....................................................................................................................... 50

6 Recommendations ............................................................................................................. 51

7 References ........................................................................................................................ 52




List of Figures

Figure 1: The CGLS Dynamic Land Cover Map V2.0 at 100 m (detailed legend in Table 3). ........ 21

Figure 2: The fractional land cover layers for the 10 base land cover classes of the CGLS Dynamic

Land Cover V2.0 product at 100 m (shown as a collage on global scale). .............................. 22

Figure 3: Continents used for validation dataset collection. ........................................................... 23

Figure 4: Selected validation sites and the stratification by Olofsson et al 2012: ‘p’ before the strata

names denote populated part of climate zone. ....................................................................... 24

Figure 5: Regional Experts and their representing regions. ........................................................... 25

Figure 6: Validation data collection process. ................................................................................. 26

Figure 7: A screenshot of Geo-Wiki based land cover validation data collection interface. ............ 27

Figure 8: A screenshot of an example sample interpretation over the 10mx10m sub-pixels included

into a PROBA-V pixel (100mx100m). ..................................................................................... 28

Figure 9: Legend of the Globeland30 2010 map. .......................................................................... 32

Figure 10: Comparison of three maps in Madagascar (left) and Northern Amazonia, South America

(right) (legends are shown in Table 3, Table 4 and Figure 9 for the CGLS_LC100m, LC-CCI,

Globeland30 map, respectively). ............................................................................................ 43

Figure 11: Comparison of three maps in Rondonia and Mato Grosso, Brazil (left) and Uzbekistan



Figure 12: Comparison of three maps in Kalimantan and Sulawesi, Indonesia (left), and Cuba



Figure 13: Examples of artefacts and inconsistencies (related to unknown forest type) in the

CGLS_LC100 V2 discrete map .............................................................................................. 46




List of Tables

Table 1: Usefulness of information on LC and LC change processes for different international

actions and programmes. ....................................................................................................... 15

Table 2: List of land cover classes requested by users. ................................................................ 15

Table 3: Discrete classification coding of the CGLS dynamic land cover map V2.0. ...................... 20

Table 4: Legend of the global CCI-LC maps, based on LCCS....................................................... 30

Table 5: Legend definition of the Globeland 30 map (http://www.globeland30.org/) ...................... 31

Table 6: Confusion matrix (%) for the discrete CGLS_LC100m V2.0 Level 1 map at global scale,

corrected by sample inclusion probabilities. ........................................................................... 34

Table 7: Overall accuracy of the discrete CGLS_LC100m V2.0 Level 1 map per continent. .......... 35

Table 8: Confusion matrix (%) for the discrete CGLS_LC100m V2.0 Level 1 map over Africa. ..... 35

Table 9: Confusion matrix for the discrete CGLS_LC100m V2.0 Level 1 map over Asia. .............. 36

Table 10: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Northern

Eurasia. ................................................................................................................................. 36

Table 11: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Europe. .... 37

Table 12: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over North

America. ................................................................................................................................ 37

Table 13: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Oceania and

Australia. ................................................................................................................................ 38

Table 14: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over South

America. ................................................................................................................................ 38

Table 15: Confusion matrix for the discrete CGLS_LC100m V2.0 Level 2 map at global scale,

corrected by sample inclusion probaiblities. ........................................................................... 39

Table 16: Overall accuracy of the discrete CGLS_LC100 V2.0 Level 2 map per continent. ........... 40

Table 17: Accuracy of the fractional land cover layers at global scale. .......................................... 40

Table 18: Accuracy of the fractional land cover layers at continental scale. .................................. 41

Table 19: Agreement of combined reference points with four previous global land cover maps and

the CGLS_LC100m V2.0 map................................................................................................ 47

Table 20: Confusion matrix (%) for the Globeland30 2010 map at global scale, calculated using the

CGLS_LC100m validation data (corrected by sample inclusion probaiblities) ........................ 48

Table 21: Overall accuracy of the Globeland30 2010 map at continental level .............................. 49




List of Acronyms

AD Applicable Documents

ASAR Advanced Synthetic Aperture Radar

ATBD Algorithm Technical Basis Document

AVHRR Advanced Very High Resolution Radiometer

CEOS-WGCV Committee on earth Observing Satellites working Group on Calibration and

Validation

CGLS Copernicus Global Land Service

CGLS_LC100m CGLS Dynamic Land Cover 100m

CORINE Coordination of Information on the Environment

ENVISAT Environmental Satellite

ESRI Environmental Systems Research Institute

ETM Enhanced Thematic Mapper

FAPAR Fraction of Absorbed Photosynthetically Active Radiation

FAO Food and Agriculture Organisation

FR Full Resolution

HRPT High Resolution Picture Transmission

GEE Google Earth Engine

GlobeLand30 Global land cover dataset at 30m resolution

IIASA International Institute for Applied Systems Analysis

JRC Joint Research Center

LC-CCI Climate Change Initiative –Land Cover product

LAI Leaf Area Index

MERIS Medium Resolution Imaging Spectrometer

MODIS Moderate-resolution imaging spectroradiometer

NDVI Normalized Difference Vegetation Index

NOAA National Oceanic and Atmospheric Administration

MAE Mean absolute error

OECD Organisation for Economic Cooperation and Development

PROBA-V Project for On-Board Autonomy – Vegetation satellite

PUM Product user manual

RMSE Root mean square error

RR Reduced Resolution

SDG Sustainable Development Goal

SEEA System of Environmental and Economic Accounting

SVP Service validation plan

TM Thematic Mapper

TOC Top of Canopy

UNCCD United Nations Convention to Combat Desertification




UNFCCC United Nations Framework Convention on Climate Change

UN LCCS United Nations Land Cover Classification System

URD User requirement document

UTM Universal Transverse Mercator

VR Validation Report

WGS World Geodetic System

WUR Wageningen University and Research




EXECUTIVE SUMMARY

The Copernicus Global Land Service (CGLS) is earmarked as a component of the Land service to

operate “a multi-purpose service component” that provides a series of bio-geophysical products on

the status and evolution of land surface at global scale. Production and delivery of the parameters

take place in a timely manner and are complemented by the constitution of long-term time series.

From 1st January 2013, the Copernicus Global Land Service is providing a series of bio-

geophysical products describing the status and evolution of land surface at global scale. Essential

Climate Variables like Leaf Area Index (LAI), the Fraction of PAR absorbed by the vegetation

(FAPAR), the surface albedo, the Land Surface Temperature, the soil moisture, the burnt areas,

the areas of water bodies, and additional vegetation indices, are generated every hour, every day

or every 10 days from Earth Observation satellite data. Production and delivery of the parameters

are performed on a reliable, automatic and timely manner and are complemented by the

constitution of long term time series.

The CGLS delivers a dynamic global land cover product at 100m spatial resolution

(CGLS_LC100m). The objective of this quality assessment was to validate the Version 2.0 of the

discrete land cover map and fractional land cover layers of CGLS_LC100m product, providing

statistically robust estimates of overall and class specific accuracies. The validation was done

using an independent set of sample points (~21 600) generated in collaboration with experts from

the WUR and other regional experts. In addition to the statistical validation, assessments also

include qualitative and quantitative comparison of the CGLS_LC100m V2.0 discrete land cover

map with other recent global land cover products.

Our validation shows that the discrete global CGLS_LC100m V2.0 2015 Level 1 map had an

overall accuracy of 80.2+/-0.7%. In terms of land cover types, bare/sparse vegetation, snow/ice

and permanent water are mapped with high accuracies, while shrubs and herbaceous wetland

classes are mapped with lowest accuracies. For the continents, accuracies were also around 80%

for the most of the continents, with exceptions to North America (77.1 ± 1.7) and Asia (83.3± 1.5).

At Level 2, when closed and open forests classes were separated, global overall accuracy is

75.1% +/-0.7%. Among the cover fraction layer, snow & ice, built-up, water and lichen/moss

fraction maps show lowest errors, followed by crops and bare and sparse vegetation fraction types.

On the other hand, herbaceous vegetation fraction product has the highest error.

Comparison of the CGLS_LC100m V2.0 discrete map with other global land cover maps showed

that this new product has higher accuracy than the existing ones. Our visual comparison of the

CGLS_LC100m V2.0, the LC-CCI 2015 and the Globeland30 2010 maps shows good agreements

between the CGLS_LC100m V2.0 and the Globeland30 2010 map for the characterization of

natural vegetation classes and croplands with some regional differences. Quantitative comparison

showed the CGLS_LC100m V2.0 map has 3% higher agreement with a combined reference

dataset than the other global land cover maps. In addition, a comparison of the Globeland30 2010




map using the CGLS_LC100 validation dataset also shows that the CGLS_LC100m V2.0 discrete

map has, at least, 10% higher accuracy than the Globeland 30 2010 map at global scale, although

there are some differences in the legend definitions and the reference years. These results indicate

that the global CGLS_LC100m dynamic Land Cover V2.0 product has good improvements in

characterizing land cover types as compared to other global land cover maps.

These achievements can be further improved by focusing on the main limitations such as better

characterization of shrubs and wetland classes and better prediction of herbaceous vegetation

fractions.




1 BACKGROUND OF THE DOCUMENT

1.1 SCOPE AND OBJECTIVES

This document describes the validation of the CGLS Dynamic Land Cover 100m product V2.0

(CGLS_LC100m V2.0). It presents the objectives of the map validation process, provides details

on the methods and the results of the validation.

The quality assessment is performed on the 2015 CGLS_LC100m V2.0 product at global scale,

including evaluation at continent level.

1.2 CONTENT OF THE DOCUMENT

This document is structured as follows:

Chapter 2 recalls the users requirements and the expected performance

Chapter 3 describes the data and methodology used for the validation

Chapter 4 presents the results of the analysis

Chapter 5 summarizes the main conclusions of the study

Chapter 6 makes recommendations based upon the results

1.3 RELATED DOCUMENTS

1.3.1 Applicable documents

AD1: Annex I – Technical Specifications JRC/IPR/2015/H.5/0026/OC to Contract Notice 2015/S

151-277962 of 7th August 2015

AD2: Appendix 1 – Copernicus Global land Component Product and Service Detailed Technical

requirements to Technical Annex to Contract Notice 2015/S 151-277962 of 7th August 2015

AD3: GIO Copernicus Global Land – Technical User Group – Service Specification and Product

Requirements Proposal – SPB-GIO-3017-TUG-SS-004 – Issue I1.0 – 26 May 2015.

1.3.2 Input

Document ID Descriptor

CGLOPS1_SSD Service Specifications of the Copernicus Global Land

Service “Vegetation and Energy” component

CGLOPS1_SVP Service Validation Plan of the Copernicus Global

Land Service




CGLOPS1_ATBD_LC100m-V2.0 Algorithm Theoretical Basis Document of the

Moderate Dynamic Land Cover Version2.0 product

CGLOPS1_URD_LC100m Users Requirement Document of Moderate Dynamic

Land Cover 100m product

CGLOPS1_PSD_LC100m Product Specifications Document of the dynamic land

cover 100m product.

CGLOPS1_TrainingDataReport_L

C100m

Report describing the training dataset used for

Dynamic Land Cover 100m product

CGLOPS1_VR_LC100m-V1.0 Validation report of the Moderate Dynamic Land

Cover Version1 product

1.3.3 Output

Document ID Descriptor

CGLOPS1_PUM_LC100m-V2.0 Product User Manual summarizing all information about the

Moderate Dynamic Land Cover Version 2.0 product




2 REVIEW OF USERS REQUIREMENTS

According to the applicable documents [AD2] and [AD3], the user’s requirements relevant for

Dynamic Moderate Land Cover are:

Definition: Dynamic global land cover products at 300m and/or 100m resolution using UN

Land Cover Classification System (LCCS)

Geometric properties:

o Pixel size of output data shall be defined on a per-product basis so as to facilitate

the multi-parameter analysis and exploitation.

o The baseline datasets pixel size shall be provided, depending on the final product,

at resolutions of 100m and/or 300m and/or 1km.

o The target baseline location accuracy shall be 1/3 of the at-nadir instantaneous field

of view.

o Pixel co-ordinates shall be given for centre of pixel.

Geographical coverage:

o geographic projection: lat long

o geodetical datum: WGS84

o pixel size: 1/112° - accuracy: min 10 digits

o coordinate position: pixel centre

o global window coordinates:

Upper Left: 180°W-75°N

Bottom Right: 180°E, 56°S

Accuracy requirements: Overall thematic accuracy of dynamic land cover mapping

products shall be >80%. The overall accuracy assessment (including confidence limits) will

be based on a stratified random sampling design and the minimum number of sampling

points per land cover class relevant to the product shall be calculated as described in

Wagner and Stehman, 2015.

Few workshops were held in 2016 to consult different stakeholders to understand users’ needs for

global land cover maps. A feasibility study was performed to define the guidelines to create the first

LC100 map. More details can be found in [CGLOPS1_URD_LC100m]. Larger consultations in

2017 and 2018 allowed collecting the requirements of wide user communities which were

translated in product specifications [CGLOPS1_PSD_LC100m].

Table 1 summarizes the usefulness of information on LC and LC change processes for different

international actions and programmes. Table 2 includes the LC classes that are required by

different JRC units and are marked by (“X”), respectively. The last column provides the following

information: either a class is included in the LC100m V2.0 product’s legend, or it can be derived by

users from the fraction layers, or additional research is needed.




Table 1: Usefulness of information on LC and LC change processes for different international actions

and programmes.

LC types Related land

change processes UNFCCC UNCCD OECD SEEA/FAO SDGs

1 Urban/built-up areas Urbanization

2 Cropland Crop expansion

3 Cropland and other

vegetation Land abandonment

4 Forest Deforestation

5 Forest Reforestation

6 Wetland Wetland degradation

7 Water body Expansion of water

surface

8 Water body Reduction of water

surface

9 Bare areas Desertification

Table 2: List of land cover classes requested by users.

Code

Level

1

Code

Level

2

UN LCCS level Land cover class

Fo

rest

mo

dell

ing

/RE

DD

+

Cro

p m

on

ito

rin

g

Bio

div

ers

ity

Mo

nit

ori

ng

En

vir

on

men

t

an

d S

ecu

rity

in

Afr

ica

Clim

ate

mo

de

llin

g

Cla

ss

in

clu

ded

in

th

e

pro

du

ct

10 A12A3A20B2 Forest/tree cover X X X X Yes

11 A12A3A20B2D

2E1

Evergreen

Needleleaf forest X X X

Yes

12 A12A3A20B2D

1E1

Evergreen

Broadleaf forest X X X

Yes

13 A12A3A20B2D

2E2

Deciduous

Needleleaf forest X X X

Yes

14 A12A3A20B2D

1E2

Deciduous

Broadleaf forest X X X

Yes

15 A12A3A20B2D Mixed forest X X Yes




Code

Level

1

Code

Level

2


Fo

rest

mo

dell

ing

/RE

DD

+

Cro

p m

on

ito

rin

g

Bio

div

ers

ity

Mo

nit

ori

ng

En

vir

on

men

t

an

d S

ecu

rity

in

Afr

ica

Clim

ate

mo

de

llin

g

Cla

ss

in

clu

ded

in

th

e

pro

du

ct

1D2

16

A12A3A10B2X

XXX (assuming

that an intact

forest is a very

dense forest)

Intact forest

X X X

To map these classes,

addition research is

required for

methodology. Either

we develop an expert

rule based on other

datasets such as:

http://www.intactforest

s.org/data.ifl.html or

new training and

validation datasets.

These classes could

potentially be included

in the next product

evolutions.

17 - Secondary forest X X X

18 A11A1 Managed forest X X X

A11A1 Plantation

forest/tree crops X X X X

A11A1 Oil palm plantation X X

- Forest logging X X X

A12A3

Dominant tree

species, e.g.

spruce, pine, birch

X X

A11A1(A2/A3) Shifting cultivation

system X X X

20 AA12A4A20B3(

B9) Shrub X X X

Yes

21 A12A4A20B(B9

)XXE1 Evergreen shrubs X

These classes could

be potentially included

in the next product

evolutions. We will

have to collect

corresponding training

and validation data.

22 A12A4A20B3(B

9)XXE2 Deciduous shrubs X

30 A12A2(A6)A20

B4

Herbaceous

vegetation X X X

Yes

A12A6A10 //

A11A1A11B4X

XXXXXF2F4F7

G4-F8

Pasture/managed

grassland X



required for

methodology. Also we

will have to develop

new training and


A122(A6)A10 Natural grassland X X

A12A2 Grass types for

Western Africa X

http://www.intactforests.org/data.ifl.html

http://www.intactforests.org/data.ifl.html




Code

Level

1

Code

Level

2


Fo

rest

mo

dell

ing

/RE

DD

+

Cro

p m

on

ito

rin

g

Bio

div

ers

ity

Mo

nit

ori

ng

En

vir

on

men

t

an

d S

ecu

rity

in

Afr

ica

Clim

ate

mo

de

llin

g

Cla

ss

in

clu

ded

in

th

e

pro

du

ct

These classes could


in the next product

evolutions.

A12A3A11B2X

XXXXXF2F4F7

G4-A12;

A12A3A11B2-

A13;

A12A1A11

Savannas X

The LC100 V2 product

includes such a LC

class as open forest,

which is a mix of trees

(more than 15%),

shrubs and grassland.

This class only partly

corresponds to

savannas because a

100m x100m pixel

may include less trees

but still be considered

as savanna.

However, users are

encouraged to use the

fraction layers to

produce their own

savanna layer by

applying specific

thresholds for tree,

shrub and grass cover.

40 A11A3

Cultivated and

managed

vegetation/agricultur

e

X X X X

Yes

41 A11A3XXXXXX

D3(D9) Irrigated cropland X X



required for



new training and

42 A11A3XXXXXX

D1 Rainfed cropland X X

43 A11A3 Big and small

farming/field size X




Code

Level

1

Code

Level

2


Fo

rest

mo

dell

ing

/RE

DD

+

Cro

p m

on

ito

rin

g

Bio

div

ers

ity

Mo

nit

ori

ng

En

vir

on

men

t

an

d S

ecu

rity

in

Afr

ica

Clim

ate

mo

de

llin

g

Cla

ss

in

clu

ded

in

th

e

pro

du

ct

44 A11A1-W8/A2 Permanent crops X X validation datasets.

These classes could


in the next product

evolutions.

45 A11A3 Row crops X

A11A2

Crop types:

long/short cycle or

winter/summer

crops

X

A11A2 Multiple crop cycles X

50 B15A1 Urban/built up X X X Yes

60 B16A1(A2) Bare/sparse

vegetation X X

Yes

70 B28A2(A3) Snow and Ice X X Yes

80 B28A1 Open water X X Yes

90 A24A1(A2/A3/A

4) Wetland X X X

Yes

A24A3 Mangroves X X

To map this class,


required for



new training and


These classes could


in the next product

evolutions.




3 QUALITY ASSESSMENT METHOD

The objective is to validate the global CGLS_LC100m V2.0 product providing statistically robust

estimates of overall and class specific accuracies. The validation also includes assessment of 9

fractional land cover layers. In addition to the statistical accuracy assessment, comparative

assessments of other maps are included in the validation.

3.1 OVERVALL PROCEDURE

The validation process follows clearly defined protocols that are detailed in the Service Validation

Plan [CGLOPS1_SVP]. A global reference dataset was collected and used for the validation of the

CGLS_LC100m V2.0 products (the discrete and fractional maps). In addition, the CGLS_LC100m

V2.0 discrete map was qualitatively and quantitatively compared against other existing global land

cover maps.

These steps are explained in detail in the following sub-sections which describe the

CGLS_LC100m V2.0 products that were the subject of the validation (Section 3.2), in-situ

reference dataset (Section 3.3), the methods for map assessment (Section 3.4), and comparison of

the CGLS_LC100m with other maps (Section 3.5).

3.2 CGLS_LC100M 2015 PRODUCTS

The second version (V2.0) of the Dynamic Land Cover map at 100 m resolution includes global

scale discrete and fractional land cover layers for 2015 as opposed to the first version (V1.0) of

maps which were provided over the African continent for 2015. These maps were generated from

the PROBA-V 100 m time-series, a database of high quality land cover training sites and several

ancillary datasets. The description of the map generation is detailed in the ATBD

[CGLOPS1_ATBD_LC100m-V2.0]. The global discrete map provides 23 classes (Table 3) and is

defined using the Land Cover Classification System (LCCS) developed by the United Nations (UN)

Food and Agriculture Organization (FAO). The UN-LCCS system was designed as a hierarchical

classification, which allows adjusting the thematic detail of the legend to the amount of information

available:

The “level 1” legend contains classes with codes that are multiples of ten in Table 3 (i.e.

class values of 10, 20, 30, etc.).

The “level 2” legend, a more detailed legend known as regional legend, has class codes of

two digits that is not a multiple of ten in Table 3 (i.e. class values of 11, 12 are sub-classes

of 10, and so on).

The “level 3” legend has three digits (i.e. 111 – 116 and 121 – 126) and are used to further

distinguish the forest types (sub-classes of 11 – open forest and 12 – closed forest).

Figure 1 shows the global discrete land cover map V2.0 at level 3.




Table 3: Discrete classification coding of the CGLS dynamic land cover map V2.0.

Map code

UN LCCS level Land Cover Class Definition according UN LCCS Color code (RGB)

0 - No input data available - 0, 0, 0

111 A12A3A10B2D2E1 Closed forest, evergreen needle leaf

tree canopy >70 %, almost all needle leaf trees remain green all year. Canopy is never without green foliage.

88, 72, 31

113 A12A3A10B2D2E2 Closed forest, deciduous needle leaf

tree canopy >70 %, consists of seasonal needle leaf tree communities with an annual cycle of leaf-on and leaf-off periods

112, 102, 62

112 A12A3A10B2D1E1 Closed forest, evergreen, broad leaf

tree canopy >70 %, almost all broadleaf trees remain green year round. Canopy is never without green foliage.

0, 153, 0

114 A12A3A10B2D1E2 Closed forest, deciduous broad leaf

tree canopy >70 %, consists of seasonal broadleaf tree communities with an annual cycle of leaf-on and leaf-off periods.

0, 204, 0

115 A12A3A10 Closed forest, mixed Closed forest, mix of types 78, 117, 31

116 A12A3A10 Closed forest, unknown Closed forest, not matching any of the other definitions

0, 120, 0

121 A12A3A11B2D2E1 Open forest, evergreen needle leaf

top layer- trees 15-70 % and second layer- mixed of shrubs and grassland, almost all needle leaf trees remain green all year. Canopy is never without green foliage.

102, 96, 0

123 A12A3A11B2D2E2 Open forest, deciduous needle leaf

top layer- trees 15-70 % and second layer- mixed of shrubs and grassland, consists of seasonal needle leaf tree communities with an annual cycle of leaf-on and leaf-off periods

141, 116, 0

122 A12A3A11B2D1E1 Open forest, evergreen broad leaf

top layer- trees 15-70 % and second layer- mixed of shrubs and grassland, almost all broadleaf trees remain green year round. Canopy is never without green foliage.

141, 180, 0

124 A12A3A11B2D1E2 Open forest, deciduous broad leaf

top layer- trees 15-70 % and second layer- mixed of shrubs and grassland, consists of seasonal broadleaf tree communities with an annual cycle of leaf-on and leaf-off periods.

160, 220, 0

125 A12A3A12 Open forest, mixed Open forest, mix of types 146, 153, 0

126 A12A3A12 Open forest, unknown Open forest, not matching any of the other definitions

100, 140, 0

20 A12A4A20B3(B9) Shrubs

These are woody perennial plants with persistent and woody stems and without any defined main stem being less than 5 m tall. The shrub foliage can be either evergreen or deciduous. 255, 187, 34

30 A12A2(A6)A20B4 Herbaceous vegetation

Plants without persistent stem or shoots above ground and lacking definite firm structure. Tree and shrub cover is less than 10 %. 255, 255, 76

90 A24A2A20 Herbaceous wetland

Lands with a permanent mixture of water and herbaceous or woody vegetation. The vegetation can be present in either salt, brackish, or fresh water. 0, 150, 160

100 A12A7 Moss and lichen Moss and lichen 250, 230, 160




Map code

UN LCCS level Land Cover Class Definition according UN LCCS Color code (RGB)

60 B16A1(A2) Bare / sparse vegetation Lands with exposed soil, sand, or rocks and never has more than 10 % vegetated cover during any time of the year 180, 180, 180

40 A11A3 Cultivated and managed vegetation/agriculture (cropland)

Lands covered with temporary crops followed by harvest and a bare soil period (e.g., single and multiple cropping systems). Note that perennial woody crops will be classified as the appropriate forest or shrub land cover type. 240, 150, 255

50 B15A1 Urban / built up Land covered by buildings and other man-made structures 250, 0, 0

70 B28A2(A3) Snow and Ice Lands under snow or ice cover throughout the year. 240, 240, 240

80 B28A1B1 Permanent water bodies lakes, reservoirs, and rivers. Can be either fresh or salt-water bodies. 0, 50, 200

200 B28A1B11 Open sea

Oceans, seas. Can be either fresh or salt-water bodies. 0, 0, 128

Figure 1: The CGLS Dynamic Land Cover Map V2.0 at 100 m (detailed legend in Table 3).

Next to the discrete map, the product also includes a set of ten continuous field layers that provide

proportional estimates for main land cover types (from 0 to 100%). This continuous classification

scheme may depict areas of heterogeneous land cover better than the standard classification

scheme and, as such, can be tailored for application use (e.g. forest monitoring, biodiversity and

conservation, climate modelling, etc.). Figure 2 shows a collage of the cover fraction layers (i.e.

shrubland, forest, moss&lichen, snow, seasonal water, permanent water, bare/sparse vegetation,

cropland, built-up, and herbaceous vegetation) for different parts of the world.




Figure 2: The fractional land cover layers for the 10 base land cover classes of the CGLS Dynamic

Land Cover V2.0 product at 100 m (shown as a collage on global scale).

3.3 IN-SITU REFERENCE PRODUCTS

The land cover products were assessed using an independent validation dataset. This section

describes briefly about the sample selection (sampling design) and reference data collection

processes (response design). A more detailed information on the data collection design can be

found in the Service Validation Plan [CGLOPS1_SVP] and in (Tsendbazar et al., 2018).

Sampling design

The validation dataset was collected through two rounds of reference data collection activities.

The first round of collection was based on stratified sampling following the stratification by Olofsson

et al. (2012) based on the Köppen climate zones and human population density. In this, climate

zones are separated from the populated parts (more than 5 persons/km2). Because this

stratification is not dependent on any existing land cover maps, the collected sample locations can

be considered as representative and suitable for validating subsequent maps that are planned

within the CGLS, with some additional considerations such as rechecking and/or updating the

reference land cover at validation locations (Tsendbazar et al., 2018).




Figure 3: Continents used for validation dataset collection.

In order to facilitate accuracy assessment at global scale as well as continental scale, we collected

validation sample sites for each continent. Approximately 2700 sample sites were planned for each

continent in addition to the sample sites collected for Africa in the validation of CGLS_LC100m

V1.0 [CGLOPS1_VR_LC100m-V1] (Tsendbazar et al., 2018). The validation data collection in

other continents followed the same approach as Africa. Due to its exceptionally large size, Asia

was divided into 2 sub-continents (Figure 3): Asia and Northern Eurasia which includes Russia

(both European and Asian part), Kazakhstan and Mongolia.

After the first round, the validation dataset contains 20,019 sample sites across the world. These

sample sites were allocated following the approach of Olofsson et al. (2012) which aims to target

land cover classes that are more likely to be misclassified, in particular those that occur in

heterogeneous landscapes (Jung et al., 2006) [CGLOPS1_VR_LC100m-V1.0. Stratification map

including the selected validation sites are depicted in Figure 4.




Figure 4: Selected validation sites and the stratification by Olofsson et al 2012: ‘p’ before the strata

names denote populated part of climate zone.

The second round of the reference data collection focuses on the rare land cover types such as

urban and water body among the LCCS Level 1 legend. We applied a requirement of, at least, 100

sample sites per LCCS level1 land cover type for each continent to ensure the rare land cover

types are represented sufficiently [CGLOPS1_SVP]. This additional collection mostly focused on

land cover types namely urban, herbaceous wetland, permanent water, snow/ice, moss/ lichen,

and shrubs.

After the second round, the validation dataset contains, in total, 21,752 sample sites.

Response design

Aspects related to the labelling of reference land cover types are specified here.

The reference land cover at each sample site was interpreted visually by regional experts who

have experiences of working with satellite based land cover analysis and image interpretation.

Regional experts and their representing regions are provided in Figure 5.




Figure 5: Regional Experts and their representing regions.

The experts were provided with an access to a designated validation data collection web-interface

which is based on Geo-Wiki platform. Validation data collection process followed several steps to

ensure a good quality reference data for validation (Figure 6). Experts were offered a remote

training to get familiarized with the interface, land cover type definition and tutorials. Since a visual

interpretation can be subject to interpreter variability and bias (Strahler et al., 2006), feedback

loops were conducted in loops of interpretations. Experts received feedback on their work after

they interpreted 10-20, 50 and 100 validation locations and the rest of the validation locations.

Experts could only continue when they have worked on the feedbacks received from validation

experts from WUR. Feedback was given for each validation interpretation. The experts have

corrected the interpretations where necessary.

Quality checks and consolidation efforts were also carried out in order to select the updated

interpretations. We further consolidated our validation dataset against some national and regional

land cover maps. We checked the land cover labels of the validation dataset against land cover

maps such as CORINE (Bossard et al., 2000), Australian Dynamic Land Cover (Geoscience

Australia, 2010), Northern American Land Cover product (Latifovic et al., 2004), and Circumpolar

Arctic Vegetation Map (Walker et al., 2005). Any validation points which did not match with these

datasets were rechecked for confirmation and consolidation. In total, as part of feedback loops and

consolidation efforts, approximately 32% of the total 21,752 sample sites were updated partially or

fully.




Figure 6: Validation data collection process.

Different information sources were provided to the regional experts for visual interpretation. These

include Google, Bing maps, ESRI-WORLD imagery as well as Sentinel-2 images. Furthermore,

time series NDVI profiles based on MODIS (MOD13Q1.005 16 days composite), Landsat (32 days

composite from Landsat-7 and 8 data), PROBA-V (maximum NDVI over 3 days from TOC daily

NDVI from PROBA-V 100m Collection 1 data) were used as information source of plant phenology:

these dataset are available through Google Earth Engine (GEE: https://earthengine.google.com).

Thumbnails of all available Sentinel-2 images were also made available for different band

combinations. In addition, experts had the possibility to load the validation point location Google

Earth to make use of historical very high resolution images.

Figure 7 shows a screenshot of the dedicated interface on the Geo-Wiki platform. An access to this

validation interface was only given to the experts involved with the validation data collection.




Figure 7: A screenshot of Geo-Wiki based land cover validation data collection interface.

The validation sample site areas match with a PROBA-V pixel (100x100m), which is aligned to

Sentinel-2 pixels based on UTM projection [CGLOPS1_PUM_LC100m-V2.0]. At each sample unit,

10mx10m subpixels were created and reference information on the land cover was labelled on

each of the subpixel. Figure 8 shows an example of sample interpretation on the validation

interface.

This response design was used due to the following advantages:

Allows objective way of quantifying the class proportions within the sample unit area in case

of heterogeneous areas.

Land cover elements such as trees, shrubs, grass and cropland, building etc., will be

recorded rather than land cover classes. This allows the collected data to be used for

different legends flexibly.

Interpretation at smaller box/points is suitable for validating/calibrating future higher-

resolution evolutions of land cover maps.




Figure 8: A screenshot of an example sample interpretation over the 10mx10m sub-pixels included

into a PROBA-V pixel (100mx100m).

It is worth to mention that, although this validation dataset was collected on the Geo-Wiki platform

and its set-up on labelling land cover at 10m resolution is similar to that of the training dataset used

for the product mapping, the validation dataset is fully independent from the training dataset. The

validation dataset was collected by a different group of experts (Figure 5). Validation data

collection was managed independently by a separate institution (WUR). Sample selection scheme

of the validation dataset is different than that of the training datasets.

Land cover information per subpixel was combined for each sample site to obtain information on

the fraction of land cover types within the sample sites. This information was then translated into

the discrete map legend using the UN LCCS (United Nations Land Cover Classification System) as

a basis [CGLOPS1_URD_LC100m]. Based on the report on the training data

[CGLOPS1_TrainingDataReport_LC100m], land cover fractions were directly converted to land

cover classes for homogeneous sample sites (e.g., 100% water proportion corresponds to water

body class). For heterogeneous sample sites where conditions can meet the definitions of multiple

land cover types, a priority rule was applied Tsendbazar et al. (2018). The priority order was

permanent water, urban, cropland, herbaceous wetland, forest, shrubs, herbaceous vegetation,

bare/sparse vegetation, lichen/moss, and snow & ice. In the legend translation, +/- 5% deviations

from the legend definition thresholds were allowed. This is mainly due to the geolocation error (or

shift) of the very high resolution images of Google Map and Bing Map.

3.4 MAP ACCURACY ASSESSMENT

Based on the validation dataset, the discrete and fractional land cover layers were assessed.

To estimate the accuracy of the land cover maps, we followed the same methods used in

Tsendbazar et al. (2018). We accounted for unequal inclusion probabilities between different strata

because sample sites were not allocated proportionally to the strata areas (Olofsson et al.,

2012;Wickham et al., 2010). Based on Pengra et al. (2015), the inclusion probability for stratum h




is defined as πh = kh / Kh, where kh is number of sample sites in stratum h and Kh is the population

size for stratum h. Number of sites is based on the 100m × 100m units. Inclusion probability for the

additional sample sites for the rare classes were calculated based on the population of possible

sample sites within the rare classes of the CGLS_LC100m V2.0 map. The estimation weight, the

inverse of inclusion probability ( h = 1 / πh ), was then calculated and used to construct the

confusion matrix accounting for unequal sample inclusion probabilities following the methods

described in Stehman et al. (2003) and Wickham et al. (2010). We then estimated the overall and

class specific accuracies and their confidence intervals (at 95% confidence level) following

Stehman (2014) which specifically addresses estimating map accuracies when the sampling strata

are different from the map classes.

Validation of the discrete maps focused on Level 1 and Level 2 legends which differ in terms of

number of forest related classes (see Section 3.2). Accuracies of the discrete map was calculated

at global scale and as well as at seven (sub) continent levels (Figure 3) described in Section 3.3.

Not all Level 1 classes are validated at (sub) continent level due to their very limited or no

appearance in some continents. These include snow and ice in Africa and Oceania & Australia;

lichen and moss in Africa, Asia, Oceania and South America.

Out of 21,752, some validation sites had no data in the CGLS_LC100m V2.0 product. These

locations are mostly in the extreme north, outside the mapped region of the CGLS_LC100m

product. Therefore, 21,594 sites were used to validate the map.

In addition to the accuracy assessment of the discrete land cover map, quality assessment was

conducted for the fractional land cover layers. Validation of the fractional layers focused on Level 1

legend which includes nine fractional layers, i.e. trees, shrubs, herbaceous vegetation, crops,

moss/lichen, bare sparse vegetation, snow/ice, built-up and permanent water fraction layers.

Seasonal water is not included in Level 1 legend, therefore this layer was not validated. To do this,

fraction information of the land cover types in the validation dataset was directly used. For each

fractional land layer, the mean absolute error (MAE) and root mean square error (RMSE) were

calculated (Foody, 1996; Pengra et al., 2015).

(Eq.1)

where RMSEc is the root mean squared error of class c, vi is the reference fraction of class c (in

percent), pi is the mapped fraction of class c, represents the estimation weight for the sample

site and n is the total number of sample sites.

(Eq.2)

These two statistics show how many percent did the classification either over-represent or under-

represent class c membership of all pixels in the image on average. The difference between the

two is that RMSE gives a larger penalty for large errors compared to MAE.




3.5 COMPARISON WITH OTHER GLOBAL LAND COVER MAPS

3.5.1 Other global land cover products used for comparison

Two widely available global land cover products were selected for comparison:

(1) The Climate Change Initiative-Land Cover (LC-CCI) delivers time series of annual global

land cover maps at 300 m from 1992 to 2015, using 5 different satellite missions, including

NOAA-AVHRR HRPT, SPOT-Vegetation, ENVISAT-MERIS FR and RR, ENVISAT-ASAR,

and PROBA-V. The LC-CCI comprises 22 land cover classes at level 1 (see Table 4). Here

we used the LC-CCI 2015 map which has an overall thematic accuracy of 74.4%.

Table 4: Legend of the global CCI-LC maps, based on LCCS.

Value Label Color

10 Cropland, rainfed 20 Cropland, irrigated or post-flooding 30 Mosaic cropland (>50%)

/ natural vegetation (tree, shrub, herbaceous cover) (<50%)

40 Mosaic natural vegetation (tree, shrub, herbaceous cover) (>50%) / cropland (<50%)

50 Tree cover, broadleaved, evergreen, closed to open (>15%) 60 Tree cover, broadleaved, deciduous, closed to open (>15%) 70 Tree cover, needleleaved, evergreen, closed to open (>15%) 80 Tree cover, needleleaved, deciduous, closed to open (>15%) 90 Tree cover, mixed leaf type (broadleaved and needleleaved) 100 Mosaic tree and shrub (>50%) / herbaceous cover (<50%) 110 Mosaic herbaceous cover (>50%) / tree and shrub (<50%) 120 Shrubland 130 Grassland 140 Lichens and mosses 150 Sparse vegetation (tree, shrub, herbaceous cover) (<15%) 160 Tree cover, flooded, fresh or brakish water 170 Tree cover, flooded, saline water 180 Shrub or herbaceous cover, flooded, fresh/saline/brakish water 190 Urban areas 200 Bare areas 210 Water bodies 220 Permanent snow and ice

(2) The Globeland30 provides global land cover maps for the year 2000 and 2010 at 30 m

resolution using Landsat TM, Landsat ETM+, and the Chinese Environmental Disaster

Alleviation Satellite (HJ-1) data (Chen et al., 2015). The Globeland30 map comprises ten

land cover classes (see Table 5 and Figure 9), of which eight occur in Africa (i.e. cultivated




land, forest, grassland, shrubland, wetland, water bodies, artificial surfaces and bare land).

Here we used the 2010 map for Africa. The overall map accuracy has been reported to be

79.26% at global level (Chen et al., 2015) but no accuracy information is available for

Africa.

Table 5: Legend definition of the Globeland 30 map (http://www.globeland30.org/)

Code Classes Definition

10 Cultivated

land

Lands used for agriculture, horticulture and gardens, including paddy

fields, irrigated and dry farmland, green houses cultivated land,

artificial tame pastures, economic cultivated land (such as grape,

coffee, and palm), and abandoned arable lands.

20 Forest Lands covered with trees, with vegetation cover over 30%, including

deciduous and coniferous forests, and sparse woodland with cover 10

- 30%.

30 Grassland Lands covered by natural grass with cover over 10%.

40 Shrubland Lands covered with shrubs with cover over 30%, e.g. deciduous and

evergreen shrubs, and desert steppe with cover over 10%.

50 Wetland Lands covered with wetland plants and water bodies, e.g. inland

marsh, lake marsh, river floodplain wetland, forest/shrub wetland, peat

bogs, mangrove and salt marsh.

60 Water bodies Water bodies in the land area, e.g. river, lake, reservoir, fish pond.

70 Tundra Lands covered by lichen, moss, hardy perennial herb and shrubs in

the polar regions, e.g. shrub tundra, herbaceous tundra, wet tundra

and barren tundra.

80 Artificial

surfaces

Lands modified by human activities, e.g. all kinds of habitation,

industrial and mining area, transportation facilities, and interior urban

green zones and water bodies.

90 Bare land Lands with vegetation cover lower than 10%, e.g. desert, sandy fields,

Gobi, bare rocks, saline and alkaline lands.

100 Permanent

snow and ice

Lands covered by permanent snow, glacier and icecap.

https://www.sciencedirect.com/topics/earth-and-planetary-sciences/greenhouse

https://www.sciencedirect.com/topics/earth-and-planetary-sciences/pasture

https://www.sciencedirect.com/topics/earth-and-planetary-sciences/arable-land




Figure 9: Legend of the Globeland30 2010 map.

3.5.2 Qualitative comparison

We assessed the CGLS_LC100m V2.0 discrete map qualitatively by comparing it with Google Map

and the two global land cover products (e.g. LC-CCI 2015 and Globeland30 2010). For this, the

CGLS_LC100m discrete map, the LC-CCI 2015 and Globeland30 2010 maps were made

accessible in the Geo-Wiki portal.

3.5.3 Quantitative comparison

We compared the accuracy of the CGLS_LC100m V2.0 discrete map with other available maps

using two different validation datasets. The first validation dataset is a combination of available

reference datasets compiled by Tsendbazar et al. (2016), while the second validation dataset is the

one described in the Section 3.3.

Comparison using a combined validation dataset

We base this comparison on the study by Tsendbazar et al. (2016). This study combined openly

available global land cover reference datasets and harmonized them into 9 generic classes,

namely forests, shrubland, grassland, cropland, wetland, urban, bare, water and snow/ice. In total,

24,687 reference points were used. The reference period for these reference datasets varied from

2000 to 2010 (Tsendbazar et al. (2016)). Although there are quite some differences with respect to

the reference years and to the legend harmonization compared to those of the CGLS_LC100 V2.0,

the reference dataset of Tsendbazar et al. (2016) could still be useful to indicate the relative quality

of the CGLS_LC100m V2.0 map compared with the previously existing global maps. Excluding

some no data points in the CGLS_LC100m V2.0 map, in total 24,593 points were used in the




comparison. CGLS_LC100m V2.0 map legend was made compliant to the legend of the combined

reference data: only the lichen/moss class was recoded as herbaceous vegetation to be

compatible to the legend of the combined reference data of Tsendbazar et al. (2016). We used this

combined reference dataset to calculate the overall agreement of the CGLS_LC100m V2.0 map

and compared it against overall agreements of four existing global land cover maps which were

studied in Tsendbazar et al. (2016): the Globcover 2009, LC-CCI 2010, MODIS2010 and

Globeland30 2010.

Comparison using the CGLS_LC100 validation dataset

We used the CGLS_LC100 validation dataset described in Section 3.3 to assess the accuracy of

the Globeland30 2010 map and compared with the accuracy of the CGLS_LC100m V2.0 map.

This map was selected because its pixels size (i.e. 30 m) is smaller than the sample site area (i.e.

100 m) of the CGLS_LC100 validation dataset. The other maps such as LC-CCI 2015 at 300 m

resolution, and MODIS land cover maps at 250 m resolution were not considered since their pixel

size is larger than the sample site area of the CGLS_LC100 validation dataset. As such, this

validation dataset cannot represent the reference land cover at coarser scales (250-300m). There

is 5-year difference in the reference year of the Globeland30 2010 map and the CGLS_LC100

validation dataset collected for 2015. This could cause inconsistency related to land cover changes

that have occurred during this period. However, the change areas are often much negligible

compared with the areas that are misclassified (Tsendbazar et al., 2016). Therefore, this

comparison is still suitable to indicate general quality of the map.

To assess the accuracy of the Globeland30 2010 map, we extracted the dominant mapped classes

value of the Globeland30 2010 map over the sample site areas of the CGLS_LC100m dataset.

Since our CGLS_LC100 validation dataset contains fractional land cover information, we used this

fraction information to translate the land cover of the validation dataset into the legend of the

Globeland30 2010 map (Table 5). The translation to the Globeland30 2010 legend used the similar

approach as the CGLS_LC translation to CGLS_LC100 legend as described in section 3.3.

The main inconsistency between the Globeland30 2010 map and the CGLS_LC100 validation

dataset is the definition of the wetland. In the validation dataset, wetland is defined as herbaceous

wetland (Table 3) while the Globeland30 2010 map includes other vegetation types as trees,

shrubs in the wetland vegetation (Table 5). In addition, tundra class does not correspond to one

type of land cover in the CGLS_LC100m validation dataset, rather this class represents

combination of different land cover types (lichen, shrubs, herbaceous vegetation, bare areas). Due

to this difference, sample locations with mapped class as “tundra” were not used in the

assessment (similar the comparison done by Tsendbazar et al. (2016)).

Afterwards, the map accuracies were calculated using the methods described in the Section 3.4 at

global and continental level. In total, 20, 780 sample locations were used in the assessment. Per

continent, sample sites varied from 2,745 in Northern Eurasia to 3,611 in Africa.




4 RESULTS

4.1 ACCURACY OF THE CGLS_LC100M V2.0 PRODUCT

The CGLS_LC100m V2.0 product (the discrete map and the nine fraction layers) were assessed

using the independent CGLS_LC100m validation dataset described in Section 3.3. The results of

the accuracy assessment are detailed below.

Discrete map

Based on the mapped and reference land cover types, a confusion or an error matrix was

calculated. The error matrix was corrected by sample inclusion probabilities (Tsendbazar et al

2018). Table 6 shows the confusion matrix for the discrete CGLS_LC100m V2.0 Level 1 map at

global scale.

Table 6: Confusion matrix (%) for the discrete CGLS_LC100m V2.0 Level 1 map at global scale,

corrected by sample inclusion probabilities.

Fo

rest

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

Lic

hen

/ m

oss

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 33.1 1.9 1.2 0.6 0.0 0.0 0.0 0.0 0.1 0.1 37.1 38.3 89.4 0.8

Shrubs 1.5 5.8 1.4 0.3 0.0 0.2 0.0 0.0 0.1 0.0 9.3 7.9 62.4 3.1

Herbaceous vegetation 1.2 2.3 13.8 0.6 0.0 0.5 0.0 0.1 0.3 1.0 20.0 19.2 69.2 1.8

Croplands 1.1 0.4 1.6 8.2 0.1 0.0 0.0 0.1 0.1 0.0 11.6 12.6 70.2 2.1

Urban 0.1 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.6 3.9 77.3 5.7

Bare/sparse vegetation 0.0 0.3 0.8 0.0 0.0 13.5 0.0 0.0 0.0 0.0 14.7 6.5 91.5 1.8

Snow/ice 0.0 0.0 0.0 0.0 0.0 0.1 1.9 0.0 0.0 0.0 2.0 2.4 94.7 3.5

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.1 0.0 2.0 0.0 0.0 2.1 4.0 94.9 2.0

Herbaceous Wetland 0.1 0.1 0.3 0.0 0.0 0.0 0.0 0.1 0.5 0.1 1.1 3.7 46.5 6.3

Lichen/moss 0.0 0.0 0.1 0.0 0.0 0.4 0.0 0.0 0.0 0.9 1.4 1.7 63.9 7.0

Total 37.3 10.9 19.2 9.7 0.7 14.7 2.0 2.3 1.2 2.2 100.0

Sample count 38.9 9.2 19.4 10.6 3.6 7.3 2.3 4.3 2.6 1.8 100.0

Producer's accuracy 88.9 53.4 71.8 83.9 76.1 91.7 98.9 86.8 43.7 42 80.2

Confidence interval ± 0.8 2.8 1.8 2.0 7.6 1.4 0.9 3.2 6.4 5.3

0.7

The overall accuracy of the CGLS_LC100m V2.0 discrete global land cover map is 80.2% +/-0.7%

(confidence intervals at 95% confidence level). In terms of class specific accuracies, forest,

bare/sparse vegetation, snow/ice and permanent water are mapped with very high accuracies (>

85%). The class accuracies of herbaceous vegetation, croplands and urban are moderate (70%-




85%) while herbaceous wetlands, lichen/moss and shrubs have lower class accuracies (<65%).

Specifically, croplands are in general overestimated at the cost of herbaceous vegetation class.

Urban class has higher confusion error with croplands, forest and herbaceous vegetation.

Herbaceous wetlands have high confusion error with herbaceous vegetation and shrubs have high

confusion error with forest and herbaceous vegetation class, what was expected considering the

spectral similarity of these classes. Lichen/moss class has higher confusion error with herbaceous

vegetation. All the class accuracies are estimated with high confidence, having confidence

intervals less than 8%.

The overall accuracies and confusion matrices were also calculated for each continent (Table 8 -

Table 14). All continents are mapped with overall accuracies around 80% with lowest 77% for

North America and highest 83.3 % for Asia at Level 1 (Table 7).

Table 7: Overall accuracy of the discrete CGLS_LC100m V2.0 Level 1 map per continent.

Number of samples Overall accuracy (%) Confidence intervals ±

Africa 3616 80.1 2.0

Asia 3071 83.3 1.5

Northern Eurasia 2976 79.8 1.6

Europe 3120 80.4 1.6

North America 2843 77.1 1.7

Oceania & Australia 2951 81.9 1.9

South America 3017 79.6 1.5

Table 8: Confusion matrix (%) for the discrete CGLS_LC100m V2.0 Level 1 map over Africa.

Africa

Fo

rest

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 25.5 1.7 1.0 0.8 0.0 0.0 0.0 0.1 29.2 34.3 87.5 2.2

Shrubs 3.0 9.7 2.5 0.9 0.0 0.3 0.0 0.1 16.6 13.1 58.3 6.4

Herbaceous vegetation 1.1 1.8 9.6 0.8 0.0 0.6 0.1 0.2 14.2 15.5 67.2 6.7

Croplands 0.5 0.8 1.1 4.9 0.0 0.0 0.0 0.1 7.4 9.9 66.4 6.4

Urban 0.1 0.0 0.1 0.0 0.3 0.0 0.0 0.0 0.5 5.7 52.8 20.5

Bare/sparse vegetation 0.0 0.3 1.2 0.0 0.0 28.7 0.0 0.0 30.3 8.7 94.9 2.9

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.0 7.1 99.6 0.6

Herbaceous Wetland 0.2 0.0 0.1 0.0 0.0 0.0 0.1 0.4 0.8 5.5 48.0 18.4

Total 30.6 14.2 15.6 7.5 0.3 29.7 1.2 1.0 100.0

Sample count 36.8 11.1 16.0 9.5 4.1 9.0 8.4 5.2 100.0

Producer's accuracy 83.5 68.2 61.4 65.0 94.1 96.8 83.0 42.0

80.1

Confidence interval ± 2.6 6.6 6.5 7.2 3.9 2.1 10.3 18.0

2.0




Table 9: Confusion matrix for the discrete CGLS_LC100m V2.0 Level 1 map over Asia.

Asia F

ore

st

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 27.1 1.2 1.0 0.8 0.0 0.0 0.0 0.0 0.0 30.1 36.7 90.1 2.0

Shrubs 0.3 1.3 0.3 0.0 0.0 0.1 0.0 0.1 0.0 2.1 3.3 63.0 9.5

Herbaceous vegetation 0.7 1.5 10.4 0.6 0.0 0.3 0.0 0.1 0.1 13.7 11.7 76.3 4.9

Croplands 2.4 0.7 2.2 13.9 0.2 0.1 0.0 0.3 0.4 20.1 18.5 69.2 4.0

Urban 0.1 0.0 0.1 0.1 1.1 0.0 0.0 0.0 0.0 1.3 3.3 80.0 7.9

Bare/sparse vegetation 0.1 0.6 1.9 0.2 0.1 27.3 0.0 0.0 0.0 30.3 16.7 90.1 2.5

Snow/ice 0.0 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.5 3.3 90.0 5.9

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.6 0.0 1.7 3.3 95.0 4.3

Herbaceous Wetland 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.2 3.3 43.0 9.8

Total 30.8 5.4 15.8 15.6 1.3 27.8 0.4 2.1 0.6 100.0

Sample count 37.2 5.6 14.6 15.0 2.8 15.6 2.9 4.3 1.9

100.0

Producer's accuracy 88.0 24.9 65.9 89.2 79.1 98.2 100.0 73.9 11.0

83.3

Confidence interval ± 2.1 4.8 4.8 3.0 14.6 0.9 0.0 9.4 5.8

1.5

Table 10: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Northern Eurasia.

Northern Eurasia

Fo

rest

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

H

erb

ace

ou

s

Wet

lan

d

Lic

hen

/ m

oss

To

tal

Sa

mp

le

cou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 40.3 2.7 2.2 0.1 0.0 0.0 0.0 0.1 0.0 0.3 45.8 34.8 87.9 2.1

Shrubs 0.3 1.9 0.3 0.0 0.0 0.0 0.0 0.0 0.1 0.1 2.7 3.4 72.0 8.8


Croplands 0.4 0.1 1.6 4.9 0.0 0.0 0.0 0.0 0.0 0.0 7.0 7.1 70.8 6.8

Urban 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.2 3.4 60.0 9.7


Snow/ice 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.1 3.4 92.0 5.3

Permanent Water 0.1 0.0 0.0 0.0 0.0 0.1 0.0 3.0 0.0 0.0 3.2 3.4 94.0 4.7


Lichen/moss 0.0 0.1 0.1 0.0 0.0 0.1 0.0 0.0 0.0 0.6 0.9 3.4 70.0 9.0

Total 42.6 7.4 30.2 5.8 0.1 3.8 0.1 3.3 2.4 4.2 100.0

Sample count 33.2 6.8 33.5 5.8 2.1 4.6 3.1 3.5 2.2 5.2

100.0

Producer's accuracy 94.5 26.2 81 85.3 97.9 73.4 100.0 89 64.4 15.2

79.8


1.6




Table 11: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Europe.

Europe F

ore

st

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

Lic

hen

/ m

oss

To

tal

Sa

mp

le

cou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 41.0 1.5 1.5 1.6 0.3 0.0 0.0 0.0 0.0 0.1 46.1 39.2 89 1.8

Shrubs 1.2 0.7 0.4 0.1 0.0 0.0 0.0 0.0 0.0 0.0 2.4 4.1 29.2 10.9


Croplands 2.3 0.4 3.7 24.7 0.3 0.0 0.0 0.0 0.1 0.0 31.5 24.7 78.4 3.0

Urban 0.1 0.0 0.1 0.0 2.0 0.0 0.0 0.0 0.0 0.0 2.3 3.2 88.0 6.4


Snow/ice 0.0 0.0 0.0 0.0 0.0 0.1 0.3 0.0 0.0 0.0 0.4 3.2 85.9 6.9

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.0 0.0 2.1 3.2 97.0 3.4


Lichen/moss 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 9.1 5.7

Total 46.3 3.6 14.6 27.5 2.8 1.2 0.4 2.1 0.5 1.0 100.0

Sample count 39.8 4.0 15.0 21.6 3.8 5.3 3.7 3.9 1.7 1.3

100.0

Producer's accuracy 88.6 19.6 60.2 89.7 73.6 41.8 85.7 95.9 58.5 0.3

80.4


1.6

Table 12: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over North America.

North America

Fo

rest

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

Lic

hen

/ m

oss

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 31.3 1.5 0.9 0.3 0.2 0.0 0.0 0.0 0.2 0.2 34.6 38.0 90.4 2.0

Shrubs 1.8 7.5 1.6 0.1 0.0 0.0 0.0 0.0 0.1 0.2 11.3 11.3 66.3 5.7


Croplands 1.3 0.3 1.5 7.8 0.0 0.0 0.0 0.0 0.0 0.0 10.8 12.3 72.3 5.1

Urban 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.5 3.5 92 5.3

Bare/sparse vegetation 0.0 0.2 0.0 0.0 0.0 1.8 0.0 0.0 0.0 0.1 2.2 3.5 81 7.7

Snow/ice 0.0 0.0 0.0 0.0 0.0 0.5 10.4 0.0 0.0 0.0 10.9 3.9 95.3 3.8

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.1 0.0 4.1 0.0 0.0 4.3 3.5 96 3.9

Herbaceous Wetland 0.1 0.1 0.2 0.0 0.0 0.0 0.0 0.0 0.5 0.0 0.9 3.5 54 9.8

Lichen/moss 0.0 0.0 0.6 0.0 0.0 2.0 0.0 0.1 0.0 4.7 7.5 5.8 63.3 7.7

Total 35.3 12.6 13.4 8.6 0.7 5.0 10.5 4.4 1.3 8.4 100.0

Sample count 38.9 12.6 13.7 9.0 4.0 5.1 3.7 3.7 2.6 6.7

100.0

Producer's accuracy 88.7 59.3 64.4 90.9 63.8 35.8 99.4 94.8 36.4 56.5

77.1


1.7




Table 13: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over Oceania and

Australia.

Oceania and Australia F

ore

st

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 25.4 2.7 0.8 0.1 0.0 0.0 0.0 0.1 29.2 40.2 87.1 2.9

Shrubs 1.8 13.2 3.3 0.0 0.0 0.1 0.0 0.0 18.3 9.0 71.8 5.7

Herbaceous vegetation 1.9 4.4 38.5 0.5 0.0 0.6 0.0 0.1 46.0 30.0 83.7 2.8

Croplands 0.1 0.1 1.2 3.3 0.0 0.0 0.0 0.0 4.7 6.2 70.6 8.6

Urban 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.1 4.4 88.6 15.9

Bare/sparse vegetation 0.0 0.0 0.1 0.0 0.0 1.2 0.0 0.0 1.3 3.4 93.0 5.0

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.1 0.2 0.0 0.4 3.4 51.0 9.8

Herbaceous Wetland 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 3.4 34.0 9.3

Total 29.2 20.4 43.9 4.0 0.1 2.1 0.2 0.1 100.0

Sample count 39.7 11.6 29.5 6.0 4.8 5.0 2.0 1.2

100.0

Producer's accuracy 87.0 64.4 87.7 83.5 85.5 57.7 91.0 17.9

81.9

Confidence interval ± 2.9 5.6 2.5 7.4 15.9 12.6 11.1 20.8

1.9

Table 14: Confusion matrix (%) for the discrete CGLS_LC100 V2.0 Level 1 map over South America.

South America

Fo

rest

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Forest 49.3 2.7 1.1 0.6 0.0 0.0 0.0 0.0 0.2 53.9 45.4 91.5 1.4

Shrubs 2.0 6.9 1.7 0.1 0.0 0.5 0.0 0.0 0.2 11.4 10.1 59.9 5.7

Herbaceous vegetation 2.0 2.9 10.9 0.8 0.0 0.2 0.0 0.0 0.5 17.4 16.5 62.6 4.5

Croplands 1.1 0.4 1.5 6.0 0.0 0.1 0.0 0.0 0.0 9.0 9.0 65.9 5.8

Urban 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.3 3.3 89.0 6.2

Bare/sparse vegetation 0.0 0.5 0.6 0.0 0.0 4.7 0.1 0.0 0.0 5.9 5.7 79.4 6.5

Snow/ice 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.2 3.3 81.0 7.7

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.1 3.3 91.0 5.6

Herbaceous Wetland 0.0 0.1 0.1 0.0 0.0 0.0 0.0 0.1 0.5 0.8 3.3 59.0 9.7

Total 54.6 13.4 15.8 7.5 0.4 5.5 0.2 1.2 1.4 100.0

Sample count 46.9 12.8 15.0 7.1 3.3 5.8 2.8 3.6 2.8

100.0


79.6

Confidence interval ± 1.5 5.2 4.5 5.6 20.3 5.8 22.6 10.2 10.1

1.5

Class specific accuracies at each continent followed a similar trend as the class specific accuracies

at global scale. The class accuracies of forest and snow/ice are high for all continents. For

permanent water, accuracies are high for all continents except for Oceania & Australia (Table 13).

For bare/sparse vegetation, accuracies are high except in continents such as Europe (Table 11),

North America (Table 12), and Oceania & Australia (Table 13) where this class is underestimated.




Accuracy of herbaceous vegetation is moderate in all continents other than Oceania & Australia

where this class is mapped with high accuracy (Table 13). Cropland class is also mapped with

moderate accuracy, although this class tend to be overestimated in most continents. Similarly, for

urban class, the accuracies are relatively high except for in Africa (Table 8) and Northern Eurasia

(Table 10).

For all continents, herbaceous wetland class is mapped with low accuracy for all continents. For

lichen/moss class, accuracy is relatively better in North America (Table 12), while it tends to be

underestimated in Europe (Table 11) and Northern Eurasia (Table 10). This class has higher

confusion error with herbaceous vegetation and bare/sparse vegetation. Shrub class has lower

accuracies in Asia (Table 9), Northern Eurasia (Table 10) and Europe (Table 11), where it is

underestimated. In Europe, this class had high confusion error with forest and herbaceous

vegetation. The class accuracies are estimated mostly within range of 10% interval (confidence

interval). But some classes in continents have higher confidence interval. This could be related to

map misclassification at some validation sites that carry larger weights (sample sites representing

strata with larger areas – see section 3.4). These sample weights are used in confidence interval

calculation.

The accuracy of the CGLS_LC100m V2.0 discrete map is also calculated at Level 2 that separates

open and closed forests (Table 15).

Table 15: Confusion matrix for the discrete CGLS_LC100m V2.0 Level 2 map at global scale,

corrected by sample inclusion probaiblities.

Clo

sed

fo

rest

Op

en f

ore

st

Sh

rub

s

Her

ba

ceo

us

veg

eta

tio

n

Cro

pla

nd

s

Urb

an

Ba

re/s

pa

rse

veg

eta

tio

n

Sn

ow

/ice

Per

ma

nen

t

Wa

ter

Her

ba

ceo

us

Wet

lan

d

Lic

hen

/ m

oss

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Closed forest 22.2 3.9 0.8 0.5 0.1 0.0 0.0 0.0 0.0 0.1 0.0 27.6 28.1 80.4 1.2

Open forest 1.2 5.8 1.1 0.7 0.5 0.0 0.0 0.0 0.0 0.0 0.0 9.4 10.1 61.9 2.5

Shrubs 0.2 1.3 5.8 1.4 0.3 0.0 0.2 0.0 0.0 0.1 0.0 9.3 7.9 62.4 3.1

Herbaceous vegetation 0.1 1.1 2.3 13.8 0.6 0.0 0.5 0.0 0.1 0.3 1.0 20.0 19.2 69.2 1.8

Croplands 0.2 1.0 0.4 1.6 8.2 0.1 0.0 0.0 0.1 0.1 0.0 11.6 12.6 70.2 2.1

Urban 0.0 0.1 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.6 3.9 77.3 5.7

Bare/sparse vegetation 0.0 0.0 0.3 0.8 0.0 0.0 13.5 0.0 0.0 0.0 0.0 14.7 6.5 91.5 1.8

Snow/ice 0.0 0.0 0.0 0.0 0.0 0.0 0.1 1.9 0.0 0.0 0.0 2.0 2.4 94.7 3.5

Permanent Water 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 2.0 0.0 0.0 2.1 4.0 94.9 2.0

Herbaceous Wetland 0.0 0.1 0.1 0.3 0.0 0.0 0.0 0.0 0.1 0.5 0.1 1.1 3.7 46.5 6.3

Lichen/moss 0.0 0.0 0.0 0.1 0.0 0.0 0.4 0.0 0.0 0.0 0.9 1.4 1.7 63.9 7.0

Total 23.9 13.3 10.9 19.2 9.7 0.7 14.7 2.0 2.3 1.2 2.2 100.0

Sample count 24.5 14.4 9.2 19.4 10.6 3.6 7.3 2.3 4.3 2.6 1.8

100.0

Producer's accuracy 92.8 43.8 53.4 71.8 83.9 76.1 91.7 98.9 86.8 43.7 42

75.1

Confidence interval ± 0.8 2.1 2.8 1.8 2.0 7.6 1.4 0.9 3.2 6.4 5.3

0.7




At Level 2, when open and closed forests are separated, the overall accuracy at global scale is

75.1% +/-0.7. All the class accuracies are the same as Level 1 (Table 6), except for the open forest

and closed forest classes. Here, closed forest class has higher accuracy than open forest class.

However, closed forest class tends to be overestimated at the cost of open forest class.

Overall accuracies are also calculated at Level 2 for each continent. Continental level overall

accuracies range from 71% (Northern Eurasia) to 79.5% (Asia) (Table 16).

Table 16: Overall accuracy of the discrete CGLS_LC100 V2.0 Level 2 map per continent.

Number of samples Overall accuracy Confidence interval ±

Africa 3616 76.5 2.0

Asia 3071 79.5 1.6


Europe 3120 73.5 1.7




Fractional land cover layers

The CGLS_LC100m V2.0 fractional land cover layers, namely tree cover, shrub cover, herbaceous

vegetation cover, crops, lichen/moss, bare/sparse vegetation, snow/ice, built-up and water, were

assessed using the cover fraction information in the validation dataset. Table 17 lists the mean

absolute error (MAE) and root mean square error (RMSE) for the fractional land cover layers.

Table 17: Accuracy of the fractional land cover layers at global scale.

Trees Shrub

Herbaceous

vegetation Crops

Lichen/

Moss

Bare and sparse

vegetation Snow/ice Built-up Water

Mean absolute error %

(MAE) 9.0 9.4 17.3 5.1 2.9 5.6 0.1 0.8 0.8

Root mean square error

% (RMSE) 17.9 17.4 28.1 15.8 15.1 15.6 3.3 5.7 5.9

Among the fractional land cover layers, rare classes, such as snow/ ice, water and built-up area,

show the lowest errors with an average deviation (MAE) from the validation data of maximum 0.8%

and RMSE lower than 6%, followed by lichen/moss, crops and bare/sparse vegetation fractions

layers (MAE ~ 5% and RMSE ~ 15%) and trees and shrubs fraction layers (MAE ~ 9%, RMSE ~

17%). Herbaceous vegetation fraction product has the highest error with MAE of 17.3% and RMSE

of 28.1%. This can be due to difficulty in separating herbaceous vegetation from other land cover

types.




MAE and RMSE were also calculated at continental scale (Table 18).

Table 18: Accuracy of the fractional land cover layers at continental scale.

MAE (%)

Trees Shrub Herbaceous

vegetation

Crops Lichen/

moss

Bare sparse

vegetation

Snow/

ice

Built-up Water

Africa 7.9 11.4 15.6 5.7 0.0 4.9 0.0 0.7 0.4

Asia 7.8 5.9 12.6 7.4 0.1 5.5 0.1 1.6 0.8

Northern Eurasia 11.8 8.4 23.4 2.8 5.7 3.9 0.0 0.3 1.1

Europe 11.7 7.9 18.0 10.7 1.1 2.6 0.2 2.6 0.6

North America 8.3 9.6 20.2 3.7 10.7 8.2 0.6 0.6 1.3

Oceania &

Australia 8.9 11.8 18.5 2.3 0.1 9.1 0.0 0.2 0.3

South America 9.7 11.4 15.8 4.9 0.2 4.9 0.1 0.3 0.7

RMSE (%)

Trees Shrub Herbaceous

vegetation

Crops Lichen/

moss

Bare sparse

vegetation

Snow/

ice

Built-up Water

Africa 14.9 17.2 24.4 17.0 0.5 13.9 0.0 6.1 4.4

Asia 17.0 12.9 22.0 18.1 2.2 14.3 2.1 8.0 6.7

Northern Eurasia 23.8 20.3 37.6 12.1 21.7 13.7 1.0 3.5 6.8

Europe 17.8 14.0 25.9 20.3 8.6 9.1 3.3 9.9 4.0

North America 16.6 18.3 32.2 13.5 29.4 20.9 7.0 4.6 6.6

Oceania &

Australia

14.8 16.5 24.9 10.7 2.9 19.0 0.0 2.3 4.7

South America 18.7 20.0 25.6 16.5 3.3 14.4 2.8 3.0 6.1

Tree fraction layer has lower errors in Africa, Asia, North America and Oceania & Australia,

followed by South America, Northern Eurasia and Europe. Shrub fraction layer has lower MAE in

Asia and Europe. Herbaceous vegetation has highest errors for all the continents comparing to

other classes. Among different continents, Northern Eurasia and North America have higher errors

while Asia, Africa and South America have lower errors. Crop fraction layer has the highest errors

in Europe while the errors are lower in Oceania & Australia, Northern Eurasia and North America.

Lichen moss class has higher errors in North America and Northern Eurasia. For bare and sparse

vegetation layer, the errors are higher in Oceania & Australia and North America, while the errors

are less in Northern Eurasia and Europe. For all continents, snow/ice, built-up, and water fraction

layers have less errors compared to other fraction layers.




4.2 COMPARISON WITH OTHER GLOBAL LAND COVER MAPS

4.2.1 Qualitative comparison

Visual comparison of the CGLS_LC100m V2.0 map and other recent global land cover map was

performed in order to investigate the visual agreement and disagreement between the maps. This

is also intended to increase the user confidence in the CGLS_LC100m V2.0 product.

A visual comparison shows that the CGLS_LC100m V2.0 discrete map agrees with the LC-CCI

2015 and Globeland30 2010 in terms of spatial distribution of the main land cover types. Due to

similarities in the legend, the general pattern of the land cover types in the CGLS_LC100m V2.0

discrete map looks like that of the Globeland30 2010 map. The LC-CCI 2015 map patterns look

differently because it characterizes more land cover types (see Table 4). As expected, there are

also disagreements between the products in some parts of the world. Some typical examples are

shown on Figure 10, Figure 11 and Figure 12 below.

The separation of natural vegetation appears to be good with the CGLS LC100m V2.0 map. For

example, central part of Madagascar is mapped as ‘mosaic herbaceous cover (>50%) / tree and

shrub (<50%)’ in the LC-CCI 2015 while it is mostly grassland as shown in the CGLS LC100m

V2.0 and Globeland30 2010 (Figure 10, left). Figure 10 (left) also reveals that CGLS LC100m map

shows a good agreement with the Globeland30 2010 map for forest. While the LC-CCI 2015

considers some parts as tree cover while the other parts as “mosaic natural vegetation”, which is

mixed trees, shrub, grass and cropland. Similarly, in Northern Amazonia in South America, large

grassland regions are captured in the CGLS LC100m V2.0 and Globeland30 2010, while the area

is mapped as mostly shrubs in the LC-CCI 2015 (Figure 10, right). The CGLS_LC100m V2.0 maps

less area as wetland than the other two maps (Figure 10, right). This could be partly due to

underestimation of wetland areas and partly due to the legend difference between the maps as the

CCI-LC2015 and Globeland30 2010 wetland classes include flooded trees and shrubs, while the

CGLS_LC100m V2.0 depicts only herbaceous wetland.

Figure 11 shows the comparisons in Rondonia and Mato Grosso, Brazil (left), and one region on

the Kazakhstan-Uzbekistan border (right). In Rondonia and Mato Grosso, most areas of grassland

captured by the CGLS LC100m V2.0 map have a good agreement with those detected by the

Globeland30 2010 map, while this area is mapped mostly as ‘mosaic cropland (>50%) / natural

vegetation’ in the LC-CCI 2015. The spatial pattern of cropland delineation shows agreement

among the three products. In Kazakhstan-Uzbekistan (Figure 11, right), the spatial pattern of

cropland type agrees between CGLS_LC100m and Globeland30m. The LC-CCI 2015 however

detected more cropland areas. Similarly, the LC-CCI 2015 also detected more bare / spare

vegetation when combining bare class and spare vegetation class comparing to the

CGLS_LC100m V2.0. The Globeland30 map may not comparable directly because it contains only

bare class. Furthermore, shrubland areas in Kyzyl Kum are delineated in the CGLS_LC100m V2.0

and LC-CCI 2015 while it was mostly seen as grassland in the Globeland30 map. Regarding to

grassland, the CGLS_LC100m V2.0 shows relative good agreements with the Globeland30m.

However the LC-CCI 2015 detected much less grassland, where it maps as shrubland, bare and




spare vegetation. The CGLS_LC100m V2.0 detected less permanent water and wetlands

comparing to the other two maps, such as in the Aral Sea.

Figure 10: Comparison of three maps in Madagascar (left) and Northern Amazonia, South America


Globeland30 map, respectively).




Figure 11: Comparison of three maps in Rondonia and Mato Grosso, Brazil (left) and Uzbekistan


Globeland30 map, respectively).




Figure 12: Comparison of three maps in Kalimantan and Sulawesi, Indonesia (left), and Cuba (right)

(legends are shown in Table 3, Table 4 and Figure 9 for the CGLS_LC100m, LC-CCI, Globeland30 map,

respectively).




Figure 12 shows the comparisons in Kalimantan and Sulawesi, Indonesia (left) and Cuba (right). In

Indonesia, the CGLS_LC100m V2.0 map depicts less cropland areas than the other two products.

This is the same situation for the city of Banjamarsin and southern tip of Sulawesi Selatan. Similar

tendency of depicting less cropland areas than the two other products is also visible in Cuba. This

could because the CGLS_LC100m V2.0 considers pasture areas as herbaceous vegetation while

the Globeland30 includes pasture as cropland. The CGLS_LC100m V2.0 map detected much

more forests in Cuba as compared to the other two products.

Visual inspection of the CGLS_LC100m V2.0 further reveals some artefacts and misclassifications.

For example, there are artefacts in the CGLS_LC100m V2.0 in the open water of Caspian sea

(Figure 13a). The Level 3 legend of the discrete map depicts considerably large forest areas

without forest type specifications, “unknown type” (darker green) (Figure 13b).

a. Caspian sea b. Northern Malaysia

Figure 13: Examples of artefacts and inconsistencies (related to unknown forest type) in the

CGLS_LC100 V2 discrete map




4.2.2 Quantitative comparison

Comparison using a combined validation dataset

Tsendbazar et al. (2016) combined six available reference datasets consisting of around 25000

points and checked the correspondence of the resulting dataset with the Globcover 2009, LC-CCI

2010, MODIS2010 and Globeland30 2010 maps. The correspondence of the CGLS_LC100m V2.0

maps was also evaluated with this combined dataset following the same procedure. Table 19 lists

the overall agreements of the maps at global and continental scales (see Tsendbazar et al. (2016)

for the division of the continents).

Table 19: Agreement of combined reference points with four previous global land cover maps and

the CGLS_LC100m V2.0 map.

Number of

samples

Globcover

2009

LC-CCI

2010

MODIS

2010

Globeland30

2010

CGLS_LC100m

V2.0 2015

Global scale 24593 60.0 66.7 69.4 67.3 72.3

Eurasia 10353 65.4 71.1 70.0 72.3 73.6

North America 4686 56.5 67.6 70.1 65.2 73.5

Australia &

Oceania 2381 49.7 59.0 69.6 61.8 69.8

Africa 3881 50.8 55.4 62.9 57.2 68.0

South America 3292 66.2 70.0 73.9 70.6 73.1

At global scale, CGLS_LC100m V2.0 2015 map has higher agreement (by 3%) with the combined

reference dataset than the other four global land cover maps. Similarly, at continental scale, the

CGLS_LC100m V2.0 2015 map has consistently higher agreement than the other maps. Only in

South America, MODIS 2010 map has slightly higher accuracy than the CGLS_LC100m V2.0 map.

MODIS 2010 map also has similar accuracies as the CGLS_LC100m V2.0 map in Australia &

Oceania, but for other continents, agreement with the combined validation dataset is lower than

that of the CGLS_LC100m V2.0 map.

It is worth to note that the objective of this comparison is only to indicate generic quality of the

CGLS_LC100m V2.0 map as compared to commonly used previous global land cover maps. One

must be cautious to compare directly the overall agreement of these maps due to the differences in

the reference years and legend definitions and other differences between the combined reference

dataset and the global land cover maps (Tsendbazar et al. 2016). This result shows that the

CGLS_LC100m V2.0 has a comparable, if not better, quality as the previous land cover maps.

Comparison using the CGLS_LC100 validation dataset

We used the CGLS_LC100 validation dataset, described in Section 3.3, to assess the accuracy of

the Globeland30 2010 map after translating the reference land cover classes into the legend of the




Globeland30 2010 map using the land cover fraction information of our validation dataset. The

confusion matrix of the Globeland30 2010 map is provided in Table 20.

Overall accuracy is 68.7% +/-0.9% (at 95% confidence level) which lower compared to the overall

accuracy of the CGLS_LC100m V2.0 map (80.2% +/-0.7%, Table 6). The accuracy of water

bodies, bare land and permanent snow classes are high and comparable to those of the

CGLS_LC100m V2 (Table 6). The producer’s accuracy forest class is substantially lower than that

of the CGLS_LC100m V2.0 map, indicating higher omission error for the Globeland30 map.

Cultivated land has lower user’s accuracy compared to that of the CGLS_LC100m V2.0, indicating

higher commission error. Classes such as grassland, shrubland and wetland have lower

accuracies that compared to the CGLS_LC100m V2.0 map. The results in Table 20 show that the

grassland and cultivated land have high confusion at global scale. This could be explained

because the pasture areas in Europe, which are considered as grasslands in the validation data,

are commonly mapped as cultivated land according to its definition on cropland.

Table 20: Confusion matrix (%) for the Globeland30 2010 map at global scale, calculated using the

CGLS_LC100m validation data (corrected by sample inclusion probaiblities)

Cu

ltiv

ate

d l

an

d

Fo

rest

Gra

ssla

nd

Sh

rub

lan

d

Wet

lan

d

Wa

ter

bo

die

s

Art

ific

ial

surf

ace

s

Ba

re l

an

d

Per

ma

nen

t

sno

w

To

tal

Sa

mp

le c

ou

nt

Use

r's

acc

ura

cy

Co

nfi

den

ce

inte

rva

l ±

Cultivated land 8.6 2.8 2.1 0.8 0.1 0.0 0.1 0.1 0.0 14.7 17.5 58.9 20

Forest 0.3 27.5 0.8 1.2 0.1 0.0 0.0 0.0 0.0 29.9 30.7 91.8 0.8

Grassland 0.8 6.3 11.4 4.7 0.2 0.0 0.0 1.2 0.0 24.6 22.3 46.3 1.8

Shrubland 0.2 2.2 0.9 3.5 0.1 0.0 0.0 0.1 0.0 7.1 7.4 49.2 3.3

Wetland 0.0 1.3 0.4 0.4 0.4 0.0 0.0 0.1 0.0 2.7 3.7 14.6 3.0

Water bodies 0.0 0.1 0.1 0.0 0.1 2.0 0.0 0.0 0.0 2.4 4.6 84.4 3.4

Artificial surfaces 0.1 0.3 0.2 0.0 0.0 0.0 0.5 0.0 0.0 1.1 4.6 48.7 7.1

Bare land 0.1 0.1 1.6 0.7 0.0 0.0 0.0 12.4 0.0 15.0 6.3 82.6 2.6

Permanent snow 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.2 2.3 2.6 2.9 91.4 3.6

Total 10.2 40.6 17.6 11.3 0.9 2.2 0.7 14.1 2.4 100.0

Sample count 11.1 42.3 17.8 9.7 2.4 4.1 3.7 6.4 2.5

100.0


68.7

Confidence interval ± 2.0. 1.2 2.1 2.6 7.6 2.6 7.5 2.1 0.4

0.9




Table 21: Overall accuracy of the Globeland30 2010 map at continental level

Number of samples Overall accuracy (%) Confidence intervals ±

Africa 3611 61.7 2.5

Europe 3024 66.5 1.9


Asia 3062 74.3 1.8




The overall accuracy of the Globeland30 2010 map was also assessed at continental level (Table

21). Continental accuracy varied from 56.7% in Oceania & Australia to 74.5% in North America.

For all continents, the accuracy of the Globeland30 2010 was consistently lower than that of

CGLS_LC100m V2.0 Level1 map (see Table 7). This could be partially due to the inconsistencies

in the class definition when translating the validation dataset used for this assessment to the

legend of the Globeland30 2010 map, and partially due to temporal gap between the reference

dataset (2015) and the map (2010).




5 CONCLUSIONS

This document reports the validation process of the CGLS_LC100m Dynamic Land Cover V2.0

product of the Copernicus Global Land Service. The discrete land cover map and the nine cover

fraction layers were validated using an independent validation dataset containing around 21 600

points generated in collaboration with regional experts.

Our assessments showed that the overall accuracy of the discrete CGLS_LC100m V2.0 Level 1

map reaches 80.2+/-0.7%. Targeted overall accuracy 80% (Chapter 2) has therefore been met.

Forest, bare/sparse vegetation, snow/ice and permanent water classes are mapped with very high

accuracies (>85%). The class accuracies of herbaceous vegetation, croplands, urban are

moderate, while herbaceous wetland, lichen and moss and shrubs classes have lower accuracies

(<65%). For the continents, accuracies are also around 80% for the most of the continents, with

exceptions to North America (77.1 ± 1.7) and Asia (83.3± 1.5). At Level 2, when closed and open

forests classes are separated, global overall accuracy is 75.1% +/-0.7%, ranging from 71-79.5%

for the different continents.

Among the fractional land cover layers, snow & ice, built-up, water and lichen/moss fraction maps

show lowest errors, followed by crops and bare and sparse vegetation fraction types. On the other

hand, fractional herbaceous vegetation cover layer has the highest error.

Our qualitative and quantitative comparisons show that the CGLS_LC100m V2.0 discrete map has

higher accuracy than other available global land cover map. Quantitative comparison using a

combined reference dataset (Tsendbazar et al. 2016) shows the CGLS_LC100m V2.0 map has

3% higher agreement than the other global land cover maps. At continental level, the discrete

CGLC-LC100m V2.0 map also consistently outperforms the other maps. The exception is South

America, where MODIS 2010 has marginally higher agreement with the combined reference data.

In addition, a comparison of the Globeland30 2010 map using our CGLS_LC100m validation

dataset also shows that the discrete CGLS_LC100m V2.0 map has at least 10% higher accuracy

than the Globeland30 2010 map. The quantitative comparisons have limitations due to the

discrepancy between the validation datasets and the global land cover maps in terms of legend

definitions and reference periods.

The visual comparison of the CGLS_LC100m V2.0, the LC-CCI 2015 and the Globeland30 2010

maps shows good agreements between the CGLS_LC100m V2.0 and the Globeland30 2010 map

for the characterization of natural vegetation classes and croplands. Classes such as wetland and

cropland tend to have smaller extent in the CGLS_LC100 V2.0 map as compared to the other

maps. This can be partially attributed to the differences in the class definitions.




6 RECOMMENDATIONS

Validation of the CGLS_LC100m V2.0 land cover product shows noticeable improvements in land

cover mapping as compared with other existing products. These achievements can be further

improved by focusing on the main limitations.

Artefacts observed in Caspian sea should be addressed for improvements. In addition,

inconsistencies in characterizing forest types (unknown forest type) should also be further taken

into actions for improvements. It might also necessary to collect additional training data on tree

types. Characterization of classes such as wetlands, lichen/moss and shrubs needs improvement

as they have low class specific accuracies. Wetland areas are often found bordering waterbody

and cropland classes. Better identification of wetland areas could make use of existing global

wetland layers (e.g. Global saltmarsh by McOwen et al., 2017 and Peatmap by Xu et al., 2018)

either at the model training stage or at output refining stage after checking their reliability.

Improvement efforts should also take into account lichen and moss classes in Europe and

Northern Eurasia where the accuracy of this class is low. Existing regional land cover maps such

as Corine CLC of Europe could be used. Herbaceous vegetation fraction layer needs improvement

as its error is the largest among the other fraction layers. In addition to PROBA-V data, integrating

other remote sensing data with more spectral bands might help to solve this issue.

The global CGLS_LC100m product is planned to be released yearly. Consequently, attention

should be paid to reduce year to year inconsistency between the yearly land cover products,

minimizing false land cover change detection as much as possible. For this purpose,

inconsistencies between the yearly land cover maps should be assessed and additional training

datasets could be collected in the inconsistent areas which can be useful for improving the land

cover mapping as well as land cover change detection. Also, there is a need to collect up-to-date

validation data for a more reliable validation of the yearly maps.

The validation presented in this document meets the Stage 3 validation requirement of the CEOS

WGCV. This validation should be progressed towards Stage 4 validation requirement which aims

at updating the validation as land cover maps for the following years are released.




7 REFERENCES Bossard, M., Feranec, J., and Otahel, J.: CORINE land cover technical guide: Addendum 2000.

Chen, J., et al. (2015). "Global land cover mapping at 30m resolution: A POK-based operational approach." ISPRS Journal of Photogrammetry and Remote Sensing 103: 7-27.

CCI-LC. CCI-LC Product User Guide; UCL-Geomatics: Louvain-la-Neuve, Belgium, 2014.

Foody, G. M.: Approaches for the production and evaluation of fuzzy land cover classifications from remotely-sensed data, Int. J. Remote Sens., 17, 1317-1340, 10.1080/01431169608948706, 1996.

Geoscience Australia: Dynamic Land Cover Dataset. Bioregional Assessment Source Dataset. http://data.bioregionalassessments.gov.au/dataset/1556b944-731c-4b7f-a03e-14577c7e68db.

Jung, M., Henkel, K., Herold, M., and Churkina, G.: Exploiting synergies of global land cover products for carbon cycle modeling, Remote Sensing of Environment, 101, 534-553, 2006.

Latifovic, R., Zhu, Z.-L., Cihlar, J., Giri, C., and Olthof, I.: Land cover mapping of North and Central America—Global Land Cover 2000, Remote Sensing of Environment, 89, 116-127, https://doi.org/10.1016/j.rse.2003.11.002, 2004.

McOwen, C.J., Weatherdon, L.V., Van Bochove, J.W., Sullivan, E., Blyth, S., Zockler, C., Stanwell-Smith, D., Kingston, N., Martin, C.S., Spalding, M., & Fletcher, S. (2017). A global map of saltmarshes. Biodiversity Data Journal, 5 Olofsson, P., Stehman, S. V., Woodcock, C. E., Sulla-Menashe, D., Sibley, A. M., Newell, J. D., Friedl, M. A., and Herold, M.: A global land-cover validation data set, part I: fundamental design principles, Int. J. Remote Sens., 33, 5768-5788, 10.1080/01431161.2012.674230, 2012.

Pengra, B., Long, J., Dahal, D., Stehman, S. V., and Loveland, T. R.: A global reference database from very high resolution commercial satellite data and methodology for application to Landsat derived 30m continuous field tree cover data, Remote Sensing of Environment, 165, 234-248, https://doi.org/10.1016/j.rse.2015.01.018, 2015.

Stehman, S. V., Wickham, J., Smith, J., and Yang, L.: Thematic accuracy of the 1992 National Land-Cover Data for the eastern United States: Statistical methodology and regional results, Remote Sensing of Environment, 86, 500-516, 2003.

Stehman, S. V.: Estimating area and map accuracy for stratified random sampling when the strata are different from the map classes, Int. J. Remote Sens., 35, 4923-4939, 2014.

Strahler, A. H., Boschetti, L., Foody, G. M., Friedl, M. A., Hansen, M. C., Herold, M., Mayaux, P., Morisette, J. T., Stehman, S. V., and Woodcock, C. E.: Global land cover validation: Recommendations for evaluation and accuracy assessment of global land cover maps, European Communities, Luxembourg, 51, 2006.

Tsendbazar, N.-E., de Bruin, S., and Herold, M.: Integrating global land cover datasets for deriving user-specific maps, Int. J. Digit. Earth, 1-19, 10.1080/17538947.2016.1217942, 2016.

Tsendbazar, N. E., Herold, M., de Bruin, S., Lesiv, M., Fritz, S., Van De Kerchove, R., Buchhorn, M., Duerauer, M., Szantoi, Z., and Pekel, J. F.: Developing and applying a multi-purpose land cover validation dataset for Africa, Remote Sensing of Environment, 219, 298-309, https://doi.org/10.1016/j.rse.2018.10.025, 2018.

https://doi.org/10.1016/j.rse.2003.11.002






Walker, D. A., Raynolds, M. K., Daniëls, F. J., Einarsson, E., Elvebakk, A., Gould, W. A., Katenin, A. E., Kholod, S. S., Markon, C. J., and Melnikov, E. S.: The circumpolar Arctic vegetation map, Journal of Vegetation Science, 16, 267-282, 2005.

Wickham, J., Stehman, S., Fry, J., Smith, J., and Homer, C.: Thematic accuracy of the NLCD 2001 land cover for the conterminous United States, Remote Sensing of Environment, 114, 1286-1296, 2010.

Xu, J.R., Morris, P.J., Liu, J.G., & Holden, J. (2018). PEATMAP: Refining estimates of global peatland distribution based on a meta-analysis. Catena, 160, 134-140