noaa national coastal data development center writing fgdc compliant metadata introduction to...

NOAA National Coastal Data Development CenterNOAA National Coastal Data Development Center

Writing FGDC Compliant MetadataWriting FGDC Compliant Metadata

Introduction to Geospatial Metadata

Introduction to Geospatial Metadata

Metadata 101

What is Metadata?

Why is it Important?

Metadata is information about data

Metadata is information about data

What is Metadata?What is Metadata?

Metadata functions

• documentation

• management

• discovery

• access

• use

Metadata functions

• documentation

• management

• discovery

• access

• use

Who, What, Where,

When, Why, and How

Who, What, Where,

When, Why, and How


WhoWho collected the data?Who processed the data?Who wrote the metadata?Who to contact for questions?Who to contact to order?Who owns the data?


WhereWhere were the data collected?Where were the data processed?Where are the data located?


WhatWhat are the data about?What project were they collected under?What are the constraints on their use?What is the quality?What are appropriate uses?What parameters were measured?What format are the data in?


WhenWhen were the data collected?When were the data processed?


HowHow were the data collected?How were the data processed?How do I access the data?How do I order the data?How much do the data cost?How was the quality assessed?

HowHow were the data collected?How were the data processed?How do I access the data?How do I order the data?How much do the data cost?How was the quality assessed?Why

Why were the data collected?

WhyWhy were the data collected?


We often use metadata without even knowing it.

We often use metadata without even knowing it.

Examples?Examples?


This is the metadata for this.

What’s Missing?

What is Metadata?

Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, 1997. Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction

Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, 1997. Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction

While the card-catalog entry is a form of metadata, it does not address topics such as

quality, accuracy, or scale. Well-written geospatial metadata describes these and many more aspects of the data.

What is Metadata?


Identification_Information Citation Citation_Information Originator: NOAA Coastal Services Center Publication_Date: 19971131 Title: Hurricane Storm Surge Geospatial_Data_Presentation_Form: Map Publication_Information Publication_Place: Charleston, SC Publisher: NOAA Coastal Services Center Larger_Work_Citation Citation_Information

Identification_Information Citation Citation_Information Originator: NOAA Coastal Services Center Publication_Date: 19971131 Title: Hurricane Storm Surge Geospatial_Data_Presentation_Form: Map Publication_Information Publication_Place: Charleston, SC Publisher: NOAA Coastal Services Center Larger_Work_Citation Citation_Information

What is Metadata?


Metadata contains vital information

Without a label, how would you know which

one to open?

Dog Food?

Ravioli?

Imagine being given two identical cans.Your task is to choose one to eat. But here’s the catch. Neither can has a label. Which would you choose?

Metadata as a Component of Data

Metadata as a Component of Data

Properly documented data

provides vital information to

interestedparties.

A Component of Data

A published map contains elements of metadata…

• Publisher

• Publication date

• Type of map

• Title / Description

• Spatial references

• Scale and accuracy

• Sources

• Legend

A Component of Data

A Component of Data

Metadata is that component of data which describes it.

Environmental Sensitivity Index Data

Metadata

RARNUM - unique combination of species, concentration, and seasonality

CONC (concentration) = Density species is found at location

Season_ID = seasonality code like to the seasonal table

Element - Biology group

A Component of Data

Metadata describes…Metadata describes…

• CONTENT

• CONDITION

• QUALITY

• CONTENT

• CONDITION

• QUALITY

Characteristics of the data

A Component of DataA Component of Data

Because metadata provides vital information about a dataset, it should

never be viewed or treated as a separate entity.

Because metadata provides vital information about a dataset, it should

never be viewed or treated as a separate entity.

Metadata is a critical component

of a complete data set.

Metadata is a critical component

of a complete data set.

A Component of DataA Component of Data

Metadata should be updated to reflect changes in the data

1980

HEWTeheran

British HondurasCape Hatteras Light

Mt. St. HelenWest Germany

2005

HHS & HUDTehranBelizeCape Hatteras LightMt. St. HelenGermany

The Valueof Metadata

The Valueof Metadata

(Why Metadata?)(Why Metadata?)

Properly documenting a data set is the key to preserving its

usefulness through time.

Standardized

Metadata at a

Glance

Standardized

Metadata at a

Glance

Establishing the FGDC Standard Establishing the FGDC Standard

Metadata - Documentation of geospatial data written in a consistent manner

FGDC - Federal Geographic Data Committee

CSDGM - Content Standard for Digital Geospatial Metadata, referred to commonly as “The Standard ” or “The Content Standard ”

Clearinghouse - A distributed catalog of metadata

Geospatial - refers to a geographic location

Metadata - Documentation of geospatial data written in a consistent manner

FGDC - Federal Geographic Data Committee

CSDGM - Content Standard for Digital Geospatial Metadata, referred to commonly as “The Standard ” or “The Content Standard ”

Clearinghouse - A distributed catalog of metadata

Geospatial - refers to a geographic location

Some Terms We Will be Using

Some Terms We Will be Using

The Federal Geographic Data Committee (FGDC) was organized

in 1990 under the Office of Management and Budget to

promote the coordinated use, sharing, and dissemination of geospatial data on a national

basis. The FGDC was tasked with creating a metadata standard to

meet these objectives.

The Federal Geographic Data Committee (FGDC) was organized

in 1990 under the Office of Management and Budget to

promote the coordinated use, sharing, and dissemination of geospatial data on a national

basis. The FGDC was tasked with creating a metadata standard to

meet these objectives.

The Metadata StandardThe Metadata Standard

Establishing the StandardEstablishing the Standard

"... each agency shall document all new geospatial data it collects or produces, either directly or indirectly, using the standard under development by the FGDC, and make that standardized documentation electronically accessible to the Clearinghouse network."

"... each agency shall document all new geospatial data it collects or produces, either directly or indirectly, using the standard under development by the FGDC, and make that standardized documentation electronically accessible to the Clearinghouse network."

The Content Standard for Digital Geospatial Metadata (CSDGM)The Content Standard for Digital Geospatial Metadata (CSDGM)

Executive Order 12906, signed by President Clinton in 1994.

Executive Order 12906, signed by President Clinton in 1994.


The Content Standard helps

the user determine...

The Content Standard helps

the user determine...

• If a set of geospatial data is available and fit for a particular use

• How to access and transfer the data set

• If a set of geospatial data is available and fit for a particular use

• How to access and transfer the data set


The Content Standard utilizes...

• Common terms

• Common definition

• Common language

• Common structure

The Content Standard utilizes...

• Common terms

• Common definition

• Common language

• Common structure

Access

constraints

Citation

currentness

entity

attrib

utedomain

lineage

Process step


The Content Standard…

• Establishes names of compound elements and data elements

• Defines the information about the values that are to be provided for the data elements

The Content Standard…

• Establishes names of compound elements and data elements

• Defines the information about the values that are to be provided for the data elements

Type=“free text”Domain=Real

currentnessentity

attributeCitation

lineage


This “Content Standard” serves as a uniform

summary description of the data set.

This “Content Standard” serves as a uniform

summary description of the data set.










HowHow were the data collected?How were the data processed?How do I access the data?How do I order the data?How much do the data cost?How was the quality assessed?

HowHow were the data collected?How were the data processed?How do I access the data?How do I order the data?How much do the data cost?How was the quality assessed?Why

Why were the data collected?

WhyWhy were the data collected?

Metadata written using the Content Standard answers these important

questions:

Metadata written using the Content Standard answers these important

questions:


What does a metadata record

look like?

What does a metadata record

look like?

A Sample Metadata Record – Text VersionA Sample Metadata Record – Text Version

A Sample Metadata Record – HTML VersionA Sample Metadata Record – HTML Version

A Sample Metadata Record – XML VersionA Sample Metadata Record – XML Version

The FGDCWorkbook

Organization and Content

The FGDCWorkbook

Organization and Content

FGDC’s Metadata Workbook

FGDC’s Metadata Workbook

• Parallels the Standard

• Defines 334 available metadata elements

• Parallels the Standard

• Defines 334 available metadata elements

• A resource for applying the FGDC Content Standard• A resource for applying the FGDC Content Standard

• Uses a Graphical Representation of the Production Rules

• Uses a Graphical Representation of the Production Rules

• Describes domain values(valid values that can be

assigned to the data element)

• Describes domain values(valid values that can be

assigned to the data element)

• Provides section and element definitions• Provides section and element definitions

Using the WorkbookUsing the Workbook

The Workbook uses graphics to illustrate the production rules of the standard.

The graphics include most of the information provided by the production rules:

The Workbook uses graphics to illustrate the production rules of the standard.

The graphics include most of the information provided by the production rules:

• How elements are grouped

• What is mandatory and what is not

• What elements can repeat and how

• many times they can repeat

• How elements are grouped

• What is mandatory and what is not

• What elements can repeat and how

• many times they can repeat

Using the Graphical RepresentationUsing the Graphical Representation

SectionSections are depicted

by this symbol

Compound Element

Compound elements are depicted using a 2-dimensional box

Data Element

Data elements are depicted using a 3-dimensional box with shadow

Using the Graphical Representation

A section is composed of groups of data elements or other compound elements that describe that particular section.

There are 7 main Sections• Identification InformationIdentification Information• Data Quality InformationData Quality Information• Spatial Organization InformationSpatial Organization Information• Spatial Reference InformationSpatial Reference Information• Entity and Attribute InformationEntity and Attribute Information• Distribution InformationDistribution Information• Metadata Reference InformationMetadata Reference Information

And 3 supporting Sections• Citation InformationCitation Information• Time Period InformationTime Period Information• Contact InformationContact Information

Section


A compound element is group of data elements or other compound elements.

The form for the definition of a compound element is:

Compound element name -- definition.Compound element name -- definition.Type:Type: compound compound

An example of a definition of a compound element is:

Description - characterization of the data set, Description - characterization of the data set, including its intended use and limitationsincluding its intended use and limitations

Type:Type: compound compound

Compound Element


Data Element

A data element is a logically primitive itemof data. Data elements are the things thatyou “fill in.”

The form for the definition of a data element is:

Data element name -- definition.Data element name -- definition.Type:Type: (choice of “integer”, “real”, “text”, “date”, or “time”) (choice of “integer”, “real”, “text”, “date”, or “time”)Domain: (describes valid values that can be assigned)Domain: (describes valid values that can be assigned)

An example of the definition of a data element is:

Abstract – a brief narrative of the data set.Abstract – a brief narrative of the data set.Type:Type: text textDomain: free text Domain: free text


CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

Compound elements are composed of other compoundand data elements.

The composition is representedby nested boxes.


Mandatory - must be provided.

MeaningData

ElementCompound

Element

What’s Mandatory? What’s Not?

Mandatory if Applicable - must be provided if the data set exhibits

the defined characteristic.

Optional - provided at the discretion of the data set producer.


Using the Graphics to Make Decisions

All elements are colored yellow, so all are mandatory and must be filled in.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2


Compound Element 1 is mandatory if applicable.

If not applicable to the data set, do not report any elements.

If applicable, it is mandatory and:

Compound Element 1.1 is mandatory.

Data Element 1.1.1 is mandatory if applicable. If not applicable,

do not report it. If applicable, it is mandatory.

Data Element 1.1.2 is mandatory.

Data Element 1.2 is optional.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2


Content Standards for Digital Geospatial Metadata

Mandatory

Legend

IdentificationInformation

DataQuality

Information

SpatialData

OrganizationInformation

SpatialReference

Information

Entityand

AttributeInformation

MetadataReference

Information

4 52 6 731

DistributionInformation

Metadata

MandatoryIf

Applicable

Section 8Citation Information

Section 10Contact Information

Section 9Time Period Information

M M / A O

Tailoring Metadata

Tailoring Metadata

Fine Tuning your metadata for your data

needs.

Fine Tuning your metadata for your data

needs.

Key ConceptsKey Concepts

• Be able to identify which extension or profile best fits dataset

• Identify discipline specific thesauri

• Be able to identify which extension or profile best fits dataset

• Identify discipline specific thesauri

Why Tailor?Why Tailor?

• Dataset specific

• Organization or Community guidelines

• Publishing or harvesting purposes

• Dataset specific

• Organization or Community guidelines

• Publishing or harvesting purposes

What if the dataset is not accurately

described by the FGDC CSDGM?

What if the dataset is not accurately

described by the FGDC CSDGM?

ExtensionsExtensions

• Extensions are used when some data does not fit into a particular section

• Example of extensions:•Remote Sensing Extensions that

includes metadata describing the sensor, platform, the information need to determine the geographical location, and the method and process of deriving geospatial information from the raw telemetry

• Extensions are used when some data does not fit into a particular section

• Example of extensions:•Remote Sensing Extensions that

includes metadata describing the sensor, platform, the information need to determine the geographical location, and the method and process of deriving geospatial information from the raw telemetry

What are Extensions?What are Extensions?

ProfilesProfiles

• Profiles extend the base standard by adding metadata elements to meet their specific community metadata requirements

• Examples of profiles:

• Biological profile

• Shoreline data profile

• Profiles extend the base standard by adding metadata elements to meet their specific community metadata requirements

• Examples of profiles:

• Biological profile

• Shoreline data profile

Remote Sensing ExtensionsRemote Sensing Extensions

Remote

Sensing

Extension

Remote

Sensing

Extension

Biological ProfileBiological Profile

• Objective is to increase the FGDC CSDGM utility for documenting biological resources data and information

• Helps broaden the understanding and implementation of the FGDC metadata content standard within the biological resources community

• Objective is to increase the FGDC CSDGM utility for documenting biological resources data and information

• Helps broaden the understanding and implementation of the FGDC metadata content standard within the biological resources community

Biological ProfileBiological Profile

BiologicalProfile

element

BiologicalProfile

element

Shoreline ProfileShoreline Profile

Ecological Metadata Language (EML)

Ecological Metadata Language (EML)

Metadata specification developed by the ecology discipline for the ecology

discipline

Metadata specification developed by the ecology discipline for the ecology

discipline

Different StandardsDifferent Standards

Other StandardsOther Standards

• These other standards can often be “mapped” or used with a “crosswalk” to compare to FGDC data

• Examples

• Marc21

• DCMI

• ISO

• These other standards can often be “mapped” or used with a “crosswalk” to compare to FGDC data

• Examples

• Marc21

• DCMI

• ISO

ReviewReview

• Know your dataset

• Know your needs

• Know your options

• Thesauri

• Extensions

• Profiles

• Know your dataset

• Know your needs

• Know your options

• Thesauri

• Extensions

• Profiles

Metadata

Enterprise

Resource

Management

Aid

(MERMAid)

Metadata

Enterprise

Resource

Management

Aid

(MERMAid)

Developed at NCDDC, MERMAid uses the framework of NCDDC’s web application server, the Z-Object Publishing

Environment (ZOPE)

Open-source leverages outside resources freely distributable

Platform independent

Utilizes an object-oriented database available in ZOPE

Offers secure, remote access through a web browser (https://mermaid.ncddc.noaa.gov)

Developed at NCDDC, MERMAid uses the framework of NCDDC’s web application server, the Z-Object Publishing

Environment (ZOPE)

Open-source leverages outside resources freely distributable

Platform independent

Utilizes an object-oriented database available in ZOPE

Offers secure, remote access through a web browser (https://mermaid.ncddc.noaa.gov)

Metadata Enterprise Resource Management Aid (MERMAid)

Technical Elements


Technical Elements

Enables the Metadata provider to:

Organize metadata- Establish unlimited metadata databases- Group records as needed

(by program, project, data type, personnel …)

Regulate workflow- Establish manager & custodian roles & permissions

(at the resource, data source and/ordatabase level)

- Monitor record status

Store supplemental information associated with a record or data set (documents, correspondence,…)


Management Features

Generates FGDC Standard, Biological, Shoreline, Remote Sensing Profile Records

Supports Ecological Metadata Language (EML)

Enhanced validation / continuous validation Import existing FGDC records in XML or TXT format

Export records in XML, TXT, HTML, or MARC XML via “save as”

Robust “Help” functions

Contents of records stored as generic objects in OODB

Metadata Enterprise Resource Management Aid (MERMAid)Capabilities

Implement record version control capabilities

- Multiple persons, same record

- Multiple versions of same record

Integrate Contact databases

- Contact Manager

Direct access to controlled vocabulary databases

- Keyword Manager

Integration with other metadata repositories


Continuing Development

Automate translation from FGDC to other standards

- ISO, DIF, Dublin Core

Import / Export records in “batch” mode

(“harvesting”)

Evaluate need for various custom extensions

- NOAA Server, ESRI, …


Continuing Development

You are given an account at this level

You need to create a Data Source

You need to create a Database

You may now start creating records

Record 1

Resource Store(training)

Resource Folder(Resource00)

Data Source(any name)

Database(any name)

ProjectProject

ProjectProject

ProjectProject

ProjectOffice

ProjectOffice

Project Office

Project Office

Department

LIneOffice

LIneOffice

Project Office

ProjectOffice

Project

Project

Project

Project ProjectProject

Example Structure

Request accounts through the NCDDC Metadata Tools web page:http://www.ncddc.noaa.gov/metadataresource/metadata-tools/view

Select 'Request an Account' to access the account request form.

Complete all fields in the account request form.

Select a short descriptive Resource Folder name. Avoid the use of personal names for your Resource Folder.

Click on the 'Submit' button.Your account 'User Name' and 'Password' will be sent via email.

Writing MetadataWriting Metadata

Tips for writing Tips for writing quality metadataquality metadata

6 Steps for Writing Quality Metadata6 Steps for Writing Quality Metadata

Organize your information

Write your metadata file

Review your file

Have someone else review

Revise

Publish

• Before you begin writing, get organized• Before you begin writing, get organized

• Document your data as you go• Document your data as you go

• Write so others can understand• Write so others can understand

• Always review your document• Always review your document

Writing MetadataWriting MetadataWriting MetadataWriting Metadata

Items requiredItems required

Sense of Humor!

ChocolateChocolate

FGDCStandard and

Workbook

FGDCStandard and

WorkbookMetadata entry toolMetadata entry tool

CoffeeCoffee

•Write simply but completely

•Document for a general audience

•Be consistent in style and terminology

•Write simply but completely

•Document for a general audience

•Be consistent in style and terminology

Keep your readers in mindKeep your readers in mind


• Define all acronyms• Define all acronyms

• Avoid using jargon• Avoid using jargon

• Clearly state data limitations• Clearly state data limitations

Keep your readers in mindKeep your readers in mind


Write a complete title that includes:Write a complete title that includes:

• What

• Where

• When

• Who

• What

• Where

• When

• Who


The title is critical in helping others find your data.

The title is critical in helping others find your data.

Which is better?

Greater Yellowstone Rivers from 1:126,700 Forest Visitor Maps (1961-1983)


• Discipline specific vocabularies

• Describes data in a standardized manner

• Tools in which to manage the knowledge held within resources and the relationships between their entries

• Goal is to facilitate interoperability and enable data discovery

• Discipline specific vocabularies

• Describes data in a standardized manner

• Tools in which to manage the knowledge held within resources and the relationships between their entries

• Goal is to facilitate interoperability and enable data discovery

Keyword ThesauriKeyword Thesauri

Global Change Master Directory (GCMD)

Global Change Master Directory (GCMD)

• Dynamic lists that evolve to meet changing needs of Earth science community

• Organized in a hierarchy constructed of Topics, Terms, and Variables

• Dynamic lists that evolve to meet changing needs of Earth science community

• Organized in a hierarchy constructed of Topics, Terms, and Variables

ISO 19115ISO 19115

• International standard for geographic metadata

• Topic Categories are used as Keywords in FGDC CSDGM

• International standard for geographic metadata

• Topic Categories are used as Keywords in FGDC CSDGM

Geographic Names Information System (GNIS)

Geographic Names Information System (GNIS)

• Federal body created in 1890 to maintain uniform geographic name usage throughout the Federal Government

• Federal body created in 1890 to maintain uniform geographic name usage throughout the Federal Government

What is a Data Dictionary?What is a Data Dictionary?

• User defined and vetted

• Consistency between data items

• User defined and vetted

• Consistency between data items

Data DictionariesData Dictionaries

Be specific. Quantify when you can.Be specific. Quantify when you can.

Vague: We checked our work and it looks complete.

Specific: We checked our work using 3 separate sets of check plots reviewed by 2 different people. We determined our work to be 95% complete based on these visual inspections.

Vague: We checked our work and it looks complete.

Specific: We checked our work using 3 separate sets of check plots reviewed by 2 different people. We determined our work to be 95% complete based on these visual inspections.


Select your key words wiselySelect your key words wisely

• Use unambiguous words

• Use descriptive words

• Fully qualify geographic locations

• Use unambiguous words

• Use descriptive words

• Fully qualify geographic locations


• Have someone else read it

• If you’re the only reviewer, put it away and read it again later

• Check for clarity and omissions

• Have someone else read it

• If you’re the only reviewer, put it away and read it again later

• Check for clarity and omissions

Review your final productReview your final product


• Can a novice understand what you wrote?

• Are your data properly documented for posterity?

• Can a novice understand what you wrote?

• Are your data properly documented for posterity?

When you review your work, ask:When you review your work, ask:


• Does the documentation present all the information needed to use or reuse the data?

• Are any pieces missing?

• Does the documentation present all the information needed to use or reuse the data?

• Are any pieces missing?

When you review your work, ask:When you review your work, ask:


Write so that others will understandWrite so that others will understand


6 Steps for Writing Quality Metadata6 Steps for Writing Quality Metadata

Organize your information

Write your metadata file

Review your file

Have someone else review

Revise

Publish

Metadata as a

Data Discovery Tool

Metadata as a

Data Discovery Tool

A Brief Look at the FGDC A Brief Look at the FGDC Clearinghouse and Clearinghouse and Geospatial One-Stop (GOS) Geospatial One-Stop (GOS) PortalPortal

A Brief Look at the FGDC A Brief Look at the FGDC Clearinghouse and Clearinghouse and Geospatial One-Stop (GOS) Geospatial One-Stop (GOS) PortalPortal

www.geodata.govwww.geodata.gov

A Brief Look at the Geospatial One-Stop (GOS) ClearinghouseA Brief Look at the Geospatial One-Stop (GOS) Clearinghouse

The geodata.gov portal is a catalog of geospatial information containing thousands of metadata records and links to live maps, features, and catalog services, downloadable data sets, images, clearinghouses, map files, and more.

The geodata.gov portal is a catalog of geospatial information containing thousands of metadata records and links to live maps, features, and catalog services, downloadable data sets, images, clearinghouses, map files, and more.

Search enables you to find and access a wide variety of geographic information within geodata.gov. You can perform a Search from the geodata.gov Home page or by clicking the Search tab.

A Brief Look at the Geospatial One-Stop (GOS) ClearinghouseA Brief Look at the Geospatial One-Stop (GOS) Clearinghouse

http://www.ncddc.noaa.gov/Metadata/Tools

[email protected]

http://www.ncddc.noaa.gov/Metadata/Tools

[email protected]


More Information


More Information

noaa national coastal data development center writing fgdc compliant metadata introduction to...

Documents

form of metadata

data cost

complete data set

information citation

information publication

critical component

component of dataproperly

information originator