session topics understanding the value and importance of proper data documentation michael moeller...

Session TopicsUnderstanding the Value and Importance

of Proper Data DocumentationMichael Moeller – Metadata Specialist, NOAA CSC

Metadata Servers and GatewaysJohn Ulmer – Senior Computer Analyst

Michael MoellerNOAA Coastal Services Center

MetadataUnderstanding the Value

and Importance of ProperData Documentation

Metadata - Documentation of geospatial data written in a consistent manner

Some Terms We Will Be Using

FGDC - Federal Geographic Data Committee

CSDGM - Content Standard for Digital Geospatial Metadata, referred to commonly as “The Standard” or “The Content Standard”

Clearinghouse - A distributed catalog of metadata

Geospatial - refers to a geographic location

First Things First

What is

Metadata?

Simply put, metadata is information

about your data.

What is Metadata?

This is the metadata for this.

What’s Missing?

Emily and M

adison

Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, 1997. Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction

Author(s) Boullosa, Carmen. Title(s) They're cows, we're pigs / by Carmen Boullosa Place New York : Grove Press, 1997. Physical Descr viii, 180 p ; 22 cm. Subject(s) Pirates Caribbean Area Fiction. Format Fiction


While the card-catalog entry is a form of metadata, it does not address topics such as

quality, accuracy, or scale. Well-written geospatial metadata describes these and many more aspects of the data.

Identification_Information Citation Citation_Information Originator: NOAA Coastal Services Center Publication_Date: 19971131 Title: Hurricane Storm Surge Geospatial_Data_Presentation_Form: Map Publication_Information Publication_Place: Charleston, SC Publisher: NOAA Coastal Services Center Larger_Work_Citation Citation_Information

Identification_Information Citation Citation_Information Originator: NOAA Coastal Services Center Publication_Date: 19971131 Title: Hurricane Storm Surge Geospatial_Data_Presentation_Form: Map Publication_Information Publication_Place: Charleston, SC Publisher: NOAA Coastal Services Center Larger_Work_Citation Citation_Information


We often use metadata

without even knowing it.

Metadata contains vital information.Imagine, if you will…

You are given two identical cans without labels. One contains cat food, the other contains tuna (dolphin-safe, of course).

You must choose one these cans,

and then eat the contents.

Other Examples?

What is Metadata?

Metadata as

a Component

of Data

Properly documented

data provides vital information

to interested parties.

A Component of Data

Metadata is that component of data which describes it.

Environmental Sensitivity Index Data

Metadata

RARNUM - unique combination of species, concentration, and seasonality

CONC (concentration) = Density species is found at location

Season_ID = seasonality code like to the seasonal table

Element - Biology group

A Component of Data

It’s data about a data set.

Title

Scale

Source

Content

Location

Publication

Access

Title

Scale

Source

Content

Location

Publication

Access

MetadataMetadataMetadataMetadata

GIS files

Imagery

Geospatial databases

GPS data

GIS files

Imagery

Geospatial databases

GPS data

Data setData setData setData set

A Component of Data

Metadata describes…

CONTENT

CONDITION

QUALITY

Characteristics of the data

Characteristics of the data

A Component of Data

Metadata is a critical component

of a complete data set.

A Component of Data

Metadata

Non-spatial orattributes

Spatial

Because metadata provides vital information about a dataset, it should never be viewed or

treated as a separate entity.

Take Home Message

Metadata is a critical and

integral component of any complete

data set.

Metadata is a critical and

integral component of any complete

data set.

A Component of Data

The Value

of Metadata

“It takes a lot less time, and most people

won’t notice the difference until

it’s too late.”

The Predominant Attitude?

Properly documenting a data

set is the key to preserving its usefulness

through time.

The Time Value of Metadata

Two similar paintings by Picasso up for

auction sold for vastly different prices.

Why?

One had metadata...

...One didn’t.

The Time Value of Metadata

Metadata should be updated to reflect changes in the data.

1980

HEWTeheran

British HondurasSnow measurementCape Hatteras Light

ArpanetI-95

I-495Mt. St. Helen

West Germany

2000

HHS & HUDTehranBelizeSnow measurementCape Hatteras LightInternetI-395I-95Mt. St. HelenGermany

The Value of Metadata

Metadata has other value associated

with it.

• Avoid duplication

• Share reliable information

• Publicize efforts

• Reduce workload

For data developers, metadata...


• Facilitates understanding

• Focuses on key elements

• Enables discovery — inside and outside of organizations

For data users, metadata...


For organizations, metadata...


• Protect investment in data

• Create an institutional memory

• Counter personnel changes

• Allow sharing of data with other agencies

• Reduce costs

• Limit potential liability

• Save time and money

Metadata as a “data discovery” tool

This saves time and money.

If it’s geospatial data you need, metadata helps

• Find data of interest

• Determine the usefulness of the data

• Determine how to access the data


This “data discovery” is all

accomplished through the use

of a clearinghouse, an on-line

searchable catalog of

standardized

metadata records.


StandardizedMetadata

Why Have a Standard?

Think for a moment how hard it would be to…

… bake a cake without standard units of measurement.

… put gas into your car without standard nozzle sizes.

… plug a lamp into a socket without standard electrical outlets.

Why Have a Standard?

The standard for

metadata ensures a level

of consistency in

data documentation.

Standards ensure consistency.

Establishing a Standard

The Federal Geographic Data Committee (FGDC) was organized in 1990 under the

Office of Management and Budget to promote the coordinated use, sharing,

and dissemination of geospatial data on a national basis. The FGDC was tasked

with creating a metadata standard to meet these objectives.

The Metadata Standard

"... each agency shall document all new

geospatial data it collects or produces, either

directly or indirectly, using the standard under

development by the FGDC, and make that

standardized documentation electronically

accessible to the Clearinghouse network."

The Content Standard for Digital Geospatial Metadata (CSDGM)

Executive Order 12906, 1994


This “Content Standard”

serves as a uniform

summary description

of the data set.


The Content Standard utilizes...

• Common terms

• Common definitions

• Common language

• Common structure Access

constraints

Citation

currentness

entity

attrib

ute

domain

lineage

Process step


The Content Standard…

• Establishes names of compound elements and data elements.

• Defines the information about the values that are to be provided for the data elements.

Type=“free text”Domain=Real

currentnessentity

attributeCitation

lineage


The Content Standard helps

the user determine...

• If a set of geospatial data is available and fit for a particular use.

•How to access and transfer the data set.


WhoWho collected the data?Who processed the data?Who wrote the metadata?Who to contact for questions?Who to contact to order?Who owns the data?

WhereWhere were the data collected?Where were the data processed?Where are the data located?

WhatWhat are the data about?What project were they collected under?What are the constraints on their use?What is the quality?What are appropriate uses?What parameters were measured?What format are the data in?

WhenWhen were the data collected?When were the data processed?

HowHow were the data collected?How were the data processed?How do I access the data?How do I order the data?How much do the data cost?How was the quality assessed?Why

Why were the data collected?

Metadata written using the Content Standard answers these important

questions:


The FGDC Workbook

FGDC’s Metadata

Workbook

Defines the 334

metadata elements.

What do I use “The Workbook” for?

• It is the definitive resource for applying the FGDC Content Standard.

• However, it does not define the production rules.

• It describes element domain values, which are valid values that can be assigned to the data element.

• It provides section and element definitions.

• It is a quick reference for production rules and structure.

Use the “Graphical Representation”for quick access.

• You will still need to use the workbook to find the definition of a particular element and its domain.

Playing the Metadata Game

The Content Standard is organized using numbered chapters called “sections.”

There are 7 main sections and 3 supporting sections.

Each section is organized into series of elements that define the information

content for metadata to document a set of digital geospatial data.

Organization of the Content Standard

DataQuality

Information

SpatialData

OrganizationInformation

SpatialReference

Information

Entityand

AttributeInformation

4 52 6 731

Metadata

The Three Supporting Sections

9 Time PeriodInformation

10 Contact

Information

8 Citation

Information

DistributionInformation

MetadataReference

Information

IdentificationInformation


The Seven Main Sections


Each section begins with the name and definition of the section. These are

followed by the component elements of the section.

Each section provides the names and definitions of its component elements, information about the types of values

that can be provided for the elements, and information about the elements

that are mandatory or repeatable.

Interpreting the Graphical Production Rules

The workbook uses graphics to illustrate the production rules of the standard. These graphics include most of the information provided by the production rules, including:

How elements are grouped. What is mandatory and what is not. What elements can repeat and how many

times they can repeat.


SectionSections are depicted by this symbol.

Compound Element

Compound elements are depicted using a 2-dimensional box.

Data Element

Data elements are depicted using a 3-dimensional box

with shadow.


Data Element

A data element is a logically primitive itemof data. Data elements are the things thatyou “fill in.”

The form for the definition of a data element is:

Data element name -- definition.Type: (choice of “integer”, “real”, “text”, “date”, or “time”)Domain: (describes valid values that can be assigned)

An example of the definition of a data element is:

Abstract -- a brief narrative summary of the data set.Type: textDomain: free text

Note: Data element definitions are containedin the text of the Content Standard,

not in the graphical production rules.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

How Elements Are Grouped

Compound elements are composed of other compound and data elements. The composition is represented by

nested boxes.

Compound Element 1 is composed of

Compound Element 1.1 and

Data Element 1.2.Compound Element 1.1

is composed of Data Element 1.1.1

and Data Element 1.1.2.

Mandatory - must be provided.

MeaningData

ElementCompound

Element

What’s Mandatory? What’s Not?

Mandatory if Applicable - must be provided if the data set exhibits

the defined characteristic.

Optional - provided at the discretion of the data set producer.

If an element can be repeatedindependently from other elements, itwill be indicated as such below theelement name.

Repeating Elements

CompoundElement 1

(can be repeatedunlimited times)

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

This group of elements

would repeat.

Compound Element 1 Compound Element 1.1 Data Element 1.1.1 Data Element 1.1.2 Data Element 1.2

Using the Graphics to Make Decisions

All elements are colored yellow, so all are mandatory and must be reported.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

Compound Element 1 is mandatory.

Compound Element 1.1 is optional.

If yes, Data Elements 1.1.1 and 1.1.2 are mandatory.

If no, do not report Compound Element 1.1, Data Element 1.1.1 or 1.1.2, and skip to Data Element 1.2.

Data Element 1.2 is mandatory.

CompoundElement 1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2

CompoundElement 1.1


Compound Element 1 is mandatory.

Compound Element 1.1 is mandatory.

Data Element 1.1.1 is mandatory.

Data Element 1.1.2 is mandatory if

applicable.

Data Element 1.2 is optional.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2


Compound Element 1 is mandatory if applicable. If not applicable to the data set, do not report any elements. If applicable, it is mandatory and:

Compound Element 1.1 is mandatory.

Data Element 1.1.1 is mandatory if applicable. If not applicable,

do not report it. If applicable, it is mandatory.

Data Element 1.1.2 is mandatory.

Data Element 1.2 is optional.

CompoundElement 1

CompoundElement 1.1

DataElement 1.1.1

DataElement 1.1.2

DataElement 1.2


WritingMetadata

It’s not THAT bad!

• First records are the hardest.• Not all fields may need to be filled in.• Tools are available.• Training classes can be taken.• Can often be produced automatically.• Can (and should) be reviewed

for updates.

Before you begin writing,get organized.

Writing Metadata

Document your data as you go.

Writing Metadata

Write so others can understand.

Writing Metadata

Always review your document.

Writing Metadata

Items required

Sense of Humor!

Chocolate

FGDCWorkbook

Metadata entry tool

Coffee

• Write simply but completely.

• Document for a general audience.

• Be consistent in style and terminology.

Keep your readers in mind.Writing Metadata

• Define all acronyms.

• Avoid using jargon.

• Clearly state data limitations.

Writing Metadata

Keep your readers in mind.

Write a complete title that includes:

• What• Where• When• Scale• Who

Writing Metadata

The title is critical in helping others find your data.

Which is better?

Greater Yellowstone Rivers from 1:126,700 Forest Visitor Maps (1961-1983)

Writing Metadata

Be specific. Quantify when you can.

Vague: We checked our work and it looks complete.

Specific: We checked our work using 3 separate sets of check plots reviewed by 2 different people. We determined our work to be 95% complete based on these visual inspections.

Writing Metadata

Select your key words wisely.

• Use unambiguous words.

• Use descriptive words.

• Fully qualify geographic locations.

Writing Metadata

• Have someone else read it.

• If you’re the only reviewer, put it away and read it again later.

• Check for clarity and omissions.

Review your final product.

Writing Metadata

• Can a novice understand what you wrote?

• Are your data properly documented for posterity?

When you review your work, ask:

Writing Metadata

• Does the documentation present all the information needed to use or reuse the data?

• Are any pieces missing?

When you review your work, ask:

Writing Metadata

Tool Time

Tool Time

A sample of some of the available tools for metadata creation, validation, and publication.

• CNS and MP“Chew ‘n spit,” checks and corrects structural errors, and “Metadata Parser”, which checks for errors in element compliance.

• NOAA CSC MetaScribe Allows you to create a template record that can be used to create large numbers of similar records.

• NOAA CSC ArcView Metadata Collector Extension for ArcView 3.x.

• TKMEText editor used for metadata entry.

• Commercially available software

Metadata Entry Tools

TKME An editor for formal metadata, TKME is intended to

simplify the process of creating metadata that conform to the standard.

NOAA CSC ArcView ® Metadata Collector The ArcView® Metadata Collection Tool was developed by the National Oceanic

and Atmospheric Administration (NOAA) Coastal Services Center in ArcView using

the Avenue® scripting language. This tool collects and compiles Federal

Geographic Data Committee (FGDC)-compliant metadata for ARC/INFO

coverage's, shapefiles, grids and supported image formats.


NOAA CSC MetaScribeMetaScribe is a template driven application designed to significantly reduce the

labor required to produce a collection of FGDC-compliant metadata records with

similar content.


TKME, along with CNS, and MP, are available as free downloads from the United States Geological Survey (USGS) Website. (geology.usgs.gov/tools/metadata)

Tool Time

The NOAA CSC Metadata Collector Extension for ArcView 3.x is downloadable from the NOAA CSC Web site. (www.csc.noaa.gov/metadata)

The NOAA CSC MetaScribe is accessible from the NOAA CSC Web site. (www.csc.noaa.gov/metadata/)

Finally...

Remember, metadata is a legacy document that

concisely sums up your data or data set.

Without metadata, your data set is

incomplete.

Michael [email protected]

www.csc.noaa.gov/metadata/

session topics understanding the value and importance of proper data documentation michael moeller...

Documents