February 17, 1999

Open Forum on Metadata Registries 1

Census Corporate Statistical Metadata Registry

By

Martin V. Appel

Daniel W. Gillman

Samuel N. Highsmith, Jr.

U.S. Bureau of the Census

February 17, 1999

Open Forum on Metadata Registries 2

Overview

• Objectives

• Definitions

• Standards

• Models

• Implementation

February 17, 1999

Open Forum on Metadata Registries 3

Census Bureau Metadata Objectives

• Gain Acceptance of One Logical Metadata Registry Model

• Support wide Range of Projects, Surveys, Applications, and Users

• Comply with Federal Standards

February 17, 1999

Open Forum on Metadata Registries 4

Statistical Metadata: Definition

• Descriptive information or documentation that facilitates sharing and understanding statistical data over the lifetime of the data.

• Includes– file location, record layout, database schemas, data

dictionary, definitions, questions, questionnaires, sample design, processing steps, data quality, etc.

DefinitionDefinitionss

February 17, 1999

Open Forum on Metadata Registries 5

Purposes of a Statistical Metadata Repository

• End-user oriented– Electronic Data Dissemination

• Production oriented– Automated Survey Design and Processing

February 17, 1999

Open Forum on Metadata Registries 6

What is the Corporate Metadata Repository (CMR)?

• Electronic card catalog• Provides access to metadata describing many

classes of objects– throughout a survey

– across many surveys

• Types of classes are– variables or data elements questionnaires

– datasets other products (reports)

– documents surveys

• Metadata quality monitored through registration process

February 17, 1999

Open Forum on Metadata Registries 7

Problems The CMR Solves?• Supports Cataloging and Reuse of Metadata

– Design and Processing

– Data Dissemination

• Facilitates– Sharing of Data / Metadata

– Data Administration

– Survey Design and Processing

– Survey Reengineering

• Complies with Required Federal Standards– GILS

– FGDC

– and other standards, e.g. DDI

8

Open Forum on Metadata Registries February 17, 1999

Standards Supporting the CMR

CMR

SDSMCensus Bur

MMSDX3.285

February 17, 1999

Open Forum on Metadata Registries 9

Models

• Repository Models– Business Data Model– Data Element Registry Model– Metamodel

• Model for Searching and Classifying– Business Process Model

• Table of Contents

February 17, 1999

Open Forum on Metadata Registries 10

Integrated Statistical Model

Metadata Repository

Metamodel

Business Data Model

Data ElementRegistry Model

11

Open Forum on Metadata Registries February 17, 1999

Repository Model Overview

Registration

Metamodel

BusinessData

Documen-tation

DataElement

12

Open Forum on Metadata Registries February 17, 1999

Survey Survey Instance

Survey Dataset

Questionnaire

Questions

Products

Sample

Universe/Frame

Business Data Model

13

Open Forum on Metadata Registries February 17, 1999

R esearch

P ro to typ e

N ear Term

C M RP rod u c tion P ilo t

L on g Term

IIS

R eg is try D eve lop m en t

14

Business Data Model + DER Model + Meta Model (access, search, security)

TOCBrowser

Tool

MetadataCRUDTool

Variables / Documentation/

Data sets/Tools

RegistrationTool

Prototype Corporate Metadata Repository

CMR w/ Standard Interface

DocsOpen

DADS FERRET

February 17, 1999

Open Forum on Metadata Registries 15

Near Term Implementation• Find a Home

• Design & Build Physical Architecture

• Production Pilot– Automated Survey Design and Processing

• DocsOpen

• Econ Questionnaire Design

– Electronic Data Dissemination Integration• DADS

• FERRET

• Product Registry

February 17, 1999

Open Forum on Metadata Registries 16

Long Term Implementation:Integrated Information Solutions

Program (IIS)• Data Integration, Access and Delivery

– provides access to integrated data sets– provides ongoing customer support

• Product Creation System

• Policies and Standards

• New Business Practices

• Metadata and Data Support

February 17, 1999

Open Forum on Metadata Registries 17

Current Situation Observations

• Each Program Area Operates Independently, Thereby:– Creating difficulty when two program areas must

coordinate

– Discouraging the notion of corporate data or metadata assets

– Minimizing economies of scale or reuse

– Discouraging the production of and access to integrated data

February 17, 1999

Open Forum on Metadata Registries 18

Integrated Information SolutionsProcess Model Overview

ProductsProducts CustomersCustomers ResourcesResources

• Prospect for Customers

• Target Customers

• Optimize Customers

• Plan and Acquire Resources

• Implement Resources

• Maintain Resources

• Conceive Products

• Plan and Design Products

• Develop Products

• Deliver Products

• Maintain Products

The IIS program will manage:

February 17, 1999

Open Forum on Metadata Registries 19

IIS Product Process

Customer Conceiveproducts

Plan anddesign

products

Developproducts

DeliverProducts

MaintainProducts

Customer

Needs

High-levelproduct concepts

ProductConcept

Final model

PreliminaryMarketing

plan

Approved product

Final marketing

plan

Product access

Questions

Support

Orders

Fee

dbac

k

CustomerDatabase

Resource AllocationTracking System

Corporate dataand metadata repository

Sou

rce

data

& m

etad

ata

App

rove

d pr

oduc

t

Fee

dbac

k su

mm

ari e

s

Pro

duct

Con

cept

App

rove

dbu

sine

ss c

ase

Feedback summaries

Decision to improve product

Product concept

Previous experience with similar products

App

rove

d m

odel

Pro

duct

co n

cept

App

rove

d p

rodu

ct

Usa

ge s

tati

stic

s

Usage statistics

Improve or discontinue decisions

February 17, 1999

Open Forum on Metadata Registries 20

February 17, 1999

Open Forum on Metadata Registries 21

Phased Implementation Strategy

• Phase 1 : (January 99 - September 00)

– Specifications for technology solutions developed

– Establish corporate metadada repository

– Pilot projects

• Phase 2: (FY2001 and FY2002)

– Implementing technology solutions

– Non-Decennial portion of DADS incorporated

• Phase 3: (FY2003)

– Bulk of IIS processing vision put in place

– DADS fully incorporated into IIS

February 17, 1999

Open Forum on Metadata Registries 22

Questions

23

Open Forum on Metadata Registries February 17, 1999

DATA ELEMENT CONCEPT DATA ELEMENT

Property Property

Representation

(1:1) (1:1)

(1:1)

(1:N)

(1:N)

Object Class

(1:N)

Object Class

X3.285 Fundamental Model

February 17, 1999

Open Forum on Metadata Registries 24

Value Domain

Data Element

Data Element Concept

Data Element Model

February 17, 1999

Open Forum on Metadata Registries 25

Administered Component

Classified Component Classification Scheme

Common Attributes

subtype

subtype

Metadata Objects ...

Registration Model

February 17, 1999

Open Forum on Metadata Registries 26

Administered Component

Documentation

Documentation Type

Documentation Model

February 17, 1999

Open Forum on Metadata Registries 27

SDSM Chapters• Access Constraint

• Archive

• Citation

• Contact

• Data

• Dataset Supplier

• Data Quality

• Dataset

• Descriptive Statistics

• Distribution Information

• Documentation

• Emprise

• Field

• Format

• Frame

• Methodology

• Planning and Design

• Pointer to Object

• Product

• Program

• Questionnaire

• Question

• Sample

• Series

• Sponsor

• Status

• Survey

• Survey Instance

• System

• Technique

• Theme

• Universe

• FGDC