1

Schema Registries

Steven Hughes, Lou Reich, Dan Crichton

NASA

April 20, 2023

2

Topics

Background/Definitions

Schema Registry Requirements

Plan

3

Definitions

Registry - Formal record of information, or the place where such records are kept.

Repository - A repository is a place where data is stored and maintained.

Registry and Repository are referenced in the Information Architecture and OAIS RM papers but they are not defined.

4

Registry vs Repository

Repository holds data Provides “get” and “put” functions to retrieve and store data

holdings using a “unique identifier”.

Some additional “operative” metadata may exist primarily for managing the data in the repository. (dates, checksums, etc).

Repository may hold “archive” metadata, but it is co-located with the data. In the repository context the metadata is not considered operative.

Repository has a “register function”. (i.e. A repository is a register in the sense that a formal record of its contents is kept by registering or “making or securing an official entry”)

Repository is similar to the “OAIS Archival Storage” functional entity

5

Registry vs Repository

Registry assigns an identifier to object

Registry uses metadata to build a catalog (or index) Makes the metadata searchable for identifying and locating stored

objects

Supports two phased data access (i.e. search and then retrieve)

Registry is similar to the OAIS functional entity “Data Management”. It supports “OAIS Access”.

Registry is considered separately from a repository since the two functional entities may be deployed separately.

Registry and Repository functions are often collocated.

6

Types of Registries

Metadata registry provides an interface to search for data about metadata. Manage meta-metadata models. (e.g. XML Schemas, Data

Dictionaries, Ontologies

Resource registry provides an interface to search for resources. Manage descriptions of any resource (i.e. object), typically

information objects such as science data products and data sets.

Service registry provides an interface to search for functional services that perform a needed action specified by a user. Manage descriptions of service interfaces (called service

descriptions), including their respective locations, methods and method parameters.

7

Schema Registry

Repository/registry that accepts, stores, and maintains XML schemas Functionality is shared across a distributed environment

Support schema design, development, validation, deployment, evolution, and maintance

8

XFDUProducer Application

XFDUConsumerApplication

XMLToolkitLibrary

PackagingPlug-ins

TransformationPlug-ins

Validation Plug-ins

PluginRegistry/ Repository

XMLRegistry/ Repository

System Programmer

Producer Consumer

CCSDS XFDU Administration

Registration Dissemination Notification Validation Query Subscription Reconstitution

Application Domain Programmer

9

Functional Requirements

Full Standards Compliance Import and validate W3C XML Schema or DTD

Version Control Schema level Object level Configuration management with audit trails Check-In, Check Out Release Workflow – local updates, checkin, integration, create

release, deploy release Subscribe/Notification

Impact Analysis Track dependencies between objects Determine impact of changes

10

Functional Requirements

Object Management Manages schemas at their object level, not at the file or file

fragment level (i.e. To the attribute or element level)

Any redundancy and duplication is resolved at import/update time

Reuse happens through reference links

Assemble schemas dynamically from the objects

Common Repository Developers work in their own personal workspaces

Shared object database provides for team-based development

11

Functional Requirements

Documentation Object annotation Documentation production on request

Multiple namespaces Relationships between elements Automatic transformations

Schema artifact respository

CortexML - Collaborative XML Schema Management

http://www.digitalml.com/en/index.php

12

Functional Requirements

Schema Validation Validate well formed XML and log errors.

XML Instance Validation Validate an XML file against an internally referenced DTD or XSD

13

Plan

Determine interest

Consider working group for schema registries

Gather requirements for schema registries

Define standard information model and services for schema registries

Identify role of schema registry in information infrastructure

14

Backup

15

DSMS NeedsRegistry Service Components

Data Element/Data Type Registry: Provides common data dictionary information that contains element definitions and data type definitions

XML Schema Registry: Provides a versioned repository for XML schemas that describe messages, file formats, and data components.

XML Stylesheet Registry: Provides a versioned repository for XML stylesheets that perform common data translations.

Namespace/Domain Registry: Provides a controlled, hierarchical namespace/domain registry.

Service Registry: Provides a dynamic Registry for Web Services.

Model Registry: Provides a repository for information models, data relationships, and other ontological information.

16

DSMS Needs CIO/Standards Efforts

Schema Registry A component of the Metadata Service.

Currently deployed in the DSMS development environment.

Data Element Registry A component of the Metadata Service.

Currently deployed in the DSMS development environment.

Service Registry Evaluating COTS and Open Source products.

17

Schema RegistryOverview

This registry provides a central repository for developers and organizations to publicly submit, publish and exchange XML-based schemas.

A component of the Metadata Service. An effort currently funded by the CIO.

Interfaces Application Program Interface (API)

Support distributed access using CORBA/RMI

Support Java and HTTP application interfaces

Command-Line

Web-Based GUI

Planned Features for Future Releases Support for Authentication and Authorization

18

Schema RegistryCurrent Deployment

Deployed with Metadata Service release 2.0.0 http://metadata.jpl.nasa.gov/ on February 9

http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment)

Features Add, remove and modify schemas and their supporting

metadata in the registry.

Metadata capture based on the Dublin Core Element Set.

Allow for versioning and promotion of schemas.

Capability to locate and access registered schemas.

Provide for URL access to registered schemas.

Provide for a web directory hierarchy for organizing schemas.

19

Data Element RegistryOverview

This registry provides the capability to register and manage a data element and associate it with a data dictionary.

A component of the Metadata Service. An effort currently funded by the CIO.

Interfaces Application Program Interface (API)

Support distributed access using CORBA/RMI

Support Java and HTTP application interfaces

Command-Line

Web-Based GUI

Planned Features for Future Releases Support for value domain

Support for element relationships

Support for Authentication and Authorization

20

Data Element RegistryCurrent Deployment

Deployed with Metadata Service release 2.0.0 http://metadata.jpl.nasa.gov/ on February 9

http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment)

Features Based on the ISO/IEC 11179 “Information technology --

Specification and standardization of data elements”.

Add, remove and modify data elements and their supporting attributes in the registry.

Allow for versioning of data elements.

Associated data elements to a data dictionary.

Extract data elements from an imported data dictionary via the Dictionary Registry component.

Capability to locate and access registered data elements.