leaving a metadata trail
DESCRIPTION
Leaving a Metadata Trail. Chapter 14. Defining Warehouse Metadata. Data about warehouse data and processing Vital to the warehouse Used by everyone. Metadata. The Key to Understanding Warehouse Information. Specifies data location Manage data Aids use of information Describes the data - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/1.jpg)
Leaving a Metadata Trail
Chapter 14
![Page 2: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/2.jpg)
Defining Warehouse Metadata
Data about warehouse data and processing Vital to the warehouse Used by everyone
MetadataMetadata
![Page 3: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/3.jpg)
The Key to Understanding Warehouse Information Specifies data location Manage data Aids use of information Describes the data Documents the development process Provides a record of changes Records enhancements over time
![Page 4: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/4.jpg)
Metadata UsersIT developers
ETT
Operational
Enduser
Metadatarepository
WarehouseEnd users
![Page 5: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/5.jpg)
Types of Metadata End user: - Key to a good warehouse - Navigation aid - Information provider ETT: - Maps structure - Source and target information - Transformations Operational: - Load, management, scheduling processes - Performance
![Page 6: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/6.jpg)
Developing a Metadata Strategy Define a strategy to ensure high-quality metadata
useful to users and developers. Primary strategy considerations: - Define goals and intended use. - Identify target users. - Choose tools and techniques. - Choose the metadata location. - Manage access to the metadata. - Manage access to the metadata. - Integrate metadata from multiple tools. - Manage change.
![Page 7: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/7.jpg)
Defining Metadata Goals and Intended Usage Define clear goals. Identify requirements. Identify intended usage.
MetadataMetadata
![Page 8: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/8.jpg)
Identifying Target Metadata Users Who are the metadata users? - Developers - End users What information do they need? How will they access the
metadata?
![Page 9: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/9.jpg)
Choosing Metadata Tools and Techniques Tools - Data modeling - ETT - End-user query and analysis Database schema definitions COBOL copybooks Middleware tools
![Page 10: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/10.jpg)
Choosing the Metadata Location
Usually the warehouse server Possibly on operational platforms Desktop tool with metalayer
Metadatarepository
Warehouse End users
ExternalsourcesOperationalDatasources
![Page 11: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/11.jpg)
Managing the Metadata Managed by the metadata manager Maintained by the metadata architect Standards produced by the metadata
architectMetadatarepository
Warehouse End users
ExternalsourcesOperationalDatasources
![Page 12: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/12.jpg)
Integrating Multiple Sets of Metadata Multiple tools may generate their
own metadata. There are many metalayer
integration issues. Metadata exchangeability is
desirable.
![Page 13: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/13.jpg)
Managing Changes to Metadata Different types of metadata have
different rates of change. Consider metadata changes
resulting from refresh cycles.
![Page 14: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/14.jpg)
Examining Types of Metadata ETT metadata End user metadata
ETTEnduser
Metadatarepository
Warehouse
![Page 15: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/15.jpg)
ETT Metadata Business rules Source tables, fields, and key values Ownership Field conversions Encoding and reference table Name changes Key value changes Default values Logic to handle multiple sources Algorithms Time stamp
Extraction
Externalsources
Operational data sources
Staging file
![Page 16: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/16.jpg)
Extraction Metadata Space and storage requirements Source location information Diverse source data Access information Security Contacts Program names Frequency details Failure procedures Validity checking information
Extraction
Externalsources
Operational data sources
Staging file
![Page 17: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/17.jpg)
Transformation Metadata Duplication routines Exception handling Key restructuring Grain conversions Program names Frequency Summarization
TransformExternalsources
Operational data sources
Staging file
![Page 18: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/18.jpg)
Transportation Metadata Method of transfer Frequency Validation procedures Failure procedures Deployment rules Contact information
Externalsources
OperationalData sources
ETT
TransportTransport
Transport
Staging file
Metadatarepository
Warehouse
![Page 19: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/19.jpg)
End-User Metadata 739516 1816 666 17.62 Associate the metadata description Analogous to Oracle Data Dictionary views
Enduser
Metadatarepository
Warehouse
![Page 20: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/20.jpg)
Example of End User Metadata
TableName
ColumnName
Data Meaning
Product Prodid 739516 Unique identifier for the product
ProductProduct
Valid_dateWare_loc
01/971816
Last refresh dateWarehouse location number
Product Ware_bin 666 Warehouse bin number
Product Weight 17.62 Packed shippingWeight in kilograms
![Page 21: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/21.jpg)
More End-User Metadata Information
Location of fact and dimensions Availability Description of contents Algorithms for derived and summary data Owners of data and telephone number
Enduser
Metadatarepository
Warehouse
![Page 22: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/22.jpg)
Historic Context of Data Supports change history Maintains the context of information
Operational Warehouse
Content
Structure
repositoryMetadata
94 95 96 97 98
![Page 23: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/23.jpg)
Types of Context Simple: - Data structure - Naming conventions - Metrics Complex: - Product definitions - Markets - Pricing External: - Economic - Political
Warehouse
94 95 96 97 98
![Page 24: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/24.jpg)
Additional Metadata Content and Considerations Summarization algorithms Relationships Stewardship Permissions Pattern analysis Reference tables
![Page 25: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/25.jpg)
Metadata Management Tools Carleton Evolutionary Technologies Hewlett Packard Informatics Oracle Designer Platinum Technology Prism Solutions Sagent
![Page 26: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/26.jpg)
Common Warehouse Metadata
Design and Administratioon
![Page 27: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/27.jpg)
Common Warehouse Metadata Future
WarehouseBuilder
Oracle8iServer
Discover
ExpressServer
Commonmetadata
![Page 28: Leaving a Metadata Trail](https://reader038.vdocuments.mx/reader038/viewer/2022103100/56813676550346895d9e0450/html5/thumbnails/28.jpg)
Summary
This lesson discussed the following topics: Definitions Integration Contents Storage Creation Selection Tools