modernizing data: issues and approaches nams-if data modernization... · 2007. 11. 13. · legacy...

35
1 November 2007 Modernizing Data: Issues and Approaches

Upload: others

Post on 31-Jan-2021

3 views

Category:

Documents


0 download

TRANSCRIPT

  • 1November 2007

    Modernizing Data:

    Issues and Approaches

  • 2November 2007

    Introduction to Treehouse Software

    Issues

    Approaches and Solutions

    Keys to Success: Standards, Metadata and Experience

    Q&A

    Agenda

  • 3November 2007

    Established in 1982

    Leading ISV for Software AG

    70% penetration in North American market

    Consultants with 20+ years SAG experience

    More than 300 man years of experience

    Over 700 customers worldwide

    20 products

    30 employees

    Introduction to Treehouse Software

  • 4November 2007

    Strategic Partners

    Anubex

    ACS Government Systems

    Benaroya

    B.O.S. Software Service und

    Vertrieb GmbH

    CCA Software Pty. Ltd.

    CA

    Cronus Consulting Pty

    Cubeware GmbH

    DataMirror Corporation (IBM)

    EDS Atlantic Region

    E-Net Corporation

    Harvest Moon Computing Pty. Ltd.

    Intelligent Commerce Network LLC

    NatWorks, Inc.

    Policy Studies, Inc.

    SAIC

    Solution Specialists, Inc.

    The Software Revolution, Inc.

    Wostmann & Associates

    Technology Partners

    Fujitsu Siemens Computers

    Hewlett-Packard

    IBM Corporation

    Microsoft

    Oracle

    SAP*

    Sun Microsystems

    Marketing Representatives

    AustraliaCCA Software Pty Ltd.

    Brazil 3CON Consultoria e Sistemas Ltd.

    Germany VersaTec IT Services

    JapanE-Net Japan

    Singapore and Far EastBachoJames Inc.

    South AfricaBateleur Software (Pty) Ltd.

    Partnerships give

    Treehouse

    international

    presence and

    broaden our

    capabilities

  • 5November 2007

    Treehouse Offerings

    Data Management

    • ADAMAGIC

    • ADAREORG

    • ADASTRIP

    • AUDITRE

    • SECURITRE

    • TRIM

    Mainframe Emulation

    • SEDIT

    • S/REXX

    • S/REXX Debugger

    Professional Services

    • Data Warehousing and

    Business Intelligence

    • Data Transfer

    • Application Modernization

    and Migration

    • DBA Services

    • Performance and Tuning

    • Product Installation

    • Training

    Data Transfer

    • tRelational

    • tRelationalPC

    • DPS

    • DPSync

    • DPS X-Link

    • NatQuery

    • NatCDC

    • tcACCESS

    • DataMirror Products

    Application Management

    • CHART

    • N2O, N2O/3GL

    • EspControl

    • PROFILER

  • 6November 2007

    Treehouse Customers

  • 7November 2007

    "I would strongly recommend the use of the tRelational and DPS products for

    any other State agencies that are currently utilizing ADABAS, especially if they

    need to do any migration to another platform for data warehouses."

    Nancy Kane

    Systems & Programming DBA Manager

    Arizona Department of Economic Security

    "tRelational and DPS are the tools that allowed us to reach our goals in a

    pragmatic and non-invasive way."

    Roberto Licandro

    Architect - IT Architectures 
B-Source SA

    Treehouse Customers

  • 8November 2007

    "In our deployment of field identification

    capabilities, having current data available on

    criminal histories as part of that process is

    critical.”

    Captain Dickerson

    Commander of Technical Services

    Sacramento County Sheriff's Department

    "DPSync allows us to take a big step forward

    without replacing CJIS.”

    Albert Locher

    Assistant Chief in the

    Sacramento County District Attorney’s Office

    "...it was clear that tRelational/DPS was the leading

    product for ADABAS-to-RDBMS data transfer.”

    Doug Kauffroath

    MIS Director for the Sacramento Superior Courts

    Treehouse Customers

  • 9November 2007

    Federal Government

    – Department of Justice

    – National Business Center

    State and Local Government

    – State of Alaska – Department of Public Safety

    – State of Arizona – Department of Economic Security

    – State of Delaware – Department of Technology Services, Department of Revenue

    – State of Utah – Department of Workforce Services

    – State of Washington

    – State of Hawaii

    – County of Sacramento

    – UK Driver and Vehicle Licensing Agency (DVLA)

    – City of Calgary

    – Service Nova Scotia

    Sample Data Transfer Customers

  • 10November 2007

    Higher Education

    – Case Western Reserve University

    – Florida Community College Software Consortium

    – Miami University of Ohio

    – Penn State University

    – University of Texas Austin

    Private Sector

    – Argus Health Systems

    – B-Source

    – Cutler Hammer

    – Dexia Banque Internationale à Luxembourg S.A.

    – First American Real Estate Solutions (FARES)

    – Food Lion

    – GMAC

    – Kansas Nebraska Energy (KNE)

    – National Fuel Gas

    Sample Data Transfer Customers

  • 11November 2007

    Legacy modernization initiatives view data modernization as a small component

    - yet -

    Failure of the data modernization process can and does cause failure of the entire project

    Issues in Data Modernization

    Whenever a particular data representation is to be expressed in a

    foreign representation, there is great opportunity for mistranslation

  • 12November 2007

    Lack of useful, up-to-date, a accurate documentation

    Discrepancies between logical and physical definitions

    Discrepancies between processing environments (e.g., development vs. production)

    Discovery/Analysis of Source Data Structures

    nd

  • 13November 2007

    Domain analysis

    Unused elements

    “Seasonal” or “state change” elements

    Record typing

    Index usage

    Validation of application assumptions

    Insight into Actual Data Content

  • 14November 2007

    Hidden aspects of the data model

    Multiple usages of fields/columns, records/rows and files/tables

    Feature and datatype usage that cannot be accurately represented in the target environment or database

    Proprietary Aspects of the Source

  • 15November 2007

    Need to not only reflect the structure and content of the legacy source, but also its behavior

    Must specify, design and implement appropriate mappings and transformations

    Requires deep technical knowledge of the source and the target technologies

    Data Modeling and Design of the Target

  • 16November 2007

    Need a central store of information on the source, the target, and the relationships between them

    Must support iterative and interim development

    Metadata needs modernization too—legacy data documentation should not be discarded

    Maintaining the Metadata Repository

  • 17November 2007

    “Lowest common denominator” leaves too much to be desired

    Throughput needs to be maximized

    Ensure correct interpretation and formatting

    Leveraging Rich Native Interfaces

  • 18November 2007

    Iteration is inevitable

    Deployments must be tested repeatedly and comparatively

    Data currency demands may escalate

    Employing an Evolutionary Methodology

  • 19November 2007

    Different modernization approaches place different demands on aspects of the technology architecture

    Necessity to make legacy data available to new applications rarely fits conveniently into “batch windows”

    Tight DBA resource control may no longer be feasible

    Impact on Production Systems

  • 20November 2007

    Approaches and Solutions

  • 21November 2007

    Extracting legacy data changes (CDC) by:

    Comparison to prior state

    Capture from database log or exit

    Programmatically by timestamp

    Generally requires initial Extract-Transform-Load (ETL)

    May be accomplished in batch, in near-real time or in real time

    Products include: DPSync, tcVISION, NatCDC, DataMirror Transformation Server (/ES)

    Data Replication

  • 22November 2007

    Advantages:

    Low impact on source

    Low volume of data to be transmitted

    Can process only committed transactions

    Disadvantages

    Always some degree of latency

    Multidirectional replication is extremely complex

    Requires robust recovery and monitoring capabilities

    Data Replication

  • 23November 2007

    Inserting one or more technology layers (servers) between the legacy source and the consuming application

    May be read-only or read/write

    Products include: tcACCESS, DPS X-Link

    SQL/SOA Enablement

  • 24November 2007

    Advantages:

    Zero latency in data

    Legacy data directly integrated into applications

    Disadvantages

    Difficult to control amount of data transmitted and resources used

    Query isolation may be assumed but not delivered

    Consuming applications need metadata

    SQL/SOA Enablement

  • 25November 2007

    Extraction of source data with appropriate transformation to meet the needs of the target platforms and applications

    Products include: tRelational/DPS, NatQuery

    Data Migration

  • 26November 2007

    Advantages:

    Applications work with native, modern data sources—no compromises

    Ultimately, reduction in cost and complexity of technology architecture

    Disadvantages

    ―Big bangs‖ are not feasible, so coexistence/phased implementation support is needed

    Must select a point in time to migrate

    Data Migration

  • 27November 2007

    Keys to Success

  • 28November 2007

    Discovery and analysis of ADABAS source data structures

    Compares FDT to PREDICT

    Flags post-implementation changes to either

    Insight into actual data content and use of proprietary ADABAS datatypes and features

    Analyses of MU/PE fields, datatypes

    Validation of application assumptions

    All information captured into metadata repository

    TSI’s tRelational/DPS as a Case Study

  • 29November 2007

    TSI’s tRelational/DPS as a Case Study

  • 30November 2007

    Data modeling and design of the RDBMS tables

    High-productivity modeling environment

    Supports mapping to many complex simultaneous RDBMS targets

    Complete, native RDBMS schema generation

    Generation of capabilities to mimic ADABAS features

    Complete, standards-based Java data-access layer generation

    TSI’s tRelational/DPS as a Case Study

  • 31November 2007

    TSI’s tRelational/DPS as a Case Study

    The product implements and applies detailed technical

    knowledge of the source and target

  • 32November 2007

    Native interfaces to source and target

    Direct processing of static data source generated by ADABAS utility

    Native formatting of output for RDBMS loader

    Supports an evolutionary methodology

    High productivity and performance enables iteration

    Early and iterative response to data quality issues

    Facilitates prototyping of new applications

    TSI’s tRelational/DPS as a Case Study

  • 33November 2007

    Minimized extraction impact

    No workload placed on ADABAS

    Support for phased ADABAS-to-RDBMS migration project implementation

    Native log processing for CDC

    Transformation to RDBMS-native SQL

    Seamless adaptation to real-time processing with DPSync

    Bidirectional and multidirectional replication capabilities available via tcACCESS and DataMirror

    TSI’s tRelational/DPS as a Case Study

  • 34November 2007

    TSI has led the ADABAS-to-RDBMS market with constant innovation for over 12 years

    Products have benefited from know-how derived from real-world projects

    TSI is able to offer an array of solutions to meet any ADABAS data modernization requirement

    TSI partners with companies like Microsoft, Oracle, TSRI, Anubex, and innoWake to mitigate the risk of ADABAS data modernization

    Experience Counts

  • 35November 2007