modernizing data: issues and approaches nams-if data modernization... · 2007. 11. 13. · legacy...
TRANSCRIPT
-
1November 2007
Modernizing Data:
Issues and Approaches
-
2November 2007
Introduction to Treehouse Software
Issues
Approaches and Solutions
Keys to Success: Standards, Metadata and Experience
Q&A
Agenda
-
3November 2007
Established in 1982
Leading ISV for Software AG
70% penetration in North American market
Consultants with 20+ years SAG experience
More than 300 man years of experience
Over 700 customers worldwide
20 products
30 employees
Introduction to Treehouse Software
-
4November 2007
Strategic Partners
Anubex
ACS Government Systems
Benaroya
B.O.S. Software Service und
Vertrieb GmbH
CCA Software Pty. Ltd.
CA
Cronus Consulting Pty
Cubeware GmbH
DataMirror Corporation (IBM)
EDS Atlantic Region
E-Net Corporation
Harvest Moon Computing Pty. Ltd.
Intelligent Commerce Network LLC
NatWorks, Inc.
Policy Studies, Inc.
SAIC
Solution Specialists, Inc.
The Software Revolution, Inc.
Wostmann & Associates
Technology Partners
Fujitsu Siemens Computers
Hewlett-Packard
IBM Corporation
Microsoft
Oracle
SAP*
Sun Microsystems
Marketing Representatives
AustraliaCCA Software Pty Ltd.
Brazil 3CON Consultoria e Sistemas Ltd.
Germany VersaTec IT Services
JapanE-Net Japan
Singapore and Far EastBachoJames Inc.
South AfricaBateleur Software (Pty) Ltd.
Partnerships give
Treehouse
international
presence and
broaden our
capabilities
-
5November 2007
Treehouse Offerings
Data Management
• ADAMAGIC
• ADAREORG
• ADASTRIP
• AUDITRE
• SECURITRE
• TRIM
Mainframe Emulation
• SEDIT
• S/REXX
• S/REXX Debugger
Professional Services
• Data Warehousing and
Business Intelligence
• Data Transfer
• Application Modernization
and Migration
• DBA Services
• Performance and Tuning
• Product Installation
• Training
Data Transfer
• tRelational
• tRelationalPC
• DPS
• DPSync
• DPS X-Link
• NatQuery
• NatCDC
• tcACCESS
• DataMirror Products
Application Management
• CHART
• N2O, N2O/3GL
• EspControl
• PROFILER
-
6November 2007
Treehouse Customers
-
7November 2007
"I would strongly recommend the use of the tRelational and DPS products for
any other State agencies that are currently utilizing ADABAS, especially if they
need to do any migration to another platform for data warehouses."
Nancy Kane
Systems & Programming DBA Manager
Arizona Department of Economic Security
"tRelational and DPS are the tools that allowed us to reach our goals in a
pragmatic and non-invasive way."
Roberto Licandro
Architect - IT Architectures B-Source SA
Treehouse Customers
-
8November 2007
"In our deployment of field identification
capabilities, having current data available on
criminal histories as part of that process is
critical.”
Captain Dickerson
Commander of Technical Services
Sacramento County Sheriff's Department
"DPSync allows us to take a big step forward
without replacing CJIS.”
Albert Locher
Assistant Chief in the
Sacramento County District Attorney’s Office
"...it was clear that tRelational/DPS was the leading
product for ADABAS-to-RDBMS data transfer.”
Doug Kauffroath
MIS Director for the Sacramento Superior Courts
Treehouse Customers
-
9November 2007
Federal Government
– Department of Justice
– National Business Center
State and Local Government
– State of Alaska – Department of Public Safety
– State of Arizona – Department of Economic Security
– State of Delaware – Department of Technology Services, Department of Revenue
– State of Utah – Department of Workforce Services
– State of Washington
– State of Hawaii
– County of Sacramento
– UK Driver and Vehicle Licensing Agency (DVLA)
– City of Calgary
– Service Nova Scotia
Sample Data Transfer Customers
-
10November 2007
Higher Education
– Case Western Reserve University
– Florida Community College Software Consortium
– Miami University of Ohio
– Penn State University
– University of Texas Austin
Private Sector
– Argus Health Systems
– B-Source
– Cutler Hammer
– Dexia Banque Internationale à Luxembourg S.A.
– First American Real Estate Solutions (FARES)
– Food Lion
– GMAC
– Kansas Nebraska Energy (KNE)
– National Fuel Gas
Sample Data Transfer Customers
-
11November 2007
Legacy modernization initiatives view data modernization as a small component
- yet -
Failure of the data modernization process can and does cause failure of the entire project
Issues in Data Modernization
Whenever a particular data representation is to be expressed in a
foreign representation, there is great opportunity for mistranslation
-
12November 2007
Lack of useful, up-to-date, a accurate documentation
Discrepancies between logical and physical definitions
Discrepancies between processing environments (e.g., development vs. production)
Discovery/Analysis of Source Data Structures
nd
-
13November 2007
Domain analysis
Unused elements
“Seasonal” or “state change” elements
Record typing
Index usage
Validation of application assumptions
Insight into Actual Data Content
-
14November 2007
Hidden aspects of the data model
Multiple usages of fields/columns, records/rows and files/tables
Feature and datatype usage that cannot be accurately represented in the target environment or database
Proprietary Aspects of the Source
-
15November 2007
Need to not only reflect the structure and content of the legacy source, but also its behavior
Must specify, design and implement appropriate mappings and transformations
Requires deep technical knowledge of the source and the target technologies
Data Modeling and Design of the Target
-
16November 2007
Need a central store of information on the source, the target, and the relationships between them
Must support iterative and interim development
Metadata needs modernization too—legacy data documentation should not be discarded
Maintaining the Metadata Repository
-
17November 2007
“Lowest common denominator” leaves too much to be desired
Throughput needs to be maximized
Ensure correct interpretation and formatting
Leveraging Rich Native Interfaces
-
18November 2007
Iteration is inevitable
Deployments must be tested repeatedly and comparatively
Data currency demands may escalate
Employing an Evolutionary Methodology
-
19November 2007
Different modernization approaches place different demands on aspects of the technology architecture
Necessity to make legacy data available to new applications rarely fits conveniently into “batch windows”
Tight DBA resource control may no longer be feasible
Impact on Production Systems
-
20November 2007
Approaches and Solutions
-
21November 2007
Extracting legacy data changes (CDC) by:
Comparison to prior state
Capture from database log or exit
Programmatically by timestamp
Generally requires initial Extract-Transform-Load (ETL)
May be accomplished in batch, in near-real time or in real time
Products include: DPSync, tcVISION, NatCDC, DataMirror Transformation Server (/ES)
Data Replication
-
22November 2007
Advantages:
Low impact on source
Low volume of data to be transmitted
Can process only committed transactions
Disadvantages
Always some degree of latency
Multidirectional replication is extremely complex
Requires robust recovery and monitoring capabilities
Data Replication
-
23November 2007
Inserting one or more technology layers (servers) between the legacy source and the consuming application
May be read-only or read/write
Products include: tcACCESS, DPS X-Link
SQL/SOA Enablement
-
24November 2007
Advantages:
Zero latency in data
Legacy data directly integrated into applications
Disadvantages
Difficult to control amount of data transmitted and resources used
Query isolation may be assumed but not delivered
Consuming applications need metadata
SQL/SOA Enablement
-
25November 2007
Extraction of source data with appropriate transformation to meet the needs of the target platforms and applications
Products include: tRelational/DPS, NatQuery
Data Migration
-
26November 2007
Advantages:
Applications work with native, modern data sources—no compromises
Ultimately, reduction in cost and complexity of technology architecture
Disadvantages
―Big bangs‖ are not feasible, so coexistence/phased implementation support is needed
Must select a point in time to migrate
Data Migration
-
27November 2007
Keys to Success
-
28November 2007
Discovery and analysis of ADABAS source data structures
Compares FDT to PREDICT
Flags post-implementation changes to either
Insight into actual data content and use of proprietary ADABAS datatypes and features
Analyses of MU/PE fields, datatypes
Validation of application assumptions
All information captured into metadata repository
TSI’s tRelational/DPS as a Case Study
-
29November 2007
TSI’s tRelational/DPS as a Case Study
-
30November 2007
Data modeling and design of the RDBMS tables
High-productivity modeling environment
Supports mapping to many complex simultaneous RDBMS targets
Complete, native RDBMS schema generation
Generation of capabilities to mimic ADABAS features
Complete, standards-based Java data-access layer generation
TSI’s tRelational/DPS as a Case Study
-
31November 2007
TSI’s tRelational/DPS as a Case Study
The product implements and applies detailed technical
knowledge of the source and target
-
32November 2007
Native interfaces to source and target
Direct processing of static data source generated by ADABAS utility
Native formatting of output for RDBMS loader
Supports an evolutionary methodology
High productivity and performance enables iteration
Early and iterative response to data quality issues
Facilitates prototyping of new applications
TSI’s tRelational/DPS as a Case Study
-
33November 2007
Minimized extraction impact
No workload placed on ADABAS
Support for phased ADABAS-to-RDBMS migration project implementation
Native log processing for CDC
Transformation to RDBMS-native SQL
Seamless adaptation to real-time processing with DPSync
Bidirectional and multidirectional replication capabilities available via tcACCESS and DataMirror
TSI’s tRelational/DPS as a Case Study
-
34November 2007
TSI has led the ADABAS-to-RDBMS market with constant innovation for over 12 years
Products have benefited from know-how derived from real-world projects
TSI is able to offer an array of solutions to meet any ADABAS data modernization requirement
TSI partners with companies like Microsoft, Oracle, TSRI, Anubex, and innoWake to mitigate the risk of ADABAS data modernization
Experience Counts
-
35November 2007