expert session - data migration
TRANSCRIPT
Expert Session
- Data Migration -
Philippe Despierre & Jan Van Vlaenderen - 27/05/2021
Short introduction
2
Jan Van Vlaenderen
- Started career as a SCM consultant in the domain of MM, WM and QM
- Data migration lead in projects since 2009
Philippe Despierre
- Started career as a FI consultant- Community lead financial
solutions since 2018- Data addict
Agenda
3
Inform on a data migration strategy
Inform on ETL methodology, based on Best Practises
Learn some highlights on data cleansing
Learn what’s on the LTMC tool by SAP S/4HANA release 2020
Data Migration Strategy - Purpose and Scope
What is “data migration” ?
5
According to Wikipedia
Data migration is the process of transferring data between storage types, formats, or computer systems. It is a
key consideration for any system implementation, upgrade or consolidation.
BUT…
the goal of a data migration project is not simply to move and transform data from one system/format to
another; it is to ensure that the moved data is of high quality, is fit-for-use, and supports the underlying
business processes and operational goals of the organization!
Data migration is necessary
It is one of the key cross stream tasks during a transition. No data, no Go-live.
Unfortunately, it is often underestimated, it is not just ‘pressing a button’.
Understand your data migration requirements early and plan for it accordingly.
Data migration is 90% preparation.
Data Migration Approach
6
Creation of data migration strategy document
Identification of objects and scope finalization
Data extract and upload methodology (auto, semi-auto,
manual)
Identify dependencies
Creation of data migration schedule
Effort estimate
Resource requirements
Functional design per object
Signoff functional design
Expertum DM Strategy Document
Object scope list
Functional design per object
Data Migration Schedule
Resource Plan
Preparation
Deliverables
Approach - ETL based on Best PractisesLegacy Data Environment
XML
Applications
Databases
Flat Files/Excel
Loading
Extract & Profile SAP
Target Environment
Cleanse
Name Parsing
Address Parsing & Correction
Material/Product Parsing
MatchingSAP
Configuration ExtractionReconciliation
Performance Analysis
Dashboards and Business Reporting
Files
IDocs
Pre-Built load routines for SAP
Objects
Validate & Load
Business Validation Rules
Automatic SAP Config
Validation
Transform
TransformData into SAP
structure
7
Approach - the bigger picture
8
Approach - ETL based on Best Practises
9
Un
it T
est Transform
• Transform extracted data in
upload datasheet as per the
transformation rule
Extract
• Creation of extract tools
• Extract restricted data
Load
• Creation of data upload tool
• Finalizing upload sheet format
• Upload of representative data
• Deliver report
Tria
l M
igra
tio
n
I
Transform
• Transform extracted data in
upload datasheet as per the
transformation rule
Extract
• Finalization of extract tools
• Extract restricted but
representative data
Load
• Finalization of data upload
tool
• Upload transformed data
• Delivery report
Tria
l M
igra
tio
n
II
Transform
• Transform extracted data in
upload datasheet as per the
transformation rule
Extract
• Extract data for all erroneous
objects
Load
• Upload of extracted data
• Deliver report
Dry
ru
n Transform
• Transform extracted data in
upload datasheet as per the
transformation rule
Extract
• Extract data for all the object
(partial)
Load
• Upload of extracted data
• Deliver report
Fin
al M
igra
tio
n
/ d
elta
Transform
• Transform extracted data in
upload datasheet as per the
transformation rule
• Sign-off
Extract
• Extract complete data after
freeze
• Sign-off
Load
• Upload of complete data set
• Deliver report
• Sign-off
Unit testing
Technical creation load tools
Technical test
Trial Migration 1
Integration testing
Limited data sets
Trial Migration 2
User Acceptance Testing
Quality data sets
Higher volumes
Dry run
Dress Rehearsal for golive
Final Data Migration
Golive
Data Migration - ETL methodology based on Best Practises
ETL - KEY step 1 : Extraction
11
Data Discovery
Identify master data
Different applications
Data Cleansing
Data purging – obsolete data deleted or marked for
deletion
Data corrections
• Identify and correct in current legacy
• Data accuracy (net weight, material descriptions, … ) are vital
Data Extraction
Validation of the cleansed data – prior
Can be automatic, semi-auto or manual
Repetitive and successive runs improving the accuracy
and completeness of the extracted data
Step 1Extraction
V
A
L
I
D
A
T
I
O
N
ETL - KEY step 2 : Transform
12
Local data mapping
From local source to destination SAP system
Identify those for which no origin was found
Data harmonization
Identification of ‘global’ data (ie extending of existing data)
Data conversion
Convert legacy system data to meet the business
requirements
Business rules will be applied
Can be automatic, semi-auto or manual
Enrichment
Enrich the inexistent data in source system (default value,
manually, …)
Validation
Validate the content of input files
Business and system validation rules will be applied
Step 2Transform
V
A
L
I
D
A
T
I
O
N
ETL - KEY step 3 : Load
13
Data from the repository (or manual data sheets) will be uploaded
based on rules and (standard) upload tools.
Progressive approach
Master data or called static data migration (material master,
business partners, …)
Data migrated over and extended period of time
Change-over approach
Transactional data or called dynamic data migration (stock, open
sales orders, …)
Data migrated during a short cutover period (e.g. weekend)
Data upload methodology depends
Quantity of the data
Availability of the data
Complexity of the data
Importance for business flows
V
A
L
I
D
A
T
I
O
N
Step 3Load
Validation during different phases
14
Validation is a crucial step in the full process. Based on a variation of business and/or system rules, data will have
to be validated and signed off after extraction (confirm data profiling is correct), transformation (confirm data is
converted to the agreed values) and after load (confirm data is loaded as expected, in the required quality –
functionally and technically)
Datavalidation
Legacysystems
Conversionenvironment
SAP
Datavalidation
Datavalidation
Extract Transform Load
Tracked in planning – defect logging and handling
Data Migration Activities - Roles
Team structure
16
Data
Migration
Team
Core
Functional
Team
(Local)
Business
Team
Team structure
17
Data
Migration
Team
Core
Functional
Team
(Local)
Business
Team
Customer, Expertum,
DM Manager
R&R - Data migration team
18
The deliverables expected
Data Migration Strategy
Data Migration Plan
Data load tools (Upload procedures) - with dev team
Data migration load template
Data load
Data
Migration
Team
Customer,
Expertum,
DM Manager
R&R - Functional core team
19
The deliverables expected
Configuration upload environment
Functional specification for data load tool
Data migration mapping templates for data structures
Data validation
Core
Functional
Team
Customer,
Expertum,
DM Manager
R&R - Functional core team
20
The deliverables expected
Source data cleansing
(Local) data mapping rules
(Local) data extraction, cleansing, profiling, and
validation tools
Country / Site specific data migration mapping
Data validation
(Local)
Business
Team
Customer,
Expertum,
DM Manager
Key topics to take “home”
21
No data, no system
Preparation is key within a data migration track
Involvement of each, cross functional, department of the company is required
Whether it is IT, business, developer, key user…. You will need them to make it a success story !
Some highlights on Data Cleansing
Functional Data Migration approach
Any
SystemSAP S/4
Transformation
Engine
Extract Load
Functional Data Migration approach
Any
System
Other
Sources
Source Table 1
Source Table 2
Source Table n
Scoring
Cleansing
Base table 1
Base table x
Transformation
De
pe
nd
en
cyM
od
el
Trans-formed
data
Target
Transformation Engine
Check tables
What’s on the LTMC tool by SAP S/4HANA release 2020
To conclude
26
Leave your contacts info to get the presentation or recording
Our next webinar - Thursday June 24th 2021 @ 4 PM
Financial KPI reporting at your fingertips
https://www.expertum.net/events-blog/financial-kpi-reporting-0
Survey : if you still have a specific question regarding this topic, you can leave it here as
well
www.expertum.net
Inspire by Experience.
Thanks for listening! Any questions?
Jan Van VlaenderenProject lead - MDM
[email protected]+32 478 67 47 91
Philippe DespierreCommunity lead finance
[email protected]+32 496 808 943