bw350 sap components extraction

SAP AG BW350 Preface-1

SAP AG 2002

BW350

SAP AG 2002

! "

R/3 System, Release 3.0B 2003/Q1 Material number 50055941


SAP AG 2002

Copyright 2002 SAP AG . All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG . The information contained herein may be changed without prior notice.

All rights reserved.

Copyright

Trademarks: Some software products marketed by SAP AG and its distributors contain proprietary software

components of other software vendors.

Microsoft, WINDOWS, NT, EXCEL, Word, PowerPoint and SQL Server are registered trademarks of Microsoft Corporation.

IBM, DB2, OS/2, DB2/6000, Parallel Sysplex, MVS/ESA, RS/6000, AIX, S/390, AS/400, OS/390, and OS/400 are registered trademarks of IBM Corporation.

ORACLE is a registered trademark of ORACLE Corporation. INFORMIX-OnLine for SAP and INFORMIX Dynamic ServerTM are registered trademarks of

Informix Software Incorporated. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. HTML, DHTML, XML, XHTML are trademarks or registered trademarks of W3C, World Wide

Web Consortium, Massachusetts Institute of Technology.

JAVA is a registered trademark of Sun Microsystems, Inc. JAVASCRIPT is a registered trademark of Sun Microsystems, Inc., used under license for

technology invented and implemented by Netscape.

SAP, SAP Logo, R/2, RIVA, R/3, ABAP, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch, BAPI, SAPPHIRE, Management Cockpit, mySAP.com Logo and mySAP.com are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other products mentioned are trademarks or registered trademarks of their respective companies.


SAP AG 2002

Course Prerequisites

#$#$#$#$

!"#$%&

%%$ %%$ %%$ %%$

'(


SAP AG 2002

Target Group

Members of the project team with knowledge and experience of the BW Administrator Workbench area of BW

& $%'& $%'& $%'& $%'

Notes to the user The training materials are not teach-yourself programs. They complement the course instructor's

explanations. Your material includes space for noting down this additional information. There may not be enough time to do all the exercises during the course. The exercises are intended to

be additional examples that are dealt with during the course. Participants can also use them to deepen their knowledge after the course.


SAP AG 2002

BW350 Course Overview

BW Overview - SAP R/3 Extraction

Extracting Data from SAP Source Systems: Basics

Delta Management of Transaction Data

Extracting Data from FI-SL

Extracting CRM Data

Tools for Generic Data Extraction

Enhancing Business Content

LO Data Extraction

Extracting Data from CO-PA

SAP AG BW350 1-1

SAP AG 2002

Concepts of Extracting Data from SAP Components

The Extraction Process Business Content (BCT) Extractors Activating Business Content Extractors


SAP AG BW350 1-2

SAP AG 2002

BW Overview - SAP R/3 Extraction: Unit Objectives

illustrate the various concepts behind data extraction and give examples of ways in which data is extracted from SAP systems

explain the flow of data during its extraction into SAP BW

activate Business Content extractors

SAP AG BW350 1-3

SAP AG 2002

BW Overview - SAP R/3 Extraction: Contents





Extracting CRM Data



LO Data Extraction


SAP AG BW350 1-4

SAP AG 2002

BW Overview - SAP R/3 Extraction: Overview Diagram

Enhancements

ApplicationApplication--SpecificSpecific

Types of ExtractorTypes of Extractor

BW ContentBW ContentExtractorsExtractors

CrossCross--ApplicationApplication

GenericGenericExtractorsExtractors

CustomerCustomer--GeneratedGeneratedExtractorsExtractors

Application-Specific

DB Tables

LIS

FI-SL

CO-PA

DB View

Function Module

FICOHR

SAP CRMLO Cockpit


DB Tables

Various SAPDB Tables

InfoSet

Basics Delta Method

Application-specific extractors - extractors developed by SAP or by customers themselves - extract data from specific tables that are connected to a corresponding application area.

The data in generic extractors can come from various sources (logical databases, cluster tables, transparent tables). This tool is not application-specific and can be used in situations where the other types of extractor are unavailable.

InfoSet Query: The query tool is called the ABAP Query or the SAP Query depending on the BW release you refer to. Although they are very similar, there are a few small differences in the functions available in the two tools.

SAP AG BW350 1-5

SAP AG 2002

BW Overview - SAP R/3 Extraction:Business Scenario

t the start of a SAP BW project (and even if you already have a 'live' BW system) you need to think about the reports that you want to run and the data you will need to be able to run them.ith this in mind, you need to decide which of the data sources from SAP Business Content you can use as default data sources or adapt to meet your requirements.

SAP AG BW350 1-6

SAP AG 2002

The 3-Tiered Architecture of SAP BW

Master DataMaster Data

Source Systems:

Components

OLAP:

OLAP ProcessorOLAP Processor

Staging EngineStaging Engine

Metadata RepositoryInfoProvider

Mobile Intelligence

- EUR5.000,00 EUR

10.000,00 EUR15.000,00 EUR20.000,00 EUR25.000,00 EUR30.000,00 EUR35.000,00 EUR

BeckerBerlin

Bayer VW Daimler

Incoming Orders

Auftragseingang

Web ApplicationBEx Analyzer (MS Excel)

BW

SE

RVER

BW SE

RVER

Adm

inis

tra

tor

Wo

rkbe

nch

Adm

inis

trat

or

Wo

rkbe

nc

h

Transaction DataTransaction Data

DBConnectXML

This slide gives an overview of the SAP Business Information Warehouse architecture in a heterogenous system landscape.

The lower level shows the source systems.

As components of the Business Framework, source systems and the Business Information Warehouse use BAPIs (Business Application Programming Interfaces) to communicate with one another. SAP delivers extraction mechanisms for live data. These mechanisms prepare data in mySAP.com components for extraction into the Business Information Warehouse.

In non-SAP source systems, the BW BAPI interface allows you to use third-party extraction tools to prepare and extract data.

Metadata and application data are managed on the Business Information Warehouse server. You use the Administrator Workbench in the Business Information Warehouse to manage the

different source systems. You use the scheduler and the monitor in the Administrator Workbench to schedule and monitor the transfer of metadata and transaction data from the assigned legacy systems.

The Business Explorer and its reporting tools form a third layer.

SAP AG BW350 1-7

SAP AG 2002

Source Systems in SAP BW

Source systems are created in the source system area of the Adminstrator Workbench. This course shall focus exclusively on the extraction of data from SAP systems.

Types of source system:

SAP systems with SAP Basis Release 3.1I and above Database systems (using SAP DB Connect) External systems (using staging BAPIs, for example, Ascential Datastage) File systems (files) SAP Business Information Warehouse

Data is extracted from the source system and updated using a DataSource.

SAP AG BW350 1-8

SAP AG 2002

Data Targets

InfoSource

Mapping & Transfer Mapping & Transfer RulesRules

Update RulesUpdate Rules

Mapping & Transfer Mapping & Transfer RulesRules

DataSource

BW Metadata

Replicated MetadataDataSource

XML DataXML Data

...

OtherOther

SAP SystemSAP

System

FileFile

DataSource

DBConnect

DBConnect

Extracting Data from SAP Source Systems

Extractors belong to the data staging mechanisms in the SAP source system. An extractor can supply the extract structure of a DataSource with data from datasets in the SAP source system.

A metadata upload replicates the DataSource and its relevant properties in BW. In BW, the DataSource can be assigned to an InfoSource. The fields of the DataSource can be assigned to InfoObjects in BW. Once you have maintained the transfer rules in the BW Administrator Workbench, and thereby determined the flow of data, you can schedule an InfoPackage in the scheduler. The process of loading the data is triggered by a request IDoc to the source system.

There are application-specific extractors that are programmed specifically for the DataSources delivered with BW Business Content. These application-specific extractors also supply the extract structures of Business Content DataSources with data.

There are generic extractors that are used to extract additional data from a SAP source system and transfer it to BW. A generic extractor does not know which tables it should read data from or which structure it should read the data to. When it is called it gets this information from the name of the DataSource for which the data is going to be extracted. A generic extractor can therefore supply various extract structures and DataSources with data.

Generic data extraction is used in various application areas of the R/3 source system, such as LIS, CO-PA, FI-SL and HR. LIS, for example, uses a generic extractor to read InfoStructures. DataSources are generated on the basis of these (individually) defined InfoStructures. We speak of customer-generated DataSources when we refer to the generic extraction of data from applications.

Independently of any particular application, you can also extract master data attributes, master data texts, or transaction data generically from any transparent tables, database views, or function areas of SAP Query or by using function modules. You can generate your own DataSources to do this. In this case we speak of generic DataSources.

The data for these types of DataSource is read generically and transferred to BW. Generic extractors allow you to extract data that is not available through Business Content.

SAP AG BW350 1-9

SAP AG 2002

DataSources in SAP Systems

Transparent Table, View,Transparent Table, View,Function Module, SAP QueryFunction Module, SAP Query

DataSource determines which fields of a business event can be transferred into SAP BW.hese fields are included in the extract structure. The DataSource also describes the properties of the extractor.ypes of DataSource: Transaction data, attributes, texts, and hierarchies.

TransparentTable

TransparentTable

TransparentTable

TransparentTable

In a source system, data that logically belongs together is stored in the form of DataSources. A DataSource contains a number of fields forming a flat structure (extract structure) that is used to

transfer data to BW. A DataSource also describes the properties of the extractor belonging to it with regards to how data is transferred to BW. During a metadata upload, the DataSource properties that are relevant to BW are replicated in the BW system.

There are four types of DataSource: DataSources for transaction data DataSources for master data DataSources for attributes DataSources for texts DataSources for hierarchies

Business Content DataSources The DataSources delivered with SAP Business Content and the DataSources delivered by partners

come in a delivery version (D-version). If you want to use a Business Content DataSource or a Partner Content DataSource to transfer data from a source system into BW, you have to first convert the DataSource from a D-version into an active version (A-version).

In the source system, the DataSources are assigned to particular application components. If , in the source system view of the BW Administrator Workbench - Modeling, you want to display the DataSources according to the application component hierarchy, you have to convert the DataSources from the D-version to the A-version in the source system.

Source system DataSources are processed in the customizing area of the extractors. In the BW Administrator Workbench - Modeling, you get to the customizing area through the context menu (right mouse click) of the source system in the source system tree. In the DataSource overview of a source system, you can access the DataSource maintenance screens directly by choosing the "Maintain" option from the context menu of a DataSource in the source system.

SAP AG BW350 1-10

SAP AG 2002

InfoSource

n InfoSource prepares consolidated data for update in the SAP BW data targets.

he structure of an InfoSource is determined by the communication structure that is made up of InfoObjects.

ypes of InfoSource

Direct InfoSource

Flexible InfoSource

An InfoSource in BW describes all the data available for a particular business event or type of business event, for example, cost center accounting. An InfoSource is a collection of related pieces of information that have been summarized into a single unit. InfoSources contain either transaction data or master data (attriubtes, text, and hierarchies.)

An InfoSource always consists of a quantity of InfoObjects that belong together logically. The structure in which these are stored is called a communication structure.

If, out of different source systems, there are fields that logically belong together, these fields can be combined to form an InfoSource in BW. The source system can be from any release. If the InfoSource is an InfoSource with a flexible update, the update rules are used to load the data from the communication structure of the InfoSource into additional data targets. With InfoSources with a direct update, master data (without update rules) is written to the master data tables directly.

For both types of InfoSource, the uploaded data is transformed by the transfer rules that have been created for each particular combination of InfoSource and source system and for each InfoObject of the communication structure. A data target can be supplied with data from several different InfoSources that in turn can be supplied with data from several different source systems.

InfoSources are listed in the Administrator Workbench in the InfoSource tree below an application component.

SAP AG BW350 1-11

SAP AG 2002

Attributes Texts

Flexible InfoSource

InfoSource with Flexible Update

ommunication structure

pdate in data targets by using update ruleso connection to hierarchy-type DataSources

InfoCube ODS Object InfoObject

Transaction Data DataSource

Texts DataSource

Attributes DataSource

With InfoSources with a flexible update, update rules are used to load the data from the communication structure into the data targets (InfoCube, ODS object, master data.) An InfoSource can supply several data targets with data. An InfoSource can contain transaction data and master data.

This function is not available with hierarchies. In releases prior to BW 3.0A, only transaction data can be updated flexibly and only master data can

be updated directly. There is therefore a difference between InfoSources for master data and InfoSources for transaction data. This difference does not, however, apply to BW 3.0A and higher releases, because both transaction data and master data can be updated flexibly in these later releases. This also means that it is not immediately obvious in the system, whether an InfoSource with a flexible update refers to transaction data or master data. It is therefore a good idea to indicate this in the name of the InfoSource.

There are the following update options:

Data from InfoSources with master data or transaction data can be stored directly in ODS objects. From the ODS objects, you can use update rules to update the data in additional ODS objects,

InfoCubes, or master data tables. It is also possible to update data in InfoCubes or master data tables without using ODS objects in

between.

SAP AG BW350 1-12

SAP AG 2002

Attributes Texts

= Direct InfoSource

InfoSource with Direct Update

Hierarchies

irect update of master data in the master data tables of an InfoObject Attributes Texts Hierarchies

o connection to transactional-type DataSourcespdate rules not used

Attributes DataSource

Texts DataSource

Hierarchies DataSource

InfoObject

With an InfoSource with a direct update, master data (characteristics with attributes, texts, or hierarchies) from an InfoObject can be updated directly (without update rules, and using only transfer rules) in the master data table. You need to assign an application component to the InfoSource to be able to do this. The characteristic is displayed in the InfoSource tree in the Administrator Workbench. Here you can assign DataSources and source systems to the characteristic. Finally, you can upload attributes, texts, and hierarchies for the characterisitc.

In the following cases, it is not possible to use an InfoObject as an InfoSource with a direct update: In combination with the 0SOURSYSTEM (source system ID) characteristic. If the characteristic has neither master data, texts, nor hierarchies. Accordingly, it is not possible

to load data for the characteristic. If you are using a unit or a key figure and not a characteristic.

The characteristic must also be an InfoSource with a direct update, if you want to generate an export DataSource for the characteristic.

SAP AG BW350 1-13

SAP AG 2002

Data Flow in SAP BW

Logical View Technical View

Data Targets

InfoSource


Replicated Metadata

DataSource

DataSource


Communication Structure

!

"Transfer Structure

Extract Structure

Transfer Structure

Mapping & Mapping & Transfer RulesTransfer Rules

"

The steps on this slide are discussed on the next page. A DataSource in combination with an InfoSource is a type of infrastructure that allows the cross-

system transfer of data between source systems and SAP BW. This infrastructure is made up of various DDIC structures and the transformation rules that link them together. DataSources therefore control the flow of data from the extraction source structure in the source system to the communication structure in SAP BW that is used to supply the data targets with data.

DataSources/InfoSources can supply either transaction data (stored in InfoCubes) or master data (attributes, texts, and hierarchies - stored in separate, transparent tables.) DataSources/InfoSources for transaction data and master data are almost identical in the way they are structured.

DataSources describe the quantity of all the data in a source system that is available for a business event or type of business event, for example, cost center accounting. InfoSources represent the total amount of data availabe in BW for the same business events, but independently of any source systems. For this reason, every live SAP BW environment contains a large number of DataSources/InfoSources that describe individual operations within the applications that are going to be analyzed.

Each time a relationship between a DataSource and an InfoSource is activated(when the transfer rules are activated) the corresponding transfer structures and rules are generated. Transfer structures always exist as pairs - in a source system and in the associated SAP BW system. The transfer structure transfers data in the format of the original application from a source system into SAP BW. From SAP BW, transformation rules that exist only in BW forward the data to the communication structure of the InfoSource. The communication structure is independent of the source system and contains all the fields from the InfoSource that it represents in BW.

SAP AG BW350 1-14

SAP AG 2002

Determining the Flow of Data

#"$#"$#"$#"$Create a DataSource in the SAP source system#!$#!$#!$#!$Replicate the DataSource DataSource is replicated in the SAP BW Fields from the extract structure that have not been hidden are

available in BW in a template DataSource#$#$#$#$Create an InfoSource and maintain the communication structure The communication structure is created manually or is generated

from the fields selected from the template DataSource in BW# $# $# $# $Create transfer rules The DataSource is assigned to an InfoSource The transfer structure is generated from the fields selected from the

template DataSource in BW and the source system Fields in the transfer structure are assigned to InfoObjects

!

"

Once a DataSource has been generated in the source system, the metadata has to be replicated in the SAP BW system. During this process, a template DataSource is generated to make it easier to generate the communication structure. The fields in the DataSource that have been assigned to BW InfoObjects can be transferred to the communication structure as the global definition of the InfoSource.

The fields in the communication structure have to be supplied with data by the transfer rules. This can be done using a field in the template DataSource, a fixed value, or a function exit. The transfer structure is determined by all the fields from the template DataSource that are used to populate the communication structure.

The transfer structure is generated automatically in the R/3 system and the BW system when the transfer rules are activated in SAP BW.

SAP AG BW350 1-15

SAP AG 2002

Setting Transfer Rules

Transfer RulesTransfer StructureTransfer Structure

Communication Structure/Transfer Rules

0COMP_CODE

InfoObject Tp Rule0CO_AREACOSTC##0CURRENCY0PROFIT_CTR

0CO_AREACOSTC##0CURRENCY0PROFIT_CTR0COMP_CODE

InfoObject - Field AssignmentInfoObject Field

0COMP_CODE0PROFIT_CTR0CURRENCYCOSTC##

KOKRS

WAERSPRCTRBUKRS

KOSTL0CO_AREA

IDOC

Transfer Method

PSA

InfoSource COSTC## GR## Cost Center 13 Communication Structure

InfoObject Description Status Type Length 0CO_AREACOSTC##0CURRENCY0PROFIT_CTR

CHARCHAR

CHARCUKY

134

510

Controlling AreaGR## Cost Center 13 CurrencyProfit Center

Source System IDES R/3

DataSource ZGR##CC_ATTR

!

"

This slide gives an overview of how transfer rules are created: 1. Defining the transfer structure from the fields of the DataSource template (these were tranferred

to SAP BW when the data was replicated.) 2. Assigning the DataSource fields to the InfoObjects in the communication structure. 3. Creating transfer rules (optional):

- Transfer rules specify which fields in the transfer structure are transferred to which fields in the communication structure.

- For each InfoObject in the communication structure you choose only one of the three types of transfer rules:

The fields are copied from the transfer structure and are not modified. A fixed value can be assigned to an InfoObject. A routine (ABAP) or form field determines the value of the InfoObject.

The transfer rules do not take effect until they have been activated. When the transfer rules are activated, the transfer structure in SAP BW and in the SAP source system is also generated.

SAP AG BW350 1-16

SAP AG 2002

Business Content

InfoProvider

InfoSources

Queries

InfoObjects

Web Templates / Web Items Workbooks

Roles

Extractors, DataSources

Plug-In Installation in

the SAP Source System

Indi

vid

ual

izat

ion

Stan

dard

izat

ion

Business Content contains many objects that make it easier to implement BW and speed up the process of establishing an effective reporting tool.

These objects are arranged in a specific way to enable you to make the best use of them: There are roles for every area of your business and for many specific industries. These roles give users targeted access to the reports and applications that they need to complete their particular tasks.

In future, cross-component applications, called analytical applications, will provide targeted access to all the information related to a particular process. Analytical applications will not only allow you to analyze a particular process, but also enable direct actions and reactions in the systems delivering the data. Areas where analytical applications will be effective include Supply Chain Management (Supplier and Catalog Analysis, Manufacturing Analysis, and so on), Customer Relationship Management (e-Business Analysis, Market Exploration, and so on), or analyses in businesses, such as Performance Analysis, Payment Analysis, or Human Capital Analysis.

SAP delivery: SAP delivers Business Content objects as D-versions. Modifications are stored in the M-versions. Active Business Content is stored in the A-versions. Only objects that have been saved as A-versions are exported from the development system. When

they are imported into the target system, these objects are imported either into the active version directly or into a modified version, depending on what type of object they are.

SAP AG BW350 1-17

SAP AG 2002

DataSources: Business Content Extractors

Enhancements








DB Tables

LIS

FI-SL

CO-PA

DB View

Function Module

FICOHR

SAP CRMLO Cockpit


DB Tables


InfoSet

Basics Delta Method

Business Content Extractors:Application-specific extractors

that are activated when Business Content

DataSources are transferred

SAP provides extractors for almost all applications. These extractors are usually programmed on a fixed basis and shipped with Business Content. A PlugIn (AddOn component) is therefore required for a SAP source system.

These extractors are delivered by DataSources. SAP ships Business Content in the SAP delivery version (D-version). You have to convert Business Content objects into active versions (A-versions) before you are able to work with them. DataSources have to therefore be activated in the SAP source system first.

Follow these steps to install and activate one of the DataSources delivered with SAP Business Content: Select your source system from the source system tree in the BW Administrator Workbench, and click on the customizing extractors option from the context menu (right mouse click) followed by - Business Information Warehouse Business Content DataSources / Activate SAP Business Content - Transfer Business Content DataSources. Refer to the documentation on customizing extractors for more information on this topic. You can also customize extractors in the SAP source system directly (transaction SBIW).

When you transfer the application component hierarchy, you transfer and activate the application component hierarchy shipped by SAP and any partner application component hierarchies.

Whenever you activate and transport the application component hierarchy into a quality and productive environment, you must do so before you transport the DataSources. The object catalog entry is R3TR DSAA APCO.

SAP AG BW350 1-18

SAP AG 2002

%%%%eplicate DataSources Per application component / source system Individually, if there are changes to existing DataSources only

ctivate the corresponding Business Content in BW InfoObjects InfoSources and transfer rules InfoProviders and data targets with update rules Frontend objects: Queries, Web templates, and so on

dministrator Workbench Business Content Select the appropriate source system(s) to be transferred Note how the objects are grouped: Transfer Packages

Installing Business Content

Business Content DataSources from a source system are available in BW for transferring data only after you have converted them to their active versions in the source system and replicated them in the BW system.

You have to convert Business Content objects from the version SAP ships them in (D-version) to their active version (A-version) before you are able to work with them. Procedure: Choose "Administration" from the main menu of the SAP Business Information Warehouse,

Administrator Workbench From the left navigation window, choose the Business Content function area by clicking on the

corresponding pushbutton or using the "Goto" menu option, "Install Business Content". Note: If you want to assign source systems, choose the corresponding option. You get to the "Select source systems as default" dialog box. Select one or more source systems by typing the corresponding indicator or indicators into the default assignment column. The source system assignment is relevant only for transfer rules, file DataSources, and InfoPackages. If several source systems are available, only those objects assigned to the selected source systems are collected ready for transfer. Objects that are not assigned to the selected source systems are ignored.

Make the settings you require in the following selection lists: - Grouping - Collection Mode - Types of object and the selection of objects that you want to transfer.

Check the following settings in particular: - Source system: This setting determines for which source systems and which of their

DataSources transfer rules are activated. - Grouping: The settings "before", "afterwards", "before and afterwards", and "only necessary

objects" allow you to collect and activate a diverse range of objects from a Business Content data flow all at the same time. It is better to build the data flow step by step rather than transfer all the objects at the same time and lose track of what you are doing.

SAP AG BW350 1-19

SAP AG 2002

Data Flow Display

oth the Business Content version and the active version of a data flow can be displayed in the Metadata Repository:

n active data flow can also be displayed in the modeling view (InfoProvider / InfoSources) of the Administrator Workbench

Periodical Plant Stock2LIS_03_S198

Goods Movement: Plant2LIS_03_S196

0IC_C01 2LIS_03_S1987SZSNAZUEWO7XZ5W71Z3U36VE

0IC_C01 2LIS_03_S1961FZNSOZ8N0OFNONKGV1JLGP7J

Goods Movement: Plantand Periodical Plant Stock0IC_C01

It can be useful to display a Business Content data flow in the metadata repository to give you an idea of how complex the data flow is with regard to transferring data. Displaying the data flow per data target in the modeling part of the Administrator Workbench, for example, provides you with a quick way of finding out how a particular data target is supplied with data.

In the metadata repository, you can display the data flow for the Business Content version and the activated version of a data target.

Choose either the "Activated objects" option or the "Business Content" option. Select object: InfoCube 0SD_C03, for example. Network display of the data flow

In the Administrator Workbench window, you can display the data flow for InfoSources and InfoCubes dynamically.

The elements that are part of the processes of extraction, data staging, and data update are displayed.

Source system, DataSource, PSA, transfer rules, InfoSource, update rules, and InfoCube are all displayed.

Point of entry: InfoSource or InfoCube Context menu: "Display Data Flow"

Double-clicking on an element in the display takes you to the maintenance transactions. Elements are automatically displayed in the correct sequence.

SAP AG BW350 1-20

SAP AG 2002

Course Overview Diagram

Enhancements








DB Tables

LIS

FI-SL

CO-PA

DB View

Function Module

FICOHR

SAP CRMLO Cockpit


DB Tables


InfoSet

Basics Delta Method




SAP AG BW350 1-21

SAP AG 2002

BW Overview - SAP R/3 Extraction: Summary

&&&&

illustrate the various concepts behind data extraction and give examples of ways in which data is extracted from SAP systems

explain the concept of Business Content activate Business Content extractors

SAP AG BW350 2-1

SAP AG 2002

Creating a connection to a SAP source system Transfer methods General control parameters for transferring data BW Service API


SAP AG BW350 2-2

SAP AG 2002

Extracting Data from SAP Source Systems: Basics Unit Objectives

explain how SAP source systems are connected to BW give examples of data transfer methods and explain

how to maintain general control parameters explain the central role of the BW Service API in the

extraction process

SAP AG BW350 2-3

SAP AG 2002

Extracting Data from SAP Source Systems: Basics Contents





Extracting CRM Data



LO Data Extraction


SAP AG BW350 2-4

SAP AG 2002

Basics: Overview Diagram

Enhancements








DB Tables

LIS

FI-SL

CO-PA

DB View

Function Module

FICOHR

SAP CRMLO Cockpit


DB Tables


InfoSet

Basics Delta Method




SAP AG BW350 2-5

SAP AG 2002

Connecting SAP Source Systems to SAP BW

SAP source system is created in the Administrator Workbench.he data is extracted from the SAP source system and updated using a DataSource.he client of a SAP component is always connected as a source system in BW.

Modeling

InfoProvider

InfoObjects

InfoSources

Source Systems

PSA

Source systems Technical Name.SOURCESYSTEMTR

Select Source System Type

SAP System from Release 3.0E (Automatic Creation)SAP System from Release 3.0D (Manual Creation)SAP Business Information WarehouseFile System (Manual Metadata, Data using File Interface)Database System (Data and Metadata using SAP DB Connect)External System (Data and Metadata Transfer using Staging BAPIs)

BW is a data warehouse and as such it is closely linked in a network to other systems. Usually, there are several source systems connected to a BW system, but the BW system itself can also act as a source system to another BW system. This is where we speak of data marts. Since modifications to a system that is part of a source system/BW connection have an impact on all the connected systems, they cannot be treated in isolation.

A connection between a source system and a BW system consists of a series of individual connections and settings that are made in both systems: RFC Connections ALE Settings Partner Agreements Ports IDoc Types IDoc Segments

BW Settings The following selection options are available when you create a SAP source system:

SAP System from Release 3.0E (Automatic Creation) SAP System from Release 3.0D (Manual Creation) SAP Business Information Warehouse

When you select the "automatic creation" option, the required RFC connections are generated at the same time (recommended.)

If you choose to create the source system manually, you have to create the RFC connections between the BW system and the SAP source system first.

The chapter entitled "Prerequisites for Creating a SAP Source System" in the BW documentation contains additional important information on this topic.

SAP AG BW350 2-6

SAP AG 2002

Connecting SAP Source Systems to SAP BW

FC (Remote Function Call) connections are created between the systemsne user is required in each of the systems to enable the systems to communicate with one anotherased on the logical system name of the client in a SAP source system

BW System

SAP Source System

RFC Connection(Dialog)

RFC Connections(Background)

BW

R/3

You use transaction SM59 to maintain RFC connections. RFC connections are based on ALE technology (Application Link Enabling):

Technology for the construction and running of distributed applications. Provides for the efficient and controlled exchange of messages to keep data conistent in loosely

connected SAP application systems. The applications are integrated using synchronous and asynchronous communication, rather than a central database.

RFC connections are created based on the logical system name. This allows a client in a SAP system landscape to be identified uniquely.

Note the following with regard to background users: The default for the user should be set in the source system in BW customizing (transaction SPRO)

to guarantee that it is set correctly. Path: BW Customizing Implementation Guide Business Information Warehouse Connections to other systems Connections between SAP systems and BW

The following basic points apply: - Both background users should be of the "system" user-type - There are special authorization profiles for both background users (see SAP Service

Marketplace information.) This is the only way to prevent these users from being used incorrectly.

- In the RFC connection, the current password for the background user must be entered in the SAP source system that is going to be connected to the BW.

A second RFC connection (with _DIALOG as a prefix to the technical name) is created to enable you to jump from the BW system into the SAP source system (for example, out of the monitor assistant or to the customizing screens in the source system) during the online processing. No user is specified here. The BW user must therefore log on using his or her source system user name and password before jumping to the SAP source system. The notes in the SAP Service Marketplace contain detailed up-to-date information on this topic.

SAP AG BW350 2-7

SAP AG 2002

ALEInbox

Outbox

Data IDocsData IDocsInfo IDocstRFC (Connection Log)

Transfer Structure

(Connection Log) tRFC

Data PackagesData Packages

no PSA

SAP BW

SAP Source System

Transfer Methods

PSA

Extractor

SAPI

Extract Structure

Transfer StructureNon-SAP Source System (Third-Party Provider)

Info IDocs

ALEInbox

Outbox

The user selects the transfer method (PSA or IDoc) when he or she maintains the transfer rules. The process of loading the data is triggered by a request IDoc to the source system.

InfoIDocs are used with both types of transfer method. Only ALE is used to transfer the InfoIDocs. In BW, InfoIDocs are also used to update the monitor that oversees data loading process.

The transfer method only determines how the data is transferred.

IDoc transfer method: A data IDoc consists of a control record, a data record, and a status record. The control record contains administration information such as receiver, sender, and client. The status record describes the status of the IDoc, for example "modified".

Warning! The data stores in the ALE inbox and outbox have to be emptied or reorganized. PSA (tRFC) transfer method: With this transfer method, a transactional Remote Function Call is

used to transfer the data directly from the source system to the SAP BW. Here, there is the option of storing the data in the PSA (the tables have the same structure as the transfer structure.) This is the preferred transfer method, because it improves performance better than the IDoc method. When you use tRFCs to transfer data, the maximum number of fields that can be used is restricted to 255. The length of a data record is also restricted to 1962 bytes when you use tRFCs (IDoc --> 1000 bytes.)

SAP AG BW350 2-8

Overview PSA: Length of the data record is maximum 1962 bytes, while the number of fields per data record is

restricted to 255, and uses tRFC as transfer log. More commonly used technology, because it improves the performance of data loads and gives you

the option of storing incoming data in the PSA.

Advantage: Higher-performance, because it allows larger data packages to be transported. Option of working with error handling.

ALE (Data IDocs) Maximum length of the data records is 1000 bytes, uses tRFC as transfer log, and can be used with

hierarchies.

Advantage: More detailed logs thanks to the control record and the status record for the data IDoc.

SAP AG BW350 2-9

SAP AG 2002

General Control Parameters for Transferring Data

n BW customizing in the SAP source system, you can make general settings for the transfer of data from a SAP source system into a BW

mportant control parameters that are maintained here for allDataSources include: Maximum number of parallel processes during the transfer of data

Maximum number of rows in a data package

Maximum size of a data package

Frequency

he last two parameters can be adjusted in an InfoPackage to meet the requirements of a specific DataSource (within the context of the default values)

The Business Information Warehouse Implementation Guide contains information and details of how to extract and transfer data from a SAP source system into a BW system. You use transaction SBIW to access the implementation guide.

The Maintain Control Parameters for Data Transfer transaction is in the "General Settings" section. The parameters that you maintain here are valid as standard values across all DataSources. In certain circumstances, the extractors for the Business Content DataSources may need their own specific settings so as not to have a negative impact on the system performance. Details of the available parameters:

1. Source system: Enter the logical system of your source client and assign to it the control parameters that you have selected.

2. Maximum size of a data package: Individual data records are transferred to BW in data packages of variable size. You use these parameters to set an average size for a data package. If you do not maintain an entry here, a default setting of 10,000 kilobytes per data package is used to transfer the data. The amount of memory required does not depend on the setting controlling the size of the data packages alone. It also depends on the width of the transfer structure, the memory requirement of the extractor involved, and with large data packages, it also depends on the number of data records contained in the data package.

3. Maximum number of rows in a data package: With large data packages, the amount of memory required depends largely on the number of data records that are being transferred in each particular data package. You use these parameters to set the maximum number of data records that a data package can contain. The default setting is a maximum of 100,000 records per data pacakge. The maximum main memory requirement per data package is approximately 2*'Max. Rows'*1000 bytes.

SAP AG BW350 2-10

4. Frequency: You set the frequency at which InfoIDocs are sent by specifying the number of data IDocs or data packages that have to be transferred before an InfoIDoc is triggered. The default frequency is 1. This means that every data package is followed by an InfoIDoc. Generally, you should set a frequency of between 5 and 10, but not greater than 20. The bigger the data package the lower you should set the frequency. In the monitor in BW, each InfoIDoc tells you whether the data is being loaded successfully or not.

5. Maximum number of parallel processes for transferring data: This field is relevant to Release 3.1I and higher. Type in a number greater than 0. The default maximum number of parallel processes permitted at any one time is 2. The optimal setting for the parameters depends on how the application server that you use to transfer data is configured.

6. Target system for batch jobs: Specify the name of the application server on which you want to run the extraction job.

You can select the DataSource-specific settings for transferring data at the same time as you define an InfoPackage in the BW Administrator Workbench. InfoPackages menu under the Scheduler DataSource Default setting for data transfer

SAP AG BW350 2-11

SAP AG 2002

The BW Service API in the BW Architecture

Data Targets

InfoSource

Mapping & Mapping & Transfer Transfer RulesRules

Mapping &Mapping & Transfer Transfer RulesRules

DataSource

BW Metadata

Replicated MetadataDataSource

FileFileXML DataXML Data

...

OtherOther DB ConnectDB Connect

SAP System

SAP System


BWS-API

DataSource

BWS-API

BWS-API

BW Service API technology is used at various points within the BW architecture. To transfer data and metadata from SAP systems. To transfer data from XML files. To transfer data between BW data targets or from one BW system to another (data marts).

SAP AG BW350 2-12

SAP AG 2002

he BW Service API is the function in a SAP system that enables data from this source system to be replicated in a SAP BW

Controlling the data transfer

Managing metadata (DataSources / extractors)

he BW Service API (S-API) is based exclusively on SAP technology and is available for SAP systems with a Basis Release of 3.1I and higher

BW Service API: Overview

SAP AG 2002

For the latest information and availability refer to the SAP Service Marketplace under "Extractors / PlugIns"

SAP AG BW350 2-13

SAP AG 2002

BW Service API: Scope

onfiguration of source system connections

lobal control parameters for transferring data to SAP BW

ransfer methods: PSA and IDoc

elta Queue: Temporary storage location for modified data records

onitoring the data extraction process by exchanging messages between the systems!!!!ess downtime during initializations and early delta initialization

Delta Queue The delta queue is a data storage location for change records in the source system. In the delta queue,

data records are written either automatically by an update process in the source system or by an extraction process using a function module following a data request from BW.

In a delta request, the data records are transferred from the BW scheduler into BW itself.

In the delta queue, data is stored in a compressed form. It can be requested from several BW systems. The delta queue can also be repeated, because the data from the most recent extraction process is always stored there. The repeat mode of the delta queue is specific to the target system.

Early Delta Initialization During the initialization of the delta method for a DataSource, change records for certain application

extractors can be written to the delta queue or into the delta tables of the application.

If you use an early delta initialization, you do not have to stop updates to data in the source system during an initialization.

Less Downtime Less downtime is realized thanks to initialization runs in offline systems.

SAP AG BW350 2-14

SAP AG 2002

BW Service API: Scope

anaging DataSources (metadata) Business Information Warehouse Implementation Guide

Preparing DataSources and Business Content application components

Technical properties of a DataSource (extractor, extract structure, delta method, and so on)

Enhancements to Business Content DataSources

Replicating DataSources in SAP BW

mplementing a customer's own DataSources Generic extraction

emoteCube technology

est tool for the extraction of data

Delta Method The delta method is a property of the extractor. It tells you how the data is being transferred. As an

attribute of the DataSource, it tells you how the data in the DataSource is being transferred to the data target. This enables you to work out, for example, the data targets for which a DataSource is suitable, how to update, and how the serialization is done.

Business Content DataSources In the source system, the DataSources are assigned to particular application components. If , in the

source system view of the BW Administrator Workbench - Modeling, you want to display the DataSources according to the application component hierarchy, you have to convert the DataSources from the Business Content version to the active version in the source system.

Generic DataSources You can use generic data extraction to extract data from any transparent tables, database views, or

SAP Query function areas, independently of any sepcific application. You do not need any knowledge of ABAP programming to be able to do this. You can also use function modules in generic data extraction. Generic DataSources enable data to be extracted to BW that neither the Business Content DataSources nor the DataSources developed by customers in the applications would be able to provide.

SAP AG BW350 2-15

RemoteCube Technology A SAP RemoteCube is a RemoteCube that enables you to define queries with direct access to

transaction data in other SAP systems.

A RemoteCube is an InfoCube with transaction data that is managed externally, rather than in the Business Information Warehouse. Only the structure of the RemoteCube is defined in BW. A BAPI is used to read the data from another system.

A virtual InfoCube with services is an InfoCube without a physical data storage location of its own in BW. A user-defined function module is used as a data source. There is a range of options that allow you to determine the properties of the data source more specifically. Services for converting the parameters and the data are provided by the Data Manager in accordance with these properties.

Test Tool for Extracting Data From Service API 3.0B, you are able to test the extraction process for DataSources in the source

system independently of any BW system. This allows you to identify any problems before the actual extraction takes place. This option should be used in particular with generic extractions or enhancements to DataSources.

SAP AG BW350 2-16

SAP AG 2002

BW Service API: Delivery

SAP Basis Release of mySAP.com components

From SAP Basis 6.20 shipment of the S-API as a special AddOn - PI_BASIS

Recommended PI_BASIS Release?

< SAP Basis 6.20 > SAP Basis 6.20

Service APImySAP.com components

mySAP.com components

Shipment of the S-API with the standard installation of mySAP.com components

Recommended Release of mySAP.com components?

Shipment of the S-API with an AddOn -the PI (PlugIn)

Recommended PI Release?

PI (Service API) PI_BASIS (Service API) PI

mySAP.com components

Example: You want to connect a SAP R/3 Enterprise with SAP Basis Release 6.20 to a SAP BW 3.0B: S-API is shipped with a PI_BASIS. You also install PI containing Business Content for the SAP R/3 Enterprise (DataSources). PI 2002.1 / PI_BASIS 2002.1 is currently recommended for BW 3.0B (valid: October 2002)

PI Release 2002.1 is the highest Release for which a PI-A version is available. From PI-Release 2002.2 there is only a PI Version of the SAP R/3 Plug-In (for a Basis Release > 6.20 there is also a PI_BASIS). PI-A is a partial shipment of PI and contains only interfaces to SAP BW and SAP BBP 1.0B.

Refer to the installation guide for the corresponding mySAP.com component or the information in SAP Service Marketplace for more information on the installation of separate PlugIns (PI) and Plug-In Basis (PI_BASIS).

SAP AG BW350 2-17

SAP AG 2002

BW Service API: PI_BASIS and PI

I_BASIS is a new software component (available from SAP Basis Release 6.20) that will deliver the BW Service API technology needed to connect to a SAP BW systemlarge proportion of the Business Content application extractors will continue to be shipped with the PI software component Business Content DataSources with extractors and extract

structure

SAP AG BW350 2-18

SAP AG 2002

Extracting Data from SAP Source Systems: Basics Summary

""""

explain how SAP source systems are connected to BW give examples of data transfer methods and explain how to

maintain general control parameters explain the central role of the BW Service API in the

extraction process

SAP AG BW350 3-1

SAP AG 2002

BW Customizing Cockpit Comparing LO extraction with LIS extraction Converting data extraction from the LIS method

to the LO method

LO Data Extraction

SAP AG BW350 3-2

SAP AG 2002

LO Data Extraction: Unit Objectives

make the required extraction settings in the LO-BW Customizing Cockpit of the R/3 system

give examples of the different data extraction options in logistics, and list the advantages of the LO method over the LIS method

convert from the LIS info structure extraction method to the LO extraction method

When BW is installed on the R/3 side (plug-ins) the customer is able to install detailed LIS structures with the naming convention S26*. The only purpose of these structures is to collect detailed data from the document level for transfer into BW. They are not intended for reporting with LIS tools. These structures are often referred to as LIS transfer structures", but they are used only to collect R/3 data and transfer it to BW. Apart from the fact that they are not used for R/3 reporting, in all other respects, these structures are LIS InfoStructures with a high level of granularity.

SAP AG BW350 3-3

SAP AG 2002

LO Data Extraction: Contents





Extracting CRM Data



LO Data Extraction


SAP AG BW350 3-4

SAP AG 2002

LO Data Extraction: Overview Diagram

Enhancements








DB Tables

LIS

FI-SL

CO-PA

DB View

Function Module

FICOHR

SAP CRMLO Cockpit


DB Tables


InfoSet

Basics Delta Method

The transaction data that arrives in the the Business Information Warehouse by means of extractors originates from very different modules. For historical reasons, this requires very different extraction mechanisms.

SAP AG BW350 3-5

SAP AG 2002

our company uses the conventional SD data extraction method (LIS Extraction).he LIS structures that you use to collect sales data are BW-specific LIS structures (S26*) that were delivered as part of PI-99 and installed in R/3.ou are responsible for converting from LIS to the new LO extraction method, and want to find out all you can about it.

LO Data Extraction: Business Scenario

SAP AG BW350 3-6

SAP AG 2002

Billing

Delivery

Customerorder

Salesdocument

MCVBAK

MCVBRK

MCLIKP

Communicationstructure

...

...

...

Extractstructure

MC11VA0HDRMC11VA0ITMMC11VA0SCLMC11V_0ITMMC11V_0SCL

MC12VC0HDRMC12VC0ITMMC12VC0SCL

MC13VD0HDRMC13VD0ITM

Data/InfoSource

2LIS_11_VAHDR2LIS_11_VAHDR2LIS_11_VAITM2LIS_11_VAITM2LIS_11_VASCL2LIS_11_VASCL2LIS_11_V_ITM2LIS_11_V_ITM2LIS_11_V_SCL2LIS_11_V_SCL

2LIS_12_VCHDR2LIS_12_VCHDR2LIS_12_VCITM2LIS_12_VCITM2LIS_12_VCSCL2LIS_12_VCSCL

2LIS_13_VDHDR2LIS_13_VDHDR2LIS_13_VDITM2LIS_13_VDITM

0SD_C040SD_C04

0SD_C05

0SD_C03

Cubes & Queries

LO Data Extraction: Data Flow into BW

BWR/3

The same R/3 communication structures are used to transfer data directly to BW using new data sources.

Note: The LO data flow bypasses the LIS InfoStructures.

SAP AG BW350 3-7

SAP AG 2002

LO Extraction: IMG (SBIW)

LBWE

LBWF

Delete set up tables: LBWG

!"!"!"!"

OLI*BW

SAP AG BW350 3-8

SAP AG 2002

Update Methods

he new update methods and the "serialized V3 update" method are available in the Logistics Extract Structures Customizing Cockpit (LBWE) of each application:

The Logistics Customizing Cockpit is the central administration tool for extract structures. You use it to transfer Logistics transaction data from the OLTP system into the BW system. The extract structures are filled by the R/3 communication structures of the individual Logistics applications.

If the R/3 communication structures are enhanced by adding new fields, these new fields are also available to the extract structure and the DataSource.

The Cockpit contains the following functions: Maintaining extract structures

Maintaining DataSources Activating updates Controlling updates

SAP AG BW350 3-9

SAP AG 2002

Maintaining Extract Structures

Extract structure maintenance: Field selection Creating new fields

The extract structure is supplied with data by the R/3 communication structures assigned to it. You can use only selected fields from the communication structures. SAP internal control fields, for example, are not available. User-appends to the communication structure are, however, available.

SAP already delivers extract structures that you can enhance by adding appends to the communication structure. Every extract structure can be maintained by you and by SAP.

After you set up the extract structure, the system generates it automatically. Missing fields (their units and characteristics) are completed. The extract structure is created hierarchically according to the R/3 communication structures. Every communication structure triggers the generation of a substructure of the actual extract structure.

SAP AG BW350 3-10

SAP AG 2002

Maintaining DataSources

DataSource Maintenance

At this point, you call up the general maintenance screen for DataSources by selecting the extract structure fields that you want to transfer into the BW system. This selection forms the basis for the process of generating the DataSource.

SAP AG BW350 3-11

SAP AG 2002

Maintaining DataSources: Procedure

MCVBAKStructure MC11VA0ITM

VKORG KUNNR MATNR

MCVBAP

Communication Structures Extract Structure

Selecting Selection Fields

Selection: Selection fields Hide fields Cancellation fields

There is a DataSource (for example, 2LIS_11_VAITM) for each extract structure available in the OLTP system (for example, MC11VA0ITM).

A maintenance screen is displayed, in which you can assign additional properties to the fields of the extract structure.

Selection fields

Hide fields Cancellation fields (field is inverted when canceled (*-1))

Among the fields for the DataSource for data from the item level, there are also some fields from the header level, to ensure that all the important data is available.

The process is very much the same as the LIS process from this point on.

SAP AG BW350 3-12

SAP AG 2002

Replicating DataSources

As far as BW is concerned, DataSources are based on R/3 extract structures. An R/3 system that is connected to a BW system provides a number of DataSources. These are

uploaded into BW by replicating them in the source system tree. DataSources are delivered in the 'D-version'. They are converted into the 'A-version' when they are

activated. They can also be modified and converted into an 'A-version' that is not the same as the default 'A-version'.

If DataSources change in the source system (for example, when your R/3 system is upgraded), you can decide if you want to activate these modified DataSources into 'A-versions' or continue to use the ones you already have.

The DataSource must be replicated in BW before it can be used by the BW system. You replicate DataSources in the Source Systems tabstrip of the BW Administrator Workbench. Select the relevant source system, and choose the Replicate DataSources option from the context menu. All the DataSources that do not yet exist in BW are transferred from the source system to the BW system.

SAP AG BW350 3-13

SAP AG 2002

Set Up Extraction

!#$#!!#$#!!#$#!!#$#!!#$#!!#$#!!#$#!!#$#! % % % %

% % % %

% % % %

% % % %

D-Management------------------------------------------

Delta Extraction

MasterData

MasterData

TransactionData Staging

Engine

SetUpTablesSetUpTables

BWR/3

When you extract transaction data using the LO method, you first need to set up the extraction. This is similar to the LIS method for refreshing statistical data.

The extraction set up reads the dataset that you want to process (for example, customer orders with the tables VBAK, VBAP and so on) and fills the relevant communication structure with data. The data is stored in cluster tables (_SETUP) from where it is read when the initialization is run.

You also use the set up tables to carry out a full update in BW.

SAP AG BW350 3-14

SAP AG 2002

Activating Extract Structures

Activate the extract structures

When you set the extract structure for update to active, data is written to the extract structure, both online and when the set up tables are filled. It is important that during the initialization phase, no-one generates or modifies application data, at least until the tables can be set up. You can minimize the downtime, during which no applications can be loaded, by initializing the delta process selectively.

SAP AG BW350 3-15

SAP AG 2002

Initialization/Simulation (OLI*BW)

Simulation:No update!

LBWF:

1. Selection criteria to limit historical data

2. Date/time to stop the program to allow the R/3 system to better support production

3. Name of run

4. New run flag

Before data can be extracted from the R/3 system, you have to initialize the dataset for the previous X number of years of transactions for each of the extract structures.

The initialization must be set up on the OLTP side. The set up fills the set up tables that are read during the initialization.

SAP recommends that you simulate the process first. This gives you the option of checking the update of the extract structures, without actually updating the data. The results are recorded in a detailed simulation log.

SAP AG BW350 3-16

SAP AG 2002

Extracting Data from SAP R/3 Archives

!# !%&%!# !%&%!# !%&%!# !%&%!# !%&%!# !%&%!# !%&%!# !%&%

InfoSource

Replicated Metadata

!#' (!%!!#' (!%!!#' (!%!!#' (!%!!#' (!%!!#' (!%!!#' (!%!!#' (!%!

Mapping & Mapping & Transfer RulesTransfer Rules

DataSource

DataSource

Delta QueueSet Up Tables

I(nitial), F(ull) D(elta)

DocumentProcessing

Update Technology

ApplicationTables

ArchivedDocuments

RefreshStatistical Data

When statistical data is refreshed in the SAP R/3 source system, the data can be read from the application tables (resident documents) or from archived documents. The following terms are important for the processing of archived documents:

Archive file Here, you type the name of the archive file, from which you want to read the documents. If you do not specify an archive, the resident documents (the documents in the system) are read. Archiving session Archiving sessions have leading numbers that are incremented automatically. The numbers for archiving runs are valid across all objects. Note the following points: When statistical data is refreshed, the application set up tables are filled for DataSources with active

extract structures. Data from these set up tables can be transferred in full update mode, I(nitialization of the delta method), and D(elta update) only.

If data is loaded from the set up table of an application and into BW in full update mode, the delta method is not affected. Make sure, however, that records that have been transferred into SAP BW already are not transferred again (particularly for later updates to an InfoCube).

SAP recommends that before the delta method is initialized, archived documents are loaded, in full update mode, into the data targets in BW. If the DataSource is already working actively on a delta method, and is posting to an ODS object as a data target, a full update is possible only if the Repair Request flag is set in the InfoPackage menu.

Please note that this function is currently only available in some Logistics applications (MM and SD at present). If this function is available, there will be a selection field for archiving sessions in the application-specific set up of the application (SBIW).

SAP AG BW350 3-17

SAP AG 2002

Initializing the Delta Process (Scheduler)

The selected dataset in the OLTP is completely updated

into BW and the delta method is initialized.

Initialize Delta

If you want to transfer data in the delta method, the delta method must be initialized in the first data request. In this process, the selection conditions for connecting delta uploads are set, and the complete dataset that corresponds to these conditions is loaded into the BW system from the R/3 system.

In this case, you must put a check mark in the Delta Method Initialization checkbox for the InfoPackage under the update parameters of the scheduler.

SAP AG BW350 3-18

SAP AG 2002

Delta Extraction with V3 Update

Schematic data flow: Logistics extraction with V3 update

Time

V3 Collect. Run(daily,

hourly, etc.)

Delta-Queue(queued qRFC)

Delta-Queue(queued qRFC)

One LUW,one Commit

Delta Request

BW (PSA, ODS, Cube)BW (PSA, ODS, Cube)

Transfer to BW

Doc.1 V1

Call V3 Module

Doc.TablesDoc.

Tables

Doc.2 V1

Call V3 Module

Doc.N V1

Cal l V3 Module

Read

and p

roce

ss al

l

queu

ed up

date

data

to a

modu

le

Read

and p

roce

ss al

l

queu

ed LU

Ws t

o a

DataS

ourc

e

Doc.TablesDoc.

Tables

Doc.TablesDoc.

Tables

Update TablesUpdate Tables

Collective update technology (V3 update) is used for updating the extraction of transaction data in Logistics applications (MM, PP, SD, LE, and so on).

This means that before it is transferred to the interface with BW (delta queue) the data is collected in the R/3 update tables from where it is picked up by an update process that is scheduled to start periodically.

In this V3 collection run, the data is transferred to the BW delta queue, from where it is picked up by requests from the BW.

For more detailed information on this topic, refer to the SAP Service Marketplace, "Know How Network" Logistics Extraction. You should also refer regularly to the composite notes on Logistics Extraction in the SAP Service Marketplace.

SAP AG BW350 3-19

SAP AG 2002

V3 Update and Serialization

!!!!erialization: Need to transfer the delta information into the SAP BW in the same sequence as it was generated in the SAP source system. This applies to both reading and processing the data. Required for updates in ODS objects.

))))unctions of the serialized V3 update have been developed.****owever, it is still not always possible to serialize document data in the V3 update: With multiple accesses to the document in a single second Time not synchronous for all instances of the SAP system Errors with previous V2 updates of the document Processing of a document in several log-on languages

Since in the Logistics extraction for transaction data, updates into ODS objects are permitted for almost all DataSources, the delta information has to be transferred into BW in the same sequence in which it was generated in the OLTP system. This is the only way to guarantee, for example, that status fields are stored consistently in ODS

objects. The sequence in which the data records were generated must be acknowledged and adhered to when

the update data is read and processed (step A) as well as when the data is transferred to BW (step B). In step B, this is guaranteed by the central delta management functions.

Since the V3 update does not actually know how to serialize update data, several corrections have been made to the SAP Basis to create a "serialized V3 update" function, so that serialization is also possible in step A.

The following problems often occur with V3 updates in the Logistics extraction of transaction data: A serialized V3 update can guarantee the correct sequence for the extraction data in a document only

if the document is not modified several times in one second. A serialized V3 update can guarantee the correct sequence for the extraction data of a document only

if the times are always synchronized exactly in all the instances of a system. This is because the update data is sorted according to the time at which the update records were generated, and this is determined by the local time that is set for the application server.

The serialized V3 update can guarantee the correct sequence for the extraction data of a document only if there were no errors in the V2 update. This is because the V3 update only processes update data for which a V2 update has been carried out successfully.

Regardless of the serialization, update errors in V2 updates have a transaction that means any V3 updates that are still open are never processed, and this can lead to inconsistencies in the BW data.

If, in an R/3 system, documents in an application are modified by several users, who are logged on in different languages, in a process call, the V3 collection run will process the update entries for only one of these languages.

SAP AG BW350 3-20

SAP AG 2002

New Update Methods from PI 2002.1

))))rom PI 2002.1, the following new update methods are available in addition to the serialized V3 update in Logistics extraction: Direct Delta Queued Delta Unserialized V3 Update

he "unserialized V3 update" method does not guarantee the serialization of the document data: Use only in certain scenarios Update in ODS objects not recommended

PI 2002.1 provides the following new update methods for Logistics extraction: Direct delta:

With every document posting, the extraction data is transferred directly into the BW delta queue. Each document posting with delta extraction becomes exactly one LUW in the corresponding BW delta queue.

Queued delta: The extraction data from the application is collected in an extraction queue instead of as update data, and can be transferred to the BW delta queue by an update collection run, as in the V3 update. Depending on the application, up to 10000 delta extractions of documents per DataSource are summarized into one LUW in the BW delta queue.

Unserialized V3 update: A V3 update module writes the extraction data for the application to the update tables as before. The extraction data is stored in the update tables until it is read and processed by an update collection run. The difference between this method and the serialized V3 update method is that the sequence of the data is not taken into consideration when the update collection run reads the data from the update tables and transfers it to the BW delta queue.

SAP AG BW350 3-21

SAP AG 2002

New Update Method: Direct Delta

Schematic data flow: Logistics extraction with direct delta

Delta Request


Transfer to BW

Doc.1 V1

E xtraction Modu le w

ith V-1 Updat e

Doc.TablesDoc.

Tables

Doc.2 V1Extract. Module wi th V-1 Update

Doc.N V1

Extr.Mod.w. V-1 Update

Read a

nd pro

cess al

l

queued

LUWs

to a

DataSo

urce

Doc.TablesDoc.

Tables

Doc.TablesDoc.

Tables

Delta-Queue(queued qRFC)Delta-Queue(queued qRFC)

Advantages and properties of the direct delta: Writing to the delta queue within the V1 update process, means that the enqueue concept in the

applications guarantees the serialization of the data document by document. This method is recommended for customers, who handle small volumes of documents, if some

downtime is acceptable during the set up and delta init request of the initialization process.

This method puts more pressure on V1 than with V3 or the delta queue. For customers with the above volume of documents, this is probably not critical.

Extraction is independent of the V2 update. No additional monitoring of update data or the extraction queue.

SAP AG BW350 3-22

SAP AG 2002

New Update Method: Queued Delta

Schematic data flow: Logistics extraction with queued delta

Time

Extraction Collect.Run(Recommended: Hourly)

Delta Queue(queued qRFC)

Delta Queue(queued qRFC)

One LUW,one Commit

Delta Request


Transfer to BW

Doc.1 V1

Filling the E xtraction Queue

from the V 1- Update

Doc.TablesDoc.

Tables

Doc.2 V1

Filli ng the Extracti on Queue

from the V1-Update

Doc.N V1Fil ling the Extract. Queue

from the V 1-Update

Read

and p

roce

ss al

l

queu

ed up

date

data

to an

appli

catio

n

Read

and p

roce

ss al

l

queu

ed LU

Ws t

o a

DataS

ourc

e

Doc.TablesDoc.

Tables

Doc.TablesDoc.

Tables

Extraction QueueExtraction Queue

Advantages and properties of the queued delta: Writing to the extraction queue within the V1 update process, means that the enqueue concept in

the applications guarantees the serialization of the data document by document. Since the data is collected in the extraction queue and processed regularly (recommended hourly)

this method is particularly suited to customers, who work with larger volumes of documents.

The collection run uses the same reports as before (RMBWV311, and so on). In the initialization process, collecting new document data during the delta init request means that

down time can be reduced to just the set up run (filling the set up tables). More immeasurable pressure on V1 than if V3 is used. Much better performance for collection run than with V3. In particular, no problems with multiple

languages.

In contrast to the V3 collection run, there is a definite end to the collection run, and subsequent processes can be scheduled. At the end of a collection run for an application, an event (&MCEX_11, for example) is triggered automatically. This event (if one is set) can be used to start a subsequent job.

Extraction is independent of the success or failure of the V2 update.

SAP AG BW350 3-23

SAP AG 2002

New Update Methods

he new Logistics Queue Overview (LBWQ) function is available to monitor the extraction queue. The new function can also be accessed from a button in the Cockpit:

SAP AG BW350 3-24

SAP AG 2002

LO Extraction - Step-by-Step

1. Maintain extract structures

2. Maintain DataSource3. Replicate DataSource in BW

4. Assign InfoSources5. Maintain communication structure/transfer rules

6. Maintain InfoCubes and update rules7. Activate extract structures

8. Delete set up tables/set up extraction9. InfoPackage for the delta initialization

10. Set up periodic V3 update11. InfoPackage for the delta upload

Note: In situations similar to the LIS extraction method, where historical transactions are set up in current or active time frames, applications should not be modified and new orders should not be created (see point 8 above).

SAP AG BW350 3-25

SAP AG 2002

Performance &Volume

SimpleHandling

StandardizedLogistics Solution

No LISFunctionality

New LO Data Extraction: Why?

The new LO extractor has the following advantages when compared with the process of extracting data using transfer structures and information structures:

Improved performance and reduced volumes of data Simple handling Standardized solution for all Logistics applications

No use of LIS functions

SAP AG BW350 3-26

SAP AG 2002

Advantages I: Performance and Data Volume

+,-%+,-%+,-%+,-%

You can deactivate the extraction of scheduling data, for example, ('slim extractors' mean smaller volumes of data to upload)

+ + + +

Only changes to data that are relevant to BW* are updated (smaller volumes of data to upload)

!%,,!%,,!%,,!%,,

Reduces the volume of data to upload by removing redundant data

",%",%",%",%

No pressure on the day-to-day running of business

*: BW-relevant data changes are changes to the contents of the fields that also exist in a relevant extract structure. Not every document change therefore results automatically in a new/old posting for BW.

SAP AG BW350 3-27

SAP AG 2002

Advantages II: Simple Handling

No LIS functions: No knowledge of LIS customizing, update settings, and

so on, is required) ) ) )

Simple to create without modificationsBW Customizing Cockpit: Central, standardized, maintenance tool for logistics

applications

SAP AG BW350 3-28

SAP AG 2002

./% !!%%./% !!%%./% !!%%./% !!%%

Technical restrictions: See notes from Online Service System

+.! 0 +.! 0 +.! 0 +.! 0 11112/2/2/2/:

Double updates, double data-saves

Advantages III: Others

Refer to the Online Service System for notes on the technical limitations. Due to technical restrictions, it is not possible to update S261/S263 in the V2 update if picking

takes place in a LUW (Logical Unit of Work) when the delivery is saved. You must not therefore use dispatch time 4 for picking-messages in the delivery, when the info

structures S261/S263 are actively updated.

SAP AG BW350 3-29

SAP AG 2002

LO Extraction: Conversion Scenario

InfoCubes

Transfer Rules

Update Rules

! ! ! !

Communication Structure

Transfer structure

Transfer structure

OLTP1: LIS InfoStructureOLTP1: LIS InfoStructure

' ($#' ($#' ($#' ($#

Transfer structure

S260S260 S260BIW1S260BIW1 S260BIW2S260BIW2

Extraction source structure ZZB

Transfer structure

S263S263 S263BIW1S263BIW1 S263BIW2S263BIW2

OLTP2: LIS InfoStructureOLTP2: LIS InfoStructure

Extraction source structure ZZB

LO data extraction can completely replace extraction using transfer structures (for example, S260 - S264 in SD) and extraction using 'traditional' information structures in the LIS (for example, S001 - S006 in standard LIS) without losing out on any information.

This means that LO extraction enables you to load more information with more detail into the SAP BW system .

SAP AG BW350 3-30

SAP AG 2002

Example: Conversion in Sales

Sales order

Request

Order/delivery

Delivery

Billing Doc.

MC11VA0HDRMC11VA0HDR

MC11VA0ITMMC11VA0ITM

MC11VA0SCLMC11VA0SCL

MC11V_0ITMMC11V_0ITM

MC11V_0SCLMC11V_0SCL

MC12VC0HDRMC12VC0HDR

MC12VC0ITMMC12VC0ITM

MC12VC0SCLMC12VC0SCL

MC13VD0HDRMC13VD0HDR

MC13VD0ITMMC13VD0ITM

VA: HeaderVA: ItemVA: Unit

Item levelUnit level

VC: HeaderVC: ItemVA: Unit

VD: HeaderVD: Item

newold

S264

S263

S261

S260

S262

With the new LO extractor, there is a communication structure available for every document level (for example, document header/document item/document unit). These communication structures form the basis for each of the corresponding extract structures.

See the SAP Homepage for more information. The paper on extracting MM transaction data is particularly recommended.

SAP AG BW350 3-31

SAP AG 2002

Step-by-Step Conversion: LIS Extraction to LO Extraction

1. Maintain extract structure

2. Maintain DataSource3. Replicate DataSource in BW

4. Assign InfoSources5. Maintain communication structure/transfer rules

6. Maintain InfoCubes and update rules7. Extract structure active and + LIS update off

8. Upload the open delta queue (from LIS)9. Empty set up tables (security measure)

10. Set up/start periodic V3 update11. Simulate delta initialization

12. InfoPackage for the delta upload

You must follow the sequence of steps described below to switch from the InfoSources that you want to replace to the new DataSources/InfoSources.

Steps 1-6: Transfer new Content objects from the Content (DataSources in R/3 and InfoSources, update

rules, transfer rules, and so on in BW.) Find the time frames in which there are no updates for the InfoStructures that you want to replace. Steps 7-10:

IMPORTANT: Make sure that no R/3 data is being updated for any of the DataSources when you carry out these steps.

Switch off the update for LIS extraction. Switch on the update for LO extraction (TA SBIW). Load the delta queue of the old InfoSource into BW so that all the delta records for the old source are posted to BW (this is important, because it makes sure that no changes or new records are lost and that the data remains consistent). Do not refresh the statistics for the new DataSources, or records will be posted twice in the InfoCube.

Steps 11-12: Simulate an initial load for the delta upload of the new InfoSources (option in the InfoPackage). This simulation run must get a 'green' light. The update can now take place. When the V3 update has been scheduled for the new DataSources, the delta queue is filled and the data can be loaded for the new DataSources.

SAP AG BW350 3-32

SAP AG 2002

LIS Data Flow

Standardanalysis

Standardanalysis

LIS interfaceLIS interface

ApplicationApplication

Updaterules

Updaterules

Sales

Informationstructure

Informationstructure

Sales order

VBAPVBAK

MCVBAPMCVBAK

S999/XYZS001/1QtyValue

...

...

NumberValue

...

...

analysis program analysis program

VBAKVBAKVBAPVBAP

S...S... S001S001 S5nnS5nn

SAP AG 2002

The LIS data flow can be described as follows: Operative transaction data (for example, sales orders, billing documents) is stored in application

data tables (for example, VBAK/VBAP for sales orders). The information is transferred from an operative document into communication structures (for

example, MCVBAK, MCVBAP) to update the data in the LIS. These R/3 communication structures perform the same functions as the BW communication structures, but they fill LIS InfoStructures in R/3 with data and not cubes in BW.

The update rules write the information from the communication structures to the corresponding information structures.

Although the term "InfoStructure" is used in R/3, the InfoStructure is a real transparent table and not a structure that contains data only at runtime.

LIS has many reporting tools (ABC analysis, correlation, graphical displays, and so on) but the limitations of a schema with only one table makes reporting slower than it would be in BW.

In the LI