how to archive data from sap net weaver bw to sap sybase iq as near line storage

SAP How-to Guide

Database & Technology – SAP NetWeaver Business Warehouse

SAP HANA™ Appliance

Applicable Releases:

SAP NetWeaver BW 7.3x SP09

SAP HANA 1.0 SPS 06

Version 1.0

September 2013

How to Archive Data from SAP NetWeaver BW to SAP Sybase IQ as Near line Storage

© Copyright 2014 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. by Netscape. SAP, R/3, SAP SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.

Business Objects and the Business Objects logo, BusinessObjects,

Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other

Business Objects products and services mentioned herein as well as their

respective logos are trademarks or registered trademarks of Business

Objects Software Ltd. Business Objects is an SAP company. Sybase and Adaptive Server, 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase, Inc.

All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of th ird-party Web pages nor provide any warranty whatsoever relating to third-party Web pages.

SAP “How-to” Guides are intended to simplify the product

implementation. While specific product features and procedures typically

are explained in a practical business context, it is not implied that those

features and procedures are the only approach in solving a specific

business problem using SAP NetWeaver. Should you wish to receive

additional information, clarification or support, please refer to SAP

Consulting.

Any software coding and/or code lines / strings (“Code”) included in this

documentation are only examples and are not intended to be used in a

productive system environment. The Code is only intended better explain

and visualize the syntax and phrasing rules of certain coding. SAP does

not warrant the correctness and completeness of the Code given herein,

and SAP shall not be liable for errors or damages caused by the usage of

the Code, except if such damages were caused by SAP intentionally or

grossly negligent.

Disclaimer

Some components of this product are based on Java™. Any code change

in these components may cause unpredictable and severe malfunctions

and is therefore expressively prohibited, as is any decompilation of these

components.

Any Java™ Source Code delivered with this product is only to be used by

SAP’s Support Services and may not be modified or altered in any way.

Document History

Document Version Description

1.00 First official release of this guide

Typographic Conventions

Type Style Description

Example Text Words or characters quoted

from the screen. These

include field names, screen

titles, pushbuttons labels,

menu names, menu paths,

and menu options.

Cross-references to other

documentation

Example text Emphasized words or

phrases in body text, graphic

titles, and table titles

Example text File and directory names and

their paths, messages,

names of variables and

parameters, source text, and

names of installation,

upgrade and database tools.

Example text User entry texts. These are

words or characters that you

enter in the system exactly

as they appear in the

documentation.

<Example

text>

Variable user entry. Angle

brackets indicate that you

replace these words and

characters with appropriate

entries to make entries in the

system.

EXAMPLE TEXT Keys on the keyboard, for

example, F2 or ENTER.

Icons

Icon Description

Caution

Note or Important

Example

Recommendation or Tip

Table of Contents

1. Introduction ........................................................................................................................... 1

2. Step-by-Step Procedure ...................................................................................................... 2

2.1 Configuring Sybase IQ as a Near-Line Storage Solution ............................................. 2

2.1.1 The configuration steps in SAP NetWeaver BW ............................................. 2

2.1.2 Configuration Steps in Sybase IQ (optional) ................................................... 8

2.2 Create Data Archiving Process in BW .......................................................................... 9

2.3 Read Archived Data ..................................................................................................... 19

2.3.1 BEx Query on Archived Data .......................................................................... 19

2.3.2 Reading NLS data in BW Transformations .................................................... 26

2.3.3 NLS data access for InfoProviders as sources within APD .......................... 27

3. InfoProvider Type Specific Features .................................................................................. 28

3.1 Archiving of non-compressed data for InfoCubes ..................................................... 28

3.2 Archiving of Write-Optimized DSOs ........................................................................... 29

4. Write into NLS with Process Chain .................................................................................... 31

4.1 Process Chain with Archiving Steps ........................................................................... 31

4.1.1 Write Phase ..................................................................................................... 32

4.1.2 Verification Phase ........................................................................................... 32

4.1.3 Deletion Phase ................................................................................................. 33

How to Archive Data from SAP BW to Sybase IQ as Near Line Storage

April 2014 1

1. Introduction

This paper’s focus is the procedure of setting up and executing the archiving process with the

Sybase IQ NLS from the BW system.

For positioning, prerequisites and other relevant information please visit: new SCN page (Roland’s

current page: http://scn.sap.com/docs/DOC-39944).

The aim of this paper is to illustrate with screenshots the setup of the NLS-IQ solution for BW as

described in the online documentation Sybase IQ as a Near-Line Storage Solution

Please also always refer to the latest version of the documentation when executing the steps

described in this paper.

http://scn.sap.com/docs/DOC-39944

http://help.sap.com/saphelp_nw73/helpdata/en/60/cf63e1bbbb49429ee6c35e6ad03a45/frameset.htm


April 2014 2

2. Step-by-Step Procedure

First, you will need to Create Database Connection and Configure Sybase IQ. Next you will use the

SAP BW system to create the archiving process to Archive SAP BW Data to Sybase IQ. Finally you

will check the archived request and test query access to data in the Sybase IQ for older data.

For detailed information on the Sybase IQ installation please refer the SAP First Guidance - SAP-

NLS Solution with Sybase IQ

2.1 Configuring Sybase IQ as a Near-Line Storage Solution

2.1.1 The configuration steps in SAP NetWeaver BW For details refer to the SAP Online documentation: Configuring Sybase IQ as a Near-Line Storage

Solution ...

Create Database Connection

First create DB Connect - call transaction DBCO and create a database connection to Sybase IQ.

For more details, refer to SAP Note 1737415.

CAUTION

The DBCO user and the Sybase IQ user both have to be created with uppercase letters.

Parameter SYBASE_IQ_CESU-8 under Connection Information must be set to 0 if you use UTF8 as the character set.

Connection Info that works is:

SYBASE_SERVER=MELN50791806A.dhcp.wdf.sap.corp SYBASE_PORT=2638

SYBASE_IQ_ENGINE=MELN50791806A_iqdemo SYBASE_DBNAME=BDFNLS

SYBASE_CONTYPE=IQ SYBASE_IQ_LOAD=1 SYBASE_IQ_CESU-8=1 SYBASE_SQL_TRACE=1

SYBASE_IQ_LOCKWAIT=600 SYBASE_IQ_BUFFER_SIZE=500000

https://scn.sap.com/docs/DOC-39627

https://scn.sap.com/docs/DOC-39627

http://help.sap.com/saphelp_nw73/helpdata/en/e8/395401e46f4edca50aefeead7f3a44/frameset.htm



April 2014 3

Note

Ensure that Sybase IQ is up and running. (Refer to Appendix A).

The default password for user DBA delivered is ‘sql’. This needs to be changed for security reasons.

Create Near-Line Storage Connection

1. Call transaction SPRO SAP NetWeaver Business Warehouse General Settings

Process Near-Line Storage Connection

2. Choose New Entries

3. Enter a name for the near-line storage connection. This name is then offered to you for

selection when you create a data archiving process.


April 2014 4

4. As the name, enter class CL_RSDA_SYB_CONNECTION.

5. Leave the Destination field empty

6. Enter a connection string under Connection Parameters. The connection string generally

comprises a list of name/value pairs, separated by a semi-colon.

Note

Each name/value pair must be entered as follows:

<Name of the parameter>=<Value of the parameter>

The following parameters are supported:

DBCON (required entry): The value of the parameter should be the name of the

database connection that you created in the first step.

BACKUP_REQUEST_EVENT (optional):

The value should be either DUMMY or the name of an event that has been defined in the Sybase IQ database and which triggers a new data backup. If this parameter is entered, and all data has been copied to Sybase IQ, the end of the write phase triggers this event. This makes it possible to perform the data backup in Sybase IQ parallel to the verification phase. Before the actual deletion process begins, the system checks for a backup in Sybase IQ. The deletion phase cannot continue until this has happened.


April 2014 5

A special event called DUMMY triggers an event in the Sybase IQ database. It checks for a more recent data backup however before the data in the request is deleted from the BW database. The DUMMY event should therefore be used if database backups are detected before the data is deleted and the data backups are automatically created by the Sybase IQ database.

If the parameter is left out or does not contain any values, no event is triggered, and the system does not check for data backups. You should only choose this setting in non-productive systems.

BACKUP_WAIT_SEC (optional)

If the BACKUP_REQUEST_EVENT has been specified, and no data backup is detected before the start of the deletion phase, the process waits for a period specified here in seconds. If no data backup is detected during this period, the archiving request terminates with error message RSDA_SYB197: "No new data backup since the end of the copy phase. The current data backup ID is still <backup ID> of <timestamp>. The parameters <backup ID> and <timestamp> refer to the columns bu_id and bu_time in the sybase IQ system view SYSIQBACKUPHISTORY. All rows with the condition selective_type in (1,2) are taken into account. This excludes all data backups that only refer to read-only files. The archiving request only be restarted once the data backup has been created.

PACKET_SIZE_MB (optional)

The package size of a single LOAD statement in MB. If this parameter is left out or does not contain any values, the package size of the LOAD statement is exactly the same as the package size of the FETCH of the online table. This is currently set to 20 MB. A package size greater than 2000 is ignored and replaced by 2000 MB.

7. Save your settings.

Tip

Use program ADBC_TEST_CONNECTION to test the DB connection to Sybase IQ configured above:

Confirm the ODBC connection settings. In our example the ODBC connection has been defined as follows:


April 2014 6


April 2014 7


April 2014 8

2.1.2 Configuration Steps in Sybase IQ (optional) For details refer to the SAP Online documentation: Configuring Sybase IQ as a Near-Line Storage

Solution

These configuration steps in Sybase IQ are optional in order to integrate an automated data backup

mechanism. ...

1. Log on to Sybase IQ with user DBA via Sybase Central or IQISQL.

2. Create an event that triggers a new data backup. As the name of the event, enter the name

that you entered in step 6 for BACKUP_REQUEST_EVENT. The way the event is handled

depends on your data backup and data recovery strategy.

Example

The following event definition shows a very simple strategy. Here, a data backup is always started whenever the event is triggered. The data backup files are stored in a directory in the database's Home directory. The name of the database forms part of the path name and of the file name:

CREATE EVENT BACKUP_REQUEST

HANDLER

BEGIN

-- Declare and set unique filename

DECLARE filename varchar(255);

SET filename = '../backup/' || db_name() || '/' || db_name() ||

'_' || dateformat(now(),'yyyymmdd_hhnnss.ssssss');

-- Perform incremental backup

BACKUP DATABASE FULL TO filename;

END




April 2014 9

2.2 Create Data Archiving Process in BW For details refer to the SAP Online documentation: Creating Data Archiving Processes

Step-by-step procedure: ...

1. From the context menu of the InfoProvider (e.g. DataStore Object), choose Create Data

Archiving Process

2. Deselect the ADK-Based Archiving checkbox, enter the Near-Line Connection BDFNLS and

Check this connection.

http://help.sap.com/saphelp_nw73/helpdata/en/4c/75184b167821d1e10000000a42189c/content.htm?frameset=/en/60/cf63e1bbbb49429ee6c35e6ad03a45/frameset.htm


April 2014 10

3. Check the status information for the selected Near-Line Connection and choose Continue

(Enter).

4. Enter selection profile if applicable

Note

The fields that are available for selection are only the InfoProvider key fields

(highlighted in red).

If there is no time characteristic then you may not archive the data in the DSO.

Where there is no time characteristic in the key of the DSO then the

“characteristic for time slice” refers to a time characteristic in the data field of

the DSO (see pictures below). The primary partitioning characteristic refers to

the selection defined in the Data Archiving Process. The way archiving works is

that the system will select all primary partitioning characteristics that are older

than the date of the “characteristic for time slice.


April 2014 11

The DSO key doesn’t contain any time characteristic.

The “characteristic for time slice” refers to a time characteristic in the data field of the DSO.


April 2014 12

5. Select semantic group if applicable

6. Accept the default settings on the Near-Line storage tab. Check and Activate your Data

Archiving Process.


April 2014 13

7. Check the Activation Log for the Data Archiving Process and leave the log by choosing

Continue (Enter).

8. Create Archiving Request. From the context menu of the InfoProvider choose Manage then

go to the Archiving tab and choose Create Archiving Request

Note

An icon appears next to the InfoProvider indicating that a Data Archiving Process was created successfully.


April 2014 14

For additional details refer to Creating and Executing Archiving Requests

9. Simulate Execution of Data Archiving Request in Dialog

Note

The period referred to in “Only Data Records Older Than” is the number of days/weeks/years relative to today (this is dynamic so can easily be used in process chains).

The absolute value is a static fixed selection period

http://help.sap.com/saphelp_nw73/helpdata/en/4c/7e3b4db7756bb4e10000000a42189e/content.htm


April 2014 15

The selection options provided in the restriction is based on the definition of the Data Archiving Process Selection. Refer to step 4 above.

10. You can simulate the Data Archiving Process from here – in the background (user

BWREMOTE is used) or in the foreground (user logged into BW)

Simulate the execution of the Data Archiving Request by choosing Only Simulate Request Processing in Dialog

11. Ignore the warning No restrictions yet defined by choosing Continue (Enter)

Confirm “Do you want to continue processing in the dialog?” with Yes


April 2014 16

Check the displayed log for errors and choose Continue (Enter).

12. Create and execute the Data Archiving Request by choosing Create Request and Execute in

Background


April 2014 17

Note

The Data Archiving Process consists of 5 steps. This is reflected in the screen

dump above. You can run this in single steps or you can do this in one step

meaning you can simulate step 10 of the process, then step 20. This will assist

in error detection.

Setting the “Continue Processing until Target Status” to 70 will execute all

steps in one go.

It is recommended to use 70 for the manual approach. Alternatively split this

into step 40, 50 and 70 within a process chain. The reason for the split within

the process chain is to better assist with error handling

13. View the results of scheduled Job BI_ARCH-NLSDS01-CREATE


April 2014 18

Check the Job’s Status. You should see a successfully finished archiving job (BI_ARCH-NLSDS01-CREATE). Leave the job overview by choosing Back

14. Check the Archiving Request

Click Refresh to refresh the Data Archiving Request list.

Your Data Archiving Request should now appear in the list.


April 2014 19

Leave the DataStore Object administration and the Data Warehousing Workbench by choosing Back twice.

2.3 Read Archived Data

The archived data is accessible via BEx Queries, Transformations for DSO lookup and Analysis

Process.

2.3.1 BEx Query on Archived Data

With BW on HANA 7.30, you can now set up queries on MultiProvider and read the NLS partitions of

the PartProviders. Configuration for NLS use is possible on four different levels: ...

1. In the properties of the InfoProvider

2. In the properties of the PartProvider (from MultiProvider)

3. In the properties of the BEx Query Designer

4. Definition of a near-line storage variable for the query selection screen

The settings for NLS use on the four different levels are described below.

For additional details refer to the SAP Online documentation: Read Stored Data

1. Properties of the InfoProvider:

In order to access archived data at InfoProvider level you have to change/adjust the InfoProvider settings appropriately.

For all InfoProvider object basically two NLS settings are possible:

http://help.sap.com/saphelp_nw73/helpdata/en/4c/2c60d7477f51e6e10000000a42189b/content.htm?frameset=/en/45/2502de55c56d7be10000000a114a6b/frameset.htm


April 2014 20

Near-Line Access Switched Off (Default)

‘X’ Near-Line Access Switched On

For MultiProviders, an additional option is available:

‘P’ Near-Line Access Set the Same as for PartProvider

Depending on the type of the InfoProvider different settings are available.

2. Properties of the PartProvider:

NLS usage can also be delegated from the query level to the PartProvider level by using the following settings:

BEx Query Designer: ‘Use Near-Line Storage According to Provider Settings’

MultiProvider: ‘Near-Line Access Set the Same as for PartProvider’

PartProvider: ‘Near-Line Access Switched On/Off

This configuration allows you to switch specific NLS PartProvider partitions on and off.


April 2014 21

3. Properties of the BEx Query Designer

The query definition is defined with NLS on the extended tab.

Options available for Near-Line Storage include:

Use Near-Line Storage According to Provider Setting’ mode

Do Not Read Near-Line Storage

Read Near-Line Storage


April 2014 22

4. Definition of a near-line storage variable for the query selection screen:

In the BEx Query Designer, you can also define an ‘NLS variable’ to let a specific query run either in

‘Read Near-Line Storage’ mode or

‘Do Not Read Near-Line Storage’ mode or

‘Use Near-Line Storage According to Provider Setting’ mode


April 2014 23

Start the Query Monitor

You can start the Query Monitor from the SAP Easy Access Menu (SAP menu -> Business Explorer -> BEx Monitor -> Query Monitor (= transaction RSRT1).

The “NLS read access” is the name of the selection variable. Now you can decide during query execution if you want to report on archived data or not.


April 2014 24

This example illustrates results of the query using variable NLS set and without NLS access.

Example:

Read Near-Line Storage:

Do Not Read Near-Line Storage

The results of the query monitor are as follows:

Note

In the Statistics Data for the Query Runtime (transaction RSRT), you can see how many PartProviders, including aggregates and near-line partitions (suffix $N), effectively contribute to the query result.

The following is another example showing how data is queried on from NLS and live data

The $X reflects the active data. The $N reflects Near-line Storage (Sybase IQ).


April 2014 25

Values show 96 records which were archived and 48 remained in the DSO in for this

selection

Note

Two tables are generated in Sybase IQ for the InfoProvider. Once contains the actual data. The other contains the MetaData. You can view data in Sybase IQ to validate that the data is indeed correct

The screen shot display data archived using the Data Arching Process

The following screen shot displays the Data Arching Process Metadata.


April 2014 26

This above Sybase IQ table reflects the BW DAP below.

2.3.2 Reading NLS data in BW Transformations

For additional details refer to the SAP Online documentation: Reading Stored Data

Please refer to SAP notes 1028450 and 985609 for additional information.

http://help.sap.com/saphelp_nw73/helpdata/en/db/d20fc1f11e424696ac61c37583d05b/content.htm?frameset=/en/45/2502de55c56d7be10000000a114a6b/frameset.htm

https://service.sap.com/sap/support/notes/1028450

https://service.sap.com/sap/support/notes/985609


April 2014 27

2.3.3 NLS data access for InfoProviders as sources within APD As of SAP NetWeaver BW 7.30, the Analysis Process Designer is enabled to read data transparently

from the online partitions and near-line partitions. This makes it easy to include historically

archived data in your analysis process.

In the example shown in the following picture an APD process (TC: RSANWB) data is to be

extracted from Cube which is partly archived.

Select the source cube with the cursor and click the right mouse button. In the context menu you

can select the Properties….. Now you can select the option ‘Read Data from Near-Line Storage’.


April 2014 28

3. InfoProvider Type Specific Features

For additional details refer to the SAP Online documentation: Creating Data Archiving Processes

3.1 Archiving of non-compressed data for InfoCubes The option to archive non-compressed data was requested by customers for whom data

compression was a time consuming operation and did not really lead to significant compression

rates (highly granular data).

Data compression was a precondition for archiving InfoCube data since release BW 7.00. Before

data can be compressed, it must be:

distributed to subsequent data marts

rolled up to existing aggregates

As of SAP NetWeaver BW 7.30, it is no longer necessary to compress data before archiving. To

ensure data consistency in different data layers, the two preconditions above must be met before

data can be archived.

The flag ’Allow Archiving of Non-Compressed Data’ is a central setting in the data archiving

process.

Note

The flag can be changed in the Data Archiving Process (DAP) even if data has already been archived. If you change this flag (Allow Archiving of Non-Compressed Data) in the DAP definition, all archiving requests that are not completely finished (Status ‘70 Deletion Phase Confirmed and Request Completed’) must be finished in the old ‘Compression Mode’ or must be set to invalid before you can archive data in the new ‘Compression Mode’

http://help.sap.com/saphelp_nw73/helpdata/en/4c/75184b167821d1e10000000a42189c/content.htm?frameset=/en/45/2502de55c56d7be10000000a114a6b/frameset.htm


April 2014 29

3.2 Archiving of Write-Optimized DSOs

For additional details refer to the SAP Online documentation: Special Features of Write-Optimized

DataStore Objects

The NLS solution allows for archiving of data from Write Optimized DSOs. The main difference

between settings for DAP for standard DSOS and write optimized DSOs is: data from write

optimized DSOs are archived request-based.

On the archiving request (context menu of an InfoProvider -> Manage-> Tab: Archiving) you can

still create a time related restriction:

http://help.sap.com/saphelp_nw73/helpdata/en/48/1f0cec04184269e10000000a421937/content.htm?frameset=/en/45/2502de55c56d7be10000000a114a6b/frameset.htm

http://help.sap.com/saphelp_nw73/helpdata/en/48/1f0cec04184269e10000000a421937/content.htm?frameset=/en/45/2502de55c56d7be10000000a114a6b/frameset.htm


April 2014 30

After starting the archiving process an additional request related restriction is generated:

As a result all request which contain data older than Time-restriction are archived in full. You can

check it in the Requests tab,


April 2014 31

4. Write into NLS with Process Chain

For additional details refer to the SAP Online documentation: Scheduling Data Archiving Processes

Using Process Chains

Prerequisite is an “Active, executable” Data Archiving Process for your desired Info Provider.

4.1 Process Chain with Archiving Steps

The sequence of process chain using NLS is as follows (see screen): ...

1. Write Phase

2. Verification Phase

3. Deletion Phase

It is recommended to at least keep the delete process separately.

http://help.sap.com/saphelp_nw73/helpdata/en/4d/9c4b25416d0b83e10000000a42189b/content.htm?frameset=/en/66/c9f936d1a84890a954e14371abcada/frameset.htm

http://help.sap.com/saphelp_nw73/helpdata/en/4d/9c4b25416d0b83e10000000a42189b/content.htm?frameset=/en/66/c9f936d1a84890a954e14371abcada/frameset.htm


April 2014 32

4.1.1 Write Phase

Apply the following settings:

Continue Processing Until Target Status: “40 Write Phase Completed Successfully”

Auom- Request Invalidation After Error: “None”

4.1.2 Verification Phase

You need the following settings:

Continue Open Archiving Request(s) : Check

Continue Processing Until Target Status: “50 Verification Phase Ended Successfully”

Auom- Request Invalidation After Error: “None”


April 2014 33

Note

The Continue Open Archiving Request(s) Check box refer to whether there are

any preceding steps that have been executed

If the “All Open Archiving Requests” check box is selected then all open

requests for this InfoProvider will be processed – including manually managed

DAP’s as well as those that have not been processed by other processed chains

4.1.3 Deletion Phase

You need following settings:

Continue Open Archiving Request(s) : Check

Continue Processing Until Target Status: “70 Deletion Phase Confirmed and Request

Completed”

Auom- Request Invalidation after Error: “None”

Important

For InfoCubes with compressed data please note special archiving setting for the non-compressed data (Allow Archiving of Non-Compressed Data ). It the flag is not turned on you have to compress your data in the cube before starting the archiving process.

E.g. include the compression step in the process chain prior the archiving steps.

www.sap.com/contactsap

www.sdn.sap.com/irj/sdn/howtoguides

how to archive data from sap net weaver bw to sap sybase iq as near line storage

Education