goldengate for nonstop enscribe to sql - · pdf filegoldengate for nonstop enscribe to sql...

GoldenGate for NonStop Enscribe to SQL Server


Objective

Upon completion of this lesson, you will be able to keep two heterogeneous databases

synchronized, in this case NonStop audited Enscribe to Microsoft SQL Server.

During this lesson, you will learn how to:

Prepare your environment to configure the GoldenGate processes

Configure and execute the initial data load

Configure and start the change capture of database operations

Configure and start the change delivery of database operations

NonStop Audited to MS SQL Server configuration

The following diagram illustrates the configuration for replicating a NonStop audited

database to an MS SQL Server target.

NonStop <source>

TMF Audit

TrailAudserv

Extract

Source

databaseManager

Windows server <target>

MS SQL Server

<dsn>

Target

<schema>Replicat

Collector

Trails

Manager

Network


Overview of Tasks

Prepare the Environment

In order to execute this lesson, the GoldenGate application must be installed on both the

source and target systems. The installation includes a sample database and scripts to generate

initial data as well as subsequent update operations. The source and target tables are created

and loaded with initial data. The GoldenGate Manager processes are also started so that other

processes may be configured and started.

Initial Data Load

To initially load data between heterogeneous databases, use the GoldenGate application to

perform your initial data synchronization while the application remains active. This lesson

demonstrates using Extract to pull data from the source files and sending the data directly to

Replicat on the target system.

Configure Change Capture

For audited tables, the Extract process is configured to capture change data directly from the

TMF audit trails and store the changes in a data queue known as GoldenGate remote trails.

Configure Change Delivery

Once the tables have been initially loaded with data and the Extract process is capturing all

operations, the Replicat process is configured to deliver the captured operations to the target

database.

<target> server

Initial

Replicat

MS SQL Server

<dsn>

Target

<schema>

<source> NonStop

Initial

ExtractSource

database

Network


Exercise 1.


Objective

The goals of this exercise are to:

Configure and start the Manager processes.

Prepare the source and target sample databases.

Prepare the source NSK environment

1. Configure a GLOBALS parameter file on the source

Execute the following commands on the <source> system.

A GLOBALS parameter can be used to uniquely identify each GoldenGate instance when

multiple environments exist on single NSK node. Use the two-character <prefix> that was

defined in the Lab Preparation.

Execute the following commands on the <source> system unless a <prefix> is not needed for

your installation.

TACL> VOLUME <install vol>.<install subvol>

TACL> TEDIT GLOBALS

ADD DEFINE =GGS_PREFIX, CLASS MAP, FILE $<prefix>

2. Configure the Manager process on the source system


The Manager process is responsible for starting and monitoring all of the other GoldenGate

processes. You will always have one manager process per GoldenGate instance.


TACL> RUN GGSCI

GGSCI> TEDIT PARAMS MGRPARM

-- GoldenGate Manager Parameter file

TCPIPPROCESSNAME <tcpip process>

PORT <port>

GGSCI> START MANAGER, CPU 1, PRI 180

Verify the results:


GGSCI> INFO MANAGER

3. Create and load source files


The installation includes scripts to create sample files and generate initial data. Note that your <prefix> may need to be substituted for the GG in subvolume GGSSOU used in this section.

Move the script to the source data volume.

TACL> VOLUME <install vol>.GGSSOU

TACL> FUP DUP <install subvol>.DEMOLDEO, GGSSOU.*

Run the scripts.

TACL> FUP /IN <install vol>.<install subvol>.DEMOFUPS/

TACL> RUN <install vol>.<install subvol>.DEMOLDEO

Within DEMOLDEO the following prompts and replies are issued:

To perform an initial load enter (I), to update

the database enter (U) or to exit the program enter (E).

?I



?E

Verify the results:

TACL> FUP COPY ECUSTMER

TACL> FUP COPY ECUSTORD

4. Generate source definitions


The DEFGEN utility is an interactive program that generates a source definition file describing

the layouts of the files/tables that are to be replicated.


TACL> RUN DEFGEN

Within DEFGEN, the following prompts and replies are issued:

Enter definitions filename (or Exit): <install vol>.GGSDEF.ECUSTDEF

File/Table to create definition for (or Exit): <install vol>.GGSSOU.ECUSTMER

Definition retrieved.

File/Table to create definition for (or Exit): <install vol>.GGSSOU.ECUSTORD

Definition retrieved.

File/Table to create definition for (or Exit): EXIT


5. Move the source definitions to the target

After exiting, FTP the file <install-vol>.GGSDEF.ECUSTDEF in ASCII format to the target

<install location>/dirdef/source.def.

Prepare the SQL Server 2005 target system

Install Manager

1. Configure Manager process on the target

Execute the following commands on the <target> system.

Note! The port numbers must be unique if you are installing GoldenGate more than once on

the same server.

Shell> cd <install location>

Shell> ggsci

GGSCI> EDIT PARAMS MGR

In the parameter file, enter the following parameter, then save and close the file.

-- GoldenGate Manager Parameter file

PORT <port>

Start Manager.

GGSCI> START MANAGER

Verify the results:

GGSCI> INFO MANAGER

Steps to create the database

2. Create the database

From Start>Programs, run SQL Server Management Studio.

Enter the name of the local SQL Server instance, then click connect.

Note: These instructions assume SQL Server Authentication which requires the

more complex, double layer, of authentication. For only Windows Authentication,

you can leave out most of the login and password entries.


This displays the main SQL Server Management Studio dialog box. Right, right-click

Database and select the New Database option.

The New Database dialog box appears. In the Name box, type the <database> from

the Lab Preparation worksheet.


Click OK to add the database.

The main SQL Server Management Studio dialog box reappears.

3. Create SQL Server logins and users

Right-click Security and select the New > Login.

The New Login dialog box appears.


For Name, type the <login>.

Select Authentication,

If you selected SQL Server Authentication, type the <password> in the two

Password fields.

Under Defaults, select your <database> for Database, and leave Language set to the

default.

Do not click OK. Instead, from Select a page in the upper left-hand box, double click User Mapping.


In the upper Users mapped to this login area, activate the checkmark next to your

database.

In the lower Database role membership for <database> area,, check the box for the

db_owner role.

Click OK to exit.

The next step creates a schema that you will associate with the new user.

While you have the database selected, click on the New Query button in the main

menu.

The right side of the dialog displays the SQL Query interface. Type in:

create schema <owner/schema>;

And press the Execute button from the toolbar.


Now you need to associate the new user and the schema that you have created with the new

database.

Right click on the Security option that is under the database and select New > User.

The New Database User dialog box appears. Enter the name of your user and press the browse button labeled with three dots (…).


The Select Login dialog displays. Press the Browse button.

Select your new user <login> from the list. Then press OK.

Your selected user now displays in the object names box. Press OK to exit from

Select Login.

Back in the New Database User dialog, enter the <schema> you added as the default

schema and press OK.

4. Create the ODBC system data source name

The GoldenGate Extract process connects to a SQL Server database through an ODBC (Open

Database Connectivity) connection. Both the Extract and Replicat components of GoldenGate

require that a system data source name (DSN) be established, which stores the information

about how to connect to the SQL Server.

Click Start > Settings > Control Panel.

Double-click Administrative Tools.

Double-click Data Sources (ODBC) to open the ODBC Data Source Administrator

dialog box.

Click the System DSN tab, and then click the Add button. The Create New Data

Source dialog box appears.

Note: For SQL Server 2000, you may use any ODBC driver supported by the

database. For SQL Server 2005, however, you must use the ODBC driver that is

included with Microsoft Data Access Components (MDAC). Both Extract and

Replicat will abend if the SQL Native Client ODBC driver that is provided with SQL

Server 2005 is used.


Select the SQL Server driver and then click Finish. The Create a New Data Source

to SQL Server dialog box appears.

For Name, type <dsn>.

For a description, type "GG Training Data source" when creating this source DSN.

This field is optional.

For Which SQL Server do you want to connect to, select the server name.

Click Next.

Answer the question about how SQL Server should verify the login by selecting the

option that you entered when creating the <login>. Then type <login> in Login ID

and <password> in Password.


Click Next.

Make certain the default database is set to the database that you created; otherwise

select Change the default database to: and select it. Leave the other settings to their

defaults to use ANSI.

Click Next.

Leave the next dialog box set to the defaults, and click Finish.

In the confirmation, click Test Data Source to test the connection.

Close the confirmation dialog box and the Create a New Data Source box.


5. Create the practice tables


Go to SQL Server Management Studio, select your database and select New Query

from the toolbar.

If the correct database is not displayed in the drop down box (showing ggsdata

above), click on the arrow and select it from the list.

Click the File > Open and navigate to the demo_mss_create.sql script.

Click Open to open the script in the New Query window.

Execute the script by clicking the Execute Query button on the toolbar.

Verify the results using the following commands in the input window:

sp_help tcustmer

go

sp_help tcustord

go

Initial Data Load

Exercise 2.

Initial Data Load using Direct Load Method

Objective


Configure and execute the initial data load capture

Configure and execute the initial data delivery

Configure initial load

1. Add the initial load capture batch task group

Execute the following commands on the <source> system to add an Extract group named

EINI <unique id>1.

GGSCI> ADD EXTRACT EINI<unique id>, SOURCEISTABLE

Verify the results:

GGSCI> INFO EXTRACT *, TASKS

2. Configure the initial load capture parameter file


GGSCI> EDIT PARAMS EINI<unique id>

--

-- GoldenGate Initial Data Capture

-- for ECUSTMER and ECUSTORD

--

EXTRACT EINI<unique id>

RMTHOST <target>, MGRPORT <port>

RMTTASK REPLICAT, GROUP RINI<unique id>

TABLE <install vol>.GGSSOU.ECUSTMER;

TABLE <install vol>.GGSSOU.ECUSTORD;

1 The process names used in lab exercises, for example EINIBD, are made up of 1) one character for the

GoldenGate process (E for Extract, R for Replicat); 2) three or four to describe the process type (INI for

initial data load, ORA for capture from or delivery to an Oracle database, etc.) and 3) two characters to

create a unique identifier (usually your initials).

Initial Data Load

3. Add the initial load delivery batch task group


GGSCI> ADD REPLICAT RINI<unique id>, SPECIALRUN

Verify the results:

GGSCI> INFO REPLICAT *, TASKS

4. Configure the initial load delivery parameter file


GGSCI> EDIT PARAMS RINI<unique id>

--

-- Change Delivery parameter file for

-- TCUSTMER and TCUSTORD changes

--

REPLICAT RINI<unique id>

TARGETDB <dsn>, USERID <login>, PASSWORD <password>

SOURCEDEFS ./dirdef/source.def

DISCARDFILE ./dirrpt/RINI<unique id>.dsc, PURGE

MAP <install vol>.GGSSOU.ECUSTMER,

TARGET <owner/schema>.TCUSTMER; MAP <install vol>.GGSSOU.ECUSTORD,

TARGET <owner/schema>.TCUSTORD;

5. Execute the initial load process


GGSCI> START EXTRACT EINI<unique id>

Verify results using GGSCI:

GGSCI> VIEW REPORT EINI<unique id>

And using the SQL Query Analyzer on the <target>.

Query Analyzer> SELECT * FROM <owner/schema>.TCUSTMER

Query Analyzer> SELECT * FROM <owner/schema>.TCUSTORD


Exercise 3.


Objective


Configure and add the Extract process that will capture changes.

Add the trail that will store the changes.

Start the capture process.

Configure change capture

1. Create the Extract parameter file

Execute the following commands on the <source> system to define an Extract named

EENS<unique id> to pull data from the Enscribe TMF audit trails and route these changes to

GoldenGate trails.

GGSCI> EDIT PARAMS EENS<unique id>

--

-- Change Capture parameter file to capture

-- ECUSTMER and ECUSTORD Changes

--

EXTRACT EENS<unique id>

TCPIPPROCESSNAME <tcpip process>

FETCHCOMPS

RMTHOST <target>, MGRPORT <port>

RMTTRAIL ./dirdat/<trail id>

TABLE <install vol>.GGSSOU.ECUSTMER;

TABLE <install vol>.GGSSOU.ECUSTORD;

Note: Record the two characters selected for your <trail id>: ______. You will need this

defining the remote trail and for identifying the trail to be read by Replicat.

2. Add the Extract group


GGSCI> ADD EXTRACT EENS<unique id>, BEGIN NOW, CPU 1, PRI 160

Verify results:

GGSCI> INFO EXTRACT EENS<unique id>


3. Define the GoldenGate remote trail

Execute the following commands on the <source> system to add the trail declared in the

EENS<unique id>Extract parameters. This will be located on the target system.

GGSCI> ADD RMTTRAIL ./dirdat/<trail id>, EXTRACT EENS<unique id>,

MEGABYTES 5

4. Start the capture process


GGSCI> START EXTRACT EENS<unique id>

Verify the results:

GGSCI> INFO EXTRACT EENS<unique id>

GGSCI> VIEW REPORT EXTRACT EENS<unique id>

Discussion points

1. Identifying a remote system

What parameters are used to identify the remote target system?

___________________________________________________________________

___________________________________________________________________

2. Using the FETCHCOMPS parameter

What action is triggered by the FETCHCOMPS parameter? What type of input record is

affected?

___________________________________________________________________

___________________________________________________________________

Hint: Search in the NSK Reference Guide for FETCHCOMPS | FETCHLASTIMAGE


Exercise 4.


Objective


Set up the checkpoint table on the target system.

Create a named group that includes the Replicat process and the checkpoint tables.

Configure the Replicat group by adding parameters.

Start the Replicat group.

Set up checkpoints

1. Edit a GLOBALS file on the target system


Edit the GLOBALS parameter file to add the checkpoint table.


Shell> ggsci

GGSCI> EDIT PARAMS ./GLOBALS

In the text editor, type:

CHECKPOINTTABLE <owner/schema>.ggschkpt

Record the checkpoint table owner and name, then save and close the file.

Table owner ____________________ name ___________________

Note: You could name the table anything you want, but for training purposes we are using ggschkpt.

2. Activate the GLOBALS parameters

For the GLOBALS configuration to take effect, you must exit the session in which the changes

were made. Execute the following command to exit GGSCI.

GGSCI> EXIT


3. Add a Replicat checkpoint table


This step adds the checkpoint table that you specified when you created the GLOBALS

parameter file. It will be created in the <dsn> database.

Run GGSCI on the target.


Shell> ggsci

Execute the following commands in GGSCI.

GGSCI> DBLOGIN SOURCEDB <dsn>, USERID <login>, PASSWORD

<password>

GGSCI> ADD CHECKPOINTTABLE

Configure delivery

4. Add the Replicat checkpoint group

Execute the following commands on the <target> system to create the Replicat group named

RMSS<unique id>.

GGSCI> ADD REPLICAT RMSS<unique id>, EXTTRAIL ./dirdat/<trail id>

Note: Refer to your Extract set up for the correct two-character <trail id>.

5. Create Replicat parameter file

Execute the following command on the <target> system to edit the Replicat

parameter file.

GGSCI> EDIT PARAM RMSS<unique id>

Add the following lines to the parameter file.

REPLICAT RMSS<unique id>

TARGETDB <dsn>, USERID <login>, PASSWORD <password>

HANDLECOLLISIONS

SOURCEDEFS ./dirdef/source.def

DISCARDFILE ./dirrpt/RMSS<unique id>.DSC, PURGE

MAP <install vol>.GGSSOU.ECUSTMER,

TARGET <owner/schema>.TCUSTMER; MAP <install vol>.GGSSOU.ECUSTORD,

TARGET <owner/schema>.TCUSTORD;


6. Start the Replicat process


Execute the following command in GGSCI to start the Replicat process.

GGSCI> START REPLICAT RMSS<unique id>

Verify the results:

GGSCI> INFO REPLICAT RMSS<unique id>

Discussion points

Search in the Windows//UNIX Reference Guide for the information on the following

questions.

1. When to use HANDLECOLLISIONS

When would you use HANDLECOLLISIONS? What does it do?

_________________________________________________________________________

_________________________________________________________________________

2. What information is supplied by SOURCEDEFS?

_________________________________________________________________________

_________________________________________________________________________

3. What is the purpose of the DISCARDFILE?

_________________________________________________________________________

_________________________________________________________________________

Generate Activity and Verify Results

Exercise 5.


Objective


Execute miscellaneous update, insert, and delete operations on the source system.

Verify the delivery of the changes to the target.

Generate database operations

1. Execute miscellaneous update, insert, and delete operations


TACL> VOLUME <install vol>.GGSSOU

TACL> RUN <install vol>.<install subvol>.DEMOLDEO

Within DEMOLDEO the following prompts and replies are issued:



?U



?E

Verify change capture and delivery

2. Verify the results on your source system

Execute the following commands on the <source> system to view the contents of the source

Enscribe files.

TACL> FUP COPY <install vol>.GGSSOU.ECUSTMER

TACL> FUP COPY <install vol>.GGSSOU.ECUSTORD


Verify your target SQL Server 2005 results

3. Verify your results on the SQL Server 2005 target

Verify and record processing statistics on the <target> with the following command

in GGSCI.

GGSCI> SEND REPLICAT RMSS<unique id>, REPORT

GGSCI> VIEW REPORT RMSS<unique id>

Verify the data by executing the following commands in the New Query window of

SQL Server Management Studio on the source:

select * from TCUSTMER;

select * from TCUSTORD;

Turn off error handling

4. Turn off initial load error handling for the running delivery process


GGSCI> SEND REPLICAT RMSS<unique id>, NOHANDLECOLLISIONS

5. Remove initial load error handling from the parameter file

GGSCI> EDIT PARAMS RMSS<unique id>

Remove the HANDLECOLLISIONS parameter.

Exercise Name

goldengate for nonstop enscribe to sql - · pdf filegoldengate for nonstop enscribe to sql...

Documents