1 em413 using adaptive server anywhere’s remote data access feature mark culp software development...

1

EM413Using Adaptive Server Anywhere’s Remote Data Access Feature

Mark CulpSoftware Development ManageriAnywhere Solutionsmark.culp@ianywhere.com

2

Goals of this presentation

• Enable you to access remote data from your ASA database

• Show you how to migrate your data to ASA

• Help you improve your cross database join performance

• What should you not do when using ASA's remote data access feature

3

Contents

• What is “Remote Data Access” ?• Basics – How to get started• What if it doesn't work? – Some helpful

hints• Data migration tools• Cross database joins• Limitations – What to avoid• Questions

4

What is “Remote Data Access” ?

5

What is “Remote Data Access” ?

• Allows an ASA server to access data from a remote source as if it were local

• Remote data may be :• iAnywhere ASA, Sybase ASE• Oracle, IBM DB2, MS SQL Server• Other ODBC data sources

• E.g. MS Access, MS Excel

6

Why use “Remote Data Access” ?

• Cross database joins• Maintain real time access to remote

data• Integration to other projects and other

data sources• Migration of legacy data• Quickly set up a remote ASA database

for Mobilink

7

ASA Server

ASAStore

QueryExecution

Engine

ClientComm.

What is … - Architecture

RemoteStore

Other

ODBCASE

jConnect ASA

Remote

Server

8

What is … - Platforms

Available on• Win32 platforms:

95,98,ME,NT,2000• UNIX platforms (since ASA 7.0.2):

Linux and SPARC Solaris• Netware (since ASA 7.0.2)

9

What is … – Also known as …

Remote Data Access capability is also known as:

• OMNI - OmniConnect (from ASE 10.5)• CIS - Component Integration Services• Proxy Tables

10

Basics – How to get started

11

Basics

Three steps to accessing remote data1. Define your remote server2. Map your external login3. Create your remote proxy tables

Then use the proxy tables in the same way you would use any other table in your query

12

ASA needs to know:• What you will call the remote server• How to connect to the remote server• The capabilities of the remote server

Basics – Step 1:Define your remote server

SQL statement:create server <server_name>

class <connection_class>

using <remote_location>

13

Step 1 – Create server … class

Connection class:• JDBC (jConnect) classes:

• asajdbc, asejdbc

• ODBC classes:• asaodbc, aseodbc – ASA and ASE• db2odbc – IBM DB2• mssodbc – MS SQL Server• oraodbc – Oracle server (version 8.0 or later)• odbc – any other ODBC data source

14

Step 1 – Create server … location (JDBC)

Remote location (JDBC):<machine_name>:<port> [ /<db_name> ]

Location appended to JDBC URL specification:'jdbc.sybase.TDS.'

/<db_name> only required if more than one database loaded on remote server

15

Step 1 – Create server … example (JDBC)

Example 1 (using jConnect):create server DemoServerViaJconnect

class 'asajdbc'using 'localhost:2638/asademo'

jConnectasajdbcDemoServer

ViaJconnect

ASA server:

asademolocalhost

:2638/asademo

Remote server:

16

Step 1 – Create server … location (ODBC)

Remote location (ODBC):<DSN_name> [ ; <other_options> ]

DSN (data source name) normally specifies parameters on how to find remote server

• Driver name • Driver specific parameters:

• E.g. ServerNameConnection is in context of the ASA server

17

Step 1 – Create server … location (ODBC)

Example 2 (using an ODBC DSN):create server DemoServer

class 'asaodbc'using 'demoDSN'

DemoServer

ASA server:

asaodbc ODBCDriver

Manager

ODBCDriver

demoDSN

Note: You can use dbdsn to create ASA DSNs:dbdsn –w demoDSN –c “eng=asademo”

asademoEng=

asademo

Remote server:

18

Step 1 – Create server … location (ODBC)

Example 3 (connect to an Excel Spreadsheet):create server ExcelData

class 'odbc'using 'ExcelDataDSN'

Note 'Read Only' Check Box

19

Step 1 – Create server … location (ODBC)

Example 4 (no ODBC DSN):create server DemoServerNoDSN

class 'asaodbc'using 'driver=Adaptive Server Anywhere

8.0; ServerName=asademo'

DemoServerNoDSN

ASA server:

asaodbc ODBCDriver

Manager

ODBCDriver

Driver=…asademo

ServerName=

asademo

Remote server:

20

Step 1 – Create server … location (UNIX)

Remote location (UNIX ODBC):<DSN_name> [ ; driver=<driver_path> ]

There is no standard ODBC driver manager on UNIX, so ASA needs to know where to find the driver (manager) that you want to use.

You must specify a driver-path either:• In your location string, or• In your file DSN specification

21

Step 1 – Create server … location (UNIX)

Example 5 (UNIX - no driver manager): create server DemoServerDirect

class 'asaodbc'using 'demoDSN'

Contents of ~/.odbc.ini:[demoDSN]driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.soservername=asademo

22

Step 1 – Create server … location (UNIX)

Example 5 continued (UNIX - no driver manager):

• Since no "driver=" found in location string, ASA looks for location of driver in the DSN

asademoDemoServer

Directasaodbc demoDSN

ASA server:

ODBCDriver

dbodbc8_r.so

Remote server:

23

Step 1 – Create server … location (UNIX)

Example 6 (UNIX - with driver manager): create server DemoServerUsingManager

class 'asaodbc'using 'driver=<manager_path>;

DSN=demoDSN'

Contents of ~/.odbc.ini:[demoDSN]driver=/opt/sybase/SYBSsa8/lib/dbodbc8_r.soservername=asademo

24

Step 1 – Create server … location (UNIX)

Example 6 continued (with driver manager):

• ASA loads ODBC driver manager• ODBC driver manager loads ODBC driver

dbodbc8_r.so

asademoDemoServer

UsingManagerasaodbc .odbc.ini

driver=…

ASA server:

ODBCDriver

Manager

ServerName=

asademo

ODBCDriver

Remote server:

25

Basics – Step 2:Map your external login

ASA may need to know:• User id and password to use when

connecting to the remote server

SQL statement:create externlogin <login_name>

to <remote_server>remote login <remote_user>[ identified by <remote_password> ]

26

Step 2 – When to create externlogin

• By default, ASA uses the names and passwords of its clients whenever it connects to a remote server on behalf of those clients.

• CREATE EXTERNLOGIN statement assigns an alternate login name and password to be used when communicating with a remote server

27

Step 2 – Map your external login

Example 7:create externlogin dba

to DemoServerremote login dba identified by sql

28

Basics – Step 3:Create proxy tables

ASA needs to know how to make the mapping from a local proxy table to its remote data:

• If the remote table already exists, use the CREATE EXISTING statement.

• If the remote table is to be created, use the CREATE … AT statement.

29

Step 3 – Create existing table

To map an existing remote table:

SQL Statement:create existing table [<owner>.]<table_name>

[ ( <column_definitions> ) ]at <location_clause>;

• This statement defines the proxy table for an existing table on the remote server.

30

Step 3 – Location clause

location_clause specifies where to locate the remote table, and is of the form:

'<server>;<database>;<owner>;<table>'where:• server - name of server as known by current

database from create server command• database - depends on type of remote server• owner - owner of remote table• table – name of the remote table

31

Step 3 – Create existing example

Example 8:create existing table dba.p_employee

at 'DemoServer;;dba;employee';

dba.p_employee

ASA local database: DemoServer:

dba.employee

proxy table: remote table:

32

Step 3 – Create new table

To create a new remote table:

SQL Statement:create table [<owner>.]<table_name>

( <column_definitions> )at <location_clause>;

This statement creates a new table on the remote server, and also defines the proxy table for that table.

33

Step 3 – Create new example

Example 9:create table p_employee_sheet

( emp_id numeric(9,0), emp_fname varchar(30) )at 'ExcelData;;;employee_sheet'

Copy data into the remote table using: insert into p_employee_sheet

select emp_id, emp_fname from p_employee

34

Some Helpful Hints

35

Hints – Dropping proxy tables

To drop a proxy table, use:

SQL Statement:drop table [<owner>.]<table_name>;

• If the proxy table was created using CREATE EXISTING, only the proxy table is dropped

• If the proxy table was created using CREATE … AT statement, then the proxy table and the remote table are both dropped.

36

Hints – Dropping proxy tables example

Example 10:drop table p_employee• Since this proxy table was created using

CREATE EXISTING, this statement drops the proxy table only; the remote table is not affected.

drop table p_employee_sheet• Since this proxy table was created using

CREATE TABLE … AT, this statement drops the proxy table, AND will also drop the remote table at 'ExcelData;;;employee_sheet'

37

Remote Data Access passthru mode

To send a statement directly to the remote server, use:

SQL statement:forward to <server_name>;<remote_statement(s)>forward to;

• The remote_statements text is sent to the remote server as given

• Text is terminated by the "forward" and "to" tokens

38

Hints – Useful system tablessys.sysservers – list of remote servers

select * from sys.sysservers

39

Hints – Useful system tablessys.systable – list of proxy tables

select table_name, remote_location, existing_objfrom systablewhere remote_location is not NULL;

40

Hints – Useful built-in procedures

sp_remote_tables server_name [, table_name ][, table_owner ] [, table_qualifier ][, with_table_type ]

• Lists tables available at remote server

sp_remote_columns server_name, table_name[, owner ] [, database]

• Lists columns in remote table

41

Remote Data Access trace CIS option

You diagnose remote data access problems by using the cis_option connection property

SQL statement:set temporary option cis_option = 7;

• ASA will display messages in console log.• Set cis_option value to 0 (zero) to turn off.

42

Trace CIS option example

Example 11:set temporary option cis_option=7;select * from p_employee;

43

Data Migration Tools

44

Data migration tools - Availability

You can import remote data using the sa_migrate set of stored procedures

• Available in ASA 7.0.2, 7.0.3 and 8.0• Source data may be from one of:

• iAnywhere ASA• Sybase ASE• Oracle, IBM DB2, MS SQL Server, MS

Access

45

Data migration tools - Basics

Basic methodology:• create base table(s) and proxy table(s)• copy data from remote table(s)• create foreign keys

proxytable

ASA local database: Remote data source:

remotetable

basetable

46

Migration – Single step process

To migrate entire remote database in one step, use stored procedure:

dbo.sa_migrate(IN local_table_owner VARCHAR(128),IN server_name VARCHAR(128),IN table_name VARCHAR(128) DEFAULT NULL,IN owner_name VARCHAR(128) DEFAULT NULL,IN database_name VARCHAR(128) DEFAULT NULL,IN migrate_data BIT DEFAULT 1,IN drop_proxy_tables BIT DEFAULT 1 )

47

Migration – Single step example

Example 12: call dbo.sa_migrate(

'dba', 'DemoServer', 'Customer', 'dba' )• This migrates the customer table definition and

data from DemoServer to my database by:• creating customer table in my database• creating customer_et proxy table• copying the customer data• dropping the customer_et proxy table

48

Migration – Single step example results

49

Migration – Multi-step process

To have more control over what is migrated, use:

1. sa_migrate_create_remote_table_list( … )2. sa_migrate_create_tables( … )3. sa_migrate_data( … )4. sa_migrate_create_remote_fks_list( … )5. sa_migrate_create_fks( … )6. sa_migrate_drop_proxy_tables( … )

50

Migration – Step 1: create table list

1. Create a list of tables to migrate usingdbo.sa_migrate_create_remote_table_list(

IN server_name varchar(128),IN table_name varchar(128) default NULL,IN owner_name varchar(128) default NULL,IN database_name varchar(128) default NULL )

• this populates the dbo.migrate_remote_table_list table

• delete any rows that you do not want

51

Migration – Step 2: create proxy tables

2. Create proxy and base tables using:dbo.sa_migrate_create_tables(

IN local_table_owner varchar(128) )

• uses rows in dbo.migrate_remote_table_list

• creates a proxy table and a base table for each remote table listed

• also creates all primary key indexes

52

Migration - Step 3: migrate data

3. Migrate data into base tables using:dbo.sa_migrate_data (

IN local_table_owner varchar(128) )

• uses rows in dbo.migrate_remote_table_list

• populates base tables with rows from proxy tables

53

Migration – Step 4: create fks list

4. Create list of foreign keys to migrate:dbo.sa_migrate_create_remote_fks_list(

IN server_name varchar(128) )

• uses rows in dbo.migrate_remote_table_list

• creates dbo.migrate_remote_fks_list table• delete any rows that you do not want

54

Migration – Step 5: create foreign keys

5. Create the foreign keys using:dbo.sa_migrate_create_fks(

IN local_table_owner varchar(128) )

• uses rows in dbo.migrate_remote_fks_list• creates a foreign key for each listed

55

Migration - Step 6: drop proxy tables

6. Drop all of the proxy tables:dbo.sa_migrate_drop_proxy_tables(

IN local_table_owner varchar(128) )

• uses rows in dbo.migrate_remote_table_list

• drops the proxy table (created in step 2) for each remote table listed

56

Migration – Multi-step example

Example 13:• call dbo.sa_migrate_create_remote_table_list

( 'DemoServer', NULL, 'dba' )• select * from dbo.migrate_remote_table_list• delete from dbo.migrate_remote_table_list

where table_name = 'customer' or table_name like 'sales_%' or table_name like 'fin

%'• call dbo.sa_migrate_create_tables( 'dba' )• call dbo.sa_migrate_data( 'dba' )• call dbo.sa_migrate_create_remote_fks_list( 'DemoServer' )• select * from dbo.migrate_remote_fks_list• call dbo.sa_migrate_create_fks( 'dba' )• call dbo.sa_migrate_drop_proxy_tables( 'dba' )

57

Migration – In case of a failure

Migration may fail if a foreign key referenced table does not exist

• You may choose to update the dbo.migrate_remote_fks_list table (delete the offending entry) and continue

• Minimally you must run dbo.sa_migrate_drop_proxy_tables to delete the temporary proxy tables that were created

58

Cross Database Joins

59

Cross database joins – What is it?

A "Cross Database Join" is a join between two or more tables where the tables reside on two or more different servers

Table A(ASA)

Join

Table B(remote)

Example A:

Table A(remote a)

Join

Table B(remote b)

Example B:

60

Cross database joins – Implementation

ASA handles cross database joins by:• Parse the query and identify remote tables• Push predicates down to remote server when

possible• Reads remote rows using forward-only cursorNote:• 6.0.x and 7.0.x uses only naïve nested loop

join operator

61

Cross database joins – Remote Join

• When the join is on tables from the same server, the join is pushed to the remote

Table A(remote a)

Join

Table B(remote a)

Example C:

62

Cross database joins – Example 14

Example 14:• R is remote table, S is ASA local table

Select R.* from R join Swhere R.k = S.k

For each row r in Rif exists( select 1 from S where S.k = r.k )

match row

63

Cross database joins – Example 15

64

Cross database joins – Example 16

65

Cross Database Joins – Why is it slow?

If ASA uses a plan that has the remote table on the right side of the join operator,then the remote data will be hauled across the communication link multiple times – once for each left hand side row.

For each row s in Sfor each row r in ( select * from R )

if s.k = r.k match row

66

Cross Joins – How to make it go faster?

• Only include remote columns that are absolutely required by the query

• Run the query using dbisql, and check the plan that is being used

• Simplify query to have fewer remote tables• Consider using local temporary tables to

hold intermediate results• Migrate data to ASA and perform join on

local table

67

Limitations:

What should you avoid ?

68

Limitations

Features not supported on remote data:• ALTER TABLE statement against remote tables• Triggers defined on proxy tables will not fire• SQL Remote• Java data types• Foreign keys that refer to remote tables are

ignored• Positioned UPDATE and DELETE• UPDATE and DELETE requiring an intermediate

temporary table• Backwards scrolling on cursors

69

What to avoid … No cycles

1. Don't create a proxy table that refers back to the same database.

Reason:Deadlock can occur if remote access waits for a lock caused by the same connection.

Corollary:Don't create cycles in your databases' remote access definitions.

70

What to avoid … Separate Servers

2. Don't create a proxy table that refers back to the same server.

Reason:Deadlock will occur when more than one concurrent remote access is attempted.

Work around: increase –gx parameterSolution:

Use separate servers to host the local and remote databases.

71

What to avoid … Join Large Tables

3. Don't join two large remote tables from different remote servers.

Reason:Slow response

Solution:Migrate one of the tables into an ASA table first.

72

What to avoid … N-way joins

4. Don't create complicated joins containing many remote tables from many different servers.

Reason:Slow response

Solution:Migrate table(s) into ASA first.

73

Summary

ASA's Remote Data Access feature is a powerful tool to access your remote data from your ASA database:

• Easy to migrate your legacy data to ASA, or• Use your remote data in place as if it were

part of the ASA database

More detailed information found in:ASA SQL User's Guide

15. Accessing Remote Data

74

Thank you

Questions ?

mark.culp@ianywhere.comwww.ianywhere.com