oracle database lite indepth · baker collins kelly roberts young . oracle database lite indepth...

Oracle Database Lite InDepth An Oracle White Paper Dec 2003

Oracle Database Lite InDepth


List of Figures .................................................................................................... 4 Introduction ....................................................................................................... 5 Oracle Lite Database Architecture.................................................................. 6

Database File Structure ................................................................................ 7 Metadata .................................................................................................... 8

POL__CONS....................................................................................... 9 POL__TBLCONS .............................................................................. 9 POL__TBLUSAGE............................................................................ 9 POL__COLUSAGE......................................................................... 10 POL__INDICES............................................................................... 10 POL__INDICESDT ........................................................................ 10 POL__TRIGGERS........................................................................... 10 POL__SYNONYMS ........................................................................ 10 POL__USERS ................................................................................... 10 ALL_DEPENDENCIES ................................................................ 10 POL__SCHEMATA......................................................................... 10 POL__VIEWS................................................................................... 10 POL__SEQ ........................................................................................ 10 POL__ALLOBJ................................................................................. 10

Multi-User mechanisms ............................................................................. 11 Query Optimisation.................................................................................... 11

Introduction ............................................................................................ 11 Execution Plan Generation................................................................... 12

Single Table Queries.......................................................................... 13 Optimizing Join Queries................................................................... 13

Query Execution Engine....................................................................... 14 Oracle Database Lite Replication.................................................................. 15

Introduction................................................................................................. 15 Definitions ................................................................................................... 16 Replication Specific Tables........................................................................ 17 Table Detail ................................................................................................. 18 Replication Processing ............................................................................... 19

Client Synchronisation........................................................................... 19 MGP Processing..................................................................................... 21

Tuning Replication ..................................................................................... 21 A Typical Tuning Example................................................................... 22

Application Deployment............................................................................ 26 Automatic replication of Application Files ........................................ 26 Customisable setup utility ..................................................................... 26 Offline instantiation ............................................................................... 26


Customisation of Replication.................................................................... 26 Client Synchronisation Customisation ................................................ 27

Client Synchronisation Parameters(default as implemented by mSync) ................................................................................................. 27

Transport Customisation ...................................................................... 29 Snapshot Replication Customisation................................................... 30

Snapshots based on views ................................................................ 30 Snapshots based on remote tables .................................................. 31 Queue based publication items........................................................ 32 Customising DML............................................................................. 34 Using Before and After Apply procedures .................................... 35 Using MyCompose methods............................................................ 35

Publication Customisation .................................................................... 35 Administration Customisation ............................................................. 37

Mobile Server ................................................................................................... 38 Web-to-go......................................................................................................... 38

Repository .................................................................................................... 39 Client Database ........................................................................................... 40 Web-To-Go Database................................................................................ 40 Workspace Customisation......................................................................... 41

Appendix A – API Template Program ........................................................ 42



LIST OF FIGURES Figure 1 - Oracle Lite Database Architecture ............................................... 6 Figure 2 - Page Layout...................................................................................... 7 Figure 3 - Example B+Tree index .................................................................. 8 Figure 4 - Oracle Database Lite Replication ...............................................15 Figure 5 - Main Replication Objects.............................................................16 Figure 6 - Timings Output.............................................................................25 Figure 7 - Mobile Server configuration........................................................38 Figure 8 - Web-To-Go ...................................................................................39 Figure 9 - WSH Utility....................................................................................41


Oracle Database Lite Indepth

INTRODUCTION Oracle Database Lite is an addition to the Oracle Database used for rapid development and deployment of high-impact, mission-critical applications for mobile, embedded and lightweight business environments. This white paper provides an in depth discussion of the technical architecture of the Oracle Database Lite product set (which includes Mobile Server, Mobile Client, Web-To-Go and the Oracle Lite database), enabling application developers to make full use of the functionality available with the product.1

1 On Win32 platforms, ODBC applications access the ODBC driver via the ODBC driver manager.


ORACLE LITE DATABASE ARCHITECTURE The Oracle Lite database client runs on Win 32, Windows CE, EPOCH and Palm OS devices. It has a very similar architecture on each d2evice.

Figure 1 - Oracle Lite Database Architecture

Java A pp lica tion

JD B C T ype 2

O D B C A pp lica tion

O D B C D riv er O L O D 2 0 4 0 .D L L

SQ L E n gin e O L SQ L 4 0 .D L L

O K A PIJava S tored P ro cs

O racle L ite C ore

O L O B J4 0 .D L L

O L JD B C 4 0 .D L L

Java A pp lica tion

O D B C A pp lica tion

O racle L ite L isten er

O L S V 2 0 4 0 .E X E

JD B C T ype 4 O L C L 2 0 4 0 .D L L

O D B C R em ote C lient

D ataba se F ile

ok A PI A pp lica tion


Database File Structure An Oracle Lite Database consists of a single file. When something is stored in an Oracle Lite file, it is referred to as an object. An Oracle Lite file is split into 4K pages. Each page is identified by a 21-bit page identifier. The first page of the file is a volume page that maintains information about the database. Each page is referenced by an entry in a page map (multiple page maps can exist per file). The page map is a bit map indicating if a particular page is free or not. The volume page contains the first page map of the database.

Figure 2 - Page Layout

Header(4bytes) Data Slot-table pgCntl_s Trailer(4bytes)

Each page has a pgCntl (page control) that indicates, amongst other things, the number of slots and amount of free space in the page. . The slot-table is variable in size. It grows backward. Each entry in the slot table contains an offset to an object stored on that page. Each object is therefore identified by the 21 bit page number on which it resides and an 11 bit slot number which identifies where on the page it is stored. For convenience, objects are grouped together into groups. Normally a group contains objects of the same type. This is not the case for a special group called the MetaGroup that maintains objects from various classes (see Below). All the objects of one type (class) equate to a table in SQL. Oracle Lite also maintains structures for Indexes (see below).

Each object is made up of an object header and an object body. If an object that exists on one page, is updated, and no longer fits on its home page, the object is moved to a different page. The object header on the new page indicates that it is the new home of the object and contains the new objects length. The original object header remains in its original slot and is updated to indicate that the object has moved and what its new page/slot location is.

Indexes are stored as B+Trees (see Figure 3 - Example B+Tree index). The B+Tree index pages follow the same page layout as described above. Each index node is represented by a page and each index entry by a slot in this page. Each index entry contains the key value for lookup and either a pointer to the child page for non-leaf pages or an array of object ids (page/slot combinations) corresponding to the key value. The index B+Tree has no fixed width or depth. Each leaf node contains a variable number of entries. The page is split if the next key value to be inserted or updated cannot fit in the page. Each page at a particular level in the B+Tree are linked by next and previous pointers.


Metadata

Every Oracle Lite database contains two special groups called MetaGroup and POL__META. These groups contain a number of different objects that describe the database. The MetaGroup group contains okapi meta classes (class names that begin ok) and the POL__META group contains class names beginning with

Figure 3 - Example B+Tree index

Root

Node

Node Node Node Node

Data Data Data Data Data Data Data Data

Adams Davis Kelly

Kelly RobertsYoung

Davis Franklin

Adams Baker Collins

Roberts Young Kelly


POL__ or ALL. The main set of objects belong to the okClass class. This basically lists all the tables in the database including other metadata ones. The okAttr class contains all the columns in the database. The following metadata tables are created at database creation time (when using the okapi create database call) and then used by the SQL Engine:

POL__CONS

Describes the defined constraints on the database. • Catalog: The fully qualified database file name. For example

C:\OLITE\MOBILE\SDK\OLDB40\POLITE.ODB • Schema: The owner of the table • Constraint: The name of the constraint. This is either entered by the user in

the DDL or system generated. • Type: Type of the constraint (The pseudo column CONSTRAINT_TYPE

will decode this value). • TblCons: okRef to POL__TBLCONS that refers to the constrained table. • AssertCons: reserved • DomainCons: reserved • Flags: for update rules. You can use UPDATE_RULE, DELETE_RULE,

and STATUS pseudo columns to extract values from this column. • To: Used by foreign key constraints to point to the parent table primary key

constraint object (i.e. the constraining table). Can be linked to POL__COLSUSAGE via column TblUsageRef, which describes the columns used in the constraint.

• Extensions: not used

POL__TBLCONS

Provided additional information for each constraint.

• Catalog: Same as POL__CONS. • Schema: Same as POL__CONS • Table: Name of the table. • ConsRef: Reference to POL__CONS object. • Flags: Reserved • Type: Reserved • Search: Used by Check constraint to store the user constraint string. For

example if the SQL Statement was “create table foo (I int constraint cons_i check (i > -10))” the “Search” will contain “I > -10”

• SearchTree: The internal representation of the constraint clause • Extensions: not used • ClassRef: Reference to the Class object for the table. • GroupRef: Reference to the group object for the table.

POL__TBLUSAGE

Describes the user defined child tables involved in constraints.

• Catalog: Same as above • Schema: Same as above • Table: Name of the table • TblConsRef: Reference to POL__TBLCONS object. • DomConsRef: not used • AssertConsRef: not used


POL__COLUSAGE

One entry for each column occurring in the constraint definition. • Column: Name of the column. • TblUsageRef: Reference to its POL__TBLUSAGE object • Position: Position of the column in the key (used to identify order of

columns in the key), null for check constraint.

POL__INDICES

Describes the user defined indexes on tables.

POL__INDICESDT

Describes the columns in the indexes

POL__TRIGGERS

Describes the triggers on tables

POL__SYNONYMS

Describes any user defined synonyms

POL__USERS

Describes any non system users defined

The SQL engine creates the following metadata tables during the execution of the first DDL statement.

ALL_DEPENDENCIES

Describes all objects that are dependant on other objects (such as Views on Tables).

POL__SCHEMATA

Describes any user defined schemas created.

POL__VIEWS

Describes any views defined.

POL__SEQ

Describes any sequences defined.

POL__ALLOBJ

Describes any user defined objects.

The above information can be accessed via either okAPI or via the SQL engine. When using the okClass and okAttr classes in a SQL statement the names must be quoted. This also applies to any columns included in the query (this is also true of columns in the POL__ tables). e.g.

SELECT “Name” FROM “okClass”


The SQL engine also recognizes a number of pseudo columns when selected from some of the metadata table these are:

From the okClass table: TABLE_SCH (schema), TABLE_NAME, TABLE_TYPE, TABLE_COMMENT, VIEW_LEN (for TABLE_TYPE = “VIEW” only), VIEW_TEXT

From the okAttr table: COLUMN_SCH, TABLE_NAME, COLUMN_NAME, COLUMN_COMMENT, TYPE_CODE, SUBTYPE_CODE, COL_PRECISION, NULLABILITY, POSITION, DEFAULT_LENGTH, DATA_DEFAULT

From the POL__CONS table: CONSTRAINT_TYPE, UPDATE_RULE, DELETE_RULE, and STATUS

From the POL__INDICESDT table: SCHEMA_NAME, INDEX_NAME, TABLE_SCHEMA, TABLE_NAME

From the POL_SYNONYM tab le: SCHEMA_NAME, INDEX_NAME, TABLE_SCHEMA, TABLE_NAME

Multi-User mechanisms Oracle Lite uses Shared Memory segments to provide for multi-user access. When the first Oracle Lite process starts up, by default, it maps 128MB of virtual memory at the address specified by the SharedAddress parameter in POLITE.INI (30000000 by default). The amount of shared memory allocated can be increased or decreased by specifying it through MaxSharedMemSize parameter in POLITE.INI (e.g. MaxSharedMemSize = 64 will set the shared memory size to 64MB. The minimum size is 32MB.). The process then opens the specified database, reads the root directory of that database and reads the pages it needs into the shared memory. The pointers within these structures are absolute pointers, to the position of the shared memory in the virtual memory address space. For this reason any second or subsequent Oracle Lite process starting up needs to map the shared memory to the SAME virtual address.

If this cannot be done because the new process is already using that address, it fails with an error. However, before it exists, it writes an address that it has free into the SuggestedSharedMemory parameter in POLITE.INI. If all OLITE processes are now terminated, and either the SharedMemory address is either changed to the SuggestedSharedMemory address or the SharedMemory parameter is not specified, the first Oracle Database Lite process to start up will use this address to map the 128MB. If it succeeds, second and subsequent Oracle Lite processes will use this address and hopefully not have a problem. This sequence of events can be repeated until all clients start successfully.

Query Optimisation

Introduction

Oracle Lite goes through a number of steps to execute a SQL query. This paper will mainly talk about executing a SQL query (a SELECT statement). The query execution involves the following steps:


1. Parsing: The input statement is scanned and is converted into an internal representation whose shape is like an abstract syntax tree. In this process grammatically incorrect statements are flagged and a syntax error is returned.

2. Normalization: The tree is “walked”, normalized and decorated. That is simple transformation are carried out and semantic information is added to the tree nodes. Some tautologies if any are evaluated. Any semantic error is caught during this tree traversal.

3. View expansion: Any references to views are expanded in line and the view tree is “walked”

4. View Optimization: The view expansions are collapsed into the main queries. For example a statement “SELECT * FROM v1,v2 where v1.c1=v2.c2” will be resolved to a query on the base tables in v1 and v2. The transformation takes place on the query tree.

5. Subquery optimization: The “IN” subqueries are analyzed and are un-nested into the containing queries if possible. The determination to un-nest a subquery is based on its position or depth of nesting as well as uniqueness of the result returned by it. The uniqueness is determined by analyzing the indexes on tables, the list of columns selected and predicates in the where clause.

6. Transitive Closure of Predicates: In this technique a predicate is analyzed and extra inferences are added to the predicate. This will later help the optimizer in selecting the better execution plan.

7. Predicate Push: The predicates are pushed down from top to bottom. This helps the queries on top of views. When a view contains UNIONs, MINUS and GROUP BY it helps to push the filtering condition to the source of data or base tables.

8. Execution Plan Generation: The query is now analyzed to generate a best execution plan. The optimizer uses a cost based approach to select the best execution plan.

9. Query Execution: The execution plan generated is used to execute the query.

Execution Plan Generation

Oracle Lite uses a “cost” based approach to select an optimal execution plan. The cost for Oracle Lite optimizer is I/O cost only. CPU cost is ignored. The optimizer is sophisticated considering its size and limited amount of statistics available to it. The Oracle Lite engine maintains the following statistics at all times. You do not have to run a separate command to update the statistics. The statistics value is always reliable.

1. Number of rows in a table 2. Number of pages in a table 3. For each index, number of distinct keys 4. Maximum and minimum values of a key


5. A pretty good estimate of number of rows selected for a given range of an index key.

Single Table Queries

Oracle Lite database engine supports the following table access methods.

1. Full table scan: This method is used when there are no indexes or the number of rows being retrieved is huge. The cost of retrieval using this approach is equal to the number pages in the table. This cost is easily available.

2. Index access method: In this method a key value or a key ranges (E.G. price between (10,15) ) is used to retrieve the rows. The key is used to find the row-ids of the matching rows. Each row-id uniquely identified the location of the row in the database. The cost of access is depended upon the height of the btree (index tree). The cost of access of a single row is “height + 1” I/O.

SINGLE TABLE I/O COST

A single query is optimized by estimating the selectivity of each predicate in the WHERE clause and estimating the cost of access for a) Full table scan, b) each available index access method. An execution plan that produces the lowest cost is picked. The selectivity does not matter for the full table scan. The selectivity for index retrieval is given as follows:

1. For an equality predicate: 1/number-of-distinct-key 2. For a range query: Number-of-rows-selected/total-number-rows-in-the-

table

The selectivity * total-number-rows is used to calculate the I/O cost. If selectivity is low index access method will be used. The preference is given to the primary key index as the data is assumed to be clustered on the primary key index and the costing formula is corrected for it.

INTERESTING ORDER OPTIMIZATION

For a single table query that contain “order by” or “group by” clause this technique is used to influence the access method chosen above. An estimate is made on the size of the result set and cost of sorting the result set. Now if an index is available that can return the rows in the right order, it may be beneficial to use that index to retrieve the data and avoid the sorting cost. If the new execution plan is less costly, it will be chosen. Basically we are willing to pay a higher cost for the index access method provided saving the sorting cost adequately compensates us.

Optimizing Join Queries

Optimizing join queries is complex because of the number of possibilities explode.

1. The local predicates are isolated (the predicates on a single table) from join predicates. Then the effective table size is estimated by applying the local predicates to the table. Whenever an index is available the calculation of selectivity is fairly accurate. It is assumed that the data is uniformly


distributed. But when the data is skewed, the estimate can go wrong and the execution plan chosen may not be optimal.

2. The selectivity of each predicate is calculated.

3. A “driving table”, the table with the smallest effective cardinality, is picked first. Its optimal access method is picked (see optimizing single table queries). The table is put in the set of “outer” tables.

4. The query is looked at to find out which possible tables can be joined to the tables in the current “outer” tables set. The “cross product” is not considered. The I/O cost is estimated for all possible joins. The least costly join is picked and the chosen table is added the “outer” table set. The step is repeated until all tables in the query have been selected. By the end the execution order of the tables is known and what access methods to pick for each table.

5. The current execution plan is saved and a new driving table chosen whose effective cardinality is the second lowest. Step 4 is repeated and the least expensive execution plan of the two plans is selected. This is repeated with third, fourth and fifth tables (if more than 5 tables exist the optimiser does not carry on).

6. The optimizer also creates hints such as when to cache the subquery results, or create an index for intermediate results of a view. The last one is useful when joining a view that is not collapsed to another table or view.

INTERESTING ORDER OPTIMIZATION

The interesting order optimization is carried out for join queries also. Again the cost is estimated two ways. Sorting + best execution plan and no sorting but using the right set of indexes. The optimal execution plan is chosen.

Query Execution Engine

The query execution engine relies on the database engine to implement the access methods on a single table. The database engine also provides basic data filtering. The SQL rich filtering involving SQL functions and aggregation is implemented in the Query Engine layer. The join operation is implemented by passing on the right set of filtering condition to the tables in a nested loop fashion. The query engine contains code to implement all SQL functions. The database engine supports sorting. Additional techniques like creating a temporary index, or caching the subquery results are also implemented in the query engine. Data conversions are also the responsibility of the query engine.


ORACLE DATABASE LITE REPLICATION Oracle Database Lite includes a lightweight fully functional replication engine.

Introduction Oracle Database Lite has built in replication functionality. At the database level Oracle Lite supports the CREATE SNAPSHOT command (and it’s okAPI equivalent). This creates a special table that automatically tracks changes to rows on that table. This can be seen through the SQL engine using the OL__ROW_STATUS pseudo column. Value 16 means an inserted row, 32 means an update row and 80 means a deleted row. By adding a + to the table name the SQL engine lists only those rows that have a non zero row status (this is an indexed operation, so quick). Surrounding this functionality is a small client side component, mSync and a large server side component consisting of the Mobile Server and the MGP (see later).

The mSync component is available as an executable, as a COM object and as an API allowing programming in Java, C, C++ etc. The server side components are documented below.

Figure 4 - Oracle Database Lite Replication

Admin Schema

Client

In Queue

Out Queue

MGP

Base Schema

Base Table

Version Table

Log Table

Error Table

Client Database

ConsCLI Database


Definitions Snapshot A vertical and/or horizontal subset of table data. A simple

snapshot’s main query is based on a single table. A complex snapshot’s main query can be based on multiple tables.

Publication Item

A publication item is the definition of a snapshot to be replicated to the client machine. A number of customisations can be made to the replication process at the publication item level. Using WTGPACK the name of the publication item is generated for you. Using the API you can choose your own publication name. The term publication item and snapshot are used interchangeably.

Publication A publication is a group of snapshots (or publication items) that will be replicated to the client database. Using WTGPACK the publication name is generated for you. Using the API you can choose your own publication name. A user subscribes to a publication and receives data for all publication items it encapsulates.

Application An application is synonymous with a publication and is used as an administration object. For web based applications, the application also encapsulates the application definition and files.

Client The definition of a user who is interested in data from a snapshot

Subscription The publication items a client is interested in

Subscription Parameters

Subsetting parameters for each client

Figure 5 - Main Replication Objects

Application Publication

Publication Item

Client

Application Access

Subscription

Subscription Parameter


Replication Specific Tables Base Schema

TableName The name of the table upon which the snapshot is based

CVR$TableName The Version table, in this table is 1 row per row in the base table name. Each time a row in the base table is updated, its corresponding row in the version table is updated.

CLG$TableName The Log table. The first pass of the MGP compose process dirty rows in the version table are copied to log table. The version table rows are then cleared as being dirty.

CEQ$TableName The Error table. Any rows that encounter errors during the apply phase are placed in the error table. The error message corresponding to the error is placed in the admin schema C$EQ table.

Mobile Admin Schema

CFM$PublicationItemName The In Queue. Rows that are dirty (new/updated/deleted) on the client are placed in this table during a replication session. They are stamped with the version number of the corresponding row on the CMP$PublicationItemName table. Once the have been applied to the base table, they are deleted.

CMP$PublicationItemName The Map Table and Out Queue. Rows that need to be sent to the client are placed in this queue. The rows in this table persist and provide a record of what has been sent to the client.

C$ALL_CLIENTS Provides a list of all clients (users), with information about last sync time etc.

C$ALL_CLIENT_ITEMS Provides a list of all publication items for each client.

C$ALL_LOGGED_TABLES A list of all the tables that are involved in replication including a flag saying if they are


dirty or not.

C$ALL_SID_LOGGED_TABLES A list of all the tables that have been dirtied by different sessions since the last MGP compose run. During the first part of the MGP compose process, the C$ALL_LOGGED_TABLES table is updated from this table. Used to prevent concurrency bottlenecks.

C$INQ Used for Queue Publication Items – see separate section on customisation.

C$PUB_LIST_Q Used for Queue Publication Items – see separate section on customisation.

C$IN_MESSAGES Logs transactions received from clients

C$MGP_HISTORY *NEW TABLE* Provides historical data about MGP processes

C$SYNC_HISTORY *NEW TABLE* Provides historical data about synchronisations at a client level.

Table Detail CVR$TableName

<Primary Key Columns>

VERSION$$ Starts at 1 for new records, incremented on each change.

DMLTYPE$$ D(elete), I(nsert), U(pdate) Null indicates MGP has processed the change

CLG$TableName same as CVR$TableName

CFM$PublicationItemName

CLID$$CS Client ID

<Columns in snapshot>

DMLTYPE$$ D(elete), I(nsert), U(pdate)

SEQNO$$

TRANID$$ The transaction id. Each sync session has is a separate transaction.

VERSION$$ The version of the row (copied from the CMP$PublicationItemName table).


CMP$PublicationItemName

CLID$$CS Client ID

RID$$ As of 5.0.2 not relevant but used for old clients that allowed for replicating tables without primary keys. The RID represented the virtual primary key created on the client database.

<Primary Key Columns>

VERSION$$ The latest version of the row replicated (or to be replicated) to the client.

DMLTYPE$$ D(elete), I(nsert), U(pdate) – current DMLTYPE of row

SEQNO$$

DTYPE$$ D(elete), I(nsert), U(pdate) – last DMLTYPE of row replicated to client.

Replication Processing When a publication item is created, all publication item specific tables (i.e. 3 per base table in the publication and the in and out queues) are created and the version tables populated. A set of triggers (Before Delete, After Insert, After Update) is placed on each of the base tables involved in the publication item. These flag the table as being dirty in the C$ALL_SID_LOGGED_TABLES table, and update the corresponding version table with a new version number and current DML type. The MGP will then populate the out queue for each client.

Client Synchronisation

Before first synchronisation, the Mobile Client for the relevant platform must be installed. On the Windows platform this will create the directory structure below.

On other mobile devices, everything is contained in one directory by default (although on Windows CE, the databases can be stored on a flash card by setting the DATADIRECTORY parameter in polite.txt – see customisable setup on how to do this during setup).

UserName

oldb40

WebToGo

Mobile Client Root

Where oldb40 contains the databases in directories identified by username and WebToGo contains the software


A number of things are setup during first synchronisation.

• A new database will be created on the client (on Windows in the relevant directory).

• A new ODBC DSN will be created for the database.

• The database will be populated with all the defined snapshots, any defined DML will be run and the snapshot tables will be populated.

The Oracle Lite core automatically keeps track of which rows in a snapshot table have changed. During second and subsequent synchronisations, the client performs 4 phases.

1. It identifies using the Oracle Lite core which rows of which snapshots have changed. These are written to a temporary file.

2. These are then passed in one HTTP call to the mobile server for processing. Upon receiving the file the Mobile Server

a. Tries to lock the clients record on the C$ETC_PASSWD table. If this is unsuccessful a message is returned ‘Server Busy’. This normally means that a client with the same name is syncing. However it can also occur if either: the client has completed one sync and is trying to initiate another before the mobile server has completed its work (certain tasks are performed after the client has been sent the out queue data and if these are not complete when the client tries to sync again, this can happen); or a client has disconnected without notifying the server and the lock has remained in place.

b. Upon success, it then locks the clients record in the C$ALL_CLIENTS table and all the clients records in the C$ALL_CLIENT_ITEMS table. If this is unsuccessful, a message is returned ‘Server Busy, MGP processing’ to the client. This is because the MGP locks individual rows in the C$ALL_CLIENT_ITEMS table when it is processing (see below).

c. Upon success, it then processes the data it has received by inserting a record in the C$IN_MESSAGES table and by inserting the data it has received into the relevant inqueues (it may also apply the data to the base tables, see below) and it prepares the data to send to the client. This is based on all dirty rows in the out queues (As identified by C$ALL_CLIENTS OUTQUEUE_DIRTY being set to ‘Y' and C$ALL_CLIENT_ITEMS DIRTY being set to 1.

d. This is then sent to the client via HTTP (one session, multiple calls).

3. The client receives this data into a temporary file,


4. The changes in the file are then applied to the client database. If this phase fails, the client can ask the mobile server to re-send the changes at a later date, which is does based on the DTYPE$$ field in the out queue.

MGP Processing

The MGP has two phases the Apply phase and the Compose phase. The MGP performs the Apply phase for all clients followed by the Compose phase for processes all clients (as determined from C$ALL_CLIENTS).

During the Apply phase, the MGP processes all dirty in queues for each client (as determined by rows in the C$IN_MESSAGES table). Each row in each in queue is examined. If the primary key fields of the in queue match that of an existing row in the base table, and the either the version numbers of the two rows do not match (by examining CVR$TableName) or some server DML has been performed on the row (because the DMLTYPE$$ is set) a conflict has occurred. If the snapshots conflict resolution rule is set to SERVER WINS, the row in the in queue is discarded (in 5.0.3 of Oracle Database Lite a record of this will be written to the CEQ$TableName table). If the snapshots conflict resolution rule is set to CLIENT WINS, the row on the server is discarded. Conflict resolution can be customized (see customization section). The Apply phase may be performed synchronously during client synchronisation. This will happen if the client database contains any updateable complete refresh snapshots. Currently, any application packaged with the WTGPACK utility will contain system generated updatable complete refresh snapshots. This behaviour is due to change in 5.0.3.

The first step of the Compose phase is to prepare the log tables of all dirty base tables. Firstly, the MGP updates the C$ALL_LOGGED_TABLES table with entries from the C$ALL_SID_LOGGED_TABLES table. Next, for each table identified as dirty (DIRTY=’Y’), the dirty version records (DMLTYPE$$ is not null) are copied into its log table (CLG$). The DMLTYPE$$ column of the version table is then cleared (unless it equals ‘D’, in which case it is deleted). The DIRTY flag in the C$ALL_LOGS table is set to ‘Y’.

The MGP then processes all dirty out queues for each client. Firstly, the clients status is set to ‘C’ on C$ALL_CLIENTS (which locks the record).

All dirty tables that are relevant to the client are selected (by joining C$ALL_CLIENT_ITEMS with C$ALL_LOGS). For each dirty table, the publication's out queue is populated with rows from the log table that are relevant to that client (based on any subscription parameters). The client item for that publication is flagged as dirty (C$ALL_CLIENT_ITEMS.DIRTY set to 1).

At the end of the compose phase the status of the client is set to ‘R’ and with the out queue set to ‘Y’ (on C$ALL_CLIENTS).

All log tables are truncated and the C$ALL_LOGS table reset.

Tuning Replication A major source of performance problems associated with replication is badly performing SQL executed by the MGP. It might be that the original snapshot defined by the application development is sub optimal. But even where the original


snapshot is optimized, as the MGP uses the SQL defined in the snapshot as a sub query in various guises, this can cause the SQL to become sub optimal.

The Oracle Database Lite provides a tool Consperf.exe to assist in the tuning of snapshots. This is documented in Appendix J of the Oracle Database Lite Administration and Deployment guide.

The Consperf tool can provide 2 reports useful in tuning snapshots as well as providing an Autotune facility to automatically tune the SQL used during replication. During MGP processing, a number of different operations need to be performed. Each operation requires a different SQL statement to be issued. The particular shape of this SQL statement can influence how optimal that statement is. By default the MGP always uses the same shape of statement. During an autotune run, each shape of SQL statement is executed and the best performing one is chosen for MGP to use in future executions. The operations that can be tuned in this way are:

• Synchronisation

o SYNC_1 – Uses OUT JOIN for synchronising insert, updates and deletes.

o SYNC_2 – Uses normal joins synchronising inserts and updates.

o SYNC_3 – Uses a base view to do inserts and updates.

• Deletes from Outqueue

o LDEL_1 – Uses an EXISTS subquery

o LDEL_2 – Uses a correlated IN subquery

o LDEL_3 – Uses a HASH ANTI JOIN subquery

o LDEL_4 – Uses an uncorrelated IN subquery

• Inserts into Outqueue

o LINS_1 – Uses an EXISTS subquery

o LINS_2 – Uses a correlated IN subquery

o LINS_3 – Uses an uncorrelated IN subquery

• Inserts into Log Table

o LG_1 – Uses an EXISTS subquery

o LG_2 – Uses a correlated IN subquery

o LG_3 – Uses an uncorrelated IN subquery

A Typical Tuning Example

Tuning should be performed either on the production system or on a representative copy. Optimizer statistics should be up-to-date (if they are not then the CONSPERF tool can be instructed to collect them prior to performing the tuning exercise). When performing a tuning exercise, it is first necessary to identify the publication to be tuned, an example client to be used during tuning and optionally a list of publication items to tune (if you do not know which snapshots


are likely to be causing a problem , you leave this blank and all publication items are included in the tuning exercise).

If you use the packaging wizard, you will not normally know the name of the publication to be tuned. Executing the following SQL as the repository owner can identify this:

select publication

from applications,

app_publication

where name=<APP NAME>'

and app_id=id;

Having established the required information, the Consperf tool can be run. It is first run in reporting mode:

CONSPERF MOBILESCHEMA=<mobile server schema> MOBILEPASSWD=<mobile server password> SERVICENAME=<tns service name> PUBLICATION=<publication name> CLIENTID=<client identifier> TOUTFILE=<timing output filename> POUTFILE=<explain plan output filename> TIMEOUT=<query timeout value in seconds> ORDERBYPUBITEM (order output by publication item name)

When CONSPERF executes each query, it sets a timeout so that it does not have to wait for long running SQL to complete. It is a good idea to make this timeout larger than the default value. A value of 30 seconds is not unreasonable but will obviously slow down the tuning run.

This will produce 2 reports in the TOUTFILE and the POUTFILE. The TOUTFILE contains the timing information for the execution of each shape of SQL and the POUTFILE parameter contains execution plans for the SQL executed.

The first task is to examine the POUTFILE, this file contains execution plans for all publication items in the publication (unless specific items have been included or excluded). Each BASE - Publication item query should be examined to ensure it is optimal. As with optimising any SQL, full table scans that produce a small %age of the rows in the table should be avoided and the normal techniques to solve these issues such as adding extra indices should be employed.

Once the base SQL has been optimised, re-run the Consperf utility. This time examine the TOUTFILE. This will contain timings for all SQL operations involved in replication. The report contains 2 main sections, the Sync Output section and the MGP Output section.

The Sync Output section shows timings for the SQL that is involved in the simulated synchronisation run for each publication item. Columns enclosed with <> show the SQL that will be used by Oracle Database Lite for replication. Other columns are alternatives that could be used. The Total column shows in


milliseconds, the total time taken to do the operation. Columns with a large negative number are queries that did not complete within the specified timeout period. The NS,BS, & AS columns show the time spent executing any registered customisation code.

The MGP Output section shows timings for the SQL that is involved in the MGP processing phase. Again columns enclosed with <> show the SQL that will be used by Oracle Database Lite for replication. Other columns are alternatives that could be used. The Total column shows in milliseconds, the total time taken to do the operation. Columns with a large negative number are queries that did not complete within the specified timeout period. The BC & AC columns show the time spent executing any registered customisation code.

Queries that could benefit from Autotuning are ones where the timing of the chosen (i.e. <> identified query) pattern of SQL within the categories mentioned above, is greater than an alternative one. Obviously queries with the highest timing difference between identified and alternative will provide the best tuning advantage.

Looking at the MGP output in Figure 6 one can see that the MGP is not using optimal queries. LDEL_3 has a timing of 6800 whilst LDEL_1 only has a timing of 4246 that is obviously less. Similarly LINS_1 has a timing of 4626 whilst LINS_3 only has a timing of 1913.

Consperf can now be run in AUTOTUNE mode. To do this we run Consperf with the same parameters as before and add the AUTOTUNE and TOLERANCE parameters:

CONSPERF MOBILESCHEMA=<mobile server schema> MOBILEPASSWD=<mobile server password> SERVICENAME=<tns service name> PUBLICATION=<publication name> CLIENTID=<client identifier> TOUTFILE=<timing output filename> POUTFILE=<explain plan output filename> TIMEOUT=<query timeout value in seconds> ORDERBYPUBITEM (order output by publication item name) AUTOTUNE TOLERANCE=<a time in seconds>

The tolerance parameter means that if a particular query takes less time by tolerance than another query it will be picked as the new default SQL template. In our example we are only seeing a difference of 2-3 seconds in the timings so we set the tolerance to 1 and run Consperf, it will pick LDEL_1 and LINS_3 as the tuned templates.


Figure 6 - Timings Output


Application Deployment Oracle Database Lite can assist with application deployment. This section concentrates on the deployment of non web based applications (See below for automatic deployment of web based applications – Web-To-Go). The Oracle Database Lite infrastructure provides a number of facilities to assist with application deployment.

• Automatic replication of application files

• Customisable setup utility

• Offline Instantiation

Automatic replication of Application Files

Using the packaging wizard or deployment API’s, application files can be registered with the repository. These are then replicated to the client in the same directory as the client database. Oracle Database Lite automatically keeps these files up-to-date. The files must be placed in port specific directories under the local application directory of the application. For instance for Windows 32 the files would be stored under the directory win32.

Customisable setup utility

By default all the setup files for Oracle Database Lite clients are held under ORACLE_HOME/Mobile/Server/Repository/WebToGo/Setup. The setup utility consists of platform specific setup executables and ini files. On ports that support it the ini file can be edited to perform additional setup functions such as copying extra files or running extra setup programs. Some of these ini files are:

• setup.ini – Mobile Client for Web-To-Go setup

• win32.ini – Mobile Client for Win32 setup

• palm.ini – Mobile Client for PalmOS setup

• cesetup.pocket_pc.sh3.ini – Mobile Client for Pocket PC (SH3 processor) setup.

Offline instantiation

Offline instantiation is a new feature of Oracle Database Lite. It allows administrators to build an installable image that includes the database so that it can be supplied to the use via a CD. This avoids the problem of the long first synchronisation phase. Only changes that are made to replicated data between instantiation and installation will be replicated on first sync.

The instructions for offline instantiation are currently supplied in <ORACLE_HOME>/mobile/sdk/consolidator/oli.. OLI is currently documented for Win32 and WinCE.

Customisation of Replication The Oracle Database Lite infrastructure is very flexible and provides a number of different entry points for customisation. The Oracle Database Lite Developers


Guide for Windows 32 covers most of these so this white paper will just describe each one in turn and provide hints and tips where relevant.

Client Synchronisation Customisation

On all platforms an mSync executable is provided to enable the client to synchronise with the server. For Web based applications, synchronisation is provided as part of the workspace. Oracle Database Lite includes a set of API’s to enable the developer to build a custom client synchronisation program.

These programs can be written in Java (on some platforms) [J], VB (and other COM aware languages – on Windows) [O] or C/C++[C]. All three programming paradigms use a structure to specify synchronisation options. The C interface has some additional calls to set certain parameters.

Client Synchronisation Parameters(default as implemented by mSync)

Parameter Default Platform Description

AppRoot [Mobile Client] home

C Allows the developer to specify where the application is stored differently from the default.

clientDBMode EMBEDDED JOC If EMBEDDED uses standard ODBC driver. If CLIENT uses branch office ODBC driver.

cookie NULL JOC HTTP cookie used during transport.

fullrefresh N/A (flag on dialog)

JO Full refresh is forced

NewPassword Password entered and Change flag set

C New password for user.

noapps (SyncNewPublications)

False JOC Do not sync new publications during this synchronisation run.

Option Flags All False C OCAPI_OPT_SENDMETADATA Sends meta-info to the server. OCAPI_OPT_DEBUG Enables debugging messages. OCAPI_OPT_DEBUG_F Saves all bytes sent and received for debugging. OCAPI_OPT_NOCOMP Disables compression. OCAPI_OPT_ABORT If set, OCAPI will try to abort the current sync session. OCAPI_OPT_FULLREFRESH



Forces OCAPI to purge all existing data and do a full refresh.

password Entered Password

JOC Password of synchronising user.

Priority false C If set to true only high priority snapshots are synchronised (see AddPublicationItem)

proxy Entered Proxy JOC HTTP proxy to use

proxyPort Entered Port JOC HTTP proxy port

pushonly False JOC If true, only upload changes from client.

reset N/A JO Reset parameters

SavePassword Entered Save Password flag

C If true, saves the password in the users profile (JO always set to true)

secure(ity) Entered Secure flag

JOC If true, Use HTTPS

server Entered URL JOC Address of Mobile Server

sync flag True C Allows the developer to selectively decide which table is synchronised with the server (1 means true).

SyncApps True C Allows the developer to decided if applications are synchronised during processing or not

syncDirection Send and Receive JOC SendOnly, ReceiveOnly

SyncTimeLog Obtained from ocapi.ini

C If true the time of synchronisation is written to the conscli database.

publication name & table name

N/A C The snapshot to be synchronised or not.

tableFlag N/A JO tableFlag[publication.item] set to “ENABLE” or “DISABLE”. Determines whether a snapshot is to be synchronised or not (equivalent of C sync flag).

TotalReceiveDataL N/A C Total bytes received by client



en sync engine.

TotalSendDataLen N/A C Total bytes sent by client sync engine

transportDriver/transportType

HTTP JO HTTP

TrType HTTP C Determines if sync will be done using HTTP or a custom transport (see below).

UpdateLog Obtained from ocapi.ini

C If true writes update information to the C$UPDATE_LOG table of the conscli database.

user(name) Entered Username

JOC Username of synchronising user.

Transport Customisation

The default transport used for synchronisation is HTTP (or HTTPS). If the synchronisation program is written in C/C++, it is possible to specify the transport used during synchronisation. Basically, the developer needs to build a shared library (DLL or .so) that exposes 4 methods with the following signatures:

connectProc (ocEnv) ocEnv – see below disconnectProc (ocEnv) ocEnv – see below sendProc (ocEnv, data, len) ocEnv – see below data – Data to be sent len – Length of data to be sent receiveProc (ocEnv, data, len) ocEnv – see below data – Data to be received len – Length of data to be received The above methods are registered with the client synchronisation engine. This calls:

1. The connectProc before commencing any work.

2. During the send phase, the sendProc is called with buffers of data of length len to be sent. It is upto the sendProc to do perform the transport.

3. During the receive phase, the receiveProc is called and should return buffers of data of length len.


4. The disconnectProc is called after the receive phase has completed.

An example of using the customized transport is given in ORACLE_HOME/Mobile/Sdk/Examples/Consolidator/opentransport. This example shows how a floppy disk can be used for replication. The replication is done in three phases:

1. A SendOnly synchronisation is performed. This performs only the send phase and thus the sendProc is called. This writes the data to be sent to the server to a floppy disk.

2. The floppy disc is sent to the server and a java program is used to read the floppy disk, write the data to the mobile server, receive the data from the mobile server and write it to the floppy disk

a. The java uses the undocumented HeliosTransport interface that it implements. This interface is that used by the mobile server during synchronisation. It has similar methods to that of the client synchronisation engine as:

i. send(data, len)

ii. recv(data,offset)

A HelioSession is created which firstly calls the recv method which reads the data from the floppy into the buffer supplied and once all the data has been read and the mobile server has processed it, it prepares the send data from the outqueues and calls the send method passing the data in a buffer.

b. An alternative to this would be to use the HTTP interface to the mobile server directly. You do this by writing the send data to the Web-To-Go URL and getting the receive data in response.

3. The floppy disk is then returned to the client and a ReceiveOnly synchronisation performed. This performs only the receive phase and thus the receiveProc is called. This reads the data from the floppy disk and supplies it back to the client sync engine to apply locally.

Snapshot Replication Customisation

Snapshots are normally based on the base tables of a schema in the same database as the mobile repository. However, this is not a requirement. Snapshots can be based on views, remote tables, and even an abstract queue object. Also, the MGP has a number of different exit points for customisation.

Snapshots based on views

The documentation3 covers the creation of a snapshot against a view. You are required to use the Java Consolidator API (in 5.0.2.8 of Oracle Database Lite, the packaging wizard supports this functionality).

For a snapshot on a view to be updateable, at least one of the tables that make up the view must have a unique primary key. For instance the view:

3 Oracle9i Lite Developer's Guide for Windows 32 Chapter 5.4.5


SELECT E.EMPNO, E.ENAME, D.DEPTNO, D.LOCATION FROM EMP E, DEPT D WHERE D.DEPTNO=E.DEPTNO

Contains 2 primary keys (EMPNO and DEPTNO) but only one of them, EMPNO is unique within the view. Therefore EMP would be identified as the snapshots parent table.

The view: SELECT E.EMPNO, E.ENAME, D.DEPTNO, D.LOCATION FROM EMP E, DEPT D

Contains 2 primary keys but neither of them are unique in the view so it cannot be updateable.

For a snapshot on a view to be fast refreshable:

• Each of the view's base tables must have a primary key.

• All primary keys from all base tables must be included in the view's column list.

• If the item is a view, and the item predicate involves multiple tables, then all tables contained in the predicate definition must have primary keys and must have corresponding publication items.

Because Oracle does not automatically allow a complex view to be updated, either INSTEAD OF triggers must be coded against the view to perform the updates to the underlying tables or DML callouts must be used (see later in this document).

The process of building a snapshot on a view is:

• Create the view.

• Publish a snapshot against the view (using WTGPACK or the API).

o Consolidator.CreatePublicationItem4(“EMPVIEW”,”SCOTT”,”EMPVIEW”,”F”,”SELECT * FROM SCOTT.EMPVIEW”,NULL,NULL);

• Write a java program to use the API (see API template program Appendix A).

o Consolidator.ParentHint(“SCOTT”,”EMPVIEW”,”SCOTT”,”EMP”);

o Consolidator.PrimaryKeyHint(“EMPVIEW”,”EMPNO”,”SCOTT”,”EMP”,”EMPNO”);

o Consolidator.PrimaryKeyHint(“EMPVIEW”,”DEPTNO”,”SCOTT”,”DEPT”,”DEPTNO”);

Snapshots based on remote tables

The documentation5 also covers the creation of a snapshot against a remote table. You are required to use the Java Consolidator API.

4 See Oracle 9i Lite Consolidator Admin API Specification


• Firstly create local synonyms for the remote table(s) in the mobile server repository (or another user), using a database link

o CREATE DATABASE LINK6 remotedb ……

o CREATE SYNONYM remotetable on remotetable@remotedb

• Execute the script <ORACLE_HOME>\Mobile\server\admin\consolidator_rmt.sql on the remote schema.

• The snapshot must be published using the Consolidator API so write a java program to use the API (see API template program Appendix A).

o Consolidator.CreatePublicationItem(“jdbc:oracle:thin:@remotehost:1521:remotesid”,“EMPVIEW”,”MOBILEADMIN”,”EMPVIEW”,”F”,”SELECT * FROM MOBILEADMIN.EMPVIEW”,NULL,NULL);

Queue based publication items

As has been documented in the Oracle Database Lite Replication section, default replication is based on a snapshot of one or more base table(s), an inqueue and an outqueue. It is possible to create a publication item that does not actually have any base tables associated with it, just an inqueue and an outqueue. All handling of processing data from the inqueue to any base tables and processing data from any base tables to the outqueue is the responsibility of the application developer. This approach gives the application developer complete control over what data is replicated to the client and how this data is handled for instances when this is a requirement. How to do this is covered in the documentation7. You are required to use the Java Consolidator API.

A Queue based publication item is created using the CreateQueuePublicationItem API. A base table needs to be referenced so that mobile server can generate an inqueue of the correct format. The API creates the inqueue automatically. The application needs to create the outqueue.

• Write a java program to use the API (see API template program Appendix A)

o CreateQueuePublicationItem(“MYQUEUE”, ”SCOTT”, ”QUEUETABLE”, ”SELECT ID, DESCRIPTION FROM QUEUETABLE”, null, null, null);

o This will publish the item and create an inqueue

• Create the outqueue

5 Oracle9i Lite Developer's Guide for Windows 32 Chapter 5.3.5 6 See Oracle SQL manual 7 Oracle9i Lite Developer's Guide for Windows 32 Chapter 5.4.9


o CREATE TABLE CTM$+publication name ( CLID$$CS VARCHAR2 (30), ID NUMBER, DESCRIPTION VARCHAR2(2000), TRANID$$ NUMBER (10), DMLTYPE$$ CHAR (1) CHECK (DMLTYPE$$ IN ('I','U','D'), )

o Note this is a true out queue and not a map table as well because is does not have a version column on it. It is therefore called CTM (To Mobile).

• When using Queue based publication items the tables, C$INQ and C$PUB_LIST_Q are important. The C$INQ table lists all queue based publication items that have received rows from clients. The C$PUB_LIST_Q table lists all queue based publication items that are going to be replicated to clients. This table is manipulated by the queue processing logic.

• Queue processing logic is built in a PL/SQL package. The Mobile Server calls this package during queue publication item processing. The package would normally reside in the MOBILEADMIN schema but can reside anywhere.

o CREATE OR REPLACE PACKAGE CONS_QPKG AS /* * Called once a client has uploaded some data * The C$INQ table can then be examined to see which * inqueues have received data. */ PROCEDURE UPLOAD_COMPLETE( CLIENTID IN VARCHAR2, TRAN_ID IN NUMBER -- IN queue tranid ); /* * Called before the client downloads data * The outqueue should be populated with all new data and * the tran id set to CURR_TRAN. The LAST_TRAN * parameter can be used to determine what was the last data * that the client thinks it has downloaded. The * HIGH_PRTY parameter means the client is doing a high * priority sync. */ PROCEDURE DOWNLOAD_INIT( CLIENTID IN VARCHAR2, LAST_TRAN IN NUMBER, CURR_TRAN IN NUMBER, HIGH_PRTY IN VARCHAR2


); /* * Called after the client has completed the download */ PROCEDURE DOWNLOAD_COMPLETE( CLIENTID IN VARCHAR2 ); END CONS_QPKG;

o The package needs registering with the repository

Consolidator.RegisterQueuePkg(“MOBILEADMIN.CONS_QPKG");

Customising DML

After creating a publication item (either via WTGPACK or the API), it is possible to register a DML package or procedure against the item. This procedure is used to perform DML operations against the item instead of using the standard MGP functionality. The procedure has a standard signature and one needs to be built per publication item. You are required to use the Java Consolidator API.

• anyname (DML, /* Indicates the DML operation to perform ‘D’ for Delete, ‘I’ for Insert or ‘U’ for Update */ COL1..COLN, /* Each column as defined in the publication item */ PK1..PKN /* List of Primary Key columns defined in the publication (in the order they appear in the base or parent table */ );

• CreatePublicationItem(“EMP”, “SCOTT”, “EMP”, “F”, “SELECT EMPNO,ENAME, SAL FROM EMP”, null, null);

• CREATE OR REPLACE PACKAGE SCOTT.EMP_DMLPKG AS PROCEDURE EMP_DML_DRIVER(DML CHAR(1), P_EMPNO SCOTT.EMPNO%TYPE, P_ENAME SCOTT.ENAME%TYPE, P_SQL SCOTT.SAL%TYPE, P_OLD SCOTT.EMPNO%TYPE); PROCEDURE EMP_INSERT(…); PROCEDURE EMP_DELETE(…);


PROCEDURE EMP_UPDATE(…); END EMP_DMLPKG;

• Consolidator.AddMobileDMLProcedure("MYPUBLICATION", "EMP", “SCOTT.EMP_DMLPKG.EMP_DML_DRIVER");

Using Before and After Apply procedures

When creating a publication item (either via WTGPACK or the API), it is possible to register a callback package against the item. The procedures in this package are called by the MGP before the apply cycle, after the apply cycle, during the apply cycle before and after a transaction being applied, before the compose cycle and after the compose cycle. The procedures can be used to do things such as disabling/enabling triggers and/or constraints, custom auditing or distribution of replicated data, or customising the data applied or composed. The package has a standard signature and one can service more than one per publication item if required. You are required to use the Java Consolidator API.

• create or replace package SCOTT.CustomTriggers as procedure BeforeApply(clientname varchar2); procedure AfterApply(clientname varchar2); procedure BeforeTranApply(tranid number); procedure AfterTranApply(tranid number); procedure BeforeCompose(clientname varchar2); procedure AfterCompose(clientname varchar2); end CustomTriggers;

• CreatePublicationItem(“EMP”, “SCOTT”, “EMP”, “F”, “SELECT EMPNO,ENAME, SAL FROM EMP”, “SCOTT”, “CUSTOMTRIGGERS”);

Using MyCompose methods

During the compose phase of the MGP a number of java methods are called to populate the clients outqueues. It is possible to override these methods using Java. The documentation8 covers the use of these methods. The java consolidator API is used to register your custom java class.

Publication Customisation

When using the Web-To-Go option or if you wish to administer your publications through the web interface, you are required to create a Web-To-Go Application. This can done automatically for you by WTGPACK. As an alternative, WTGPACK can create a Jar file for you, which can be used to create an application

8 8 Oracle9i Lite Developer's Guide for Windows 32 Chapter 5.4.1


via the web interface or via the Web-To-Go API ResourceManager.publishApplication method.

The Jar file can be created using the jar program. The jar file contains a web.xml file in its root and any application files in their respective directories (jsp are normally stored in the root directory of the application). The web.xml file describes the application and has the format:

<workspace> <application name="Application Name"> <app_info> <directory>Application Directory</directory> <virtualpath>Application VirtualPath</virtualpath> <defaultpage>Application Default Page</defaultpage> <icon>Application Icon File</icon> -can be <icon/> <description>Application Description</description> <localroot>Application Root Directory</localroot> </app_info> <app_db connection="Number of Connections" sharing="Y/N"> <dbuser>Application Database User</dbuser> <dbpwd/> </app_db> <role name="Application Roles"/> <servlet name="Any Application Servlets" virtualpath="Servlet Virtual Path"/> <snapshot name="Application Snapshots" owner="Snapshot Owner" order="Snapshot Weight"> <snaplatform platform="Snapshot Platform" template="Snapshot Select Statement" updatable="Y/N" conflict="server/client" refresh="fast/full" dml_procedure=""/> </snapshot> </application> </workspace> If you are not using Web-To-Go and do not wish to use the web interface to manage your publications, there is not a requirement to create a Web-To-Go application. However the WTGPACK program will always create a Web-To-Go application. To use Oracle Database Lite without creating applications, it is necessary to use the Java Consolidator API.

If you wish to use a Web-To-Go Application but also wish to use the Java Consolidator API to add new publication items. The first step is to create your Web-To-Go application. There is not an API call to do this so you will need to use the publishApplication API described above. For the Web-To-Go application to be useable it needs to contain at least 1 snapshot. This can be a dummy snapshot that can be removed later.


1. Create your web.xml file

<workspace> <application name="Example Application"> <app_info> <directory>.</directory> <virtualpath>/Example</virtualpath> <defaultpage>index.html</defaultpage> <icon/> <description>Example Application to be manipulated via API</description> <localroot>.</localroot> </app_info> <app_db connection="10" sharing="N"> <dbuser>scott</dbuser> <dbpwd/> </app_db> <role name="DEFAULT ROLE"/> <snapshot name="DUMMY" owner="SCOTT" order="1"> <snaplatform platform="WIN32" template="SELECT X FROM SCOTT.DUMMY" updatable="N" conflict="SERVER" refresh="FAST" dml_procedure=""/> </snapshot> </application> </workspace>

2. Create your Jar file jar –cvf myapp.jar web.xml

3. Publish your application ResourceManager.publishApplication9(new File(“web.xml”), “Administrator”, “admin”, “http://mymobileserver.us.oracle.com/Web-To-Go”, false, true);

This creates your application in the repository, generates a publication for the application and links the two together and then adds the dummy publication item.

4. Create a publication item Consolidator.CreatePublicationItem10(“MYITEM”,”SCOTT”,”EMP”,”F”,”SELECT * FROM SCOTT.EMP”,NULL,NULL);

5. Retrieve the applications publication String pn=ResourceManager.getPublicationName9(“/Example”, “WIN32”);

6. Add the publication item to the publication Consolidator.AddPublicationItem(pn, "MYITEM", null, null, "S");

Administration Customisation

All the functions of the GUI Control Center can be performed via the oracle.mobile.admin.ResourceManager API. This enables the developers to build their own admin interface.

9 See Oracle 9i Lite WebToGo oracle.mobile.admin API Specification 10 See Oracle 9i Lite Consolidator Admin API Specification


MOBILE SERVER As documented above, the mobile server is used to support replication between client and server. The mobile server also acts as a servlet execution engine, which supports all administration tasks required to support client replication as well as providing an environment for executing web based applications. Multiple mobile servers (on multiple machines) can be used to increase scalability. A load balancer will be required to split requests between servers. Whilst mobile server can act as a http server in its own right, it is recommended that 9iAS be used to increase

scalability and availability. Currently, native Apache can also be used as the http server.

WEB-TO-GO The Oracle Database Lite product on Windows 32 allows the developer to build a web based application that provides seamless online/offline functionality. Based on the functionality documented above, Web-To-Go provides a lightweight http server and servlet execution engine to execute on the client. This is called the client Mobile Server (and is the WebToGo.exe11). Web-To-Go also extends the standard application replication functionality of Oracle Database Lite to provide a sophisticated application deployment infrastructure. Development of Web-To-Go applications is well described in the Web-To-Go Developers guide. So this paper will concentrate on some of the components of a Web-To-Go system in more detail.

11 When using the WebToGo.exe as a web server, the use of LocalHost is not supported. you must either use the full machine name or a fixed IP address (which could be provided by the Microsoft loopback adaptor.

Figure 7 - Mobile Server configuration

ClientMobile Server

Mobile Server

9iASApache

9iASApache

LOAD BALANCER

Client

Database


Repository The Web-To-Go repository is used to store both the administration application and any deployed application. The repository is split into 2 parts metadata and application files. The metadata is always stored in the database. By default the application files are stored on the file system of the Mobile Server12. They are located under ORACLE_HOME/Mobile/Server/Repository. Applications are then stored below this directory according to their virtual path. When the Mobile Server starts it loads up what is called a workspace. This is basically a portal that provides links to applications that can be run. The workspace is identified by the Web-To-Go.ora parameter file parameters:

CUSTOM_WORKSPACE=YES [Default: NO] CUSTOM_DIRECTORY=myworkspace [Default: Web-To-Go] DEFAULT_PAGE=index.html [Default: index.htm] CUSTOM_FIRSTSERVLET=myservlet [Default: WLTop ]

As can be seen, by default Mobile Server mounts the /Web-To-Go directory under the Repository directory as its home page. This provides the initial logon for the user.

12 The repository can also be stored in an Oracle Server database or an Oracle Database Lite database. For information on the Oracle Database Lite database see WebToGo database below

Figure 8 - Web-To-Go

Client Client Database

WebToGo Database

WebToGo Server (webtogo.exe)

Mobile Server (webtogo.exe)

Repository

DB


The Mobile Server identifies which applications are available for a particular user after logon from the database. The tables involved are:

APPLICATIONS Every application published to the mobile server is registered in this table. The information includes the application name, the application directory (under the Repository) and the application virtual path.

USERS List of defined users.

ROLES List of defined roles

APP_ROL Roles applicable to a particular application

APP_ROL_USR Links the users to the applications.

Client Database In Web-To-Go the database that contains the data is a regular replicated Oracle Lite database. Normally when using Oracle Lite databases, the SYSTEM user is defined without any password so any password can be used in the connect string. With a Web-To-Go client database the password is set to that of the Web-To-Go user that replicated the database.

Web-To-Go Database When a Web-To-Go application goes offline, some application metadata and the application files (html, servlets etc) that make up the application are replicated to the client machine in the Web-To-Go.odb file (in a special file system structure). The client mobile server, mounts this database as a file system. This virtual file system can be examined by shutting down the client mobile server and using the WSH utility.

wsh –L system/x@Web-To-Go.


The WSH utility has various commands to allow one to manipulate the virtual file system.

Workspace Customisation As has been previously discussed the Mobile Server uses the workspace identified by the CUSTOM_DIRECTORY parameter or /Web-To-Go if CUSTOM_WORKSPACE=NO. The Web-To-Go developers guide documents how to build a custom workspace. The MDK comes with a demo custom workspace upon which you can base your own customized workspace. This is located in ORACLE_HOME/Mobile/sdk/wtgsdk/myworkspace. The crserver scripts assume that you are running the MDK on the same server as the mobile server. If you are not, you will need to copy the custom workspace to the machine upon which the mobile server is located. By default users navigate to the setup servlet (/Web-To-Go/setup). This servlet queries the BUNDLES table so this can be used to customize what setup options the user sees.

Figure 9 - WSH Utility


APPENDIX A – API TEMPLATE PROGRAM package oracle.rj; import java.sql.SQLException; import java.sql.*; import oracle.sql.*; import oracle.lite.sync.Consolidator; import java.util.*; import java.io.*; public class ConsolidatorTemplate { static String CONS_SCHEMA = null; static String DEFAULT_PASSWORD = null; static String USERNAME; static String CONNSTR; static long TRANSACTIONID; public static void main(String argv[]) throws Throwable { if(argv.length != 3) { System.out.println("Usage java ConsolidatorTemplate <Schema> <Passwd> <JDBC URL> "); return; } CONS_SCHEMA = argv[0] ; DEFAULT_PASSWORD = argv[1] ; CONNSTR = argv[2] ; // Open a logger // All output to System.out is suppressed once the connection is opened FileWriter fw = new FileWriter("c:\\temp\\tf.txt"); // Establish Connection oracle.mobile.admin.ResourceManager.openConnection(CONS_SCHEMA, DEFAULT_PASSWORD, CONNSTR); // Insert in here any Consolidator or ResourceManager API calls such as // Consolidator.CreatePublicationItem (“MYITEM”, // ”SCOTT”, // ”EMP”, // ”F”, // ”SELECT * FROM SCOTT.EMP”, // NULL, // NULL); // Commit Transaction oracle.mobile.admin.ResourceManager.commitTransaction(); // Close Connection oracle.mobile.admin.ResourceManager.closeConnection(); fw.close(); } }

Oracle Database Lite InDepth December 2003 Author: Richard Jacobs Contributing Authors: Boris.Berdichevskiy, Johnathan Haulund, Ravindra Prakash, Sharad Singani, Martin Graf, Philip Stephenson Oracle Corporation World Headquarters 500 Oracle Parkway Redwood Shores, CA 94065 U.S.A. Worldwide Inquiries: Phone: +1.650.506.7000 Fax: +1.650.506.7200 www.oracle.com Copyright © 2003, Oracle. All rights reserved. This document is provided for information purposes only and the contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. We specifically disclaim any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without our prior written permission. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners.

oracle database lite indepth · baker collins kelly roberts young . oracle database lite indepth...

Documents