spectrum data federation guide - pitney...

Spectrum™ Technology PlatformVersion 10.0

Spectrum™ Data Federation Guide

1 - Introduction

Data Federation 5What is Spectrum™ Data Federation? 5Core Components - Spectrum™ Data

Federation 6

2 - Workflow - Spectrum™

DataFederation

Workflow - Spectrum™ Data Federation 9

3 - Establishing Data SourceConnections

Data Source Connections 12

4 - Designing Baseviews

Introduction to Baseview 21Creating a Baseview 21Modifying a Baseview 22Schema - Baseview 23Deleting a Baseview 25

5 - Designing Metaviews

Introduction to Metaview 27Creating a Metaview 27Deleting a Metaview 37

6 - Designing Virtual DataSources

Virtual Data Source 39Creating a Virtual Data Source 39Modifying a Virtual Data Source 40Deploying or Undeploying a Virtual Data

Source 40Previewing a Virtual Data Source 41Deleting a Virtual Data Source 41

7 - Common Features

Lineage 43

8 - Using the AdministrationUtility

Getting Started with the Administration Utility 45Using a Script with the Administration Utility 46Categories of Commands 46System 48

9 - Conventions

Naming Conventions 51Query Conventions 51

10 - Configurations

JDBC Driver Configuration 53Required JAR Files 55

Table of Contents

11 - Limitations

Common Limitations 58Siebel Connection Limitations 58Write Feature Limitations 59

Appendices

Appendix A:Reserved Keywords 61Appendix B:Supported Datatypes for Conversion

Transformations 66

3Spectrum™ Technology Platform 10.0 Spectrum™ Data Federation Guide

1 - Introduction

In this section

Data Federation 5What is Spectrum™ Data Federation? 5Core Components - Spectrum™ Data Federation 6

Data Federation

Data Virtualization is a data management approach that refers to the virtual abstraction of dataderived from diverse data sources, from the data consumer. This enables you to utilize the virtualdata irrespective of its technical details vis-a-vis its physical location, format, storage technology,API used, and other such details.

Data Federation, an aspect of data virtualization, is the aggregation of data derived from multipleheterogeneous and autonomous data sources, like databases, flat files, web services, and ERM/CRMapplications, which a consumer application can view as one integrated data source using a singlepoint of access.

Data Federation provides:

1. A uniform user interface to access multiple heterogeneous data sources providing a standardizedand simplified access to aggregated data

2. Increased data access speed on a real time basis3. Reduced system workload as the data is virtually rearranged as opposed to physical

rearrangement4. Autonomy of the multiple heterogeneous data sources, vis-a-vis:

• Design: Varied designs for data storage and usage on the different heterogeneous data sources• Communication: Various communication channels between the multiple data sources and thedata consumer.

• Execution: Autonomous execution of commands as applicable on the different data sources• Association: Independence of the data sources to associate and disassociate from the federationat any given time

5. Significantly reduced development and support time

What is Spectrum™ Data Federation?

The Spectrum™ Data Federation Module allows you to create an abstracted virtual data sourceproviding a unified access to diverse heterogeneous data sources through a uniform user interface.Using the module, you can collate the data from multiple data sources and organize it as a specificrepresentation of your business model.

The Spectrum™ Technology Platform is placed between one or more data sources and your ownapplication. It connects to the data sources and collates the data from them for access by yourapplication. Spectrum™ Data Federation creates a federated view of this collated data, providing aunified business-relevant view of the entire federated data.


Introduction

The Spectrum™ Data Federation Module provides the below key features:

1. Virtualization: The module provides access to an integrated virtual data source.2. Federation: The module aggregates the data from the multiple data sources into one virtual data

source. This allows you to access them as one data source entity.3. Abstraction: The module hides the technical complexities of the multiple data sources, allowing

access to the virtually aggregated data without engaging in any structural, syntactical orcommunication related technical details. Standard JDBC protocol and ANSI SQL database querysyntax is used to access the aggregated virtual data source.

4. Data Presentation: Themodule presents the retrieved data in a user-specified format, irrespectiveof the structural and syntactical details of the exact data source from which the data has beenderived.

Figure 1: Overview - Spectrum™ Data Federation

Core Components - Spectrum™ Data Federation

The key components of Spectrum™ Data Federation are Data Source Connections, Baseviews,Metaviews, and Virtual Data Sources.

Allows you to connect to and access any number of autonomous datasources simultaneously from within the Spectrum™ Data FederationModule.

Data SourceConnection

Provides a uniform representation of the underlying data derived from allthe configured Data Source Connections.

Baseview

A logical representation of specific business entities created by collatingchunks of data derived from one or more Baseviews.

Metaview


Introduction

Provides a logical data source to group the user-defined business entitiesor Metaviews generated. Multiple Virtual Data Sources can be created,and deployed as needed for your business requirements.

Virtual Data Source

Key Features - MetaviewThe Metaview provides powerful features like Transformations and Criteria Builder to createlogical business entities.

A set of pre-defined conversions which can be applied on the Baseview data,in order to generate a business-specific data representation in the Metaview.

Transformations

You can select from a range of pre-defined mathematical, logical, constants,range-defining, and similar functions to apply on the Baseview data to generatea business-specific Metaview.

A database query builder to help you place required filters and clauses onthe Baseview data to create the required Metaview.

Criteria Builder


Introduction

2 - Workflow -Spectrum™ DataFederation

In this section

Workflow - Spectrum™ Data Federation 9

Workflow - Spectrum™ Data Federation

The Spectrum™ Data Federation Module provides you an interface to create a Virtual Data Sourceby modelling data federated from multiple sources into business specific entities. It establishesmultiple Data Source Connections and creates a Baseview to achieve a uniform representation ofthe entire available data. This Baseview in turn is used to create Metaviews which represent relevantbusiness entities. The Metaviews are organized into Virtual Data Sources to help you view and usethe unified data in a business-relevant model.

The following diagram illustrates the workflow of the Data Federation Module.

Figure 2: Workflow - Spectrum™ Data Federation

1. Create a Data Source Connection for each of the individual data sources you wish to collatedata from.

To create a JDBC connection, use the Management Console. To create a non-JDBCconnection, use the Spectrum™ Data Federation web interface.

Note:



2. Create a Baseview, which will provide a uniform representation of the data collated from themultiple data sources.

3. Create a Metaview by generating required logical business entities, deriving data from one ormore Baseviews. Use the Transformations andCriteria Builder features to generate the requiredbusiness specific views of the source data in the Metaview.

4. Create a Virtual Data Source (VDS) by collating the created business-specific Metaviews. Thiswould serve as the virtual database to be used by your client application.

5. Deploy the created Virtual Data Source.6. Use the deployed Virtual Data Source. A Virtual Data Source can be accessed:

• From within Spectrum™ Technology Platform

1. Define a JDBC connection using the Management Console.2. Use this JDBC connection in the Enterprise Designer, in the DB stages.

• From outside Spectrum™ Technology Platform

This can be done by loading the JDBC driver, setting up the required connection(s) and executingANSI SQL queries to access the data.



3 - Establishing DataSource Connections

In this section

Data Source Connections 12

Data Source Connections

The Spectrum™ Data Federation Module lets you establish connections to various types of datasources. The module currently supports connections to Amazon SimpleDB, Apache Cassandra,Flat File, salesforce, and Siebel data sources.

The Data Federation Module provides specific screens to set up each type of data source. Thesescreens let you enter your authentication credentials and other necessary details to connect to thedesired data source.

JDBC Connection

Using a JDBC ConnectionTo use a JDBC connection, open Management Console.

In Management Console, you can configure a JDBC driver and then create a JDBC connectionusing the driver.

For more information, refer the Connecting to Data section of the Administration Guide.

Amazon SimpleDB Connection

Adding an Amazon SimpleDB ConnectionTo add a new Amazon SimpleDB connection navigate to Connections > Amazon SimpleDB, andenter the AWS Security Credentials to authenticate your access to the required Amazon AWSaccount and the SimpleDB data source.

1. Go to Connections > Amazon SimpleDB.

2. By default, the screen opens in the create mode. Otherwise, click to add a new AmazonSimpleDB connection.

3. Enter a Connection Name for the Amazon SimpleDB connection.4. In the Access Key ID field, enter the 20-character alpha-numeric sequence provided to you to

access the Amazon AWS account.5. In the Secret Access Key field, enter the confidential 40-character key needed to authenticate

the connection.6. Click Save.

A message confirms the successful creation of the connection.

To modify an Amazon SimpleDB connection select the required connection, modify thedetails as required, and click Save.

Note:


Establishing Data Source Connections

Deleting an Amazon SimpleDB ConnectionTo delete an Amazon SimpleDB connection navigate to Connections > Amazon SimpleDB, selectthe particular connection, and click Delete.

1. Go to Connections > Amazon SimpleDB.2. Select the Amazon SimpleDB connection to be deleted.3. Click Delete.

If the connection is in use you will be notified which Baseviews, Metaviews or Virtual Data Sourcesare using it, allowing you to either choose to abort the deletion or proceed anyway.

If you choose to proceed, a message confirms the successful deletion.

Apache Cassandra Connection

Adding an Apache Cassandra ConnectionTo add a new Apache Cassandra connection navigate to Connections > Apache Cassandra, andenter the authentication details for access to the required Apache Cassandra data source.

1. Go to Connections > Apache Cassandra.

2. By default, the screen opens in the create mode. Otherwise, click to add a new ApacheCassandra connection.

3. Enter a Connection Name for the Apache Cassandra connection.4. In the Host field, enter the machine name or the IP on which the Apache Cassandra data source

is installed.5. In the Keyspace field, enter the name of the keyspace of the data center you wish to access.6. Enter the Port on which the Apache Cassandra data source is configured.7. Click Show advanced (optional) settings... to enter further connection details. Else, directly go

to Step 12.8. Enter the Username provided for the particular Apache Cassandra connection.9. Enter the Password to authenticate the access.10. In the Consistency Level, select the required success criteria for data reads and writes. This

refers to the required number of replica nodes to be hit for a successful data transaction. Thiscan be at least one, or all, or a combination of available nodes.

11. In the Fetch Size, enter the number of resultset rows you wish to fetch on each read transaction.12. Click Save.

A message confirms the successful creation of the connection.

To modify an Apache Cassandra connection select the required connection, modify thedetails as required, and click Save.

Note:



Deleting an Apache Cassandra ConnectionTo delete an Apache Cassandra connection navigate toConnections >Apache Cassandra, selectthe particular connection, and click Delete.

1. Go to Connections > Apache Cassandra.2. Select the Apache Cassandra connection to be deleted.3. Click Delete.



Flat File Connection

Adding a Flat File ConnectionTo add a new Flat File connection navigate to Connections > Flat File, and enter the file's accessdetails and content type details to allow the Data Federation Module to read it correctly.

You can create either a Delimited or Fixed Width Flat File Connection.

To modify a Flat File connection select the required connection, modify the details as required,and click Save.

Note:

Adding a Delimited Flat File ConnectionTo add a new Delimited Flat File connection navigate to Connections > Flat File, and select theRecord Type as Delimited. Enter the file's access details and content type details to allow the DataFederation Module to read the file correctly.

1. Go to Connections > Flat File.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Flat Fileconnection.

3. Enter a Connection Name for the Flat File data connection.4. Enter the File Path by clicking Browse and selecting the directory of the file.5. Select the Character Encoding of the flat file from the drop-down.6. Select the Record Type as Delimited.7. In Field Delimiter, select the expected separator between any two fields of a file record.8. Select the Text Qualifier (optional), if any, that encloses the field values of a file record.9. In Line Separator, the value Default is selected, indicating that the expected line separator

depends on whether Spectrum™ Technology Platform is running on a Unix or Windows system.10. To specify whether the first row of the file is a header row, shift the First Row is Header Row

slider to either Yes or No.



11. To specify whether the data type of the various fields in any record of the file should beautomatically detected, shift the Detect data type from file slider to either Yes or No.

12. Click Save.A message confirms the successful creation of the connection.

In order to view a sample record fetched using the created Delimited Flat File connection. clickPreview in the header bar. File records will be fetched and the Fields sorted according to the detailsprovided by you.

Adding a Fixed Width Flat File ConnectionTo add a new Fixed Width Flat File connection navigate to Connections > Flat File and select theRecord Type as Fixed Width. Enter the file's access details and content type details to allow theData Federation Module to read the file correctly.

1. Go to Connections > Flat File.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Flat Fileconnection.

3. Enter a Connection Name for the Flat File data connection.4. Enter the File Path by clicking Browse and selecting the directory of the file.5. Select the Character Encoding of the flat file from the drop-down.6. Select the Record Type as Fixed Width.7. In the Record Length field, enter the total number of characters in a file record.Repeat Step 8 to Step 13 to enter details of all fields expected in a file record.8. Click Add Field to add a row for a field in a file record.9. In the Name column, enter the name for the field value.10. In the Type column, select the data type of the field value.11. In the Start Position column, enter the position in the file record at which of the field value begins.

For the first field in a file record, the Start Position counting begins from 1.

12. In the Length field, enter the total number of characters the field covers, including the characterat the Start Position.The sum of the Start Position and Length values for any field should be less than or equal tothe Record LengthIf the File Record is:

01234Rob Smith29PitneyBowes

Record Length = 27

For the field 'Name':

Start Position = 6

Length = 9

Name = Rob Smith



13. Check the Trim checkbox if you wish to trim any white spaces at the beginning and/or end of afield value.


In order to view a sample record fetched using the created Fixed Width Flat File connection. clickPreview in the header bar. File records will be fetched and the Fields sorted according to the detailsprovided by you.

Date Time Formats in a File ConnectionWhile reading date and time values from files using a File Connection in the Spectrum™ DataFederation Module, the values need to adhere to certain specific date-time formats.

Accepted Date Time Formats

• Date: "yyyy-mm-dd"• Datetime: "yyyy-mm-dd HH:mm:ss"• Time: "HH:mm:ss"

These are as per standard date-time notations.

Delimited Files

If the Detect type feature is turned on while configuring the Delimited File Connection, then thedate and time values in the file records, which adhere to the above formats, are automaticallydetected as Date type.

If a date-time value does not adhere to one of the accepted formats, the value is read as a Stringtype value instead of a Date type value.

Fixed Width Files

For FixedWidth files, date type values are configured while creating the FixedWidth File Connection.Hence these values are read as Date type values, irrespective of whether they adhere to the acceptedformats or not.

If the date-time value in a Fixed Width file does not adhere to the accepted formats, it needs to behandled using Transformations at the Metaview creation stage by applying the below Conversioncategory function to the value:

parsedate(String date, String format)

In this, the date is the value received from the file, while the format is the date-time format in whichthe value is received from the file. This helps to parse the date-time value correctly.

For example, if the date = 23-Feb-2008, then the format = dd-MMM-yyyy.

Resulting Value Formats

While previewing data in a Virtual Data Source:



• If the value has been read as a date/time value, it is reflected in one of the accepted date/timeformats in the preview.

• If the value has been read as a String value, it is reflected as it is in the preview.

Deleting a Flat File ConnectionTo delete a Flat File connection navigate toConnections > Flat File, select the particular connection,and click Delete.

1. Go to Connections > Flat File.2. Select the Flat File connection to be deleted.3. Click Delete.



Salesforce Connection

Adding an Salesforce ConnectionTo add a new Salesforce connection navigate to Connections > Salesforce, and enter yourauthentication credentials to access the required Salesforce data source.

1. Go to Connections > Salesforce.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Salesforceconnection.

3. Enter a Connection Name for the Salesforce connection.4. In the Username field, enter the email ID registered on the Salesforce data store.5. In the Password field, enter a combination of the Salesforce portal login password and the

security token generated through the Salesforce portal.For example, if your password is Sales@Test, and the security token provided to you by Salesforceis 56709367, then the Password to authenticate this Salesforce connection would beSales@Test56709367.


To modify a Salesforce connection select the required connection, modify the details asrequired, and click Save.

Note:

Deleting a Salesforce ConnectionTo delete a Salesforce connection navigate to Connections > Salesforce, select the particularconnection, and click Delete.

1. Go to Connections > Salesforce.



2. Select the Salesforce connection to be deleted.3. Click Delete.



Siebel Connection

Adding a Siebel ConnectionTo add a new Siebel connection navigate to Connections > Siebel, and enter your authenticationcredentials to access the required Siebel data source.

It is assumed that a Siebel connection user is familiar with the Siebel CRM Tools andTechnologies, and terms like Siebel Business Objects/Components and links betweendifferent Business Components.

Note:

1. Go to Connections > Siebel.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Siebelconnection.

3. Enter a Connection Name for the Siebel data connection.4. In the Host-Name field, enter the name of the host on which the Siebel Server is installed.5. In the Port field, enter the listening port number for the SCBroker component (use 2321 for the

SCBPort).6. In the Server field, enter the name of your Siebel Enterprise Server.7. In the Object Manager field, enter the name of the Object Manager.8. Enter the Username provided to access the required Siebel data store.9. Enter the Password to authenticate the access.10. In the Locale field, select the type of Object Manager and Language Pack for the Siebel Business

application you are connecting to.

For locales other than English, locale specific JAR files need to be installed for a Siebelconnection. For more information, see Required JAR Files on page 55.

Note:


Within a Baseview, Siebel business components are displayed in the format <BusinessObject>.<Business Component>.

To modify a Siebel connection select the required connection, modify the details as required, andclick Save.



A Business Component which is not a part of any Business Object is not displayed inBaseviews.

Note:

Deleting a Siebel ConnectionTo delete a Siebel connection navigate to Connections > Siebel, select the particular connection,and click Delete.

1. Go to Connections > Siebel.2. Select the Siebel connection to be deleted.3. Click Delete.





4 - DesigningBaseviews

In this section

Introduction to Baseview 21Creating a Baseview 21Modifying a Baseview 22Schema - Baseview 23Deleting a Baseview 25

Introduction to Baseview

A Baseview is a representation of the data derived from a data source. It is created by selecting aconfigured and active data source connection, and selecting its required tables to be included inthe Baseview.

Once saved, a Baseview allows you to view the schema of the data in the selected Baseview tables,as present in the original data source. The schema depicts the relationship of the tables in the datasource and their various properties.

From a single data source, multiple Baseviews can be created to suit different purposes.

For example, for a system administrator, one Baseview may be created from a data source byincluding its system tables, while for use by end users a separate Baseview may be created fromthe same data source by including the required data tables.

Creating a Baseview

To create a Baseview, navigate to Baseview, and select the connection and its tables using whichthe Baseview is to be defined.

1. Click Baseview in the top bar.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Baseview.3. Enter a name for the Baseview in the Baseview name field.4. Select the required connection type based on which the Baseview is to be created from the

Connection type field.The connections of the selected connection type get populated in the Connection name field.For example, if you select database as a Data source type then the Data source should listthe connections like SQL, Oracle, MYSQL, Sybase etc.

5. From the Connection name field, select the desired connection.6. Click Go.

This lists all the tables/views of the selected connection in the below grid.7. Filter the search results according to the table/view names, owners, and/or types.

For example, if you want to filter the results based on table names beginning with the text "sa",type "sa" in the filter box at the top of the Name column of the grid. The grid lists the tables withnames beginning with "sa".


Designing Baseviews

If you want to filter tables based on owner names beginning with the text "sys", type "sys" in thefilter box at the top of the Owner column of the grid. The grid lists the tables with owner namesbeginning with "sys".

If you want to filter tables based on type names beginning with the text "view", type "view" in thefilter box at the top of the Type column of the grid. The grid lists the tables with type namesbeginning with "view".

8. Select the tables/views from the grid that you want to include in the Baseview, by checking thecorresponding checkboxes against the required table.Select the checkbox in the header row if you wish to select all the listed tables.The Total Items field below the grid displays the total number of tables present in the selectedconnection, while the Selected Items field displays the number of tables selected for inclusionin the Baseview from the connection.

9. To choose between viewing either the selected, unselected or all the tables of the connection,use the Show field.

• All: To view all the tables of the selected connection.• Selected: To view the tables selected for inclusion in the Baseview• Not Selected: To view the tables not selected for inclusion in the Baseview.

10. Click Save.

To view a hierarchical illustration of the Spectrum™ Data Federation entities designed so far, whichhas this created Baseview in focus, click Lineage in the top bar. For more information, see Lineageon page 43.

In a Baseview, for a Siebel connection, the business components are displayed in the formatBusiness Object.Business Component.

Note:

Modifying a Baseview

To modify an existing Baseview, navigate to Baseview in the header, select the Baseview you wishto modify and change its data source connection and/or selected tables as required.

1. Click Baseview in the top bar.2. Select the Baseview you wish to modify. You can search for a particular Baseview name using

the textbox above the Baseview list.On selecting a Baseview, its constituent tables are listed in the grid on the right. By default theShow field displays the Selected option, listing the tables already included in the Baseview.

3. Modify the selected connection for the Baseview in the Connection name field, if required.4. To view the unselected tables or all the tables of the selected connection, select the appropriate

value in the Show field.


Designing Baseviews

5. Select/deselect the tables that you want to add or remove from the existing Baseview.6. Click Save.

To view a hierarchical illustration of the Spectrum™ Data Federation flow designed so far, whichhas this created Baseview in focus, click Lineage in the top bar. For more information, see Lineageon page 43.

Schema - Baseview

The schema shows the properties and relationships of the tables selected in the Baseview, as theyexist in the data source. In the View Schema screen, for each selected table in the Baseview, thejoins and properties like primary keys, foreign keys, and / or indexes, as defined in the data source,are displayed.

The various sections of the View Schema screen display the properties of the selected table asmentioned below:

Displays the properties of each column of the selected table.Columns

Displays the primary key and its properties for the selected table.Primary Keys

Displays the foreign keys and their properties for the selected table.Foreign Keys

Displays the indexes and their properties for the selected table, andwhether the indexes are unique or non unique. If an index is non uniquethen the NU check box is displayed as checked, else it is unchecked.

Indexes

The properties of the selected table displayed in the schema are read-only and cannot bemodified.

Note:

Connection based Schema SectionsThe sections Columns, Primary Keys, Foreign Keys and Indexes are displayed in the ViewScehma screen based on the type of the data source.

The schema sections displayed for the different data connection types are as below:JDBC connection Displays the name, datatype, and size of each column.

The section also displays whether a column is nullableor a primary key.

Columns

Displays the primary key, primary key name, and keysequence.

Primary Keys

Displays the foreign key column name, foreign key name,primary key catalog name, primary key table name,primary key column name, and primary key name.

Foreign Keys

Displays the index name, column name, and whether theindexes are unique or non unique. If an index is non

Indexes


Designing Baseviews

unique then the NU check box is displayed as checkedotherwise it is displayed as unchecked.

Amazon SimpleDBconnection

Displays the column name and datatype for the tableyou select.

Columns

Apache Cassandraconnection

Displays the column name and datatype. The sectionalso displays whether a column is updatable,searchable, or a primary key.

Columns

Displays the primary key and primary key name.Primary Keys

Salesforceconnection

Displays the name, datatype, and size of eachcolumn. The section also displays whether a columnis nullable or a primary key.

Columns

Displays the primary key and primary key name.Primary KeysDisplays the foreign key column name, foreign keyname, and reference table name.

Foreign Keys

File connection Displays the column name and datatype for the tableyou select.

Columns

Viewing Schema of a BaseviewTo view the schema, select a saved Baseview and click View Schema.

1. Click Baseview in the top bar.2. Create a new Baseview and save it. Else select an existing Baseview.3. Click View Schema.

By default the View Schema button is disabled. It is only enabled for a Baseview once theBaseview is saved.

4. Use the Preview feature at the bottom left of the screen to set the focus on a particular sectionof the canvas.

5. Select a Baseview table whose properties you wish to view. The selected table's name is displayedin the PROPERTIES window on the right.

If there are multiple tables on the canvas, you can use the Search box to select a table.In the Search box, type a table name and then select the appropriate result from thesuggestion list.

Note:

6. Click PROPERTIES on the right to expand it and view the selected table's properties.The Table Properties window opens.

7. Click on any of the available sections Columns, Primary Keys, Foreign Keys, and Indexes toview the table properties. The sections are available based on the Data Source Connection.

In the Columns section, the size of a column is shown as zero if it is not defined in thedata source.

Note:


Designing Baseviews

Click Back to go back to the Baseview canvas.

Deleting a Baseview

1. Click Baseview in the top bar.2. Select the Baseview you wish to delete.3. Click Delete.

If the Baseview has any dependent Metaview(s) and/or Virtual Data Source(s), a warningmessageis displayed.

4. Click Yes to confirm.A message confirms the successful deletion of the Baseview.

If you delete a Baseview which has dependent Metaviews and Virtual Data Sources:

1. A dependent Metaview of the deleted Baseview is rendered unusable as the mapped tables ofthe deleted Baseview are no longer available in the Metaview.

2. A dependent Virtual Data Source of the deleted Baseview continues to function as defined untilit is redeployed after the Baseview deletion.


Designing Baseviews

5 - DesigningMetaviews

In this section

Introduction to Metaview 27Creating a Metaview 27Deleting a Metaview 37

Introduction to Metaview

A Metaview is a logical representation of business-specific entities. You can create a Metaview byderiving the required data from one or more Baseviews, and applying conversions on it using thefeatures of Transformations and Criteria Builder to replicate your business model.

Creating a Metaview

To create a Metaview, add new tables, map them to existing baseview table columns, and use theTransformations and Criteria Builder features to convert, compare and filter the data as required.

1. Click Metaview in the top bar.2. By default, the screen opens in the create mode. Otherwise, click on the Metaview dropdown

and select New to add a new Metaview.3. Enter a name for the Metaview in the Metaview name field adjacent to the Metaview selection

dropdown.4. To add any number of tables in a metaview, click Add Table. Expand the Entry Attributes

section which displays the column names and datatypes of the Metaview tables. For moreinformation, see Adding Tables in a Metaview on page 28.

5. To add source tables to the Metaview from available Baseviews, expand the Source Tablessection of the Metaview Editor, select a Baseview from the drop-down and drag the requiredtables of the Baseview into the canvas to map them to the Metaview tables. For more information,see Adding Source Baseview Tables in a Metaview on page 28.

6. To apply transformations to the columns of the Metaview tables, expand the Transformationssection. For more information, see Transformations on page 29.Different transformations can be applied on the Metaview table columns using Baseview tablecolumn mappings, arithmetic, string, datetime and other utility functions along with modificationsusing constants.

7. To apply data filters as well as filter groupings on the Metaview, expand the Criteria Buildersection. Add data filter groups and filter criteria as required for the various Metaview tables andcolumns. For more information, see Criteria Builder on page 35.

8. Click Save.

To view a hierarchical illustration of the Spectrum™ Data Federation entities designed so far, whichhas this created Metaview in focus, click Lineage in the top bar. For more information, see Lineageon page 43.


Designing Metaviews

Adding Tables in a MetaviewBefore creating Metaview you must create a Baseview.

1. To add a table in the Metaview, clickAdd Table in theMetaview Editor. You can add any numberof tables to a Metaview.The new table is displayed on the canvas with one default column.

2. To name the table, double-click on the table title bar in the canvas and name it.The title bar of the Metaview Editor displays the name of the selected Metaview table on thecanvas.

3. To add columns to a table, first select the table on the canvas. Then click Add Column. Renamethe column by double-clicking the column within the table on the canvas.Expand the Entry Attributes section to view the column names and datatypes of the selectedMetaview table. You can rename a table column in the Entry Attributes section as well.

The datatype of each Metaview table column is derived from the column of the source Baseviewtable to which the former is mapped.

1. To delete a column of a Metaview table, select the table on the canvas, and in the Entry Attributes

section click against the column.2. To delete a Metaview table, select the table on the canvas, and clickDelete Table in theMetaview

Editor.

Adding Source Baseview Tables in a MetaviewTo add source Baseview tables for a Metaview, add a Metaview table on the canvas and open theSource Tables section in the Metaview Editor.

Ensure you have added a table in the Metaview. The Source Tables section is available in theMetaview Editor if at least one table has been added to the Metaview.

Ensure at least one Baseview is configured and available in the system in order to definesource tables for a Metaview.

Note:

1. Click on the Source Tables section in the Metaview Editor.2. To display the Baseview tables used in the Metaview on the canvas, check the Show checkbox

on the Source Tables section header.3. Select the required Baseview from the Baseview dropdown in this section.

The tables in the selected Baseview are displayed below.4. Drag the required Baseview table on to the canvas.5. To link the columns of the Metaview table to the desired column of the Baseview table, click on

the column of the Metaview table and drag the pointer to the desired column of the Baseviewtable.


Designing Metaviews

A column of a Metaview table derives values from the respective column of the Baseview tableto which it is linked.

You can link a Baseview column using Transformations as well. Refer Applying a ColumnTransformation on page 30.

The datatype of the Metaview table column is derived from the Baseview table columnto which it is linked.

Note:

6. Drag the tables and links on the canvas to position them as desired.7. Click Save.

Note: 1. To delete a link between the columns of a Metaview table and a Baseview table, selectthe link and press Delete on your keyboard.

2. You can add tables from multiple Baseviews to a Metaview, by repeating the steps 3to 6.

Example - Linking Source Tables in a MetaviewFor example, in a Metaview, you add a new table Item and the Baseview tablesNew and Watch.

You link:

1. The Name column of the Metaview table Item to the ItemName column of theBaseview table New

2. The Symbol column of the Metaview table Item to the SYMBOL column of theBaseview table Watch

The relationship is depicted on the canvas as below:

TransformationsTransformations allow you to apply various conversions on a value and populate the convertedvalue into a column of a Metaview table.

The source value can be from a Baseview table column or a constant.


Designing Metaviews

You can define create complex transformations by defining the components as nested transformationswithin the parent transformation.

Applying a TransformationTo apply a transformation to any column of a Metaview table, open the Transformations sectionin the Metaview Editor.

1. Click on the Transformations section in the Metaview Editor.

2. To add a transformation on a column of a Metaview table, click against the column.For example, if you wish to trim the value of the column ItemName of the Baseview table Newvalue before filling it in the linked Name column of the Metaview table Item, select the Item table

on the canvas, and click against the column in the Transformations section.The Transformation Editor window opens.

3. In the Transformation Editor window, select the Undefined column value on the left. You needto define the resulting value for the destination column of the Metaview table.

4. Select either Column, Constant, or Function. The selected option is used to derive the valueof the particular column of the Metaview table.

5. Define the transformation as required.The transformation applied on the column of the Metaview table is displayed on the left in theTransformation Editor.

6. Click OK.

The applied transformation is displayed on the canvas for the particular column of the Metaviewtable.

Applying a Column TransformationA column transformation allows you to link the required column of a Baseview source table to definea column of a Metaview table or a parameter of a function transformation.

You can also apply a column transformation by linking a Baseview table column with a Metaviewtable column on the canvas directly.


2. To add a column transformation on a column of a Metaview table, click against the column.The Transformation Editor window opens.

3. In the Transformation Editor window, select the Undefined column value on the left. This isthe resulting column value of the Metaview table you need to define using the required function.

4. Select the Column option.5. In the Baseview Name field, select the Baseview whose table's column you wish to link.6. In the Table Name field, select a table of the selected Baseview.7. In the Column Name field, select the column you wish to link.

The selected column of the Baseview table shows on the left.8. Click OK.


Designing Metaviews

The applied column transformation is reflected on the canvas with a link.

Applying a Constant TransformationA constant transformation allows you to apply a constant value to define a column of a Metaviewtable or a parameter of a function transformation.


2. To add a constant transformation on a column of a Metaview table, click against the column.The Transformation Editor window opens.


4. Select the Constant option.5. From the Type field, select the datatype of the constant value you wish to define.6. In the Value field, enter a constant value of the datatype selected in the previous step.

The constant value shows on the left.7. Click OK.

The applied constant transformation is reflected on the canvas with a transformation node.

Applying a Function TransformationA function transformation allows you to apply functions of string, numeric, conversion, and datetimecategories to define a column of a Metaview table or a parameter of a function transformation.


2. To add a function transformation on a column of a Metaview table, click against the column.The Transformation Editor window opens.


4. Select the Function option.5. From the Category field select the category of the function.

The function categories available:

• String• Numeric• Conversion• Datetime

6. From the Name field select the function you wish to apply .Based on the number of parameters expected in the function, as many undefined parametersare displayed on the left.

7. Select a parameter on the left to define it for the function.8. Define the parameter using either of the options Column, Constant, or Function.

Ensure that the resulting datatype of the defined parameter matches the expected datatypefor the parameter in the parent function.

Note:


Designing Metaviews

9. Repeat step 8 for each parameter of the parent function.10. Click OK.

The applied function transformation is reflected on the canvas with a transformation node.

Example - Applying a Function Transformation to a column of a Metaview tableFor example, if you wish to trim spaces from the right of each value of the columnItemName of the Baseview table New and fill in the trimmed value into the linkedName column of the Metaview table Item:

1. Select the Item table on the canvas.

2. Click against the Name column in the Transformations section of theMetaview Editor.

3. In the Transformation Editor window, select undefined on the left. This is thecolumn you wish to define using a function transformation.

4. On the right, select the Function option.5. In the Category field, select String.6. In the Name field, select the rtrim(string string) function.7. On the left, select the undefined function parameter. As shown in the function

name, the expected datatype of the parameter is String.8. Define the parameter using either of the optionsColumn,Constant, or Function,

such that the resulting datatype of the parameter is String.9. Click OK.

The applied function transformation is depicted on the canvas as below:


Designing Metaviews

Conversion Function TransformationA conversion function allows you to convert values from one datatype into another datatype, fromone format into another format, or parse values according to desired formats.

A conversion function is applied to convert the data from a source table column data or a constantinto the desired datatype or format. This converted data populates a column of a Metaview table.

The various conversion functions available are:

This function allows you to convert values of one datatype into anotherdatatype.

cast

This function allows you convert a value of any datatype into a String valueusing a provided format.

format

This function allows you to parse a string value into the desired datatypeusing a provided format .

parse

This function allows you to convert a CLOB type value into bytes.to_bytes

This function allows you to convert a BLOB type value into chars.to_chars

Applying a Conversion Transformation

The cast, format, and parse conversion functions have two parameters: a source value of a datatypewhich needs to be converted into a different target datatype.

1. In the Transformations section of the Metaview Editor click against the column to beedited.The Transformation Editor window opens.

2. In the Transformation Editor window, select the Undefined column value on the left.3. Select the Function option on the right.4. From the Category field, select Conversion.5. Select the conversion function you wish to apply from the Name field.

The first parameter of this function is the value which needs to be converted. The secondparameter is the target datatype into which the first parameter value needs to be converted.

6. Select the first undefined parameter on the left.7. Define the first parameter of the conversion function using either of the optionsColumn,Constant,

or Function.

Ensure the resulting datatype matches the expected datatype for the first parameter ofthe cast function.

Note:

8. Select the second undefined parameter on the left.9. Select the Constant option on the right.10. From the Type field, select the datatype String.11. In the Value field, enter the desired target datatype into which the first parameter value is to be

converted. Refer Supported Datatypes for Conversion Transformations on page 67.


Designing Metaviews

The target datatype value shows on the left for the second parameter.12. Click OK.

The applied conversion function transformation is reflected on the canvas with a link.

Example - Applying a Cast Conversion Function TransformationFor example, if you wish to convert an Integer value from the column BALANCE ofthe Baseview table CUSTOMER of the Baseview ClientSheet into a Float value topopulate the column CUSTBALANCE of a Metaview table CUSTPROFILE:

1. Select the CUSTPROFILE Metaview table on the canvas.

2. Click against the CUSTBALANCE column in the Transformations section ofthe Metaview Editor.

3. In the Transformation Editor window, select undefined on the left.4. On the right, select the Function option.5. In the Category field, select Conversion.6. In theName field, select the cast(Integer value,string target) function.7. On the left, select the first undefined function parameter. The expected datatype

of the first parameter is Integer.8. For the first parameter, select the column BALANCE of the Baseview table

CUSTOMER.

• Select the Column option.• In the Baseview Name field, select the Baseview ClientSheet.• In the Table Name field, select a table CUSTOMER.• In the Column Name field, select the column BALANCE.

The Integer value from this column becomes the first parameter of the cast function.9. On the left, select the second undefined function parameter. The expected

datatype of the second parameter is string.10. For the second parameter:

• Select the Constant option.• From the Type field, select the datatype String.• In the Value field, enter Float. This is the target datatype.

The target datatype Float becomes the second parameter of the cast function.11. Click OK.

The applied cast conversion function transformation is depicted on the canvas.

Editing a Transformation

1. To edit a transformation, in the Transformations section of the Metaview Editor, clickagainst the column to be edited.


Designing Metaviews

In case of constant and function transformations, you can also double click the transformationnode on the canvas to edit it.

2. In the Transformation Editor window, select the column or function parameter on the left whichyou wish to modify.

3. Select either Column, Constant, or Function. The selected option is used to derive the valueof the particular column of the Metaview table.

4. Define the transformation as required.The transformation applied on the column of the Metaview table is displayed on the left in theTransformation Editor.

5. Click OK.

The edited transformation is displayed on the canvas for the particular column of the Metaview table.

Deleting a Transformation

1. Delete a transformation using either of the below ways:.

• Select the transformation function node or the transformation link on the canvas and pressDelete on your keyboard.

• In the Transformations section of theMetaview Editor press against the transformationyou wish to delete.

2. Click Save.

Criteria BuilderThe Criteria Builder allows you to define the required criteria to filter the data retrieved from thesource Baseviews while populating a Metaview's tables.

Components - Criteria Builder

The criteria for a Metaview consists of a parent group, which comprises of a hierarchy of criteriaand child groups.

A group is defined using multiple criteria and child groups. You can create a hierarchyof child groups and criteria up to any desired level within the parent group.

For each group, you need to specify whether the data fetched from the source Baseviewtables needs to satisfy all, at least one or none of the defined criteria of that group.

Group

A criterion is defined using either one or two operands and a logical operator.

The selected logical operator either compares the left operand to the values defined inthe right operand, or places a condition on the left operand.

Criterion

The column of the Baseview table on which you wish to define thecriterion.

Left Operand


Designing Metaviews

The right operand is defined as any one of the following:Right Operand

• A constant• A set of constants• A column of any of the Baseview tables• A set of columns of any of the Baseview tables

The logical operator defines the condition to be fulfilled for thecriterion. The operator can:

Operator

• Place a condition on the left operand.• Place a condition comparing the left operand with the rightoperand.

Creating a Criteria

1. Click on the Criteria Builder section in the Metaview Editor.2. Click Add Group.3. Select a grouping method for the criteria defined in the group by clicking on <group>. Specify

whether it is necessary to fulfill all, at least one, or none of the criteria defined in the group.4. To add a child group, select the parent group, and click Add Group.5. To add a criterion, select the parent group, and click Add Criterion.

a) To define the left operand of the criterion, click Column 1 to select one of the Baseview tablecolumns .

a) To select a logical operator from the provided list, click <comparison>.If the selected logical operator requires a right operand, a placeholder for the right operandis displayed to be defined.

b) To define the right operand of the criterion, if required, click Column 2.Define the right operand as any of the below:

• A constant• A set of constants• A column of any of the Baseview tables• A set of columns of any of the Baseview tables

6. Repeat steps 4 and 5 to create the required criteria.7. Click Save in the header.

All, any, or none of the defined set of criteria need to hold true in order to retrieve the required datafrom the connected Baseview tables into the Metaview table columns.

To modify the criteria, select the child group or criterion you wish to edit, modify it as required, andclick Save.


Designing Metaviews

To expand a group and view its definition, click within the Criteria Builder section. To

collapse a group's contents, click .

Note:

Editing a Criteria

1. Click on the Criteria Builder section in the Metaview Editor.2. Select the group, child group or criterion you wish to edit.3. Make the required changes.4. Click Save in the header.

Deleting a Criteria

1. Click on the Criteria Builder section in the Metaview Editor.2. Select the group, child group or criterion you wish to delete.3. Click Delete.4. Click Save in the header.

Deleting a Metaview

1. Click Metaview in the top bar.2. Click on the Metaview dropdown and select the Metaview you wish to delete.3. Click Delete in the header.

A window prompts you to confirm the deletion.4. Click Yes.

A message confirms the successful deletion of the Metaview.


Designing Metaviews

6 - Designing VirtualData Sources

In this section

Virtual Data Source 39Creating a Virtual Data Source 39Modifying a Virtual Data Source 40Deploying or Undeploying a Virtual Data Source 40Previewing a Virtual Data Source 41Deleting a Virtual Data Source 41

Virtual Data Source

A Virtual Data Source is the business-specific virtual representation of the data collected fromdifferent heterogeneous data sources. The collected data is represented in a business specificmodel by strategically grouping the relevant Metaviews and Baseviews.

Multiple Virtual Data Sources can be created to suit different business requirements, by appropriatelygrouping available Metaviews and Baseviews.

Once created, a Virtual Data Source needs to be deployed for its data to be accessed by the clientapplication. A deployed Virtual Data Source can be undeployed from use, when required.

Creating a Virtual Data Source

To create a Virtual Data Source, navigate to Virtual Data Source, and select the Baseviews and/orMetaviews to include in it.

1. Go to Virtual Data Source.

2. By default, the screen opens in the create mode. Otherwise, click to add a new Virtual DataSource.

3. Enter a name in the text box for the new Virtual Data Source.4. Select the Metaviews and / or Baseviews to include in the Virtual Data Source.

When you add a Metaview to the Virtual Data Source, a message prompts you to add its relatedBaseviews as well. Click Yes to add the related Baseviews of the Metaview to the Virtual DataSource. If not, the deployment of a Metaview, whose related Baseviews have not been added,will fail as part of the deployment of the Virtual Data Source.

5. Click Save.

To view a hierarchical illustration of the Spectrum™ Data Federation enitites designed so far, whichhas this created Virtual Data Source in focus, click Lineage in the top bar. For more information,see Lineage on page 43.


Designing Virtual Data Sources

Modifying a Virtual Data Source

To modify an existing Virtual Data Source, navigate to Virtual Data Source, select the Virtual DataSource you wish to modify, and add / remove the Metaviews and / or Baseviews from it as required.

Only an undeployed Virtual Data Source can be modified. To modify a deployed Virtual Data Source,first undeploy it and then modify it.

1. Go to Virtual Data Source.2. Select the Virtual Data Source from the left sidebar which you wish to modify.3. You can remove added Metaviews and Baseviews from a Virtual Data Source, as well as add

other available Metaviews and / or Baseviews to it.

• To remove an addedMetaview / Baseview from this Virtual Data Source, uncheck the checkboxagainst it.

• To add other available Metaviews / Baseviews to this Virtual Data Source, select "Not Selected"in the Show drop-down. In the grid below, which displays a list of all the other availableMetaviews and Baseviews, check the checkbox against the desired Metaviews and / orBaseviews to add them to the Virtual Data Source.

If you select a Metaview to add to the Virtual Data Source, but do not select the Baseviews onwhich the Metaview is dependant, a message prompts you to select those Baseviews as well.Click Yes to add these related Baseviews to the Virtual Data Source. If not, the deployment ofthe Metaview, whose related Baseviews have not been added, will fail during the deployment ofthe Virtual Data Source.

4. Click Save.

To view a hierarchical illustration of the Spectrum™ Data Federation entities designed so far, whichhas this created Virtual Data Source in focus, click Lineage in the top bar. For more information,see Lineage on page 43.

Deploying or Undeploying a Virtual Data Source

To deploy or undeploy an existing Virtual Data Source, navigate to Virtual Data Source, select thedesired Virtual Data Source, and click Deploy / Undeploy.

1. Go to Virtual Data Source.2. Select the Virtual Data Source from the left sidebar which you wish to deploy / undeploy.3. You can deploy an available Virtual Data Source for use in your application, or undeploy an

already deployed Virtual Data Source.

• Click Deploy to use the data from the Virtual Data Source in your application.



• Click Undeploy to detach the Virtual Data Source from your application.

Note: 1. On undeploying a Virtual Data Source which is in use currently, any further usage ofconnections created previously for using this undeployed Virtual Data Source will resultin a runtime exception.

If the same Virtual Data Source is deployed again, a new connection needs to be createdto use it.

2. If a Baseview or Metaview on which a Virtual Data Source is dependent is deleted, theVirtual Data Source continues to function until it is redeployed. A warning message isdisplayed to convey the list of entities which no longer exist.

Previewing a Virtual Data Source

To preview a deployed Virtual Data Source, select it, click Preview, and run the required SQL query.

An undeployed Virtual Data Source cannot be previewed.Note:

1. Go to Virtual Data Source.2. Select the desired deployed Virtual Data Source from the left sidebar.3. Click Preview.4. In the Query textbox, type in the SQL query to retrieve desired data from the selected Virtual

Data Source. Enclose the table name within double-quotes.select * from "EMPLOYEE_SALARY";

5. Click Fetch Data.

Deleting a Virtual Data Source

Only an undeployed Virtual Data Source can be deleted. To delete a deployed Virtual Data Source,first undeploy it and then delete it.

1. Go to Virtual Data Source.2. Select the Virtual Data Source from the left sidebar which you wish to delete.3. Click Delete.

A message confirms the successful deletion of the Virtual Data Source.



7 - Common Features

In this section

Lineage 43

Lineage

The Lineage button lets you view a hierarchical illustration of the designed Spectrum™ DataFederation flow. You can arrive at the Lineage screen from within aBaseview,Metaview, or VirtualData Source.

The illustration maintains focus on the component from within which you arrive at the Lineagescreen.

For instance, if you click the Lineage button from within a Baseview, the tree-hierarchy illustrationkeeps the selected Baseview in focus while displaying the data federation flow designed so far,including the Data Source Connections, Metaviews and Virtual Data Sources added, if any.

Viewing LineageFrom within theBaseview,Metaview, or Virtual Data Source screen, select the respective desiredentity, and click Lineage in the top bar.

1. Go to Baseview, Metaview, or Virtual Data Source.2. Select the respective entity for which you wish to view the hierarchical illustration.3. Click Lineage in the top bar.

The Lineage screen displays the designed Spectrum™ Data Federation flow, keeping the selectedBaseview / Metaview / Virtual Data Source in focus. The name of the selected entity is displayed inthe header of the Lineage screen.

If you hover over a Lineage component using your mouse, the name of the corresponding Baseview/ Metaview / Virtual Data Source is displayed in a tooltip.

The legend to the bottom right of the Lineage screen identifies the symbols used to denoteData Source Connections, Baseviews, Metaviews, and Virtual Data Sources.

Note:


Common Features

8 - Using theAdministration Utility

In this section

Getting Started with the Administration Utility 45Using a Script with the Administration Utility 46Categories of Commands 46System 48

Getting Started with the Administration Utility

The Administration Utility provides command line access to several administrative functions. Youcan use it in a script, allowing you to automate certain administrative tasks. You can also use itinteractively. Not all administrative functions are available in the Administration Utility. Use theManagement Console to access the functions that are not available in the Administration Utility.

The Administration Utility requires Java 7. If you are installing the Administration Utility onthe same computer where the Spectrum™ Technology Platform server is installed, Java 7

Note:

is already installed since it is included with the server. If you want to install the AdministrationUtility on another computer, make sure that the computer has Java 7 installed.

1. Open a web browser and go to the Spectrum™ Technology Platform Welcome Page at:

http://<servername>:<port>

For example, if you installed Spectrum™ Technology Platform on a computer named"myspectrumplatform" and it is using the default HTTP port 8080, you would go to:

http://myspectrumplatform:8080

2. Click Platform Client Tools.3. Click Command Line Clients.4. Under Administration Utility, click Download and download the zip file to the computer where

you want to use the Administration Utility.5. Extract the contents of the zip file.6. To launch the command line interface, do one of the following:

• If you are running the server on a Unix or Linux system, execute cli.sh.• If you are running the server on a Windows system, execute cli.cmd.

If necessary, modify the .sh or .cmd file to use the path to your Java installation.Note:

7. Connect to the Spectrum™ Technology Platform server that you want to administer. To connectto the server, execute the following command:

connect --h servername:port --u username --p password --s SSLTrueOrFalse

For example:

connect --h myserver:8080 --u admin --p myPassword1 --s true

8. Once you are connected you can execute commands. For a list of available commands, typehelp.

9. When you are done, type exit to exit the Administration Utility.


Using the Administration Utility

Using a Script with the Administration Utility

The Administration Utility can execute a series of commands from a script file. This is useful if youwant to automate or standardize administrative actions through the use of a script instead of manuallyexecuting commands.

1. Using a text editor, create a script file. A script file contains the commands that you want toexecute.

To add a command to a script file, type the command and the necessary parameters as youwould if you were entering the command at the command prompt. Enter one command per line.

To insert comments into a script file, use the following notation:

Indicates the start of a block comment./*

Indicates the end of a block comment.*/

Indicates an inline comment. Use at the start of a line only.//

Indicates an inline comment. Use at the start of a line only.;

2. Save the script either on the computer where you run the Administration Utility or in a locationthat is accessible from the computer where you run the Administration Utility. You can use anyfile name and extension you choose. The recommend file extension is .cli.

3. Open the Administrative tool and connect to the Spectrum™ Technology Platform server usingthe connect command. For more information on this command, see connect on page 48.

4. Use the script command to execute the script. For more information on this command, see scripton page 48.

Categories of Commands

The Administration Utility commands for Spectrum™ Data Federation are broadly categorized asData Source, Baseview, Metaview, and Virtual Data Source commands.

The Data Source commands provide the functionality to export and/or import all thedata sources supported by Spectrum™ Data Federation except JDBC specific datasources.

To get the list of various commands supported under this category, typesdfdatasource in the Administration Utility and press the Tab key.

Data Source

To get the details of a particular command and its various parameters, type help<command name>.



For example:

help sdfdatasource export

The Baseview commands provide the functionality to export and/or import allBaseviews defined in Spectrum™ Data Federation.

To get the list of various commands supported under this category, typesdfbaseview in the Administration Utility and press the Tab key.

Baseview


For example:

help sdfbaseview export

The Metaview commands provide the functionality to export and/or import allMetaviews defined in Spectrum™ Data Federation.

To get the list of various commands supported under this category, typesdfmetaview in the Administration Utility and press the Tab key.

Metaview


For example:

help sdfmetaview export

The commands of this category provide the functionality to export and/or import allVirtual Data Sources defined in Spectrum™ Data Federation.

To get the list of various commands supported under this category, typesdfvirtualdatasource in the Administration Utility and press the Tab key.

Virtual DataSource


For example:

help sdfvirtualdatasource export



System

connectThe connect command opens a session with the Spectrum™ Technology Platform server youspecify. You must issue the connect command before you can issue other commands.

Usageconnect --h HostName --u UserName --p Password --s TrueOrFalse

DescriptionArgumentRequired

The host name and port to connect to, separated by a colon. Forexample, to connect to MyServer on port 8080 you would specify--h MyServer:8080.

--h HostNameYes

The user name to use to authenticate to the server.--u UserNameYes

The password for the user.--p PasswordYes

Specifies whether to create a secure connection using SSL. You canonly connect to the server over a secure connection if the server has

--s TrueOrFalseNo

been configured to support SSL communication. TrueOrFalse mustbe one of the following:true

Use SSL.

falseDo not use SSL. This is the default setting.

Example

This example opens a connection to the server MyServer on port 8080 with the username admin and the password myPassword1.

connect --h MyServer:8080 --u admin --p myPassword1

scriptThe script command directs the command line interface to execute a script containing a seriesof commands. You can use a script to automate administrative tasks.



Usagescript --file ScriptFile --linenumbers TrueOrFalse

DescriptionArgumentRequired

Specifies the path to the script file.--file ScriptFileYes

Specifies whether to display line numbers whileexecuting the script, where TrueOrFalse is oneof the following:true

--linenumbers TrueOrFalseNo

Displays line numbers whileexecuting the script.

falseDoes not display line numberswhile executing the script.This is the default value.



9 - Conventions

In this section

Naming Conventions 51Query Conventions 51

Naming Conventions

• The names of Data Source Connections, Baseviews, Metaviews and Virtual Data Sources mustbegin with a letter, and consist of only alphanumeric characters and underscores.

• The names of Data Source Connections, Baseviews, Metaviews and Virtual Data Sources cancontain reserved words.

• The names of the tables and columns of a baseview's underlying data source can contain specialcharacters and reserved words.

• The names of tables and columns of a metaview can contain reserved words.

Query Conventions

• While previewing a Virtual Data Source using an SQL query, enclose a table name or columnname containing reserved words or special characters within double quotes.

If a table name or column name does not contain special characters or reserved words, do notenclose it in double quotes.

See Reserved Keywords on page 62.

For example, the table name CUSTOMER_DETAILS in the below query does not need to beenclosed within double quotes, since it is not a reserved word:

select * from CUSTOMER_DETAILS

But in the below query, the column name DATE, being a reserved word, needs to be enclosedwithin double quotes, while the CUSTOMER_ID column does not need to be enclosed:

select "DATE", CUSTOMER_ID from CUSTOMER_DETAILS

• If a column name includes a pair of double quotes, enclose the column name in two additionalpairs of double quotes while using it in a query.

For example, if the table CUSTOMER_DETAILS has the column "CITY", such that the columnname includes the double quotes, use this column in a query as below:

select """CITY""" from CUSTOMER_DETAILS


Conventions

10 - Configurations

In this section

JDBC Driver Configuration 53Required JAR Files 55

JDBC Driver Configuration

To Access a Virtual Data Source from within the Spectrum™ Technology Platform:

For users accessing a Virtual Data Source of the Spectrum™ Data Federation Module from withinthe Spectrum™ Technology Platform, the JDBC driver is pre-installed as part of the Spectrum™

Technology Platform installation.

The Spectrum™Data Federation option is available in theManagement Console, withinResources >JDBC Drivers.

To Access a Virtual Data Source from an External Application:

For users accessing the Spectrum™ Data Federation from an external application, the sdf-jdbc.jarneeds to be downloaded and installed manually. For more information, see Installing JDBC Driverfor an External Application on page 55.

Accessing a Virtual Data Source from within Spectrum™ Technology PlatformFor users accessing a Virtual Data Source of the Spectrum™ Data Federation Module from withinthe Spectrum™ Technology Platform, the JDBC driver is pre-installed as part of the Spectrum™

Technology Platform installation.

You need to create a connection to the required Virtual Data Source to use it.

Creating a Connection to use a Virtual Data SourceTo use a Virtual Data Source of the Spectrum™ Data Federation Module, you need to create aconnection using the Management Console.

In the Management Console, ensure that the Spectrum™ Data Federation option is presentwithin Resources > JDBC Drivers.

Note:

1. In the Management Console, go to Resources > Connections.2. Click Add.3. Enter a name for the connection in the Connection name field.4. Select Spectrum™ Data Federation in the Database driver field.5. In the Connection Options grid, enter the below details:

a) Enter a valid Spectrum™ username in the user field.

The entered user must have the 'Execute' right for the required Virtual Data Source.Note:

b) Enter the password for the user.


Configurations

c) In the host field, enter the IP or the machine name of the Spectrum™ server on which thedesired Virtual Data Source has been deployed.

d) In the port field, enter the port number to access the Spectrum™ Data Federation Module.The default port is 32750.

To see how to change the port, see Changing the Port for Spectrum™ DataFederation on page 54.

Note:

e) Enter the name of the Virtual Data Source you wish to use in the virtualDataSource field.

6. Click Test to test whether the defined connection works properly. A message confirms a successfulconnection.

7. Click OK.

Changing the Port for Spectrum™ Data Federation

1. Go to the path {Spectrum™ server installationdirectory}\server\modules\datafederation\conf.

2. Open the file federation_config.properties.3. Edit the property edf.socket.port to the desired port.4. Save the file.5. Restart the Spectrum™ Technology Platform server.

The new port can now be used to connect to desired Virtual Data Sources of the Spectrum™ DataFederation Module.

Accessing a Virtual Data Source from an External ApplicationFor users accessing a Virtual Data Source of the Spectrum™ Data Federation Module from anexternal application, the JDBC driver needs to be downloaded and installed manually.

Once the JDBC driver is installed, connections to the Spectrum™ Data Federation can be createdusing a simple Java code, in order to use the Virtual Data Sources.

To see the steps to install the JDBC driver and using the same to create connections to the desiredVirtual Data Source, see Installing JDBC Driver for an External Application on page 55.


Configurations

Required JAR Files

Connection JAR FilesTo connect to a Siebel server with a locale other than English, theSiebelJI_<locale>.jar file needs to installed.

For example, if the locale selected is French, the required jar file isSiebelJI_fra.jar.

Siebel Connection

For more information, see Installing a Siebel Locale JAR File on page 55.

Configuration JAR FilesFor users accessing the Spectrum™ Data Federation from an externalapplication, the sdf-jdbc.jar needs to be downloaded and installed. For more

JDBC Driver

information, see Installing JDBC Driver for an External Application on page55.

Installing a Siebel Locale JAR FileTo connect to a Siebel server with a locale other than English, the locale-specific jar file needs tobe installed. The jar file name follows the convention SiebelJI_<locale>.jar.

For example, if the locale selected is French, the required jar file is SiebelJI_fra.jar.

The English locale jar file, SiebelJI_enu.jar comes pre-installed with the Spectrum™ DataFederation installation.

Note:

1. Login to the Siebel server system, to which the connection is to be created.2. Navigate to the path {Siebel-server-installation-directory}\siebsrvr\CLASSES.3. Download the required locale specific Siebel JAR file from the mentioned path.4. Place the downloaded jar file in the mentioned path on the Spectrum™ server:

{installation-path}\Pitney Bowes\Spectrum\server\app\lib.

You can now create a Siebel connection in the locale for which you installed the jar file.

Installing JDBC Driver for an External ApplicationTo install the JDBC driver in order to use the Spectrum™ Data Federation with your externalapplication, the relevant jar needs to be installed.

1. In the Spectrum™ Data Federation web UI, click on the admin dropdown in the header.2. Click on Download Spectrum™ Data Federation JDBC Driver.3. Save the sdf-jdbc.jar file in the default browser directory.


Configurations

You do not need to restart the server.Note:

Once the JDBC driver is installed, the below details are needed to connect to the Spectrum™ serverusing a Java code or any application:

1. Spectrum™ username2. Password to authenticate the Spectrum™ username3. JDBC Driver Class Name = com.pb.spectrum.sdf.driver.SDFDriver

4. Connection String Template =jdbc:spectrum://${host}:${port}/${virtualDataSource}.

Fill in the server IP or machine name in place of {host}, the port for the Spectrum™ DataFederation in {port}, and the name of the Virtual Data Source you wish to access in{virtualDataSource}.

using valid Spectrum™ username and password and connection properties.

You can now connect to any desired Virtual Data Source.


Configurations

11 - Limitations

In this section

Common Limitations 58Siebel Connection Limitations 58Write Feature Limitations 59

Common Limitations

Common Limitations

1. Spectrum™ Data Federation does not interpret case sensitive field names as different names. Iftwo tables or column names use the same name but different cases, they are interpreted asduplicate names. If a Virtual Data Source contains such field names, the deployment of thatVirtual Data Source fails.

For example, fields like DeDup Token, Dedup Token, and DEdup Token are interpreted asduplicate names. The deployment of a Virtual Data Source containing such case-differentiatedfields fails.

2. If a Data Source, Baseview, Metaview or Virtual Data Source has a dependency on a JDBCspecific data source, it cannot be imported or exported using the Administration Utility.

Siebel Connection Limitations

1. Fields of Siebel Business Components with the same names but differing cases are handled byleaving the first occurrence unchanged, while appending all subsequent occurrences with _PB_and an increasing numeral value.

For example, fields like DeDup Token, Dedup Token, and DEdup Token in a Siebel businesscomponent are renamed to DeDup Token, Dedup Token_PB_1, and DEdup Token_PB_2respectively and used.

2. If a field name contains a dot (.) in a Siebel schema, the dots are removed from the field namefor usage in the Spectrum™ Data Federation Module.

3. While creating a Baseview from a Siebel connection, links between any two Siebel BusinessComponents are displayed only if the link name matches the Business Component name in theSiebel schema.

For example, in a Siebel schema, if a link is defined between the Business Components Accountand Contact, while another Business Component Contact Custom uses the links of Custom,then in the Baseview created for this Siebel connection, no link is displayed between Accountand Contact Custom. Links are displayed only between Account and Contact, as the originallink is between these two in the Siebel schema.

4. Fields of a Business Component which are marked as inactive in the Siebel schema are notdisplayed in Baseviews, Metaviews, and Virtual Data Sources.


Limitations

Write Feature Limitations

In Spectrum™ Data Federation, while writing to a database source using theWrite to DB stage inthe Enterprise Designer, there are certain limitations.

1. Spectrum™Data Federation does not support unified writes tomultiple connections simultaneously.You can write to only a single data source at a time.

2. Creation of tables while writing to a database source using a Spectrum™ Data Federationconnection is not supported. If you click Create Table while using a Spectrum™ Data Federationconnection, a runtime exception is displayed.

3. Dropping and recreating any table while writing to a database using a Spectrum™ Data Federationconnection is not supported.

4. Delete and truncate operations are not supported while writing to a database using Spectrum™

Data Federation. Only insert and update operations are supported.5. Spectrum™ Data Federation performs a commit after completion of each batch insert. TheBatch

count to commit cannot be changed to any other value other than 1.

In the Runtime tab of theWrite to DB stage in the Enterprise Designer, the Batch count tocommit field can only have the value 1 for Spectrum™ Data Federation.

A batch consists of 1000 records by default.Note:

6. For batch inserts, the featureCommit at the end is not supported. This feature refers to performinga commit after all batches are complete. In Spectrum™ Data Federation, a commit is performedat the end of each batch insert completion.

7. With an Amazon SimpleDB connection, the update option is not permitted in the Write to DBstage. The insert option handles both insert and update requirements, by differentiating betweenan insert and an update using the unique value of the ItemName column, present in all AmazonSimpleDB tables.

This limitation is because an update query requires a Primary Key for each record of the tableto be updated, which is not supported by Amazon SimpleDB databases.


Limitations

Appendices

In this section

Reserved Keywords 61Supported Datatypes for Conversion Transformations 66

A - Reserved Keywords

In this section

Reserved Keywords 62

Reserved Keywords

DCBA

DATALINKCALLBEGINADDDATECALLEDBETWEENALLDAYCASCADEDBIGDECIMALALLOCATEDEALLOCATECASEBIGINTALTERDECCASTBIGINTEGERANDDECIMALCHARBINARYANYDECLARECHARACTERBLOBAREDEFAULTCHECKBOTHARRAYDELETECLOBBREAKARRAY_AGGDEREFCLOSEBYTEASDESCCOLLATEASCDESCRIBECOLUMNASENSITIVEDETERMINISTICCOMMITASYMETRICDISCONNECTCONNECTATOMICDISTINCTCONSTRAINTAUTHORIZATIONDLNEWCOPYCONTINUEDLPREVIOUSCOPYCONVERTDLURELCOMPLETEWRITECORRESPONDINGDLURLCOMPLETECREATEDLURLCOMPLETEONLYCRITERIADLURLPATHCROSSDLURLPATHONLYCURRENT_DATEDLURLPATHWRITECURRENT_TIMEDLURLSCHEMECURRENT_TIMESTAMPDLURLSEVERCURRENT_USERDLVALUECURSORDOUBLECYCLEDROPDYNAMIC

HGFE

HASGETFALSEEACHHAVINGGLOBALFETCHELEMENTHOLDGRANTFILTERELSEHOURGROUPFLOATEND

GROUPINGFORERRORFOREIGNESCAPEFREEEXCEPTFROMEXECFULLEXECUTEFUNCTIONEXISTS


Reserved Keywords

EXTERNAL

LKJI

JOINIDENTITY LANGUAGEIF LARGEIMMEDIATE LATERALIMPORT LEADINGIN LEAVEINDICATOR LEFTINNER LIKEINOUT LIKE_REGEXINPUT LIMITINSENSITIVE LOCALINSERT LOCALTIMEINT LOCALTIMESTAMPINTEGER LONGINTERSECT LOOPINTERVALINTOISISOLATION

PONM

PARAMETEROBJECTNATIONALMAKEDEPPARTITIONOFNATURALMAKENOTDEPPRECISIONOFFSETNCHARMATCHPREPAREOLDNCLOBMEMBERPRIMARYONNEWMERGEPROCEDUREONLYNOMETHOD

OPENNOCACHEMINUTEOPTIONNONEMODIFIESOPTIONSNOTMODULEORNULLMONTHORDERNUMERICMULTISETOUTOUTEROUTPUTOVEROVERLAPS

TSRQ

TABLESAVEPOINTRANGETEMPORARYSCROLLREADSTHENSEARCHREAL


Reserved Keywords

TIMESECONDRECURSIVETIMESTAMPSELECTREFERENCESTIMEZONE_HOURSENSITIVEREFERENCINGTIMEZONE_MINUTESESSION_USERRELEASETOSETRETURNTRAILINGSHORTRETURNSTRANSLATESIMILARREVOKETRANSLATIONSMALLINTRIGHTTREATSOMEROLLBACKTRIGGERSPECIFICROLLUPTRUESPECIFICTYPEROW

SQLROWSSQLEXCEPTIONSQLSTATESQLWARNINGSTARTSTATICSTRINGSUBMULTILISTSYMETRICSYSTEMSYSTEM_USER

XWVU

XMLWHENVALUEUNIONXMLAGGWHENEVERVALUESUNIQUEXMLATTRIBUTESWHEREVARCHARUNKNOWNXMLBINARYWHILEVARYINGUPDATEXMLCASTWINDOWVIRTUALUSERXMLCOMMENTWITHUSINGXMLCONCATWITHINXMLDOCUMENTWITHOUTXMLELEMENTXMLEXISTSXMLFORESTXMLITERATEXMLNAMESPACESXMLPARSEXMLPIXMLQUERYXMLSERIALIZEXMLTABLEXMLTEXTXMLVALIDATE

ZY


Reserved Keywords

YEAR


Reserved Keywords

B - SupportedDatatypes forConversionTransformations

In this section

Supported Datatypes for Conversion Transformations 67

Supported Datatypes for Conversion TransformationsSpectrum™ Data Federation supports the datatypes mentioned in the table below while applyingconversion transformations.

Table 1: Supported Datatypes for Conversion Transformations

DescriptionType

Variable length character string with a maximum length of4000

String or Varchar

Variable length binary string with a nominal maximum lengthof 8192

Varbinary

A single Unicode characterChar

A single bit, or Boolean, with two possible valuesBoolean

Numeric, integral type, signed 8-bitByte or Tinyint

Numeric, integral type, signed 16-bitShort or Smallint

Numeric, integral type, signed 32-bit.

The serial type also implies not null and has anauto-incrementing value that starts at 1. Serial types are notautomatically UNIQUE.

Integer or Serial

Numeric, integral type, signed 64-bitLong or Bigint

Numeric, integral type, arbitrary precision of up to 1000digits

Biginteger

Numeric, floating point type, 32-bit IEEE 754 floating-pointnumbers

Float or Real

Numeric, floating point type, 64-bit IEEE 754 floating-pointnumbers

Double

Numeric, floating point type, arbitrary precision of up to 1000digits

Bigdecimal or Decimal

Datetime, representing a single day (year, month, day)Date


Supported Datatypes for Conversion Transformations

DescriptionType

Datetime, representing a single time (hours, minutes,seconds, milliseconds)

Time

Datetime, representing a single date and time (year, month,day, hours, minutes, seconds, milliseconds, nanoseconds)

Timestamp


Supported Datatypes for Conversion Transformations

Notices

© 2015 Pitney Bowes Software Inc. All rights reserved. MapInfo and Group 1 Software are trademarksof Pitney Bowes Software Inc. All other marks and trademarks are property of their respectiveholders.

USPS® Notices

Pitney Bowes Inc. holds a non-exclusive license to publish and sell ZIP + 4® databases on opticaland magnetic media. The following trademarks are owned by the United States Postal Service:CASS, CASS Certified, DPV, eLOT, FASTforward, First-Class Mail, Intelligent Mail, LACSLink,NCOALink, PAVE, PLANET Code, Postal Service, POSTNET, Post Office, RDI, SuiteLink , UnitedStates Postal Service, Standard Mail, United States Post Office, USPS, ZIP Code, and ZIP + 4.This list is not exhaustive of the trademarks belonging to the Postal Service.

Pitney Bowes Inc. is a non-exclusive licensee of USPS® for NCOALink® processing.

Prices for Pitney Bowes Software's products, options, and services are not established, controlled,or approved by USPS® or United States Government. When utilizing RDI™ data to determineparcel-shipping costs, the business decision on which parcel delivery company to use is not madeby the USPS® or United States Government.

Data Provider and Related Notices

Data Products contained on this media and used within Pitney Bowes Software applications areprotected by various trademarks and by one or more of the following copyrights:© Copyright United States Postal Service. All rights reserved.© 2014 TomTom. All rights reserved. TomTom and the TomTom logo are registered trademarks ofTomTom N.V.© 1987 - 2014 HERE. All rights reserved.

Fuente: INEGI (Instituto Nacional de Estadística y Geografía)

Based upon electronic data © National Land Survey Sweden.© Copyright United States Census Bureau© Copyright Nova Marketing Group, Inc.

Portions of this program are © Copyright 1993-2007 by Nova Marketing Group Inc. All RightsReserved© Copyright Second Decimal, LLC© Copyright Canada Post Corporation

This CD-ROM contains data from a compilation in which Canada Post Corporation is the copyrightowner.© 2007 Claritas, Inc.

The Geocode Address World data set contains data licensed from the GeoNames Project(www.geonames.org) provided under the Creative Commons Attribution License ("Attribution


Copyright

http://www.geonames.org

License") located at http://creativecommons.org/licenses/by/3.0/legalcode. Your use of theGeoNames data (described in the Spectrum™ Technology Platform User Manual) is governed bythe terms of the Attribution License, and any conflict between your agreement with Pitney BowesSoftware, Inc. and the Attribution License will be resolved in favor of the Attribution License solelyas it relates to your use of the GeoNames data.

ICU Notices

Copyright © 1995-2011 International Business Machines Corporation and others.

All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software andassociated documentation files (the "Software"), to deal in the Software without restriction, includingwithout limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of theSoftware, and to permit persons to whom the Software is furnished to do so, provided that the abovecopyright notice(s) and this permission notice appear in all copies of the Software and that both theabove copyright notice(s) and this permission notice appear in supporting documentation.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS ORIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,FITNESSFORAPARTICULARPURPOSEANDNONINFRINGEMENTOFTHIRDPARTYRIGHTS.IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICEBE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THISSOFTWARE.

Except as contained in this notice, the name of a copyright holder shall not be used in advertisingor otherwise to promote the sale, use or other dealings in this Software without prior writtenauthorization of the copyright holder.


Copyright

http://creativecommons.org/licenses/by/3.0/legalcode

3001 Summer Street

Stamford CT 06926-0700

www.pitneybowes.com

spectrum data federation guide - pitney...

Documents