table of contents - github pagesteiid.github.io/teiid-documents/9.2.x/teiid-documents.pdf ·...
Post on 20-May-2020
34 Views
Preview:
TRANSCRIPT
1.1
1.2
1.3
1.3.1
1.3.2
1.3.2.1
1.3.2.1.1
1.3.2.1.2
1.3.2.1.3
1.3.2.1.4
1.3.2.1.5
1.3.2.1.6
1.3.2.1.7
1.3.2.1.8
1.3.2.1.9
1.3.2.1.10
1.3.2.1.11
1.3.2.1.12
1.3.2.1.13
1.3.2.1.14
1.3.2.1.15
1.3.2.1.15.1
1.3.2.1.15.2
1.3.2.2
1.3.3
1.3.4
1.3.5
1.3.6
1.3.6.1
1.3.6.2
1.3.6.3
1.3.6.4
1.3.6.5
1.3.6.6
1.3.7
1.3.8
1.3.8.1
TableofContentsIntroduction
LegalNotice
Administrator’sGuide
InstallationGuide
DeployingVDBs
DeployingVDBDependencies
AccumuloDataSources
AmazonSimpleDBDataSources
CassandraDataSources
FileDataSources
Ftp/FtpsDataSources
GoogleSpreadsheetDataSources
JDGLibraryModeDataSources
JDGHotRodDataSources
JDBCDataSources
LDAPDataSources
MongoDBDataSources
PhoenixDataSources
SalesforceDataSources
SolrDataSources
WebServiceDataSources
KerberoswithRESTbasedServices
OAuthAuthenticationWithRESTBasedServices
VDBVersioning
Logging
ClusteringinTeiid
Monitoring
PerformanceTuning
MemoryManagement
Threading
CacheTuning
SocketTransports
LOBs
OtherConsiderations
TeiidConsole
AdminShell
GettingStarted
1
1.3.8.2
1.3.8.3
1.3.8.4
1.3.8.5
1.3.8.6
1.3.9
1.3.10
1.3.11
1.3.12
1.3.13
1.4
1.4.1
1.4.2
1.4.2.1
1.4.2.2
1.4.3
1.4.4
1.4.5
1.4.6
1.5
1.5.1
1.5.1.1
1.5.1.1.1
1.5.1.1.2
1.5.1.1.3
1.5.1.1.4
1.5.1.1.5
1.5.1.1.6
1.5.1.1.7
1.5.1.2
1.5.1.3
1.5.1.4
1.5.1.4.1
1.5.1.4.2
1.5.1.4.3
1.5.1.4.4
1.5.1.4.5
1.5.1.5
1.5.1.5.1
1.5.1.5.2
Executingascriptfile
LogFileandRecordedScriptfile
DefaultConnectionProperties
HandlingMultipleConnections
InteractiveShellNuances
OtherScriptingEnvironments
SystemProperties
TeiidManagementCLI
DiagnosingIssues
MigrationGuideFromTeiid8.x
CachingGuide
ResultsCaching
MaterializedViews
ExternalMaterialization
InternalMaterialization
CodeTableCaching
TranslatorResultsCaching
HintsandOptions
ProgrammaticControl
ClientDeveloper’sGuide
JDBCSupport
ConnectingtoaTeiidServer
DriverConnection
DataSourceConnection
StandaloneApplication
WildFlyDataSource
UsingMultipleHosts
SSLClientConnections
AdditionalSocketClientSettings
PreparedStatements
ResultSetLimitations
JDBCExtensions
StatementExtensions
PartialResultsMode
Non-blockingStatementExecution
ResultSetExtensions
ConnectionExtensions
UnsupportedJDBCMethods
UnsupportedClassesandMethodsin"java.sql"
UnsupportedClassesandMethodsin"javax.sql"
2
1.5.2
1.5.2.1
1.5.2.2
1.5.2.3
1.5.2.4
1.5.3
1.5.3.1
1.5.4
1.5.5
1.5.6
1.5.7
1.5.8
1.5.9
1.5.9.1
1.5.10
1.5.11
1.5.12
1.5.12.1
1.5.12.2
1.5.12.3
1.5.12.4
1.6
1.6.1
1.6.1.1
1.6.1.1.1
1.6.1.1.2
1.6.1.2
1.6.1.2.1
1.6.1.3
1.6.1.3.1
1.6.1.4
1.6.2
1.6.2.1
1.6.2.1.1
1.6.2.1.2
1.6.2.2
1.6.2.2.1
1.6.2.2.2
1.6.2.2.3
1.6.2.2.4
ODBCSupport
InstallingtheODBCDriverClient
ConfiguringtheDataSourceName(DSN)
DSNLessConnection
ODBCConnectionProperties
ODataSupport
ODataVersion4.0Support
UsingTeiidwithHibernate
UsingTeiidwithEclipseLink
GeoServerIntegration
QGISIntegration
Reauthentication
ExecutionProperties
XMLextensions
SETStatement
SHOWStatement
Transactions
LocalTransactions
RequestLevelTransactions
UsingGlobalTransactions
Restrictions
Developer’sGuide
DevelopingJEEConnectors
ConnectorEnvironmentSetup
BuildEnvironment
ArchetypeTemplateConnectorProject
ImplementingtheTeiidFramework
ra.xmlfileTemplate
PackagingtheAdapter
AddingDependentLibraries
DeployingtheAdapter
TranslatorDevelopment
EnvironmentSetup
Settingupthebuildenvironment
ArchetypeTemplateTranslatorProject
ImplementingtheFramework
CachingAPI
CommandLanguage
ConnectionstoSource
DependentJoinPushdown
3
1.6.2.2.5
1.6.2.2.6
1.6.2.2.7
1.6.2.2.8
1.6.2.2.9
1.6.2.3
1.6.2.4
1.6.2.5
1.6.2.5.1
1.6.2.6
1.6.3
1.6.3.1
1.6.3.2
1.6.3.2.1
1.6.4
1.6.5
1.6.6
1.6.7
1.6.8
1.6.8.1
1.7
1.7.1
1.7.2
1.8
1.8.1
1.8.2
1.8.2.1
1.8.2.2
1.8.2.3
1.8.2.4
1.8.2.5
1.8.2.6
1.8.2.7
1.8.2.8
1.8.3
1.8.3.1
1.8.3.2
1.8.3.3
1.8.3.4
1.8.3.4.1
ExecutingCommands
ExtendingtheExecutionFactoryClass
LargeObjects
TranslatorCapabilities
TranslatorProperties
ExtendingTheJDBCTranslator
DelegatingTranslator
Packaging
AddingDependentModules
Deployment
UserDefinedFunctions
SourceSupportedFunctions
SupportforUser-DefinedFunctions(Non-Pushdown)
ArchethypeTemplateUDFProject
AdminAPI
CustomLogging
RuntimeUpdates
CustomMetadataRepository
PreParser
ArchethypeTemplatePreParserProject
EmbeddedGuide
LogginginTeiidEmbedded
SecureEmbeddedwithPicketBox
ReferenceGuide
DataSources
VirtualDatabases
DevelopingaVirtualDatabase
DDLVDB
UsingXML&DDL
VDBProperties
SchemaObjectDDL
MultiSourceModels
MetadataRepositories
RESTServiceThroughVDB
SQLSupport
Identifiers
Expressions
Criteria
ScalarFunctions
NumericFunctions
4
1.8.3.4.2
1.8.3.4.3
1.8.3.4.4
1.8.3.4.5
1.8.3.4.6
1.8.3.4.7
1.8.3.4.8
1.8.3.4.9
1.8.3.4.10
1.8.3.4.11
1.8.3.4.12
1.8.3.4.13
1.8.3.4.14
1.8.3.5
1.8.3.5.1
1.8.3.5.2
1.8.3.5.3
1.8.3.5.4
1.8.3.5.5
1.8.3.5.5.1
1.8.3.5.5.2
1.8.3.5.5.3
1.8.3.5.5.4
1.8.3.5.6
1.8.3.5.7
1.8.3.5.8
1.8.3.5.9
1.8.3.5.10
1.8.3.5.11
1.8.3.5.12
1.8.3.6
1.8.3.6.1
1.8.3.6.2
1.8.3.6.3
1.8.3.6.4
1.8.3.7
1.8.3.7.1
1.8.3.7.2
1.8.3.8
1.8.3.8.1
StringFunctions
Date_TimeFunctions
TypeConversionFunctions
ChoiceFunctions
DecodeFunctions
LookupFunction
SystemFunctions
XMLFunctions
JSONFunctions
SecurityFunctions
SpatialFunctions
MiscellaneousFunctions
NondeterministicFunctionHandling
DMLCommands
SetOperations
Subqueries
WITHClause
SELECTClause
FROMClause
XMLTABLE
ARRAYTABLE
OBJECTTABLE
TEXTTABLE
WHEREClause
GROUPBYClause
HAVINGClause
ORDERBYClause
LIMITClause
INTOClause
OPTIONClause
DDLCommands
TempTables
AlterView
AlterProcedure
AlterTrigger
XMLSELECTCommand
QueryStructure
DocumentGeneration
Procedures
ProcedureLanguage
5
1.8.3.8.2
1.8.3.8.3
1.8.3.9
1.8.4
1.8.4.1
1.8.4.2
1.8.4.3
1.8.4.4
1.8.5
1.8.5.1
1.8.6
1.8.6.1
1.8.6.2
1.8.6.3
1.8.6.4
1.8.6.5
1.8.7
1.8.7.1
1.8.7.2
1.8.7.3
1.8.7.4
1.8.8
1.8.8.1
1.8.8.2
1.8.9
1.8.9.1
1.8.9.2
1.8.9.3
1.8.9.4
1.8.9.5
1.8.9.6
1.8.9.7
1.8.9.8
1.8.9.9
1.8.9.10
1.8.9.10.1
1.8.9.10.2
1.8.9.10.3
1.8.9.10.4
1.8.9.10.5
VirtualProcedures
UpdateProcedures
Comments
Datatypes
SupportedTypes
TypeConversions
SpecialConversionCases
EscapedLiteralSyntax
UpdatableViews
preservedTable
TransactionSupport
AutoCommitTxnExecutionProperty
UpdatingModelCount
JDBCandTransactions
TransactionalBehaviorwithJBossDataSourceTypes
LimitationsandWorkarounds
DataRoles
Permissions
RoleMapping
XMLDefinition
Customizing
SystemSchema
SYS
SYSADMIN
Translators
AmazonSimpleDBTranslator
ApacheAccumuloTranslator
ApacheSOLRTranslator
CassandraTranslator
DelegatingTranslators
FileTranslator
GoogleSpreadsheetTranslator
InfinispanHotRodTranslator
InfinispanLibraryModeTranslator
JDBCTranslators
ActianVectorTranslator
ApachePhoenixTranslator
ClouderaImpalaTranslator
DB2Translator
DerbyTranslator
6
1.8.9.10.6
1.8.9.10.7
1.8.9.10.8
1.8.9.10.9
1.8.9.10.10
1.8.9.10.11
1.8.9.10.12
1.8.9.10.13
1.8.9.10.14
1.8.9.10.15
1.8.9.10.16
1.8.9.10.17
1.8.9.10.18
1.8.9.10.19
1.8.9.10.20
1.8.9.10.21
1.8.9.10.22
1.8.9.10.23
1.8.9.10.24
1.8.9.10.25
1.8.9.10.26
1.8.9.10.27
1.8.9.10.28
1.8.9.10.29
1.8.9.10.30
1.8.9.10.31
1.8.9.11
1.8.9.12
1.8.9.13
1.8.9.14
1.8.9.15
1.8.9.16
1.8.9.17
1.8.9.18
1.8.9.19
1.8.9.20
1.8.9.21
1.8.9.22
1.8.9.23
1.8.10
GreenplumTranslator
H2Translator
HiveTranslator
HSQLTranslator
InformixTranslator
IngresTranslators
IntersystemsCacheTranslator
JDBCANSITranslator
JDBCSimpleTranslator
MetaMatrixTranslator
MicrosoftAccessTranslators
MicrosoftSQLServerTranslator
ModeShapeTranslator
MySQLTranslators
NetezzaTranslator
OracleTranslator
OSISoftPITranslator
PostgreSQLTranslator
PrestoDBTranslator
RedshiftTranslator
SAPHanaTranslator
SybaseIQTranslator
SybaseTranslator
TeiidTranslator
TeradataTranslator
VerticaTranslator
JPATranslator
LDAPTranslator
LoopbackTranslator
MicrosoftExcelTranslator
MongoDBTranslator
ObjectTranslator
ODataTranslator
ODataV4Translator
SwaggerTranslator
OLAPTranslator
SalesforceTranslators
SAPGatewayTranslator
WebServicesTranslator
FederatedPlanning
7
1.8.10.1
1.8.10.2
1.8.10.3
1.8.10.4
1.8.10.5
1.8.10.6
1.8.10.7
1.8.10.8
1.8.11
1.8.11.1
1.8.11.2
1.8.11.3
1.8.11.4
1.8.12
1.9
1.9.1
1.9.2
1.9.3
1.9.4
1.9.5
1.9.6
1.9.7
1.9.8
1.9.9
PlanningOverview
QueryPlanner
QueryPlans
FederatedOptimizations
SubqueryOptimization
XQueryOptimization
FederatedFailureModes
ConformedTables
Architecture
Terminology
DataManagement
QueryTermination
Processing
BNFforSQLGrammar
SecurityGuide
LoginModules
TeiidServerTransportSecurity
JDBC/ODBCSSLconnectionusingself-signedSSLcertificates
DataSourceSecurity
KerberossupportthroughGSSAPI
CustomAuthorizationValidator
SAMLBasedSecurityForOData
OAuth2BasedSecurityForODataUsingKeyCloak
SAMLBasedSecurityForODataUsingKeyCloak
8
LegalNotice
ContributetoTeiidDocumentation
Thepagesthemselveshavecommentingenabled.Youshouldbeabletoclickonanytextareatoaddacomment.
ThedocumentationprojectishostedonGitHubat(teiid/teiid-documents).
ForsimplechangesyoucanjustusetheonlineeditingcapabilitiesofGitHubbynavigatingtotheappropriatesourcefileandselectingfork/edit.
Forlargerchangesfollowthese3steps:
Step.1clonethesources
gitclonegit@github.com:teiid/teiid-documents.git
Step.2doedit
Useanytexteditortoedittheadocfiles,AsciiDocSyntaxQuickReferencecanhelpyouinAsciiDocSyntax.
Step.3submityourchange
Onceyoufinishedthissteps,thecontentofdocumentswillupdatedautomatically.
Testlocally
Youmayneedtestlocally,tomakesurethechangesarecorrect,todothisinstallgitbook,thenexecutethefollowingcommandsfromthecheckoutlocation:
$gitbookinstall
$gitbookserve-w
Onceabovecommandsexecutessuccessfully,thehttpformatdocumentcanbetestlocallyviahttp://localhost:4000/.
Introduction
9
Generatehtml/pdf/epub/mobi
Youmaylocallycreaterenderedformsofthedocumentation.Todothisinstallgitbookandebook-convert,thenexecutethefollowingcommandsfromthecheckoutlocation:
$gitbookbuild./teiid-documents
$gitbookpdf./teiid-documents.pdf
$gitbookepub./teiid-documents.epub
$gitbookmobi./teiid-documents.mobi
Onceabovecommandsexecutessuccessfully,theteiid-documentsfolder,teiid-documents.pdf,teiid-documents.epub,andteiid-documents.mobiwillbegenerated.
Introduction
10
LegalNotice
1801VarsityDriveRaleigh,NC27606-2072USAPhone:+19197543700Phone:8887334281Fax:+19197543701POBox13588ResearchTrianglePark,NC27709USA
Copyright©2016byRedHat,Inc.Thiscopyrightedmaterialismadeavailabletoanyonewishingtouse,modify,copy,orredistributeitsubjecttothetermsandconditionsoftheGNULesserGeneralPublicLicense,aspublishedbytheFreeSoftwareFoundation.
RedHatandtheRedHat"ShadowMan"logoareregisteredtrademarksofRedHat,Inc.intheUnitedStatesandothercountries.
Allothertrademarksreferencedhereinarethepropertyoftheirrespectiveowners.
TheGPGfingerprintofthesecurity@redhat.comkeyis:
CA2086862BD69DFC65F6ECC4219180CDDB42A60E
LegalNotice
11
Administrator’sGuideThisguideisintendedforanyuserwhoassumesroleofadeveloper/administratorofTeiidinstance.ThisguideguidesuserthroughinstallationofTeiidServer,configurationofdifferentservicesanddeploymentofTeiidartifactssuchasVDBs.BeforeonecandelveintoTeiiditisveryimportanttolearnfewbasicconstructsofTeiid,likewhatisVDB?whatisModel?etc.Forthatpleasereadtheshortintroductionherehttp://teiid.jboss.org/basics/
Administrator’sGuide
12
InstallationGuideTeiidneedstobeinstalledintoanexistingWildFly10.0.0installation.
Note Teiidprovidesanembeddedkit,howeveritshouldbeconsideredatechpreviewasitsAPIswilllikelyevolveandthereissparsedocumentation.
StepstoinstallTeiid
DownloadtheWildFlyapplicationserver.Installtheserverbyunzippingintoaknownlocation.Ex:/apps/jboss-install
Note YoumayalsochoosetouseanexistingASinstallation.HoweverifapreviousversionofTeiidwasalreadyinstalled,youmustremovetheoldTeiiddistributionartifactsbeforeinstallingthenewversion.
DownloadTeiid.UnzipthedownloadedartifactinsidetheWildFlyinstallation.Teiid9.2directorystructurematchesWildFlydirectly-itisjustanoverlay.ThiswilladdnecessarymodulesandconfigurationfilestoinstallTeiidinWildFly10.0.0inbothStandaloneandDomainmodes.Teiidprovidesseparateconfigurationfilesforbothstandalonemodeanddomainmode.BasedonmodetypeyouselectedtorunWildFly10.0.0,youmayhavetorunaCLIscripttocompletetheTeiidinstallation.
The"Domain"moderecommendedinaclusteredenvironmenttotakeadvantageofclusteredcachingandclustersafedistributionofevents.Teiid’sdefaultconfigurationforDomainmodethroughCLIscriptconfiguredforhighavailabilityandclusteredcaching.
StandaloneMode
ifyouwanttostartthe"standalone"profile,executethefollowingcommand
<jboss-install>/bin/standalone.sh-c=standalone-teiid.xml
InstallingTeiidusingCLIscript
TheaboveisstartingWildFlyinaseparateTeiidspecificconfigurationthatisbasedstandalone.xml.However,ifyoualreadyworkingwithapredefinedconfigurationforexampledefaultstandalone.xmlandwouldliketoinstallTeiidintothatconfiguration,thenyoucanexecutethefollowingJBossCLIscript.First,starttheserver
<jboss-install>/bin/standalone.sh
theninaseparateconsolewindowexecute
<jboss-install>/bin/jboss-cli.sh--file=bin/scripts/teiid-standalone-mode-
install.cli
thiswillinstallTeiidsubsystemintotherunningconfigurationoftheWildFly10.0.0instandalonemode.
Note:Ifyouareusingstandalonehaorstandalonefull-ha,youshouldusetheteiid-standalone-ha-mode-install.cliscriptinstead.
DomainMode
InstallationGuide
13
Tostarttheserverin"Domain"mode,installWildFly10.0.0andTeiid9.2onalltheserversthataregoingtobepartofthecluster.Selectoneoftheserversasthe"master"domaincontroller,therestoftheserverswillbeslavesthatconnecttothe"master"domaincontrollerforalltheadministrativeoperations.PleaserefertoWildFly10.0.0provideddocumentationforfulldetails.
Onceyouconfiguredalltheservers,startthe"master"nodewithfollowingcommand
<jboss-install>/bin/domain.sh
andon"slave"nodes
<jboss-install>/bin/domain.sh
Theslavenodesfetchtheirdomainconfigurationfromthe"master"node.
Oncealltheserversareup,completetheinstallationtorunindomainmodebyexecutingthefollowingcommandagainstthe"master"node.Notethatthisonlyneedstoberunonceperdomain(i.e.cluster)install.ThisscriptwillinstallTeiidinthehaandfull-haprofiles.Itwillalsore-configuremain-server-grouptostartthehaprofile.Onceindomainmode,youcannotstaticallydeployresourcesbydroppingtheminthedomain/deploymentsfolder,sothisscriptwilldeploythedefaultresources(file,ldap,salesforceandwsconnectors)usingtheCLIinterface.
<jboss-install>/bin/jboss-cli.sh--file=scripts/teiid-domain-mode-install.cli
Thatsit!.WildFlyandTeiidarenowinstalledandrunning.Seebelowinstructionstocustomizevariousothersettings.
OnceVDBshavebeendeployed,userscannowconnecttheirJDBCapplicationstoTeiid.IfyouneedhelponconnectingyourapplicationtoTeiidusingJDBCcheckouttheClientDeveloper’sGuide.
DirectoryStructureExplained
ThisshowsthecontentsoftheTeiid9.2deployment.ThedirectorystructureisexactlythesameunderanyJBossprofile.
DirectoryStructure
/bin
/scripts
/docs
/teiid
/datsources
/schema
/examples
/domain
/configuration
/modules
/system
/layers
/base
/org/jboss/teiid/*
/standalone
/configuration
standalone-teiid.xml
InstallationGuide
14
Name Description
bin/scripts ContainsinstallationandutilityCLIscriptsforsettingupTeiidindifferentconfigurations.
docs/teiid Containsdocuments,examples,sampledatasourceXMLfragmentsandschemafiles.
/standalone/configurationstandalone-teiid.xml-MasterconfigurationfilefortheTeiidsystem.ThisfilecontainstheTeiidsubsystem,inadditiontothestandardWildFlywebprofilesubsystems
/domain/configuration/ -
/modules/system/layers/base/org/jboss/teiid/* ThisdirectorycontainstheTeiidmodulesforWildFly10.0.0system
/modules/system/layers/base/org/jboss/teiid/client
ThisdirectorycontainsTeiidclientlibraries.IthastheTeiidJDBCdriverjar,"teiid-9.2.0.Final-jdbc.jar",andalsocontains"teiid-hibernate-dialect-9.2.0.Final.jar"thatcontainsTeiid’sHibernatedialect.
\{standaloneordomain}/tmp/teiid
Thisdirectoryunderstandaloneordomain,containstemporaryfilescreatedbyTeiid.Thesearemostlycreatedbythebuffermanager.ThesefilesarenotneededacrossaVMrestart.CreationofTeiidlobvalues(forexamplethroughSQL/XML)willtypicallycreateonefileperlobonceitexceedstheallowableinmemorysizeof8KB.Inheavyusagescenarios,considerpointingthebufferdirectoryatapartitionthatisroutinelydefragmented.
\{standaloneordomain}/data/teiid-data Thisdirectoryunderstandaloneordomain,containscachedvdbmetadatafiles.Donoteditthemmanually.
InstallationGuide
15
DeployingVDBsAVDBistheprimarymeanstodefineaVirtualDatabaseinTeiid.AusercancreateaVDBusingTeiidDesigner-http://www.jboss.org/teiiddesigner/-orfollowtheinstructionsintheReferenceGuidetocreateaVDBwithoutTeiidDesigner.
Onceyouhavea"VDB"builtitcanbedeployed/undeployedinTeiidruntimeindifferentways.
WarningIfVDBversioningisnotusedtogivedistinctversionnumbers,overwritingaVDBofthesamenamewillterminateallconnectionstotheoldVDB.ItisrecommendedthatVDBversioningbeusedforproductionsystems.
Caution RemovinganexistingVDBwillimmediatelycleanupVDBfileresources,andwillautomaticallyterminateexistingsessions.
CautionTheruntimenamesofdeployedVDBartifactsmusteitherbe*.vdbforazipfileor*-vdb.xmlforanxmlfileor-vdb.ddlforDDLfile.FailuretonamethedeploymentproperlywillresultinadeploymentfailureastheTeiidsubsystemwillnotknowhowtoproperlyhandletheartifact.
Tip ifyouhaveexistingVDBincombinationof*.vdbor-vdb.xmlformat,youcanmigratetoallDDLversionusingthe"teiid-convert-vdb.bat"or"teiid-convert-vdb.sh"utilityinthe"bin"directoryoftheinstallation.
DirectFileDeployment
CopytheVDBfileintothe
<jboss-install>/standalone/deployments
directory.ThencreateanemptymarkerfilewithsamenameastheVDBwithextension".dodeploy"inthesamedirectory.Forexample,ifyourvdbnameis"enterprise.vdb",thenmarkerfilenamemustbe"enterprise.vdb.dodeploy".MakesurethattherearenootherVDBfileswiththesamename.IfaVDBalreadyexistswiththesamename,thenthisVDBwillbereplacedwiththenewVDB.ThisisthesimplestwaytodeployaVDB.Thisismostlydesignedforquickdeploymentduringdevelopment,whentheTeiidserverisavailablelocallyonthedeveloper’smachine.
Note ThisonlyworksintheStandalonemode.ForDomainmode,youmustuseoneoftheotheravailablemethods.
AdminConsoleDeployment(Web)
Usetheadminwebconsoleat:
http://<host>:<port>/console
MoredetailsforthiscanbefoundintheAdminConsoleVDBdeploymentsection.ThisistheeasiestwaytodeployaVDBtoaremoteserver.
CLIbasedDeploymentWildFly10.0.0providescommandlineinterface(CLI)fordoinganykindofadministrativetask.Execute
DeployingVDBs
16
bin/jboss-cli.sh--connect
commandandrun
#instandalonemode
deploy/path/to/my.vdb
#indomainmode
deploy/path/to/my.vdb--server-groups=main-server-group
todeploytheVDB.Notethatindomainmode,youneedtoeitherselectaparticular"server-group"orallavailableservergroupsaredeploymentoptions.CheckoutCLIdocumentationformoregeneralusageoftheCLI.
AdminShellDeploymentTeiidprovidesagroovybasedAdminShellscriptingtool,whichcanbeusedtodeployaVDB.Seethe"deploy"method.ConsulttheAdminShelldocumentationformoreinformation.NotethatusingtheAdminShellscripting,youcanautomatedeploymentofartifactsinyourenvironment.WhenusingAdminShell,indomainmode,theVDBisdeployedtoalltheavailableservers.
AdminAPIDeploymentTheAdminAPI(lookinorg.teiid.adminpi.*)providesJavaAPImethodsthatletsauserconnecttoaTeiidruntimeanddeployaVDB.IfyouneedtoprogramaticallydeployaVDBusethismethod.ThismethodispreferableforOEMusers,whoaretryingtoextendtheTeiid’scapabilitiesthroughtheirapplications.WhenusingAdminAPI,indomainmode,theVDBisdeployedtoalltheservers.
DeployingVDBs
17
DeployingVDBDependenciesApartfromdeployingtheVDB,theuserisalsoresponsibleforprovidingallthenecessarydependentlibraries,configurationforcreatingthedatasourcesthatareneededbythemodels(schemas)definedin"META-INF/vdb.xml"fileinsideyourVDB.Forexample,ifyouaretryingtointegratedatafromOracleRDBMSandFilesourcesinyourVDB,thenyouareresponsibleforprovidingtheJDBCdriverfortheOraclesourceandanynecessarydocumentsandconfigurationthatareneededbytheFileTranslator.
DatasourceinstancesmaybeusedbysingleVDB,ormaybesharedwithasotherVDBsorotherapplications.Considersharingconnectionstodatasourcesthathaveheavy-weightandresourceconstrained.
WiththeexceptionofJDBCsources,othersupporteddatasourceshaveacorrespondingJCAconnectorintheTeiidkit.Eitherdirectlyeditthestandalone-teiid.xmloruseCLItocreatetherequireddatasourcesbytheVDB.Exampleconfigurationsareprovidedforallthesourcesin"<jboss-install>/docs/teiid/datasources"directory.NotethatintheDomainmode,youmustuseCLIoradmin-consoleorAdminShelltoconfigurethedatasources.
Somedatasourcesmaycontainpasswordsorothersensitiveinformation.SeetheWIKIarticleEncryptingDataSourcePasswordstonotstorepasswordsinplaintext.
OncetheVDBanditsdependenciesaredeployed,thenclientapplicationscanconnectusingtheJDBCAPI.Ifthereareanyerrorsinthedeployment,aconnectionattemptwillnotbesuccessfulandamessagewillbelogged.Youcanusetheadmin-consoletoolorcheckthelogfilesforerrorsandcorrectthembeforeproceeding.CheckClientDeveloper’sGuideonhowtouseJDBCtoconnecttoyourVDB.
DeployingVDBDependencies
18
ApacheAccumuloDataSourcesAccumulodatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreateaAccumulodatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutethefollowingcommandusingtheCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid:add(jndi-name=java:/accumul
o-ds,class-name=org.teiid.resource.adapter.accumulo.AccumuloManagedConnectionFactory,enabled=true,use-java-c
ontext=true)
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid/config-properties=ZooKeeper
ServerList:add(value=localhost:2181)
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid/config-properties=Username:
add(value=user)
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid/config-properties=Password:
add(value=password)
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid/config-properties=InstanceN
ame:add(value=instancename)
/subsystem=resource-adapters/resource-adapter=accumulo/connection-definitions=teiid/config-properties=Roles:add
(value=public)
/subsystem=resource-adapters/resource-adapter=accumulo:activate
runbatch
AllthepropertiesthataredefinedontheRARfileare
PropertyName Description Required Default
ZooKeeperServerList
Acommaseparatedlistofzookeeperserverlocations.Eachlocationcancontainanoptionalport,oftheformathost:port
true none
Username ConnectionUser’sName true none
Password ConnectionUser’spassword true none
InstanceName Accumuloinstancename true none
Rolesoptionalvisibilityforuser,supplymultiplewithcommaseparated
false none
TofindoutallthepropertiesthataresupportedbythisAccumuloConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=accumulo)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/accumulo"directoryunder"resource-adapters"subsystem.Shutdowntheserver
DeployingVDBDependencies
19
beforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
20
AmazonSimpleDBDataSourcesSimpleDBdatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreateaSimpleDBdatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutethefollowingcommandusingtheCLIonceyouconnectedtotheServer.Makesureyouprovidethecorrectaccesskeys.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=simpledb/connection-definitions=simpledbDS:add(jndi-name=java:/si
mpledbDS,class-name=org.teiid.resource.adapter.simpledb.SimpleDBManagedConnectionFactory,enabled=true,use-ja
va-context=true)
/subsystem=resource-adapters/resource-adapter=simpledb/connection-definitions=simpledbDS/config-properties=Acce
ssKey:add(value=xxx)
/subsystem=resource-adapters/resource-adapter=simpledb/connection-definitions=simpledbDS/config-properties=Secr
etAccessKey:add(value=xxx)
/subsystem=resource-adapters/resource-adapter=simpledb:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisSimpleDBConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=simpledb)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/simpledb"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
21
CassandraDataSourcesCassandradatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreateaCassandradatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutethefollowingcommandusingtheCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=cassandra/connection-definitions=cassandraDS:add(jndi-name=java:/
cassandraDS,class-name=org.teiid.resource.adapter.cassandra.CassandraManagedConnectionFactory,enabled=true,u
se-java-context=true)
/subsystem=resource-adapters/resource-adapter=cassandra/connection-definitions=cassandraDS/config-properties=Ad
dress:add(value=127.0.0.1)
/subsystem=resource-adapters/resource-adapter=cassandra/connection-definitions=cassandraDS/config-properties=Ke
yspace:add(value=my-keyspace)
/subsystem=resource-adapters/resource-adapter=cassandra:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisCassandraConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=cassandra)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/cassandra"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
22
FileDataSourcesFiledatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.Therearemanywaystocreatethefiledatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingtheCLIonceyouconnectedtotheServer.Makesureyouprovidethecorrectdirectorynameandotherpropertiesbelow.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=file/connection-definitions=fileDS:add(jndi-name=java:/fileDS,cl
ass-name=org.teiid.resource.adapter.file.FileManagedConnectionFactory,enabled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=file/connection-definitions=fileDS/config-properties=Parentdirect
ory:add(value=/home/rareddy/testing/)
/subsystem=resource-adapters/resource-adapter=file/connection-definitions=fileDS/config-properties=AllowParentP
aths:add(value=true)
/subsystem=resource-adapters/resource-adapter=file:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisFileConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=file)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/file"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
23
Ftp/FtpsDataSourcesFtp/FtpsdatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreatetheFtp/Ftpsdatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingtheCLIonceyouconnectedtotheServer.Makesureyouprovidethecorrectdirectorynameandotherpropertiesbelow.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
/subsystem=resource-adapters/resource-adapter=ftp:add(module=org.jboss.teiid.resource-adapter.ftp)
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS:add(jndi-name=${jndi.name}",cla
ss-name=org.teiid.resource.adapter.ftp.FtpManagedConnectionFactory,enabled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS/config-properties=ParentDirector
y:add(value="${ftp.parent.dir}")
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS/config-properties=Host:add(value
="${ftp.parent.host}")
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS/config-properties=Port:add(value
=${ftp.parent.port}")
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS/config-properties=Username:add(v
alue=${ftp.parent.username}")
/subsystem=resource-adapters/resource-adapter=ftp/connection-definitions=ftpDS/config-properties=Password:add(v
alue=${ftp.parent.password}")
/subsystem=resource-adapters/resource-adapter=ftp:activate()
TofindoutallthepropertiesthataresupportedbythisFtp/FtpsConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=ftp)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/ftp"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
24
GoogleSpreadsheetDataSources
Authenticationtoyourgoogleaccountmaybedoneintwoways.ClientLogin(requiresloginandpassword)orOAuth(morecomplicated,requiresusertosupplyrefreshtoken).
GoogleJCAconnectorisnamedteiid-connector-google.rar.Theexamplesincludeasamplegoogle.xmlfile.TheJCAconnectorhasnumberofconfig-propertiestodriveauthentication.TheJCAconnectorconnectstoexactlyonespreadsheet.
Configproperty Description
AuthMethod MethodtoaccessGoogle.ThispropertycanonlybeOAuth2.
RefreshToken UseguidebelowtoretrieveRefreshToken.RequestaccesstoGoogleDriveandSpreadsheetAPI.
SpreadsheetName RequiredpropertywithnameoftheSpreadsheetthatisdatasourceforthisconnector.
BatchSize Maximumnumberofrowsthatcanbefetchedatatime.Defaultsto4096.
GettinganOAuthRefreshTokenUsethisrequest:
https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive+https%3A%2F%2Fspreadsheets.google.com%2Ffeeds&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&client_id=217138521084.apps.googleusercontent.com
ThencopytheauthorizationcodeintofollowingPOSTrequestandrunitincommandline:
curl\--data-urlencodecode=<AUTH_CODE>\
--data-urlencodeclient_id=217138521084.apps.googleusercontent.com\
--data-urlencodeclient_secret=gXQ6-lOkEjE1lVcz7giB4Poy\
--data-urlencoderedirect_uri=urn:ietf:wg:oauth:2.0:oob\
--data-urlencodegrant_type=authorization_codehttps://accounts.google.com/o/oauth2/token
Therefreshtokenwillbeintheresponse
ImplementationDetails
GoogleTranslatorisimplementedusingGDataAPI[1](javalibraryfor[4])andGoogleVisualizationProtocol[2].AuthenticationisimplementedusingOAuth2fordevices[3].1.https://developers.google.com/google-apps/spreadsheets/2.https://developers.google.com/chart/interactive/docs/querylanguage3.https://developers.google.com/accounts/docs/OAuth2ForDevices4.https://developers.google.com/google-apps/spreadsheets
DeployingVDBDependencies
25
JBossDataGrid(JDG)LibraryModeDataSourcesJDGLibaryModedatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.ThisconnectorhasthefollowingoptionsthatcanbeconfiguredtoaccessanJDGcacherunninginlibrarymode:
UsingJNDI
Usingconfigurationfile
ItisassumedthatyouwillbeinstallingtheJDGLibraryModeEAPModulesKit,whichcanbeobtainedontheRedHat’sCustomerPortalathttps://access.redhat.com/jbossnetwork/restricted/listSoftware.html.
Configuration
AccessingCacheinLibraryMode
OneofthefollowingpropertiesisrequiredfordefininghowthecachewillbeaccessedviatheCacheManager:
PropertyName Description
CacheJndiName JNDInameusedtofindtheCacheContainer
ConfigurationFileNameForLocalCache TheJDGConfigurationxmlfilewillbeusedtocreateandconfiguretheCacheManagerasalocalcache
ConfiguringWhichCacheandPojo
ThefollowingpropertyisrequiredasitprovidesthemappingtotheJDGcacheandpojothatwillbeaccessed.
PropertyName PropertyTemplate Description
CacheTypeMap cacheName:className[:pkFieldName[:cacheKeyJavaType]]Definesthecachenameandpojocomponentsforaccessingthecache
WherecacheName-Nameofthecachetoaccess
className-Classnameofthepojothatisusedtostorethedatainthecache
pkFieldName-[Optional]identifiesthepojoclassattributethatisusedastheprimarykey.Itisrequiredforupdatesorifthepojocontainscomplexobjectsusedinrelationships(1-to-1or1-to-many).ThepkFieldNameMUSTmatchacorrespondinggetter/settermethodinthepojo
cacheKeyJavaType-[Optional]identifyprimarykeyjavatypewhendifferentthanclassattributetype
Thefollowingconfiguresthemodulethatcontainsthepojostoredinthecache:
PropertyName Req. Description
module N SpecifytheWildFlymodulethatcontainsthecacheclassesthatweredefinedinCacheTypeMap
DeployingVDBDependencies
26
PojoJar
Thepojoclassistheobjectthatwillbeusedtostorethedatainthecache.Itshouldbebuiltaccordingly:
Totakeadvantageofthecachebeingindexedenabled,shouldannotatetheclass.See[JDGdocumentation|https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Grid/6.6/html-single/Infinispan_Query_Guide/index.html#chap-Annotating_Objects_and_Querying]
Theclassshouldbepackagedintoajarsothatitcanbedeployedasamodule
Toconfiguretheuseofthepojo,dothefollowing:
Deploythepojojarasamoduleinthejboss-asserver.Thendefinethe"lib"propertyinthe-vdb.xmlandassignthecorrectmodulename.Thiscanbedoneusingthefollowingtemplate:
<propertyname="lib"value="{pojo_module_name}"></property>
TheJDGcommonsmodule,org.infinispan.commons,slot="jdg-6.x"orslotforversioninstalled,needstohavethepojodependencyadded:
<modulename="{pojo_module_name}"export="true"/>
UsingCacheforExternalMaterializationThefollowingaretheadditionalpropertiesthatneedtobeconfiguredifusingtheRemoteCacheforexternalmaterialization:
PropertyName Req. Description
StagingCacheName Y Cachenameforthestagingcacheusedinmaterialization
AliasCacheName YCachenameforthealiascacheusedintrackingaliasingofthecachesusedinmaterialization
ServerConfiguration
Therearemanywaystocreatethedatasource;usingCLI,AdminShell,admin-consoleetc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingCLIonceyouconnectedtotheServer.Makesureyouprovidethecorrectdirectorynameandotherpropertiesbelow.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=infinispan/connection-definitions=infinispanDS:add(jndi-name=java
:/infinispanDS,class-name=org.teiid.resource.adapter.infinispan.libmode.InfinispanManagedConnectionFactory,en
abled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=infinispan/connection-definitions=infinispanDS/config-properties=
CacheTypeMap:add(value=trades:org.somewhere.Trade;tradeId)
/subsystem=resource-adapters/resource-adapter=infinispan/connection-definitions=infinispanDS/config-properties=
Module:add(value=org.somewhere)
/subsystem=resource-adapters/resource-adapter=infinispan/connection-definitions=infinispanDS/config-properties=
CacheJndiName:add(value=java:/myCache)
runbatch
DeployingVDBDependencies
27
TofindoutallthepropertiesthataresupportedbythisInfinispanConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=infinispan)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin*"<jboss-install>/docs/teiid/datasources/infinispan"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
NoteAvoidClassloadingIssues:IfyouareusingaservletorothertypeofwebapplicationtocreatetheDefaultCacheManagerforthecache,besurenottoincludetheInfinispanjardependenciesintheapplication,butaddtheirmoduledependencies.
DeployingVDBDependencies
28
JBossDataGrid(JDG)HotRodDataSourcesJDGcachesrunninginclient/servermodecanuseaTeiidspecificJCAconnectorforaccessingasadatasource,whichisdeployedintoWildFly10.0.0duringinstallation.ThisconnectorcanbeconfiguredtosupporttheaccessingofaremoteJDGcacheusingthetheHotRodclient.
ItisassumedthatyouwillbeinstallingtheJDGHotrodClientEAPModulesKit,whichcanbeobtainedontheRedHat’sCustomerPortalathttps://access.redhat.com/jbossnetwork/restricted/listSoftware.html.
Tohandlepojotypesinthesamecache,whereeachpojoobjectcanbeaccessedbyperforminga"get(key)"onthecache,willrequireadifferentconfiguredresource-adapterforeachpojotype.
TherearetwooptionsforhowtheJDGschemacanbeconfiguredintheconnector;protobufannotationsorprotobuf(.proto)filewithmarshaller(s).Thefollowingaretherequirements.
Requirement
(option1)Minimum,JDG6.2-thisrequiresyouprovideaprotobufdefinitionfileandpojomarshaller(s)forthepojotoconfiguretheJDGschema
(option2)Minimum,JDG6.6-thiscanbeusedwhenthepojohasdefinedprotobufannotationswhichareusedtoconfiguretheJDGschema.
SeeInfinispanHotRodTranslatorfordetailsonhowthethemetadatawillbeexposedorcanbemanuallyconfiguredbasedontheschemathat’sdefinedforthisdatasource.
Configuration
Therearemanydifferentwaystoconfigurethisresource-adapterbasedondifferentusagescenarios:
complexityofthepojo(i.e,pojocontainsotherpojo’s).
usingeitherpojoprotobufannotationsorprotobufdefinitionfiles,toconfiguretheJDGschema
wantingtheenableHotRodclienttouseSSL
utilizingtheJDGcacheformaterialization
AccessingHotRodRemoteCache
OneofthefollowingpropertiesisrequiredfordefininghowtheRemoteCacheManagerwillbecreated/accessed:
PropertyName Req. PropertyTemplate Description
CacheJndiName N JNDInametofindtheCacheContainer
RemoteServerList N host:port\[;host:port….\]
Specifythehostandportsthatwillbeclusteredtogethertoaccessthecaches
HotRodClientPropertiesFile N TheHotRodproperties
DeployingVDBDependencies
29
cache
ConfiguringWhichCacheandPojo
ThefollowingpropertyisrequiredasitprovidesthemappingtotheJDGcacheandpojothatwillbeaccessed.
PropertyName PropertyTemplate Description
CacheTypeMap cacheName:className[:pkFieldName[:cacheKeyJavaType]]Definesthecachenameandpojocomponentsforaccessingthecache
WherecacheName-Nameofthecachetoaccess
className-Classnameofthepojothatisusedtostorethedatainthecache
pkFieldName-[Optional]identifiesthepojoclassattributethatisusedastheprimarykey.Itisrequiredforupdatesorifthepojocontainscomplexobjectsusedinrelationships(1-to-1or1-to-many).ThepkFieldNameMUSTmatchacorrespondinggetter/settermethodinthepojo
cacheKeyJavaType-[Optional]identifyprimarykeyjavatypewhendifferentthanclassattributetype
JDGSchemausingProtobufDefinitionandMarshaller(s)
Thefollowingpropertiesarerequiredwhentheprotobufdefinitionfile(.proto)andthepojomarshaller(s)arebeingusedtoconfiguretheJDGschema:
PropertyName Req. PropertyTemplate Description
ProtobufDefinitionFile Y
PathtotheGoogleProtobufdescriptorfilethat’spackagedinajar(ex:/quickstart/addressbook.proto)
MessageMarshallers Y className:marshallerClassName\[,className:marshallerClassName,..\]
ContainsClassname(s)mappedtoitsrespectivemessagemarshaller(s)thatistoberegisteredforserialization
MessageDescriptor YMessagedescriptorpackagenameintheprotobufdescriptorfile
Thefollowingpropertyshouldbedefinedwhenusingprotobufdefinitionfileandmarshallers:
PropertyName Req. PropertyTemplate Description
module N
SpecifytheWildFlymodulethatcontainsthecacheclassesthatneedtobeloaded
JDGSchemausingPojoAnnotations
Thepojoclassistheobjectthatisusedtostorethedatainthecache.Itshouldbebuiltaccordingly:
DeployingVDBDependencies
30
Ifthepojoistobeusedtodefinetheschema,thenshouldusetheprotobufannotations.SeeJDGProtobufAnnotationsathttps://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Grid/6.6/html-single/Infinispan_Query_Guide/index.html#Custom_Fields_Indexing_with_Protobuf
Theclassshouldbepackagedintoajarsothatitcanbedeployedasamodule.
Iftherootpojo,definedinCacheTypeMapproperty,containsotherpojo(s)(e.g.,1-to-1or1-to-manyrelationship),thenthosechildpojo’smustbedefinedintheChildClassespropertysothattheycangetregisteredintheJDGschema.Thefollowingpropertymustbeconfigured:
PropertyName PropertyTemplate Description
ChildClasses className[,className,..]]commaseparatedlistofclassnamesthatindicatethechildclassestoregisterintheJDGschema
EnableHotrodClientSSL
ToenabletheHotrodclientintheresource-adaptertocommunicateusingSSL,configurethefollowingpropertiesontheresourceadapter:
PropertyName Description
TrustStoreFileName Filenameofthetruststore
TrustStorePassword TrustStorePassword
KeyStoreFileName Filenameofthekeystore
KeyStorePassword KeyStorePassword
SNIHostName [optional]SNIHostName
UsingforExternalMaterialization
ThefollowingaretheadditionalpropertiesthatneedtobeconfiguredifusingtheRemoteCacheforexternalmaterialization:
PropertyName Req. Description
StagingCacheName Y Cachenameforthestagingcacheusedinmaterialization
AliasCacheName Y
Cachenameforthealiascacheusedintrackingaliasingofthecachesusedinmaterialization.Thiscachecanbesharedwithotherconfiguredmaterializations.
Examples
Therearemanywaystocreatethedatasource,usingCLI,AdminShell,admin-consoleetc.
1stExample
DeployingVDBDependencies
31
The1stexampleisanxmlsnippetofaresource-adapterthatisusingprobufsandmarshallerstoconfiguretheJDGschema:
SampleResourceAdapterdefiningProtobufDefinitionandMarshaller
<resource-adapterid="infinispanRemQS">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.infinispan.hotrod"/>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.infinispan.hotrod.Infinis
panManagedConnectionFactory"jndi-name="java:/infinispanRemote"enabled="true"use-java-context="true"pool-name
="infinispanDS">
<config-propertyname="CacheTypeMap">
addressbook:org.jboss.as.quickstarts.datagrid.hotrod.query.domain.Person;id
</config-property>
<config-propertyname="ProtobufDefinitionFile">
/quickstart/addressbook.proto
</config-property>
<config-propertyname="MessageDescriptor">
quickstart.Person
</config-property>
<config-propertyname="Module">
com.client.quickstart.pojos
</config-property>
<config-propertyname="MessageMarshallers">org.jboss.
as.quickstarts.datagrid.hotrod.query.domain.Person:org.jboss.as.quickstarts.datagrid.hotrod.query.marshallers.P
ersonMarshaller,org.jboss.as.quickstarts.datagrid.hotrod.query.domain.PhoneNumber:org.jboss.as.quickstarts.data
grid.hotrod.query.marshallers.PhoneNumberMarshaller,org.jboss.as.quickstarts.datagrid.hotrod.query.domain.Phone
Type:org.jboss.as.quickstarts.datagrid.hotrod.query.marshallers.PhoneTypeMarshaller
</config-property>
<config-propertyname="RemoteServerList">
127.0.0.1:11322
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
2ndExample
The2ndexampleshowingapojoexamplewithannotationsandthexmlsnippetoftheresource-adapter:
SamplePojowithAnnotations
publicclassPerson{
@ProtoField(number=2,required=true)
publicStringname;
@ProtoField(number=1,required=true)
publicintid;
@ProtoField(number=3)
publicStringemail;
privateList<PhoneNumber>phones;
publicStringgetName(){
returnname;
}
publicvoidsetName(Stringname){
this.name=name;
}
publicintgetId(){
returnid;
}
publicvoidsetId(intid){
this.id=id;
}
DeployingVDBDependencies
32
publicStringgetEmail(){
returnemail;
}
publicvoidsetEmail(Stringemail){
this.email=email;
}
publicList<PhoneNumber>getPhones(){
returnphones;
}
publicvoidsetPhones(List<PhoneNumber>phones){
this.phones=phones;
}
}
SampleResourceAdapterusingPojowithannotations
<resource-adapterid="infinispanRemQSDSL">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.infinispan.dsl"/>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.infinispan.dsl.Infinispan
ManagedConnectionFactory"jndi-name="java:/infinispanRemoteDSL"enabled="true"use-java-context="true"pool-name
="infinispanRemoteDSL">
<config-propertyname="RemoteServerList">
127.0.0.1:11322
</config-property>
<config-propertyname="CacheTypeMap">
addressbook_indexed:org.jboss.as.quickstarts.datagrid.hotrod.query.domain.Perso
n;id
</config-property>
<config-propertyname="ChildClasses">
org.jboss.as.quickstarts.datagrid.hotrod.query.domain.PhoneNumber
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
3rdExample
The3rdexampleisusingtheJDGdatasourceformaterialization.
SampleResourceAdapterforexternalmaterialization
<resource-adapterid="infinispanRemQSDSL">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.infinispan.hotrod"/>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.infinispan.hotrod.Infinis
panManagedConnectionFactory"jndi-name="java:/infinispanRemoteDSL"enabled="true"use-java-context="true"pool-
name="infinispanRemoteDSL">
<config-propertyname="CacheTypeMap">
addressbook_indexed:org.jboss.as.quickstarts.datagrid.hotrod.query.domain.Perso
n;id
</config-property>
<config-propertyname="StagingCacheName">
addressbook_indexed_mat
</config-property>
<config-propertyname="AliasCacheName">
aliasCache
</config-property>
<config-propertyname="Module">
com.client.quickstart.addressbook.pojos
</config-property>
DeployingVDBDependencies
33
<config-propertyname="RemoteServerList">
127.0.0.1:11322
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
DeployingVDBDependencies
34
JDBCDataSourcesThefollowingisanexamplehighlightingconfiguringanOracledatasource.Theprocessisnearlyidenticalregardlessofthedatabasevendor.TypicallytheJDBCjarandtheconfigurationlikeconnectionURLandusercredentialschange.
Thereareconfigurationtemplatesforallthedatasourcesinthe"<jboss-install>/docs/teiid/datasources"directory.AcompletedescriptionhowadatasourcecanbeaddedintoWildFlyisalsodescribedhere.Thebelowwepresenttwodifferentwaystocreateadatasource.
DeployingasingleJDBCJarFile
FirststepinconfiguringthedatasourceisdeployingtherequiredJDBCjarfile.Forexample,ifyouaretryingtocreateaOracledatasource,firstyouneedtodeploythe"ojdbc6.jar"filefirst.ExecutefollowingcommandusingtheCLIonceyouconnectedtotheServer.
deploy/path/to/ojdbc6.jar
Tip Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallycopythis’ojdbc6.jar"tothe"<jboss-install>/standalone/deployments"directory,toautomaticallydeploywithoutusingtheCLItool.
CreatingamodulefortheDriver
Youmayalsocreateamoduletohavemorecontroloverthehandlingofthedriver.Incaseswherethedriverisnotcontainedinasinglefile,thismaybepreferabletocreatinga"uber"jarasthedependenciescanbemanagedseparately.
Creatingamoduleforadriverisnodifferentthananyothercontainermodule.Youjustincludethenecessaryjarsasresourcesinthemoduleandreferenceothermodulesasdependencies.
<modulexmlns="urn:jboss:module:1.0"name="com.mysql">
<resources>
<resource-rootpath="mysql-connector-java-5.1.21.jar"/>
</resources>
<dependencies>
<modulename="javax.api"/>
...
</dependencies>
</module>
CreateDataSourceNowthatyouhavetheJDBCdriverdeployedorthemodulecreated,itistimetocreateadatasourceusingthisdriver.TherearemanywaystocreatethedatasourceusingCLI,AdminShell,admin-consoleetc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentialsandedittheJNDInametomatchtheJNDInameyouusedinVDB.
/subsystem=datasources/data-source=oracel-ds:add(jndi-name=java:/OracleDS,driver-name=ojdbc6.jar,connection-u
rl=jdbc:oracle:thin:{host}:1521:orcl,user-name={user},password={password})
/subsystem=datasources/data-source=oracel-ds:enable
DeployingVDBDependencies
35
Thedriver-namewillmatchthenameofjarormodulethatyoudeployedforthedriver.
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin*"<jboss-install>/docs/teiid/datasources/oracle"directoryunder"datasources"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
36
LDAPDataSourcesLDAPdatasourcesuseaTeiidspecificJCAconnectorwhichisdeployedintoWildFly10.0.0duringinstallation.Therearemanywaystocreatetheldapdatasource,usingCLI,AdminShell,admin-consoleetc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=ldap/connection-
definitions=ldapDS:add(jndi-name=java:/ldapDS,class-
name=org.teiid.resource.adapter.ldap.LDAPManagedConnectionFactory,enabled=true,
use-java-context=true)
/subsystem=resource-adapters/resource-adapter=ldap/connection-
definitions=ldapDS/config-properties=LdapUrl:add(value=ldap://ldapServer:389)
/subsystem=resource-adapters/resource-adapter=ldap/connection-
definitions=ldapDS/config-properties=LdapAdminUserDN:add(value=
{cn=???,ou=???,dc=???})
/subsystem=resource-adapters/resource-adapter=ldap/connection-
definitions=ldapDS/config-properties=LdapAdminUserPassword:add(value={pass})
/subsystem=resource-adapters/resource-adapter=ldap/connection-
definitions=ldapDS/config-properties=LdapTxnTimeoutInMillis:add(value=-1)
/subsystem=resource-adapters/resource-adapter=ldap:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisLDAPConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=ldap)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin*"<jboss-install>/docs/teiid/datasources/ldap"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
Note Touseananonymousbind,settheLdapAdminUserDNandLdapAdminUserPasswordtoemptyvalues.
Tip Ifyouexperiencestaleconnectionsinthepool,youshouldenableeitherthevalidate-on-matchorthebackground-validationpoolsettings.
DeployingVDBDependencies
37
MongoDBDataSourcesMongoDBdatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreateaMongoDBdatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutethefollowingcommandusingtheCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=mongodb/connection-definitions=mongodbDS:add(jndi-name="java:/mon
goDS",class-name=org.teiid.resource.adapter.mongodb.MongoDBManagedConnectionFactory,enabled=true,use-java-co
ntext=true)
/subsystem=resource-adapters/resource-adapter=mongodb/connection-definitions=mongodbDS/config-properties=Remote
ServerList:add(value="{host}:27017")
/subsystem=resource-adapters/resource-adapter=mongodb/connection-definitions=mongodbDS/config-properties=Databa
se:add(value="{db-name}")
/subsystem=resource-adapters/resource-adapter=mongodb:activate
runbatch
AllthepropertiesthataredefinedontheRARfileare
PropertyName Description Required Default
RemoteServerListAcommaseparatedlistofserverlocations.Eachlocationcancontainanoptionalport,oftheformathost:port
Username ConnectionUser’sName false none
Password ConnectionUser’spassword false none
Database MongoDBdatabasename true none
SecurityType
MongoDBTypeofAuthenticationtobeused.Allowedvaluesare"None","SCRAM_SHA_1","MONGODB_CR","Kerberos","X509".IfyouareusingMongoDBversionlessthan3.0,MongoDBbydefaultuses"MONGODB_CR",thusthisvalueneedtobesetaccordinglyorsettoNone.
false SCRAM_SHA_1
AuthDatabase
MongoDBDatabaseNameforuserauthenticationincasewhenSecurityType'MONGODB-CR'isused.Thisisanoptionalvalue.
false none
Ssl UseSSLConnections false none
TofindoutallthepropertiesthataresupportedbythisMongoDBConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=mongodb)
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-
DeployingVDBDependencies
38
Tip install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/mongodb"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
39
PhoenixDataSourcesThefollowingisaexampleforsettingupPhoenixDataSources,whichispreconditionforApachePhoenixTranslator.InadditiontotheDataSourcessetup,thisarticlealsocovermappingPhoenixtabletoanexistingHBasetableandcreatinganewPhoenixtable.
ThereareconfigurationtemplatesforPhoenixdatasourcesinthe"<jboss-install>/docs/teiid/datasources"directory.AcompletedescriptionhowadatasourcecanbeaddedintoWildFlyisalsodescribedhere.
ConfiguringaPhoenixdatasourceinWildFly
ConfiguringaPhoenixdatasourceisnearlyidenticaltoconfiguringJDBCDataSources.ThefirststepisdeployingthePhoenixdriverjar.UsingbelowCLIcommandtodeployPhoenixdriver:
moduleadd--name=org.apache.phoenix--resources=/path/to/phoenix-[version]-client.jar--dependencies=javax.a
pi,sun.jdk,org.apache.log4j,javax.transaction.api
/subsystem=datasources/jdbc-driver=phoenix:add(driver-name=phoenix,driver-module-name=org.apache.phoenix,driver-
class-name=org.apache.phoenix.jdbc.PhoenixDriver)
TheDriverjarcanbedownloadfromphoenixdocument.
ThesecondstepsiscreatingtheDataSourcebaseonabovedeployeddriver,whichisalsolikecreatingJDBCDataSource.UsingbelowCLIcommandtocreateDataSource:
/subsystem=datasources/data-source=phoenixDS:add(jndi-name=java:/phoenixDS,driver-name=phoenix,connection-ur
l=jdbc:phoenix:{zookeeperquorumserver},enabled=true,use-java-context=true,user-name={user},password={pass
word})
/subsystem=datasources/data-source=phoenixDS/connection-properties=phoenix.connection.autoCommit:add(value=true)
PleasemakesuretheURL,Driver,andotherpropertiesareconfiguredcorrectly:
jndi-name-TheJNDInameneedtomatchtheJNDInameyouusedinVDB
driver-name-TheDrivernameneedtomatchthedriveryoudeployedinabovesteps
connection-url-TheURLneedtomatchtheHBasezookeeperquorumserver,theformatlikejdbc:phoenix[:<zookeeperquorum>[:<portnumber>][:<rootnode>]],’jdbc:phoenix:127.0.0.1:2181’isaexample
user-name/password-TheusercredentialsforPhoenixConnection
ThePhoenixConnectionAutoCommitdefaultisfalse.Setphoenix.connection.autoCommittotrueifyouwillbeexecutingINSERT/UPDATE/DELETEstatementsagainstPhoenix.
MappingPhoenixtabletoanexistingHBasetable
MappingPhoenixtabletoanexistingHBasetablehas2steps.Thefirststepisinstallingphoenix-[version]-server.jartotheclasspathofeveryHBaseregionserver.AneasywaytodothisistocopyitintotheHBaselib-formoredetailspleaserefertothephoenixdocumentation.
ThesecondstepisexecutingtheDDLtomapaPhoenixtabletoanexistingHBasetable.TheDDLcaneitherbeexecutedviaPhoenixCommandLine,orexecutedbyJDBC.
DeployingVDBDependencies
40
TheFollowingisaexampleformappinganexistingHBaseCustomerwiththefollowingstructure:
Asdepictedabove,theHBaseCustomertablehave2columnfamilies,customerandsales,andeachhas2columnqualifiers,name,city,productandamountrespectively.WecanmapthisTabletoPhoenixviaDDL:
CREATETABLEIFNOTEXISTS"Customer"("ROW_ID"VARCHARPRIMARYKEY,"customer"."city"VARCHAR,"customer"."name"
VARCHAR,"sales"."amount"VARCHAR,"sales"."product"VARCHAR)
FormoreaboutmappingPhoenixtabletoanexistingHBasetablepleaserefertothephoenixdocumentation.
CreatinganewPhoenixtable
CreatinganewPhoenixtableisjustlikemappingtoanexistingHBasetable.Phoenixwillcreateanymetadata(table,columnfamilies)thatdonotexist.SimilartotheaboveexampletheDDLtocreatethePhoenix/HBaseCustomertablewouldbe:
CREATETABLEIFNOTEXISTS"Customer"("ROW_ID"VARCHARPRIMARYKEY,"customer"."city"VARCHAR,"customer"."name"
VARCHAR,"sales"."amount"VARCHAR,"sales"."product"VARCHAR)
DefiningForeignTableinVDBFinally,weneeddefineaForeignTableinVDBthatmaptoPhoenixtable,thefollowingprinciplesshouldbeconsideredindefiningForeignTable:
nameinsourceoptioninTableusedtomatchPhoenixtablename
nameinsourceoptioninColumnusedtomatchHBaseTable’sColumns
createaprimarykeyisrecommended,theprimarykeycolumnshouldmatchPhoenixtable’sprimarykey/HBaserowid.
With"MappingPhoenixtabletoanexistingHBasetable"section’s`Customer'table,belowisaexample:
CREATEFOREIGNTABLECustomer(
PKstringOPTIONS(nameinsource'ROW_ID'),
citystringOPTIONS(nameinsource'"city"'),
namestringOPTIONS(nameinsource'"name"'),
amountstringOPTIONS(nameinsource'"amount"'),
productstringOPTIONS(nameinsource'"product"'),
CONSTRAINTPK0PRIMARYKEY(PK)
)OPTIONS(nameinsource'"Customer"',"UPDATABLE"'TRUE');
Note "Constraintviolation.Xmaynotbenull"exceptionmaythrownifupdatingatablewithoutdefiningaprimarykey.
DeployingVDBDependencies
41
DeployingVDBDependencies
42
SalesforceDataSourcesSalesforcedatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.Therearetwoversionsofthesalesforceresourceadapter-salesforce,whichcurrentlyprovidesconnectivitytothe22.0SalesforceAPIandsalesforce-34,whichprovidesconnectivitytothe34.0SalesforceAPI.Theversion22.0supporthasbeendeprecated.
IfyouneedconnectivitytoanAPIversionotherthanwhatisbuiltin,pleaseutilizetheprojecthttps://github.com/teiid/salesforcetogeneratenewresourceadapter/translatorpair.
Therearemanywaystocreatethesalesforcedatasource,usingCLI,AdminShell,admin-consoleetc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingtheCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=salesforce/connection-definitions=sfDS:add(jndi-name=java:/sfDS,
class-name=org.teiid.resource.adapter.salesforce.SalesForceManagedConnectionFactory,enabled=true,use-java-con
text=true)
/subsystem=resource-adapters/resource-adapter=salesforce/connection-definitions=sfDS/config-properties=URL:add(
value=https://www.salesforce.com/services/Soap/u/22.0)
/subsystem=resource-adapters/resource-adapter=salesforce/connection-definitions=sfDS/config-properties=username
:add(value={user})
/subsystem=resource-adapters/resource-adapter=salesforce/connection-definitions=sfDS/config-properties=password
:add(value={password})
/subsystem=resource-adapters/resource-adapter=salesforce:activate
runbatch
Thesalesforce-34connectiondefinitionconfigurationissimilartotheabove.Theresourceadapternamewouldinsteadbesalesforce-34,andtheurlwouldpointto34.0instead.
TofindoutallthepropertiesthataresupportedbythisSalesforceConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=salesforce)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin*"<jboss-install>/docs/teiid/datasources/salesforce"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
MutualAuthentication
IfyouneedtoconnecttoSalesforceusingMutualAuthentication,followthedirectionstosetupSalesforceathttps://help.salesforce.com/apex/HTViewHelpDoc?id=security_keys_uploading_mutual_auth_cert.htm&language=en_USthenconfigurethebelowCXFconfigurationfileontheresource-adapterbyaddingfollowingpropertytoabovecliscript
/subsystem=resource-adapters/resource-adapter=salesforce/connection-definitions=sfDS/config-properties=ConfigFi
le:add(value=${jboss.server.config.dir}/cxf-https.xml)
cxf-https.xml
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
DeployingVDBDependencies
43
xmlns:sec="http://cxf.apache.org/configuration/security"
xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
xmlns:jaxws="http://java.sun.com/xml/ns/jaxws"
xsi:schemaLocation="http://cxf.apache.org/transports/http/configurationhttp://cxf.apache.org/schemas/confi
guration/http-conf.xsdhttp://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/
spring-beans-2.0.xsdhttp://cxf.apache.org/configuration/securityhttp://cxf.apache.org/schemas/configuration/s
ecurity.xsd">
<http-conf:conduitname="*.http-conduit">
<http-conf:clientConnectionTimeout="120000"ReceiveTimeout="240000"/>
<http-conf:tlsClientParameterssecureSocketProtocol="SSL">
<sec:trustManagers>
<sec:keyStoretype="JKS"password="changeit"file="/path/to/truststore.jks"/>
</sec:trustManagers>
</http-conf:tlsClientParameters>
</http-conf:conduit>
</beans>
moreinformationaboutCXFconfigurationfilecanbefoundathttp://cxf.apache.org/docs/client-http-transport-including-ssl-support.html#ClientHTTPTransport(includingSSLsupport)-ConfiguringSSLSupport
h==OAuthSecuritywith"RefreshToken"
ThebelowlayoutthedirectionstouseRefreshTokenbasedOAuthAuthenticationwithSalesforce.
1)createconnectedapp(mayneedtosetupcustomdomain)2)addprofileand/orpermissionssettotheconnectedapp3)grabthe"callbackurl"(oneneedtosetashttps://localhost:443/_callback"4)Runthroughtheteiid-oauth-util.shin"<eap>/bin"directory,useclient_id,client_pass,andcallbackfromconnectedapp5)use"https://login.salesforce.com/services/oauth2/authorize"authorizelink6)use"https://login.salesforce.com/services/oauth2/token"foraccesstokenurl7)theyougetarefreshtokenfromit8)createasecurity-domainbyexecutingCLI
/subsystem=security/security-domain=oauth2-security:add(cache-type=default)
/subsystem=security/security-domain=oauth2-security/authentication=classic:add
/subsystem=security/security-domain=oauth2-security/authentication=classic/login-
module=Kerberos:add(code=org.teiid.jboss.oauth.OAuth20LoginModule,flag=required,
module=org.jboss.teiid.security,
module-options=[client-id=xxxx,client-secret=xxxx,refresh-token=xxxx,
access-token-uri=https://login.salesforce.com/services/oauth2/token])
reload
thiswillgeneratefollowingXMLinthestandalone.xmlordomain.xml(thiscanalsobedirectlyaddedtothestandalone.xmlordomain.xmlfilesinsteadofexecutingtheCLI)
standalone.xml
<security-domainname="oauth2-security">
<authentication>
<login-modulecode="org.teiid.jboss.oauth.OAuth20LoginModule"flag="required"module="org.jboss.teiid.s
ecurity">
<module-optionname="client-id"value="xxxx"/>
<module-optionname="client-secret"value="xxxx"/>
<module-optionname="refresh-token"value="xxxx"/>
<module-optionname="access-token-uri"value="https://login.salesforce.com/services/oauth2/token"/>
</login-module>
</authentication>
</security-domain>
9)Thentousetheabovesecuritydomaininthesalesforcedatasourceconfiguration,add"<security-domain>oauth2-security</security-domain>"
DeployingVDBDependencies
44
OAuthSecuritywith"JWTToken"basedSteps
ThebelowlayoutthedirectionstouseJWTtokenbasedOAuthAuthenticationwithSalesforce.
1)CreateaSelf-SignedcertificatelocallyoronSalesForce.(user→setup→security-controls→CertificateandKeyManagement)2)Downloadthecertificateandalsoputinkeystoreanddownloadkeystore.KeystoreisneededforTeiid,certificateforthesalesforcesetup3)CreateconnectedappandselectOAuth,andselectallthescopes(somepostssayrefresh-tokenofflineismust)4)createaprofileand/orpermissionsetassigntotheconnectedapp.Ibelievebeforeyoucancreateaconnectedappyouneedtosetupcustomdomain5)Whenyoucreatingconnectedappmakesureyouaddthecertificatein"DigitalCertificate"6)NowinTeiidcreatesecurity-domainbyexecutingCLI
/subsystem=security/security-domain=oauth2-jwt-security:add(cache-type=default)
/subsystem=security/security-domain=oauth2-jwt-security/authentication=classic:add
/subsystem=security/security-domain=oauth2-jwt-
security/authentication=classic/login-
module=oauth:add(code=org.teiid.jboss.oauth.OAuth20LoginModule,flag=required,
module=org.jboss.teiid.security,
module-options=[client-id=xxxx,client-secret=xxxx,access-token-
uri=https://login.salesforce.com/services/oauth2/token,jwt-
audience=https://login.salesforce.com,jwt-subject=your@sf-login.com,
keystore-type=JKS,keystore-password=changeme,keystore-
url=${jboss.server.config.dir}/salesforce.jks,certificate-alias=teiidtest,
signature-algorithm-name=SHA256withRSA])
reload
thiswillgeneratefollowingXMLinthestandalone.xmlordomain.xml(thiscanalsobedirectlyaddedtothestandalone.xmlordomain.xmlfilesinsteadofexecutingtheCLI)
standalone.xml
<security-domainname="oauth2-jwt-security">
<authentication>
<login-modulecode="org.teiid.jboss.oauth.JWTBearerTokenLoginModule"flag="required"module="org.jboss.
teiid.security">
<module-optionname="client-id"value="xxxxx"/>
<module-optionname="client-secret"value="xxxx"/>
<module-optionname="access-token-uri"value="https://login.salesforce.com/services/oauth2/token"/>
<module-optionname="jwt-audience"value="https://login.salesforce.com"/>
<module-optionname="jwt-subject"value="your@sf-login.com"/>
<module-optionname="keystore-type"value="JKS"/>
<module-optionname="keystore-password"value="changeme"/>
<module-optionname="keystore-url"value="${jboss.server.config.dir}/salesforce.jks"/>
<module-optionname="certificate-alias"value="teiidtest"/>
<module-optionname="signature-algorithm-name"value="SHA256withRSA"/>
</login-module>
</authentication>
</security-domain>
7)Thentousetheabovesecuritydomaininthesalesforcedatasourceconfiguration,add"<security-domain>oauth2-jwt-security</security-domain>"
Morehelpfullinks
https://developer.salesforce.com/blogs/developer-relations/2011/03/oauth-and-the-soap-api.htmlhttps://help.salesforce.com/apex/HTViewHelpDoc?id=remoteaccess_oauth_jwt_flow.htm&language=en_US#create_tokenhttp://salesforce.stackexchange.com/questions/31904/how-and-when-does-a-salesforce-saml-oauth2-user-give-permission-to-use-
DeployingVDBDependencies
45
a-connehttp://salesforce.stackexchange.com/questions/30596/oauth-2-0-jwt-bearer-token-flowhttp://salesforce.stackexchange.com/questions/88396/invalid-assertion-error-in-jwt-bearer-token-flow
DeployingVDBDependencies
46
SolrDataSourcesSolrdatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.TherearemanywaystocreateaSolrdatasource,usingCLI,AdminShell,admin-console,etc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutethefollowingcommandusingtheCLIonceyouconnectedtotheServer.MakesureyouprovidethecorrectURLandusercredentials.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=solr/connection-definitions=solrDS:add(jndi-name=java:/solrDS,cl
ass-name=org.teiid.resource.adapter.solr.SolrManagedConnectionFactory,enabled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=solr/connection-definitions=solrDS/config-properties=url:add(valu
e=http://localhost:8983/solr/)
/subsystem=resource-adapters/resource-adapter=solr/connection-definitions=solrDS/config-properties=CoreName:add
(value=collection1)
/subsystem=resource-adapters/resource-adapter=solr:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisSolrConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=solr)
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin"<jboss-install>/docs/teiid/datasources/solr"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
DeployingVDBDependencies
47
WebServiceDataSourcesWebservicedatasourcesuseaTeiidspecificJCAconnectorthatisdeployedintoWildFly10.0.0duringinstallation.Therearemanywaystocreatethefiledatasource,usingCLI,AdminShell,admin-consoleetc.TheexampleshownbelowusestheCLItool,asthisworksinbothStandaloneandDomainmodes.
ExecutefollowingcommandusingtheCLIonceyouconnectedtotheServer.Makesureyouprovidethecorrectendpointandotherpropertiesbelow.Addanyadditionalpropertiesrequiredbytheconnectorbyduplicatingthe"connection-definitions"commandbelow.EdittheJNDInametomatchtheJNDInameyouusedinVDB.
batch
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS:add(jndi-name=java:/wsDS,
class-name=org.teiid.resource.adapter.ws.WSManagedConnectionFactory,enabled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS/config-properties=EndPoint
:add(value={end_point})
/subsystem=resource-adapters/resource-adapter=webservice:activate
runbatch
TofindoutallthepropertiesthataresupportedbythisWebServiceConnectorexecutethefollowingcommandintheCLI.
/subsystem=teiid:read-rar-description(rar-name=webservice)
TheWebServiceDataSourcesupportsspecifyingaWSDLusingtheWsdlproperty.IftheWsdlpropertyisset,thentheServiceName,EndPointName,andNamespaceUripropertiesshouldalsobeset.TheWsdlpropertymaybeaURLorfilelocationortheWSDLtouse.
Tip
Developer’sTip-IfWildFly10.0.0isrunninginstandalonemode,youcanalsomanuallyeditthe"<jboss-install>/standalone/configuration/standalone-teiid.xml"fileandaddtheXMLconfigurationdefinedin*"<jboss-install>/docs/teiid/datasources/web-service"directoryunder"resource-adapters"subsystem.Shutdowntheserverbeforeyoueditthisfile,andrestartafterthemodificationsaredone.
Allavailableconfigurationpropertiesofwebresource-adapter
PropertyName appliesto Required DefaultValue Description
EndPoint HTTP&SOAP true n/a URLforHTTP,ServiceEndpointfor
SOAP
SecurityType HTTP&SOAP false none
TypeofAuthenticationtousedwiththewebservice.Allowedvalues["None","HTTPBasic","WSSecurity","Kerberos","OAuth"]
AuthUserName HTTP&SOAP false n/a Namevalueforauthentication,used
inHTTPBasicandWsSecurity
AuthPassword HTTP&SOAP false n/a Passwordvalueforauthentication,
usedinHTTPBasicandWsSecurity
ConfigFile HTTP&SOAP false n/a CXFclientconfigurationFileorURL
EndPointName HTTP&SOAP false teiid
LocalpartoftheendpointQNametousewiththisconnection,needstomatchonedefinedincxffile
DeployingVDBDependencies
48
ServiceName SOAP false n/a LocalpartoftheserviceQNametousewiththisconnection
NamespaceUri SOAP false http://teiid.org NamespaceURIoftheserviceQNametousewiththisconnection
RequestTimeout HTTP&SOAP false n/a Timeoutforrequest
ConnectTimeout HTTP&SOAP false n/a Timeoutforconnection
Wsdl SOAP false n/a WSDLfileorURLforthewebservice
CXFConfiguration
EachwebservicedatasourcemaychooseaparticularCXFconfigfileandportconfiguration.TheConfigFileconfigpropertyspecifiestheSpringXMLconfigurationfilefortheCXFBusandportconfigurationtobeusedbyconnections.Ifnoconfigfileisspecifiedthenthesystemdefaultconfigurationwillbeused.
Only1portconfigurationcanbeusedbythisdatasource.YoumayexplicitlysetthelocalnameoftheportQNametouseviatheConfigNameproperty.ThenamespaceURIfortheQNameinyourconfigfileshouldmatchyourWSDL/namespacesettingonthedatasourceorusethedefaultofhttp://teiid.org.SeetheCXFDocumentationandthesectionsbelowonWS-Security,Logging,etc.forexamplesofusingtheCXFconfigurationfile.
SampleSpringXMLConfigurationToSetTimeouts
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
xsi:schemaLocation="http://cxf.apache.org/transports/http/configuration
http://cxf.apache.org/schemas/configuration/http-conf.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<http-conf:conduitname="{http://teiid.org}configName.http-conduit">
<http-conf:clientConnectionTimeout="120000"ReceiveTimeout="240000"/>
</http-conf:conduit>
</beans>
Intheconduitname\{http://teiid.org[http://teiid.org]}configName.http-conduit,thenamespace,\{http://teiid.org[http://teiid.org]},maybesetviathenamespacedatasourceproperty.Typicallythatwillonlyneeddonewhenalsosupplyingthewsdlsetting.Thelocalnameisfollowedby.http-conduit.ItwillbebasedupontheconfigNamesetting,withadefaultvalueofteiid.
SeetheCXFdocumentationforallpossibleconfigurationoptions.
Note ItisnotrequiredtousetheSpringconfigurationtosetjusttimeouts.TheConnectionTimeoutandReceiveTimeoutcanbesetviatheresourceadapterconnectTimeoutandrequestTimeoutpropertiesrespectively.
Security
DeployingVDBDependencies
49
ToenabletheuseofWS-Security,theSecurityTypeshouldbesettoWSSecurity.AtthistimeTeiiddoesnotexpectaWSDLtodescribetheservicebeingused.ThusaSpringXMLconfigurationfileisnotonlyrequired,itmustinsteadcontainalloftherelevantpolicyconfiguration.Andjustaswiththegeneralconfiguration,eachdatasourceislimitedtospecifyingonlyasingleportconfigurationtouse.
batch
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS:add(jndi-name=java:/wsDS,
class-name=org.teiid.resource.adapter.ws.WSManagedConnectionFactory,enabled=true,use-java-context=true)
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS/config-properties=ConfigFi
le:add(value=${jboss.server.home.dir}/standalone/configuration/xxx-jbossws-cxf.xml)
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS/config-properties=ConfigNa
me:add(value=port_x)
/subsystem=resource-adapters/resource-adapter=webservice/connection-definitions=wsDS/config-properties=Security
Type:add(value=WSSecurity)
/subsystem=resource-adapters/resource-adapter=webservice:activate
runbatch
Thecorrespondingxxx-jbossws-cxf.xmlfilethataddsatimestamptotheSOAPheader
ExampleWS-Securityenableddatasource
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://cxf.apache.org/jaxws
http://cxf.apache.org/schemas/jaxws.xsd">
<jaxws:clientname="{http://teiid.org}port_x"
createdFromAPI="true">
<jaxws:outInterceptors>
<bean/>
<refbean="Timestamp_Request"/>
</jaxws:outInterceptors>
</jaxws:client>
<bean
id="Timestamp_Request">
<constructor-arg>
<map>
<entrykey="action"value="Timestamp"/>
<map>
</constructor-arg>
</bean>
</beans>
NotethattheclientportconfigurationismatchedtothedatasourceinstancebytheQName\{http://teiid.org[http://teiid.org]}port_x,wherethenamespacewillmatchyournamespacesettingorthedefaultofhttp://teiid.org.Theconfigurationmaycontainotherportconfigurationswithdifferentlocalnames.
FormoreinformationonconfiguringCXFinterceptors,pleaseconsulttheCXFdocumentation
KerberosWS-SecurityKerberosisonlysupportedwhentheWSDLpropertyisdefinedinresource-adapterconnectionconfigurationandonlywhenWSDLBasedProceduresareused.WSDLfilemustcontainWS-Policysection,thenWS-Policysectioniscorrectlyinterpretedandenforcedontheendpoint.ThesampleCXFconfigurationwilllooklike
DeployingVDBDependencies
50
<?xmlversion="1.0"encoding="UTF-8"?>
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http="http://cxf.apache.org/transports/http/configuration"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xmlns:cxf="http://cxf.apache.org/core"
xmlns:p="http://cxf.apache.org/policy"
xmlns:sec="http://cxf.apache.org/configuration/security"
xsi:schemaLocation="http://www.springframework.org/schema/beanshttp://www.springframework.org/sc
hema/beans/spring-beans.xsdhttp://cxf.apache.org/jaxwshttp://cxf.apache.o
rg/schemas/jaxws.xsdhttp://cxf.apache.org/transports/http/configurationhttp://cxf.apache.org/sche
mas/configuration/http-conf.xsdhttp://cxf.apache.org/configuration/securityhttp://cxf.apac
he.org/schemas/configuration/security.xsdhttp://cxf.apache.org/corehttp://cxf.apache.org/schemas/co
re.xsdhttp://cxf.apache.org/policyhttp://cxf.apache.org/schemas/policy.xsd">
<beanclass="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
<cxf:bus>
<cxf:features>
<p:policies/>
<cxf:logging/>
</cxf:features>
</cxf:bus>
<jaxws:clientname="{http://webservices.samples.jboss.org/}HelloWorldPort"createdFromAPI="true">
<jaxws:properties>
<entrykey="ws-security.kerberos.client">
<beanclass="org.apache.cxf.ws.security.kerberos.KerberosClient">
<constructor-argref="cxf"/>
<propertyname="contextName"value="alice"/>
<propertyname="serviceName"value="bob@service.example.com"/>
</bean>
</entry>
</jaxws:properties>
</jaxws:client>
</beans>
andyouwouldneedtoconfigurethesecurity-domaininthestandalone-teiid.xmlfileunderthe'security'subsystemas
<security-domainname="alice"cache-type="default">
<authentication>
<login-modulecode="Kerberos"flag="required">
<module-optionname="storeKey"value="true"/>
<module-optionname="useKeyTab"value="true"/>
<module-optionname="keyTab"value="/home/alice/alice.keytab"/>
<module-optionname="principal"value="alice@EXAMPLE.COM"/>
<module-optionname="doNotPrompt"value="true"/>
<module-optionname="debug"value="true"/>
<module-optionname="refreshKrb5Config"value="true"/>
</login-module>
</authentication>
</security-domain>
forcompletelistofkerberospropertiespleaserefertothistestcase
LoggingTheCXFconfigpropertymayalsobeusedtocontroltheloggingofrequestsandresponsesforspecificorallports.Logging,whenenabled,willbeperformedatanINFOleveltotheorg.apache.cxf.interceptorcontext.
Exampleloggingdatasource
batch
/subsystem=resource-adapters/resource-adapter=webservice/connection-
DeployingVDBDependencies
51
definitions=wsDS:add(jndi-name=java:/wsDS,class-
name=org.teiid.resource.adapter.ws.WSManagedConnectionFactory,enabled=true,use-
java-context=true)
/subsystem=resource-adapters/resource-adapter=webservice/connection-
definitions=wsDS/config-
properties=ConfigFile:add(value=${jboss.server.home.dir}/standalone/configuration/x
xx-jbossws-cxf.xml)
/subsystem=resource-adapters/resource-adapter=webservice/connection-
definitions=wsDS/config-properties=ConfigName:add(value=port_x)
/subsystem=resource-adapters/resource-adapter=webservice:activate
runbatch
Correspondingxxx-jbossws-cxf.xml
Exampleloggingdatasource
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://cxf.apache.org/jaxws
http://cxf.apache.org/schemas/jaxws.xsd">
<jaxws:clientname="{http://teiid.org}port_y"
createdFromAPI="true">
<jaxws:features>
<beanclass="org.apache.cxf.feature.LoggingFeature"/>
</jaxws:features>
</jaxws:client>
</beans>
TransportSettings
TheCXFconfigpropertymayalsobeusedtocontrollowlevelaspectsoftheHTTPtransport.SeetheCXFdocumentationforallpossibleoptions.
ExampleDisablingHostnameVerification
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
xsi:schemaLocation="http://cxf.apache.org/transports/http/configuration
http://cxf.apache.org/schemas/configuration/http-conf.xsd
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd">
<http-conf:conduitname="{http://teiid.org}port_z.http-conduit">
<!--WARNING!disableCNcheck=trueshouldNOTbeusedinproduction-->
<http-conf:tlsClientParametersdisableCNcheck="true"/>
</http-conf:conduit>
</beans>
ConfiguringSSLSupport(Https)ForusingtheHTTPS,youcanconfigureCXFfileasbelow
DeployingVDBDependencies
52
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:sec="http://cxf.apache.org/configuration/security"
xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
xmlns:jaxws="http://java.sun.com/xml/ns/jaxws"
xsi:schemaLocation="http://cxf.apache.org/transports/http/configurationhttp://cxf.apache.org/schemas/confi
guration/http-conf.xsdhttp://www.springframework.org/schema/beanshttp://www.springframework.org/schema/beans/
spring-beans-2.0.xsdhttp://cxf.apache.org/configuration/securityhttp://cxf.apache.org/schemas/configuration/s
ecurity.xsd">
<http-conf:conduitname="*.http-conduit">
<http-conf:clientConnectionTimeout="120000"ReceiveTimeout="240000"/>
<http-conf:tlsClientParameterssecureSocketProtocol="SSL">
<sec:trustManagers>
<sec:keyStoretype="JKS"password="changeit"file="/path/to/truststore.jks"/>
</sec:trustManagers>
</http-conf:tlsClientParameters>
</http-conf:conduit>
</beans>
forallthehttp-conduitbasedconfigurationseehttp://cxf.apache.org/docs/client-http-transport-including-ssl-support.html.YoucanalsoconfigureforHTTPBasic,kerberos,etc.
DeployingVDBDependencies
53
KerberoswithRESTbasedServices
Note "Kerberosinws-securitywithSOAPservices"-
CheckoutthecxfconfigurationtoallowKerberosinSOAPwebservicesathttp://cxf.apache.org/docs/security.html
ThekerberossupportisbasedSPNEGOasdescribedinhttp://cxf.apache.org/docs/client-http-transport-including-ssl-support.html#ClientHTTPTransport%28includingSSLsupport%29-SpnegoAuthentication%28Kerberos%29.Theretwotypesofkerberossupport
Negotiation
Withthisconfiguration,RESTserviceisconfiguredwithKerberosJAASdomain,tonegotiateatoken,thenuseitaccessthewebservice.Forthisfirstcreateasecuritydomaininstandalone.xmlfileasbelow
<security-domainname="MY_REALM"cache-type="default">
<authentication>
<login-modulecode="Kerberos"flag="required">
<module-optionname="storeKey"value="true"/>
<module-optionname="useKeyTab"value="true"/>
<module-optionname="keyTab"value="/home/username/service.keytab"/>
<module-optionname="principal"value="host/testserver@MY_REALM"/>
<module-optionname="doNotPrompt"value="true"/>
<module-optionname="debug"value="false"/>
<module-optionname="addGSSCredential"value="true"/>
</login-module>
</authentication>
</security-domain>
andthejboss-cxf-xxx.xmlfileneedstobesetas
<beansxmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:sec="http://cxf.apache.org/configuration/security"
xmlns:http-conf="http://cxf.apache.org/transports/http/configuration"
xsi:schemaLocation="http://cxf.apache.org/transports/http/configurationhttp://cxf.apache.org/schemas/c
onfiguration/http-conf.xsdhttp://www.springframework.org/schema/beanshttp://www.springframework.org/schema/be
ans/spring-beans-2.0.xsdhttp://cxf.apache.org/configuration/securityhttp://cxf.apache.org/schemas/configurati
on/security.xsd">
<http-conf:conduitname="*.http-conduit">
<http-conf:authorization>
<sec:AuthorizationType>Negotiate</sec:AuthorizationType>
<sec:Authorization>MY_REALM</sec:Authorization>
</http-conf:authorization>
</http-conf:conduit>
</beans>
Theresourceadaptercreationneedstodefinethefollowingproperties
<config-propertyname="ConfigFile">path/to/jboss-cxf-xxxx.xml</config-property>
<config-propertyname="ConfigName">test</config-property>
NoteEventhoughaboveconfigurationconfiguresthevalueof"ConfigName",thecxfframeworkcurrentlyinthecaseofJAX-RSclientdoesnotgiveoptiontouseit.Forthatreasonuse"*.http-conduit"whichwillapplytoalltheHTTPcommunicationsunderthisresourceadapter.
DeployingVDBDependencies
54
Delegation
IfincasetheuserisalreadyloggedintoTeiidusingKerberosusingJDBC/ODBCorusedSPNEGOinweb-tierandusedpass-throughauthenticationintoTeiid,thenthereisnoneedtonegotiateanewtokenfortheKerberos.Thesystemcandelegatetheexistingtoken.
Toconfigurefordelegation,setupsecuritydomaindefinedexactlyasdefinedin"negotiation",andjboss-cxf-xxx.xmlfile,howeverremovethefollowinglinefromjboss-cxf-xxx.xmlfile,asitisnotgoingtonegotiatenewtoken.
<sec:Authorization>MY_REALM</sec:Authorization>
Addthefollowingpropertiesinwebserviceresourceadaptercreation.Oneconfiguresthat"kerberos"securitybeingused,theseconddefinesasecuritydomaintobeusedatthedatasource,inthiscasewewanttouseasecuritydomainthatpassesthroughaloggedinuser
<config-propertyname="SecurityType">Kerberos</config-property>
<security>
<security-domain>passthrough-security</security-domain>
</security>
Toconfigurein"passthrough-security"securitydomain,the"security"subsystemaddfollowingXMLfragment
<security-domainname="passthrough-security"cache-type="default">
<authentication>
<login-modulecode="Kerberos"flag="required"module="org.jboss.security.negotiation">
<module-optionname="delegationCredential"value="REQUIRED"/>
</login-module>
</authentication>
</security-domain>
IfincasethereisnodelegationCredentialisavailableonthecontext,theaccesswillfail.
DeployingVDBDependencies
55
OAuthAuthenticationWithRESTBasedServices
SingleuserOAuthauthentication
WebServicesresource-adaptercanbeconfiguredtoparticipateinOAuth1.0aandOAuth2authenticationschemes.UsingTeiidalongwith"ws"translatorand"web-services"resourceadapteroncewriteapplicationscommunicatingwithwebsiteslikeGoogleandTwitter.
InordertosupportOAuthauthentication,thereissomepreparationandconfigurationworkinvolved.IndividualwebsitestypicallyprovidedeveloperfacingRESTbasedAPIsforaccessingtheircontentonthewebsitesandalsoprovidewaystoregistercustomapplicationsonuser’sbehalf,wheretheycanmanagetheAuthorizationofservicesofferedbythewebsite.Thefirststepistoregisterthiscustomapplicationonthewebsiteandcollectconsumer/APIkeysandsecrets.Theweb-siteswillalsolisttheURLS,wheretorequestforvariousdifferenttypesoftokensforauthorizationusingthesecredentials.AtypicalOAuthauthenticationflowisdefinedasbelow
Theaboveimagetakenfromhttps://developers.google.com/accounts/docs/OAuth2
Toaccommodateabovedefinedflow,Teiidprovidesautilitycalled"teiid-oauth-util.sh"or"teiid-oauth-util.bat"forwindowsinthe"bin"directoryofyourserverinstallation.Byexecutingthisutility,itwillaskforvariouskeys/secretsandURLsforthegeneratingtheAccessTokenthatisusedintheOAuthauthenticationandintheendoutputaXMLfragmentlikebelow.
$./teiid-oauth-util.sh
SelecttypeofOAuthauthentication
1)OAuth1.0A
2)OAuth2.0
2
===OAuth2.0Workflow===
DeployingVDBDependencies
56
EntertheClientID=10-xxxjb.apps.googleusercontent.com
EntertheClientSecret=3L6-xxx-v9xxDlznWq-o
EntertheUserAuthorizationURL=https://accounts.google.com/o/oauth2/auth
Enterscope(hitenterfornone)=profile
Cut&PastetheURLinawebbrowser,andAuthticate
AuthorizeURL=https://accounts.google.com/o/oauth2/auth?client_id=10-
xxxjb.apps.googleusercontent.com&scope=profile&response_type=code&redirect_uri=urn%
3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&state=Auth+URL
EnterTokenSecret(AuthCode,Pin)frompreviousstep=4/z-RT632cr2hf_vYoXd06yIM-
xxxxx
EntertheAccessTokenURL=https://www.googleapis.com/oauth2/v3/token
RefreshToken=1/xxxx_5qzAF52j-EmN2U
AddthefollowingXMLintoyourstandalone-teiid.xmlfileinsecurity-domains
subsystem,
andconfiguredatasourcesecurtytothisdomain
<security-domainname="oauth2-security">
<authentication>
<login-modulecode="org.teiid.jboss.oauth.OAuth20LoginModule"
flag="required"module="org.jboss.teiid.web.cxf">
<module-optionname="client-id"value="10-
xxxjb.apps.googleusercontent.com"/>
<module-optionname="client-secret"value="3L6-xxx-v9xxDlznWq-o"/>
<module-optionname="refresh-token"value="1/xxxx_5qzAF52j-EmN2U"/>
<module-optionname="access-token-uri"
value="https://www.googleapis.com/oauth2/v3/token"/>
</login-module>
</authentication>
</security-domain>
TheXMLfragmentattheenddefinestheJAASLoginModuleconfiguration,editthestandalone-teiid.xmlandadditunder"security-domains"subsystem.Userneedstousethissecurity-domainintheirresourceadapterasthesecurityproviderforthisdatasource.Anexampleresource-adapterconfigurationtodefinethedatasourcetothewebsiteinstandalone-teiid.xmlfilelookslike
<resource-adapterid="webservice3">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.webservice"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory"jndi-name=
"java:/googleDS"enabled="true"use-java-context="true"pool-name="teiid-ws-ds">
<config-propertyname="SecurityType">
OAuth
</config-property>
DeployingVDBDependencies
57
<security>
<security-domain>oauth2-security</security-domain>
</security>
</connection-definition>
</connection-definitions>
</resource-adapter>
---
Then,anyquerywrittenusingthe"ws"translatorandaboveresource-adapterwillbeautomaticallyAuthorizedw
iththetargetwebsiteusingOAuth,whenyouaccessaprotectedURL.
===OAuthwithDelegation
Intheaboveconfigurationasingleuserisconfiguredtoaccessthewebsite,howeverifyouwanttodelegate
loggedinuser’scredentialasOAuthauthentication,thenuserneedstoextendtheaboveLoginModule
_(org.teiid.jboss.oauth.OAuth20LoginModuleororg.teiid.jboss.oauth.OAuth10LoginModule)_andautomatetheproce
ssdefinedinthe"teiid-oauth-util.sh"todefinetheAccessToken
detailsdynamically.Sincethisprocesswillbedifferentfordifferentwebsites(itinvolvesloginandauthen
tication),Teiidwillnotbeabletoprovidesinglesolution.However,usercanextendtheloginmoduletoprov
idethisfeaturemuchmoreeasilysincetheywillbeworkingwithtargetedwebsites.
DeployingVDBDependencies
58
VDBVersioningVDBVersioningisafeaturethatallowsmultipleversionsofaVDBtobedeployedatthesametimewithadditionalsupporttodeterminewhichversionwillbeused.Ifaspecificversionisrequested,thenonlythatVDBmaybeconnectedto.Ifnoversionisset,thenthedeployedVDBsaresearchedfortheappropriateversion.Thisfeaturehelpssupportmorefluidmigrationscenarios.
VersionProperty
WhenauserconnectstoTeiidthedesiredVDBversioncanbesetasaconnectionproperty(SeetheClientDeveloper’sGuide)inJDBCorusedaspartoftheVDBnameforODataandODBCaccess.
Thevdbversionissetineitherthevdb.xml,whichisusefulforanxmlfiledeployment,orthroughanamingconventionofthedeploymentname-vdbname.version.vdb,e.g.marketdata.2.vdb.Thedeployerisresponsibleforchoosinganappropriateversionnumber.IfthereisalreadyaVDBname/versionthatmatchesthecurrentdeployment,thenconnectionstothepreviousVDBwillbeterminatedanditscacheentrieswillbeflushed.AnynewconnectionswillthenbemadetothenewVDB.
AsimpleintegerversionactuallytreatedasthesemanticversionX.0.0.Ifdesiredafullsemanticversioncanbeusedinstead.Asemanticversionisuptothreeintegersseparatedbyperiods.
Trailingversioncomponentsthataremissingaretreatedaszeros-version1isthesameas1.0.0andversion1.1isthesameas1.1.0.
JDBCandODBCclientsmayuseaversionrestriction--vdbname.X.orvdbname.X.X.-notethetrailing'.'whichmeansaVDBthatmustmatchthepartialversionspecified.Forexamplevdbname.1.2.couldmatchany1.2.Xversion,butwouldnotallow1.3+or1.1andearlier.
ConnectionTypeOncedeployedaVDBhasanupdatablepropertycalledconnectiontype,whichisusedtodeterminewhatconnectionscanbemadetotheVDB.Theconnectiontypecanbeoneof:
NONE-disallownewconnections.
BY_VERSION-thedefaultsetting.AllowconnectionsonlyiftheversionisspecifiedorifthisistheearliestBY_VERSIONvdbandtherearenovdbsmarkedasANY.
ANY-allowconnectionswithorwithoutaversionspecified.
TheconnectiontypemaybechangedeitherthroughtheAdminConsoleortheAdminAPI.
DeploymentScenarios
IfonlyaselectfewapplicationsaretomigratetothenewVDBversion,thenafreshlydeployedVDBwouldbeleftasBY_VERSION.Thisensuresthatonlyapplicationsthatknowthenewversionmayuseit.
IfonlyaselectfewapplicationsaretoremainonthecurrentVDBversion,thentheirconnectionsettingswouldneedtobeupdatedtoreferencethecurrentVDBbyitsversion.ThenthenewlydeployedvdbwouldhaveitsconnectiontypesettoANY,whichallowsallnewconnectionstobemadeagainstthenewerversion.Ifarollbackisneededinthisscenario,thenthenewlydeployedvdbwouldhaveitsconnectiontypesettoNONEorBY_VERSIONaccordingly.
VDBVersioning
59
VDBVersioning
60
LoggingTheTeiidsystemprovidesawealthofinformationvialogging.Tocontrollogginglevel,contexts,andloglocations,youshouldbefamiliarwithlog4jandthecontainer’sstandalone-teiid.xmlordomain-teiid.xmlconfigurationfilesdependinguponthestartupmodeofWildFly.
AllthelogsproducedbyTeiidareprefixedby"org.teiid".ThismakesitextremelyeasytocontrolofofTeiidloggingfromasinglecontext.Notehoweverthatchangestothelogconfigurationfilemanuallyrequirearestarttotakeaffect.CLIbasedlogcontextmodificationsarepossible,howeverdetailsarebeyondthescopeofthisdocument.
Ifyouexpectahighvolumeoflogginginformationoruseexpensivecustomaudit/commandloggers,itisagoodideatouseanaynchappendertominimizetheperformanceimpact.Forexampleyoucanuseaconfigurationsnippetliketheonebelowtoinsertanasynchhandlerinfrontofthetargetappender.
<periodic-rotating-file-handlername="COMMAND_FILE">
<levelname="DEBUG"/>
<formatter>
<pattern-formatterpattern="%d{HH:mm:ss,SSS}%-5p[%c](%t)%s%E%n"/>
</formatter>
<filerelative-to="jboss.server.log.dir"path="command.log"/>
<suffixvalue=".yyyy-MM-dd"/>
</periodic-rotating-file-handler>
<async-handlername="ASYNC">
<levelname="DEBUG"/>
<queue-lengthvalue="1024"/>
<overflow-actionvalue="block"/>
<subhandlers>
<handlername="COMMAND_FILE"/>
</subhandlers>
</async-handler>
<loggercategory="org.teiid.COMMAND_LOG">
<levelname="DEBUG"/>
<handlers>
<handlername="ASYNC"/>
</handlers>
</logger>
LoggingContextsWhileallofTeiid’slogsareprefixedwith"org.teiid",therearemorespecificcontextsdependingonthefunctionalareaofthesystem.Notethatlogsoriginatingfromthird-partycode,includingintegratedorg.jbosscomponents,willbeloggedthroughtheirrespectivecontextsandnotthrough"org.teiid".SeethetablebelowforinformationoncontextsrelevanttoTeiid.Seethecontainer’sstandalone-teiid.xmlforamorecompletelistingofloggingcontextsusedinthecontainer.
Context Description
com.arjuna Third-partytransactionmanager.Thiswillincludeinformationaboutalltransactions,notjustthoseforTeiid.
org.teiidRootcontextforallTeiidlogs.Note:therearepotentiallyothercontextsusedunderorg.teiidthanareshowninthistable.
org.teiid.PROCESSORQueryprocessinglogs.Seealsoorg.teiid.PLANNERforqueryplanninglogs.
Logging
61
org.teiid.PLANNER Queryplanninglogs.
org.teiid.SECURITY Session/Authenticationevents-seealsoAUDITlogging
org.teiid.TRANSPORT Eventsrelatedtothesockettransport.
org.teiid.RUNTIME Eventsrelatedtoworkmanagementandsystemstart/stop.
org.teiid.CONNECTOR Connectorlogs.
org.teiid.BUFFER_MGR Bufferandstoragemanagementlogs.
org.teiid.TXN_LOG Detaillogofalltransactionoperations.
org.teiid.COMMAND_LOG Seecommandlogging
org.teiid.AUDIT_LOG Seeauditlogging
org.teiid.ADMIN_API AdminAPIlogs.
org.teiid.ODBC ODBClogs.
CommandLogging
CommandloggingcapturesexecutingcommandsintheTeiidSystem.Thisincludesusercommands(thathavebeensubmittedtoTeiidatanINFOlevel),datasourcecommands(thatarebeingexecutedbytheconnectorsataDEBUGlevel),andqueryplancommands(forwhenthequeryplanningisperformedataTRACElevel)aretrackedthroughcommandlogging.
Theusercommand,"STARTUSERCOMMAND",isloggedwhenTeiidstartsworkingonthequeryforthefirsttime.Thisdoesnotincludethetimethequerywaswaitinginthequeue.Andacorrespondingusercommand,"ENDUSERCOMMAND",isloggedwhentherequestiscomplete(i.e.whenstatementisclosedorallthebatchesareretrieved).Thereisonlyonepairoftheseforeveryuserquery.
Thequeryplancommand,"PLANUSERCOMMAND",isloggedwhenTeiidfinishesthequeryplanningprocess.Thereisnocorrespondingendinglogentry.
Thedatasourcecommand,"STARTDATASRCCOMMAND",isloggedwhenaqueryissenttothedatasource.Andacorrespondingdatasourcecommand,"ENDSRCCOMMAND",isloggedwhentheexecutionisclosed(i.ealltherowshasbeenread).TherecanbeonepairforeachdatasourcequerythathasbeenexecutedbyTeiid,andtherecanbenumberofpairsdependinguponyourquery.
Withthisinformationbeingcaptured,theoverallqueryexecutiontimeinTeiidcanbecalculated.Additionally,eachsourcequeryexecutiontimecanbecalculated.Iftheoverallqueryexecutiontimeisshowingaperformanceissue,thenlookateachdatasourceexecutiontimetoseewheretheissuemaybe.
Toenablecommandloggingtothedefaultloglocation,simplyenabletheDETAILlevelofloggingfortheorg.teiid.COMMAND_LOGcontext.
Note "Wanttologtoadatabase?"-IfyouwouldliketologCommandlogmessagestoanydatabase,thenlookatthehttps://github.com/teiid/teiid-extensionsproject.TheinstallationzipfileisavailableinTeiiddownloadspage.
Logging
62
Toenablecommandloggingtoanalternativefilelocation,configureaseparatefileappenderfortheDETAILloggingoftheorg.teiid.COMMAND_LOGcontext.Anexampleofthisisshownbelowandcanalsobefoundinthestandalone-teiid.xmldistributedwithTeiid.
<periodic-rotating-file-handlername="COMMAND_FILE">
<levelname="DEBUG"/>
<formatter>
<pattern-formatterpattern="%d{HH:mm:ss,SSS}%-5p[%c](%t)%s%E%n"/>
</formatter>
<filerelative-to="jboss.server.log.dir"path="command.log"/>
<suffixvalue=".yyyy-MM-dd"/>
</periodic-rotating-file-handler>
<loggercategory="org.teiid.COMMAND_LOG">
<levelname="DEBUG"/>
<handlers>
<handlername="COMMAND_FILE"/>
</handlers>
</logger>
SeetheDeveloper’sGuidetodevelopacustomloggingsolutioniffilebasedlogging,oranyotherbuilt-inLog4jlogging,isnotsufficient.
Thefollowingisanexampleofadatasourcecommandandwhatonewouldlooklikewhenprintedtothecommandlog:
2012-02-2216:01:53,712DEBUG[org.teiid.COMMAND_LOG](Worker1_QueryProcessorQueue11STARTDATASRCCOMMAND:st
artTime=2012-02-2216:01:53.712
requestID=Ku4/dgtZPYk0.5sourceCommandID=4txID=nullmodelName=DTHCPtranslatorName=jdbc-simplesessionID=Ku4/d
gtZPYk0
principal=user@teiid-security
sql=HCP_ADDR_XREF.HUB_ADDR_ID,CPN_PROMO_HIST.PROMO_STAT_DTFROMCPN_PROMO_HIST,HCP_ADDRESS,HCP_ADDR_XREF
WHERE(HCP_ADDRESS.ADDR_ID=CPN_PROMO_HIST.SENT_ADDR_ID)AND(HCP_ADDRESS.ADDR_ID=HCP_ADDR_XREF.ADDR_ID)AND
(CPN_PROMO_HIST.PROMO_STAT_CDNOTLIKE'EMAIL%')AND(CPN_PROMO_HIST.PROMO_STAT_CD<>'SENT_EM')AND
(CPN_PROMO_HIST.PROMO_STAT_DT>{ts'2010-02-2216:01:52.928'})
Notethefollowingpiecesofinformation:
modelName:thisrepresentsthephysicalmodelforthedatasourcethatthequeryisbeingissued.
translatorName:showstypeoftranslatorusedtocommunicatetothedatasource.
principal:showstheuseraccountwhosubmittedthequery
startTime/endTime:thetimeoftheaction,whichisbasedonthetypecommandbeingexecuted.
sql:isthecommandsubmittedtotheengineortothetranslatorforexecution-whichisNOTnecessarilythefinalsqlcommandsubmittedtotheactualdatasource.Butitdoesshowwhatthequeryenginedecidedtopushdown.
ENDeventswilladditionallycontain:
finalRowCount:thenumberofrowsreturnedtotheenginebythesourcequery.
cpuTime:thenumberofnanosecondsofcputimeusedbythesourcecommand.Canbecomparedtothestart/endwallclocktimestodeterminecpuvs.idletime.
AuditLogging
Auditloggingcapturesimportantsecurityevents.Thisincludestheenforcementofpermissions,authenticationsuccess/failures,etc.
Logging
63
Toenableauditloggingtothedefaultloglocation,simplyenabletheDEBUGlevelofloggingfortheorg.teiid.AUDIT_LOGcontext.
Note"Wanttologtoadatabase?"-IfyouwouldliketologAuditlogmessagestoanydatabase,thenlookatthehttps://github.com/teiid/teiid-extensionsproject.TheinstallationzipfilewillbeavailableinTeiiddownloadspage.
Toenableauditloggingtoanalternativefilelocation,configureaseparatefileappenderfortheDETAILloggingoftheorg.teiid.AUDIT_LOGcontext.SeetheDeveloper’sGuidetodevelopacustomloggingsolutioniffilebased,oranyotherbuilt-inLog4j,loggingisnotsufficient.
AdditionalLoggingInformationOnceasessionhasbeencreated,eachlogmadebyTeiidwillincludethesessionidandvdbname/versionintheMDC(mappeddiagnosticcontext)withkeysofteiid-sessionandteiid-vdbrespectively.
AnyloginthescopeofaquerywillincludetherequestidintheMDCwithkeyofteiid-request.
Customloggers,orformatpatterns,cantakeadvantageofthisinformationtobettercorrelatelogentries.SeeforexampleTeiiddefaultstandalone-teiid.xmlthatusesapatternformatwhichincludesthesessionidpriortothemessage:
<pattern-formatterpattern="%d{HH:mm:ss,SSS}%-5p[%c](%t)%X{teiid-session}%s%E%n"/>
Logging
64
ClusteringinTeiidSinceTeiidisinstalledinWildFly,thereisnoadditionalconfigurationneededbeyondwhatwasperformedwhenTeiidissetupinDomainMode.SeetheDomainModesectionintheTeiidInstallationGuide.JustmakesurethatyouinstalledTeiidineveryWildFlynodeandstartedallWildFlyinstancesintheDomainmodethattobeapartofthecluster.
Typicallyuserscreateclusterstoimprovetheperformanceofthesystemthrough:
LoadBalancing:TakelookattheClientDeveloper’sGuideonhowtouseloadbalancingbetweenmultiplenodes.
FailOver:TakelookattheClientDeveloper’sGuideonhowtousefailoverbetweenmultiplenodes.
DistributedCaching:Thisisautomaticallydoneforyouonceyouconfigureitasspecifiedabove.
Eventdistribution:metadataanddatamodificationswillbedistributedtoallclustermembers.
IntheDomainmode,theonlywayausercandeployanyartifactsisusingeitherCLIorusingtheAdminAPIorAdminShell.CopyingVDBdirectlyintothe"deployments"directoryisnotsupported.
ClusteringinTeiid
65
MonitoringTeiidprovidesinformationaboutitscurrentoperationalstate.Thisinformationcanbeusefulintuning,monitoring,andmanagingloadandthrough-put.Theruntimedatacanbeaccessedusingadministrativetools(i.e.AdminConsole,AdminShellorAdminAPI).
Query/Sessiondetails:
Name Description
CurrentSessions Listcurrentconnectedsessions
CurrentRequest Listcurrentexecutingrequests
CurrentTransactions Listcurrentexecutingtransactions
QueryPlan Retrievesthequeryplanforaspecificrequest
Thereareadministrativeoptionsforterminatingsessions,queries,andtransactions.
Metrics:
Session/Query
Name Property Description Comment
SessionCount sessionCountIndicatesthenumberofuserconnectionscurrentlyactive
Toensurenumberofsessionsarenotrestrictedatpeaktimes,checkmax-sessions-allowed(default10000)issetaccordinglyandreviewsessions-expiration-timelimit
QueryCount queryCount Indicatesthenumberofqueriescurrentlyactive.
ActiveQueryPlanCount
ENGINE_STATISTIC.active-plans-count
Numberofqueryplanscurrentlybeingprocessed
Toensuremaximumthrough-put,seetheQueryEnginesectioninThreadingontuning.
WaitingQueryPlanCount
ENGINE_STATISTIC.waiting-plans-count
Numberofqueryplanscurrentlywaiting
MaxWaitingQueryPlanWatermark
ENGINE_STATISTIC.max-waitplan-watermark
Themaximumnumberofqueryplansthathavebeenwaitingatonetime,sincethelasttimetheserverstarted
Monitoring
66
LongRunningQueries longRunningQueries
Listcurrentexecutingqueriesthathavesurpassedthequerythreshold(query-threshold-in-seconds).
Setupalerttowarnwhenoneormorequeriesareconsumingresourcesforanextendedperiodoftime.Ifrunningtoolong,anoptionistocancelrequestorincreasethreshold.
BufferManager
Fortuningsuggestions,seeMemoryManagement.
Name Property Description Comment
DiskWriteCount ENGINE_STATISTIC.buffermgr-disk-write-count
Diskwritecountforthebuffermanager.
DiskReadCount ENGINE_STATISTIC.buffermgr-disk-read-count
Diskreadcountforthebuffermanager.
CacheWriteCount ENGINE_STATISTIC.buffermgr-cache-write-count
Cachewritecountforthebuffermanager.
CacheReadCount ENGINE_STATISTIC.buffermgr-cache-read-count
Cachereadcountforthebuffermanager.
DiskSpaceUsed(MB) ENGINE_STATISTIC.buffermgr-diskspace-used-mb
Indicatesamountofstoragespacecurrentlyusedbybufferfiles
Setupalerttowarnwhenusedbufferspaceisatanunacceptablelevel,basedonthesettingofmax-buffer-space
Totalmemoryinuse(KB)
ENGINE_STATISTIC.total-memory-inuse-kb
Estimateofthecurrentmemoryusageinkilobytes.
Totalmemoryinusebyactiveplans(KB)
ENGINE_STATISTIC.total-memory-inuse-active-plans-kb
Estimateofthecurrentmemoryusagebyactiveplansinkilobytes
Plan/ResultCache
Fortuningsuggestions,seeCacheTuning.
Name Property Description
PreparedPlanCacheSize PREPARED_PLAN_CACHE.total-entries Currentnumberofentriesincache.
PreparedPlanCache#ofRequests PREPARED_PLAN_CACHE.request-count Totalnumberofrequests
madeagainstcache.
PreparedPlanCacheHitRatio% PREPARED_PLAN_CACHE.hit-ratio Percentageofpositivecache
hits
ResultSetCacheSize QUERY_SERVICE_RESULT_SET_CACHE.total-entries
Currentnumberofentriesincache.
Monitoring
67
ResultSetCache#ofRequests
QUERY_SERVICE_RESULT_SET_CACHE.request-count
Totalnumberofrequestsmadeagainstcache.
ResultSetCacheHitRatio% QUERY_SERVICE_RESULT_SET_CACHE.hit-ratio Percentageofpositivecache
hits.
Monitoring
68
PerformanceTuningPerformancetuningcanbedonebychangingthepropertysettingsdefinedintheteiidsubsystemanditssubcomponents.
ExecutethefollowingcommandonCLItoseethepossiblesettingsattherootoftheteiidsubsystem:
/subsystem=teiid:read-resource-description
Thereareseveralcategoriesofproperties:
1. MemoryManagement
2. BufferManager:allpropertiesthatstartwith"buffer-service"
3. CacheTuning:allpropertiesthatstartwith"resultset-cache"or"preparedplan-cache"
4. Threading
5. LOBs
6. OtherConsiderations
SocketTransportsettingsforoneofthesupportedtransporttypes(i.e.,jdbc,odbc,embedded)canbeviewedbyexecutingthefollowingcommand:
/subsystem=teiid/transport={jdbc|odbc|embedded}:read-resource-description
PerformanceTuning
69
MemoryManagementTheBufferManagerisresponsiblefortrackingbothmemoryanddiskusagebyTeiid.ConfiguringtheBufferManagerproperlyalongwithdatasourcesandthreadingensureshighperformance.InmostinstancesthoughthedefaultsettingsaresufficientastheywillscalewiththeJVMandconsiderotherpropertiessuchasthesettingformaxactiveplans.
ExecutefollowingcommandonCLItofindallpossiblesettingsonBufferManager:
/subsystem=teiid:read-resource
Allthepropertiesthatstartwith"buffer-service"usedtoconfigureBufferManager.ShownbelowaretheCLIwriteattributecommandstochangeBufferManager’ssettings(allshowthedefaultsetting):
/subsystem=teiid:write-attribute(name=buffer-service-use-disk,value=true)
/subsystem=teiid:write-attribute(name=buffer-service-encrypt-files,value=false)
/subsystem=teiid:write-attribute(name=buffer-service-processor-batch-size,value=256)
/subsystem=teiid:write-attribute(name=buffer-service-max-open-files,value=64)
/subsystem=teiid:write-attribute(name=buffer-service-max-file-size,value=2048)
/subsystem=teiid:write-attribute(name=buffer-service-max-processing-kb,value=-1)
/subsystem=teiid:write-attribute(name=buffer-service-max-reserve-kb,value=-1)
/subsystem=teiid:write-attribute(name=buffer-service-max-buffer-space,value=51200)
/subsystem=teiid:write-attribute(name=buffer-service-max-inline-lobs,value=true)
/subsystem=teiid:write-attribute(name=buffer-service-memory-buffer-space,value=-1)
/subsystem=teiid:write-attribute(name=buffer-service-max-storage-object-size,value=8388608)
/subsystem=teiid:write-attribute(name=buffer-service-memory-buffer-off-heap,value=false)
Note Notethatitisnotrecommendthattochangethesepropertiesuntilthereisanunderstandingoftheproperties(elaboratedbelow)andanypotentialissuethatisbeingexperienced.
SomeofBufferManager’spropertiesaredescribedbelow.Notethattheperformancetuningadviceishighlightedininfoboxes.
max-reserve-kb(default-1)-settingdeterminesthetotalsizeinkilobytesofbatchesthatcanbeheldbytheBufferManagerinmemory.Thisnumberdoesnotaccountforpersistentbatchesheldbysoft(suchasindexpages)orweakreferences.Thedefaultvalueof-1willauto-calculateatypicalmaxbaseduponthemaxheapavailabletotheVM.Theauto-calculatedvalueassumesa64bitarchitectureandwilllimitbufferusageto40%ofthefirstgigabyteofmemorybeyondthefirst300megabytes(whichareassumedforusebytheASandotherTeiidpurposes)and50%ofthememorybeyondthat.Theadditionalcaveathereisthatifthesizeofthememorybufferspaceisnotspecified,thenitwilleffectivelybeallocatedoutofthemaxreservespace.Asmalladjustmentisalsomadetothemaxreservetoaccountforbatchtrackingoverhead.
Note Withdefaultsettingsandan8GBVMsize,thenmax-reserve-kbwillatamaxuse:(1024-300)*0.4)+(7*1024*0.5=4373.6MBor4,478,566KB
TheBufferManagerautomaticallytriggerstheuseofacanonicalvaluecacheifenabledwhenmorethan25%ofthereserveisinuse.ThiscandramaticallycutthememoryusageinsituationswheresimilarvaluesetsarebeingreadthroughTeiid,butdoesintroducealookupcost.IfyouareprocessingsmallorhighlysimilardatasetsthroughTeiid,andwishtoconservememory,youshouldconsiderenablingvaluecaching.
Warning
Memoryconsumptioncanbesignificantlymoreorlessthanthenominaltargetdependinguponactualcolumnvaluesandwhethervaluecachingisenabled.Largenonbuilt-intypeobjectscanexceedtheirdefaultsizeestimate.Ifanoutofmemoryerrorsoccur,thensetalowermax-reserve-kbvalue.Alsonotethatsourcelobvaluesareheldbymemoryreferencesthatarenotclearedwhenabatchispersisted.Withheavylobusageyoushouldensurethatbuffersofothermemoryassociatedwithlobreferencesareappropriatelysized.
MemoryManagement
70
max-processing-kb(default-1)-settingdeterminesthetotalsizeinkilobytesofbatchesthatcanbeguaranteedforusebyoneactiveplanandmaybeinadditiontothememoryheldbasedonmax-reserve-kb.TypicalminimummemoryrequiredbyTeiidwhenalltheactiveplansareactiveis#active-plans*max-processing-kb.Thedefaultvalueof-1willauto-calculateatypicalmaxbaseduponthemaxheapavailabletotheVMandmaxactiveplans.Theauto-calculatedvalueassumesa64bitarchitectureandwilllimitnominalprocessingbatchusagetolessthan10%oftotalmemory.
Note
Withdefaultsettingsincluding20active-plansandan8GBVMsize,thenmax-processing-kbwillbe:(.07*8*1024)/20^.8=537.4MB/11=52.2MBor53,453KBperplan.Thisimpliesanominalrangebetween0and1060MBthatmaybereservedwithroughly53MBperplan.Youshouldbecautiousinadjustingmax-processing-kbonyourown.Typicallyitwillnotneedadjustedunlessyouareseeingsituationswhereplansseemmemoryconstrainedwithlowperforminglargesorts.
max-file-size(default2GB)-Eachintermediateresultbuffer,temporaryLOB,andtemporarytableisstoredinitsownsetofbufferfiles,whereanindividualfileislimitedtomax-file-sizemegabytes.Considerincreasingthestoragespaceavailabletoallsuchfilesbyincreasingmax-buffer-space,ifyourinstallationmakesuseofinternalmaterialization,makesheavyuseofSQL/XML,orprocesseslargerowcounts.
processor-batch-size(default256)-Specifiesthetargetrowcountofabatchofthequeryprocessor.Abatchisusedtorepresentbothlineardatastores,suchassavedresults,andtemporarytablepages.Teiidwilladjusttheprocessor-batch-sizetoaworkingsizebaseduponanestimateofthedatawidthofarowrelativetoanominalexpectationof2KB.Thebasevaluecanbedoubledorhalveduptothreetimesdependinguponthedatawidthestimation.Forexampleasinglesmallfixedwidth(suchasaninteger)columnbatchwillhaveaworkingsizeofprocessor-batch-size*8rows.Abatchwithhundredsofvariablewidthdata(suchasstring)willhaveaworkingsizeofprocessor-batch-size/8rows.Anyincreaseintheprocessorbatchsizebeyondthefirstdoublingshouldbeaccompaniedwithaproportionalincreaseinthemax-storage-object-sizetoaccommodatethelargerstoragesizeofthebatches.
Note
AdditionalconsiderationsareneedediflargeVMsizesand/ordatasetsarebeingused.Teiidhasanon-negligibleamountofoverheadperbatch/tablepageontheorderof100-200bytes.Ifyouaredealingwithdatasetswithbillionsofrowsandyourunintomemoryissues,thenafterexaminingtherootcauseifyouseethatit’ssolelyrelatedtomemoryheldbyasignificantnumberofbatchreferences,thenconsiderincreasingtheprocessor-batch-sizetoforcetheallocationoflargerbatchesandtablepages.Ageneralguidelinewouldbetodoubleprocessor-batch-sizeforeverydoublingoftheeffectiveheapforTeiidbeyond4GB-processor-batch-size=512foran8GBheap,processor-batch-size=1024fora16GBheap,etc.
max-storage-object-size(default8288608or8MB)-Themaximumsizeofabufferedmanagedobjectinbytesandrepresentstheindividualbatchpagesize.Iftheprocessor-batch-sizeisincreasedand/oryouaredealingwithextremelywideresultsets(severalhundredcolumns),thenthedefaultsettingof8MBforthemax-storage-object-sizemaybetoolow.Theinline-lobssettingalsocanincreasethesizeofbatchescontainingsmalllobs.Thesizingformax-storage-object-sizeisintermsofserializedsize,whichwillbemuchclosertotherawdatasizethantheJavamemoryfootprintestimationusedformax-reserved-kb.max-storage-object-sizeshouldnotbesettoolargerelativetomemory-buffer-spacesinceitwillreducetheperformanceofthememorybuffer.Thememorybuffersupportsonly1concurrentwriterforeachmax-storage-object-sizeofthememory-buffer-space.Notethatthisvaluedoesnottypicallyneedtobeadjustedunlesstheprocessor-batch-sizeisadjusted,inwhichcaseconsideradjustingitinproportiontotheincreaseoftheprocessor-batch-size.
NoteIfexceptionsoccurrelatedtomissingbatchesand"TEIID30001Maxblocknumberexceeded"isseenintheserverlog,thenincreasethemax-storage-object-sizetosupportlargerstorageobjects.Alternativelyyoucouldmaketheprocessor-batch-sizesmaller.
memory-buffer-space(default-1)-ThiscontrolstheamountofonoroffheapmemoryallocatedasbytebuffersforusebytheTeiidbuffermanagermeasuredinmegabytes.Thissettingdefaultsto-1,whichautomaticallydeterminesasettingbaseduponwhetheritisonoroffheapandthevalueformax-reserve-kb.Thememorybuffersupportsonly1concurrentwriterforeachmax-storage-object-sizeofthememory-buffer-space.Anyadditionalspaceservesasacachefortheserializedforofbatches.
Note
Whenleftatthedefaultsettingthecalculatedmemorybufferspacewillbeapproximately40%ofthemax-reserve-kbsize.Ifthememorybufferisonheapandthemax-reserve-kbisautomaticallycalculated,thenthememorybufferspacewillbesubtractedoutoftheeffectivemax-reserve-kb.Ifthememorybufferisoffheapandthemax-reserve-kbisautomaticallycalculated,thenit’ssizewillbereducedslightlytoallowforeffectivelymore
MemoryManagement
71
workingmemoryinthevm.
memory-buffer-off-heap(defaultfalse)-TakeadvantageoftheBufferManagermemorybuffertoaccesssystemmemorywithoutallocatingittotheheap.Settingmemory-buffer-off-heapto"true"willallocatetheTeiidmemorybufferoffheap.DependingonwhetheryourinstallationisdedicatedtoTeiidandtheamountofsystemmemoryavailable,thismaybepreferabletoon-heapallocation.TheprimarybenefitisadditionalmemoryusageforTeiidwithoutadditionalgarbagecollectiontuning.Thisbecomesespeciallyimportantinsituationswheremorethan32GBofmemoryisdesiredfortheVM.Notethatwhenusingoff-heapallocation,thememorymuststillbeavailabletothejavaprocessandthatsettingthevalueofmemory-buffer-spacetoohighmaycausetheVMtoswapratherthanresideinmemory.Withlargeoff-heapbuffersizes(greaterthanseveralgigabytes)youmayalsoneedtoadjustVMsettings.
Note
Oracle/SunVM-therelevantVMsettingsareMaxDirectMemorySizeandUseLargePages.Forexampleadding:'-XX:MaxDirectMemorySize=12g-XX:+UseLargePages'totheVMprocessargumentswouldallowforaneffectiveallocationofapproximatelyan11GBTeiidmemorybuffer(thememory-buffer-spacesetting)accountingforanyadditionaldirectmemorythatmaybeneededbytheASorapplicationsrunningintheAS.
DiskUsage
max-buffer-space(default-1)-Fortablepageandresultbatchesthebuffermanagerwillhavealimitednumberoffilesthatarededicatedtoaparticularstoragesize.However,asmentionedintheinstallation,creationofTeiidlobvalues(forexamplethroughSQL/XML)willtypicallycreateonefileperloboncethelobexceedstheallowableinmemorysizeof32KB.Inheavyusagescenarios,considerpointingthebufferdirectoryonapartitionthatisroutinelydefragmented.BydefaultTeiidwilluseupto50GBofdiskspace.ThisistrackedintermsofthenumberofbyteswrittenbyTeiid.Forlargedatasets,youmayneedtoincreasethemax-buffer-spacesetting.
Limitations
It’salsoimportanttokeepinmindthatTeiidhasmemoryandotherhardlimitswhichbreaksdownalongseverallinesintermsof#ofstorageobjectstracked,diskstorage,streamingdatasize/rowlimits,etc.
1. Thebuffermanagerhasamaxaddressablespaceof16terabytes-butduetofragmentationyou’dexpectthatthemaxusablewouldbeless.ThisisthemaximumamountofstorageavailabletoTeiidforalltemporarylobs,internaltables,intermediateresults,etc.
2. Themaxsizeofanobject(batchortablepage)thatcanbeserializedbythebuffermanageris32GB-butyoushouldapproachthatlimit(thedefaultlimitis8MB).Abatch/pageissetorrowsthatareflowingthroughTeiidengineandisdynamicallyscaledbasedupontheestimateddatawidthsothattheexpectedmemorysizeisconsistent.
3. Themax-processing-kbandmax-reserve-kbarebaseduponmemoryfootprintestimationsandnotexactsizes-actualmemoryusageandgarbagecollectioncyclesareinfluencedbyalotofotherfactors.
Handlingasourcethathastera/petabytesofdatadoesn’tbyitselfimpactTeiidinanyway.Whatmattersistheprocessingoperationsthatarebeingperformedand/orhowmuchofthatdatadoweneedtostoreonatemporarybasisinTeiid.Withasimpleforward-onlyquery,Teiidwillreturnapetabytesofdatawithminimalmemoryusage.
OtherConsiderationsforSizing
Eachbatch/tablepagerequiresaninmemorycacheentryofapproximately~128bytes-thusthetotaltrackedmaxbatchesarelimitedbytheheapandisalsowhywerecommendtoincreasetheprocessingbatchsizeonlargermemoryorscenariosmakinguseoflargeinternalmaterializations.Theactualbatch/tableitselfismanagedbybuffermanager,whichhaslayeredmemorybufferstructurewithspilloverfacilitytodisk.
MemoryManagement
72
UsinginternalmaterializationisbasedontheBufferManager.BufferManagersettingsmayneedtobeupdatedbaseduponthedesiredamountofinternalmaterializationperformedbydeployedvdbs.
Ifanoutofmemoryerroroccursitisbesttofirstcaptureaheapdumptodeterminewherememoryisbeingheld-tweakingtheBufferManagersettingsmaynotbenecessarydependinguponthecause.
CommonConfigurationScenarios
Inadditiontoscenariosoutlinedabove,acommonscenariowouldbetominimizetheamountofonheapspaceconsumedbyTeiid.Thiscanbedonebymovingthememorybuffertooffheapwiththememory-buffer-off-heapsettingorbyrestrictingthemax-reserve-kbsetting.Reducingthemax-processing-kbsettingshouldgenerallynotbenecessary,unlessthereisaneedtoseverlyrestricttheheapusagebeyondthemax-reserve-kbsetting.
MemoryManagement
73
Transport
max-socket-threads(default0)-Themaxnumberofthreadsdedicatedtotheinitialrequestprocessing.Zeroindicatestousethesystemdefaultofmaxavailableprocessors.AlltheaccesstoTeiid(JDBC,ODBC,etc)iscontrolledby"transport"elementintheconfiguration.Socketthreadsareconfiguredforeachtransport.TheyhandleNIOnon-blockingIOoperationsaswellasdirectlyservicinganyoperationthatcanrunwithoutblocking.Forlongerrunningoperations,thesocketthreadsqueuewithworkthequeryengine.
QueryEngine
max-threads(default64)-Thequeryenginehasseveralsettingsthatdetermineitsthreadutilization.max-threadssetsthetotalnumberofthreadsavailableintheprocesspoolforqueryenginework(processingplans,transactioncontroloperations,processingsourcequeries,etc.).Youshouldconsiderincreasingthemaximumthreadsonsystemswithalargenumberofavailableprocessorsand/orwhenit’scommontoissuenon-transactionalqueriesthatissuealargenumberofconcurrentsourcerequests.
max-active-plans(default20)-Shouldalwaysbesmallerthanmax-threads.Bydefault,thread-count-for-source-concurrencyiscalculatedby(max-threads/max_active_plans)*2todeterminethethreadsavailableforprocessingconcurrentsourcerequestsforeachuserquery.Increasingthemax-active-plansshouldbeconsideredforworkloadswithahighnumberoflongrunningqueriesand/orsystemswithalargenumberofavailableprocessors.Ifmemoryissuesarisefromincreasingthemax-threadsandmax-active-plans,thenconsiderdecreasingtheamountofheapheldbythebuffermanagerordecreasingtheprocessor-batch-sizetolimitthebasenumberofmemoryrowsconsumedbyeachplan.
thread-count-for-source-concurrency(default0)-Shouldalwaysbesmallerthanmax-threads,setsthenumberofconcurrentlyexecutingsourcequeriesperuserrequest.0indicatestousethedefaultcalculatedvaluebasedon2*(max-threads/max-active-plans).Settingthisto1forcesserialexecutionofallsourcequeriesbytheprocessingthread.Anynumbergreaterthan1limitsthemaximumnumberofconcurrentlyexecutionsourcerequestsaccording.Usingtherespectivedefaults,thismeansthateachuserrequestwouldbeallowed6concurrentlyexecutingsourcequeries.Ifthedefaultcalculatedvalueisnotapplicabletoyourworkload,forexample,ifyouhavequeriesthatgeneratemoreconcurrentlongrunningsourcequeries,youshouldadjustthisvalue.
time-slice-in-milliseconds(default2000)-Providescourseschedulingoflongrunningprocessorplans.Planswhoseexecutionexceedatimeslicewillbere-queuedforadditionalprocessingtoallowforotherplanstobeinitiated.Thetimesliceisfromtheperspectiveoftheengineprocessingthread.Thisvalueisnothonoredexactlyastheplanmaynotbeatare-startablepointwhenthetimesliceexpires.ThisisnotareplacementforthethreadschedulingperformedbyJavaandtheoperatingsystem,ratheritjustensuresthatTeiidallowsotherworktobestartedifthecurrentsetofactiveplansincludeslongrunningqueries.
Threading
74
CacheTuningCachingcanbetunedforcachedresults(includinguserqueryresultsandprocedureresults)andpreparedplans(includinguserandstoredprocedureplans).Eventhoughitispossibletodisableorotherwiseseverelyconstrainthesecaches,thiswouldprobablyneverbedoneinpracticeasitwouldleadtopoorperformance.
CachestatisticscanbeobtainedthroughtheAdminConsoleorAdminshell.Thestatisticscanbeusedtohelptunecacheparametersandensureahitratio.
Plansarecurrentlyfullyheldinmemoryandmayhaveasignificantmemoryfootprint.Whenmakingextensiveuseofpreparedstatementsand/orvirtualprocedures,thesizeoftheplancachemaybeincreasedproportionallytonumberofgigabytesintendedforusebyTeiid.
Whiletheresultcacheparameterscontrolthecacheresultentries(maxnumber,eviction,etc.),theresultbatchesthemselvesareaccessedthroughtheBufferManager.Ifthesizeoftheresultcacheisincreased,youmayneedtotunetheBufferManagerconfigurationtoensurethereisenoughbufferspace.
Resultsetandpreparedplancacheshavetheirentriesinvalidatedbydataandmetadataevents.BydefaulttheseeventsarecapturedbyrunningcommandsthroughTeiid.SeetheDevelopersGuideforfurthercustomization.Teiidstorescompiledformsofupdateplansortriggeractionswiththepreparedplan,sothatifmetadatachanges,forexamplebydisablingatrigger,changesmaytakeeffectimmediately.Thedefaultmax-stalenessforresultsetcachingis0secondsorimmediateinvalidation.Considerincreasingthisvaluetoincreaseresultsetcachehits.Evenwithasettingof0,fulltransactionalconsistencyisnotguaranteed-rathertheunderlyingInfinispancachemustbeconfiguredwithatransactionmodeofXA.
CacheTuning
75
SocketTransportsTeiidseparatestheconfigurationofitssockettransportsforJDBCandODBC.Typicalinstallationswillnotneedtoadjustthedefaultthreadandbuffersizesettings.Thedefaultvaluesforinput-buffer-sizeandoutput-buffer-sizearesetto0,whichwillusethesystemdefault.Beforeadjustingthesevalues,keepinmindthateachJDBC/ODBCconnectionwillcreateanewsocket.Settingthesevaluestoalargebuffersizeshouldonlybedoneifthenumberofclientsareconstrained.AllJDBC/ODBCsocketoperationsarenon-blocking,sosettingthenumberofmax-socket-threadshigherthanthemaximumeffectiveparallelismofthemachineshouldnotresultingreaterperformance.Thedefaultvalue0indicatesthesystemdefaultof2*availableprocessorswillbeused.
NoteIfyouareusingmorethanthe2defaultsockettransportsonamachinewithahighnumberofactualorvirtualcores,youmayneedtoconsidermanuallyconfiguringthemaxthreadsforeachtotransporttocutdownonthenumberofthreadscreated.
JDBCclientsmayneedtoadjustlow-leveltransportvalues,inadditiontoSSLClientConnectionpropertiesviaateiid-client-settings.propertiesfile.Thisfilealsocontainsbuffer,socketpooling,andmaxObjectSize(effectivelythemaximumresponsesize)settings.
SocketTransports
76
LOBsLOBsandXMLdocumentsarestreamedfromtheTeiidServertotheTeiidJDBCAPI.Normally,thesevaluesarenotmaterializedintheservermemory-avoidingpotentialout-of-memoryissues.Whenusingstylesheetsandnon-streamingXQuerywholeXMLdocumentsmustbematerializedontheserver.EvenwhenusingtheXMLQueryorXMLTablefunctionsanddocumentprojectionisapplied,memoryissuesmayoccurforlargedocuments.
LOBsarebrokenintopieceswhenbeingcreatedandstreamed.Themaximumsizeofeachpiecewhenfetchedbytheclientcanbeconfiguredwiththe"lob-chunk-size-in-kb"propertyonTeiidconfiguration.Thedefaultvalueis100KB.WhendealingwithextremelylargeLOBs,youmayconsiderincreasingthisvaluetodecreasetheamountofround-tripstostreamtheresult.Settingthevaluetoohighmaycausetheserverorclienttohavememoryissues.
SourceLOBvalues(LOBsfromphysicalsources)aretypicallyaccessedbyreference,ratherthanhavingthevaluecopiedtoatemporarylocation.ThuscaremustbetakentoensurethatsourceLOBsarereturnedinamemory-safemanner.ThiscautionismoreforthesourcedrivervendorstonottoconsumeVMmemoryforLOBs.TranslatorsviathecopyLobspropertycaninsteadcopylobvaluestoatemporarylocation.
Cachedlobswillbecopiedratherthanrelyingonthereferencetothesourcelob.
TemporarylobscreatedbyTeiidwillbecleanedupwhentheresultsetorstatementisclosed.Torelyonimplicitgarbagecollectionbasedcleanupinsteadofstatementclose,theTeiidsessionvariableclean_lobs_onclosecanbesettofalse(byissuingthequery"SELECTteiid_session_set('clean_lobs_onclose',false)"-whichcanbedoneforexampleviathenewconnectionsqlinthedatasourcedefinition).Thiscanbeusedforlocalclientscenariosthatreliedontheimplicitbehavior,suchasDesignergeneratedRESTVDBs.
LOBs
77
OtherConsiderationsWhenusingTeiidinadevelopmentenvironment,youmayconsidersettingthemax-source-rows-allowedpropertytoreasonablysmalllevelvalue(e.g.10000)topreventlargeamountsofdatafrombeingpulledfromsources.Leavingtheexception-on-max-source-rowssetto"true"willalertthedeveloperthroughanexceptionthatanattemptwasmadetoretrievemorethanthespecifiednumberofrows.
OtherConsiderations
78
TeiidConsoleTeiidConsoleisawebbasedadministrativeandmonitoringtoolforTeiid.TeiidConsoleisextensionofWildFlyconsolethatisbuiltusingGWTbasedtechnologies.TheTeiidkitdoesnotincludefilesforTeiidConsole.ThisisseparatedownloadyoucandownloadfromTeiidDownloads.
Installation
OnceyoudownloadtheTeiidConsole,unzipthecontentsovertheWildFlyrootdirectoryandalltherequiredfileswillbeoverlayedcorrectlytoinstallTeiidConsole.TheTeiidConsole,bydefaultissecured,soyouwouldneedamanagementrealmuseridandpasswordtologin.Inthe<install>/bindirectory,use
Addingamanagementuserinlinux
./add-user.sh
AddingamanagementuserinWindows
add-user.bat
thenfollowthepromptstocreateManagementRealmuser.Onceyouhavecreatedamanagementuser,youneedtousethesecredentialstologintotheTeiidConsole.IfyouhavestartedyourWildFlyindefaultmode,thenyoucanaccesstheTeiidConsoleathttp://localhost:9990/console/App.html.Ifyouhavealteredthebindingorportnumbersthenmodifytheaddressaccordingly.
ProfileView
Profileviewisforconfiguration.Clickon"profile"ontoprighthandcornerofthemainconsolescreen,togotoProfileView.Youcanclickon"Teiid"inleftnavigationtreetogotoTeiidconfiguration.Thereyouhavethreechoices
QueryEngine-viewandconfigurecoreTeiidengineproperties.
Translators-view,addandremovetheTranslatorsconfiguredinTeiid.
Transports-view,addandremovetransportstotheTeiidengine.
TeiidConsole
79
UsingthisviewyoucanchangeanyconfigurationvalueofTeiid.Notethatvariousdifferentconfigurationpropertiesaresub-dividedintodifferenttabs.Youcanclick"NeedHelp"linkonthesepagestoseethedescriptionofeachfieldonthepage.
Note ServerRestart-Notethatsomepropertiesrequireyoutorestarttheserverbeforetheycantakeeffect.
RuntimeViewRuntimeviewshowsruntimeinformationaboutAS7andTeiidsubsystem.RuntimeinformationaboutTeiidcanbeviewedbyselecting"VirtualDatabases"onlefthandnavigationaltree.
TeiidConsole
80
UsingthispageusercanviewmanydifferentsettingsinthecontextaVDB.AlltheVDBsdeployedintheserverareshownintopleveltable.WhenyouselectandhighlightaVDB,moredetailsaboutthatVDBaredisplayedinthesub-tabsbelow.Eachofthesesub-tabsaredividedintogroupingofthefunctionality.
Summary
ThistabshowsthedescriptionandanypropertiesassociatedwithVDB,alongwithanyotherVDBsthisVDBimports.ThistabisdesignedtogiveaquickoverviewoftheVDBstatus.
Models
ThispanelshowsallthemodelsthataredefinedinagivenVDB,andshowseachmodelstranslatornameandsourceconnectionJNDIname.Italsoshowsthetypeofmodelsandifitismulti-sourceornot.Whenaparticularmodelisselecteditwillshowallpropertiesofthatmodelthataredefinedandalsoshowsanyerrorsassociatedwiththemodel.WhenyourVDBisnotdeployedinthe"active"status,youwouldneedtoverifytheseerrorsandfixtoresolveanydeploymentissues.
The"DDL"buttonshowstheschemaforthegivenmodel.NotethatthisdoesnotworkforXMLbasedmodelsthataredesignedusingTeiidDesigner.
ThetoolletstheusereditthetranslatornameorJNDInamebydoubleclickingonthemandmodifyingthem.ThisusefulifyouwouldliketochangetheJNDInameinagivenenvironment.
Overrides
IfyouhaveoverriddenanytranslatorsintheTeiidDesigner,thispanelwillshowthealltheoverriddentranslatorsandtheirproperties.
TeiidConsole
81
Caching
Cachingpanelshowscachingstatisticsofresultsetcacheastohoweffectivelythecacheisbeingused.ItalsoshowsalltheinternalmaterializedviewsintheVDBandtheirloadstatusastowhentheywereloaded.ItalsogivesoptionstoinvalidateaspecificvieworalltheviewsinaVDB,sothattheycanrefresh/reloadthecontentsfromsource.
ThispanelalsoprovidesaUItoflushtheentiretheresultsetcachecontentsorpreparedplancachecontentsfortheselectedVDB.
DataRoles
DataRolespanelshowstheallthepoliciesthatdefinedintheVDBusingtheTeiidDesignerorhandcodedinthevdb.xmlfile.Foreachselectedpolicy,itwillalsolistthe"permissions"forthatpolicyastowhatkindofauthorizationsuserhasandshowsthemappedenterpriseroleassignmentstothatpolicy.Youcanevenadd/removeaenterpriseroletothepolicyusingthethisUI.
Requests
ThispanelshowsallthecurrentrequestsagainsttheselectedVDBatthetimeofVDBselection.Youcanclick"refresh"togetamoreuptodaterequests.Thetoptableinthepanelshowstheusersubmittedrequeststotheteiidengine,whenoneofthoserequestsareselected,thenthebottomtableshowsallthesourcelevelqueriesthataresubmittedtothephysicalsourcesbyTeiidengine.
UsingthisUI,usercanalsosubmita"cancel"requesttoauserlevelquery.Since"cancel"asynchronousoperation,theoperationisnotguaranteedasquerymayalreadybeenfinished,bythetimecancelissubmitted.
Sessions
ThispanelshowsalltheactivesessionsthatareconnectedtotheselectedVDB.Itshowstheirconnectionpropertiesandalsogivesanoptiontoterminateeitheraselectedsessionorallthesessions.
FAQ
HowtodeployaVDBinstandalonemode?
IntheDeploymentsview,clickaddandselecttheVDBtodeploy.AlsomakesureyouenabletheVDBonceitisdeployed.
HowtocreateDatasource?
IntheConfigurationview,gotoSubsystem→Datasources→XA/Non-XA,clickaddandfollowthewizardtocreateJDBCdatasource.
IfyoutryingtocreateconnectiontoTeiidbasedFile,SalesforceorWSbasedconnections,selectSubsystem→ResourceAdaptorsandclickadd.
HowtoaddCOMMANDLogging?
IntheConfigurationview,gotoSubsystem→Logging,clickview,onLogCategoriestab,clickaddorg.teiid.COMMAND_LOGinDEBUGmode.ThedefaultlogwillbeintheFILEhandler.Youcanevenaddotherhandlerifchoosetodoso.
ChangeTeiidJDBCPortinstandalonemode?
IntheConfigurationview,gotoSocketBindingclickView,viewthestandard-socketsselectteiid-jdbcandedit.
TeiidConsole
82
TeiidConsole
83
AdminShellTheAdminShelltoolingprovidesscriptingbasedprogrammingenvironmentsthatenableusertoaccess,monitorandcontrolaTeiidServer.BoththecommandlineandgraphicalconsoletoolsarebuiltonfunctionalityprovidebytheGroovy(http://groovy.codehaus.org/)project.TheAdminShelltoolscanbeusedinad-hocscriptingmodeortorunpre-definedscripts.
AdminShellfeatures:
1. fullyfunctionalprogrammingenvironmentwithresourceflowcontrolandexceptionmanagement.SeeGroovydocsforthefullpowerofthelanguage.
2. quickadministrativetool.TheusercanconnecttoarunningTeiidServerandinvokeanyoftheAdminAPImethods,suchas"deploy"or"createDataSource",tocontroltheTeiidSystem.Sincethiscanbescriptdriven,thesetaskscanbeautomatedandre-runatalatertime.
3. simplifieddataaccesstool.TheusercanconnecttoaVDB,issueanySQLcommands,andviewtheresultsofthequeryviaGroovySqlextensions.
4. migrationtool.ThiscanbeusedtodevelopscriptslikemovingtheVirtualDatabases(VDB),ConnectionFactories,andConfigurationfromonedevelopmentenvironmenttoanother.Thiswillenableuserstotestandautomatetheirmigrationscriptsbeforeproductiondeployments.
5. testingtool.TheJUnit(http://junit.org)testframeworkisbuiltin,seeGroovyUnitTests.Usercanwriteregressiontestsforcheckingsystemhealth,ordataintegritythatcanbeusedtovalidateasystemfunctionalityautomaticallyinsteadofmanualverificationbyQApersonnel.
Download
AdminShellisdistributedalongwithotherTeiiddownloadsunder"teiid-{version}-adminshell-dist.zip"name.Downloadandunzipthisfiletoanydirectory.Onceyouhaveunzippedthefile,inrootdirectoryyouwillfind"adminshell"and"adminshell-console"executablescriptstolaunchthecommandlineandgraphicaltoolsrespectively.
Windows:Doubleclickorexecute"adminshell.cmd"
nix:Executethe"adminshell.sh"script
AdminShell
84
GettingStartedTolearnthebasicsofGroovytakealookattheirdocumentsandtutorialsontheirwebsite.
BasicknowledgeoftheJavaprogramminglanguageandtypesisrequiredinordertoeffectivelydesignanddevelopscriptsusingtheAdminShell.TolearnJavalanguagefindlearningresourcesathttp://java.sun.com.
YoucanlearnabouttheTeiidAdminAPIeitherusing"adminHelp()"functionorbyusingtheJavaDocs.
AdminShellisaspecializedversionofGroovywhichworksinseveraldifferentmodes:interactiveshell,graphicalconsole,orscriptrunmode.Ininteractiveshellmode(launchedviaadminshell),theusercaninvokeconnecttoaliveTeiidsystemandissueanyad-hoccommandstocontrolthesystem.Theinteractivebuffercanbeusedtodevelopascriptandtheinteractivesessioninputandoutputcanbecapturedintoalogfile,moreonthislaterinthedocument.
Ingraphicalmode(lanchedviaadminshell-console),theusercandevelopandrunscriptsusingatexteditorthatsupportssyntaxhighlighting.
Inthescriptrunmode,theusercanexecute/playbackpreviouslydevelopedscripts.Thismodeespeciallyusefultoautomateanytestingortoperformanyrepeatedconfigurations/migrationschangestoaTeiidsystem.
EssentialRules
TouseAdminShellsuccessfully,therearesomebasicsyntacticalrulestokeepinmind.
1. Ininteractiveshellmode,mostcommands(asseenbythehelpcommand)areusedtocontrolshellbehaviorandarenotgeneralGroovyscriptingconstructs.Adminmethodswilltypicallybecalledusingfunctionalnotation:
connectAsAdmin()
1. Allcommandsandfunctionsarecasesensitive.
2. AnendingsemicolonisoptionalforGroovystatements.
3. Ifafunctionrequiresinputparameter(s),theyshouldbedeclaredinside"("and")".Afunctionmayhavemorethanoneparameter.Stringparameterscanbewrappedindoubleorsinglequotes.Example:
connectAsAdmin("localhost","9990","user","password","conn1")
1. Localadminconnectionmaynotrequireapassword.Remoteconnectionswilltypicallybeoverthe9993httpsportinstead.OtherJavamethodsandclassescanbeusedfromyourscripts,iftherequiredJavaclasslibrariesarealreadyinclasspath.Youmayplaceadditionaljarsinthelibdirectorytohavebeautomaticallypartoftheclasspath.Anexampleshowinganimport:
importmy.package.*;
myObject=newMyClass();
myObject.doSomething();
Toexecutethecommandsandarbitraryscriptininteractivemodeyouenterthemfirstandpressentertoexecute,thenenterthenextline,soon.
Toexitthetoolintheinteractivemode,firstdisconnectifyouareconnectedtotheTeiidsystembyexecuting"disconnect();"thentype"exit".Inthescriptmode,whenexecutionofthescriptfinishesthetoolwillexitautomatically,howeveryoustillhavetodisconnectfromTeiidsysteminthescript.
GettingStarted
85
Note IfSSListurnedontheTeiidserver,youwouldneedtoadjusttheconnectionURLandtheclientSSLsettingsasnecessary(typicallythiswillonlybeneededfor2-waySSL).
Help
TheadminHelp()methodslistsalltheavailableadministrativeAPImethodsintheAdminShell.PleasenotethatnoneoftheGroovyShellcommandsorotheravailablefunctioncallswillbeshowninthislist
adminHelp();
Togetaspecificdefinitionaboutamethodandit’srequiredinputparameters,useadminHelp("method")
adminHelp("deploy");
/*
*DeployaVDBfromfile
*/
voiddeploy(
String/*filename*/)
throwsAdminException
throwsFileNotFoundException
ThesqlHelp()methodslistsallSqlextensionmethods.
sqlHelp();
Togetaspecificdefinitionaboutamethodandit’srequiredinputparameters,usesqlHelp("method")
BasicCommands
ThelistbelowcontainssomecommoncommandsusedinAdminShell.
BasicCommands
println"xxx";//printsomethingtoconsole
adminHelp();//showsalltheavailableadmincommands;
sql=connect();//getanextendedGroovySqlconnectionusingconnection.propertiesfile
sql.execute(<SQL>);//runanySQLcommand.
connectAsAdmin();//connectasadmin;noneedhavethevdbname.SQLcommandswillnotworkunderthisconnect
ion
printlngetConnectionName();//returnsthecurrentconnectionname
useConnection(<connectionname>);//switchestousingthegivenconnectionsettings
disconnect();//disconnectsthecurrentconnectioninthecontext
Someexamples
ExampleofDeployingaVDB
connectAsAdmin();
deploy("/path/to/<name>.vdb");
GettingStarted
86
//checktovalidatethedeployment
VDBvdb=getVDB("<name>",1);
if(vdb!=null){
print(vdb.getName()+"."+vdb.getVersion()+"isdeployed";
}
else{
print("<name>.vdbfailedtodeploy";
}
CreateaDatasource(oracle)
connectAsAdmin();
//firstdeploytheJDBCjarfileforOracle
deploy("ojdbc6.jar");
props=newProperties();
props.setProperty("connection-url","jdbc:oracle:thin:@<host>:1521:<sid>");
props.setProperty("user-name","scott");
props.setProperty("password","tiger");
createDataSource("oracleDS","ojdbc6.jar",props);
CreateaResourceAdapter(file)basedDatasource
connectAsAdmin();
props=newProperties();
props.setProperty("jndi-name","java:/fileDS");
props.setProperty("ParentDirectory","${jboss.server.base.dir}/myData");
props.setProperty("AllowParentPaths","true");
//NOTE:the2ndargument,template-name,mustmatchthe'id'ofoneoftheresource-adaptersthatarecurrent
lydefinedintheserver
createDataSource("MyFile","file",props);
ExecuteSQLQueryagainstTeiid
sql=connect("jdbc:teiid:<vdb>@mm://<host>:31000","user","user");
//select
sql.eachRow("select*fromsys.tables"){println"${it}"}
//update,insert,delete
sql.execute(<sqlcommand>);
Note LearnmoreaboutGroovySQL
GettingStarted
87
ExecutingascriptfileToexecuteascriptfile"foo.groovy"inadirectory"some/directory"intheinteractivecomamndlinetool,executeasfollowing
.some/directory/foo.groovy
"foo.groovy"isreadintocurrentcontextoftheshellasifyoutypedinthewholedocument.Ifyourscriptonlycontainedmethodcalls,youcanexplicitlyinvokethecalltoexecute.
Fullexecutesyntaxmayalsobeused,andisrequiredoutsideoftheinteractivecommandlinetool:
evaluate("some/directory/foo.groovy"asFile)
Toexecutethesamefilewithoutenteringinteractivemode,run
./adminshell.sh.some/directory/foo.groovy
ParameterscanbepassedinasJavaSystemproperties.Forexample
./adminshell.sh-Dparam=value.some/directory/foo.groovy
Insidethescriptfile,youcanaccessthesepropertiesusingSystem.getProperty
value=System.getProperty(“param”);//willreturn"value"
Executingascriptfile
88
LogFileandRecordedScriptfile
Duringtheinteractivemode,inputisrecordedinahistoryfile.Thisfilecanbeaccessedviatheuparrowintheinteractiveshell.
Usercanalsocapturethecommandsenteredduringainteractivesessiontotheirownscriptfilebyusing"startRecording"and"stopRecording"commands.Forexample,
recordstartdirectory/filename.txt
<commandsandscript..>
recordstop
Allinputandoutputbetweenthestartandstoparecapturedinthe"directory/filename.txt"file.Thisgivestheuseranoptiontocaptureonlycertainportionsoftheinteractivesessionandtolaterrefineascriptoutofrecordedfile.
LogFileandRecordedScriptfile
89
DefaultConnectionPropertiesThefile"connection.properties"intheinstallationdirectoryoftheAdminShelldefinesthedefaultconnectionpropertieswithwhichusercanconnecttoTeiidsystem.Thefollowingpropertiescanbedefinedusingthisfile
jdbc.user=user
jdbc.password=user
jdbc.url=jdbc:teiid:admin@mm://localhost:31000;
admin.host=localhost
admin.port=9990
admin.user=admin
admin.password=admin
Acallto"connect()"or"connectionAsAdmin()"withoutanyinputparameters,willconnecttotheTeiidsystemusingthepropertiesdefinedinpropertiesfile.However,ausercanalwayspassinparametersintheconnectmethodtoconnecttoasameordifferentserverthanonementionedinthe"connection.properties".Lookalltheallthedifferentconnectmethodsusingthe"adminHelp()"method.
NoteNotethatitisnotsecuretoleavethepasswordsincleartext,asdefinedabove.Pleasetakenecessarymeasurestosecurethepropertiesfile,ordonotusethisfeatureandalwayspassinpasswordinteractivelyorsomeothersecureway.
NoteAtanygiventimeusercanbeactivelyconnectedtomorethanonesystemorhavemorethanoneconnectiontosamesystem.Tomanagetheconnectionscorrectlyeachconnectioniscreatedgivenauniqueconnectionname.TolearnmoreaboutthislookatHandlingMultipleConnections.
DefaultConnectionProperties
90
HandlingMultipleConnectionsUsingAdminShell,ausercanactivelymanagemorethanoneconnectiontoasingleormultipleTeiidsystems.Forexample,twoseparateconnectionscanbemaintained,onetothedevelopmentserverandonetotheintegrationserveratthesametime.ThisispossiblebecauseAdminShellsupportsafeaturecallednamedconnections.
Everytimeaconnectionismade,theconnectionhasanexplicitoranimplicitlyassignedname.Ifanotherconnectcommandisexecutedthenanewconnectionismadewithauniquenameandexecutionwillbeswitchedtousethenewconnection.Thepreviousconnectionwillbeheldasitisinitscurrentstate,andwillnotbeclosed.
Youcanusethefollowingcommandtofindoutthecurrentconnection’sname
name=getConnectionName();
Knowingthenamesoftheconnectionthatuserisworkingwithisimportanttoswitchtheactiveconnectiontoapreviousconnection.Toswitchtheactiveconnection,usethefollowingcommandandsupplythenameoftheconnectiontobeused
useConnection("name");
Ifusersuppliesthesamenameastheactiveconnectionastheyarecurrentlyparticipatingin,thenthisoperationwillsimplyreturnwithoutanymodifications.Thereisnolimitationthenumberofsimultaneousconnections.
Thefollowingshowsanexampleofusingandswitchingbetweentwoconnections.
//createsaconnection
connectAsAdmin();
//capturetheconnectionname
conn1=getConnectionName();
deploy("file.vdb")
//createsasecondconnection
connectAsAdmin();
conn2=getConnectionName();
deploy("file.vdb")
//switchtheconnectionto"conn1"
useConnection(conn1);
//closetheconnectioninthe"conn1"
disconnectAll();
HandlingMultipleConnections
91
InteractiveShellNuancesTheinteractiveshellusesaspecialshellinterpretterandthereforehasdifferentbehaviorthanjustwrittingascriptinGroovy.SeetheGroovyShellDocumentationformoreonitsusage.Notabledifferences:
DefstatementsdonotdefineavariableinthecontextoftheShell,e.g.donotusedefx=1,usex=1
Shellcommands(asseenthroughhelp)usingthenon-functionalshellsyntaxareonlyavailableintheshell.
GroovyclassesusingannotationscannotbeparsedintheShell.
InteractiveShellNuances
92
OtherScriptingEnvironmentsTheAdminShellmethods(namedcontextualconnections,AdminAPIwrapper,andhelpsystem)havenodirectdependenciesonGroovyandcanbeusedinotherscriptinglanguages.
TousetheAdminShellmethodsinanotherlanguage,simplyimportthestaticmethodsandAdminclassesintoyourscript.Youwillalsoneedtoensurethatthe<adminshelldist>/lib/teiid-7.6-client.jarand<adminshelldist>/lib/teiid-adminshell-7.6.jarareinyourclasspath.ThesnippetbelowshowimportstatementsthatwouldworkinJava,BeanShell,Groovy,etc.
importstaticorg.teiid.adminshell.AdminShell.*;
importstaticorg.teiid.adminshell.GroovySqlExtensions.*;
importorg.teiid.adminapi.*;
Notethattheprovidedshellandconsoleexecutablesautomaticallyhavetheproperclasspathsetandinjecttheproperimportsintorunningscripts.Ifyouwishtousescriptsinanon-TeiidGroovyenvironment,youwillneedtomanuallyaddtheseimportsandaddtheadmin/clientjarstotheclasspath.
OtherScriptingEnvironments
93
SystemPropertiesSomeofTeiid’slow-levelbehaviorcanbeconfiguredviasystemproperties,ratherthanthroughconfigurationfiles.AtypicalplacetosetsystempropertiesforWildFlylaunchesisinthe<install>/bin/<mode>.conf.Apropertysettinghastheformat-Dproperty=value.
General
org.teiid.allowNanInfinity-defaultstofalse.SettotruetoallownumericfunctionstoreturnNaN(NotANumber)and+-Infinity.NotethatthesevaluesarenotcoveredbytheSQLspecification.
org.teiid.useValueCache-defaultstofalse.Settotruetoenablethecanonicalvaluecache.ValuecachingisuseddynamicallywhenbuffermemoryisconsumedtoreuseidenticalvaluesandthusreducethememoryconsumedbyTeiid.Thereisacomputationcostassociatedwiththecachelookup,soenablingthissettingisnotappropriateforinstallationshandlinglargevolumesofdissimilardata.
org.teiid.ansiQuotedIdentifiers-defaultstotrue.SettofalsetoemulateTeiid6.xandpriorbehavioroftreatingdoublequotedvalueswithoutleadingidentifierpartsasstringliterals,whichisnotexpectedbytheSQLspecification.
org.teiid.subqueryUnnestDefault-defaultstofalse.Iftrue,theoptimizerwillaggressivelyunnestsubqueriesinWHEREpredicates.Ifpossiblethepredicatewillbeunnestedtoatraditionaljoinandwillbeeligiblefordependentjoinplanning.
org.teiid.ODBCPacketSize-defaultsto307200.TargetsizeinbytesoftheODBCresultsbuffer.Thisisnotahardmaximum,lobsandwiderowsmayuselargerbuffers.
org.teiid.decimalAsDouble-defaultstofalse.Settotruetoparseexactfixedpointliterals,e.g.1.0,asdoublevaluesratherthanasdecimal/BigDecimalvaluesandtoreturnadoublevaluefromtheAVGfunctionforintegralvaluesinthesamewayasreleasesearlierthan8.0.
org.teiid.comparableLobs-defaultstofalse.SettotruetoallowblobandclobcolumnvaluestobecomparableinTeiid.Sourcetypemetadatawilldetermineifthecomparisoncanbepusheddown.
org.teiid.comparableObject-defaultstofalse.SettotruetoallowobjectcolumnvaluestobecomparableinTeiid.Sourcetypemetadatawilldetermineifthecomparisoncanbepusheddown.Theobjectinstancesareexpectedtocorrectlyimplementjava.lang.Comparable.compareTo.IftheinstanceobjectisnotComparable,thenClassCastExceptionsmaythethrown.
org.teiid.padSpace-defaultstofalse.SettotruetocomparestringsasifPADSPACEcollationisbeingused,thatisstringsareeffectivelyrightpaddedtothesamelengthforcomparison.Ifthispropertyisset,itisnotnecessarytousethetrimStringstranslatoroption.
org.teiid.collationLocale-notsetbydefault,whichmeansthatJava’snatural(UTF-16)stringcomparisonwillbeused.SettoaJavalocalestringlanguage[_country[_varient]],wherelanguage,country,andvariantaretwolettercodes-seejava.util.Localeformoreonvalidcodes.Notethateveniforg.teiid.comparableLobsisset,clobvalueswillnotbecomparedusingthelocalecollator.
org.teiid.clientVdbLoadTimeoutMillis-defaultsto5minutes.Thedefaultamountoftimeaclient(currentlyonlylocalclients)willwaittomakeaconnectiontoanactiveVDBbeforethrowinganexception.ClientsmayoverridethissettingviatheloginTimeoutconnectionproperty.
org.teiid.enDateNames-defaultstofalse.SettotruetouseEnglishmonthanddaynamesforthesystemfunctiondayNameandmonthName,ratherthanreturningnamesfromtheJavadefaultlocale.Priorto8.2dayNameandmonthNamealwaysreturnedEnglishnames.
org.teiid.pushdownDefaultNullOrder-defaultstofalse.Settotruetomimic8.1andpriorreleasebehaviorofpushingtheTeiid’sdefaultnullorderofnullslowifthesourcehasadifferentdefaultnullorderandsupportsexplicitnullordering.
SystemProperties
94
org.teiid.requireTeiidCollation-defaultstofalse.SettotruetoforceallsortstobeinTeiid’scollation(seeorg.teiid.collationLocale).
org.teiid.implicitMultiSourceJoin-defaultstotrue.SettofalsetodisableTeiid8.2andpriorreleasebehaviorofimplicitlypartitioningjoinsbetweenmulti-sourcetables.Whensettofalseandexplicitpredicatesuchastbl1.source_name=tbl2.source_nameisrequiredtopartitiontheresultsofthejoin.
org.teiid.maxStringLength-defaultsto4000.SetsthenominalmaximumlengthofstringsinTeiid-mostoperationsinTeiidwilltruncatestringsthatarelargerthanthisvalue.Settingthisvaluecanalsoadjustthemaxsizeoflobbytesheldinmemory.NOTE:sourcesmaynotappropriatelyhandlestringvaluesthatarelargerthanthesourcesupports.
org.teiid.assumeMatchingCollation-defaultstofalse.IffalseandatranslatordoesnotspecifyacollationLocale,thenasortinvolvingcharacterdataforasort/mergejoinwillnotbepushed.TeiiddefaultstotheJavaUCS-2collation,whichmaynotmatchthedefaultcollationforsources,particulartables,orcolumns.Youmaysetthesystempropertyorg.teiid.assumeMatchingCollationtotruetorestoretheolddefaultbehaviororselectivelyupdatethetranslatorstoreportacollationLocalematchingorg.teiid.collationLocale(UCS-2ifunset).
Note Stringsarealwaysfullyheldinmemory.Donotsetthisvaluetoohighasyoumayexperienceoutofmemoryerrors.
org.teiid.calendarTimestampDiff-defaultstotrue.SettofalsetousetheTeiid8.2andoldcomputationoftimestampdiff.notethat:usingtheoldbehaviorcanresultindifferingresultsbetweenpushedandnon-pushedversionsoftimestampdiffforintervalsgreaterthansecondsassourcesusedatepartandnotapproximateintervaldifferences.
org.teiid.compactBufferFiles-defaultstofalse.SettotruetohaveTeiidkeepthebufferfilesmorecompact(minimizingsparseregions).
org.teiid.maxMessageSize-defaultsto2097152.Themaximumsizeofmessagesinbytesthatareallowedfromclients.Increaseonlyifclientsroutinelyuselargequeriesand/ornon-lobbindvalues.
org.teiid.maxStreamingLobSize-defaultsto4294967296.Themaximumsizeoflobsinbytesthatareallowedtobestreamedaspartofthemessagefromclients.
org.teiid.defaultIndependentCardinality-defaultsto10.Thenumberofindependentrowsorlessthatcanautomaticallytriggeradependentjoin.Increasewhentablestypicallyonlyhavecardinalitysetandmoredependentjoinsaredesired.
org.teiid.checkPing-defaultstotrue.CanbesettofalsetodisablepingcheckingforremoteJDBCconnections.Pingcheckingshouldonlybedisabledinspecificcircumstances,suchaswhenusinganexternalloadbalancerandnotutilizingtheTeiiddefaultloadbalancinglogic.
org.teiid.defaultNullOrder-defaultstoLOW.CanbeoneofLOW,FIRST,HIGH,LAST.SetsthedefaultnullorderfortheTeiidengine.Thiswillnotbeusedforsourceorderingunlessorg.teiid.pushdownDefaultNullOrderisalsoset.
org.teiid.iso8601Week-defaultstotrue.SettotruetouseISO8601rulesforweekcalculationsregardlessofthelocale.Whentruetheweekfunctionwillrequirethatweek1ofayearcontainstheyear’sfirstThursday.PushdownweekvalueswillbecalculatedasISOregardlessofthissetting.
org.teiid.widenComparisonToString-defaultstofalse.Settotruetoenablewideningofvaluestostringincomparisons,whichwasthedefaultbehaviorpriortoTeiid9.Forexamplewiththissettingasfalsetimestamp_col<'a'willproduceanexceptionwhereaswhensettotrueitwouldeffectivelyevaluatecast(timestamp_colasstring)<`a'.
org.teiid.aggressiveJoinGrouping-defaultstotrue.Settofalsetonotaggressivelygroupjoins(typicallyallowedifthereexistsanexplicitrelationship)againstthesamesourceforpushdownandrelymoreuponacostbasedordering.
Security
org.teiid.allowCreateTemporaryTablesByDefault-defaultstofalse.Settotruetousethepre-8.0behaviorofallowinganyauthenticatedusertocreatetemptableswithoutanexplicitpermission.
SystemProperties
95
org.teiid.allowFunctionCallsByDefault-defaultstofalse.Settotruetousethepre-8.0behaviorofallowinganyauthenticatedusertocallanynon-systemfunctionwithoutanexplicitpermission.
org.teiid.ignoreUnauthorizedAsterisk-defaultstofalse.Iftrueunauthorizedcolumns(asdeterminedbydatarolechecking)arenotpartofselectallorqualifiedselectallexpansion.Iffalse,theclientmaysetthesessionvariableignore_unauthorized_asterisktotruetoachievethesamebehavior.
org.teiid.sanitizeMessages-defaultstofalse.IftruequeryrelatedexceptionandwarningswillhavetheirmessagesreplacedwithjusttheTeiidcode.Serversidestacktraceswillalsoberemovedwhensenttotheclient.ThisshouldbeenabledifthereisaconcernaboutSQLorvaluesbeingpresentintheexception/logs.Iftheloglevelisincreasedtodebugfortherelevantlogger,thenthesanitizeMessagessettingwillhavenoeffect.
org.teiid.ODBCRequireSecure-defaultstotrue.IftruesettingtheSSLconfigtologinorenabledwillrequireclientstoconnectappropriatelywitheitheraGSSloginorSSLrespectively.SettingthepropertytofalsewillallowclienttouseanyauthenticationandnoSSL(whichwasthebehaviorofthepgtransportpriorto8.9CR2).
PostgreSQLCompatibility
Note TheseaffectTeiidglobally,andnotjustthroughtheODBCtransport.
org.teiid.backslashDefaultMatchEscape-defaultstofalse.Settotruetouse'\'asthedefaultescapecharacterforLIKEandSIMILARTOpredicateswhennoescapeisspecified.OtherwiseTeiidassumestheSQLspecificationcompliantbehavioroftreatingeachnon-wildcardcharacterasanexactmatchcharacter.
org.teiid.addPGMetadata-defaultstotrue.Whensettofalse,theVDBwillnotincludePostgresqlbasedsystemmetadata.
org.teiid.honorDeclareFetchTxn-defaultstofalse.Whenfalsethewrappingbegin/commitofaUseDeclareFetchcursorwillbeignoredasTeiiddoesnotrequireatransaction.
Client
SystempropertiescanalsobesetforclientVMs.SeeAdditionalSocketClientSettings.
SystemProperties
96
TeiidManagementCLITheWildFlyCLIisacommandlinebasedadministrativeandmonitoringtoolforTeiid.ManysnippetsofCLIscriptingareshownthroughouttheAdminGuide-especiallyaroundmanagingdatasources.AdminShellprovidesabindingintotheGroovyscriptinglanguageandhigherlevelmethodsthatareoftenneededwheninteractingwithTeiid.ItisstillusefultoknowtheunderlyingCLIcommandsinmanycircumstances.ThebelowisaseriesusefulCLIcommandsforadministeringaTeiidServer.PleasealsorefertotheASdocumentationformoreoninteractingwiththeCLI-includinghowtonavigate,listoperations,etc.
TableofContentsVDBOperationsAuthenticationOperationsSourceOperationsTranslatorOperationsRuntimeOperations
VDBOperations
deployadminapi-test-vdb.xml
undeployadminapi-test-vdb.xml
/subsystem=teiid:restart-vdb(vdb-name=AdminAPITestVDB,vdb-version=1,model-names=TestModel)
/subsystem=teiid:list-vdbs()
/subsystem=teiid:get-vdb(vdb-name=AdminAPITestVDB,vdb-version=1)
/subsystem=teiid:change-vdb-connection-type(vdb-name=AdminAPITestVDB,vdb-version=1,connection-type=ANY)
/subsystem=teiid:add-data-role(vdb-name=AdminAPITestVDB,vdb-version=1,data-role=TestDataRole,mapped-role=tes
t)
/subsystem=teiid:remove-data-role(vdb-name=AdminAPITestVDB,vdb-version=1,data-role=TestDataRole,mapped-role=
test)
AuthenticationOperations
/subsystem=teiid:read-attribute(name=authentication-security-domain)
/subsystem=teiid:write-attribute(name=authentication-security-domain,value=teiid-security)
/subsystem=teiid:read-attribute(name=authentication-max-sessions-allowed)
/subsystem=teiid:write-attribute(name=authentication-max-sessions-allowed,value=1000)
/subsystem=teiid:read-attribute(name=authentication-sessions-expiration-timelimit)
/subsystem=teiid:write-attribute(name=authentication-sessions-expiration-timelimit,value=0)
/subsystem=teiid:read-attribute(name=authentication-type)
/subsystem=teiid:write-attribute(name=authentication-type,value=USERPASSWORD)
/subsystem=teiid:read-attribute(name=authentication-trust-all-local)
/subsystem=teiid:write-attribute(name=authentication-trust-all-local,value=true)
SourceOperations
/subsystem=teiid:add-source(vdb-name=AdminAPITestVDB,vdb-version=1,source-name=text-connector-test,translato
r-name=file,model-name=TestModel,ds-name=java:/test-file)
TeiidManagementCLI
97
/subsystem=teiid:remove-source(vdb-name=AdminAPITestVDB,vdb-version=1,source-name=text-connector-test,model-
name=TestModel)
/subsystem=teiid:update-source(vdb-name=AdminAPITestVDB,vdb-version=1,source-name=text-connector-test,transl
ator-name=file,ds-name=java:/marketdata-file)
TranslatorOperations
/subsystem=teiid:list-translators()
/subsystem=teiid:get-translator(translator-name=file)
/subsystem=teiid:read-translator-properties(translator-name=file,type=OVERRIDE)
/subsystem=teiid:read-rar-description(rar-name=file)
RuntimeOperations
/subsystem=teiid:workerpool-statistics()
/subsystem=teiid:cache-types()
/subsystem=teiid:clear-cache(cache-type=PREPARED_PLAN_CACHE)
/subsystem=teiid:clear-cache(cache-type=QUERY_SERVICE_RESULT_SET_CACHE)
/subsystem=teiid:clear-cache(cache-type=PREPARED_PLAN_CACHE,vdb-name=AdminAPITestVDB,vdb-version=1)
/subsystem=teiid:clear-cache(cache-type=QUERY_SERVICE_RESULT_SET_CACHE,vdb-name=AdminAPITestVDB,vdb-version=1)
/subsystem=teiid:cache-statistics(cache-type=PREPARED_PLAN_CACHE)
/subsystem=teiid:cache-statistics(cache-type=QUERY_SERVICE_RESULT_SET_CACHE)
/subsystem=teiid:engine-statistics()
/subsystem=teiid:list-sessions()
/subsystem=teiid:terminate-session(session=sessionid)
/subsystem=teiid:list-requests()
/subsystem=teiid:cancel-request(session=sessionId,execution-id=1)
/subsystem=teiid:list-requests-per-session(session=sessionId)
/subsystem=teiid:list-transactions()
/subsystem=teiid:mark-datasource-available(ds-name=java:/accounts-ds)
/subsystem=teiid:get-query-plan(session=sessionid,execution-id=1)
TeiidManagementCLI
98
DiagnosingIssuesYoumayexperiencesituationswhereyouincurperformanceissuesorunexpectedresults/exceptions.Therestofthischapterwillfocusonqueryplanningandprocessingissues.Configurationoroperationalissuesrelatedtothecontaineraretypicallymoreisolatedandeasiertoresolve.
GeneralDiagnosticProcess
Whenthereisanissue,startbyisolatingaproblemqueryasmuchaspossible.IfaccessingthroughODBCorOData,doesasimilarissueoccurjustthroughJDBC.Inwhateverscenariotheissueoccurs,theparticularsmatter-whatsources,ifthereisatransaction,load,etc.
Don’tmaketoomanyassumptions
Forexamplememoryconsumptioncanbeheavilydependentupondrivers,andaresultingoutofmemoryissuemayonlybeindirectlyrelatedtoTeiid
Startwiththequeryplan-especiallywithperformanceissues
Theremaybesimplificationsorchangespossibletoviewsandproceduresutilizedbytheuserquery.
Ensurethatrelevantcostingmetadataissetand/orthathintsyouhaveprovidedarebeingappliedasexpected.
UtilizeLogging
Planningissuesmaybeunderstoodwiththedebugplan
Thecommandlog
Afulldebug/tracelevellogcanshedevenmorelight–butitmaynotbeeasytofollow.
Youcancorrelatewhatishappeningbycontext,thread,sessionid,andrequestid.
Ifnoresolutionisfound,engagethecommunityandutilizeprofessionalsupport
QueryPlans
Oncetheproblemhasbeenisolatedasmuchaspossible,youshouldfurtherexaminethequeryplan.Theonlycircumstancewhenthisisnotpossibleiswhenthereareplanningerrors.Inthiscasethelogs,eitherfulldebugorjustthedebugplan,isstillusefultothenloganissuewiththecommunityorwithsupport.
Ifyouhaven’tdonesoalready,youshouldstartbyfamiliarizingyourselfwithFederatedPlanning-especiallythesectionsonthequeryplan.
Thefinalprocessorplanisgenerallywhatismeantwhenreferringtoby“thequeryplan”.TheplancanbeviewedinanXMLoraplaintextformat.
Designeristypicallythefirstplacetolookatqueryplansfromthequeryexecutionview.It’seasiesttolookatusingDesigneroranXMLawareviewerastheplantreescangetlarge.
YoucanalsouseTeiidExtensionsorSQLstatementstogettheplan:
SETSHOWPLANON
SELECT...
SHOWPLAN
DiagnosingIssues
99
Onceyouhavetheplan,youcan:
Doublecheckthathintsaretakingeffect
Makesurethingsseemcorrect
Lookfirstatallofthesourcequeriesontheaccessnodes.Generallyamissingpushdown,suchaspredicateiseasytospot
Focusonproblemsourcequeriesandtheirparentnodesifyoualreadyhaveexecutiontimes
It’salsoagoodideatovalidatequeryplansduringthedevelopmentandtestingofaVDB.Alsoanyengagementwiththecommunityorsupportwilllikelyneedthequeryplanaswell.
Iftheplanisobtainedfromanexecutedquery,thentheplanwillalsoshowexecutionstatistics.Itisespeciallyusefultoseethestatswhenprocessinghasfinishedandallrowshavebeenfetched.Whileseveralstatsarecollected,it’smostusefultosee“NodeOutputRows”and“NodeNextBatchProcessTime”.
Exampletextformofaqueryplanshowingstats:
ProjectNode
+RelationalNodeID:6
+OutputColumns:x(double)
+Statistics:
0:NodeOutputRows:6
1:NodeNextBatchProcessTime:2
2:NodeCumulativeNextBatchProcessTime:2
3:NodeCumulativeProcessTime:2
4:NodeNextBatchCalls:8
5:NodeBlocks:7
+CostEstimates:EstimatedNodeCardinality:-1.0
+Child0:
AccessNode
+RelationalNodeID:7
+OutputColumns
+Statistics:
0:NodeOutputRows:6
1:NodeNextBatchProcessTime:0
2:NodeCumulativeNextBatchProcessTime:0
3:NodeCumulativeProcessTime:0
4:NodeNextBatchCalls:2
5:NodeBlocks:1
...
Inadditiontotheexecutionstats,notetherearealsocostestimates.Thevaluesforthecostestimatesarederivedfromthestatisticvaluessetofeachtable/columnabouttherowcount,numberofdistinctvalues,numberofnullvalues,etc.Unlikesystemsthatownthedata,Teiiddoesnotbuildhistogramsorotherin-depthmodelsofthedata.Thesesvaluesaremeanttobeapproximationswithnominallydistributionassumptions.Thecostinginformationfromthemetadataonlymattersforphysicalentitiesaswe’llrecomputethehighervaluesinplanningaftermergevirtualandotherplanmodifications.Ifyouseethatjoinisbeingimplementedinefficiently,thenfirstmakesurereasonablecostingvaluesarebeingsetonthetablesinvolved.Statisticscanbegatheredforsomesourcesatdesigntimeordeploytime.Inenvironmentsthatfluctuaterapidly,youmayneedtoissueruntimecostingupdatesviasystemprocedures.
Note:ifyoucardinalityvaluesareunknown-shownas'NodeCardinality:-1.0'intheplan-andnohintsareused,thentheoptimizerwillnotassumethatdependentjoinplansshouldbeused.
PushdownInhibited
Oneofthemostcommonissuesthatcausesperformanceproblemsiswhennotenoughoftheplanispushedtoagivensourceleadingtotoomanyrowsbeingfetchedand/ortoomuchprocessinginTeiid.
DiagnosingIssues
100
Pushdownproblemsfallintotwocategories:
Somethingthatcannotbepusheddown.Forexamplenotallsystemfunctionsaresupportedbyeachsource.Formattingfunctionsinparticulararenotbroadlysupported.
Aplanningorotherissuethatpreventsotherconstructsfrombeingpusheddown
Temptablesormaterializationcaninhibitpushdownwhenjoining
Windowfunctionsandaggregationwhennotpushedcanpreventfurtherpushdown
Ifpushdownisinhibitedthentheconstructwillbemissingfromtheaccessnodeissuingthesourcequery,andwillinsteadbebeatahighernode:
<nodename="SelectNode">...<propertyname="Criteria"><value>pm1.g1.e2=1</value>
<nodename="AccessNode">...<propertyname="Query"><value>SELECTpm1.g1.e1,pm1.g1.e2FROMpm1.g1</value>
Whenpushdownisinhibitedbythesource,itshouldbeeasytospotinthedebugplanwithloglinesimilarto:
LOWRelationalPlannerSubqueryInisnotsupportedbysourcepm1-e1IN/*+NO_UNNEST*/(SELECTe1FROMpm2.g
1)wasnotpushed
CommonIssues
Beyondpushdownbeinginhibited,othercommonissuesare:
SlightdifferencesinTeiid/Pushdownresults
forexampleTeiidproducesadifferentforagivenfunctionthanthesource
Sourcequeryformisnotoptimalorincorrect
Thereisanunaccountedfortypeconversion
forexamplethereisnochar(n)typeinTeiid
Acastmaycauseasourceindexnottobeused
JoinPerformance
Costingvaluesnotsetleadingtoanon-performantplan.
Usehintsifneeded.
Teiidwillreplaceouterwithinnerjoinswhenpossible,butjustincasereviewouterjoinusageintheuserqueryandviewlayers
XQuery
XQuery/XPathcanbedifficulttogetcorrectwhennotassistedbytooling.Havinganincorrectnamespaceforexamplecouldsimplyresultinnoresultsratherthanexception.
WithXMLQUERY/XMLTABLEeachXPath/XQueryexpressioncanhavealargeimpactonperformance.Inparticulardescendantaccess'//'canbecostly.Justaccessingelementsinthedirectparentageisefficientthough.
Thelargerthedocumentbeingprocessed,themorecarefulyouneedtobetoensurethatdocumentprojectionandstreamprocessingcanbeused.Streamingtypicallyrequiresasimplecontextpath-'a/b/c'
OutofMemory
DiagnosingIssues
101
Outofmemoryerrorscanbedifficulttotrackdown.Inalmostallcases,itisbesttodeterminetheactualmemoryconsumptionutilizingaheapdump-whichcanbeobtainedusingthevmHeapDumpOnOutOfMemoryErroroptionorviaatoolsuchasVisualVM.Youmayalsosimplyincreasethesizeoftheheap,butthatmaysimplydelaytheissuefromreappearing.
Logging
Thequeryplanalonedoesnotprovideafullaccountingofprocessing.Somedecisionsaredelayeduntilexecutionorcanonlybeseenintheserverlogs:
TheENAHANCEDSORTJOINnodemayexecutecanexecuteoneofthreedifferentjoinstrategiesdependingontheactuallyrowcountsfound,thiswillnotbeseenunlessthequeryplanisobtainedattheendofexecution.
Theeffectoftranslationisnotyetaccountedforastheplanshowstheengineformofthequery
Thefulltranslationcanbeseeninwithcommandloggingatatracelevelorwithdebug/traceloggingingeneral.
Thequeryplandoesn’tshowtheexecutionstatsofindividualthesourcequeries,whichisshowninthecommandlog
Theforfullpictureofexecutiondowntoallthebatchfetches,you’lljustneedthefullserverdebug/tracelog
PlanDebugLog
Thelogicalplanoptimizationisrepresentedbytheplanningdebuglogandismoreusefultounderstandwhyplanningdecisionsweremade.
SETSHOWPLANDEBUG
SELECT...
SHOWPLAN
Youwilltypicallynotneedtousethislevelofdetailtodiagnoseissues,butitisusefultoprovidetheplandebuglogtosupportwhenplanningissuesoccur.
DiagnosingIssues
102
MigrationGuideFromTeiid8.xTeiidstrivestomaintainconsistencybetweenallversions,butwhennecessarybreakingconfigurationandVDB/sqlchangesaremade-andthentypicallyonlyformajorreleases.
Youshouldconsultthereleasenotesforcompatibilityandconfigurationchangesfromeachminorversionthatyouareupgradingover.Thisguideexpandsuponthereleasenotesincludedinthekittocoverchangessince8.x.
IfpossibleyoushouldmakeyourmigrationtoTeiid9byfirstusingTeiid8.13.8.13isanon-featuretransitionalreleasethatiseffectivelyTeiid8.12runningonWildFly9.0.2.Seealso8.13MigrationGuide
JRESupportTeiid9.1usesWildFly10.0.0.BoththeserverkitandusageofTeiidEmbeddedwillrequireJava1.8+.Theclientdrivermaystillusea1.6runtime.
Teiid9.0usesWildFly9.0.2.BoththeserverkitandusageofTeiidEmbeddedwillrequireJava1.7+.Theclientdrivermaystillusea1.6runtime.
ConfigurationChanges
YouwillneedtoapplyyourTeiidandotherconfigurationchangesstartingwithanewbaseconfigurationforWildFly,suchasthestandalone-teiid.xmlincludedinthekit.Notethat9999porthasbeenremovedbydefault.Adminconnectionsareexpectedtouseeither9990(http)or9993(https).
SecurityRelated
Thereisnowasinglesessionservice.Sessionservicerelatedproperties,prefixedbyauthentication,arenolongerspecifiedpertransport.Insteadtheynowappearasasinglesiblingtothetransports.
Oldstandalone.xmlConfiguration
<transportname="local"/>
<transportname="odata">
<authenticationsecurity-domain="teiid-security"/>
</transport>
<transportname="jdbc"protocol="teiid"socket-binding="teiid-jdbc">
<authenticationsecurity-domain="teiid-security"/>
</transport>
<transportname="odbc"protocol="pg"socket-binding="teiid-odbc">
<authenticationsecurity-domain="teiid-security"/>
<sslmode="disabled"/>
</transport>
Newstandalone.xmlConfiguration
<authenticationsecurity-domain="teiid-security"/>
<transportname="local"/>
<transportname="odata"/>
<transportname="jdbc"protocol="teiid"socket-binding="teiid-jdbc"/>
<transportname="odbc"protocol="pg"socket-binding="teiid-odbc">
<sslmode="disabled"/>
</transport>
MigrationGuideFromTeiid8.x
103
Thedefaultmaximumnumberofsessionswasincreasedto10000toaccommodateforthischange.
Inadditionthereisanewpropertytrust-all-localthatdefaultstotrueandallowsunauthenticatedaccessbylocalpass-throughconnectionsovertheembeddedtransport-thiswaseffectivelythedefaultbehaviorof8.xandbeforewhennosecurity-domainwassetontheembeddedtransport.Youmaychoosetodisallowthattypeofaccessbysettingthepropertytofalseinstead.
Theauthentication-security-domainpropertywillonlyacceptasinglesecuritydomain,andwillnotinterpretthevalueasacommaseparatedlist.Thedefaultbehaviorhasalsochangedforusernames-theyarelongerallowedtobequalifiedbythesecuritydomain.Usetheauthentication-allow-security-domain-qualifierpropertytoallowtheoldbehaviorofacceptingusernamesthataresecuritydomainqualified.
RoleBasedCredentialMapIdentityLoginModule
TheRoleBasedCredentialMapIdentityLoginModuleclasshasbeenremoved.ConsideralternativeloginmoduleswithrolesassignmentstorestrictaccesstotheVDB.
LocalTransport
TheembeddedtransportwasrenamedtolocaltoavoidconfusionwithTeiidembedded.
Behavioral
widenComparisonToString
Theresolver’sdefaultbehaviorwastowidencomparisonstostring,but9.0nowdefaultsorg.teiid.widenComparisonToStringtofalse.Forexamplewiththissettingasfalseacomparisonsuchas"timestamp_col<'a'"willproduceanexceptionwhereaswhensettotrueitwouldeffectivelyevaluate"cast(timestamp_colasstring)<'a'".Ifyouexperienceresolvingerrorswhenavdbisdeployedyoushouldupdatethevdbifpossiblebeforerevertingtotheoldresolvingbehavior.
reportAsViews
TheJDBCclientwillreportTeiidviewsinthemetadataastabletypeVIEWratherthanTABLEbydefault.UsetheconnectionpropertyreportAsViews=falsetousepre-9.0behavior.
DefaultPrecision/Scale
Ifacolumnisspecifiedwithaprecisionof0orleftasthedefaultinDDLmetadataitwillbetreatedashavingthenominalinternalmaximumvalueof32767.Thismaycausetheprecisionandscaletobereporteddifferently,whichmayhavebeen2147483647insomeplacesor20inJDBCDatabaseMetaData.
CompatibilityChanges
DDLDelimiters
Notusingasemicolondelimiterbetweenstatementsisdeprecatedandshouldonlybereliedonforbackwardscompatibility.
SystemMetadata
Withdatarolesenabledsystemtables(SYS,SYSADMIN,andpg_catalog)willonlyexposetables,columns,procedures,etc.forwhichtheuserisentitledtoaccess.AREADpermissionisexpectedfortables/columns,whileanEXECUTEpermissionisexpectedforfunctions/procedures.Allnon-hiddenschemaswillstillbevisiblethough.
MigrationGuideFromTeiid8.x
104
TheOIDcolumnshasbeenremoved.TheUIDcolumnshouldbeusedinsteadorthecorrespondingpg_catalogtablewillcontainanOIDvalues.
ParentuidcolumnshavebeenaddedtotheSYSTables,Procedures,KeyColumns,andColumnstables.
XMLDocumentModel
TheXMLDocumentModelhasbeendeprecated.PleaseconsidermigratingtoODataorutilizingSQL/XMLfunctionsforconstructingdocuments.
Kitting/BuildChanges
AdminJAR
For8.13theentrypointforcreatingremoteadminconnection,AdminFactory,wasmovedintotheteiid-jboss-adminjarratherthanbeinglocatedinteiid-admin.
APIChanges
TheAuthorizationValidatorandPolicyDeciderinterfaceshadminorchanges.AuthorizationValidatorhasanadditionalmethodtodeterminemetadatafiltering,andPolicyDeciderhadisTempAccessablecorrectedtoisTempAccessible.
SemanticversioningrequiredthechangeoftheVDBversionfieldfromanintegertoastring.Thisaffectedthefollowingpublicclasses:
VDBSessionEventListenerVDBImportExecutionContextMetadataRepository
Therearealsoduplicate/deprecatedmethodson:
EventDistributorAdmin
UsingtheTranslatorPropertyannotationwithoutasetternowrequiresthatreadOnly=truebesetontheannotation.
TheJDBCDatabaseMetaDataandCommandContextgetUserNamemethodswillnowreturnjustthebaseusernamewithoutthesecuritydomain.
EmbeddedKit
TheEmbeddedKithasbeenremoved.YoushouldfollowtheEmbeddedExamplestousemaventopullthedependenciesyouneedforyourproject.
Therewereextensivechangesindependencymanagementforhowtheprojectisbuilt.Thesechangesallowedustoremovetheneedforresourceadapterjarsbuiltwiththelibclassifier.Ifyouneedtoreferencetheseartifactsfrommaven,justomittheclassifier.
LegacyDrivers
ThedriversforJRE1.4/1.5systemshavebeendiscontinued.Ifyoustillneedaclientforthoseplatforms,youshouldusetheappropriate8.xdriver.
OData
TheODatav2warbaseduponodata4jhasbeenremoved.YoushouldutilizetheODatav4warserviceinstead.
MigrationGuideFromTeiid8.x
105
Thenamesofthewarshavebeenchangedtostripversioninformation-thismakesiteasiertocaptureadeployment-overlayintheconfigurationsuchthatitwon’tbechangedfromoneTeiidversiontothenext.
teiid-odata-odata2.warhasbecometeiid-odata.warteiid-olingo-odata4.warhasbecometeiid-olingo-odata4.war
Tochangepropertiesinanweb.xmlfileoraddotherfilestothedefaultodatawar,youshoulduseadeploymentoverlayinstead.
Materialization
Thesemanticversioningchangerequiresthematerializationstatustablestochangetheirversioncolumnfromanintegertostring.Boththesourceandthesourcemodelwillneedtobeupdatedwiththecolumntypechange.
MigrationGuideFromTeiid8.x
106
CachingGuideTeiidprovidesseveralcapabilitiesforcachingdataincluding:
1. Materializedviews
2. ResultSetcaching
3. Codetablecaching
Thesetechniquescanbeusedtosignificantlyimproveperformanceinmanysituations.
Withtheexceptionofexternalmaterializedviews,thecacheddataisaccessedthroughtheBufferManager.ForbetterperformancetheBufferManagersettingshouldbeadjustedtothememoryconstraintsofyourinstallation.SeetheCacheTuningformoreonparametertuning.
CachingGuide
107
ResultsCachingTeiidprovidesthecapabilitytocachetheresultsofspecificuserqueriesandvirtualprocedurecalls.Thiscachingtechniquecanyieldsignificantperformancegainsifusersofthesystemsubmitthesamequeriesorexecutethesameproceduresoften.
SupportSummary
CachingofuserqueryresultsincludingXMLdocumentmodelresults.
Cachingofvirtualprocedureresults.
ScopingofresultsisautomaticallydeterminedtobeVDB/user(replicated)orsessionlevel.Thedefaultlogicwillbeinfluencedbyeveryfunctionevaluated,considertheDETERMINISMpropertyonallsourcemodels/tables/procedures,andtheScopefromtheExecutionContextorCacheDirective.
Configurablenumberofcacheentriesandtimetolive.
Administrativeclearing.
UserInteraction
UserQueryCache
UserqueryresultsetcachingwillcacheresultsetsbasedonanexactmatchoftheincomingSQLstringandPreparedStatementparametervaluesifpresent.CachingonlyappliestoSELECT,setquery,andstoredprocedureexecutionstatements;itdoesnotapplytoSELECTINTOstatements,orINSERT,UPDATE,orDELETEstatements.
Endusersorclientapplicationsexplicitlystatewhethertouseresultsetcaching.ThiscanbedonebysettingtheJDBCResultSetCacheModeexecutionpropertytotrue(defaultfalse)
Propertiesinfo=newProperties();
…
info.setProperty("ResultSetCacheMode","true");
Connectionconn=DriverManager.getConnection(url,info);
orbyaddingaCacheHinttothequery.Notethatifeitherofthesemechanismsareused,Teiidmustalsohaveresultsetcachingenabled(thedefaultisenabled).
Themostbasicformofthecachehint,/*+cache*/,issufficienttoinformtheenginethattheresultsofthenon-updatecommandshouldbecached.
PreparedStatementResultSetCaching
PreparedStatementps=connection.prepareStatement("/*+cache*/selectcolfromtwherecol2=?");
ps.setInt(1,5);
ps.execute();
TheresultswillbecachedwiththedefaultttlandusetheSQLstringandtheparametervalueaspartofthecachekey.
Thepref_memandttloptionsofthecachehintmayalsobeusedforresultsetcachequeries.Ifacachehintisnotspecified,thenthedefaulttimetoliveoftheresultsetcachingconfigurationwillbeused.
AdvancedResultSetCaching
ResultsCaching
108
/*+cache(pref_memttl:60000)*/selectcolfromt
Inthisexamplethememorypreferencehasbeenenabledandthetimetoliveissetto60000millisecondsor1minute.Thettlforanentryisactuallytreatedasit’smaximumageandtheentrymaybepurgedsoonerifthemaximumnumberofcacheentrieshasbeenreached.
Note Eachqueryisre-checkedforauthorizationusingthecurrentuser’spermissions,regardlessofwhetherornottheresultshavebeencached.
ProcedureResultCache
Similartomaterializedviews,cachedvirtualprocedureresultsareusedautomaticallywhenamatchingsetofparametervaluesisdetectedforthesameprocedureexecution.UsageofthecachedresultsmaybebypassedwhenusedwiththeOPTIONNOCACHEclause.UsageiscoveredinHintsandOptions.
CachedVirtualProcedureDefinitionToindicatethatavirtualprocedureshouldbecached,it’sdefinitionshouldincludeaCacheHint.
ProcedureCaching
/*+cache*/
BEGIN
...
END
Resultswillbecachedwiththedefaultttl.
Thepref_memandttloptionsofthecachehintmayalsobeusedforprocedurecaching.
Procedureresultscachekeysincludetheinputparametervalues.Topreventoneprocedurefromfillingthecache,atmost256cachekeysmaybecreatedperprocedureperVDB.
Acachedprocedurewillalwaysproduceallofitsresultspriortoallowingthoseresultstobeconsumedandplacedinthecache.Thisdiffersfromnormalprocedureexecutionwhichinsomesituationsallowsthereturnedresultstobeconsumedinastreamingmanner.
CacheConfigurationBydefaultresultsetcachingisenabledwith1024maximumentrieswithamaximumentryageof2hours.Thereareactually2cachesconfiguredwiththesesettings.OnecacheholdsresultsthatarespecifictosessionsandislocaltoeachTeiidinstance.TheothercacheholdsVDBscopedresultsandcanbereplicated.Seetheteiidsubsystemconfigurationfortuning.TheusermayalsooverridethedefaultmaximumentryageviatheCacheHint.
Resultsetcachingisnotlimitedtomemory.Thereisnoexplicitlimitonthesizeoftheresultsthatcanbecached.CachedresultsareprimarilystoredintheBufferManagerandaresubjecttoit’sconfiguration-includingtherestrictionofmaximumbufferspace.
Whiletheresultdataisnotheldinmemory,cachekeys-includingparametervalues-maybeheldinmemory.Thusthecacheshouldnotbegivenanunlimitedmaximumsize.
Resultsetcacheentriescanbeinvalidatedbydatachangeevents.Themax-stalenesssettingdetermineshowlonganentrywillremaininthecaseafteroneofthetablesthatcontributedtotheresultshasbeenchanged.SeetheDeveloper’sGuideforfurthercustomization.
ResultsCaching
109
ExtensionMetadata
Youcanusetheextensionmetadataproperty
{http://www.teiid.org/ext/relational/2012}data-ttl
asamodelpropertyoronasourcetabletoindicateadefaultTTL.AnegativevaluemeansnoTTL,0meansdonotcache,andapositivenumberindicatesthetimetoliveinmilliseconds.IfnoTTLisspecifiedonthetable,thentheschemawillbechecked.TheTTLforthecacheentrywillbetakenastheleastpositivevalueamongallTTLs.Thussettingthisvalueasamodelpropertycanquicklydisableanycachingagainstaparticularsource.
Forexample,settingthepropertyinthevdb.xml:
<vdbname="vdbname"version="1">
<modelname="Customers">
<propertyname="teiid_rel:data-ttl"value="0"/>
...
CacheAdministration
TheresultsetcachecanbeclearedthroughtheAdminAPIusingtheclearCachemethod.Theexpectedcachekeyis"QUERY_SERVICE_RESULT_SET_CACHE".
ClearingtheResultSetCacheinAdminShell
connectAsAdmin()
clearCache("QUERY_SERVICE_RESULT_SET_CACHE")
SeetheAdministrator’sGuideformoreonusingtheAdminAPIandAdminShell.
Limitations
XML,BLOB,CLOB,andOBJECTtypecannotbeusedaspartofthecachekeyforpreparedstatementofprocedurecachekeys.
TheexactSQLstring,includingthecachehintifpresent,mustmatchthecachedentryfortheresultstobereused.Thisallowscacheusagetoskipparsingandresolvingforfasterresponses.
ResultsetcachingistransactionalbydefaultusingtheNON_XAtransactionmode.IfyouwantfullXAsupport,thenchangetheconfigurationtouseNON_DURABLE_XA.
ClearingtheresultscacheclearsallcacheentriesforallVDBs.
ResultsCaching
110
MaterializedViewsTeiidsupportsmaterializedviews.Materializedviewsarejustlikeotherviews,buttheirtransformationsarepre-computedandstoredjustlikearegulartable.WhenqueriesareissuedagainsttheviewsthroughtheTeiidServer,thecachedresultsareused.Thissavesthecostofaccessingalltheunderlyingdatasourcesandre-computingtheviewtransformationseachtimeaqueryisexecuted.
Materializedviewsareappropriatewhentheunderlyingdatadoesnotchangerapidly,orwhenitisacceptabletoretrievedatathatis"stale"withinsomeperiodoftime,orwhenitispreferredforend-userqueriestoaccessstageddataratherthanplacingadditionalqueryloadonoperationalsources.
SupportSummary
Cachingofrelationaltableorviewrecords(pre-computingalltransformations)
Model-baseddefinitionofvirtualgroupstocache
UserabilitytooverrideuseofmaterializedviewcacheforspecificqueriesthroughHintsandOptions
Approach
Theoverallstrategytowardmaterializationshouldbetoworkontheintegrationmodelfirst,thenoptimizeasneededfromthetopdown.
Resultsetcaching,ideallyhintdriven,shouldbeusediftherelotsofrepeateduserqueries.Ifresultsetcachingisinsufficient,thenmoveontointernalmaterializationforviewsthatareclosesttoconsumers(minimallyornotlayered)thatareintroducingperformanceissues.Keepinmindthattheuseofmaterializationinlinesaccesstothematerializationtableratherthantheviewsoscenariosthatintegrateontopofthematerializationmaysufferiftheywererelyingonpushing/optimizingtheworkoftheviewwithsurroundingconstructs.
Baseduponthelimitationsofinternalmaterialization,thenswitchtoexternalmaterializationasneeded.
MaterializedViewDefinition
Materializedviewsaredefinedinbysettingthematerializedpropertyonatableorviewinavirtual(view)relationalmodel.Settingthisproperty’svaluetotrue(thedefaultisfalse)allowsthedatageneratedforthisvirtualtabletobetreatedasamaterializedview.
Important Itisimportanttoensurethatallkey/indexinformationispresentasthesewillbeusedbythematerializationprocesstoenhancetheperformanceofthematerializedtable.
Thetargetmaterializedtablemayalsobesetintheproperties.Ifthevalueisleftblank,thedefault,theninternalmaterializationwillbeused.Otherwiseforexternalmaterialization,thevalueshouldreferencethefullyqualifiednameofatable(orpossiblyview)withthesamecolumnsasthematerializedview.Formostbasicscenariosthesimplicityofinternalmaterializationmakesitthemoreappealingoption.
Reasonstouseexternalmaterialization
Thecacheddataneedstobefullydurable.Internalmaterializationdoesnotsurviveaclusterrestart.
MaterializedViews
111
Fullcontrolisneededofloadingandrefresh.Internalmaterializationdoesofferseveralsystemsupportedmethodsforrefreshing,butdoesnotgivefullaccesstothematerializedtable.
Controlisneededoverthematerializedtabledefinition.InternalmaterializationdoessupportIndexes,buttheycannotbedirectlycontrolled.Constraintsorotherdatabasefeaturescannotbeaddedtointernalmaterializationtables.
Thedatavolumeislarge.Internalmaterialization(andtemptablesingeneral)havememoryoverheadforeachpage.Aroughguidelineisthattherecanbe100millionrowsinallmaterializedtablesacrossallVDBsforeverygigabyteofheap.
Important
MaterializedviewtablesdefaulttotheVDBscope.Bydefaultifamaterializedviewdefinitiondirectlyortransitivelycontainsanon-deterministicfunctioncall,suchasrandomorhasRole,theresultingtablewillcontainonlytheinitiallyevaluatedvalues.Inmostinstancesyoushouldconsidernestingamaterializedviewwithoutthedeterministicresultsthatisjoinedwithrelevantnon-deterministicvaluesinaparentview.Youmayalsoscopethematerializedviewtobesessionspecific,butthatmaylimitthereuseoftheresultsinmanysituations.
MaterializedViews
112
ExternalMaterializationThisdocumentwillexplainwhatTeiidExternalMaterializationisandhowtouseit.
TableofContentsWhatisitExternalMaterializedDataSourceSystems
RDBMSSystemsJBossDataGrid(JDG)
ViewOptionsMaterializationManagement
1.CreationofStatusTable2.CreationofViewandMaterializedTableMaterializationTableLoadingEagerMaterializationTableUpdate
UsingJDGasMaterializationStoreDefineMaterializedViewinDesignerMaterializationwithEmbeddedTeiidAppendix-1:DDLforcreatingMatViewStatusTableAppendix-2:ExampleVDBwithExternalMaterializedViewOptions
Whatisit
InTeiid,aviewisavirtualtablebasedonthecomputing(loading/transforming/federating)ofacomplexSQLstatementacrossheterogeneousdatasources.TeiidexternalmaterializationprocesscancachetheViewdatatoanexternaldatasourcesystemsonaperiodicbasis.WhenauserissuesqueriesagainstthisView,therequestwillberedirectedtothisexternaldatasourcesystemwherecachedresultswillbereturned,ratherthanre-computingresultsfromsourcesystems.MaterializationcanprovetobetimeandresourcesavingifyourViewtransformationiscomplexand/oraccesstothesourcesystemsisconstrained.
ExternalMaterialization
113
MaterializedView-Materializedviewisjustlikeotherviews,withadditionaloptionsinViewOptions,toenablepre-computingandcachingdatatoanexternaldatasourcesystem.
MaterializedTable-MaterializedtablerepresentsthetargettableforthematerializedView,hasthesamestructureasthematerializedview,butexistsontheexternaldatasourcesystem.
MatViewStatusTable-Eachmaterializedviewhasareferenceto'Status'table,thisusedtosavetheMaterializedviews'refreshstatus.ThistabletypicallyexistsonthesamephysicalsourcewiththeMaterializedTable.
Anexternalmaterializedviewgivestheadministratorfullcontrolovertheloadingandrefreshstrategies.RefertoMaterializationManagementfordetails.
ExternalMaterializedDataSourceSystemsThefollowingarethetypesofdatasourcesthathavebeentestedtoworkintheexternalmaterializationprocess:
RDBMSSystems
RDBMS-arelationaldatabaseshouldwork.Exampledatabases;Oracle,Postgresql,MySQL,MSSqlServer,SAPHana,etc.
Ifthedatabasesupportsatransactionalrenameoperation,youcanusethedefaultloadstrategythatusesastagingtableandrelyonrenamingthestagingtabletothelivetableintheafterloadscript.
NoteTEIID-4294raisesthatnoteverydatabasesupportsatransactionalrename,eitherasseparateorablockofstatements.IfthisisthecaseyoushouldconsiderusingaLOADNUMBERcolumn,oracustomloadstrategythatmaintainsonlyasingletable.
JBossDataGrid(JDG)
JBossDataGrid(JDG)-forin-memorycachingofresults.
ExternalMaterialization
114
ViewOptions
ThefollowingViewpropertiesareextensionpropertiesthatusedinthemanagementoftheMaterializedView.
PropertyName Description Optional Default
MATERIALIZED Setthevalueto'TRUE'fortheViewtobematerialized. false n/a
MATERIALIZED_TABLE
Definesthenameoftargettable,thisalsohintsthematerializationisusingexternalmaterialization.OmittingthispropertyandsettingtheMATERIALIZEDproeprtytrue,invokesinternalmaterialization.
false n/a
UPDATABLE AllowupdatingMaterializedViewviaDMLUPDATE true false
teiid_rel:ALLOW_MATVIEW_MANAGEMENTAllowTeiidbasedautomaticmanagementofload/refreshstrategiesofView.
true false
teiid_rel:MATVIEW_STATUS_TABLE
FullyqualifiedStatusTableNametomanagetheload/refreshoftheView.SeebelowfortablestructureandDDLforit.
false n/a
teiid_rel:MATVIEW_LOADNUMBER_COLUMN
NameofcolumnintheMATERIALIZED_TABLEthatcanholdstatusinformationaboutload/refreshloadprocess.ThecolumntypeMUSTbelong,andtypicallynamedas"LoadNumber".
false NONE
teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT DDL/DMLcommandtorunbeforetheactualloadofthecache true
Whennotdefined,noscriptwillberun
teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT
DDL/DMLcommandtorunaftertheactualloadofthecache.teiid_rel:MATVIEW_STAGE_TABLEtoMATVIEWtable
true
Whennotdefined,noscriptwillberun
teiid_rel:MATVIEW_SHARE_SCOPE
Allowedvaluesare{IMPORTED,FULL},whichdefineifthecachedcontentsaresharedamongdifferentVDBversionsanddifferentimportedVDBsandparentVDB.
true IMPORTED
teiid_rel:ON_VDB_START_SCRIPT DDL/DMLcommandtorunstartofvdb true n/a
teiid_rel:ON_VDB_DROP_SCRIPT
DDL/DMLcommandtorunatVDBun-deploy;typicallyusedforcleaningthecache/statustables.DONOTusethisscripttodeletethecontentsofStatustable,whencachescopesettingsareconfiguredfor{FULL}scope,if true n/a
ExternalMaterialization
115
anotherversionoftheVDBisstillactive.Deletionofthisinformationwillreloadthematerializationtable.
teiid_rel:MATVIEW_ONERROR_ACTION
Actiontobetakenwhenmatviewcontentsarerequestedbutcacheisinvalid.Allowedvaluesare(THROW_EXCEPTION=throwsanexception,IGNORE=ignoresthewarningandsuppliedinvalidateddata,WAIT=waitsuntilthedataisrefreshedandvalidthenprovidestheupdateddata)
true WAIT
teiid_rel:MATVIEW_TTL
timetoliveinmilliseconds.Providepropertyorcachehintonviewtransformation-propertytakesprecedence.
true
2^63milliseconds-effectivelythetablewillnotrefresh,butwillbeloadedasingletimeinitially
teiid_rel:MATVIEW_WRITE_THROUGH
WhentrueTeiidwillperformboththeunderlyingupdateandthecorrespondingupdateagainstthematerializationtargetforaninsert/update/deleteissuedagainsttheview.
true false
Tip forscriptsthatneedmorethanonestatementexecuted,useaprocedureblockBEGINstatement;statement;…END
Important
Whenavdbisimportedintoanothervdb,materializiedviewsareautomaticallysharedacrossthesevdbs.Theteiid_rel:MATVIEW_SHARE_SCOPEpropertymustbesetto'IMPORTED'or'FULL'onimportingVDB’smaterializedviewstoenablesharingacrossthebothvdbs.Thebelowtableshowsanexampleofhowthispropertyworks
Forexample:TableAisinVDBX.1andTableCinVDBY.1TableA&BinVDBX.2andimportsY.1thendependingonscopesettingthesystemwillcachesharingwillworkas
Scope X.1 Y.1 X.2
IMPORTED A-owncopy C-Sharedw/X.2 A-owncopy,B-owncopy,C-SharedfromY.1
FULL A-Sharedwith/X.* C-Sharedw/X.2A-Sharedwith/X,B-Sharedw/X,C-Sharedfrom/Y.1
MaterializationManagement
WhendesigningViews,youcandefineadditionalmetadataandextensionproperties(refertoabovesection)ontheviewstocontroltheloadingandrefreshingofexternalmaterializationcache.Thisoptionprovidesalimited,butapowerfulwaytomanagethematerializationviews.BelowwewillliststepsneedtotaketoconfigureaViewtobematerialized.
ExternalMaterialization
116
1.CreationofStatusTable
Tomanageandreporttheloadingandrefreshingactivityofmaterializationoftheview,aMaterializedTableandStatusTableneedbebedefinedinoneofthesourcemodelsintheVDB.Createthesetablesonthephysicaldatabase,beforeyoudeploytheVDB.
ThebelowdefinestheDDLforcreatingtheStatustable.
CREATETABLEstatus
(
VDBNamevarchar(50)notnull,
VDBVersionvarchar(50)notnull,
SchemaNamevarchar(50)notnull,
Namevarchar(256)notnull,
TargetSchemaNamevarchar(50),
TargetNamevarchar(256)notnull,
Validbooleannotnull,
LoadStatevarchar(25)notnull,
Cardinalitylong,
Updatedtimestampnotnull,
LoadNumberlongnotnull,
PRIMARYKEY(VDBName,VDBVersion,SchemaName,Name)
);
Appendix-1:DDLforcreatingMatViewStatusTablecontainsaseriesofverifiedschemasagainstdifferentRDBMSsources.Thesecanbemodifiedtosuityourdatabase,pleasemakesurethenamesanddatatypesmatchexactly.
Warning
Somedatabases,suchasMySQLwiththeInnoDBbackend,maynotallowalargeprimarykeysuchastheoneforthestatustable.Ifyouexperiencethis,youshouldconsidermakingthefieldsizesshorter(suchasthetablename),usingadifferentdatabasetoholdthestatus,orusingasmallerindex(forexamplejustovervdbnameandvdbversion).
2.CreationofViewandMaterializedTable
DefinetheViewanditstransformationeitherusingtheDesignerordirectlyinDDLinaVDB’smodel.ThenprovidetheextensionpropertiesontheViewasdefinedinViewOptions
SettheMATERIALIZEDto'TRUE'andtheMATERIALIZED_TABLEpointtoatargettableisnecessaryforexternalmaterialization,UPDATABLEisoptional,setitto'TRUE'ifwanttheexternalmaterializedviewbeupdatable,thismustbesettotrue,ifyouwanttoissueincrementaleagerupdatestotheview.DefinetheTTLtodefinetheload/refreshsemantics.
InananotherPHYSICALmodelintheVDB(wheretheStatustabledefined),definetheMaterializedtable,wheretheMaterializedTableshouldhavethesamestructureasViewitisrepresenting,withadditional"LoadNumber"columnwith"long"datatype.
OnceaView,whichisdefinedwiththeaboveproperties,isdeployed,thefollowingsequenceofeventswilltakeplace:
Tip ExampleVDBbasedonDDLisdefinedbelowforreference.
MaterializationTableLoading
UpondeploymentoftheVDBtotheTeiidserver,SYSADMIN.loadMatViewusedtoperformacompleterefreshofmaterializedtable,thisprocedurereadstheextensionpropertiesdefinedfromViewOptionstocustomizetheload.Thefollowingdescribesthesequenceofeventsthatoccurinsidethisprocedure
1. Inserts/updatesanentryinteiid_rel:MATVIEW_STATUS_TABLE,whichindicatesthatthecacheisbeingloaded.
2. Executesteiid_rel:MATVIEW_BEFORE_LOAD_SCRIPTifdefined.
ExternalMaterialization
117
3. Runsaquerytoloadthecachecontents.ThismakesuseofView’stransformationtoloadthecontents.
4. Executesteiid_rel:MATVIEW_AFTER_LOAD_SCRIPTifdefined.
5. Updatesteiid_rel:MATVIEW_STATUS_TABLEentrytosetmaterializedviewstatusstatusto"LOADED"andvalid.Iffailurehappensitwillbemarkedassuch.
Tip Thestart/stopscriptsarenotclusteraware-thatistheywillrunoneachclustermemberastheVDBisdeployed.Whendeployingintoaclusteredenvironment,thescriptsshouldbewritteninsuchawayastobeclustersafe.
EagerMaterializationTableUpdate
SYSADMIN.updateMatViewusedtoperformaeagerincrementalupdatebasedoncriteriaprovided.Ifyouknowthatcertaindatapointsinthesourcesystemwerechangedafterlastfullrefreshofthematerializedview,youcancallthisprocedurewithacriteriabasedontheviewthatcoverthosechangedvalues,andthisprocedurewillupdateonlythoseaffectedrowsinthematerializedtable.
Note:ThisscriptisnotinvokedautomaticallybyTeiid,asthesourceupdateeventsmaybeoccurringoutsideofTeiid.Thisprocedureneedstobeinvokedbyuser,whenhe/sheknowsthatthereischangeinthesourcesystems.
UsingJDGasMaterializationStoreTheJDGcanbeusedasamaterializationtargettosavecachecontentsoftheViewinaJDGserver.JDGcanbeclusterawareandisaccessedusingtheJDGHotRodClient.SeetheJDGHotRodDataSourceforhowtoconfigureaccessingtheremotecache.
Toconfigureforexternalmaterialization,seetheHotRodTranslator.ThisexplainshowtoconfigureMaterializationManagementthatisspecificforusingJDGremotecacheandisessentialformanagingtheunderlyingmultiplecachesneededinordertoperformmaterialization.
OncetheMaterializedViewiscompletelydefined,deployittoaTeiidServer.ThencreateanewsessionandissueaqueryagainstMaterializedView.Youwillfindit’stime-savingandcpu-savingifyourquerysentencesarecomplexandacrossmultiple,heterogeneousdatastores.
DefineMaterializedViewinDesigner
CreatematerializedviewsandcorrespondingphysicalmaterializedtargettablesinDesigner.Thiscanbedonethroughsettingthematerializedandtargettablemanually,orbyselectingthedesiredviews,rightclicking,thenselectingModeling→"CreateMaterializedViews"
GeneratetheDDLforyourphysicalmodelmaterializationtargettables.Thiscanbedonebyselectingthemodel,rightclicking,thenchoosingExport→"MetadataModeling"→"DataDefinitionLanguage(DDL)File".Thisscriptcanbeusedtocreatethedesiredschemaforyourmaterializationtargetonwhateversourceyouchoose.
MaterializationwithEmbeddedTeiidThedefinitionoftheviewandmaterializedtable,statustableallofthestepsdefinedabovearesameastheyarepartoftheVDB.FollowsamestepsaboveanddeploytheVDBintheembeddedTeiidasshownbelow.
EmbeddedServerserver=newEmbeddedServer();
…
server.addConnectionFactory("name",Object);
…
server.addTranslator("name",ExecutionFactory);
EmbeddedConfigurationconfig=newEmbeddedConfiguration();
ExternalMaterialization
118
config.setTransactionManager(EmbeddedHelper.getTransactionManager());
server.start(config);
server.deployVDB("matView-vdb.xml");
AnUsageExample
Appendix-1:DDLforcreatingMatViewStatusTable
h2
CREATETABLEstatus
(
VDBNamevarchar(50)notnull,
VDBVersionvarchar(50)notnull,
SchemaNamevarchar(50)notnull,
Namevarchar(256)notnull,
TargetSchemaNamevarchar(50),
TargetNamevarchar(256)notnull,
Validbooleannotnull,
LoadStatevarchar(25)notnull,
Cardinalitylong,
Updatedtimestampnotnull,
LoadNumberlongnotnull,
PRIMARYKEY(VDBName,VDBVersion,SchemaName,Name)
);
MariaDB
CREATETABLEstatus
(
VDBNamevarchar(50)notnull,
VDBVersionvarchar(50)notnull,
SchemaNamevarchar(50)notnull,
Namevarchar(256)notnull,
TargetSchemaNamevarchar(50),
TargetNamevarchar(256)notnull,
Validbooleannotnull,
LoadStatevarchar(25)notnull,
Cardinalitybigint,
Updatedtimestampnotnull,
LoadNumberbigintnotnull,
PRIMARYKEY(VDBName,VDBVersion,SchemaName,Name)
)OPTIONS(UPDATABLEtrue);
Appendix-2:ExampleVDBwithExternalMaterializedViewOptions
ThebelowVDBdefinesthreemodels,one"Source"modelthatdefinesyoursourcedatabasewhereyourbusinessdataisin,"ViewModel"definesa"Person"viewwhichisderivedfromsubsetofthedatafromyourtableinthe"Source"model’stable(s).Notethatviewtablealsomarkedwithfewextensionpropertiestoallowexternalmaterialization.The"materialized"modeldefinesasourcedatabasemodel,whereithasatablewithexacttablestructureastheViewModel’smaterializedviewwithadditionalcolumncalled"LoadNumber".Notethe"materializedtablealsocontainsthe"status"table.BoththesetablesmustbecreatedmanuallyonthesourcedatabasebeforeVDBisdeployedtotheserver.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="example"version="1">
<modelname="Source">
<propertyname="importer.useFullSchemaName"value="false"/>
<sourcename="source"translator-name="h2"connection-jndi-name="java:/my-ds"/>
ExternalMaterialization
119
</model>
<modelname="ViewModel"type="VIRTUAL">
<metadatatype="DDL"><![CDATA[
CREATEVIEWPerson(
idvarchar,
namevarchar,
dobdate,
PRIMARYKEY(id)
)OPTIONS(
MATERIALIZED'TRUE',UPDATABLE'TRUE',
MATERIALIZED_TABLE'materialized.PersonCached',
"teiid_rel:MATVIEW_TTL"20000,
"teiid_rel:ALLOW_MATVIEW_MANAGEMENT"'true',
"teiid_rel:MATVIEW_LOADNUMBER_COLUMN"'LoadNumber',
"teiid_rel:MATVIEW_STATUS_TABLE"'materialized.status'
)
AS
SELECTp.id,p.name,p.dobFROMSource.PersonASp;
]]>
</metadata>
</model>
<modelname="materialized"type="PHYSICAL">
<sourcename="matview"translator-name="h2"connection-jndi-name="java:/matview-ds"/>
<metadatatype="DDL"><![CDATA[
CREATEVIEWPersonCached(
idvarchar,
namevarchar,
dobdate,
LoadNumberlong,
PRIMARYKEY(id)
);
CREATETABLEstatus(
VDBNamevarchar(50)notnull,
VDBVersionvarchar(50)notnull,
SchemaNamevarchar(50)notnull,
Namevarchar(256)notnull,
TargetSchemaNamevarchar(50),
TargetNamevarchar(256)notnull,
Validbooleannotnull,
LoadStatevarchar(25)notnull,
Cardinalitylong,
Updatedtimestampnotnull,
LoadNumberlongnotnull,
PRIMARYKEY(VDBName,VDBVersion,SchemaName,Name)
);
]]>
</metadata>
</model>
</vdb>
ExternalMaterialization
120
InternalMaterializationInternalmaterializationcreatesTeiidtemporarytablestoholdthematerializedtable.Whilethesetablesarenotfullydurable,theyperformwellinmostcircumstancesandthedataispresentateachTeiidinstancewhichremovesthesinglepointoffailureandnetworkoverheadofanexternaldatabase.Internalmaterializationalsoprovidesbuilt-infacilitiesforrefreshingandmonitoring.SeeMemoryLimitationsregardingsizelimitations.
TableofContentsViewOptionsLoadingAndRefreshing
UsingSystemProcedureUsingTTLSnapshotRefresh
UpdatableIndexesClusteringConsiderations
ViewOptions
Thematerializedoptionmustbesetfortheviewtobematerialized.TheCacheHint,whenusedinthecontextofaninternalmaterializedviewtransformationquery,providestheabilitytofinetunethematerializedtable.Thecachingoptionsarealsosettableviaextensionmetadata:
PropertyName Description Optional Default
materialized Setfortheviewtobematerialized false true
UPDATABLEAllowupdatingMaterializedViewviaDMLUPDATE
true false
teiid_rel:ALLOW_MATVIEW_MANAGEMENT
AllowTeiidbasedmanagementofthettlandinitialloadratherthantheimplicitbehavior.
true false
teiid_rel:MATVIEW_PREFER_MEMEORY Sameasthepref_memcachehintoption. true false
teiid_rel:MATVIEW_TTL
TriggeraScheduledExecutorServicewhichexecuterefreshMatViewrepeatedlywithaspecifiedtimetolive
true null
teiid_rel:MATVIEW_UPDATABLE
AllowupdatingMaterializedViewviarefreshMatView,refreshMatViewRow,refreshMatViewRows
true false.
teiid_rel:MATVIEW_SCOPE true VDB
InternalMaterialization
121
teiid_rel:MATVIEW_SCOPE Sameasthescopecachehintoption.
true VDB
teiid_rel:MATVIEW_WRITE_THROUGH
WhentrueTeiidwillperformboththeunderlyingupdateandthecorrespondingupdateagainstthematerializationtargetforaninsert/update/deleteissuedagainsttheview.
true false
Thepref_memoptionalsoappliestointernalmaterializedviews.Internaltableindexpagesalreadyhaveamemorypreference,sotheperf_memoptionindicatesthatthedatapagesshouldprefermemoryaswell.
Allinternalmaterializedviewrefreshandupdateshappenatomically.InternalmaterializedviewssupportREAD_COMMITTED(usedalsoforREAD_UNCOMMITED)andSERIALIZABLE(usedalsoforREPEATABLE_READ)transactionisolationlevels.
AsampleVDBdefininganinternalmaterialization
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="sakila"version="1">
<modelname="pg">
<sourcename="pg"translator-name="postgresql"connection-jndi-name="java:/sakila-ds"/>
</model>
<modelname="sakila"type="VIRTUAL">
<metadatatype="DDL"><![CDATA[
CREATEVIEWactor(
actor_idinteger,
first_namevarchar(45)NOTNULL,
last_namevarchar(45)NOTNULL,
last_updatetimestampNOTNULL
)OPTIONS(materializedtrue,
UPDATABLE'TRUE',
"teiid_rel:MATVIEW_TTL"120000,
"teiid_rel:MATVIEW_PREFER_MEMORY"'true',
"teiid_rel:ALLOW_MATVIEW_MANAGEMENT"'true',
"teiid_rel:MATVIEW_UPDATABLE"'true',
"teiid_rel:MATVIEW_SCOPE"'vdb')
ASSELECTactor_id,first_name,last_name,last_updatefrompg."public".actor;
]]>
</metadata>
</model>
</vdb>
LoadingAndRefreshingAninternalmaterializedviewtableisinitiallyinaninvalidstate(thereisnodata).
Ifteiid_rel:ALLOW_MATVIEW_MANAGEMENTisspecifiedastrue,thentheinitialloadwilloccuronvdbstartup.
Ifteiid_rel:ALLOW_MATVIEW_MANAGEMENTisnotspecifiedorfalse,thentheloadofthematerializationtablewilloccuronimplicitonthefirstquerythataccessesthetable.
Whenarefreshhappenswhilethematerializationtableisinvalidallotherqueriesagainstthematerializedviewwillblockuntiltheloadcompletes.
UsingSystemProcedure
InternalMaterialization
122
InsomesituationsadministratorsmaywishtobettercontrolwhenthecacheisloadedwithacalltoSYSADMIN.refreshMatView.Theinitialloadmayitselftriggertheinitialloadofdependentmaterializedviews.Aftertheinitialloaduserqueriesagainstthematerializedviewtablewillonlyblockifitisinaninvalidstate.ThevalidstatemayalsobecontrolledthroughtheSYSADMIN.refreshMatViewprocedure.
InvalidatingRefresh
CALLSYSADMIN.refreshMatView(viewname=>'schema.matview',invalidate=>true)
matviewwillberefreshedanduserquerieswillblockuntiltherefreshiscomplete(orfails).
Whiletheinitialloadmaytriggeratransitiveloadingofdependentmaterializedviews,subsequentrefreshesperformedwithrefreshMatViewwillusedependentmaterializedviewtablesiftheyexist.Onlyoneloadmayoccuratatime.IfaloadisalreadyinprogresswhentheSYSADMIN.refreshMatViewprocedureiscalled,itwillreturn-1immediatelyratherthanpreemptingthecurrentload.
UsingTTLSnapshotRefresh
TheCacheHintorextensionpropertiesmaybeusedtoautomaticallytriggerafullsnapshotrefreshafteraspecifiedtimetolive(ttl).Thebehaviorisdifferentdependingonwhetherthematerializationismanagedornon-managed.
Fornon-managedviewsthettlstartsfromthetimethetableisfinishedloadingandtherefreshwillbeinitiatedafterthettlhasexpiredonaviewaccess.
Formanagedviewsthettlisafixedintervalandrefresheswillbetriggeredregardlessofviewusage.
IneithercasetherefreshisequivalenttoCALLSYSADMIN.refreshMatView('viewname',*),wheretheinvalidationbehavior*isdeterminedbythevdbpropertylazy-invalidate.Bydefaultttlrefreshesareinvalidating,whichwillcauseotheruserqueriestoblockwhileloading.Thatisoncethettlhasexpired,thenextaccesswillberequiredtorefreshthematerializedtableinablockingmanner.Ifyouwouldratherthatthettlisenforcedlazily,suchthatthecurrentcontentsarenotreplaceduntiltherefreshcompletes,setthevdbpropertylazy-invalidate=true.
Auto-refreshTransformationQuery*
/*+cache(ttl:3600000)*/selectt.col,t1.colfromt,t1wheret.id=t1.id
Theresultingmaterializedviewwillbereloadedeveryhour(3600000milliseconds).
TTLSnapshotRefreshLimitations
Theautomaticttlrefreshmaynotbesuitableforcomplexloadingscenariosasnestedmaterializedviewswillbeusedbytherefreshquery.
Thenon-managedttlrefreshisperformedlazily,thatisitisonlytriggerbyusingthetableafterthettlhasexpired.Forinfrequentlyusedtableswithlongloadtimes,thismeansthatdatamaybeusedwellpasttheintendedttl.
Updatable
Inadvanceduse-casesthecachehintmayalsobeusedtomarkaninternalmaterializedviewasupdatable.AnupdatableinternalmaterializedviewmayusetheSYSADMIN.refreshMatViewRowproceduretoupdateasinglerowinthematerializedtable.Ifthesourcerowexists,thematerializedviewtablerowwillbeupdated.Ifthesourcerowdoesnotexist,thecorrepsondingmaterializedrowwillbedeleted.Tobeupdatablethematerializedviewmusthaveasinglecolumnprimarykey.CompositekeysarenotyetsupportedbySYSADMIN.refreshMatViewRow.TransformationQuery:
/*+cache(updatable)*/selectt.col,t1.colfromt,t1wheret.id=t1.id
InternalMaterialization
123
UpdateSQL:
CALLSYSADMIN.refreshMatViewRow(viewname=>'schema.matview',key=>5)
Giventhattheschema.matviewdefinesanintegercolumncolasitsprimarykey,theupdatewillcheckthelivesource(s)fortherowvalues.
Theupdatequerywillnotusedependentmaterializedviewtables,socareshouldbetakentoensurethatgettingasinglerowfromthistransformationqueryperformswell.SeetheReferenceGuideforinformationoncontrollingdependentjoins,whichmaybeapplicabletoincreasingtheperformanceofretrievingasinglerow.Therefreshquerydoesusenestedcaches,sothisrefreshmethodshouldbeusedwithcaution.
Whentheupdatableoptionisnotspecified,accessingthematerializedviewtableismoreefficientbecausemodificationsdonotneedtobeconsidered.Therefore,onlyspecifytheupdatableoptionifrowbasedincrementalupdatesareneeded.Evenwhenperformingrowupdates,fullsnapshotrefreshesmaybeneededtoensureconsistency.
TheEventDistributoralsoexposestheupdateMatViewRowasalowerlevelAPIforProgrammaticControl-careshouldbetakenwhenusingthisupdatemethod.
IndexesInternalmaterializedviewtableswillautomaticallycreateauniqueindexforeachuniqueconstraintandanon-uniqueindexforeachindexdefinedonthematerializedview.Theprimarykey(ifitexists)oftheviewwillautomaticallybepartofaclusteredindex.
Thesecondaryindexesarealwayscreatedasorderedtrees-bitmaporhashindexesarenotsupported.Teiid’smetadataforindexesiscurrentlylimited.Wearenotcurrentlyabletocaptureadditionalinformation,sortdirection,additionalcolumnstocover,etc.Youmayworkaroundsomeoftheselimitationsthough.
Functionbasedindexaresupported,butcanonlybespecifiedthroughDDLmetadata.IfyouarenotusingDDLmetadata,consideraddinganothercolumntotheviewthatprojectsthefunctionexpression,thenplaceanindexonthatnewcolumn.Queriestotheviewwillneedtobemodifiedasappropriatethoughtomakeuseofthenewcolumn/index.
Ifadditionalcoveredcolumnsareneeded,theymaysimplybeaddedtotheindexcolumns.Thishoweverisonlyapplicabletocomparabletypes.Addingadditionalcolumnswillincreasetheamountofspaceusedbytheindex,butmayallowitsusagetoresultinhigherperformancewhenonlythecoveredcolumnsareusedandthemaintableisnotconsulted.
ClusteringConsiderations
Eachmemberinaclustermaintainsitsowncopyofeachmaterializedtableandassociatedindexes.Anattemptismadetoensureeachmemberreceivesthesamefullrefresheventsastheothers.Fullconsistencyforupdatablematerializedviewshoweverisnotguaranteed.Periodicfullrefreshesofupdatablematerializedviewtableshelpsensureconsistencyamongmembers.
InternalMaterialization
124
CodeTableCachingTeiidprovidesashortcuttocreatinganinternalmaterializedviewtableviathelookupfunction.
Thelookupfunctionprovidesawaytoaccelerategettingavalueoutofatablewhenakeyvalueisprovided.Thefunctionautomaticallycachesallofthekey/returnpairsforthereferencedtable.Thiscachingisperformedondemand,butwillproactivelyloadtheresultstoothermembersinacluster.Subsequentlookupsagainstthesametableusingthesamekeyandreturncolumnswillusethecachedinformation.
Thiscachingsolutionisappropriateforintegrationof"referencedata"withtransactionaloroperationaldata.Referencedataisusuallystaticandsmalldatasetsthatareusedfrequently.ExamplesareISOcountrycodes,statecodes,anddifferenttypesoffinancialinstrumentidentifiers.
UsageThiscachingmechanismisautomaticallyinvokedwhenthelookupscalarfunctionisused.Thelookupfunctionreturnsascalarvalue,soitmaybeusedanywhereanexpressionisexpected.Eachtimethisfunctioniscalledwithauniquecombinationofreferencedtable,returncolumn,andkeycolumn(thefirst3argumentstothefunction).
SeetheLookupFunctionintheReferenceGuideformoreinformationonuseofthelookupfunction.
CountryCodeLookup
lookup('ISOCountryCodes','CountryCode','CountryName','UnitedStates')
Limitations
Theuseofthelookupfunctionautomaticallyperformscaching;thereisnooptiontousethelookupfunctionandnotperformcaching.
Nomechanismisprovidedtorefreshcodetables
Onlyasinglekey/returncolumniscached-valueswillnotbesession/userspecific.
MaterializedViewAlternative
Thelookupfunctionisashortcuttocreateaninternalmaterializedviewwithanappropriateprimarykey.Inmanysituations,itmaybebettertodirectlycreatetheanalogousmaterializedviewratherthantouseacodetable.
CountryCodeLookupAgainstAMatView
SELECT(SELECTCountryCodeFromMatISOCountryCodesWHERECountryName=tbl.CountryName)asccFROMtbl
HereMatISOCountryCodesisaviewselectingfromISOCountryCodesthathasbeenmarkedasmaterializedandhasaprimarykeyandindexonCountryName.Thescalarsubquerywillusetheindextolookupthecountrycodeforeachcountrynameintbl.
Reasonstouseamaterializedview:
Morecontrolofthepossiblereturncolumns.Codetableswillcreateamaterializedviewforeachkey/valuepair.Iftherearemultiplereturncolumnsitwouldbebettertohaveasinglematerializedview.
Propermaterializedviewshavebuilt-insystemprocedure/tablesupport.
CodeTableCaching
125
Morecontrolviathecachehint.
TheabilitytouseOPTIONNOCACHE.
Thereisalmostnoperformancedifference.
Stepstocreateamaterializedview:
1. Createaviewselectingtheappropriatecolumnsfromthedesiredtable.Ingeneral,thisviewmayhaveanarbitrarilycomplicatedtransformationquery.
2. Designatetheappropriatecolumn(s)astheprimarykey.Additionalindexescanbeaddedifneeded.
3. Setthematerializedpropertytotrue.
4. Addacachehinttothetransformationquery.Tomimicthebehavioroftheimplicitinternalmaterializedviewcreatedbythelookupfunction,usetheHintsandOptions/*+cache(pref_mem)*/toindicatethatthetabledatapagesshouldprefertoremaininmemory.
Justaswiththelookupfunction,thematerializedviewtablewillbecreatedonfirstuseandreusedsubsequently.SeetheMaterializedViewsformore.
CodeTableCaching
126
TranslatorResultsCachingTranslatorscancontributecacheentriesintotheresultsetcacheviatheuseoftheCacheDirectiveobject.Theresultingcacheentriesbehavejustasiftheywerecreatedbyauserquery.SeetheTranslatorCachingAPIformoreonthisfeature.
TranslatorResultsCaching
127
CacheHint
Aquerycachehintcanbeusedto:
Indicatethatauserqueryiseligibleforresultsetcachingandsetthecacheentrymemorypreference,timetolive,etc.
Setthematerializedviewmemorypreference,timetolive,orupdatablity.
Indicatethatavirtualprocedureshouldbecachableandsetthecacheentrymemorypreference,timetolive,etc.
/*+cache[([pref_mem][ttl:n][updatable][scope:session|user|vdb])]*/sql...
ThecachehintshouldappearatthebeginningoftheSQL.Itcanbeappearasanyoneoftheleadingcomments.ItwillnothaveanyaffectonINSERT/UPDATE/DELETEstatementsorINSTEADOFTRIGGERS.
pref_mem-ifpresentindicatesthatthecachedresultsshouldprefertoremaininmemory.Theresultsmaystillbepagedoutbaseduponmemorypressure.
Note
Careshouldbetakentonotoverusethepref_memoption.ThememorypreferenceisimplementedwithJavasoftreferences.Whilesoftreferencesareeffectiveatpreventingoutofmemoryconditions.Toomuchmemoryheldbysoftreferencescanlimittheeffectiveworkingmemory.ConsultyourJVMoptionsforclearingsoftreferencesifyouneedtotunetheirbehavior.
ttl:n-ifpresentnindicatesthetimetolivevalueinmilliseconds.ThedefaultvalueforresultsetcachingisthedefaultexpirationforthecorrespondingInfinispancache.Thereisnodefaulttimetoliveformaterializedviews.
updatable-ifpresentindicatesthatthecachedresultscanbeupdated.Thisdefaultstofalseformaterializedviewsandtotrueforresultsetcacheentries.
scope-Therearethreedifferentcachescopes:session-cachedonlyforcurrentsession,user-cachedforanysessionbythecurrentuser,vdb-cachedforanyuserconnectedtothesamevdb.Forcachedqueriesthepresenceofthescopeoverridesthecomputedscope.Materializedviewsontheotherhanddefaulttothevdbscope.Formaterializedviewsexplicitlysettingthesessionoruserscopeswillresultinanon-replicatedsessionscopedmaterializedview.
Thepref_mem,ttl,updatable,andscopevaluesforamaterializedviewmayalsobesetviaextensionpropertiesontheview-usingtheteiid_relnamespacewithMATVIEW_PREFER_MEMORY,MATVIEW_TTL,MATVIEW_UPDATABLE,andMATVIEW_SCOPErespectively.Ifbotharepresent,theuseofanextensionpropertysupersedestheusageofthecachehint.
Limitations
Theformofthequeryhintmustbematchedexactlyforthehinttohaveaffect.Forauserqueryifthehintisnotspecifiedcorrectly,e.g./*+cach(pref_mem)*/,itwillnotbeusedbytheenginenorwilltherebeaninformationallog.Itiscurrentlyrecommendedthatyouverify(seeClientDevelopersGuide)inyourtestingthattheusercommandinthequeryplanhasretainedtheproperhint.
OPTIONNOCACHE
IndividualqueriesmayoverridetheuseofcachedresultsbyspecifyingOPTIONNOCACHEonthequery.0ormorefullyqualifiedvieworprocedurenamesmaybespecifiedtoexcludeusingtheircachedresults.Ifnonamesarespecified,cachedresultswillnotbeusedtransitively.
FullNOCACHE
HintsandOptions
128
SELECT*fromvg1,vg2,vg3WHERE…OPTIONNOCACHE
Nocachedresultswillbeusedatall.
SpecificNOCACHE
SELECT*fromvg1,vg2,vg3WHERE…OPTIONNOCACHEvg1,vg3
Onlythevg1andvg3cacheswillbeskipped,vg2oranycachedresultsnestedundervg1andvg3willbeused.
OPTIONNOCACHEmaybespecifiedinprocedureorviewdefinitions.Inthatway,transformationscanspecifytoalwaysusereal-timedataobtaineddirectlyfromsources.
HintsandOptions
129
ProgrammaticControlTeiidexposesabeanthatimplementstheorg.teiid.events.EventDistributorinterface.ItcanbelookedupinJNDIunderthenameteiid/event-distributor-factory.TheEventDistributorexposesmethodslikedataModification(whichaffectsresultsetcaching)orupdateMatViewRow(whichaffectsinternalmaterialization)toalerttheTeiidenginethattheunderlyingsourcedatahasbeenmodified.Theseoperations,whichworkclusterwidewillinvalidatethecacheentriesappropriatelyandreloadthenewcachecontents.
NoteChangeDataCapture-Ifyoursourcesystemhasanybuilt-inchangedatacapturefacilitiesthatcanscrapelogs,installtriggers,etc.tocapturedatachangeevents,theycancapturedandcanbepropagatedtoTeiidenginethroughapojobean/MDB/SessionBeandeployedinWildFlyengine.
ThebelowshowsacodeexampleashowusercanuseEventDistributorinterfaceintheirowncodethatisdeployedinthesameWildFlyVMusingaPojo/MDB/SessionBean.ConsultWildFlydocumentsdeployingasbeanastheyoutofscopeforthisdocument.
EventDistributorCodeExample
publicclassChangeDataCapture{
publicvoidinvalidate(){
InitialContextic=newInitialContext();
EventDistributored=((EventDistributorFactory)ic.lookup("teiid/event-distributor-factory")).getEventD
istributor();
//thisbelowlineindicatesthatCustomertableinthe"model-name"schemahasbeenchanged.
//thisresultincachereload.
ed.dataModification("vdb-name","version","model-name","Customer");
}
}
NoteUpdatingCostinginformation-TheEventDistributorinterfacealsoexposesmanymethodsthatcanbeusedtoupdatethecostinginformationonyoursourcemodelsforoptimizedqueryplanning.Notethatthesevaluesvolatileandwillbelostduringaclusterre-start,asthereisnorepositorytopersist.
ProgrammaticControl
130
ClientDeveloper’sGuideThisguideintendedfordevelopersthataretryingtowrite3rdpartyapplicationsthatinteractwithTeiid.Thiswillguideyouthroughconnectionmechanisms,extensionstoJDBCAPI,ODBC,SSLetc.BeforeonecandelveintoTeiiditisveryimportanttolearnfewbasicconstructsofTeiid,likewhatisVDB?whatisModel?etc.Forthatpleasereadtheshortintroductionherehttp://teiid.jboss.org/basics/
ClientDeveloper’sGuide
131
JDBCSupportTeiidprovidesarobustJDBCdriverthatimplementsmostoftheJDBCAPIaccordingtothelatestspecificationandsupportedJavaversion.MosttoolingdesignedtoworkwithJDBCshouldworkseamlesslywiththeTeiiddriver.Whenindoubt,seeUnsupportedJDBCMethodsforfunctionalitythathasyettobeimplemented.
Ifyou’reneedsgobeyondJDBC,TeiidhasalsoprovidedJDBCExtensionsforasynchhandling,federation,andotherfeatures.
GeneratedKeys
TeiidsupportsreturninggeneratedkeysforJDBCsourcesandfromTeiidtemptableswithSERIALprimarykeycolumns.Howeverthecurrentimplementationwillreturnonlythelastsetofkeysgeneratedandwillreturnthekeyresultsdirectlyfromthesource-noviewprojectionofotherintermediatehandlingisperformed.Formostscenarios(singlesourceinserts)thishandlingissufficient.AcustomsolutionmayneedtobedevelopedifyouareusingaFOREACHROWinsteadoftriggertoprocessyourinsertsandtargetmultipletablesthateachreturngeneratedkeys.ItispossibletodevelopaUDFthatalsomanipulatesthereturnedgeneratedkeys-seetheorg.teiid.CommandContextmethodsdealingwithgeneratedkeysformore.
Note GeneratedKeysisnotsupportedwhentheJDBCBatchedupdatesisusedtoinsertthevaluesintothesourcetable.
JDBCSupport
132
ConnectingtoaTeiidServerTheTeiidJDBCAPIprovidesJavaDatabaseConnectivity(JDBC)accesstoanyVirtualDatabase(VDB)deployedonaTeiidServer.TheTeiidJDBCAPIiscompatiblewiththeJDBC4.0specification;however,itdoesnotfullysupportallmethods.Advancedfeatures,suchasupdatableresultsetsorSQL3datatypes,arenotsupported.
JavaclientapplicationsconnectingtoaTeiidServerwillneedtouseJava1.6JDK.PreviousversionsofJavaarenotsupported.
BeforeyoucanconnecttotheTeiidServerusingtheTeiidJDBCAPI,pleasedofollowingtasksfirst.
1. InstalltheTeiidServer.Seethe"AdminGuide"forinstructions.
2. BuildaVirtualDatabase(VDB).YoucanbuildaVDBwithoutaDesigner,oryoucanusetheEclipsebasedGUItoolDesigner.Checkthe"ReferenceGuide"forinstructionsonhowtobuildaVDB.IfyoudonotknowwhatVDBis,thenstartwiththisdocument.
3. DeploytheVDBintoTeiidServer.CheckAdministrator’sGuideforinstructions.
4. StarttheTeiidServer(WildFly),ifitisnotalreadyrunning.
NowthatyouhavetheVDBdeployedintheTeiidServer,clientapplicationscanconnecttotheTeiidServerandissueSQLqueriesagainstdeployedVDBusingJDBCAPI.IfyouarenewtoJDBC,seeJava’sdocumentationaboutJDBC.Teiidshipswithteiid-9.2.0.Final-jdbc.jarthatcanbefoundinthedownloads.
YoucanalsoobtaintheTeiidJDBCfromtheJBossPublicMavenRepositoryhttp://repository.jboss.org/nexus/content/groups/public/usingthecoordinates:
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid</artifactId>
<classifier>jdbc</classifier>
<version>9.2.0.Final</version>
</dependency>
Against
MainclassesintheclientJAR:
org.teiid.jdbc.TeiidDriver-allowsJDBCconnectionsusingtheDriverManagerclass.
org.teiid.jdbc.TeiidDatasource-allowsJDBCconnectionsusingtheDataSourceXADataSourceclass.YoushouldusethisclasstocreatemanagedorXAconnections.
OnceyouhaveestablishedaconnectionwiththeTeiidServer,youcanusestandardJDBCAPIclassestointerrogatemetadataandexecutequeries.
ConnectingtoaTeiidServer
133
DriverConnectionUseorg.teiid.jdbc.TeiidDriverasthedriverclass.
UsethefollowingURLformatforJDBCconnections:
jdbc:teiid:<vdb-name>[@mm[s]://<host>:<port>][;prop-name=prop-value]*
NoteTheJDBCclientwillhavebothJREandservercompatibilityconsiderations.Unlessotherwisestatedaclientjarwilltypicallybeforwardandbackwardscompatiblewithonemajorversionoftheserver.Youshouldattempttokeeptheclientup-to-datethoughasfixesandfeaturesaremadeontotheclient.
URLComponents
1. <vdb-name>-NameoftheVDByouareconnectingto.OptionallyVDBnamecanalsocontainversioninformationinsideit.Forexample:"myvdb.2",thisisequivalenttosupplyingthe"version=2"connectionpropertydefinedbelow.However,useofvdbnameinthisformatandthe"version"propertyatthesametimeisnotallowed.
2. mm-definesTeiidJDBCprotocol,mmsdefinesasecurechannel(seeSSLClientConnectionsformore)
3. <host>-definestheserverwheretheTeiidServerisinstalled.IfyouareusingIPv6bindingaddressasthehostname,placeitinsquarebrackets.ex:[::1]
4. <port>-definestheportonwhichtheTeiidServerislisteningforincomingJDBCconnections.
5. [prop-name=prop-value]-additionallyyoucansupplyanynumberofnamevaluepairsseparatedbysemi-colon[;].AllsupportedURLpropertiesaredefinedintheconnectionpropertiessection.PropertyvaluesshouldbeURLencodediftheycontainreservedcharacters,e.g.(’?’,'=',';',etc.)
Note hostandportmaybeacommaseparatedlisttospecifymultiplehosts.
LocalConnections
Tomakealocalin-VMconnection,omittheprotocolandhost/port.
URLConnectionProperties
ThefollowingtableshowsallthesupportedconnectionpropertiesthatcanusedwithTeiidJDBCDriverURLconnectionstring,orontheTeiidJDBCDataSourceclass.
Table1.ConnectionProperties
PropertyName Type Description
ApplicationName StringNameoftheclientapplication;allowstheadministratortoidentifytheconnections
FetchSize intSizeoftheresultset;Thedefaultsizeif500.⇐0indicatesthatthedefaultshouldbeused.
partialResultsMode booleanEnable/disablesupportpartialresultsmode.Defaultfalse.SeethePartialResultsModesection.
ConnectingtoaTeiidServer
134
autoCommitTxn String
Onlyappliesonlywhen"autoCommit"issetto"true".ThisdetermineshowaexecutedcommandneedstobetransactionallywrappedinsidetheTeiidenginetomaintainthedataintegrity.
ON-Alwayswrapcommandindistributedtransaction
OFF-Neverwrapcommandindistributedtransaction
DETECT(default)-Iftheexecutedcommandisspanningmorethanonesourceitautomaticallyusesdistributedtransaction.Transactionsformoreinformation.
disableLocalTxn boolean
If"true",theautoCommitsetting,commitandrollbackwillbeignoredforlocaltransactions.Defaultfalse.
user String Username
Password String Credentialforuser
ansiQuotedIdentifiers boolean
SetstheparsingbehaviorfordoublequotedentriesinSQL.Thedefault,true,parsesdobuledquotedentriesasidentifiers.Ifsettofalse,thendoublequotedvaluesthatarevalidstringliteralswillbeparsedasstringliterals.
version integer VersionnumberoftheVDB
resultSetCacheMode boolean ResultSetcachingisturnedon/off.Defaultfalse.
autoFailover boolean
Iftrue,willautomaticallyselectanewserverinstanceafteracommunicationexception.Defaultfalse.Thisistypicallynotneededwhenconnectionsaremanaged,astheconnectioncanbepurgedfromthepool.
SHOWPLAN String
(typicallynotsetasaconnectionproperty)CanbeON,OFF,DEBUG;
ONreturnsthequeryplanalongwiththeresults
DEBUGadditionallyprintsthequeryplannerdebuginformationinthelogandreturnsitwiththeresults.BoththeplanandthelogareavailablethroughJDBCAPIextensions.
DefaultOFF.
NoExec String
(typicallynotsetasaconnectionproperty)CanbeON,OFF;ONpreventsqueryexecution,butparsingandplanningwillstilloccur.DefaultOFF.
PassthroughAuthentication boolean
Onlyappliesto"local"connections.Whenthisoptionissetto"true",thenTeiidlooksforalreadyauthenticatedsecuritycontextonthecallingthread.Ifonefounditusesthatuserscredentialstocreatesession.Teiidalsoverifiesthatthesameuserisusingthisconnectionduringthelifeoftheconnection.ifitfindsadifferentsecuritycontextonthecallingthread,
ConnectingtoaTeiidServer
135
itswitchestheidentityontheconnection,ifthenewuserisalsoeligibletologintoTeiidotherwiseconnectionfailstoexecute.
useCallingThread boolean
Onlyappliesto"local"connections.Whenthisoptionissetto"true"(thedefault),thenthecallingthreadwillbeusedtoprocessthequery.Iffalse,thenanenginethreadwillbeused.
QueryTimeout integer
Defaultquerytimeoutinseconds.Mustbe>=0.0indicatesnotimeout.CanbeoverridenbyStatement.setQueryTimeout.Default0.
useJDBC4ColumnNameAndLabelSemantics boolean
AchangewasmadeinJDBC4toreturnunaliasedcolumnnamesastheResultSetMetadatacolumnname.Priortothis,ifacolumnaliaswereuseditwasreturnedasthecolumnname.SettingthispropertytofalsewillenablebackwardscompatibilitywhenJDBC3andoldersupportisstillrequired.Defaultstotrue.
jaasName String
JAASconfigurationname.OnlyapplieswhenconfiguringaGSSauthentication.DefaultstoTeiid.SeetheSecurityGuideforconfigurationrequiredforGSS.
kerberosServicePrincipleName String
Kerberosauthenticatedprinciplename.OnlyapplieswhenconfiguringaGSSauthentication.SeetheSecurityGuideforconfigurationrequiredforGSS
encryptRequest boolean
Onlyappliestonon-SSLsocketconnections.When"true"therequestmessageandanyassociatepayloadwillbeencryptedusingtheconnectioncryptor.Defaultfalse.
disableResultSetFetchSize boolean
Insomesituationstoolingmaychooseundesirablefetchsizesforprocessingresults.SettotruetodisablehonoringResultSet.setFetchSize.Defaultfalse.
loginTimeout integer
Thelogintimeoutinseconds.Mustbe>=0.0indicatesnospecifictimeout,butothertimeoutsmayapply.Ifaconnectioncannotbecreatedinapproximatelythethetimeoutvalueanexceptionwillbethrown.Adefaultof0doesnotmeanthattheloginwillwaitindefinitely.Typicallyisanactivevdbcannotbefoundtheloginwillfailatthattime.Localconnectionsthatspecifyavdbversionhowevercanwaitbydefaultforuptoorg.teiid.clientVdbLoadTimeoutMillis.
reportAsViews boolean
IfDatabaseMetaDatawillreportTeiidviewsasaVIEWtabletype.IffalsethenTeiidviewswillbereportedasaTABLE.Defaulttrue.
ConnectingtoaTeiidServer
136
DataSourceConnectionTouseadatasourcebasedconnection,useorg.teiid.jdbc.TeiidDataSourceasthedatasourceclass.TheTeiidDataSourceisalsoanXADatasource.TeiidDataSourceclassisalsoSerializable,soitpossibleforittobeusedwithJNDInamingservices.
TeiidsupportstheXAprotocol,XAtransactionswillbeextendedtoTeiidsourcesthatalsosupportXA.
Alltheproperties(exceptforversion,whichisknownonTeiidDataSourceasDatabaseVersion)definedintheDriverConnection#URLConnectionPropertieshavecorresponding"set"methodsontheorg.teiid.jdbc.TeiidDataSource.PropertiesthatareassumedfromtheURLstringhaveadditional"set"methods,whicharedescribedinthefollowingtable.
Table1.DatasourceProperties
PropertyName Type Description
DatabaseName String
Thenameofavirtualdatabase(VDB)deployedtoTeiid.OptionallyDatabasenamecanalsocontain"DatabaseVersion"informationinsideit.Forexample:"myvdb.2",thisisequivalenttosupplyingthe"DatabaseVersion"propertysettovalueof2.However,useofDatabasenameinthisformatanduseofDatabaseVersionpropertyatthesametimeisnotallowed.
ServerName String
ServerhostnamewheretheTeiidruntimeinstalled.IfyouareusingIPv6bindingaddressasthehostname,placeitinsquarebrackets.ex:[::1]
AlternateServers String
Optionaldelimitedlistofhost:portentries.SeetheUsingMultipleHostsformoreinformation.IfyouareusingIPv6bindingaddressasthehostname,placetheminsquarebrackets.ex:[::1]
AdditionalProperties String
OptionalsettingofpropertiesthathasthesameformatasthepropertystringinaconnectionURL.
PortNumber integerPortnumberonwhichtheServerprocessislisteningon.
secure boolean
Secureconnection.FlagtoindicatetouseSSL(mms)basedconnectionbetweenclientandserver
DatabaseVersion integer VDBversion
DataSourceName String Namegiventothisdatasource
LoadBalance boolean
Settofalsetodisablethedefaultloadbalancingbehaviorofselectinganewserverwhenapooledconnectionisreturnedtothepool.
ConnectingtoaTeiidServer
137
NoteAdditionalProperties-AllthepropertiesfromURLConnectionPropertiescanbeusedonDataSourceusingtheAdditionalPropertiessettermethodifthecorrespondingsettermethodisnotalreadyavailable.Forexample,youcanadd"useCallingThread"propertyas<xa-datasource-propertyname="AdditionalProperties">useCallingThread=false</xa-datasource-property>
ConnectingtoaTeiidServer
138
StandaloneApplicationTouseeitherDriverorDataSourcebasedconnections,addtheclientJARtoyourJavaclientapplication’sclasspath.SeethesimpleclientexampleinthekitforafullJavasampleofthefollowing.
DriverConnection
SampleCode:
publicclassTeiidClient{
publicConnectiongetConnection(Stringuser,Stringpassword)throwsException{
Stringurl="jdbc:teiid:myVDB@mm://localhost:31000;ApplicationName=myApp";
returnDriverManager.getConnection(url,user,password);
}
}
DatasourceConnection
SampleCode:
publicclassTeiidClient{
publicConnectiongetConnection(Stringuser,Stringpassword)throwsException{
TeiidDataSourceds=newTeiidDataSource();
ds.setUser(user);
ds.setPassword(password);
ds.setServerName("localhost");
ds.setPortNumber(31000);
ds.setDatabaseName("myVDB");
returnds.getConnection();
}
}
ConnectingtoaTeiidServer
139
WildFlyDataSourceTeiidcanbeconfiguredasaJDBCdatasourceinaWildFlyServertobeaccessedfromJNDIorinjectedintoyourJEEapplications.DeployingTeiidasdatasourceinWildFlyisexactlysameasdeployinganyotherRDBMSresourceslikeOracleorDB2.
DefiningasdatasourceisnotlimitedtoWildFly,youcanalsodeployasdatasourceinGlassfish,Tomcat,Websphere,Weblogicetcservers,howevertheirconfigurationfilesaredifferentthanWildFly.Consulttherespectivedocumentationoftheenvironmentinwhichyouaredeploying.
AspecialcaseexistsiftheTeiidinstanceyouareconnectingtoisinthesameVMastheWildFlyinstance.Ifthatmatchesyoudeployment,thenfollowtheLocalJDBCConnectioninstructions
InstallationSteps
1. IfyouareworkingwithanASinstancethatalreadyhasTeiidinstalledthenrequiredmodule/jarfilesarealreadyinstalled.IftheASinstancedoesnothaveTeiidinstalled,thenyoushouldcreateamodulefortheclientjar.Underthepathmodule/org/jboss/teiid/clientaddtheclientjarandamodule.xmldefinedas:
SampleTeiidClientModule
<modulexmlns="urn:jboss:module:1.1"name="org.jboss.teiid.client">
<resources>
<resource-rootpath="teiid-{version}-jdbc.jar"/>
</resources>
<dependencies>
<modulename="javax.api"/>
<modulename="javax.transaction.api"/>
</dependencies>
</module>
Note PriortoTeiid8.12.3amoduledependencyonsun.jdkwasalsorequired.
1. UsetheCLIoreditthestandalone-teiid.xmlordomain-teiid.xmlfileandaddadatasourceintothe"datasources"subsystem.
Basedonthetypeofdeployment(XA,driver,orlocal),thecontentsofthiswillbedifferent.Seethefollowingsectionsformore.ThedatasourcewillthenbeaccessiblethroughtheJNDInamespecifiedinthebelowconfiguration.
DataSourceConnection
MakesureyouknowthecorrectDatabaseName,ServerName,Portnumberandcredentialsthatarespecifictoyourdeploymentenvironment.
SampleXADataSourceintheWildFlyusingtheTeiidDataSourceclassorg.teiid.jdbc.TeiidDataSource
<datasources>
<xa-datasourcejndi-name="java:/teiidDS"pool-name="teiidDS"enabled="true"use-java-context="true"use
-ccm="true">
<xa-datasource-propertyname="PortNumber">31000</xa-datasource-property>
<xa-datasource-propertyname="DatabaseName">{db-name}</xa-datasource-property>
<xa-datasource-propertyname="ServerName">{host}</xa-datasource-property>
<driver>teiid</driver>
<xa-pool>
<min-pool-size>10</min-pool-size>
ConnectingtoaTeiidServer
140
<max-pool-size>20</max-pool-size>
<is-same-rm-override>true</is-same-rm-override>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
<no-tx-separate-pools/>
</xa-pool>
<security>
<user-name>{user}</user-name>
<password>{password}</password>
</security>
</xa-datasource>
<drivers>
<drivername="teiid"module="org.jboss.teiid.client">
<driver-class>org.teiid.jdbc.TeiidDriver</driver-class>
<xa-datasource-class>org.teiid.jdbc.TeiidDataSource</xa-datasource-class>
</driver>
</drivers>
</datasources>
DriverbasedconnectionYoucanalsouseTeiid’sJDBCdriverclassorg.teiid.jdbc.TeiidDrivertocreateadatasource
<datasources>
<datasourcejndi-name="java:/teiidDS"pool-name="teiidDS">
<connection-url>jdbc:teiid:{vdb}@mm://{host}:31000</connection-url>
<driver>teiid</driver>
<pool>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<user-name>{user}</user-name>
<password>{password}</password>
</security>
</datasource>
<drivers>
<drivername="teiid"module="org.jboss.teiid.client">
<driver-class>org.teiid.jdbc.TeiidDriver</driver-class>
<xa-datasource-class>org.teiid.jdbc.TeiidDataSource</xa-datasource-class>
</driver>
</drivers>
</datasources>
LocalJDBCConnectionIfyouaredeployingyourclientapplicationonthesameWildFlyinstanceastheTeiidruntimeisinstalled,thenyouwillwanttoconfiguretheconnectiontoby-passmakingasocketbasedJDBCconnection.Byusingaslightlydifferentdatasourceconfigurationtomakea"local"connection,theJDBCAPIwilllookupalocalTeiidruntimeinthesameVM.
Warning
SinceDataSourcesstartbeforeTeiidVDBsaredeployed,leavetheminpoolsizeof0forlocalconnections.OtherwiseerrorsmayoccuronthestartupoftheTeiidDataSource.AlsonotethatlocalconnectionsspecifyingaVDBversionwillwaitfortheirVDBtobeloadedbeforeallowingaconnection.SeeloginTimeoutandtheorg.teiid.clientVdbLoadTimeoutMillissystemproperty.
WarningDonotincludeanyadditionalcopyofTeiidjarsintheapplicationclassloadthatisutilizingthelocalconnection.Eveniftheexactsameversionoftheclientjarisincludedinyourapplicationclassloader,youwillfailtoconnecttothelocalconnectionwithaclasscastexception.
ConnectingtoaTeiidServer
141
NoteBydefaultlocalconnectionsusetheircallingthreadtoperformprocessingoperationsratherthanusinganenginethreadwhilethecallingthreadisblocked.TodisablethisbehaviorsettheconnectionpropertyuseCallingThreads=false.Thedefaultistrue,andisrecommendedintransactionalqueries.
Localdatasource
<datasources>
<datasourcejndi-name="java:/teiidDS"pool-name="teiidDS">
<connection-url>jdbc:teiid:{vdb}</connection-url>
<driver>teiid-local</driver>
<pool>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<user-name>{user}</user-name>
<password>{password}</password>
</security>
</datasource>
<drivers>
<drivername="teiid-local"module="org.jboss.teiid">
<driver-class>org.teiid.jdbc.TeiidDriver</driver-class>
<xa-datasource-class>org.teiid.jdbc.TeiidDataSource</xa-datasource-class>
</driver>
</drivers>
</datasources>
ThisisessentiallythesameastheXAconfiguration,but"ServerName"and"PortNumber"arenotspecified.LocalconnectionshaveadditionalfeaturessuchasusingPassthroughAuthentication
ConnectingtoaTeiidServer
142
UsingMultipleHostsAgroupofTeiidServersinthesameWildFlyclustermaybeconnectedusingfailoverandload-balancingfeatures.Toenablethesesfeaturesintheirsimplestform,theclientneedstospecifymultiplehostnameandportnumbercombinationsontheURLconnectionstring.
ExampleURLconnectionstring
jdbc:teiid:<vdb-name>@mm://host1:31000,host1:31001,host2:31000;version=2
IfyouareusingaDataSourcetoconnecttoTeiidServer,usethe"AlternateServers"property/methodtodefinethefailoverservers.Theformatisalsoacommaseparatedlistofhost:portcombinations.
TheclientwillrandomlypickonetheTeiidserverfromthelistandestablishasessionwiththatserver.Ifthatservercannotbecontacted,thenaconnectionwillbeattemptedtoeachoftheremainingserversinrandomorder.Thisallowsforbothconnectiontimefail-overandrandomserverselectionloadbalancing.
FailOver
Postconnectionfailoverwillbeused,iftheautoFailoverconnectionpropertyonJDBCURLissettotrue.Postconnectionfailoverworksbysendingaping,atmosteverysecond,totesttheconnectionpriortouse.Ifthepingfails,anewinstancewillbeselectedpriortotheoperationbeingattempted.Thisisnottrue"transparentapplicationfailover"astheclientwillnotrestartthetransaction/query/recreatesessionscopedtemptables,etc.Sothisfeatureshouldbeusedwithcaution.
LoadBalancing
Postconnectionloadbalancingcanbeutilizedinoneoftwoways.FirstifyouareusingTeiidDataSourceandtheConnectionsreturnedbyTeiidPooledConnectionshavetheirclosemethodcalled,thenanewserverinstancewillbeselectedautomatically.HoweverwhenusingdriverbasedconnectionsorevenwhenusingTeiidDataSourceinaconnectionpool(suchasWildFly),theautomaticloadbalancingwillnothappen.Secondyoucanexplicitlytriggerloadbalancingthroughtheuseofthesetstatement:
SETNEWINSTANCETRUE
Typicallyyouwillnotneedwanttoissuethisstatementmanually,butinsteaduseitastheconnectiontestqueryonyourDataSourceconfiguration.
WildFlyDataSourceWithPostConnectionLoadBalancing
<datasources>
<datasourcejndi-name="java:/teiidDS"pool-name="teiidDS">
<connection-url>jdbc:teiid:{vdb}@mm://{host}:31000</connection-url>
<driver>teiid</driver>
<pool>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
<check-valid-connection-sql>SETNEWINSTANCETRUE</check-valid-connection-sql>
</pool>
<security>
<user-name>{user}</user-name>
<password>{password}</password>
</security>
ConnectingtoaTeiidServer
143
</datasource>
<drivers>
<drivername="teiid"module="org.jboss.teiid.client">
<driver-class>org.teiid.jdbc.TeiidDriver</driver-class>
<xa-datasource-class>org.teiid.jdbc.TeiidDataSource</xa-datasource-class>
</driver>
</drivers>
</datasources>
Teiidbydefaultmaintiansapoolofextrasocketconnectionsthatarereused.Forloadbalancing,thisreducesthepotentialcostofswitchingaconnectiontoanotherserverinstance.Thedefaultsettingistomaintain16connections(canbesetviaorg.teiid.sockets.maxCachedInstancesinateiid-client-settings.propertiesfile).Ifyou’reclientisconnectingtolargenumbersofTeiidinstancesandyou’reusingpostconnectiontimeloadbalancing,thenconsiderincreasingthenumberofcachedinstances.Youmayeithersetananalogoussystempropertyorcreateateiid-client-settings.properties(seetheteiid-client-settings.orig.propertiesfileintheclientjar)fileandplaceitintotheclasspathaheadoftheclientjar.
Note
Sessionleveltemporarytables,currentlyrunningtransactions,sessionlevelcacheentries,andPreparedPlansforagivensessionwillnotbeavailableonotherclustermembers.Therefore,itisrecommendedthatpostconnectiontimeloadbalancingisonlyusedwhenthelogicalconnectioncouldhavebeenclosed,buttheactualconnectionisreused(thetypicalconnectionpoolpattern).
AdvancedConfigurationServerdiscovery,loadbalancing,failover,retry,retrydelay,etc.maybecustomizeifthedefaultpolicyisnotsufficient.Seetheorg.teiid.net.socket.ServerDiscoveryinterfaceanddefaultimplementaionorg.teiid.net.socket.UrlServerDiscoveryforhowtostartwithyourcustomization.TheUrlServerDiscoveryimplemenationprovidesthefollowing:discoveryofserversfromtheURLhosts(DataSourceserver/alternativeServers),randomselectionforloadbalancingandfailover,1connectionattemptperhost,nobiasing,blacklisting,orotheradvancedfeatures.Typicallyyou’llwanttoextendtheUrlServerDiscoverysothatitcanbeusedasthefallbackstrategyandtoonlyimplementthenecessarychangedmethods.It’simportanttoconsiderthat1ServerDiscoveryinstancewillbecreatedforeachconnection.Anysharingofinformationbetweeninstancesshouldbedonethroughstaticstateorsomeothersharedlookup.
YourcustomizedserverdiscoveryclasswillthenneedtobereferencedbythediscoveryStategyconnection/DataSourcepropertybyitsfullclassname.
Youmayalsochoosetouseanexternaltcploadbalancer,suchashaproxy.TheTeiiddriver/DataSourceshouldthentypicallybeconfiguredtojustusethesinglehost/portofyourloadbalancer.
ConnectingtoaTeiidServer
144
ClientSSLSettings
ThefollowingsectionsdefinethepropertiesrequiredforeachSSLmode.NotethatwhenconnectingtoTeiidServerwithSSLenabled,youMUSTusethe"mms"protocol,insteadof"mm"intheJDBCconnectionURL,forexample
Note AnonymousSSLmodeisnotsupportedforsomeJREs,seetheTeiidServerTransportSecurityforalternatives.
jdbc:teiid:<myVdb>@mms://<host>:<port>
Therearetwodifferentsetsofpropertiesthataclientcanconfiguretoenable1-wayor2-waySSL.SeealsotheTeiidServerTransportSecuritychapterifyouareresponsibleforconfiguringtheserveraswell.
Option1:JavaSSLproperties
ThesearestandardJavadefinedsystempropertiestoconfiguretheSSLunderanyJVM,TeiidisnotuniqueinitsuseofSSL.ProvidethefollowingsystempropertiestotheclientVMprocess.
1-waySSL
-Djavax.net.ssl.trustStore=<dir>/server.truststore(required)
-Djavax.net.ssl.trustStorePassword=<password>(optional)
-Djavax.net.ssl.keyStoreType(optional)
2-waySSL
-Djavax.net.ssl.keyStore=<dir>/client.keystore(required)
-Djavax.net.ssl.keyStrorePassword=<password>(optional)
-Djavax.net.ssl.trustStore=<dir>/server.truststore(required)
-Djavax.net.ssl.trustStorePassword=<password>(optioanl)
-Djavax.net.ssl.keyStroreType=<keystoretype>(optional)
Option2:TeiidSpecificProperties
Usethisoptionwhentheabove"javax"basedpropertiesarealreadyinusebythehostprocess.ForexampleifyourclientapplicationisaTomcatprocessthatisconfiguredforhttpsprotocolandtheaboveJavabasedpropertiesarealreadyinuse,andimportingTeiid-specificcertificatekeysintothosehttpscertificatekeystoresisnotallowed.
Inthisscenario,adifferentsetofTeiid-specificSSLpropertiescanbesetassystempropertiesordefinedinsidethea"teiid-client-settings.properties"file.Asample"teiid-client-settings.properties"filecanbefoundinsidethe"teiid-<version>-client.jar"fileattherootcalled"teiid-client-settings.orig.properties".Extractthisfile,makeacopy,changethepropertyvaluesrequiredforthechosenSSLmode,andplacethisfileintheclientapplication’sclasspathbeforethe"teiid-<version>-client.jar"file.
SSLpropertiesanddefinitionsthatcanbesetina"teiid-client-settings.properties"fileareshownbelow.
########################################
#SSLSettings
########################################
#
#Thekeystoretype.DefaultstoJKS
#
org.teiid.ssl.keyStoreType=JKS
#
#Thekeystorealgorithm,defaultsto
ConnectingtoaTeiidServer
145
#thesystemproperty"ssl.TrustManagerFactory.algorithm"
#
#org.teiid.ssl.algorithm=
#
#Theclasspathorfilesystemlocationofthe
#keystore.
#
#Thispropertyisrequiredonlyifperforming2-way
#authenticationthatrequiresaspecificprivate
#key.
#
#org.teiid.ssl.keyStore=
#
#Thekeystorepassword(notrequired)
#
#org.teiid.ssl.keyStorePassword=
#
#Thekeyalias(notrequired,ifgivennamedcertificateisused)
#
#org.teiid.ssl.keyAlias=
#
#Thekeypassword(notrequired,usedifthekeypasswordisdifferentthanthekeystorepassword)
#
#org.teiid.ssl.keyPassword=
#
#Theclasspathorfilesystemlocationofthe
#truststore.
#
#Thispropertyisrequiredifperforming1-way
#authenticationthatrequirestrustnotprovided
#bythesystemdefaults.
#
#org.teiid.ssl.trustStore=
#
#Thetruststorepassword(notrequired)
#
#org.teiid.ssl.trustStorePassword=
#
#Thecipherprotocol,defaultstoTLSv3
#
org.teiid.ssl.protocol=TLSv1
#
#WhethertoallowanonymousSSL
#(theTLS_DH_anon_WITH_AES_128_CBC_SHAciphersuite)
#defaultstotrue
#
org.teiid.ssl.allowAnon=true
##Whethertoallowtrustallservercertificates#defaultstofalse#
#org.teiid.ssl.trustAll=false
ConnectingtoaTeiidServer
146
##Whethertocheckforexpiredservercertificates(noaffectinanonymousmodeorwithtrustAll=true)#defaultstofalse#
#org.teiid.ssl.checkExpired=false
1-waySSL
org.teiid.ssl.trustStore=<dir>/server.truststore(required)
2-waySSL
org.teiid.ssl.keyStore=<dir>/client.keystore(required)
org.teiid.ssl.trustStore=<dir>/server.truststore(required)
ConnectingtoaTeiidServer
147
AdditionalSocketClientSettingsA"teiid-client-settings.properties"filecanbeusedtoconfigureTeiidlowlevelandSSLsocketconnectionproperties.Currentlyonlyasinglepropertiesfileisexpectedperdriver/classloadercombination.Asample"teiid-client-settings.properties"filecanbefoundinsidethe"teiid-<version>-client.jar"fileattherootcalled"teiid-client-settings.orig.properties".Tocustomizethesettings,extractthisfile,makeacopy,changethepropertyvaluesaccordingly,andplacethisfileintheclientapplication’sclasspathbeforethe"teiid-<version>-client.jar"file.Typicallyclientswillnotneedtoadjustthenon-SSLproperties.Forreferencethepropertiesare:
########################################
#MiscSocketConfiguration
########################################
#
#Thetimeinmillisecondsforsockettimeouts.
#Timeoutsduringtheinitialization,handshake,or
#aserverpingmaybetreatedasanerror.
#
#Thisisthelowerboundforallothertimeouts
#theJDBClogintimeout.
#
#Typicallythisshouldbeleftatthedefaultof1000
#(1second).Settingthisvaluetoolowmaycauseread
#errors.
#
org.teiid.sockets.soTimeout=1000
#
#Themaxnumberofcachedserverinstances
#toreuse.Aserverinstanceisaconnected
#sockettoaparticularclustermemberwith
#clientencryptionandorSSLalreadyestablished.
#
#Cachinginstanceshelpsin2circumstances:
#-whenConnectionpoolingisnotbeingused.
#-load-balancingperformancetoacluster
#whenusingConnectionpoolingoftheDataSource.
#
#Thisvalueshouldtypicallybeamultipleofthe
#clustersize.
#
#Setto0todisableinstancecaching.
#
org.teiid.sockets.maxCachedInstances=16
#
#Setthemaxtimetolive(inmilliseconds)fornon-execution
#synchronouscalls.
#
org.teiid.sockets.synchronousttl=240000
#
#Setthesocketreceivebuffersize(inbytes)
#0indicatesthatthedefaultsocketsettingwillbeused.
#
org.teiid.sockets.receiveBufferSize=0
#
#Setthesocketsendbuffersize(inbytes)
ConnectingtoaTeiidServer
148
#0indicatesthatthedefaultsocketsettingwillbeused.
#
org.teiid.sockets.sendBufferSize=0
#
#SettotruetoenableNagle'salgorithmtoconservebandwidth
#byminimizingthenumberofsegmentsthataresent.
#
org.teiid.sockets.conserveBandwidth=false
#
#Maximumnumberofbytesperservermessage.
#Mayneedtobeincreasedwhenusingcustomtypesand/orlargebatchsizes.
#
org.teiid.sockets.maxObjectSize=33554432
#
#Settotruetodisableclientping.
#Defaultvalueis'false'thatmeansclientpingisenable.Theclientpingkeepstherelevantserverfromter
minatingsessions.
#Ifthedefaultloadbalancingstrategyisused,thenclientpingisnecessary.
#
org.teiid.sockets.DisablePing=false
Note Allpropertieslistedin"teiid-client-settings.properties"canalsobesetasSystemproperties.
ConnectingtoaTeiidServer
149
PreparedStatementsTeiidprovidesastandardimplementationofjava.sql.PreparedStatement.PreparedStatementscanbeveryimportantinspeedingupcommonstatementexecution,sincetheyallowtheservertoskipparsing,resolving,andplanningofthestatement.SeetheJavadocumentationformoreinformationonPreparedStatementusage.
PreparedStatementConsiderations
ItisnotnecessarytopoolclientsideTeiidPreparedStatements,sinceTeiidperformsplancachingontheserverside.
Thenumberofcachedplansisconfigurable(seetheAdminGuide),andarepurgedbytheleastrecentlyused(LRU).
Cachedplansarenotdistributedthroughacluster.Anewplanmustbecreatedforeachclustermember.
PlansarecachedfortheentireVDBorforjustaparticularsession.Thescopeofaplanisdetectedautomaticallybaseduponthefunctionsevaluatedduringit’splanningprocess.
StoredproceduresexecutedthroughaCallableStatementhavetheirplanscachedjustasaPreparedStatement.
Bindvariabletypesinfunctionsignatures,e.g."wheret.col=abs(?)"canbedeterminedifthefunctionhasonlyonesignatureorifthefunctionisusedinapredicatewherethereturntypecanbedetermined.Inmorecomplexsituationsitmaybenecessarytoaddatypehintwithacastorconvert,e.g.upper(convert(?,string)).
PreparedStatements
150
ResultSetLimitationsTYPE_SCROLL_SENSITIVEisnotsupported.
UPDATABLEResultSetsarenotsupported.
ReturningmultipleResultSetsfromProcedureexecutionisnotsupported.
ResultSetLimitations
151
JDBCExtensionsThesearecustomextensionstoJDBCAPIfromTeiidtosupportvariousfeatures.
JDBCExtensions
152
StatementExtensionsTheTeiidstatementextensioninterface,org.teiid.jdbc.TeiidStatement,providesfunctionalitybeyondtheJDBCstandard.Tousetheextensioninterface,simplycastorunwapthestatementreturnedbytheConnection.Thefollowingmethodsareprovidedontheextensioninterface:
Table1.ConnectionProperties
MethodName Description
getAnnotations
GetthequeryengineannotationsifthestatementwaslastexecutedwithSHOWPLANON/DEBUG.Eachorg.teiid.client.plan.Annotationcontainsadescription,acategory,aseverity,andpossiblyaresolutionofnotesrecordedduringqueryplanningthatcanbeusedtounderstandchoicesmadebythequeryplanner.
getDebugLogGetthedebuglogifthestatementwaslastexecutedwithSHOWPLANDEBUG.
getExecutionPropertyGetthecurrentvalueofanexecutionpropertyonthisstatementobject.
getPlanDescription
GetthequeryplandescriptionifthestatementwaslastexecutedwithSHOWPLANON/DEBUG.Theplanisatreemadeupoforg.teiid.client.plan.PlanNodeobjects.TypicallyPlanNode.toString()orPlanNode.toXml()willbeusedtoconverttheplanintoatextualform.
getRequestIdentifier
Getanidentifierforthelastcommandexecutedonthisstatement.Ifnocommandhasbeenexecutedyet,nullisreturned.
setExecutionProperty
Settheexecutionpropertyonthisstatement.SeetheExecutionPropertiessectionformoreinformation.ItisgenerallypreferabletousetheSETStatementunlesstheexecutionpropertyappliesonlytothestatementbeingexecuted.
setPayload
Setaper-commandpayloadtopasstotranslators.Currentlytheonlybuilt-inuseisforsendinghintsforOracledatasource.
JDBCExtensions
153
PartialResultsModeTheTeiidServersupportsa"partialresults"querymode.Thismodechangesthebehaviorofthequeryprocessorsotheserverreturnsresultsevenwhensomedatasourcesareunavailable.
Forexample,supposethattwodatasourcesexistfordifferentsuppliersandyourdataDesignershavecreatedavirtualgroupthatcreatesaunionbetweentheinformationfromthetwosuppliers.Ifyourapplicationsubmitsaquerywithoutusingpartialresultsquerymodeandoneofthesuppliers’databasesisdown,thequeryagainstthevirtualgroupreturnsanexception.However,ifyourapplicationrunsthesamequeryin"partialresults"querymode,theserverreturnsdatafromtherunningdatasourceandnodatafromthedatasourcethatisdown.
Whenusing"partialresults"mode,ifasourcethrowsanexceptionduringprocessingitdoesnotcausetheuser’squerytofail.Rather,thatsourceistreatedasreturningnomorerowsafterthefailurepoint.Mostcommonly,thatsourcewillreturn0rows.
ThisbehaviorismostusefulwhenusingUNIONorOUTERJOINqueriesastheseoperationshandlemissinginformationinausefulway.Mostotherkindsofquerieswillsimplyreturn0rowstotheuserwhenusedinpartialresultsmodeandthesourceisunavailable.
Foreachsourcethatisexcludedfromthequery,awarningwillbegenerateddescribingthesourceandthefailure.ThesewarningscanbeobtainedfromtheStatement.getWarnings()method.ThismethodreturnsaSQLWarningobjectbutinthecaseof"partialresults"warnings,thiswillbeanobjectoftypeorg.teiid.jdbc.PartialResultsWarningclass.Thisclasscanbeusedtoobtainalistofallthefailedsourcesbynameandtoobtainthespecificexceptionthrownbyeachresourceadaptor.
Note
SinceTeiidsupportscursoringbeforetheentireresultisformed,itispossiblethatadatasourcefailurewillnotbedetermineduntilafterthefirstbatchofresultshavebeenreturnedtotheclient.Thiscanhappeninthecaseofunions,butnotjoins.Toensurethatallwarningshavebeenaccumulated,thestatementshouldbecheckedaftertheentireresultsethasbeenread.
Note Ifotherwarningsarereturnedbyexecution,thenthepartialresultswarningsmayoccurafterthefirstwarninginthewarningchain.
PartialresultsmodeisoffbydefaultbutcanbeturnedonforallqueriesinaConnectionwitheithersetPartialResultsMode("true")onaDataSourceorpartialResultsMode=trueonaJDBCURL.Ineithercase,partialresultsmodemaybetoggledlaterwithaSETStatement.
SettingPartialResultsMode
Statementstatement=...obtainstatementfromConnection...
statement.execute("setpartialResultsModetrue");
GettingPartialResultsWarnings
statement.execute("setpartialResultsModetrue");
ResultSetresults=statement.executeQuery("SELECTNameFROMAccounts");
while(results.next()){
...//processtheresultset
}
SQLWarningwarning=statement.getWarnings();
while(warning!=null){
if(warninginstanceofPartialResultsWarning){
PartialResultsWarningpartialWarning=(PartialResultsWarning)warning;
CollectionfailedConnectors=partialWarning.getFailedConnectors();
Iteratoriter=failedConnectors.iterator();
while(iter.hasNext()){
StringconnectorName=(String)iter.next();
SQLExceptionconnectorException=partialWarning.getConnectorException(connectorName);
System.out.println(connectorName+":"+connectorException.getMessage());
}
JDBCExtensions
154
}
warning=warning.getNextWarning();
}
Warning
Insomeinstances,typicallyJDBCsources,thesourcenotbeinginitiallyavailablewillpreventTeiidfromautomaticallydeterminingtheappropriatesetofsourcecapabilities.Ifyougetanexceptionindicatingthatthecapabilitiesforanunavailablesourcearenotvalidinpartialresultsmode,thenitmaybenecessarytomanuallysetthedatabaseversionorsimilarpropertyonthetranslatortoensurethatthecapabilitiesareknownevenifthesourceisnotavailable.
JDBCExtensions
155
Non-blockingStatementExecutionJDBCqueryexecutioncanindefinitelyblockthecallingthreadwhenastatementisexecutedoraresultsetisbeingiterated.Insomesituationsyoumaynotwishtohaveyourcallingthreadsheldintheseblockedstates.Whenusingembedded/localconnections,youmayoptionallyusetheorg.teiid.jdbc.TeiidStatementandorg.teiid.jdbc.TeiidPreparedStatementinterfacestoexecutequerieswithacallbackorg.teiid.jdbc.StatementCallbackthatwillbenotifiedofstatementevents,suchasanavailablerow,anexception,orcompletion.Yourcallingthreadwillbefreetoperformotherwork.Thecallbackwillbeexecutedbyanengineprocessingthreadasneeded.Ifyourresultsprocessingisitselfblockingandyouwantqueryprocessingtobeconcurrentwithresultsprocessing,thenyourcallbackshouldimplementonRowhandlinginamulti-threadedmannertoallowtheenginethreadtocontinue.
Non-blockingPreparedStatementExecution
PreparedStatementstmt=c.prepareStatemen(sql);
TeiidPreparedStatementtStmt=stmt.unwrap(TeiidPreparedStatement.class);
tStmt.submitExecute(newStatementCallback(){
@Override
publicvoidonRow(Statements,ResultSetrs){
//anylogicthataccessesthecurrentrow...
System.out.println(rs.getString(1));
}
@Override
publicvoidonException(Statements,Exceptione)throwsException{
s.close();
}
@Override
publicvoidonComplete(Statements)throwsException{
s.close();
},newRequestOptions()
});
Thenon-blockinglogicislimitedtostatementexecutiononly.OtherJDBCoperations,suchasconnectioncreationorbatchedexecutionsdonotyethavenon-blockingoptions.
IfyouaccessforwardpositionsintheonRowmethod(callingnext,isLast,isAfterLast,absolute),theymaynotyetbevalidandaorg.teiid.jdbc.AsynchPositioningExceptionwillbethrown.ThatexceptionisrecoverableifcaughtorcanbeavoidedbycallingTeiidResultSet.available()todetermineifyourdesiredpositioningwillbevalid.
ContinuousExecution
TheRequestOptionsobjectmaybeusedtospecifyaspecialtypeofcontinuousasynchexecutionviathecontinuousorsetContinuousmethods.Incontinuousmodethestatementwillbecontinuouslyre-executed.Thisisintendedforconsumingreal-timeorotherdatastreamsprocessedthroughaSQLplan.Acontinuousquerywillonlyterminateonanerrororwhenthestatementisexplicitlyclosed.TheSQLforacontinuousqueryisnodifferentthananyotherstatement.Careshouldbetakentoensurethatretrievalsfromnon-continuoussourcesisappropriatelycachedforreuse,suchasbyusingmaterializedviewsorsessionscopedtemptables.
Acontinuousquerymustdothefollowing:
returnaresultset
beexecutedwithaforward-onlyresultset
cannotbeusedinthescopeofatransaction
JDBCExtensions
156
Sinceresourceconsumptionisexpectedtobedifferentinacontinuousplan,itdoesnotcountagainsttheservermaxactiveplanlimit.Typicallycustomsourceswillbeusedtoprovidedatastreams.SeetheDeveloper’sGuide,inparticularthesectiononReusableExecutionsformore.
Whentheclientwishestoendthecontinuousquery,theStatement.close()orStatement.cancel()methodshouldbecalled.Typicallyyourcallbackwillclosewheneveritnolongneedstoprocessresults.
SeealsotheContinuousStatementCallbackforuseastheStatementCallbackforadditionalmethodsrelatedtocontinuousprocessing.
JDBCExtensions
157
ResultSetExtensionsTheTeiidresultsetextensioninterface,org.teiid.jdbc.TeiidResultSet,providesfunctionalitybeyondtheJDBCstandard.Tousetheextensioninterface,simplycastorunwaparesultsetreturnedbyaTeiidstatement.Thefollowingmethodsareprovidedontheextensioninterface:
Table1.ConnectionProperties
MethodName Description
available
Returnsanestimateoftheminimumnumberofrowsthatcanberead(afterthecurrent)withoutblockingortheendoftheResultSetisreached.
JDBCExtensions
158
ConnectionExtensionsTeiidconnections(definedbytheorg.teiid.jdbc.TeiidConnectioninterface)supportthechangeUsermethodtoreauthenticateagivenconnection.Ifthereauthenticationissuccessfulthecurrentconnectionmybeusedwiththegivenidentity.Existingstatements/resultsetsarestillavailableforuseundertheoldidentity.SeetheJBossASissueJBAS-1429formoreonusingreauthenticationsupportwithJCA.
JDBCExtensions
159
UnsupportedJDBCMethodsBasedupontheJDBCinJDK1.6,thisappendixdetailsonlythoseJDBCmethodsthatTeiiddoesnotsupport.Unlessspecifiedbelow,TeiidsupportsallotherJDBCMethods.
ThosemethodslistedwithoutcommentsthrowaSQLExceptionstatingthatitisnotsupported.
Wherespecified,somelistedmethodsdonotthrowanexception,butpossiblyexhibitunexpectedbehavior.Ifnoargumentsarespecified,thenallrelated(overridden)methodsarenotsupported.Ifanargumentislistedthenonlythoseformsofthemethodspecifiedarenotsupported.
UnsupportedJDBCMethods
160
UnsupportedClassesandMethodsin"java.sql"
Classname Methods
Blob
[source,java]----getBinaryStream(long,long)-throwsSQLFeatureNotSupportedExceptionsetBinaryStream(long)--throwsSQLFeatureNotSupportedExceptionsetBytes--throwsSQLFeatureNotSupportedExceptiontruncate(long)-throwsSQLFeatureNotSupportedException----
CallableStatement
[source,java]----getObject(intparameterIndex,Map<String,Class<?>>map)-throwsSQLFeatureNotSupportedExceptiongetRef-throwsSQLFeatureNotSupportedExceptiongetRowId-throwsSQLFeatureNotSupportedExceptiongetURL(StringparameterName)-throwsSQLFeatureNotSupportedExceptionregisterOutParameter-ignoresregisterOutParameter(StringparameterName,*)-throwsSQLFeatureNotSupportedExceptionsetRowId(StringparameterName,RowIdx)-throwsSQLFeatureNotSupportedExceptionsetURL(StringparameterName,URLval)-throwsSQLFeatureNotSupportedException----
Clob
[source,java]----getCharacterStream(longarg0,longarg1)-throwsSQLFeatureNotSupportedExceptionsetAsciiStream(longarg0)-throwsSQLFeatureNotSupportedExceptionsetCharacterStream(longarg0)-throwsSQLFeatureNotSupportedExceptionsetString-throwsSQLFeatureNotSupportedExceptiontruncate-throwsSQLFeatureNotSupportedException----
Connection
[source,java]----createBlob-throwsSQLFeatureNotSupportedExceptioncreateClob-throwsSQLFeatureNotSupportedExceptioncreateNClob-throwsSQLFeatureNotSupportedExceptioncreateSQLXML-throwsSQLFeatureNotSupportedExceptioncreateStruct(StringtypeName,Object[]attributes)-throwsSQLFeatureNotSupportedExceptiongetClientInfo-throwsSQLFeatureNotSupportedExceptionreleaseSavepoint-throwsSQLFeatureNotSupportedExceptionrollback(Savepointsavepoint)-throwsSQLFeatureNotSupportedExceptionsetHoldability-throwsSQLFeatureNotSupportedExceptionsetSavepoint-throwsSQLFeatureNotSupportedExceptionsetTypeMap-throwsSQLFeatureNotSupportedExceptionsetRealOnly-effectivelyignored----
DatabaseMetaData
[source,java]----getAttributes-throwsSQLFeatureNotSupportedExceptiongetClientInfoProperties-throwsSQLFeatureNotSupportedExceptiongetRowIdLifetime-throwsSQLFeatureNotSupportedException----
NClob NotSupported
PreparedStatement
[source,java]----setRef-throwsSQLFeatureNotSupportedExceptionsetRowId-throwsSQLFeatureNotSupportedExceptionsetUnicodeStream-
UnsupportedJDBCMethods
161
throwsSQLFeatureNotSupportedException----
Ref NotImplemented
ResultSet
[source,java]----deleteRow-throwsSQLFeatureNotSupportedExceptiongetHoldability-throwsSQLFeatureNotSupportedExceptiongetObject(,Map<String,Class<?>>map)-throwsSQLFeatureNotSupportedExceptiongetRef-throwsSQLFeatureNotSupportedExceptiongetRowId-throwsSQLFeatureNotSupportedExceptiongetUnicodeStream-throwsSQLFeatureNotSupportedExceptiongetURL-throwsSQLFeatureNotSupportedExceptioninsertRow-throwsSQLFeatureNotSupportedExceptionmoveToInsertRow-throwsSQLFeatureNotSupportedExceptionrefreshRow-throwsSQLFeatureNotSupportedExceptionrowDeleted-throwsSQLFeatureNotSupportedExceptionrowInserted-throwsSQLFeatureNotSupportedExceptionrowUpdated-throwsSQLFeatureNotSupportedExceptionsetFetchDirection-throwsSQLFeatureNotSupportedExceptionupdate-throwsSQLFeatureNotSupportedException----
RowId NotSupported
Savepoint notSupported
SQLData NotSupported
SQLInput notSupported
SQLOutput NotSupported
UnsupportedJDBCMethods
162
UnsupportedClassesandMethodsin"javax.sql"
Classname Methods
RowSet* NotSupported
UnsupportedJDBCMethods
163
ODBCSupportOpenDatabaseConnectivity(ODBC)isastandarddatabaseaccessmethoddevelopedbytheSQLAccessgroupin1992.ODBC,justlikeJDBCinJava,allowsconsistentclientaccessregardlessofwhichdatabasemanagementsystem(DBMS)ishandlingthedata.ODBCusesadrivertotranslatetheapplication’sdataqueriesintocommandsthattheDBMSunderstands.Forthistowork,boththeapplicationandtheDBMSmustbeODBC-compliant–thatis,theapplicationmustbecapableofissuingODBCcommandsandtheDBMSmustbecapableofrespondingtothem.
TeiidcanprovideODBCaccesstodeployedVDBsintheTeiidruntimethroughPostgreSQL’sODBCdriver.ThisispossiblebecauseTeiidhasaPostgreSQLserveremulationlayeraccessibleviasocketclients.
Note Bydefault,ODBCisenabledandrunningononport35432.
Thepgemulationisnotcomplete.TheintentionoftheODBCaccessistoprovidenon-JDBCconnectivitytoissueTeiidqueries-notpgsqlqueries.WhilemanyPostgreSQLconstructsaresupportedthedefaultbehaviorforqueriesmatchesTeiid’sexpectations.SeeSystemPropertiesforoptionalpropertiesthatfurtheremulatepgsqlhandling.
Ifyoudonotwishtogloballyenableorg.teiid.backslashDefaultMatchEscape,thenhaveyourODBCclientissue"selectcast(teiid_session_set('backslashDefaultMatchEscape',true)asboolean)"toalterthedefaultbehaviorforjustthecurrentsession.
KnownLimitations:Updateablecursorsarenotsupported.Youwillreceiveparsingerrorscontainingthepgsystemcolumnctidifthisfeatureisnotdisabled.
LOsupportisnotavailable.LOBswillbereturnedasstringorbyteaasappropriateusingthetransportmaxlobsizesetting.
TheTeiidobjecttypewillmaptothePostgreSQLUNKNOWNtype,whichcannotbeserializedbytheODBClayer.Cast/Convertshouldbeusedtoprovideatypehintwhenappropriate-forexampleteiid_session_setreturnsanobjectvalue."SELECTteiid_session_set('x','y')"willfail,but"SELECTcast(teiid_session_set('x','y')asstring)"willsucceed.
Multi-dimensionalarraysarenotsupported.
InstallationBeforeanapplicationcanuseODBC,youmustfirstinstalltheODBCdriveronsamemachinethattheapplicationisrunningonandthencreateaDataSourceName(DSN)thatrepresentsaconnectionprofileforyourTeiidVDB.
Forawindowsclient,seetheWindowsInstallationGuide.
Configuration
Warning
BydefaultTeiidsupportsplaintextpasswordauthenticationforODBC.Iftheclient/serverarenotconfiguredtouseSSLorGSSauthentication,thepasswordwillbesentinplaintextoverthenetwork.IfyouneedsecurepasswordsintransitandarenotusingSSL,thenconsiderinstallingasecuritydomainthatwillacceptsafepasswordvaluesfromtheclient(forexampleencryptedorhashed).
SeetheSecurityGuidefordetailsonconfiguringSSLforandusingKerberoswiththepgtransport.
Forawindowsclient,seetheConfiguringtheDataSourceName.
SeealsoDSNLessConnection.
ODBCSupport
164
ConnectionSettings
Alltheavailablepgdriverconnectionoptionswiththeirdescriptionsthatcanbeusedaredefinedherehttp://psqlodbc.projects.pgfoundry.org/docs/config.html.Whenusingthesepropertiesontheconnectionstring,theirpropertynamesaredefinedherehttp://psqlodbc.projects.pgfoundry.org/docs/config-opt.html.
HoweverTeiiddoesnothonorallproperties,andsome,suchasUpdatableCursors,willcausequeryfailures.
Table1.PrimaryODBCSettingsForTeiid
Name Description
UpdateableCursors&RowVersioning Shouldnotbeused.
Useserversideprepare&ParseStatements&DisallowPremature
Itisrecommendedthat"Useserversideprepare"isenabledand"ParseStatements"/"DisallowPremature"aredisabled
SSLmode SeeSecurityGuide
UseDeclare/Fetchcursors&FetchMaxCount Shouldbeusedtobettermanageresourceswhenlargeresultsetsareused
Logging/debugsettingscanbeutilizedasneeded.
Settingsthatmanipulatedatatypes,metadata,oroptimizationssuchas"ShowSystemTables","Trueis-1","Backendgeneticoptimizer","ByteaasLongVarBinary","BoolsasChar",etc.areignoredbytheTeiidserverandhavenoclientsideeffect.Ifthereisaneedfortheseoranyothersettingstohaveadefinedaffect,pleaseopenanissuewiththeproduct/project.
Anyothersettingthatdoeshaveaclientsideaffect,suchas"LF<→CR/LFconversion",maybeusedifdesiredbutthereiscurrentlynoserversideusageofthesetting.
TeiidConnectionSettings
MostTeiidspecificconnectionpropertiesdonotmaptoODBCclientconnectionsettings.IfyoufindyourselfinthissituationandcannotusepostconnectionSETstatements,thentheVDBitselfmaytakedefaultconnectionpropertiesforODBC.UseVDBpropertiesoftheformconnection.XXXtocontrolthingslikepartialresultsmode,resultsetcaching,etc.
Theapplicationnamemaybesetbysomeclients.Ifnot,youmayuseaSETstatement-"SETapplication_namename"-tosetthenameevenaftertheconnectionismade.
ODBCSupport
165
InstallingtheODBCDriverClientAPostgreSQLODBCdriverneededtomaketheODBCconnectiontoTeiidisnotbundledwiththeTeiiddistribution.TheappropriatedriverneedsbedownloadeddirectlyfromthePostgreSQLwebsite.Wehavetestedwith8.04.200versionoftheODBCdriver.
MicrosoftWindows
1. DownloadtheODBC8.4driverfromPostgreSQLdownloadsite.Ifyouarelookingfor64-bitWindowsdriverdownloadthedriverfromhere.Laterversionsofthedrivermaybeused,butarenotconsideredfullysupported.
2. ExtractthecontentsoftheZIPfileintoatemporarylocationonyoursystem.Forexample:"c:\temp\pgodbc"
3. Doubleclickon"psqlodbc.msi"fileor(.exefileinthecaseof64bit)tostartinstallationofthedriver.
4. TheWizardappearsas
Click"Next".5.Thenextstepofthewizarddisplays.
InstallingtheODBCDriverClient
166
Carefullyreadit,andcheckthe"IacceptthetermsintheLicenseAgreement",ifyouareagreeingtothelicensingterms.Thenclick"Next".6.Thenextstepofthewizarddisplays.
Ifyouwanttoinstallinadifferentdirectorythanthedefaultthatisalreadyselected,clickthe"Browse"buttonandselectadirectory.Click"Next"tostartinstallingintheselecteddirectory.7.Thenextstepofthewizarddisplays.
InstallingtheODBCDriverClient
167
Thisstepsummarizesthechoicesyouhavemadeinthewizard.Reviewthisinformation.Ifyouneedtochangeanything,youcanusetheBackbuttontoreturntoprevioussteps.Click"Install"toproceed.8.1.Theinstallationwizardcopiesthenecessaryfilestothelocationyouspecified.Whenitfinishes,thefollowingscreendisplays.
Click"Finish"tocomplete.
Other*nixPlatformInstallations
InstallingtheODBCDriverClient
168
ForallotherplatformsotherthanMicrosoftWindows,theODBCdriverneedsbuiltfromthesourcefilesprovided.DownloadtheODBCdriversourcefilesfromthePostgreSQLdownloadsite.Untarthefilestoatemporarylocation.Forexample:"~/tmp/pgodbc".Buildandinstallthedriverbyrunningthecommandsbelow.
Note Youshouldusesuperuseraccountoruse"sudo"commandforrunningthe"makeinstall"command.
%tar-zxvfpsqlodbc-xx.xx.xxxx.tar.gz
%cdpsqlodbc-xx.xx.xxxx
%./configure
%make
%makeinstall
Some*nixdistributionsmayalreadyprovidebinaryformsoftheappropriatedriver,whichcanbeusedasanalternativetobuildingfromsource.
InstallingtheODBCDriverClient
169
ConfiguringtheDataSourceName(DSN)SeeTeiidsupportedoptionsforadescriptionoftheclientconfiguration.
WindowsInstallation
OnceyouhaveinstalledtheODBCDriverClientsoftwareonyourworkstation,youhavetoconfigureittoconnecttoaTeiidRuntime.NotethatthefollowinginstructionsarespecifictotheMicrosoftWindowsPlatform.
Todothis,youmusthaveloggedintotheworkstationwithadministrativerights,andyouneedtousetheControlPanel’sDataSources(ODBC)applettoaddanewdatasourcename.
EachdatasourcenameyouconfigurecanonlyaccessoneVDBwithinaTeiidSystem.TomakemorethanoneVDBavailable,youneedtoconfiguremorethanonedatasourcename.
Followthebelowstepsincreatingadatasourcename(DSN)
1. FromtheStartmenu,selectSettings>ControlPanel.
2. TheControlPaneldisplays.DoubleclickAdministrativeTools.
3. ThenDouble-clickDataSources(ODBC).
4. TheODBCDataSourceAdministratorappletdisplays.ClickthetabassociatedwiththetypeofDSNyouwanttoadd.
5. TheCreateNewDataSourcedialogboxdisplays.IntheSelectadriverforwhichyouwanttosetupadatasourcetable,selectPostgreSQLUnicode.
6. ClickFinish
7. ThePostgreSQLODBCDSNSetupdialogboxdisplays.
IntheDataSourceNameeditbox,typethenameyouwanttoassigntothisdatasource.IntheDatabaseeditbox,typethenameofthevirtualdatabaseyouwanttoaccessthroughthisdatasource.IntheServereditbox,typethehostnameorIPaddressofyourTeiidruntime.IfconnectingviaafirewallorNATaddress,thefirewalladdressorNATaddressshouldbeentered.InthePorteditbox,typetheportnumbertowhichtheTeiidSystemlistensforODBCrequests.Bydefault,TeiidlistenesforODBCrequestsonport35432IntheUserNameandPasswordeditboxes,supplytheusernameandpasswordfortheTeiidruntimeaccess.ProvideanydescriptionaboutthedatasourceintheDescriptionfield.
1. ClickontheDatasourcebutton,youwillseethisbelowfigure.Configureoptionsasshown.
ConfiguringtheDataSourceName(DSN)
170
Clickon"page2"andmakesuretheoptionsareselectedasshown
1. Click"save"andyoucanoptionallyclick"test"tovalidateyourconnectioniftheTeiidisrunning.YouhaveconfiguredaTeiid’svirtualdatabaseasadatasourceforyourODBCapplications.NowyoucanuseapplicationssuchasExcel,AccesstoquerythedataintheVDB
ConfiguringtheDataSourceName(DSN)
171
Other*nixPlatformInstallations
BeforeyoucanaccessTeiidusingODBConany*nixplatforms,youneedtoeitherinstallaODBCdrivermanagerorverifythatonealreadyexists.AstheODBCDrivermanagerTeiidrecommendsunixODBC.IfyouareworkingwithRedHatLinuxorFedorayoucancheckthegraphical"yum"installertosearch,findandinstallunixODBC.OtherwiseyoucandownloadtheunixODBCmanagerhere.Toinstall,simplyuntarthecontentsofthefiletoatemporarylocationandexecutethefollowingcommandsassuperuser.
./configure
make
makeinstall
CheckunixODBCwebsitesiteformoreinformation,ifyourunintoanyissuesduringtheinstallation.
Now,tooverifythatPostgreSQLdriverinstalledcorrectlyfromearlierstep,executethefollowingcommand
odbcinst-q-d
ThatshouldshowyoualltheODBCdriversinstalledinyoursystem.NowitistimetocreateaDSN.Edit"/etc/odbc.ini"fileandaddthefollowing
[<DSNname>]
Driver=/usr/lib/psqlodbc.so
Description=PostgreSQLDataSource
Servername=<TeiidHostnameorip>
Port=35432
Protocol=7.4-1
UserName=<user-name>
Password=<password>
Database=<vdb-name>
ReadOnly=no
ServerType=Postgres
ConnSettings=
UseServerSidePrepare=1
Debug=0
Fetch=10000
#enablebelowwhendealinglargeresultsetstoenablecursoring
#UseDeclareFetch=1
Notethatyouneed"sudo"permissionstoeditthe"/etc/odbc.ini"file.ForalltheavailableconfigurableoptionsthatyoucanuseindefiningaDSNcanbefoundhereonpostgreSQLODBCpage.
OnceyouaredonewithdefiningtheDSN,youcanverifyyourDSNusingthefollowingcommand
isql<DSN-name>[<user-name><password>]<commands.sql
where"commands.sql"filecontainstheSQLcommandsyouwouldliketoexecute.Youcanalsoomitthecommands.sqlfile,thenyouwillbeprovidedwithainteractiveshell.
Tip YoucanalsouselanguageslikePerl,Python,C/C++withODBCportstoPostgres,oriftheyhavedirectPostgresconnectionmodulesyoucanusethemtootoconnectTeiidandissuequeriesanretrieveresults.
ConfiguringtheDataSourceName(DSN)
172
ConfiguringtheDataSourceName(DSN)
173
DSNLessConnectionYoucanalsoconnecttoTeiidVDBusingODBCwithoutexplicitlycreatingaDSN.However,inthesescenariosyourapplicationneeds,whatiscalledas"DSNlessconnectionstring".Thebelowisasampleconnectionstring
ForWindows:
ODBC;DRIVER={PostgreSQLUnicode};DATABASE=<vdb-name>;SERVER=<host-name>;PORT=
<port>;Uid=<username>;Pwd=<password>;c4=0;c8=1;
For*nix:
ODBC;DRIVER={PostgreSQL};DATABASE=<vdb-name>;SERVER=<host-name>;PORT=<port>;Uid=
<username>;Pwd=<password>;c4=0;c8=1;
SeetheTeiidsupportedoptions.
DSNLessConnection
174
ConfiguringConnectionPropertieswithODBCWhenworkingwithODBCconnection,theusercansettheconnectionpropertiesDriverConnection#URLConnectionPropertiesthatareavailableinTeiidbyexecutingthecommandlikebelow.
SET<property-name>TO<property-value>
forexampletoturnontheresultsetcachingyoucanissue
SETresultSetCacheModeTO'true'
AnotheroptionistosetthisasVDBpropertyinvdb.xmlfileas
<vdbname="...">
<propertyname="connection.resultSetCacheMode"value="true"/>
...
</vdb>
ODBCConnectionProperties
175
ODataSupport
WhatisOData
TheOpenDataProtocol(OData)isaWebprotocolforqueryingandupdatingdatathatprovidesawaytounlockyourdataandfreeitfromsilosthatexistinapplicationstoday.ODatadoesthisbyapplyingandbuildinguponWebtechnologiessuchasHTTP,AtomPublishingProtocol(AtomPub)andJSONtoprovideaccesstoinformationfromavarietyofapplications,services,andstores.TheprotocolemergedfromexperiencesimplementingAtomPubclientsandserversinavarietyofproductsoverthepastseveralyears.ODataisusedtoexposeandaccessinformationfromavarietyofsourcesincluding,butnotlimitedto,relationaldatabases,filesystems,contentmanagementsystemsandtraditionalWebsites.
ODataisconsistentwiththewaytheWebworks-itmakesadeepcommitmenttoURIsforresourceidentificationandcommitstoanHTTP-based,uniforminterfaceforinteractingwiththoseresources(justliketheWeb).ThiscommitmenttocoreWebprinciplesallowsODatatoenableanewlevelofdataintegrationandinteroperabilityacrossabroadrangeofclients,servers,services,andtools.
copiedfromhttp://odata.org
TeiidSupportforOData
WhenausersuccessfullydeploysaVDBintoaTeiidServer,theODataprotocolsupportisimplicitlyprovidedbytheTeiidserverwithoutanyfurtherconfiguration.
ODatasupportiscurrentlynotavailableintheTeiidEmbeddedprofile.
ODatasupportisimplementedanddeployedthroughWARfile(s).Accessissimilartoaccessingtoanywebresourcesdeployedonthecontainer.Thewarfile(s)arelocatedat<containerroot>/modules/org/jboss/teiid/deployments/*.war.
TeiidprovidesODataVersion4.0support.LegacyODataVersion2.0supporthasbeenremoved,butcouldbemaintainedasit’sownproject-pleasecontactthecommunityifyoustillneedthisfeatureandwanttomaintainit.
ODataSupport
176
ODataVersion4.0SupportTeiidstrivestobecompliantwiththeODataspecification.TherestofthischapterhighlightsomespecificsofODataandTeiid’ssupport,butyoushouldalsoconsultthespecification.
HowtoAccessthedata?
Forexample,ifyouhaveavdbbynamenorthwinddeployedthathasacustomerstableinaNWmodel,thenyoucanaccessthattablewithanHTTPGETviatheURL:
http://localhost:8080/odata/northwind/NW/customers
thiswouldbeakintomakingaJDBC/ODBCconnectionandissuingtheSQL:
SELECT*FROMNW.customers
Note Usecorrectcase(upperorlower)intheresourcepath.UnlikeSQL,thenamesusedintheURIascase-sensitive.
ThereturnedresultsfromODataquerycanbeinAtom/AtomPubXMLorJSONformat.JSONresultsarereturnedbydefault.
QueryBasicsUserscansubmitpredicateswithalongtheirquerytofiltertheresults:
http://localhost:8080/odata/northwind/NW/customers?$filter=nameeq'bob'
NoteSpacesaround'eq'areforreadabilityoftheexampleonly;inrealURLstheymustbepercent-encodedas%20.ODatamandatespercentencodingforallspacesinURLs.http://docs.oasis-open.org/odata/odata/v4.0/odata-v4.0-part2-url-conventions.html
thiswouldbesimilartomakingaJDBC/ODBCconnectionandissuingtheSQL
SELECT*FROMNW.customerswherename='bob'
Torequesttheresulttobeformattedinaspecificformat,addthequeryoption$format
http://localhost:8080/odata/northwind/NW/customers?$format=JSON
Queryoptionscanbecombinedasneeded.Forexampleformatwithafilter:
http://localhost:8080/odata/northwind/NW/customers?$filter=nameeq'bob'&$format=xml
ODataallowsforqueryingnavigationsfromoneentitytoanother.Anavigationissimilartotheforeignkeyrelationshipsinrelationaldatabases.
Forexample,ifthecustomerstablehasanexportedkeytotheorderstableonthecustomersprimarykeycalledthecustomer_fk,thenanODataGETcouldbeissuedlike:
ODataVersion4.0Support
177
http://localhost:8080/odata/northwind/NW/customers(1234)/customer_fk?$filter=orderdategtdatetime'2012-12-31T2
1:23:38Z'
thiswouldbeakintomakingaJDBC/ODBCconnectionandissuingtheSQL:
SELECTo.*FROMNW.ordersojoinNW.customerscono.customer_id=c.idwherec.id=1234ando.orderdate>{ts'
2012-12-3121:23:38'}
NoteMoreComprehensiveDocumentationaboutODATA-Fordetailedprotocolaccessyoucanreadthespecificationathttp://odata.org.YoucanalsoreadthisveryusefulwebresourceforanexampleofaccessinganODataserver.
Notseeingalltherows?
Seetheconfigurationsectionbelowformoredetails.Generallybatchingisbeingutilized,whichtoolingshouldunderstandautomatically,andadditionalquerieswitha$skiptokenqueryoptionspecifiedareneeded:
http://localhost:8080/odata/northwind/NW/customers?$skiptoken=xxx
"EntitySetNotFound"error?
Whenyouissuetheabovequeryareyouseeingamessagesimilartobelow?
{"error":{"code":null,"message":"CannotfindEntitySet,Singleton,ActionImportorFunctionImportwithname'xx
x'."}}
Then,itmeansthateitheryousuppliedthemodel-name/table-namecombinationwrong,checkthespellingandcase.
Itispossiblethattheentityisnotpartofthemetadata,suchaswhenatabledoesnothaveanyPRIMARYKEYorUNIQUEKEY(s).
Howtoupdateyourdata?
UsingtheODataprotocolitispossibletoperformCREATE/UPDATE/DELETEoperationsalongwithREADoperationsshownabove.TheseoperationsusedifferentHTTPmethods.
INSERT/CREATEisaccomplishedthroughanHTTPmethod"POST".
ExamplePOST
POST/service.svc/CustomersHTTP/1.1
Host:host
Content-Type:application/json
Accept:application/json
{
"CustomerID":"AS123X",
"CompanyName":"ContosoWidgets",
"Address":{
"Street":"58ContosoSt",
"City":"Seattle"
}
}
AnUPDATEisperformedwithanHTTP"PUT".
ODataVersion4.0Support
178
ExamplePUTUpdateofCustomer
PUT/service.svc/Customers('ALFKI')HTTP/1.1
Host:host
Content-Type:application/josn
Accept:application/json
{
"CustomerID":"AS123X",
"CompanyName":"UpdatedCompanyName",
"Address":{
"Street":"UpdatedStreet"
}
}
TheDELETEoperationusestheHTTP"DELETE"method.
ExampleDelete
DELETE/service.svc/Customers('ALFKI')HTTP/1.1
Host:host
Content-Type:application/json
Accept:application/json
Security
BydefaultODataaccessissecuredusingHTTPBasicauthentication.TheuserwillbeauthenticatedagainstTeiid’sdefaultsecuritydomain"teiid-security".Usersareexpectedtohavetheodatarole.Besuretocreateuserwiththisrolewhenyouareusingadd-user.shscripttocreateanewuser.
However,ifyouwishtochangethesecuritydomainuseadeployment-overlaytooverridetheweb.xmlfileintheodata4fileinthe<modules>/org/jboss/teiid/main/deploymentsdirectory.
ODataWARcanalsosupportKerberos,SAMLandOAuth2authentications,forconfiguringthethesesecurityschemespleaseseeSecurityGuide
Configuration
TheODataWARfilecanbeconfiguredwithfollowingpropertiesintheweb.xmlfile.
PropertyName Description DefaultValue
batch-size Numberofrowstosendbackeachtime,-1returnsallrows 256
skiptoken-cache-timeTimeintervalbetweentheresultsbeingrecycled/expiredbetween$skiptokenrequests
300000
invalid-xml10-character-replacement XML1.0replacementcharacterfornonUTF-8characters.
local-transport-name TeiidLocaltransportnameforconnection odata
ReplacementstringifaninvalidXML1.0characterappearsinthedata-notethatthisreplacementwilloccurevenifJSONisrequested.No
ODataVersion4.0Support
179
value(thedefault)meansthatanexceptionwillbethrownwithXMLresultsifsuchacharacterisencountered.
proxy-base-uri Definestheproxyserver’sURItobeusedinODataresponses. n/a
connection.XXX
SetsXXXasanexecutionpropertyonthelocalconnection.Canbeusedforexampletoenableresultsetcachemode.
n/a
Note
"BehindProxyorInCloudEnvironments?"-IftheTeiidserverisconfiguredbehindaproxyserverordeployedincloudenvironment,orusingaload-balancerthentheURIoftheserverwhichishandlingtheODatarequestisdifferentfromURIofproxy.TogeneratevalidlinksintheODataresponsesconfigure"proxy-base-uri"propertyintheweb.xml.Ifthisvalueisavailableassystempropertythendefinethepropertyvaluelikebelow
<init-param>
<param-name>proxy-base-uri</param-name>
<param-value>${system-property-name}</param-value>
</init-param>
Tomodifytheweb.xml,createadeployment-overlayusingthecliwiththemodifiedcontents:
deployment-overlayadd--name=myOverlay--content=/WEB-INF/web.xml=/modified/web.xml--deployments=teiid-odata-
odata4.war--redeploy-affected
TeiidODataserver,implementscursoringlogicwhentheresultrowsexceedtheconfiguredbatchsize.Oneveryrequest,onlybatch-sizenumberofrowsarereturned.Eachsuchrequestisconsideredanactivecursor,withaspecifiedamountofidletimespecifiedbyskip-token-cache-time.Afterthecursoristimedout,thecursorwillbeclosedandremainingresultswillbecleanedup,andwillnolongerbeavailableforfurtherqueries.Sincethereisnosessionbasedtrackingofthesecursors,iftherequestforskiptokencomesaftertheexpiredtime,theoriginalquerywillbeexecutedagainandtriestorepositionthecursortorelativeabsolutepotion,howevertheresultsarenotguaranteedtobesameastheunderlyingsourcesmayhavebeenupdatedwithnewinformationmeanwhile.
Limitations
Thefollowingfeaturelimitationscurrentlyapply.
searchisnotsupported
deltaprocessingisnotsupported
data-aggregationextensiontospecificationisnotsupported.
$itusageislimitedtoonlyprimitivecollectionproperties
ClientToolsforAccessODataaccessisreallywheretheusercomesin,dependinguponyourprogrammingmodelandneedstherearevariouswaysyouwriteyouraccesslayerintoOData.Thefollowingaresomesuggestions:
YourBrowser:TheODataExplorerisanonlinetoolforbrowsinganODatadataservice.
ODataVersion4.0Support
180
Olingo:IsaJavaframeworkthatsupportsODataV4,hasbothconsumerandproducerframework.
Microsofthasvarious.Netbasedlibraries,seehttp://odata.github.io/
WindowsDesktop:LINQPadisawonderfultoolforbuildingODataqueriesinteractively.Seehttps://www.linqpad.net/
ShellScripts:useCURLtool
Forlatestinformationotherframeworksandtoolsavailablepleaseseehttp://www.odata.org/ecosystem/
ODataMetadata(HowTeiidinterpretstherelationalschemaintoOData’s$metadata)ODatadefinesitsschemausingConceptualSchemaDefinitionLanguage(CSDL).EveryVDB,thatisdeployedinanACTIVEstateinTeiidserverexposesitsmetadatainCSDLformat.Forexampleifyouwantretrievemetadataforyourvdbnorthwind,youneedtoissueaquerylike
http://localhost:8080/odata/northwind/NW/$metadata
SinceODataschemamodelisnotarelationalschemamodel,TeiidusesthefollowingsemanticstomapitsrelationalschemamodeltoODataschemamodel.
RelationalEntity MappedODataEntity
ModelName SchemaNamespace,EntityContainerName
Table/View EntityType,EntitySet
TableColumns EntityType’sProperties
PrimaryKey EntityType’sKeyProperties
ForeignKey NavigationPropertyonEntityType
Procedure FunctionImport,ActionImport
Procedure’sTableReturn ComplexType
Teiidbydesigndoesnotdefineany"embedded"ComplexTypeintheEntityType.
SinceODataaccessismorekeybased,itis*MANDATORY*thateverytableTeiidexposesthroughODatamusthaveaPKoratleastoneUNIQUEkey.Atablewhichdoesnoteitherofthesewillbedropeedoutofthe$metadata
ODataVersion4.0Support
181
UsingTeiidwithHibernate
Configuration
Forthemostpart,interactingwithTeiidVDBs(VirtualDatabases)throughHibernateisnodifferentfromworkingwithanyothertypeofdatasource.First,dependingonwhereyourHibernateapplicationwillreside,eitherinthesameVMastheTeiidRuntimeoronaseparateVM,willdeterminewhichjar’sareused.
RunninginsameVMintheWildFlyserver,thentheteiid-client-{version}.jarandteiid-hibernate-dialect-{version}.jaralreadyresidein<jboss-install>/modules/org/jboss/teiid/client
RunningseparateVM’s,youneedtheTeiidJDBCDriverJARandTeiid’sHibernateDialectJARintheHibernate’sclasspath.TheHibernateJARcanbefoundin<jboss-install>/modules/org/jboss/teiid/client,teiid-hibernate-dialect-{version}.jarandtheTeiidJDBCDriverJARneedstobedownloaded.
TheseJARfileshavetheorg.teiid.dialect.TeiidDialectandorg.teiid.jdbc.TeiidDriverandorg.teiid.jdbc.TeiidDataSourceclasses.
YouconfigureHibernate(viahibernate.cfg.xml)asfollows:
SpecifytheTeiiddriverclassintheconnection.driver_classproperty:
<propertyname="connection.driver_class">
org.teiid.jdbc.TeiidDriver
</property>
SpecifytheURLfortheVDBintheconnection.urlproperty(replacingtermsinanglebracketswiththeappropriatevalues):
<propertyname="connection.url">
jdbc:teiid:<vdb-name>@mm://<host>:<port>;user=<user-name>;password=<password>
</property>
Tip BesuretouseaLocalJDBCConnectionifHibernateisinthesameVMastheapplicationserver.
SpecifytheTeiiddialectclassinthedialectproperty:
<propertyname="dialect">
org.teiid.dialect.TeiidDialect
</property>
Alternatively,ifyouputyourconnectionpropertiesinhibernate.propertiesinsteadofhibernate.cfg.xml,theywouldlooklikethis:
hibernate.connection.driver_class=org.teiid.jdbc.TeiidDriver
hibernate.connection.url=jdbc:teiid:<vdb-name>@mm://<host>:<port>
hibernate.connection.username=<user-name>
hibernate.connection.password=<password>
hibernate.dialect=org.teiid.dialect.TeiidDialect
NotealsothatsinceyourVDBswilllikelycontainmultiplesourceandviewmodelswithidenticaltablenames,youwillneedtofullyqualifytablenamesspecifiedinHibernatemappingfiles:
UsingTeiidwithHibernate
182
<classname="<Classname>"table="<Source/viewmodelname>.[<schemaname>.]<Tablename>">
...
</class>
ExampleMapping
<classname="org.teiid.example.Publisher"table="BOOKS.BOOKS.PUBLISHERS">
...
</class>
IdentifierGenerationIdentifiergenerationbasedupontablevalues,suchasthehilogenerator,requirethattheidentifiertable(s)beexposedthroughTeiid.
GUIDandIdentity(usinggeneratedkeyretrieval)identifiergenerationstrategyaredirectlysupported.
Limitations
ManyHibernateusecasesassumeadatasourcehastheability(withproperuserpermissions)toprocessDataDefinitionLanguage(DDL)statementslikeCREATETABLEandDROPTABLEaswellasDataManipulationLanguage(DML)statementslikeSELECT,UPDATE,INSERTandDELETE.TeiidcanhandleabroadrangeofDML,butdoesnotdirectlysupportDDLagainstaparticularsource.
Sequencegenerationisnotdirectlysupported.
UsingTeiidwithHibernate
183
UsingTeiidwithEclipseLink
Overview
WecanuseTeiidwithHibernate,wealsohaveaquickstartshowhowHibernateontopofTeiid.BothHibernateandEclipselinkarefullysupportJSR-317(JPA2.0),primarypurposeofthisdocumentisdemonstratehowuseTeiidwithEclipseLink.
Configuration
Forthemostpart,interactingwithTeiidVDBs(VirtualDatabases)throughEclipselinkisnodifferentfromworkingwithanyothertypeofdatasource.First,dependingonwhereyourEclipselinkapplicationwillreside,eitherinthesameVMastheTeiidRuntimeoronaseparateVM,willdeterminewhichjar’sareused.
RunninginsameVMintheWildFlyserver,theteiid-client-{version}.jarandteiid-eclipselink-platform-{version}.jarareneeded
RunningseparateVM’s,youneedtheTeiidJDBCDriverJAR(DownloadTeiidJDBCDriverJAR)andTeiid’sEclipselinkPlatformJAR(teiid-eclipselink-platform{version}.jar)intheEclipselink’sclasspath.
TheseJARfileshavetheorg.teiid.eclipselin.platform.TeiidPlatformandorg.teiid.jdbc.TeiidDriverclasses.
YouconfigureEclipseLink(viapersistence.xml)asfollows:
SpecifytheTeiiddriverclass,connectionurl
<propertyname="javax.persistence.jdbc.driver"value="org.teiid.jdbc.TeiidDriver"/>
<propertyname="javax.persistence.jdbc.url"value="jdbc:teiid:<vdb-name>@mm://<host>:<port>"/>
<propertyname="javax.persistence.jdbc.user"value="<username>"/>
<propertyname="javax.persistence.jdbc.password"value="<password>"/>
SpecifytheTeiidplatformclass
<propertyname="eclipselink.target-database"value="org.teiid.eclipselink.platform.TeiidPlatform"/>
Limitations
ManyEclipselinkusecasesassumeadatasourcehastheability(withproperuserpermissions)toprocessDataDefinitionLanguage(DDL)statementslikeCREATETABLEandDROPTABLEaswellasDataManipulationLanguage(DML)statementslikeSELECT,UPDATE,INSERTandDELETE.TeiidcanhandleabroadrangeofDML,butdoesnotdirectlysupportDDLagainstaparticularsource.
Sequencegenerationisnotdirectlysupported.
UsingTeiidwithEclipseLink
184
GeoServerIntegrationGeoServerisanopensourceserverforgeospatialdata.ItcanbeintegratedwithTeiidtoservegeospatialdatafromavarietyofsources.
Prerequisites
HaveGeoServerinstalled.BydefaultthiswillbeinadifferentcontainerthantheTeiidWildFlyinstance,butitshouldbepossibletodeployintothesameWildFlyinstance.
YourTeiidinstallationshouldalreadybesetupforODBCaccess.Thisallowsthebuilt-insupportofGeoServerforPostGIS/PostgreSQLtobeused.
HaveaVDBdeployedthatexposesoneormoretablescontaininganappropriateGeometrycolumn.
a. TheTeiidsystemtableGEOMETY_COLUMNSwillbeusedbyGeoServer.Pleaseensurethattherelevantgeometrycolumnshavetheappropriatesridandcoord_dimensions,whichmayrequiresettingthe{http://www.teiid.org/translator/spatial/2015}sridand{http://www.teiid.org/translator/spatial/2015}coord_dimensionextensionpropertyonthegeometrycolumn.
GeoServerConfiguration
ThisprocesswillneedtoberepeatedforeachVDBschemayouareexposingthatcontainsgeospatialdata.
1. UsingtheGeoServeradminwebapplication,selectStores→AddnewStore.UnderVectorDataSources,selectPostGIS.
2. Usingthenon-JNDIconnection,fillintheTeiidserverhost,ODBCport,database(VDBNamewithoptionalversion),user,andpassword,schema(schema/modelfromthetargetVDB).
i. IfyourVDBscontaintargetschemaortablenameswith%or_,Teiidmustbeconfiguredtousethesamedefaultlikeescapecharacter'\'asPostgreSQLtoproperlyrespondtometadataqueries.Eitherthesystempropertyorg.teiid.backslashDefaultMatchEscapemustbesettotrueortheTeiidsessionvariablebackslashDefaultMatchEscapemustbesettotrue-forexampleenter"selectcast(teiid_session_set('backslashDefaultMatchEscape',true)asboolean)"inthe"SessionstartupSQL"toconfigurejustthisGeoServerconnectionpool.
3. FollowthetypicalGeoServerinstructionsforcreatingaLayerbasedupontheTeiidstore.
i. NotethatthePostGISfunctionST_Estimated_ExtentisnotsupportedbyTeiidandtheexecutionwillbeshowninthelogsasanerrorwhenselectingtocomputetheboundingboxfromthedata.
AdditionalConsiderations
IfyouareintegratingaPostgreSQLsource,youmustnotre-exposethegeometry_columnstable.ThisisbecauseGeoServermakesunqualifiedqueriesthatreferencegeometry_columnsandthequeryshouldresolveagainsttheTeiidsystemtableinstead.
GeoServerIntegration
185
QGISIntegrationQGISisanopensourcegeospatialplatform.ItcanbeintegratedwithTeiidtoservegeospatialdatafromavarietyofsources.
Prerequisites
HaveQGISinstalled.Teiidintegrationwaslasttestedwithversion2.14.
YourTeiidinstallationshouldalreadybesetupforODBCaccess.Thisallowsthebuilt-insupportofQGISforPostGIS/PostgreSQLtobeused.
HaveaVDBdeployedthatexposesoneormoretablescontaininganappropriateGeometrycolumn.
a. TheTeiidsystemtableGEOMETY_COLUMNSwillbeusedbyQGIS.Pleaseensurethattherelevantgeometrycolumnshavetheappropriatesridandcoord_dimensions,whichmayrequiresettingthe{http://www.teiid.org/translator/spatial/2015}sridand{http://www.teiid.org/translator/spatial/2015}coord_dimensionextensionpropertyonthegeometrycolumn.
QGISConfiguration
ThisprocesswillneedtoberepeatedforeachVDBschemayouareexposingthatcontainsgeospatialdata.
1. IntheQGISGUIbrowserpanelrightclickonPostGISandselect"NewConnection".
2. FillintheTeiidserverhost,ODBCport,database(VDBNamewithoptionalversion),user,andpassword.
i. IfyourVDBscontaintargetschemaortablenameswith%or_,Teiidmustbeconfiguredtousethesamedefaultlikeescapecharacter'\'asPostgreSQLtoproperlyrespondtometadataqueries.Eitherthesystempropertyorg.teiid.backslashDefaultMatchEscapemustbesettotrue.
3. FollowthetypicalQGISinstructionsforcreatingaLayerbybrowsingtotheappropriateschemaandselectingatablethatexposesageometry.
AdditionalConsiderationsIfyouareintegratingaPostgreSQLsource,youmustnotre-exposethepostgressystemtablesincludingthePostGISgeometry_columnsorgeography_columnstables.ThisisbecauseQGISmakesunqualifiedreferencestothesetables,whichmaythenbeambiguous.
Operationsinvolvingcreatingordeletingschemasortableswillnotwork.
Thelogsmaycontainmessagesrelatedtoinformation_schema.tables-thisistodetermineiftheqgis_editor_widget_stylestableexists.Thatiscurrentlynotsupported.
QGISIntegration
186
ReauthenticationTeiidallowsforconnectionstobereauthenticatedsothattheidentityontheconnectioncanbechangedratherthancreatingawholenewconnection.IfusingJDBC,seethechangeUserConnectionextension.IfusingODBC,orsimplyneedastatementbasedmechanismforreauthentication,seealsotheSETStatementforSESSIONAUTHORIZATION.
Reauthentication
187
ExecutionPropertiesExecutionpropertiesmaybesetonaperstatementbasisthroughtheTeiidStatementinterfaceorontheconnectionviatheSETStatement.Forconvenience,thepropertykeysaredefinedbyconstantsontheorg.teiid.jdbc.ExecutionPropertiesinterface.
Table1.ExecutionProperties
PropertyName/StringConstant Description
PROP_TXN_AUTO_WRAP/autoCommitTxn Sameastheconnectionproperty.
PROP_PARTIAL_RESULTS_MODE/partialResultsMode SeethePartialResultsMode
PROP_XML_FORMAT/XMLFormat
DeterminestheformattingofXMLdocumentsreturnedbyXMLdocumentmodels.SeetheXMLextensions#Documentformattingsection.
PROP_XML_VALIDATION/XMLValidation
DetermineswhetherXMLdocumentsreturnedbyXMLdocumentmodelswillbevalidatedagainsttheirschemaafterprocessing.SeetheReferenceGuide’s"XMLSELECTCommand"chapterand"documentvalidation"section.
RESULT_SET_CACHE_MODE/resultSetCacheMode Sameastheconnectionproperty.
SQL_OPTION_SHOWPLAN/SHOWPLAN Sameastheconnectionproperty.
NOEXEC/NOEXEC Sameastheconnectionproperty.
JDBC4COLUMNNAMEANDLABELSEMANTICS/
useJDBC4ColumnNameAndLabelSemanticsSameastheconnectionproperty.
ExecutionProperties
188
XMLextensionsTheXMLextensionsapplyontoXMLresutlsfromqueriestoXMLdocumentmodels,andnottoXMLproducedbySQL/XMLorreadfromsomeothersource.
Documentformatting
ThePROP_XML_FORMATexecutionpropertycanbesettomodifythewaythatXMLdocumentsareformattedfromXMLdocumentmodels.ValidvaluesfortheconstantaredefinedinthesameExecutionPropertiesinterface:
1. XML_TREE_FORMAT-ReturnsaversionoftheXMLformattedfordisplay.TheXMLwilluselinebreaksandtabsasappropriatetoformattheXMLasatree.Thisformatisslowerduetotheformattingtimeandthelargerdocumentsize.
2. XML_COMPACT_FORMAT-ReturnsaversionoftheXMLformattedforoptimalperformance.TheXMLisasinglelongstringwithoutanyunnecessarywhitespace.
3. NotSet-Ifnoformatisset,theformattingflagontheXMLdocumentintheoriginalmodelishonored.Thismayproduceeitherthe"tree"or"compact"formofthedocumentdependingonthedocumentsetting.
Schemavalidation
ThePROP_XML_VALIDATIONexecutionpropertycanbesettoindicatethattheservershouldvalidateXMLdocumentmodeldocumentsagainsttheirschemabeforereturningthemtotheclient.Ifschemavalidationison,thentheserversendaSQLWarningifthedocumentdoesnotconformtotheschemaitisassociatedwith.UsingschemavalidationwillreducetheperformanceofyourXMLqueries.
XMLextensions
189
SETStatementExecutionpropertiesmayalsobesetontheconnectionbyusingtheSETstatement.TheSETstatementisnotyetalanguagefeatureofTeiidandishandledonlyintheJDBCclient.
SETSyntax:
SET[PAYLOAD](parameter|SESSIONAUTHORIZATION)value
SETSESSIONCHARACTERISTICSASTRANSACTIONISOLATIONLEVEL(READUNCOMMITTED|READCOMMITTED|REPEATABLEREAD|SERIALIZABLE)
SyntaxRules:
Theparametermustbeanidentifier-itcancontainspacesorotherspecialcharactersonlyifquoted.
Thevaluemaybeeitheranon-quotedidentifieroraquotedstringliteralvalue.
Ifpayloadisspecified,e.g."SETPAYLOADxy",thenasessionscopedpayloadpropertiesobjectwillhavethecorrespondingnamevaluepairset.Thepayloadobjectisnotfullysessionscoped.ItwillberemovedfromthesessionwhentheXAConnectionhandleisclosed/returnedtothepool(assumestheuseofTeiidDataSource).ThesessionscopedpayloadissupersededbytheusageofTeiidStatement.setPayload.
UsingSETSESSIONCHARACTERISTICSASTRANSACTIONISOLATIONLEVELisequivalenttocallingConnection.setTransactionIsolationwiththecorrespondinglevel.
TheSETstatementismostcommonlyusedtocontrolplanningandexecution.
SETSHOWPLAN(ON|DEBUG|OFF)
SETNOEXEC(ON|OFF)
EnablingPlanDebug
Statements=connection.createStatement();
s.execute("SETSHOWPLANDEBUG");
...
Statements1=connection.createStatement();
ResultSetrs=s1.executeQuery("selectcolfromtable");
ResultSetplanRs=s1.exeuteQuery("SHOWPLAN");
planRs.next();
StringdebugLog=planRs.getString("DEBUG_LOG");
QueryPlanwithoutexecutingthequery
s.execute("SETNOEXECON");
s.execute("SETSHOWPLANDEBUG");
...
e.execute("SETNOEXECOFF");
TheSETstatementmayalsobeusedtocontrolauthorization.ASETSESSIONAUTHORIZATIONstatementwillperformaReauthenticationgiventhecredentialscurrentlysetontheconnection.TheconnectioncredentialsmaybechangedbyissuingaSETPASSWORDstatement.ASETPASSWORDstatementdoesnotperformareauthentication.
ChangingSessionAuthorization
Statements=connection.createStatement();
s.execute("SETPASSWORD'someval'");
s.execute("SETSESSIONAUTHORIZATION'newuser'");
SETStatement
190
SETStatement
191
SHOWStatementTheSHOWstatementcanbeusedtoseeavariteyofinformation.TheSHOWstatementisnotyetalanguagefeatureofTeiidandishandledonlyintheJDBCclient.
SHOWUsage:
SHOWPLAN-returnsaresultsetwithaclobcolumnPLAN_TEXT,anxmlcolumnPLAN_XML,andaclobcolumnDEBUG_LOGwitharowcontainingthevaluesfromthepreviouslyexecutedquery.IfSHOWPLANisOFFornoplanisavailable,norowsarereturned.IfSHOWPLANisnotsettoDEBUG,thenDEBUG_LOGwillreturnanullvalue.
SHOWANNOTATIONS-returnsaresultsetwithstringcolumnsCATEGORY,PRIORITY,ANNOTATION,RESOLUTIONandarowforeachannotationonthepreviouslyexecutedquery.IfSHOWPLANisOFFornoplanisavailable,norowsarereturned.
SHOW<property>-theinverseofSET,showsthepropertyvalueforthegivenproperty,returnsaresultsetwithasinglestringcolumnwithanamematchingthepropertykey.
SHOWALL-returnsaresultsetwithaNAMEstringcolumnandaVALUEstringcolumnwitharowentryforeverypropertyvalue.TheSHOWstatementismostcommonlyusedtoretrievethequeryplan,seetheplandebugexample.
SHOWStatement
192
TransactionsTeiidsupportsthreetypesoftransactionsfromaclientperspective:
1. Global
2. Local
3. RequestLevel
AllareimplementedbytheTeiidServerasXAtransactions.SeetheJTAspecificationformoreonXATransactions.
Transactions
193
LocalTransactionsALocaltransactionfromaclientperspectiveaffectsonlyasingleresource,butcancoordinatemultiplestatements.
JDBCSpecific
TheConnectionclassusestheautoCommitflagtoexplicitlycontrollocaltransactions.Bydefault,autoCommitissettotrue,whichindicatesrequestlevelorimplicittransactioncontrol.
AnexampleofhowtouselocaltransactionsbysettingtheautoCommitflagtofalse.
LocaltransactioncontrolusingautoCommit
//Setautocommittofalseandstartatransaction
connection.setAutoCommit(false);
try{
//Executemultipleupdates
Statementstatement=connection.createStatement();
statement.executeUpdate("INSERTINTOAccounts(ID,Name)VALUES(10,'Mike')");
statement.executeUpdate("INSERTINTOAccounts(ID,Name)VALUES(15,'John')");
statement.close();
//Committhetransaction
connection.commit();
}catch(SQLExceptione){
//Ifanerroroccurs,rollbackthetransaction
connection.rollback();
}
Thisexampledemonstratesseveralthings:
1. SettingautoCommitflagtofalse.Thiswillstartatransactionboundtotheconnection.
2. Executingmultipleupdateswithinthecontextofthetransaction.
3. Whenthestatementsarecomplete,thetransactioniscommittedbycallingcommit().
4. Ifanerroroccurs,thetransactionisrolledbackusingtherollback()method.
Anyofthefollowingoperationswillendalocaltransaction:
1. Connection.setAutoCommit(true)–ifpreviouslysettofalse
2. Connection.commit()
3. Connection.rollback()
4. Atransactionwillberolledbackautomaticallyifittimesout.
TurningOffJDBCLocalTransactionControls
Insomecases,toolsorframeworksaboveTeiidwillcallsetAutoCommit(false),commit()androllback()evenwhenallaccessisread-onlyandnotransactionsarenecessary.InthescopeofalocaltransactionTeiidwillstartandattempttocommitanXAtransaction,possiblycomplicatingconfigurationorcausingperformancedegradation.
Inthesecases,youcanoverridethedefaultJDBCbehaviortoindicatethatthesemethodsshouldperformnoactionregardlessofthecommandsbeingexecuted.Toturnofftheuseoflocaltransactions,addthispropertytotheJDBCconnectionURL
LocalTransactions
194
disableLocalTxn=true
TipTurningofflocaltransactionscanbedangerousandcanresultininconsistentresults(ifreadingdata)orinconsistentdataindatastores(ifwritingdata).Forsafety,thismodeshouldbeusedonlyifyouarecertainthatthecallingapplicationdoesnotneedlocaltransactions.
TransactionStatements
Transactioncontrolstatements,whicharealsoapplicabletoODBCclients,explicitlycontrolthelocaltransactionboundaries.Therelevantstatementsare:
STARTTRANSACTION-synonymforconnection.setAutoCommit(false)
COMMIT-synonymforconnection.setAutoCommit(true)
ROLLBACK-synonymforconnection.rollback()andreturningtoautocommitmode.
LocalTransactions
195
RequestLevelTransactionsRequestleveltransactionsareusedwhentherequestisnotinthescopeofaglobalorlocaltransaction,whichimplies"autoCommit"is"true".Inarequestleveltransaction,yourapplicationdoesnotneedtoexplicitlycallcommitorrollback,rathereverycommandisassumedtobeitsowntransactionthatwillautomaticallybecommittedorrolledbackbytheserver.
TheTeiidServercanperformupdatesthroughvirtualtables.Theseupdatesmightresultinanupdateagainstmultiplephysicalsystems,eventhoughtheapplicationissuestheupdatecommandagainstasinglevirtualtable.Often,ausermightnotknowwhetherthequeriedtablesactuallyupdatemultiplesourcesandrequireatransaction.
Forthatreason,theTeiidServerallowsyourapplicationtoautomaticallywrapcommandsintransactionswhennecessary.Becausethiswrappingincursaperformancepenaltyforyourqueries,youcanchoosefromanumberofavailablewrappingmodestosuityourenvironment.Youneedtochoosebetweenthehighestdegreeofintegrityandperformanceyourapplicationneeds.Forexample,ifyourdatasourcesarenottransaction-compliant,youmightturnthetransactionwrappingoff(completely)tomaximizeperformance.
Youcansetyourtransactionwrappingtooneofthefollowingmodes:
1. ON:Thismodealwayswrapseverycommandinatransactionwithoutcheckingwhetheritisrequired.Thisisthesafestmode.
2. OFF:Thismodeneverautomaticallywrapsacommandinatransactionorcheckwhetheritneedstowrapacommand.Thismodecanbedangerousasitwillallowmultiplesourceupdatesoutsideofatransactionwithoutanerror.Thismodehasbestperformanceforapplicationsthatdonotuseupdatesortransactions.
3. DETECT:Thismodeassumesthattheuserdoesnotknowtoexecutemultiplesourceupdatesinatransaction.TheTeiidServercheckseverycommandtoseewhetheritisamultiplesourceupdateandwrapsitinatransaction.Ifitissinglesourcethenusesthesourcelevelcommandtransaction.YoucansetthetransactionmodeasapropertywhenyouestablishtheConnectionoronaper-querybasisusingtheexecutionproperties.Formoreinformationonexecutionproperties,seethesectionExecutionProperties
MultipleInsertBatches
WhenissuinganINSERTwithaqueryexpression(orthedeprecatedSELECTINTO),multipleinsertbatcheshandledbyseparatesourceINSERTSmaybeprocessedbytheTeiidserver.CareshouldbetakentoensurethattargetedsourcessupportXAorthatcompensatingactionsaretakenintheeventofafailure.
RequestLevelTransactions
196
UsingGlobalTransactionsGlobalorclientXAtransactionsareonlyapplicabletoJDBCclients.Theyalltheclienttocoordinatemultipleresourcesinasingletransaction.TotakeadvantageofXAtransactionsontheclientside,usetheTeiidDataSource(orTeiidEmbeddedwithtransactiondetectionenabled).
WhenanXAConnectionisusedinthecontextofaUserTransactioninanapplicationserver,suchasJBoss,WebSphere,orWeblogic,theresultingconnectionwillalreadybeassociatedwiththecurrentXAtransaction.NoadditionalclientJDBCcodeisnecessarytointeractwiththeXAtransaction.
UsagewithUserTransaction
UserTransactionut=context.getUserTransaction();
try{
ut.begin();
Datasourceoracle=lookup(...)
Datasourceteiid=lookup(...)
Connectionc1=oracle.getConnection();
Connectionc2=teiid.getConnection();
//dosomethingwithOracleconnection
//dosomethingwithTeiidconnection
c1.close();
c2.close();
ut.commit();
}catch(Exceptionex){
ut.rollback();
}
InthecasethatyouarenotrunninginaJEEcontainerenvironmentandyouhaveyourowntransactionmangertoco-ordinatetheXAtransactions,codewilllooksomewhatlikebelow.
ManualUsageofXAtransactions
XAConnectionxaConn=null;
XAResourcexaRes=null;
Connectionconn=null;
Statementstmt=null;
try{
xaConn=<XADataSourceinstance>.getXAConnection();
xaRes=xaConn.getXAResource();
Xidxid=<newXidinstance>;
conn=xaConn.getConnection();
stmt=conn.createStatement();
xaRes.start(xid,XAResource.TMNOFLAGS);
stmt.executeUpdate("insertinto…");
<otherstatementsonthisconnectionorotherresourcesenlistedinthistransaction>
xaRes.end(xid,XAResource.TMSUCCESS);
if(xaRes.prepare(xid)==XAResource.XA_OK){
xaRes.commit(xid,false);
}
}
catch(XAExceptione){
xaRes.rollback(xid);
}
finally{
<cleanup>
}
UsingGlobalTransactions
197
WiththeuseofglobaltransactionsmultipleTeiidXAConnectionsmayparticipateinthesametransaction.TheTeiidJDBCXAResource"isSameRM"methodreturns"true"onlyifconnectionsaremadetothesameserverinstanceinacluster.IftheTeiidconnectionsaretodifferentserverinstancesthentransactionalbehaviormaynotbethesameasiftheyweretothesameclustermember.Forexample,iftheclienttransactionmanagerusesthesameXIDforeachconnection(whichitshouldnotsinceisSameRMwillreturnfalse),duplicateXIDexceptionsmayarisefromthesamephysicalsourceaccessedthroughdifferentclustermembers.Morecommonlyiftheclienttransactionmanagerusesadifferentbranchidentifierforeachconnection,issuesmayarisewithsourcesthatlockorisolatechangesbaseduponbranchidentifiers.
UsingGlobalTransactions
198
Restrictions
ApplicationRestrictions
Theuseofglobal,local,andrequestleveltransactionsareallmutuallyexclusive.Requestleveltransactionsonlyapplywhennotinaglobalorlocaltransaction.Anyattempttomixglobalandlocaltransactionsconcurrentlywillresultinanexception.
EnterpriseInformationSystem(EIS)Support
TheunderlyingresourceadaptorsthatrepresenttheEISsystemandtheEISsystemitselfmustsupportXAtransactionsiftheywanttoparticipateindistributedXAtransactionthroughTeiid.IfsourcesystemdoesnotsupporttheXA,thenitcannotparticipateinthedistributedtransaction.However,thesourceisstilleligibletoparticipateindataintegrationwithouttheXAsupport.
TheparticipationintheXAtransactionisautomaticallydeterminedbasedontheresourceadaptorsXAcapability.Itisuser’sresponsibilitytomakesurethattheyconfigureaXAresourcewhentheyrequirethemtoparticipateindistributedtransaction.
Restrictions
199
Developer’sGuideThisguidecontainsinformationfordeveloperscreatingcustomsolutionswithTeiid.ItcoverscreatingJEEJCAconnectorswiththeTeiidframework,TeiidTranslators,TeiidUserDefinedFunctions(UDFs)aswellasrelatedtopics.
IntegratingdatafromaEnterpriseInformationSystem(EIS)intoTeiid,isseparatedintotwoparts.
1. ATranslator,whichisrequired.
2. AnoptionalResourceAdapter,whichwilltypicallybeaJCAResourceAdapter(alsocalledaJEEConnector)
ATranslatorisusedto:
TranslateaTeiid-specificcommandintoanativecommand
Executethecommand
ReturnbatchesofresultstranslatedtoexpectedTeiidtypes.
AResourceAdapterisusedto:
Handlesallcommunicationswithindividualenterpriseinformationsystem(EIS),whichcanincludedatabases,datafeeds,flatfiles,etc.
CanbeaJCAConnectororanyothercustomconnectionprovider.ThereasonTeiidrecommendsandusesJCAisthisspecificationdefineshowonecanwrite,package,andconfigureaccesstoEISsysteminconsistentmanner.Therearealsovariouscommercial/opensourcesoftwarevendorsalreadyprovidingJCAConnectorstoaccessavarietyofback-endsystems.Refertohttp://java.sun.com/j2ee/connector/.
AbstractsTranslatorsfrommanycommonconcerns,suchasconnectioninformation,resourcepooling,orauthentication.+GivenacombinationofaTranslator+ResourceAdapter,onecanconnectanyEISsystemtoTeiidfortheirdataintegrationneeds.
DoYouNeedaNewTranslator?
Teiidprovidesseveraltranslatorsforcommonenterpriseinformationsystemtypes.Ifyoucanuseoneoftheseenterpriseinformationsystems,youdonotneedtodevelopacustomone.
Teiidoffersnumerousbuilt-intranslators,including:
JDBCTranslator-Workswithmanyrelationaldatabases.TheJDBCtranslatorisvalidatedagainstthefollowingdatabasesystems:Oracle,MicrosoftSQLServer,IBMDB2,MySQL,Postgres,Derby,Sybase,H2,andHSQL.Inaddition,theJDBCTranslatorcanoftenbeusedwithother3rd-partydriversandprovidesawiderangeofextensibilityoptionstospecializebehavioragainstthosedrivers.
FileTranslator-Providesaproceduralwaytoaccessthefilesystemtohandletextfiles.
WSTranslator-ProvidesproceduralaccesstoXMLcontentusingWebServices.
LDAPTranslator-AccessestoLDAPdirectoryservices.
SalesforceTranslator-WorkswithSalesforceinterfaces.
Toseeafulllistofavailabletranslators,seeTranslators
Developer’sGuide
200
Ifthere’snotanavailabletranslatorthatmeetsyourneed,Teiidprovidestheframeworkfordevelopingyourowncustomtranslator.SeetheTranslatorDevelopmentsection,asitwilldescribehowtodevelop,packageanddeployacustomdevelopedtranslator.
DoYouNeedaNewResourceAdapter?
Asmentionedabove,foreveryTranslatorthatneedstogatherdatafromexternalsourcesystems,itrequiresaresourceadapter.
ThefollowingaresomeofresourceadaptersthatareavailabletoTeiid:
DataSource:ThisisprovidedbytheWildFlycontainer.ThisisusedbytheJDBCTranslator.
File:ProvidesaJEEJCAbasedConnectortoaccessdefineddirectoryonthefilesystem.ThisisusedbytheFileTranslator
WS:ProvidesJEEJCAConnectortoinvokeWebServicesusingWildFlyWebservicesstack.ThisisusedbytheWSTranslator
LDAP:ProvidesJEEJCAconnectortoaccessLDAP;UsedbytheLDAPTranslator.
Salesforce:ProvidesJEEJCAconnectortoaccessSalesforcebyinvokingtheirWebServiceinterface.UsedbytheSalesForceTranslator.
Toseeafulllist,seeDeployingVDBDependencies
Ifthere’snotanavailableresource-adapterthatmeetsyourneed,TeiidprovidestheframeworkfordevelopingyourownJEEJCAConnector.SeetheDevelopingJEEConnectorssection,asitwilldescribehowtodevelop,packageanddeployaresourceadapter.
OtherTeiidDevelopment
Teiidishighlyextensibleinotherways:
YoumayaddUserDefinedFunctions.RefertoUserDefinedFunctions.
Youmayadaptloggingtoyourneeds,whichisespeciallyusefulforcustomauditorcommandlogging.RefertoCustomLogging.
Youmaychangethesubsystemforcustomauthenticationandauthorization.RefertoCustomLoginModules.
Developer’sGuide
201
DevelopingJEEConnectors
Developing(Custom)JEEConnectors(ResourceAdapters)
ThischapterexamineshowtousefacilitiesprovidedbytheTeiidAPItodevelopaJEEJCAConnector.PleasenotethatthesearestandardJEEJCAconnectors,nothingspecialneedstobedoneforTeiid.AsanaidtoourTranslatordevelopers,weprovidedabaseimplementationframework.IfyoualreadyhaveaJCAConnectororsomeothermechanismtogetdatafromyoursourcesystem,youcanskipthischapter.
IfyouarenotfamiliarwithJCAAPI,pleasereadtheJCA1.5Specificationathttp://java.sun.com/j2ee/connector/.TherearelotofonlinetutorialsonhowtodesignandbuildaJCAConnector.Thebelowarehigh-levelstepsforcreatingaverysimpleconnector,howeverbuildingactualconnectorthatsupportstransactions,securitycangetmuchmorecomplex.
1. UnderstandtheJEEConnectorspecificationtohavebasicideaaboutwhatJCAconnectorsarehowtheyaredevelopedandpackaged.Refertohttp://java.sun.com/j2ee/connector/.
2. GatherallnecessaryinformationaboutyourEnterpriseInformationSystem(EIS).Youwillneedtoknow:
APIforaccessingthesystem
Configurationandconnectioninformationforthesystem
Expectationforincomingqueries/metadata
Theprocessingconstructs,orcapabilities,supportedbyinformationsystem.
Requiredpropertiesfortheconnection,suchasURL,username,etc.
3. BaseclassesforalloftherequiredsupportingJCASPIclassesareprovidedbytheTeiidAPI.TheJCACCIsupportisnotprovidedfromTeiid,sinceTeiidusestheTranslatorAPIasit’scommonclientinterface.Youwillwanttoextend:
BasicConnectionFactory–DefinestheConnectionFactory
BasicConnection–representsaconnectiontothesource.
BasicResourceAdapter–Specifiestheresourceadapterclass
4. Packageyourresourceadapter.RefertoPackagingtheAdapter.
5. Deployyourresourceadapter.RefertoPackagingtheAdapter.
ForsampleresourceadaptercoderefertotheTeiidSourcecodeathttps://github.com/teiid/teiid/tree/master/connectors/.
RefertotheJBossApplicationServerConnectorsdocumentationathttp://docs.jboss.org/jbossas/jboss4guide/r4/html/ch7.chapt.html.
DevelopingJEEConnectors
202
ConnectorEnvironmentSetupTosetuptheenvironmentfordevelopingacustomconnector,youhave2options:
1. Manuallysetupthebuildenvironment-structure,frameworkclasses,andresources.
2. UsetheTeiidConnectorArchetypetemplatetogeneratetheinitialproject.
ConnectorEnvironmentSetup
203
BuildEnvironmentForEclipseusers(withoutmavenintegration),createajavaprojectandadddependenciestoteiid-common-core,teiid-apiandJEEconnector-apijars.
Formavenusersaddthefollowingasyourdependencies:
<?xmlversion="1.0"encoding="UTF-8"?>
<projectxsi:schemaLocation="http://maven.apache.org/POM/4.0.0http://maven.apache.org/maven-v4_0_0.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<artifactId>connector-{name}</artifactId>
<groupId>org.company.project</groupId>
<name>NameConnector</name>
<packaging>rar</packaging>
<description>Thisconnectorisasample</description>
<dependencies>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-common-core</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.resource</groupId>
<artifactId>connector-api</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
</project>
Wherethe${teiid-version}propertyshouldbesettotheexpectedversion,suchas9.2.0.Final.YoucanfindTeiidartifactsintheJBossmavenrepository.The$\{version.connector.api}versionlastusedwas1.5.
ConnectorEnvironmentSetup
204
ArchetypeTemplateConnectorProjectOnewaytostartdevelopingacustomconnector(resource-adapter)istocreateaprojectusingtheTeiidarchetypetemplate.Whentheprojectiscreatedfromthetemplate,itwillcontaintheessentialclassesandresourcesforyoutobeginaddingyourcustomlogic.Additionally,themavendependenciesaredefinedinthepom.xmlsothatyoucanbegincompilingtheclasses.
Note Theprojectwillbecreatedasanindependentprojectandhasnoparentmavendependencies.It’sdesignedtobebuiltindependentofbuildingTeiid.
Youhave2optionsforcreatingaconnectorproject;inEclipsebycreatinganewmavenprojectfromthearchetypeorbyusingthecommandlinetogeneratetheproject.
CreateProjectinEclipseTocreateaJavaprojectinEclipsefromanarchetype,performthefollowing:
OpentheJAVAperspective
FromthemenuselectFile–>New—>Other
Inthetree,expandMavenandselectMavenProject,pressNext
Onthe"SelectprojectnameandLocation"window,youcanacceptthedefaults,pressNext
Onthe"SelectanArchetype"window,selectConfigurebutton
Addtheremotecatalog:https://repository.jboss.org/nexus/content/repositories/releases/thenclickOKtoreturn
Enter"teiid"inthefiltertoseetheTeiidarchetypes.
Selecttheconnector-archetypev8.7.1,thenpressNext
Enteralltheinformation(i.e.,GroupID,ArtifactID,etc.)neededtogeneratetheproject,thenclickFinish
Theprojectwillbecreatedandnameaccordingtothe*ArtifactID*.
CreateProjectusingCommandLine
Tocreateacustomconnectorprojectfromthecommandline,youcanusethefollowingtemplatecommand:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/release
s/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=connector-archetype\
-DarchetypeVersion=8.7.1\
-DgroupId=${groupId}\
-DartifactId=connector-${connector-name}\
-Dpackage=org.teiid.resource.adapter.${connector-name}\
-Dversion=${teiid.version}\
-Dconnector-name=${connector-name}\
-Dvendor-name=${vendor-name}\
-Dteiid-version=${teiid-version}
where:
-DarchetypeGroupId-isthegroupIDforthearchetypetousetogenerate
ConnectorEnvironmentSetup
205
-DarchetypeArtifactId-istheartifactIDforthearchetypetousetogenerate
-DarchetypeVersion-istheversionofthearchetypetousetogenerate
-DgroupId-(userdefined)groupIDforthenewconnectorprojectpom.xml
-DartifactId-(userdefined)artifactIDforthenewconnectorprojectpom.xml
-Dpackage-(userdefined)thepackagestructurewherethejavaandresourcefileswillbecreated
-Dversion-(userdefined)theversionthatthenewconnectorprojectpom.xmlwillbe
-Dconnector-name-(userdefined)thename(type)ofthenewconnectorproject,usedtocreatethejavacla
ssnamesandrar
-Dvendor-name-nameoftheVendorforthedatasource,updatestherar
-Dteiid-version-theTeiidversiontheconnectorwilldependupon
Thefollowingisanexample:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/release
s/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=connector-archetype\
-DarchetypeVersion=8.7.1\
-DgroupId=org.jboss.teiid.connectors\
-Dpackage=org.teiid.resource.adapter.myType\
-DartifactId=connector-myType\
-Dversion=0.0.1-SNAPSHOT\
-Dconnector-name=myType\
-Dvendor-name=MyVendor\
-Dteiid-version=8.7.0.Final
Whenexecuted,youwillbeaskedtoconfirmtheproperties
Confirmpropertiesconfiguration:
groupId:org.jboss.teiid.connectors
artifactId:connector-myType
version:0.0.1-SNAPSHOT
package:org.teiid.resource.adapter.myType
connector-name:myType
vendor-name:MyVendor
teiid-version:8.7.0.Final
Y::
typeY(yes)andpressenter,andthecreationoftheconnectorprojectwillbedone
Uponcreation,adirectorybasedonthe*artifactId*willbecreated,thatwillcontaintheproject.Note:Theprojectwillnotcompilebecausethe${connector-name}ConnectioninterfaceintheConnectionImplhasnotbeenaddedasadependencyinthepom.xml.Thiswillneedtobedone.
Nowyouarereadytostartaddingyourcustomcode.
ConnectorEnvironmentSetup
206
ImplementingtheTeiidFrameworkIfyouaregoingtousetheTeiidframeworkfordevelopingaJCAconnector,followthesesteps.Therequiredclassesareinorg.teiid.resource.apipackage.PleasenotethatTeiidframeworkdoesnotmakeuseJCA’sCCIframework,onlytheJCA’sSPIinterfaces.
DefineManagedConnectionFactory
DefinetheConnectionFactoryclass
DefinetheConnectionclass
Definetheconfigurationpropertiesina"ra.xml"file
DefineManagedConnectionFactoryExtendtheBasicManagedConnectionFactory,andprovideaimplementationforthe"createConnectionFactory()"method.Thismethoddefinesafactorymethodthatcancreateconnections.
Thisclassalsodefinesconfigurationvariables,likeuser,password,URLetctoconnecttotheEISsystem.Defineanattributeforeachconfigurationvariable,andthenprovideboth"getter"and"setter"methodsforthem.Notetouseonly"java.lang"objectsastheattributes,DONOTuseJavaprimitivesfordefiningandaccessingtheproperties.Seethefollowingcodeforanexample.
publicclassMyManagedConnectionFactoryextendsBasicManagedConnectionFactory
{
@Override
publicObjectcreateConnectionFactory()throwsResourceException
{
returnnewMyConnectionFactory();
}
//configpropertyname(metadataforthesearedefinedinsidethera.xml)
StringuserName;
publicStringgetUserName(){returnthis.userName;}
publicvoidsetUserName(Stringname){this.userName=name;}
//configpropertycount(metadataforthesearedefinedinsidethera.xml)
Integercount;
publicIntegergetCount(){returnthis.count;}
publicvoidsetCount(Integervalue){this.count=value;}
}
DefinetheConnectionFactoryclass
ExtendtheBasicConnectionFactoryclass,andprovideaimplementationforthe"getConnection()"method.
publicclassMyConnectionFactoryextendsBasicConnectionFactory
{
@Override
publicMyConnectiongetConnection()throwsResourceException
{
returnnewMyConnection();
}
}
ImplementingtheTeiidFramework
207
SincetheManagedconnectionobjectcreatedthe"ConnectionFactory"classithasaccesstoalltheconfigurationparameters,if"getConnection"methodneedstodopassanyofcredentialstotheunderlyingEISsystem.TheConnectionFactoryclasscanalsogetreferencetothecallinguser’sjavax.security.auth.Subjectduring"getConnection"methodbycalling
Subjectsubject=ConnectionContext.getSubject();
This"Subject"objectcangiveaccesstologged-inuser’scredentialsandrolesthataredefined.Notethatthismaybenull.
Notethatyoucandefine"security-domain"forthisresourceadapter,thatisseparatefromtheTeiiddefined"security-domain"forvalidatingtheJDBCenduser.However,itistheuser’sresponsibilitytomakethenecessaryloginsbeforetheContainer’sthreadaccessesthisresourceadapter,andthiscangetoverlycomplex.
DefinetheConnectionclass
ExtendtheBasicConnectionclass,andprovideaimplementationbasedonyouraccessoftheConnectionobjectintheTranslator.Ifyourconnectionisstateful,thenoverride"isAlive()"and"cleanup()"methodsandprovideproperimplementations.ThesearecalledtocheckifaConnectionisstaleorneedtoflushthemfromtheconnectionpooletc.bytheContainer.
publicclassMyConnectionextendsBasicConnection
{
publicvoiddoSomeOperation(command)
{
//dosomeoperationwithEISsystem..
//ThisismethodyouuseintheTranslator,youshouldknow
//whatneedtobedonehereforyoursource..
}
@Override
publicbooleanisAlive()
{
returntrue;
}
@Override
publicvoidcleanUp()
{
}
}
XATransactions
IfyourEISsourcecanparticipateinXAtransactions,thenonyourConnectionobject,overridethe"getXAResource()"methodandprovidethe"XAResource"objectfortheEISsystem.RefertoDefinetheConnectionclass.Also,Youneedtoextendthe"BasicResourceAdapter"classandprovideimplementationformethod"publicXAResource[]getXAResources(ActivationSpec[]specs)"toparticipateincrashrecovery.
Notethat,onlywhentheresourceadaptersareXAcapable,thenTeiidcanmakethemparticipateinadistributedtransactions.IftheyarenotXAcapable,thensourcecanparticipateindistributedquerybutwillnotparticipateinthetransaction.Transactionsemanticsaredefinedbyhowyouyouconfigured"connection-factory"ina"resource-adapter".i.e.jta=true/false.
Definetheconfigurationpropertiesina"ra.xml"fileDefinea"ra.xml"filein"META-INF"directoryofyourRARfile.Anexamplefileisprovidedinra.xmlfileTemplate.
ImplementingtheTeiidFramework
208
ForeveryattributedefinedinsidetheyourManagedConnectionFactoryclass,definethefollowingXMLconfigurationforthatattributeinsidethe"ra.xml"file.ThesepropertiesareusedbyusertoconfigureinstanceofthisConnectorinsideaContainer.Also,duringthestartuptheContainerreadsthesepropertiesfromthisfileandknowshowtoinjectprovidedvaluesinthedatasourcedefinitionintoaninstanceof"ManagedConnectionFactory"tocreatetheConnection.RefertoDevelopingJEEConnectors#DefineManagedConnectionFactory.
<config-property>
<description>
{$display:"${display-name}",$description:"${description}",$allowed="${allowed}",
$required="${true|false}",$defaultValue="${default-value}"}
</description>
<config-property-name>${property-name}</config-property-name>
<config-property-type>${property-type}</config-property-type>
<config-property-value>${optioal-property-value}</config-property-value>
</config-property>
Theformatandcontentsof"<description>"elementmaybeusedasextendedmetadatafortooling.Thespecialformatmustbeginandendwithcurlybracese.g.\{…}.Thisuseofthespecialformatandallpropertiesisoptional.Propertynamesbeginwith'$'andareseparatedfromthevaluewith':'.Doublequotesidentifiesasinglevalue.Apairofsquarebrackets,e.g.[…],containingcommaseparateddoublequotedentriesdenotesalistvalue.
Extendedmetadataproperties
$display:Displaynameoftheproperty
$description:Descriptionabouttheproperty
$required:Thepropertyisarequiredproperty;oroptionalandadefaultissupplied
$allowed:Ifpropertyvaluemustbeincertainsetoflegalvalues,thisdefinesalltheallowedvalues
$masked:Thetoolsneedtomasktheproperty;Donotshowinplaintext;usedforpasswords
$advanced:NotesthisasAdvancedproperty
$editable:Propertycanbemodified;orread-only
Notethatalltheseareoptionalproperties;howeverintheabsenceofthismetadata,Teiidtoolingmaynotworkasexpected.
ImplementingtheTeiidFramework
209
ra.xmlfileTemplateThisappendixcontainsanexampleofthera.xmlfilethatcanbeusedasatemplatewhencreatinganewConnector.
<?xmlversion="1.0"encoding="UTF-8"?>
<connectorxmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee
http://java.sun.com/xml/ns/j2ee/connector_1_5.xsd"version="1.5">
<vendor-name>${comapany-name}</vendor-name>
<eis-type>${type-of-connector}</eis-type>
<resourceadapter-version>1.0</resourceadapter-version>
<license>
<description>${licensetext}</description>
<license-required>true</license-required>
</license>
<resourceadapter>
<resourceadapter-class>org.teiid.resource.spi.BasicResourceAdapter</resourceadapter-class>
<outbound-resourceadapter>
<connection-definition>
<managedconnectionfactory-class>${connection-factory}</managedconnectionfactory-class>
<!--repeatforeveryconfigurationproperty-->
<config-property>
<description>
{$display:"${short-name}",$description:"${description}",$allowed:[${value-list}],
$required:"${required-boolean}",$defaultValue:"${default-value}"}
</description>
<config-property-name>${property-name}</config-property-name>
<config-property-type>${property-type}</config-property-type>
<config-property-value>${optional-property-value}</config-property-value>
</config-property>
<!--usethebelowasisifyouusedtheConnectionFactoryinterface-->
<connectionfactory-interface>
javax.resource.cci.ConnectionFactory
</connectionfactory-interface>
<connectionfactory-impl-class>
org.teiid.resource.spi.WrappedConnectionFactory
</connectionfactory-impl-class>
<connection-interface>
javax.resource.cci.Connection
</connection-interface>
<connection-impl-class>
org.teiid.resource.spi.WrappedConnection
</connection-impl-class>
</connection-definition>
<transaction-support>NoTransaction</transaction-support>
<authentication-mechanism>
<authentication-mechanism-type>BasicPassword</authentication-mechanism-type>
<credential-interface>
javax.resource.spi.security.PasswordCredential
</credential-interface>
</authentication-mechanism>
<reauthentication-support>false</reauthentication-support>
</outbound-resourceadapter>
ImplementingtheTeiidFramework
210
</resourceadapter>
</connector>
$\{…}indicatesavaluetobesuppliedbythedeveloper.
ImplementingtheTeiidFramework
211
PackagingtheAdapterOncealltherequiredcodeisdeveloped,itistimetopackagethemintoaRARartifact,thatcanbedeployedintoaContainer.ARARartifactissimilartoaWAR.ToputtogetheraRARfileitreallydependsuponthebuildsystemyouareusing.
Eclipse:YoucanstartoutwithbuildingJavaConnectorproject,itwillproducetheRARfile
Ant:Ifyouareusing"ant"buildtool,thereis"rar"buildtaskavailable
Maven:Ifyouareusingmaven,use<packaging>elementvalueas"rar".Teiidusesmaven,youcanlookatanyofthe"connector"projectsforsample"pom.xml"file.SeeBuildEnvironmentforanexampleofapom.xmlfile.
MakesurethattheRARfile,underits"META-INF"directoryhasthe"ra.xml"file.Ifyouareusingmavenrefertohttp://maven.apache.org/plugins/maven-rar-plugin/.IntherootoftheRARfile,youcanembedtheJARfilecontainingyourconnectorcodeandanydependentlibraryJARfiles.
PackagingtheAdapter
212
AddingDependentLibrariesAddMANIFEST.MFfileintheMETA-INFdirectory,andthefollowinglinetoaddthecoreTeiidAPIdependenciesforresourceadapter.
Dependencies:org.jboss.teiid.common-core,org.jboss.teiid.api,javax.api
Ifyourresourceadapterdependsuponanyotherthirdpartyjarfiles,.dllor.sofilestheycanbeplacedattherootoftherarfile.IfanyoftheselibrariesarealreadyavailableasmodulesinWildFly,thenyoucanaddthemodulenametotheaboveMANIFEST.MFfiletodefineasdependency.
PackagingtheAdapter
213
DeployingtheAdapterOncetheRARfileisbuilt,deployitbycopyingtheRARfileinto"deploy"directoryofWildFly’schosenprofile.TypicallytheserverdoesnotneedtoberestartedwhenanewRARfileisbeingadded.Alternatively,youcanalsouse"admin-console",awebbasedmonitoringandconfigurationtool,todeploythisfileintothecontainer.
OncetheConnector’sRARfileisdeployedintotheWildFlycontainer,nowyoucancreateaninstanceofthisconnectortobeusedwithyourTranslator.CreatinganinstanceofthisConnectorisnodifferentthancreatinga"ConnectionFactory"inWildFly.Again,youhavetwowaystocreatea""ConnectionFactory".
Editstandalone.xmlordomain.xmlfile,andaddfollowingXMLinthe"resource-adapters"subystem.
<!--Ifsusbsytemisalreadydefined,onlycopythecontentsunderitandedittosuityourneeds-->
<subsystemxmlns="urn:jboss:domain:resource-adapters:1.0">
<resource-adapters>
<resource-adapter>
<archive>teiid-connector-sample.rar</archive>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.MyManagedConnectionFactory"jndi
-name="${jndi-name}"
enabled="true"
use-java-context="true"
pool-name="sample-ds">
<config-propertyname="UserName">jdoe</config-property>
<config-propertyname="Count">12</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
</resource-adapters>
</subsystem>
Therearelotmorepropertiesthatyoucandefineforpooling,transactions,security,etc.,inthisfile.ChecktheWildFlydocumentationforalltheavailableproperties.
Alternatively,youcanusethewebbased""admin-console"configurationandmonitoringprogram,tocreateanewConnectionFactory.HaveyourRARfilenameandneededconfigurationpropertieshandyandfilloutwebformtocreatetheConnectionFactory.
DeployingtheAdapter
214
Translator(Custom)DevelopmentBelowarethehigh-levelstepsforcreatingcustomTranslators,whichisdescribedinthissection.Thissectionwillcoverhowtodoeachofthefollowingstepsindetail.Italsoprovidesadditionalinformationforadvancedtopics,suchasstreaminglargeobjects.
ForsampleTranslatorcode,refertotheTeiidsourcecodeathttps://github.com/teiid/teiid/tree/master/connectors/.
1. CreateaneworreuseanexistingResourceAdapterfortheEISsystem,tobeusedwiththisTranslator.RefertoCustomResourceAdapters.
2. DecidewhethertousetheTeiidarchetypetemplatetocreateyourinitialcustomtranslatorprojectandclassesormanuallycreateyourenvironment.RefertoEnvironmentSetup.
3. ImplementtherequiredclassesdefinedbytheTranslatorAPI.RefertoImplementingtheFramework.1)CreateanExecutionFactory–Extendtheorg.teiid.translator.ExecutionFactoryclass2)CreaterelevantExecutions(andsub-interfaces)–specifieshowtoexecuteeachtypeofcommand
4. Definethetemplateforexposingconfigurationproperties.RefertoPackaging.
5. DeployyourTranslator.RefertoDeployment.
6. DeployaVirtualDatabase(VDB)thatusesyourTranslator.
7. ExecutequeriesviaTeiid.
TranslatorDevelopment
215
TranslatorEnvironmentSetupTosetuptheenvironmentfordevelopingacustomtranslator,youhave2options;
1. Manuallysetupthebuildenvironment-structure,frameworkclasses,andresources.
2. UsetheTeiidTranslatorArchetypetemplatetogeneratetheinitialproject.
EnvironmentSetup
216
SettingupthebuildenvironmentForEclipseusers(withoutmavenintegration),createajavaprojectandadddependenciesto"teiid-common-core","teiid-api"andJEE"connector-api"jars.
Formavenusersaddthefollowingasyourdependencies:
<dependencies>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-api</artifactId>
<version>${teiid-version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-common-core</artifactId>
<version>${teiid-version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.resource</groupId>
<artifactId>connector-api</artifactId>
<version>${version.connector.api}</version>
<scope>provided</scope>
</dependency>
</dependencies>
Wherethe${teiid-version}propertyshouldbesettotheexpectedversion,suchas9.2.0.Final.YoucanfindTeiidartifactsintheJBossmavenrepository.The$\{version.connector.api}versionlastusedwas1.5.
EnvironmentSetup
217
ArchetypeTemplateTranslatorProjectOnewaytostartdevelopingacustomtranslatoristocreateaprojectusingtheTeiidarchetypetemplate.Whentheprojectiscreatedfromthetemplate,itwillcontaintheessentialclasses(i.e.,ExecutionFactory)andresourcesforyoutobeginaddingyourcustomlogic.Additionally,themavendependenciesaredefinedinthepom.xmlsothatyoucanbegincompilingtheclasses.
Note Theprojectwillbecreatedasanindependentprojectandhasnoparentmavendependencies.It’sdesignedtobebuiltindependentofbuildingTeiid.
Youhave2optionsforcreatingatranslatorproject;inEclipsebycreatinganewmavenprojectfromthearchetypeorbyusingthecommandlinetogeneratetheproject.
CreateProjectinEclipseTocreateaJavaprojectinEclipsefromanarchetype,performthefollowing:
OpentheJAVAperspective
FromthemenuselectFile–>New—>Other
Inthetree,expandMavenandselectMavenProject,pressNext
Onthe"SelectprojectnameandLocation"window,youcanacceptthedefaults,pressNext
Onthe"SelectanArchetype"window,selectConfigurebutton
Addtheremotecatalog:https://repository.jboss.org/nexus/content/repositories/releases/thenclickOKtoreturn
Enter"teiid"inthefiltertoseetheTeiidarchetypes.
Selectthetranslator-archetypev8.7.1,thenpressNext
Enteralltheinformation(i.e.,GroupID,ArtifactID,etc.)neededtogeneratetheproject,thenclickFinish
Theprojectwillbecreatedandnameaccordingtothe*ArtifactID*.
CreateProjectusingCommandLine
Tocreateacustomtranslatorprojectfromthecommandline,youcanusethefollowingtemplatecommand:
mvnarchetype:generate\
-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/releases/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=translator-archetype\
-DarchetypeVersion=8.7.1\
-DgroupId=${groupId}\
-DartifactId=translator-${translator-name}\
-Dpackage=org.teiid.translator.${translator-name}\
-Dversion=${version}\
-Dtranslator-name=${translator-name}\
-Dteiid-version=${teiid-version}
where:
-DarchetypeGroupId-isthegroupIDforthearchetypetousetogenerate
-DarchetypeArtifactId-istheartifactIDforthearchetypetousetogenerate
EnvironmentSetup
218
-DarchetypeVersion-istheversionforthearchetypetousetogenerate
-DgroupId-(userdefined)groupIDforthenewtranslatorprojectpom.xml
-DartifactId-(userdefined)artifactIDforthenewtranslatorprojectpom.xml
-Dpackage-(userdefined)thepackagestructurewherethejavaandresourcefileswillbecreated
-Dversion-(userdefined)theversionthatthenewconnectorprojectpom.xmlwillbe
-Dtranslator-name-(userdefined)thename(type)ofthenewtranslatorproject,usedtocreatethejavacl
assnames
-Dteiid-version-theTeiidversiontheconnectorwilldependupon
Thefollowingisanexampletoexecute:
mvnarchetype:generate\
-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/releases/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=translator-archetype\
-DarchetypeVersion=8.7.1\
-DgroupId=org.jboss.teiid.connector\
-DartifactId=translator-myType\
-Dpackage=org.teiid.translator.myType\
-Dversion=0.0.1-SNAPSHOT\
-Dtranslator-name=MyType\
-Dteiid-version=8.7.0.Final
Whenexecuted,youwillbeaskedtoconfirmtheproperties
Confirmpropertiesconfiguration:
groupId:org.jboss.teiid.connector
artifactId:translator-myType
version:0.0.1-SNAPSHOT
package:org.teiid.translator.myType
teiid-version:8.7.0.Final
translator-name:MyType
Y::
typeY(yes)andpressenter,andthecreationofthetranslatorprojectwillbedone
Uponcreation,adirectorybasedonthe*artifactId*willbecreated,thatwillcontaintheproject.'cd'intothatdirectoryandexecuteatestbuildtoconfirmtheprojectwascreatedcorrectly:
mvncleaninstall
Thisshouldbuildsuccessfully,andnowyouarereadytostartaddingyourcustomcode.
EnvironmentSetup
219
ImplementingtheFramework
ImplementingtheFramework
220
CachingAPITranslatorsmaycontributecacheentriestotheresultsetcachebytheuseoftheCacheDirectiveobject.TranslatorswishingtoparticipateincachingshouldreturnaCacheDirectivefromtheExecutionFactory.getCacheDirectivemethod,whichiscalledpriortoexecution.ThecommandpassedtogetCacheDirectivewillalreadyhavebeenvettedtoensurethattheresultsareeligibleforcaching.Forexampleupdatecommandsorcommandswithpusheddependentsetswillnotbeeligibleforcaching.
IfthetranslatorreturnsnullfortheCacheDirective,whichisthedefaultimplementation,theenginewillnotcachethetranslatorresultsbeyondthecurrentcommand.Itisuptoyourcustomtranslatororcustomdelegatingtranslatortoimplementyourdesiredcachingpolicy.
Note Inspecialcircumstanceswherethetranslatorhasperformedit’sowncaching,itcanindicatetotheenginethattheresultsshouldnotbecachedorreusedbysettingtheScopetoScope.NONE.
ThereturnedCacheDirectivewillbesetontheExecutionContextandisavailableviatheExecutionContext.getCacheDirective()method.HavingExeuctionFactory.getCacheDirectivecalledpriortoexecutionallowsthetranslatortopotentiallybeselectiveaboutwhichresultstoevenattempttocache.Sincethereisaresourceoverheadwithcreatingandstoringthecachedresultsitmaynotbedesirabletoattempttocacheallresultsifit’spossibletoreturnlargeresultsthathavealowusagefactor.IfyouareunsureaboutwhethertocacheaparticularcommandresultyoumayreturnaninitialCacheDirectivethenchangetheScopetoScope.NONEatanytimepriortothefinalcacheentrybeingcreatedandtheenginewillgiveupcreatingtheentryandreleaseit’sresources.
IfyouplanonmodifyingtheCacheDirectiveduringexecution,justmakesuretoreturnanewinstancefromtheExecutionFactory.getCacheDirectivecall,ratherthanreturningasharedinstance.
TheCacheDirectivereadAllBooleanfieldisusedtocontrolwhethertheentireresultshouldbereadifnotalloftheresultswereconsumedbytheengine.IfreadAllisfalsethenanypartialusageoftheresultwillnotresultinitbeingaddedasacacheentry.Partialuseisdeterminedafteranyimplicitorexplicitlimithasbeenapplied.TheotherfieldsontheCacheDirectiveobjectmaptothecachehintoptions.Seethetablebelowforthedefaultvaluesforalloptions.
option default
scope Session
ttl rscachettl
readAll true
updatable true
prefersMemory false
ImplementingtheFramework
221
CommandLanguage
Language
TeiidsendscommandstoyourTranslatorinobjectform.Theseclassesarealldefinedinthe"org.teiid.language"package.TheseobjectscanbecombinedtorepresentanypossiblecommandthatTeiidmaysendtotheTranslator.However,itispossibletonotifyTeiidthatyourTranslatorcanonlyacceptcertainkindsofconstructsviathecapabilitiesdefinedonthe"ExecutionFactory"class.RefertoTranslatorCapabilitiesformoreinformation.
ThelanguageobjectsallextendfromtheLanguageObjectinterface.Languageobjectsshouldbethoughtofasatreewhereeachnodeisalanguageobjectthathaszeroormorechildlanguageobjectsoftypesthataredependentonthecurrentnode.
AllcommandssenttoyourTranslatorareintheformoftheselanguagetrees,wheretherootofthetreeisasubclassofCommand.Commandhasseveralsub-classes,namely:
QueryExpression
Insert-alsorepresentsanupsert,seetheisUpsertflag.
Update
Delete
BatchedUpdates
Call
Importantcomponentsofthesecommandsareexpressions,criteria,andjoins,whichareexaminedincloserdetailbelow.Formoreontheclassesandinterfacesdescribedhere,refertotheTeiidJavaDocshttp://docs.jboss.org/teiid/7.6/apidocs.
ExpressionsAnexpressionrepresentsasinglevalueincontext,althoughinsomecasesthatvaluemaychangeasthequeryisevaluated.Forexample,aliteralvalue,suchas5representsanintegervalue.Ancolumnreferencesuchas"table.EmployeeName"representsacolumninadatasourceandmaytakeonmanyvalueswhilethecommandisbeingevaluated.
Expression–baseexpressioninterface
ColumnReference–representsancolumninthedatasource
Literal–representsaliteralscalarvalue.
Parameter–representsaparameterwithmultiplevalues.ThecommandshouldbeaninstanceofBatchedCommand,whichprovidesallvaluesviagetParameterValues.
Function–representsascalarfunctionwithparametersthatarealsoExpressions
AggregateFunction–representsanaggregatefunctionwhichcanholdasingleexpression
WindowFunction–representsanwindowfunctionwhichholdsanAggregateFunction(whichisalsousedtorepresentanalyticalfunctions)andaWindowSpecification
ScalarSubquery–representsasubquerythatreturnsasinglevalue
SearchedCase,SearchedWhenClause–representsasearchedCASEexpression.ThesearchedCASEexpressionevaluatesthecriteriainWHENclausestilloneevaluatestoTRUE,thenevaluatestheassociatedTHENclause.
ImplementingtheFramework
222
Array–representsanarrayofexpressions,currentlyonlyusedbytheengineinmulti-attributedependentjoins-seethesupportsArrayTypecapability.
Condition
Acriteriaisacombinationofexpressionsandoperatorsthatevaluatestotrue,false,orunknown.CriteriaaremostcommonlyusedintheWHEREorHAVINGclauses.
Condition–thebasecriteriainterface
Not–usedtoNOTanothercriteria
AndOr–usedtocombineothercriteriaviaANDorOR
SubuqeryComparison–representsacomparisoncriteriawithasubqueryincludingaquantifiersuchasSOMEorALL
Comparison–representsacomparisoncriteriawith=,>,<,etc.
BaseInCondition–baseclassforanINcriteria
In–representsanINcriteriathathasasetofexpressionsforvalues
SubqueryIn–representsanINcriteriathatusesasubquerytoproducethevalueset
IsNull–representsanISNULLcriteria
Exists–representsanEXISTScriteriathatdetermineswhetherasubquerywillreturnanyvalues
Like–representsaLIKE/SIMILARTO/LIKE_REGEXcriteriathatcomparesstringvalues
TheFROMClause
TheFROMclausecontainsalistofTableReference’s.
NamedTable–representsasingleTable
Join–hasaleftandrightTableReferenceandinformationonthejoinbetweentheitems
DerivedTable–representsatabledefinedbyaninlineQueryExpression
AlistofTableReferenceareusedbydefault,inthepushdownquerywhennoouterjoinsareused.Ifanouterjoinisusedanywhereinthejointree,therewillbeatreeofJoinswithasingleroot.ThislatterformistheANSIpreferredstyle.IfyouwishallpushdownqueriescontainingjoinstobeinANSIstylehavethecapability"useAnsiJoin"returntrue.RefertoCommandFormformoreinformation.
QueryExpressionStructure
QueryExpressionisthebaseforbothSELECTqueriesandsetqueries.ItmayoptionallytakeanOrderBy(representingaSQLORDERBYclause),aLimit(representaSQLLIMITclause),oraWith(representsaSQLWITHclause).
SelectStructure
EachQueryExpressioncanbeaSelectdescribingtheexpressions(typicallyelements)beingselectedandanTableReferencespecifyingthetableortablesbeingselectedfrom,alongwithanyjoininformation.TheSelectmayoptionallyalsosupplyanCondition(representingaSQLWHEREclause),aGroupBy(representingaSQLGROUPBY
ImplementingtheFramework
223
clause),ananCondition(representingaSQLHAVINGclause).
SetQueryStructure
AQueryExpressioncanalsobeaSetQuerythatrepresentsonoftheSQLsetoperations(UNION,INTERSECT,EXCEPT)ontwoQueryExpression.TheallflagmaybesettoindicateUNIONALL(currentlyINTERSECTandEXCEPTALLarenotallowedinTeiid)
WithStructure
AWithclausecontainsnamedQueryExpressionsheldbyWithItemsthatcanbereferencedastablesinthemainQueryExpression.
InsertStructure
EachInsertwillhaveasingleNamedTablespecifyingthetablebeinginsertedinto.ItwillalsohasalistofColumnReferencespecifyingthecolumnsoftheNamedTablethatarebeinginsertedinto.ItalsohasInsertValueSource,whichwillbealistofExpressions(ExpressionValueSource)oraQueryExpression
UpdateStructure
EachUpdatewillhaveasingleNamedTablespecifyingthetablebeingupdatedandlistofSetClauseentriesthatspecifyColumnReferenceandExpressionpairsfortheupdate.TheUpdatemayoptionallyprovideacriteriaConditionspecifyingwhichrowsshouldbeupdated.
DeleteStructure
EachDeletewillhaveasingleNamedTablespecifyingthetablebeingdeletedfrom.Itmayalsooptionallyhaveacriteriaspecifyingwhichrowsshouldbedeleted.
CallStructure
EachCallhaszeroormoreArgumentobjects.TheArgumentobjectsdescribetheinputparameters,theoutputresultset,andtheoutputparameters.
BatchedUpdatesStructure
EachBatchedUpdateshasalistofCommandobjects(whichmustbeeitherInsert,UpdateorDelete)thatcomposethebatch.
LanguageUtilities
Thissectioncoversutilitiesavailablewhenusing,creating,andmanipulatingthelanguageinterfaces.
ImplementingtheFramework
224
DataTypes
TheTranslatorAPIcontainsaninterfaceTypeFacilitythatdefinesdatatypesandprovidesvaluetranslationfacilities.Thisinterfacecanbeobtainedfromcalling"getTypeFacility()"methodonthe"ExecutionFactory"class.
TheTypeFacitlityinterfacehasmethodsthatsupportdatatypetransformationanddetectionofappropriateruntimeorJDBCtypes.TheTypeFacility.RUNTIME_TYPESandTypeFacility.RUNTIME_NAMESinterfacesdefinesconstantsforallTeiidruntimedatatypes.AllExpressioninstancesdefineadatatypebasedonthissetoftypes.Theseconstantsareoftenneededinunderstandingorcreatinglanguageinterfaces.
LanguageManipulationInTranslatorsthatsupportafullersetofcapabilities(thosethatgenerallyaretranslatingtoalanguageofcomparabletoSQL),thereisoftenaneedtomanipulateorcreatelanguageinterfacestomoveclosertothesyntaxofchoice.Someutilitiesareprovidedforthispurpose:
SimilartotheTypeFacility,youcancall"getLanguageFactory()"methodonthe"ExecutionFactory"togetareferencetotheLanguageFactoryinstanceforyourtranslator.Thisinterfaceisafactorythatcanbeusedtocreatenewinstancesofalltheconcretelanguageinterfaceobjects.
SomehelpfulutilitiesforworkingwithConditionobjectsareprovidedintheLanguageUtilclass.ThisclasshasmethodstocombineConditionwithANDortobreakanConditionapartbasedonANDoperators.Theseutilitiesarehelpfulforbreakingapartacriteriaintoindividualfiltersthatyourtranslatorcanimplement.
RuntimeMetadataTeiidusesalibraryofmetadata,knownas"runtimemetadata"foreachvirtualdatabasethatisdeployedinTeiid.TheruntimemetadataisasubsetofmetadataasdefinedbymodelsintheTeiidmodelsthatcomposethevirtualdatabase.WhilebuilingyourVDBintheDesigner,youcandefinewhatcalled"ExtensionModel",thatdefinesanynumberofarbitarypropertiesonamodelanditsobjects.Atruntime,usingthisruntimemetadatainterface,yougetaccesstothosesetpropertiesdefinedduringthedesigntime,todefine/hintanyexecutionbehavior.
TranslatorgetsaccesstotheRuntimeMetadatainterfaceatthetimeofExcecutioncreation.Translatorscanaccessruntimemetadatabyusingtheinterfacesdefinedinorg.teiid.metadatapackage.ThispackagedefinesAPIrepresentingaSchema,Table,ColumnsandProcedures,andwaystonavigatetheseobjects.
MetadataObjects
AllthelanguageobjectsextendAbstractMetadataRecordclass
Column-returnsColumnmetadatarecord
Table-returnsaTablemetadatarecord
Procedure-returnsaProceduremetadatarecord
ProcedureParameter-returnsaProcedureParametermetadatarecord
Onceametadatarecordhasbeenobtained,itispossibletouseitsmetadataaboutthatobjectortofindotherrelatedmetadata.
AccesstoRuntimeMetadata
ImplementingtheFramework
225
TheRuntimeMetadatainterfaceispassedinforthecreationofan"Execution".See"createExecution"methodonthe"ExecutionFactory"class.ItprovidestheabilitytolookupmetadatarecordsbasedontheirfullyqualifiednamesintheVDB.
TheprocessofgettingaTable’spropertiesissometimesneededfortranslatordevelopment.Forexampletogetthe"NameInSource"propertyorallextensionproperties:
ObtainingMetadataProperties
//gettingtheTablemetadatafromanTableisstraight-forward
Tabletable=runtimeMetadata.getTable("table-name");
StringcontextName=table.getNameInSource();
//Thepropswillcontainextensionproperties
Map<String,String>props=table.getProperties();
LanguageVisitors
Framework
TheAPIprovidesalanguagevisitorframeworkintheorg.teiid.language.visitorpackage.Theframeworkprovidesutilitiesusefulinnavigatingandextractinginformationfromtreesoflanguageobjects.
ThevisitorframeworkisavariantoftheVisitordesignpattern,whichisdocumentedinseveralpopulardesignpatternreferences.Thevisitorpatternencompassestwoprimaryoperations:traversingthenodesofagraph(alsoknownasiteration)andperformingsomeactionateachnodeofthegraph.Inthiscase,thenodesarelanguageinterfaceobjectsandthegraphisreallyatreerootedatsomenode.Theprovidedframeworkallowsforcustomizationofbothaspectsofvisiting.
ThebaseAbstractLanguageVisitorclassdefinesthevisitmethodsforallleaflanguageinterfacesthatcanexistinthetree.TheLanguageObjectinterfacedefinesanacceptVisitor()method–thismethodwillcallbackonthevisitmethodofthevisitortocompletethecontract.AbaseclasswithemptyvisitmethodsisprovidedasAbstractLanguageVisitor.TheAbstractLanguageVisitorisjustavisitorshell–itperformsnoactionswhenvisitingnodesanddoesnotprovideanyiteration.
TheHierarchyVisitorprovidesthebasiccodeforwalkingalanguageobjecttree.TheHierarchyVisitorperformsnoactionasitwalksthetree–itjustencapsulatestheknowledgeofhowtowalkit.Ifyourtranslatorwantstoprovideacustomiterationthatwalkstheobjectsinaspecialorder(toexcludenodes,includenodesmultipletimes,conditionallyincludenodes,etc)thenyoumusteitherextendHierarchyVisitororbuildyourowniterationvisitor.Ingeneral,thatisnotnecessary.
TheDelegatingHierarchyVisitorisaspecialsubclassoftheHierarchyVisitorthatprovidestheabilitytoperformadifferentvisitor’sprocessingbeforeandafteriteration.Thisallowsusersofthisclasstoimplementeitherpre-orpost-orderprocessingbasedontheHierarchyVisitor.TwohelpermethodsareprovidedonDelegatingHierarchyVisitortoaidinexecutingpre-andpost-ordervisitors.
ProvidedVisitorsTheSQLStringVisitorisaspecialvisitorthatcantraverseatreeoflanguageinterfacesandoutputtheequivalentTeiidSQL.Thisvisitorcanbeusedtoprintlanguageobjectsfordebuggingandlogging.TheSQLStringVisitordoesnotusetheHierarchyVisitordescribedinthelastsection;itprovidesbothiterationandprocessingtypefunctionalityinasinglecustomvisitor.
TheCollectorVisitorisahandyutilitytocollectalllanguageobjectsofacertaintypeinatree.Someadditionalhelpermethodsexisttodocommontaskssuchasretrievingallelementsinatree,retrievingallgroupsinatree,andsoon.
WritingaVisitor
ImplementingtheFramework
226
Writingyourownvisitorcanbequiteeasyifyouusetheprovidedfacilities.Ifthenormalmethodofiteratingthelanguagetreeissufficient,thenjustfollowthesesteps:
CreateasubclassofAbstractLanguageVisitor.Overrideanyvisitmethodsneededforyourprocessing.Forinstance,ifyouwantedtocountthenumberofelementsinthetree,youneedonlyoverridethevisit(ColumnReference)method.Collectanystateinlocalvariablesandprovideaccessormethodsforthatstate.
Decidewhethertousepre-orderorpost-orderiteration.NotethatvisitationorderisbaseduponsyntaxorderingofSQLclauses-notprocessingorder.
WritecodetoexecuteyourvisitorusingtheutilitymethodsonDelegatingHierarchyVisitor:
//Getobjecttree
LanguageObjectobjectTree=…
//Createyourvisitorinitializeasnecessary
MyVisitorvisitor=newMyVisitor();
//Callthevisitorusingpre-ordervisitation
DelegatingHierarchyVisitor.preOrderVisit(visitor,objectTree);
//Retrievestatecollectedwhilevisiting
intcount=visitor.getCount();
ImplementingtheFramework
227
ConnectionstoSource
Obtainingconnections
Theextended"ExecutionFactory"mustimplementthegetConnection()methodtoallowtheConnectorManagertoobtainaconnection.
ReleasingConnections
OncetheConnectorManagerhasobtainedaconnection,itwillusethatconnectiononlyforthelifetimeoftherequest.Whentherequesthascompleted,thecloseConnection()methodcalledonthe"ExecutionFactory".Youmustalsooverridethismethodtoproperlyclosetheconnection.
Incases(suchaswhenaconnectionisstatefulandexpensivetocreate),connectionsshouldbepooled.IftheresourceadapterisJEEJCAconnectorbased,thenpoolingisautomaticallyprovidedbytheWildFlycontainer.IfyourresourceadapterdoesnotimplementtheJEEJCA,thenconnectionpoolingsemanticsarelefttotheusertodefineontheirown.
ImplementingtheFramework
228
DependentJoinPushdownDependentjoinsareatechniqueusedinfederationtoreducethecostofcrosssourcejoins.Joinvaluesfromonesideofajoinaremadeavailabletotheothersidewhichreducesthenumberoftuplesneededtopreformthejoin.TranslatorsmayindicatesupportfordependentjoinpushdownviathesupportsDependentJoinandsupportsFullDependentJoincapabilities.Thehandlingofpushdowndependentjoinqueriescanbecomplicated.
Note SeetheJDBCTranslatorforthereferenceimplementationofdependentjoinpushdownhandlingbasedupthecreationtemporarytables.
KeyPushdown
Themoresimplisticmodeofdependentjoinpushdownistopushonlythekey(equi-join)valuestoeffectivelyevaluateasemi-join-thefulljoinwillstillbeprocessedbytheengineaftertheretrieval.Theordering(ifpresent)andallofthenon-dependentcriteriaconstructsonthepushdowncommandmustbehonored.Thedependentcriteria,whichwillbeaComparisonwithaParameter(possiblyinArrayform),maybeignoredinpartorintotaltoretrieveasupersetofthetuplesrequested.
PushdownkeydependentjoinquerieswillbeinstancesofSelectwiththerelevantdependentvaluesavailableviaSelect.getDependentValues().AdependentvaluetuplelistisassociatedtoParametersbyidviatheParameter.getDepenentValueId()identifier.Thedependenttuplelistproviderowsthatarereferencedbythecolumnpositions(availableviaParameter.getValueIndex()).Careshouldbetakenwiththetuplevaluesastheymayguaranteedtobeordered,butwillbeuniquewithrespecttoalloftheParameterreferencesagainstthegivendependentvaluetuplelist.
FullPushdownInsomescenarios,typicallywithsmallindependentdatasetsorextensiveprocessingabovethejointhatcanbepushedtothesource,itisadvantageousforthesourcetohandlethedependentjoinpushdown.ThisfeatureismarkedassupportedbythesupportsFullDependentJoincapability.Herethesourceisexpectedtoprocessthecommandexactlyasspecified-thedependentjoinisnotoptional
FullpushdowndependentjoinquerieswillbeinstancesofQueryExpressionwiththerelevantdependentvaluesavailableviaspecialcommontabledefinitionsusingQueryExpression.getWith().TheindependentsideofafullpushdownjoinwillappearasacommontableWithItemwithadependentvaluetuplelistavailableviaWithItem.getDependentValues().ThedependentvaluetupleswillpositionallymatchthecolumnsdefinedbyWithItem.getColumns().Thedependentvaluetuplelistisnotguaranteedtobeinanyparticularorder.
ImplementingtheFramework
229
ExecutingCommands
ExecutionModes
TheTeiidqueryengineusesthe"ExecutionFactory"classtoobtainthe"Execution"interfaceforthecommanditisexecuting.Theactualqueriesthemselvesaresenttotranslatorsintheformofasetofobjects,whicharefurtherdescribedinCommandLanguage.RefertoCommandLanguage.Translatorsareallowedtosupportanysubsetoftheavailableexecutionmodes.
ExecutionInterface Commandinterface(s) Description
ResultSetExecution QueryExpressionAquerycorrespondingtoaSQLSELECTorsetquerystatement.
UpdateExecutionInsert,Update,Delete,
BatchedUpdates
Aninsert,update,ordelete,correspondingtoaSQLINSERT,UPDATE,orDELETEcommand
ProcedureExecution Call
Aprocedureexecutionthatmayreturnaresultsetand/oroutputvalues.
TypesofExecutionModes
AlloftheexecutioninterfacesextendthebaseExecutioninterfacethatdefineshowexecutionsarecancelledandclosed.ProcedureExecutionalsoextendsResultSetExecution,sinceproceduresmayalsoreturnresultsets.
ExecutionContext
Theorg.teiid.translator.ExecutionContextprovidesaconsiderableamountofinformationrelatedtothecurrentexecution.AnExecutionContextinstanceismadeavailabletoeachExecution.Specificusageishighlightedinthisguidewhereapplicable,butyoumayuseanyinformationalgettermethodasdesired.ExampleusagewouldincludecallingExecutionContext.getRequestId(),ExecutionContext.getSession(),etc.forloggingpurposes.
CommandContext
Aorg.teiid.CommandContextisavailableviatheExecutionContext.getCommandContext()method.TheCommandContextcontainsinformationaboutthecurrentuserquery,includingtheVDB,theabilitytoaddclientwarnings-addWarning,orhandlegeneratedkeys-isReturnAutoGeneratedKeys,returnGeneratedKeys,andgetGeneratedKeys.
GeneratedKeys
Toseeiftheuserqueryexpectsgeneratedkeystobereturned,consulttheCommandContext.isReturnAutoGeneratedKeys()method.Ifyouwishtoreturngeneratedkeys,youmustfirstcreateaGeneratedKeysinstancetoholdthekeyswiththereturnGeneratedKeysmethodpassingthecolumnnamesandtypesofthekeycolumns.OnlyoneGeneratedKeysmaybeassociatedwiththeCommandContextatanygiventime.
SourceHints
ImplementingtheFramework
230
TheTeiidsourcemeta-hintisusedtoprovidehintsdirectlytosourceexecutionsviauserortransformationqueries.Seethereferenceformoreonsourcehints.Ifspecifiedandapplicable,thegeneralandsourcespecifichintwillbesuppliedviatheExecutionContextmethodsgetGeneralHintandgetSourceHint.SeethesourcefortheOracleExecutionFactoryforanexampleofhowthissourcehintinformationcanbeutilized.
ResultSetExecution
TypicallymostcommandsexecutedagainsttranslatorsareQueryExpression.Whilethecommandisbeingexecuted,thetranslatorprovidesresultsviatheResultSetExecution’s"next"method.The"next"methodshouldreturnnulltoindicatetheendofresults.Note:theexpectedbatchsizecanbeobtainedfromtheExecutionContext.getBatchSize()methodandusedasahintinfetchingresultsfromtheEIS.
UpdateExecution
Eachexecutionreturnstheupdatecount(s)expectedbytheupdatecommand.IfpossibleBatchedUpdatesshouldbeexecutedatomically.TheExecutionContext.isTransactional()methodcanbeusedtodetermineiftheexecutionisalreadyunderatransaction.
ProcedureExecution
Procedurecommandscorrespondtotheexecutionofastoredprocedureorsomeotherfunctionalconstruct.Aproceduretakeszeroormoreinputvaluesandcanreturnaresultsetandzeroormoreoutputvalues.Examplesofprocedureexecutionwouldbeastoredprocedureinarelationaldatabaseoracalltoawebservice.
Ifaresultsetisexpectedwhenaprocedureisexecuted,allrowsfromitwillberetrievedviatheResultSetExecutioninterfacefirst.Then,ifanyoutputvaluesareexpected,theywillberetrievedviathegetOutputParameterValues()method.
AsynchronousExecutions
Insomescenarios,atranslatorneedstoexecuteasynchronouslyandallowtheexecutingthreadtoperformotherwork.Toallowasynchronousexecution,youshouldthrowaDataNotAvailableExecptionduringaretrievalmethod,ratherthanexplicitlywaitingorsleepingfortheresults.TheDataNotAvailableExceptionmaytakeadelayparameteroraDateinitsconstructortoindicatewhentopollnextforresults.Anynon-negativedelayvalueindicatesthetimeinmillisecondsuntilthenextpollingshouldbeperformed.TheDataNotAvailableException.NO_POLLINGexception(oranyDataNotAvailableExceptionwithanegativedelay)canbethrowntoindicatethattheexecutionwillcallExecutionContext.dataAvailable()toindicateprocessingshouldresume.
Note ADataNotAvailableExceptionshouldnotbethrownbytheexecutemethod,asthatcanresultintheexecutemethodbeingcalledmultipletimes.
Note Sincetheexecutionandtheassociatedconnectionarenotcloseduntiltheworkhascompleted,careshouldbetakenifusingasynchronousexecutionsthatholdalotofstate.
Apositiveretrydelayisnotaguaranteeofwhenthetranslatorwillbepollednext.IftheDataNotAvailableExceptionisconsumedwhiletheenginethinksmoreworkcanbeperformedorthereareothershorterdelaysissuedfromothertranslators,thentheplanmaybere-queuedearlierthanexpected.YoushouldsimplyrethrowaDataNotAvailableExceptionifyourexecutionisnotyetready.AlternativelytheDataNotAvailableExceptionmaybemarkedasstrict,whichdoesprovideaguaranteethattheExecutionwillnotbecalleduntilthedelayhasexpiredorthegivenDatehasbeenreached.UsingtheDateconstructor
ImplementingtheFramework
231
makestheDataNotAvailableExceptionautomaticallystrict.Duetoenginethreadpoolcontention,platformtimeresolution,etc.astrictDataNotAvailableExceptionisnotareal-timeguaranteeofwhenthenextpollforresultswilloccur,onlythatitwillnot
occurbeforethen.
NoteIfyourExecutionFactoryreturnsonlyasynchexecutionsthatperformminimalwork,thenconsiderhavingExecutionFactory.isForkablereturnfalsesothattheengineknowsnottospawnaseparatethreadforaccessingyourExecution.
ReusableExecutions
AtranslatormayreturninstancesofReusableExecutionsfortheexpectedExecutionobjects.TherecanbeoneReusableExecutionperqueryexecutingnodeintheprocessingplan.ThelifecycleofaReusableExecutionisdifferentthatanormalExecution.Afteranormalcreation/execute/closecycletheReusableExecution.resetiscalledforthenextexecutioncycle.Thismayoccurindefinitelydependingonhowmanytimesaprocessingnodeexecutesitsquery.ThebehavioroftheclosemethodisnodifferentthanaregularExecution,itmaynotbecalleduntiltheendofthestatementiflobsaredetectedandanyconnectionassociatedwiththeExecutionwillalsobeclosed.Whentheusercommandisfinished,theReusableExecution.dispose()methodwillbecalled.
IngeneralReusableExecutionsaremostusefulforcontinuousqueryexecutionandwillalsomakeuseoftheExecutionCotext.dataAvailable()methodforAsynchronousExecutions.SeetheClientDeveloper’sGuideforexecutingcontinuousstatements.Incontinuousmodetheuserquerywillbecontinuouslyre-executed.AReusableExecutionallowsthesameExecutionobjecttobeassociatedwiththeprocessingplanforagivenprocessingnodeforthelifetimeoftheuserquery.Thiscansimplifyasynchresourcemanagement,suchasestablishingqueuelisteners.Returninganullresultfromthenext()methodReusableExecutionjustaswithnormalExecutionsindicatesthatthecurrentpushdowncommandresultshaveended.Oncethereset()methodhasbeencalled,thenextsetofresultsshouldbereturnedagainterminatedwithanullresult.
BulkExecutionNonbatchedInsert,Update,Deletecommandsmayhavemulti-valuedParameterobjectsifthecapabilitiesshowssupportforBulkUpdate.Commandswithmulti-valued\{{Parameters}}srepresentmultipleexecutionsofthesamecommandwithdifferentvalues.AswithBatchedUpdates,bulkoperationsshouldbeexecutedatomicallyifpossible.
CommandCompletionAllnormalcommandexecutionsendwiththecallingofclose()ontheExecutionobject.Yourimplementationofthismethodshoulddotheappropriateclean-upworkforallstatecreatedintheExecutionobject.
CommandCancellationCommandssubmittedtoTeiidmaybeabortedinseveralscenarios:
ClientcancellationviatheJDBCAPI(orotherclientAPIs)
Administrativecancellation
Clean-upduringsessiontermination
Clean-upifaqueryfailsduringprocessingUnliketheotherexecutionmethods,whicharehandledinasingle-threadedmanner,callstocancelhappenasynchronouslywithrespecttotheexecutionthread.
ImplementingtheFramework
232
Yourconnectorimplementationmaychoosetodonothinginresponsetothiscancellationmessage.Inthisinstance,Teiidwillcallclose()ontheexecutionobjectaftercurrentprocessinghascompleted.Implementingthecancel()methodallowsforfasterterminationofqueriesbeingprocessedandmayallowtheunderlyingdatasourcetoterminateitsoperationsfasteraswell.
ImplementingtheFramework
233
ExtendingtheExecutionFactoryClassThemainclassinthetranslatorimplementationisExecutionFactory.AbaseclassisprovidedintheTeiidAPI,soacustomtranslatormustextendorg.teiid.translator.ExecutionFactorytoconnectandqueryanenterprisedatasource.Thisextendedclassmustprovideano-argconstructorthatcanbeconstructedusingJavareflectionlibraries.ThisExecutionFactoryneedstodefine/overridethefollowingelements.
packageorg.teiid.translator.custom;
@Translator(name="custom",description="ConnecttoMyEIS")
publicclassCustomExecutionFactoryextendsExecutionFactory<MyConnectionFactory,MyConnection>{
publicCustomExecutionFactory(){
}
}
Definetheannotation@Translatoronextended"ExecutionFactory"class.Thisannotationdefinesthename,whichisusedastheidentifierduringdeployment,andthedescriptionofyourtranslator.ThisnameiswhatyouwillbeusingintheVDBandelsewhereintheconfigurationtorefertothistranslator.
ConnectionFactoryDefinesthe"ConnectionFactory"interfacethatisdefinedinresourceadapter.Thisisdefinedaspartofclassdefinitionofextended"ExecutionFactory"class.Referto"MyConnectionFactory"sampleintheDevelopingJEEConnectorschapter.
ConnectionDefinesthe"Connection"interfacethatisdefinedintheresourceadapter.Thisisdefinedaspartofclassdefinitionofextended"ExecutionFactory"class.Referto"MyConnection"classsampleinvtheDevelopingJEEConnectorschapter.
ConfigurationPropertiesIfthetranslatorrequiresexternalconfiguration,thatdefineswaysfortheusertoalterthebehaviorofaprogram,thendefineanattributevariableintheclassanddefine"get"and"set"methodsforthatattribute.Also,annotateeach"get"methodwith@TranslatorPropertyannotationandprovidethemetadataabouttheproperty.
Forexample,ifyouneedapropertycalled"foo",byprovidingtheannotationontheseproperties,theTeiidtoolingcanautomaticallyinterrogateandprovideagraphicalwaytoconfigureyourTranslatorwhiledesigningyourVDB.
privateStringfoo="blah";
@TranslatorProperty(display="Fooproperty",description="descriptionaboutFoo")
publicStringgetFoo()
{
returnfoo;
}
publicvoidsetFoo(Stringvalue)
{
returnthis.foo=value;
}
ImplementingtheFramework
234
The@TranslatorPropertydefinesthefollowingmetadatathatyoucandefineaboutyourproperty
display:Displaynameoftheproperty
description:Descriptionabouttheproperty
required:Thepropertyisarequiredproperty
advanced:Thisisadvancedproperty;Adefaultvaluemustbeprovided.Apropertycannotbe"advanced"and"required"atsametime.
masked:Thetoolsneedtomasktheproperty;Donotshowinplaintext;usedforpasswords
Onlyjavaprimitive(int,boolean),primitiveobjectwrapper(java.lang.Integer),orEnumtypesaresupportedasTranslatorproperties.Complexobjectsarenotsupported.Thedefaultvaluewillbederivedfromcallingthegettermethod,ifavailable,onanewlyconstructedinstance.Allpropertiesshouldhaveadefaultvalue.Ifthereisnoapplicabledefault,thenthepropertyshouldbemarkedintheannotationasrequired.Initializationwillfailifarequiredpropertyvalueisnotprovided.
InitializingtheTranslator
Overrideandimplementthestartmethod(besuretocall"super.start()")ifyourtranslatorneedstodoanyinitializingbeforeitisusedbytheTeiidengine.ThismethodwillbecalledbyTeiid,onceafteralltheconfigurationpropertiessetaboveareinjectedintotheclass.
ExtendedTranslatorCapabilities
Thesearevariousmethodsthattypicallybeginwithmethodsignature"supports"onthe"ExecutionFactory"class.ThesemethodsneedtobeoverriddentodescribetheexecutioncapabilitiesoftheTranslator.RefertoTranslatorCapabilitiesformoreonthesemethods.
Execution(andsub-interfaces)
Basedontypesofexecutionsyouaresupporting,thefollowingmethodsneedtobeoverriddentoprovideimplementationsfortheirrespectivereturninterfaces.
createResultSetExecution-Overrideifyouaredoingreadbasedoperationthatisreturningarowsofresults.Forex:select
createUpdateExecution-Overrideifyouaredoingwritebasedoperations.Forex:insert,update,delete
createProcedureExecution-Overideifyouaredoingprocedurebasedoperations.Forex;storedprocedures.Thisworkswellfornon-relationalsources.Youcanchoosetoimplementalltheexecutionmodesorjustwhatyouneed.Seemoredetailsonthisbelow.
MetadataOverrideandimplementthemethodgetMetadataProcessor(),ifyouwanttoexposethemetadataaboutthesourceforuseinVDBs.Thisdefinesthetables,columnnames,procedures,parameters,etc.foruseinthequeryengine.ThismethodisusedbyDesignertoolingwhentheTeiidConnectionimporterisused.AsampleMetadataProcessormaylooklike
publicclassMyMetadataProcessorimplementsMetadataProcessor<Connection>{
publicvoidprocess(MetadataFactorymf,Connectionconn){
Objectsomedata=connection.getSomeMetadata();
ImplementingtheFramework
235
Tabletable=mf.addTable(tableName);
Columncol1=mf.addColumn("col1",TypeFacility.RUNTIME_NAMES.STRING,table);
columncol2=mf.addColumn("col2",TypeFacility.RUNTIME_NAMES.STRING,table);
//addapushdownfunctionthatcanalsobeevaluatedintheengine
Methodmethod=...
Functionf=mf.addFunction("func",method);
//addapushdownaggregatefunctionthatcanalsobeevaluatedintheengine
MethodaggMethod=...
Functionaf=mf.addFunction("agg",aggMethod);
af.setAggregateAttributes(newAggregateAttributes());
...
}
}
IfyourMetadataProcessorneedsexternalpropertiesthatareneededduringtheimportprocess,youcandefinethemonMetadataProcessor.Forexample,todefineaimportpropertycalled"ColumnNamePattern",whichcanbeusedtofilterwhichcolumnsaredefinedonthetable,canbedefinedinthecodelikethefollowing
@TranslatorProperty(display="ColumnNamePattern",category=PropertyType.IMPORT,description="Patterntode
rivecolumnnames")
publicStringgetColumnNamePattern(){
returncolumnNamePattern;
}
publicvoidsetColumnNamePattern(StringcolumnNamePattern){
this.columnNamePattern=columnNamePattern;
}
Notethecategorytype.Theconfigurationpropertydefinedintheprevioussectionisdifferentfromthisone.Configurationpropertiesdefinetheruntimebehavioroftranslator,whereas"IMPORT"propertiesdefinethemetadataimportbehavior,andaidincontrollingwhatmetadataisexposedbyyourtranslator.
Thesepropertiescanbeautomaticallyinjectedthrough"import"propertiessetthroughDesignerwhenusingthe"TeiidConnection"importerorthepropertiescanbedefinedunderthe<model>constructinthevdb.xmlfile,like
<vdbname="myvdb"version="1">
<modelname="legacydata"type="PHYSICAL">
<propertyname="importer.ColumnNamePattern"value="col*"/>
....
<sourcename=.../>
</model>
</vdb>
ExtensionMetadataPropertiesTheremaybetimeswhenimplementingacustomtranslator,thebuiltinmetadataaboutyourschemaisnotenoughtoprocesstheincomingqueryduetovarianceofsemanticswithyoursourcequery.Toaidthisissue,Teiidprovidesamechanismcalled"ExtensionMetadata",whichisamechanismtodefinecustompropertiesandthenaddthosepropertiesonmetadataobject(table,procedure,function,column,indexetc.).Forexample,inmycustomtranslatoratablerepresentsafileondisk.Icoulddefineaextensionmetadatapropertyas
publicclassMyMetadataProcessorimplementsMetadataProcessor<Connection>{
publicstaticfinalStringNAMESPACE="{http://my.company.corp}";
@ExtensionMetadataProperty(applicable={Table.class},datatype=String.class,display="Filename",descript
ion="FileName",required=true)
publicstaticfinalStringFILE_PROP=NAMESAPCE+"FILE";
ImplementingtheFramework
236
publicvoidprocess(MetadataFactorymf,Connectionconn){
Objectsomedata=connection.getSomeMetadata();
Tabletable=mf.addTable(tableName);
table.setProperty(FILE_PROP,somedata.getFileName());
Columncol1=mf.addColumn("col1",TypeFacility.RUNTIME_NAMES.STRING,table);
columncol2=mf.addColumn("col2",TypeFacility.RUNTIME_NAMES.STRING,table);
}
}
The@ExtensionMetadataPropertydefinesthefollowingmetadatathatyoucandefineaboutyourproperty
applicable:Metadataobjectthisisapplicableon.ThisisarrayofmetadataclasseslikeTable.class,Column.class.
datatype:Thejavaclassindicatingthedatatype
display:Displaynameoftheproperty
description:Descriptionabouttheproperty
required:Indicatesifthepropertyisarequiredproperty
Howthisisused?
Whenyoudefineanextensionmetadatapropertylikeabove,duringtheruntimeyoucanobtainthevalueofthatproperty.Ifyougetthequeryobjectwhichcontains`SELECT*FROMMyTable',MyTablewillberepresentedbyanobjectcalled"NamedTable".Soyoucandothefollowing
for(TableReferencetr:query.getFrom()){
NamedTablet=(NameTable)tr;
Tabletable=t.getMetadataObject();
Stringfile=table.getProperty(FILE_PROP);
..
}
NowyouhaveaccessedthefilenameyousetduringtheconstructionoftheTableschemaobject,andyoucanusethisvaluehoweveryouseemfeasibletoexecuteyourquery.Withthecombinationofbuiltinmetadatapropertiesandextensionmetadatapropertiesyoucandesignandexecutequeriesforavarietyofsources.
Logging
Teiidprovidesorg.teiid.logging.LogManagerclassforloggingpurposes.CreatealoggingcontextandusetheLogManagertologyourmessages.ThesewillbeautomaticallysenttothemainTeiidlogs.Youcaneditthe"jboss-log4j.xml"inside"conf"directoryoftheWildFly’sprofiletoaddthecustomcontext.TeiidusesLog4Jasitsunderlyingloggingsystem.
Exceptions
Ifyouneedtobubbleupanyexceptionuseorg.teiid.translator.TranslatorExceptionclass.
ImplementingtheFramework
237
ImplementingtheFramework
238
LargeObjectsThissectionexamineshowtousefacilitiesprovidedbytheTeiidAPItouselargeobjectssuchasblobs,clobs,andxmlinyourTranslator.
DataTypes
Teiidsupportsthreelargeobjectruntimedatatypes:blob,clob,andxml.Ablobisa"binarylargeobject",aclobisa"characterlargobject",and"xml"isa"xmldocument".Columnsmodeledasablob,clob,orxmlaretreatedsimilarlybythetranslatorframeworktosupportmemory-safestreaming.
WhyUseLargeObjectSupport?
TeiidallowsaTranslatortoreturnalargeobjectthroughtheTeiidtranslatorAPIbyjustreturningareferencetotheactuallargeobject.AccesstothatLOBwillbestreamedasappropriateratherthanretrievedallatonce.Thisisusefulforseveralreasons:
1. Reducesmemoryusagewhenreturningtheresultsettotheuser.
2. Improvesperformancebypassinglessdataintheresultset.
3. Allowsaccesstolargeobjectswhenneededratherthanassumingthatuserswillalwaysusethelargeobjectdata.
4. Allowsthepassingofarbitrarilylargedatavalues.However,thesebenefitscanonlytrulybegainediftheTranslatoritselfdoesnotmaterializeanentirelargeobjectallatonce.Forexample,theJavaJDBCAPIsupportsastreaminginterfaceforblobandclobdata.
HandlingLargeObjectsTheTranslatorAPIautomaticallyhandleslargeobjects(Blob/Clob/SQLXML)throughthecreationofspecialpurposewrapperobjectswhenitretrievesresults.
Oncethewrappedobjectisreturned,thestreamingofLOBisautomaticallysupported.TheseLOBobjectsthencanforexampleappearinclientresults,inuserdefinedfunctions,orsenttoothertranslators.
AExecutionisusuallyclosedandtheunderlyingconnectioniseitherclosed/releasedassoonasallrowsforthatexecutionhavebeenretrieved.However,LOBobjectsmayneedtobereadaftertheirinitialretrievalofresults.WhenLOBsaredetectedthedefaultclosingbehaviorispreventedbysettingaflagviatheExecutionContext.keepAlivemethod.
Whenthe"keepAlive"aliveflagisset,thentheexecutionobjectisonlyclosedwhenuser’sStatementisclosed.
executionContext.keepExecutionAlive(true);
InsertingorUpdatingLargeObjects
LOBswillbepassedtotheTranslatorinthelanguageobjectsasLiteralcontainingajava.sql.Blob,java.sql.Clob,orjava.sql.SQLXML.Youcanusetheseinterfacestoretrievethedatainthelargeobjectanduseitforinsertorupdate.
ImplementingtheFramework
239
ImplementingtheFramework
240
TranslatorCapabilitiesTheExecutionFactoryclassdefinesallthemethodsthatdescribethecapabilitiesofaTranslator.TheseareusedbytheConnectorManagertodeterminewhatkindsofcommandsthetranslatoriscapableofexecuting.AbaseExecutionFactoryclassimplementsallthebasiccapabilitiesmethods,whichsaysyourtranslatordoesnotsupportanycapabilities.YourextendedExecutionFactoryclassmustoverridethethenecessarymethodstospecifywhichcapabilitiesyourtranslatorsupports.Youshouldconsultthedebuglogofqueryplanning(setshowplandebug)toseeifdesiredpushdownrequiresadditionalcapabilities.
CapabilityScope
Notecapabilitiesaredeterminedandcachedforthelifetimeofthetranslator.Capabilitiesbasedonconnection/userarenotsupported.
Capabilities
ThefollowingtableliststhecapabilitiesthatcanbespecifiedintheExecutionFactoryclass.
Table1.AvailableCapabilities
Capability Requires Description
SelectDistinct TranslatorcansupportSELECTDISTINCTinqueries.
SelectExpression TranslatorcansupportSELECTofmorethanjustcolumnreferences.
SelectExpressionArrayType SelectExpression,ArrayType
TranslatorcansupportSELECTofarrayexpressions.
SelectWithoutFrom TranslatorcansupportaSELECTofscalarvalueswithoutaFROMclause
AliasedTable TranslatorcansupportTablesintheFROMclausethathaveanalias.
InnerJoins Translatorcansupportinnerandcrossjoins
SelfJoinsAliasedGroupsandatleastoneofthejointypesupports.
TranslatorcansupportaselfjoinbetweentwoaliasedversionsofthesameTable.
OuterJoins TranslatorcansupportLEFTandRIGHTOUTERJOIN.
FullOuterJoins TranslatorcansupportFULLOUTERJOIN.
DependentJoins Basejoinandcriteriasupport
Translatorsupportskeysetdependentjoinpushdown.SeeDependentJoinPushdown.WhensettheMaxDependentInPredicatesandMaxInCriteriaSizevaluesarenotusedbytheengine,ratherallindependentvaluesaremadeavailabletothepushdowncommand.
ImplementingtheFramework
241
FullDependentJoins Basejoinandcriteriasupport
Translatorsupportsfulldependentjoinpushdown.SeeDependentJoinPushdown.WhensettheMaxDependentInPredicatesandMaxInCriteriaSizevaluesarenotusedbytheengine,rathertheentireindependentdatasetismadeavailabletothepushdowncommand.
LateralJoin Translatorsupportslateraljoinpushdownwithsidewayscorrelation.
LateralJoinCondition LateralJoin Translatorsupportslateraljoinpushdownwithajoincondition.
SubqueryInOnJoinandbasesubquerysupport,suchasExistsCriteria
TranslatorcansupportsubqueriesintheONclause.Defaultstotrue.
InlineViews AliasedTable TranslatorcansupportanamedsubqueryintheFROMclause.
ProcedureTable TranslatorcansupportatablethatreturnsatableintheFROMclause.
BetweenCriteria Notcurrentlyused-betweencriteriaisrewritenascompoundcomparisions.
CompareCriteriaEquals Translatorcansupportcomparisoncriteriawiththeoperator=.
CompareCriteriaOrdered Translatorcansupportcomparisoncriteriawiththeoperator⇒or⇐.
CompareCriteriaOrderedExclusiveTranslatorcansupportcomparisoncriteriawiththeoperator>or<.DefaultstoCompareCriteriaOrdered
LikeCriteria TranslatorcansupportLIKEcriteria.
LikeCriteriaEscapeCharacter LikeCriteria TranslatorcansupportLIKEcriteriawithanESCAPEcharacterclause.
SimilarTo TranslatorcansupportSIMILARTOcriteria.
LikeRegexCriteria TranslatorcansupportLIKE_REGEXcriteria.
InCriteria MaxInCriteria TranslatorcansupportINpredicatecriteria.
InCriteriaSubquery TranslatorcansupportINpredicatecriteriawherevaluesaresuppliedbyasubquery.
IsNullCriteria TranslatorcansupportISNULLpredicatecriteria.
OrCriteria TranslatorcansupporttheORlogicalcriteria.
ImplementingtheFramework
242
NotCriteriaTranslatorcansupporttheNOTlogicalcriteria.IMPORTANT:Thiscapabilityalsoappliestonegationofpredicates,suchasspecifyingISNOTNULL,<(not⇒),>(not⇐),etc.
ExistsCriteria TranslatorcansupportEXISTSpredicatecriteria.
QuantifiedCompareCriteriaAll TranslatorcansupportaquantifiedcomparisoncriteriausingtheALLquantifier.
QuantifiedCompareCriteriaSome TranslatorcansupportaquantifiedcomparisoncriteriausingtheSOMEorANYquantifier.
OnlyLiteralComparisonTranslatorifonlyLiteralcomparisons(equality,ordered,like,etc.)aresupportedfornon-joinconditions.
Convert(intfromType,inttoType)
Usedforfinegrainedcontrolofconvert/castpushdown.TheExecutionFactory.getSupportedFunctions()
shouldcontainSourceSystemFunctions.CONVERT.Thismethodcanthenreturnfalsetoindicatealackofspecificsupport.SeeTypeFacility.RUNTIME_CODESforthepossibletypecodes.TheenginewilldoesnotcareaboutanunnecessaryconversionwherefromType==toType.Bydefaultlobconversionisdisabled.
OrderBy TranslatorcansupporttheORDERBYclauseinqueries.
OrderByUnrelated OrderBy TranslatorcansupportORDERBYitemsthatarenotdirectlyspecifiedintheselectclause.
OrderByNullOrdering OrderBy TranslatorcansupportORDERBYitemswithNULLSFIRST/LAST.
OrderByWithExtendedGrouping OrderByTranslatorcansupportORDERBYdirectlyoveraGROUPBYwithanextendedgroupingelementsuchasaROLLUP.
GroupBy TranslatorcansupportanexplicitGROUPBYclause.
GroupByRollup GroupBy TranslatorcansupportGROUPBY(currentlyasingle)ROLLUP.
GroupByMultipleDistinctAggregates GroupByTranslatorcansupportGROUPBYtocreatemultipledistinctaggregates(SeeIMPALA-110).
Having GroupBy TranslatorcansupporttheHAVINGclause.
AggregatesAvg TranslatorcansupporttheAVGaggregatefunction.
ImplementingtheFramework
243
AggregatesCount TranslatorcansupporttheCOUNTaggregatefunction.
AggregatesCountStar TranslatorcansupporttheCOUNT(*)aggregatefunction.
AggregatesDistinct Atleastoneoftheaggregatefunctions.
TranslatorcansupportthekeywordDISTINCTinsideanaggregatefunction.Thiskeywordindicatesthatduplicatevalueswithinagroupofrowswillbeignored.
AggregatesMax TranslatorcansupporttheMAXaggregatefunction.
AggregatesMin TranslatorcansupporttheMINaggregatefunction.
AggregatesSum TranslatorcansupporttheSUMaggregatefunction.
AggregatesEnhancedNumericTranslatorcansupporttheVAR_SAMP,VAR_POP,STDDEV_SAMP,STDDEV_POPaggregatefunctions.
ScalarSubqueriesTranslatorcansupporttheuseofasubqueryinascalarcontext(whereveranexpressionisvalid).
ScalarSubqueryProjection ScalarSubqueries Translatorcansupporttheuseofaprojectedscalarsubquery.
CorrelatedSubqueries Atleastoneofthesubquerypushdowncapabilities.
Translatorcansupportacorrelatedsubquerythatreferstoanelementintheouterquery.
CorrelatedSubqueryLimit CorrelatedSubqueriesDefaultstoCorrelatedSubqueriessupport.Translatorcansupportacorrelatedsubquerywithalimitclause.
CaseExpressions Notcurrentlyused-simplecaseisrewritenassearchedcase.
SearchedCaseExpressionsTranslatorcansupportsearchedCASEexpressionsanywherethatexpressionsareaccepted.
Unions TranslatorsupportUNIONandUNIONALL
Intersect TranslatorsupportsINTERSECT
Except TranslatorsupportsExcept
SetQueryOrderBy Unions,Intersect,orExcept TranslatorsupportssetquerieswithanORDERBY
RowLimit Translatorcansupportthelimitportionofthelimitclause
ImplementingtheFramework
244
RowOffset Translatorcansupporttheoffsetportionofthelimitclause
FunctionsInGroupBy GroupBy Translatorcansupportnon-columnreferencegroupingexpressions.
InsertWithQueryExpression TranslatorsupportsINSERTstatementswithvaluesspecifiedbyanQueryExpression.
BatchedUpdatesTranslatorsupportsabatchofINSERT,UPDATEandDELETEcommandstobeexecutedtogether.
BulkUpdate Translatorsupportsupdateswithmultiplevaluesets
CommonTableExpressions TranslatorsupportstheWITHclause.
SubqueryCommonTableExpressions CommonTableExpressions TranslatorsupportsaWITHclauseinsubqueries.
ElementaryOlapOperationsTranslatorsupportswindowfunctionsandanalyticfunctionsRANK,DENSE_RANK,andROW_NUMBER.
WindowOrderByWithAggregates ElementaryOlapOperations Translatorsupportswindowedaggregateswithawindoworderbyclause.
WindowDistinctAggregates ElementaryOlapOperations,AggregatesDistinct
Translatorsupportswindoweddistinctaggregates.
AdvancedOlapOperations ElementaryOlapOperations Translatorsupportsaggregateconditions.
OnlyFormatLiterals
functionsupportforaparse/formatfunctionandanimplementationofthesupportsFormatLiteralmethod.
TranslatorsupportsonlyliteralformatpatternsthatmustbevalidatedbythesupportsFormatLiteralmethod.
FormatLiteral(Stringliteral,Formattype) OnlyFormatLiterals Translatorsupportsthegivenliteralformat
string.
ArrayType Translatorsupportsthepushdownofarrayvalues.
OnlyCorrelatedSubqueries CorrelatedSubqueries
TranslatorONLYsupportscorrelatedsubqueries.Uncorrelatedscalarandexistssubquerieswillbepre-evaluatedpriortopush-down.
SelectWithoutFrom SelectExpressions TranslatorsupportsselectingvalueswithoutaFROMclause,e.g.SELECT1.
Upsert Translatorsupportsanupsertstyleinsert.
NotethatanypushdownsubquerymustitselfbecompliantwiththeTranslatorcapabilities.
ImplementingtheFramework
245
CommandForm
ThemethodExecutionFactory.useAnsiJoin()shouldreturntrueiftheTranslatorpreferstheuseofANSIstylejoinstructureforjointreesthatcontainonlyINNERandCROSSjoins.
ThemethodExecutionFactory.requiresCriteria()shouldreturntrueiftheTranslatorrequirescriteriaforanyQuery,Update,orDelete.ThisisareplacementforthemodelsupportpropertyWhereAll.
ScalarFunctionsThemethodExecutionFactory.getSupportedFunctions()canbeusedtospecifywhichsystem/userdefinedscalaranduserdefinedaggregatefunctionstheTranslatorsupports.Theconstantsinterfaceorg.teiid.translator.SourceSystemFunctionscontainsthestringnamesofallpossiblebuilt-inpushdownfunctions,whichincludesthefourstandardmathoperators:+,-,*,and/.
NotallsystemfunctionsappearinSourceSystemFunctions,sincesomesystemfunctionswillalwaysbeevaluatedinTeiid,aresimplealiasestootherfunctions,orarerewrittentoamorestandardexpression.
ThisdocumentationforsystemfunctionscanbefoundatScalarFunctions.IftheTranslatorstatesthatitsupportsafunction,itmustsupportalltypecombinationsandoverloadedformsofthatfunction.
Atranslatormayalsoindicatesupportforscalarfunctionsthatareintendedforpushdownevaluationbythattranslator,butarenotregisteredasuserdefinedfunctionsviaamodel/schema.ThesepushdownfunctionsarereportedtotheengineviatheExecutionFactory.getPushDownFunctions()listasFunctionMethodmetadataobjects.TheFuncitonMethodrepresentationallowthetranslatortocontrolallofthemetadatarelatedtothefunction,includingtypesignature,determinism,varargs,etc.ThesimplestwaytoaddapushdownfunctioniswithacalltoExecutionFactory.addPushDownFunction:
FunctionMethodaddPushDownFunction(Stringqualifier,Stringname,StringreturnType,String...paramTypes)
Thisresultingfunctionwillbeknownassys.qualifier.name,butcanbecalledwithjustnameaslongasthefunctionnameisunique.ThereturnedFunctionMethodobjectmaybefurthermanipulateddependingupontheneedsofthesource.AnexampleofaddingacustomconcatvarargfunctioninanExecutionFactorysubclass:
publicvoidstart()throwsTranslatorException{
super.start();
FunctionMethodfunc=addPushDownFunction("oracle","concat","string","string","string");
func.setVarArgs(true);
...
}
PhysicalLimitsThemethodExecutionFactory.getMaxInCriteriaSize()canbeusedtospecifythemaximumnumberofvaluesthatcanbepassedinanINcriteria.ThisisanimportantconstraintasanINcriteriaisfrequentlyusedtopasscriteriabetweenonesourceandanotherusingadependentjoin.
ThemethodExecutionFactory.getMaxDependentInPredicates()isusedtospecifythemaximumnumberofINpredicates(ofatmostMaxInCriteriaSize)thatcanbepassedaspartofadependentjoin.Forexampleifthereare10000valuestopassaspartofthedependentjoinandaMaxInCriteriaSizeof1000andaMaxDependentInPredicatessettingof5,thenthedependentjoinlogicwillformtwosourcequerieseachwith5INpredicatesof1000valueseachcombinedbyOR.
ThemethodExecutionFactory.getMaxFromGroups()canbeusedtospecifythemaximumnumberofFROMClausegroupsthatcanusedinajoin.-1indicatesthereisnolimit.
ImplementingtheFramework
246
UpdateExecutionModes
ThemethodExecutionFactory.supportsBatchedUpdates()canbeusedtoindicatethattheTranslatorsupportsexecutingtheBatchedUpdatescommand.
ThemethodExecutionFactory.supportsBulkUpdate()canbeusedtoindicatethattheTranslatoracceptsupdatecommandscontaingmultivaluedLiterals.
Notethatifthetranslatordoesnotsupporteitheroftheseupdatemodes,thequeryenginewillcompensatebyissuingtheupdatesindividually.
DefaultBehavior
ThemethodExecutionFactory.getDefaultNullOrder()specifiesthedefaultnullorder.CanbeoneofUNKNOWN,LOW,HIGH,FIRST,LAST.ThisisonlyusedifORDERBYissupported,butnullorderingisnot.
ThemethodExecutionFactory.getCollation()specifiesthedefaultcollation.Ifsettoavaluethatdoesnotmatchthecollationlocaledefinedbyorg.teiid.collationLocale,thensomeorderingmaynotbepusheddown.
ThemethodExecutionFactory.getRequiredLikeEscape()specifiestherequiredlikeescapecharacter.Usedonlywhenasourcesupportsaspecificescape.
UseofConnections
Method Description Default
is/setSourceRequiredTrueindicatesasourceconnectionisrequiredforfetchingthemetadataofthesourceorexecutingqueries.
true
is/setSourceRequiredForMetadataTrueindicatesasourceconnectionisrequiredforfetchingthemetadataofthesource.
SourceRequired
TransactionBehavior
ExecutionFactory.get/setTransactionSupportspecifiesthehighestleveloftransactionsupportedbyconnectionstothesource.ThisisusedasahinttotheenginefordecidingwhentostartatransactionintheautoCommitTxn=DETECTmode.DefaultstoXA.
ImplementingtheFramework
247
TranslatorPropertiesDuringtranslatordevelopment,atranslatordevelopercandefinethree(3)differenttypesofpropertysetsthatcanhelpcustomizethebehaviorofthetranslator.Thesectionsbelowdescribeseachone.
TranslatorOverrideProperties
Onthe"ExecutionFactory"classatranslatordevelopercandefineanynumberof"getter/setter"methodswiththe@TranslatorPropertyannotation.Theseproperties(alsoreferredtoaexecutionproperties)canbeusedforextendingthecapabilitiesofthetranslator.Itisimportanttodefinedefaultvaluesforalltheseproperties,asthesepropertiesarebeingdefinedtochangethedefaultbehaviorofthetranslator.Ifneeded,thevaluesforthesepropertiesaresuppliedin"vdb.xml"fileduringthedeploytimewhenthetranslatorisusedtorepresentvdb’smodel.Asampleexampleisgivenbelow:
@TranslatorProperty(display="CopyLOBs",description="Iftrue,returnedLOBswillbecopied,ratherthanstreame
dfromthesource",advanced=true)
publicbooleanisCopyLobs(){
returncopyLobs;
}
publicvoidsetCopyLobs(booleancopyLobs){
this.copyLobs=copyLobs;
atruntimethesepropertiescanbedefinedinvdb.xmlas
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="vdb"version="1">
<modelname="PM1">
<sourcename="connector"translator-name="my-translator-override"/>
</model>
<translatorname="my-translator-override"type="my-translator">
<propertyname="CopyLobs"value="true"/>
</translator>
</vdb>
MetadataImportPropertiesIfatranslatorisdefiningschemainformationbasedonthephysicalsource(i.e.implementinggetMetadatamethodonExecutionFactory)itisconnectedto,thenimportpropertiesprovideawaytocustomizethebehavioroftheimportprocess.Forexample,intheJDBCtranslatoruserscanexcludecertaintablesthatmatcharegularexpressionetc.Todefineaimportproperty,the@TranslatorProperyannotationisusedonanygetter/settermethodonthe"ExecutionFactory"classoranyclassthatimplementsthe"MetadataProcessor"interface,withcategorypropertydefinedas"PropertyType.IMPORT".Forexample.
@Translator(name="my-translator",description="MyTranslator")
publicclassMyExecutionFactoryextendsExecutionFactory<ConnectionFactory,MyConnection>{
...
publicMetadataProcessor<C>getMetadataProcessor(){
returnMyMetadataProcessor();
}
}
publicMyMetadataProcessorimplementsMetadataProcessor<MyConnection>{
publicvoidprocess(MetadataFactorymetadataFactory,MyConnectionconnection)throwsTranslatorException{
//schemagenerationcodehere
ImplementingtheFramework
248
}
@TranslatorProperty(display="HeaderRowNumber",category=PropertyType.IMPORT,description="Rownumberthat
containstheheaderinformation")
publicintgetHeaderRowNumber(){
returnheaderRowNumber;
}
publicvoidsetHeaderRowNumber(intheaderRowNumber){
this.headerRowNumber=headerRowNumber;
}
}
Belowisanexampleshowinghowtouseimportpropertieswithavdb.xmlfile
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="vdb"version="1">
<modelname="PM1">
<propertyname="importer.HeaderRowNumber"value="12"/>
<sourcename="connector"translator-name="my-translator"/>
</model>
</vdb>
Note"DesignerIntegration"-Whenpropertiesaredefinedusingtheannotationmechanismandwhenusingthe"TeiidConnection"importerintheDesigner,thesepropertieswillautomaticallyshowupinappropriatewizardforinput.
ExtensionMetadataProperties
Duringtheexecutionofthecommandintranslator,atranslatorisresponsibletoconvertTeiidsuppliedSQLcommandintodatasourcespecificquery.Mostoftimesthisconversionisnotatrivialtaskcanbeconvertedfromoneformtoanother.Therearemanycasesbuilt-inmetadataisnotsufficientandadditionalmetadataaboutsourceisusefultoformarequesttotheunderlyingphysicalsourcesystem.ExtensionMetadataPropertiesonesuchmechanismtofillthegapinthemetadata.Thesecanbedefinedspecificforagiventranslator.
Atranslatorisaplugin,thatiscommunicatingwithTeiidengineaboutit’ssourcewithit’smetadata.MetadatainthiscontextisdefinitionsofTables,Columns,Procedures,Keysetc.ThismetadatacanbedecoratedwithadditionalcustommetadataandfedtoTeiidqueryengine.Teiidqueryenginekeepsthisextendedmetadataintactalongwithitsschemaobjects,andwhenauserqueryissubmittedtothethetranslatorforexecution,thisextendedmetadatacanberetrievedformakingdecisionsinthetranslatorcode.
Extendedpropertiesaredefinedusingannotationclasscalled@ExtensionMetadataPropertyonthefieldsinyour"MetadataProcessor"or"ExcutionFactory"classes.
Forexample,saytranslatorrequiresa"encoding"propertyonTable,todothecorrectun-marshalingofdata,thispropertycanbedefinedas
publicclassMyMetadataProcessorimplementsMetadataProcessor<MyConnection>{
publicstaticfinalStringURI="{http://www.teiid.org/translator/mytranslator/2014}";
@ExtensionMetadataProperty(applicable=Table.class,datatype=String.class,display="Encoding",description="
Encoding",required=true)
publicstaticfinalStringENCODING=URI+"encode";
publicvoidprocess(MetadataFactorymf,FileConnectionconn)throwsTranslatorException{
..
Tablet=mf.addTable(tableName);
t.setProperty(ENCODING,"UTF-16");
//addcolumnsetc.
ImplementingtheFramework
249
..
}
}
Nowduringtheexecution,ontheCOMMANDobjectsuppliedtothe"Execution"class,usercan
Selectselect=(Select)command;
NamedTabletableReferece=select.getFrom().get(0);
Tablet=tableReference.getMetadataObject();
Stringencoding=t.getProperty(MyMetadataProcessor.ENCODING,false);
//usetheencodingvalueasneededtomarshalorunmarshaldata
Note
"DesignerIntegration"-Whenextendedpropertiesaredefinedusingtheannotationmechanism,whenusing"TeiidConnection"importerintheDesigner,youdonotneedtodefinethe"MetadataExtensionDefn"indesignerandregistertotousewithyourmodel,therequireddefinitionsareautomaticallydownloadedandconfiguredtouse.(ThisfeatureisnotavailableincurrentDesignerversion)
ImplementingtheFramework
250
ExtendingTheJDBCTranslatorTheJDBCTranslatorcanbeextendedtohandlenewJDBCdriversanddatabaseversions.ThisisoneofthemostcommonneedsofcustomTranslatordevelopment.ThischapteroutlinestheprocessbywhichausercanmodifythebehavioroftheJDBCTranslatorforanewsource,ratherthanstartingfromscratch.
TodesignaJDBCTranslatorforanyRDMSthatisnotalreadyprovidedbytheTeiid,extendtheorg.teiid.translator.jdbc.JDBCExecutionFactoryclassinthe"translator-jdbc"module.TherearethreetypesofmethodsthatyoucanoverridefromthebaseclasstodefinethebehavioroftheTranslator.
Extension Purpose
Capabilities SpecifytheSQLsyntaxandfunctionsthesourcesupports.
SQLTranslation CustomizewhatSQLsyntaxisused,howsource-specificfunctionsaresupported,howproceduresareexecuted.
ResultsTranslation CustomizehowresultsareretrievedfromJDBCandtranslated.
TableofContentsCapabilitiesExtensionSQLTranslationExtensionResultsTranslationExtensionAddingFunctionSupportUsingFunctionModifiersInstallingExtensions
CapabilitiesExtension
Thisextensionmustoverridethemethodsthatbeginwith"supports"thatdescribetranslatorcapabilities.RefertoCommandLanguage#TranslatorCapabilitiesforalltheavailabletranslatorcapabilities.
Themostcommonexampleisaddingsupportforascalarfunction–thisrequiresbothdeclaringthatthetranslatorhasthecapabilitytoexecutethefunctionandoftenmodifyingtheSQLTranslatortotranslatethefunctionappropriatelyforthesource.
AnothercommonexampleisturningoffunsupportedSQLcapabilities(suchasouterjoinsorsubqueries)forlesssophisticatedJDBCsources.
SQLTranslationExtension
TheJDBCExcecutionFactoryprovidesseveralmethodstomodifythecommandandthestringformoftheresultingsyntaxbeforeitissenttotheJDBCdriver,including:
ChangebasicSQLsyntaxoptions.SeetheuseXXXmethods,e.g.useSelectLimitreturnstrueforSQLServertoindicatethatlimitsareappliedintheSELECTclause.
RegisteroneormoreFunctionModifiersthatdefinehowascalarfunctionshouldbemodifiedortransformed.
ModifyaLanguageObject.-seethetranslate,translateXXX,andFunctionModifiers.translatemethods.Modifythepassedinobjectandreturnnulltoindicatethatthestandardsyntaxoutputshouldbeused.
ExtendingTheJDBCTranslator
251
ChangethewaySQLstringsareformedforaLanguageObject.--seethetranslate,translateXXX,andFunctionModifiers.translatemethods.Returnalistofparts,whichcancontainstringsandLanguageObjects,thatwillbeappendedinordertotheSQLstring.IftheincomingLanguageObjectappearsinthereturnedlistitwillnotbetranslatedagain.RefertoUsingFunctionModifiers.
ResultsTranslationExtension
TheJDBCExecutionFactoryprovidesseveralmethodstomodifythejava.sql.Statementandjava.sql.ResultSetinteractions,including:
1. OverridingthecreateXXXExecutiontosubclassthecorrespondingJDBCXXXExecution.TheJDBCBaseExecutionhasprotectedmethodstogettheappropriatestatement(getStatement,getPreparedStatement,getCallableStatement)andtobindpreparedstatementvaluesbindPreparedStatementValues.
2. RetrievevaluesfromtheJDBCResultSetorCallableStatement-seetheretrieveValuemethods.
AddingFunctionSupportRefertoUserDefinedFunctionsforaddingnewfunctionstoTeiid.Thisexamplewillshowyouhowtodeclaresupportforthefunctionandmodifyhowthefunctionispassedtothedatasource.
Followingisasummaryofallcodingstepsinsupportinganewscalarfunction:
1. Overridethecapabilitiesmethodtodeclaresupportforthefunction(REQUIRED)
2. ImplementaFunctionModifiertochangehowafunctionistranslatedandregisteritforuse(OPTIONAL)ThereisacapabilitiesmethodgetSupportedFunctions()thatdeclaresallsupportedscalarfunctions.
Anexampleofanextendedcapabilitiesclasstoaddsupportforthe"abs"absolutevaluefunction:
packagemy.connector;
importjava.util.ArrayList;
importjava.util.List;
publicclassExtendedJDBCExecutionFactoryextendsJDBCExecutionFactory
{
@Override
publicListgetSupportedFunctions()
{
ListsupportedFunctions=newArrayList();
supportedFunctions.addAll(super.getSupportedFunctions());
supportedFunctions.add("ABS");
returnsupportedFunctions;
}
}
Ingeneral,itisagoodideatocallsuper.getSupportedFunctions()toensurethatyouretainanyfunctionsupportprovidedbythetranslatoryouareextending.
ThismaybeallthatisneededtosupportaTeiidfunctioniftheJDBCdatasourcesupportsthesamesyntaxasTeiid.Thebuilt-inSQLtranslationwilltranslatemostfunctionsas:"function(arg1,arg2,…)".
UsingFunctionModifiers
ExtendingTheJDBCTranslator
252
Insomecasesyoumayneedtotranslatethefunctiondifferentlyoreveninsertadditionalfunctioncallsaboveorbelowthefunctionbeingtranslated.TheJDBCtranslatorprovidesanabstractclassFunctionModifierforthispurpose.
DuringthestartmethodamodifierinstancecanberegisteredagainstagivenfunctionnameviaacalltoJDBCExecutionFactory.registerFunctionModifier.
TheFunctionModifierhasamethodcalledtranslate.Usethetranslatemethodtochangethewaythefunctionisrepresented.
AnexampleofoverridingthetranslatemethodtochangetheMOD(a,b)functionintoaninfixoperatorforSybase(a%b).Thetranslatemethodreturnsalistofstringsandlanguageobjectsthatwillbeassembledbythetranslatorintoafinalstring.Thestringswillbeusedasisandthelanguageobjectswillbefurtherprocessedbythetranslator.
publicclassModFunctionModifierextendsFunctionModifier
{
publicListtranslate(Functionfunction)
{
Listparts=newArrayList();
parts.add("(");
Expression[]args=function.getParameters();
parts.add(args[0]);
parts.add("%");
parts.add(args[1]);
parts.add(")");
returnparts;
}
}
InadditiontobuildingyourownFunctionModifiers,thereareanumberofpre-builtgenericfunctionmodifiersthatareprovidedwiththetranslator.
Modifier Description
AliasModifier Handlessimplyrenamingafunction("ucase"to"upper"forexample)
EscapeSyntaxModifier WrapsafunctioninthestandardJDBCescapesyntaxforfunctions:\{fnxxxx()}
Toregisterthefunctionmodifiersforyoursupportedfunctions,youmustcalltheExecutionFactory.registerFunctionModifier(Stringname,FunctionModifiermodifier)method.
publicclassExtendedJDBCExecutionFactoryextendsJDBCExecutionFactory
{
@Override
publicvoidstart()
{
super.start();
//registerfunctions.
registerFunctionModifier("abs",newMyAbsModifier());
registerFunctionModifier("concat",newAliasModifier("concat2"));
}
}
Supportforthetwofunctionsbeingregistered("abs"and"concat")mustbedeclaredinthecapabilitiesaswell.Functionsthatdonothavemodifiersregisteredwillbetranslatedasusual.
InstallingExtensions
ExtendingTheJDBCTranslator
253
OnceyouhavedevelopedanextensiontotheJDBCtranslator,youmustinstallitintotheTeiidServer.TheprocessofpackagingordeployingtheextendedJDBCtranslatorsisexactlyasanyotherothertranslator.SincetheRDMSisaccessiblealreadythroughitsJDBCdriver,thereisnoneedtodeveloparesourceadapterforthissourceasWildFlyprovidesawrapperJCAconnector(DataSource)foranyJDBCdriver.
RefertoPackagingandDeploymentformoredetails.
ExtendingTheJDBCTranslator
254
DelegatingTranslatorInsomeinstancesyoumaywishtoextendseveraldifferntkindsoftranslatorswiththesamefunctionality.Ratherthancreateseparatesubclassesforeachextension,youcanusethedelegatingtranslatorframeworkwhichprovidesyouwithaproxyingmechanismtooverridetranslatorbehavior.Itimplementadelegatingtranslator,yourcommontranslatorlogicshouldbeaddedtoasubclassofBaseDelegatingExecutionFactorywhereyoucanoverrideanyofthedelegationmethodstoperformwhateverlogicyouwant.
ExampleBaseDelegatingExecutionFactorySubclass
@Translator(name="custom-delegator")
publicclassMyTranslatorextendsBaseDelegatingExecutionFactory<Object,Object>{
@Override
publicExecutioncreateExecution(Commandcommand,
ExecutionContextexecutionContext,RuntimeMetadatametadata,
Objectconnection)throwsTranslatorException{
if(commandinstanceofSelect){
//modifythecommandorreturnadifferentexecution
...
}
//thesupercallwillbetothedelegateinstance
returnsuper.createExecution(command,executionContext,metadata,connection);
}
...
}
Youwillbundleanddeployyourcustomdelegatingtranslatorisjustlikeanyothercustomtranslatordevelopment.Toyouuseyourdelegatingtranslatorinavdb,youdefineatranslatoroverridethatwiresinthedelegate.
ExampleTranslatorOverride
<translatortype="custom-delegator"name="my-translator">
<propertyvalue="delegateName"name="nameofthedelegateinstance"/>
<!--anycustompropertiesyoumayhaveonyourcustomtranslator-->
</translator>
Fromthepreviousexamplethetranslatortypeiscustom-delegator.Nowmy-translatorcanbeusedasatranslator-nameonasourceandwillproxyallcallstowhateverdelegateinstanceyouassign.
Note Notethatthedelegateinstancecanbeanytranslatorinstance,whetherconfiguredbyit’sowntranslatorentryorjustthenameofastandardtranslatortype.
DelegatingTranslator
255
PackagingOncethe"ExecutionFactory"classisimplemented,packageitinaJARfile.Thenaddthefollowingnamedfilein"META-INF/services/org.teiid.translator.ExecutionFactory"withcontentsspecifyingthenameofyourmainTranslatorfile.Notethat,thenamemustexactlymatchtoabove.Thisisjava’sstandardserviceloaderpattern.ThiswillregistertheTranslatorfordeploymentwhenthejarisdeployedintoWildFly.
org.teiid.translator.custom.CustomExecutionFactory
Packaging
256
AddingDependentModulesAddaMANIFEST.MFfileintheMETA-INFdirectory,andthecoreTeiidAPIdependenciesforresourceadapterwiththefollowingline.
Dependencies:org.jboss.teiid.common-core,org.jboss.teiid.api,javax.api
Ifyourtranslatordependsuponanyotherthirdpartyjarfiles,ensureamoduleexistsandaddthemodulenametotheaboveMANIFEST.MFfile.
Packaging
257
DeploymentAtranslatorJARfilecanbedeployedintoTeiidServerintwodifferentways
AsWildFlymodule
Createamoduleunder"jboss-as/modules"directoryanddefinethetranslatornameandmodulenameintheteiidsubsysteminstandalone-teiid.xmlfileordomain-teiid.xmlfileandrestarttheserver.ThedependentTeiidoranyotherjavaclasslibrariesmustbedefinedinmodule.xmlfileofthemodule.Forproductionprofilesthisisrecommended.
AsJARdeployment
FordevelopmenttimeorquickdeploymentyoucandeploythetranslatorJARusingtheCLIorAdminShelloradminconsoleprograms.WhenyoudeployinJARformthedependenciestoTeiidjavalibrariesandanyotherthirdpartylibrariesmustbedefinedunderMETA-INF/MANIFEST.MFfile.
Deployment
258
UserDefinedFunctionsIfyouneedtoextendTeiid’sscalaroraggregatefunctionlibrary,thenTeiidprovidesameanstodefinecustomorUserDefinedFunctions(UDF).
ThefollowingareusedtodefineaUDF.
FunctionNameWhenyoucreatethefunctionname,keeptheserequirementsinmind:
YoucannotoverloadexistingTeiidSystemfunctions.
Thefunctionnamemustbeuniqueamonguser-definedfunctionsinitsmodelforthenumberofarguments.Youcanusethesamefunctionnamefordifferentnumbersoftypesofarguments.Hence,youcanoverloadyouruser-definedfunctions.
Thefunctionnamecannotcontainthe`.'character.
Thefunctionnamecannotexceed255characters.
InputParameters-definesatypespecificsignaturelist.Allargumentsareconsideredrequired.
ReturnType-theexpectedtypeofthereturnedscalarvalue.
Pushdown-canbeoneofREQUIRED,NEVER,ALLOWED.Indicatestheexpectedpushdownbehavior.IfNEVERorALLOWEDarespecifiedthenaJavaimplementationofthefunctionshouldbesupplied.IfREQUIREDisused,thenusermustextendtheTranslatorforthesourceandaddthisfunctiontoitspushdownfunctionlibrary.
invocationClass/invocationMethod-optionalpropertiesindicatingthemethodtoinvokewhentheUDFisnotpusheddown.
Deterministic-ifthemethodwillalwaysreturnthesameresultforthesameinputparameters.Defaultstofalse.Itisimportanttomarkthefunctionasdeterministicifitreturnsthesamevalueforthesameinputsasthiswillleadtobetterperformance.SeealsotheRelationalextensionbooleanmetadataproperty"deterministic"andtheDDLOPTIONproperty"determinism".Defaultstofalse.Itisimportanttomarkthefunctionasdeterministicifitreturnsthesamevalueforthesameinputsasthiswillleadtobetterperformance.SeealsotheRelationalextensionbooleanmetadataproperty"deterministic"andtheDDLOPTIONproperty"determinism".
EvenPushdownrequiredfunctionsneedtobeaddedasaUDFtoallowTeiidtoproperlyparseandresolvethefunction.Pushdownscalarfunctionsdifferfromnormaluser-definedfunctionsinthatnocodeisprovidedforevaluationintheengine.Anexceptionwillberaisedifapushdownrequiredfunctioncannotbeevaluatedbytheappropriatesource.
UserDefinedFunctions
259
SourceSupportedFunctionsWhileTeiidprovidesanextensivescalarfunctionlibrary,itcontainsonlythosefunctionsthatcanbeevaluatedwithinthequeryengine.Inmanycircumstances,especiallyforperformance,asourcefunctionallowsforcallingasourcespecificfunction.ThesemanticsofdefiningthesourcefunctionassimilarorsametooneofdefiningtheUDF.
Forexample,supposeyouwanttousetheOracle-specificfunctionsscoreandcontainslike:
SELECTscore(1),ID,FREEDATAFROMDocsWHEREcontains(freedata,'nick',1)>0
Thescoreandcontainsfunctionsarenotpartofbuilt-inscalarfunctionlibrary.Whileyoucouldwriteyourowncustomscalarfunctiontomimictheirbehavior,it’smorelikelythatyouwouldwanttousetheactualOraclefunctionsthatareprovidedbyOraclewhenusingtheOracleFreeTextfunctionality.
InordertoconfigureTeiidtopushtheabovefunctionevaluationtoOracle,Teiidprovidesafewdifferentwaysonecanconfiguretheirinstance.
ExtendingtheTranslator
TheExecutionFactory.getPushdownFunctionsmethodcanbeusedtodescribefunctionsthatarevalidagainstallinstancesofagiventranslatortype.Thefunctionnamesareexpectedtobeprefixedbythetranslatortype,orsomeotherlogicalgrouping,e.g.salesforce.includes.ThefullnameofthefunctiononceimportedintothesystemwillqualifiedbytheSYSschema,e.g.SYS.salesforce.includes.
AnyfuncitonsaddedviathesemechanismsdonotneedtobedeclaredinExecutionFactory.getSupportedFunctions.Anyoftheadditionalhandling,suchasaddingaFunctionModifier,coveredaboveisalsoapplicablehere.Allpushdownfunctionswillhavefunctionnamesettoonlythesimplename.Schemaorotherqualificationwillberemoved.Handling,suchasfunctionmodifiers,cancheckthefunctionmetadataifthereisthepotentialforanambiguity.
Forexample,toextendtheOracleConnector
Required-extendtheOracleExecutionFactoryandaddSCOREandCONTAINSassupportedpushdownfunctionsbyeitheroverridingoraddingadditionalfunctionsin"getPushDownFunctions"method.Forthisexample,we’llcalltheclassMyOracleExecutionFactory.Addtheorg.teiid.translator.Translatorannotationtotheclass,e.g.@Translator(name="myoracle")
OptionallyregisternewFunctionModifiersonthestartoftheExecutionFactorytohandletranslationofthesefunctions.Giventhatthesyntaxofthesefunctionsissameasothertypicalfunctions,thisprobablyisn’tneeded-thedefaulttranslationshouldwork.
CreateanewtranslatorJARcontainingyourcustomExecutionFactory.RefertoPackagingandDeploymentforinstructionsonusingtheJARfile.OncethisisextendedtranslatorisdeployedintheTeiidServer,use"myoracle"astranslatornameinsteadofthe"oracle"inyourVDB’sOraclesourceconfiguration.
Ifyousourcehandingofthefunctioncanbedescribedbysimpleparametersubstitutionintoastring,thenyoumaynotneedtoextendthetranslatorforasourcespecificfunction.Youcanusetheextensionpropertyteiid_rel:native-querytodefinethesyntaxhandling-seealsoDDLMetadataforfunctions.
WhenUsingDesigner
SourceSupportedFunctions
260
IfyouaredesigningyourVDBusingtheDesigner,youcandefineafunctiononany"source"model,andthatfunctionisautomaticallyaddedaspushdownfunctionwhentheVDBisdeployed.ThereisnoadditionalneedforaddingJavacode.
WithoutDesigner
IfyouarenotusingDesigner,seedefiningthemetadatausingDDL,youcandefineyoursourcefunctionintheVDBas
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNFUNCTIONSCORE(valinteger)RETURNSinteger;
....(othertables,proceduresetc)
]]>
</metadata>
</model>
</vdb>
Bydefaultwhenasourcecanprovidemetadata,theSourcemodel’smetadataisautomaticallyretrievedfromthesourceiftheywereJDBC,File,WebService.TheFileandWebServicesourcesarestatic,soonecannotaddadditionalmetadataonthem.HoweverontheJDBCsourcesyoucanretrievethemetadatafromsourceandthenusercanappendadditionalmetadataontopofthem.Forexample
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="NATIVE,DDL"><![CDATA[
CREATEFOREIGNFUNCTIONSCORE(valinteger)RETURNSinteger;
]]>
</metadata>
</model>
</vdb>
TheaboveexampleusesNATIVEmetadatatype(NATIVEisthedefaultforsource/physicalmodels)firsttoretrieveschemainformationfromsource,thenusesDDLmetadatatypetoaddadditionalmetadata.OnlymetadatanotavailableviatheNATIVEtranslatorlogicwouldneedtobespecifiedviaDDL.
Alternatively,ifyouareusingcustomMetadataRepositorywithyourVDB,thenprovidethe"function"metadatadirectlyfromyourimplementation.ex.
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="{metadata-repo-module}"></metadata>
</model>
</vdb>
Intheaboveexample,usercanimplementMetadataRepositoryinterfaceandpackagetheimplementationclassalongwithitsdependenciesinaWildFlymoduleandsupplythemodulenameintheaboveXML.FormoreinformationonhowtowriteaMetadataRepositoryrefertoCustomMetadataRepository.
SourceSupportedFunctions
261
SupportforUser-DefinedFunctions(Non-Pushdown)Todefineanon-pushdownfunction,aJavafunctionmustbeprovidedthatmatchestheVDBdefinedmetadata.UserDefinedFunction(orUDF)andUserDefinedAggregateFunction(orUDAF)maybecalledatruntimejustlikeanyotherfunctionoraggregatefunctionrespectively.
MetadatainDesigner
AuserdefinedfunctioncreatedonanyVDBonviewmodelbycreatingaFunctionjustasabasetable.YouwouldrequirealltheinformationdefinedonUserDefinedFunctionstocreateaUDF.MakesureyouprovidetheJAVAcodeimplementationdetailsinthepropertiesdialogfortheUDF.
MetadatawithoutDesigner
WhendefiningthemetadatawithoutDesigner,youcandefineaUDForUDAF(UserDefinedAggregateFunction)asshownbelow.
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="VIRTUAL">
<metadatatype="DDL"><![CDATA[
CREATEVIRTUALFUNCTIONcelsiusToFahrenheit(celsiusdecimal)RETURNSdecimalOPTIONS(JAVA_CLASS'
org.something.TempConv',JAVA_METHOD'celsiusToFahrenheit');
CREATEVIRTUALFUNCTIONsumAll(arginteger)RETURNSintegerOPTIONS(JAVA_CLASS'org.something.Sum
All',JAVA_METHOD'addInput',AGGREGATE'true',VARARGS'true',"NULL-ON-NULL"'true');]]></metadata>
</model>
</vdb>
YoumustcreateaJavamethodthatcontainsthefunction’slogic.ThisJavamethodshouldacceptthenecessaryarguments,whichtheTeiidSystemwillpasstoitatruntime,andfunctionshouldreturnthecalculatedoralteredvalue.
SeeDDLMetadataforallpossibleoptionsrelatedtofunctionsdefinedviaDDL.
WritingtheJavaCoderequiredbytheUDF
ThenumberofinputargumentsandtypesmustmatchthefunctionmetadatadefinedintheVDBmetadata.
CodeRequirementsForUDFs
Thejavaclasscontainingthefunctionmethodmustbedefinedpublic.
Note OneimplementationclasscancontainmorethanoneUDFimplementationmethods.
Thefunctionmethodmustbepublicandstatic.
CodeRequirementsForUDAFs
Thejavaclasscontainingthefunctionmethodmustbedefinedpublicandextendorg.teiid.UserDefinedAggregate
Thefunctionmethodmustbepublic.
OtherConsiderations
Anyexceptioncanbethrown,butTeiidwillrethrowtheexceptionasaFunctionExecutionException.
SupportforUser-DefinedFunctions(Non-Pushdown)
262
Youmayoptionallyaddanadditionalorg.teiid.CommandContextargumentasthefirstparameter.TheCommandContextinterfaceprovidesaccesstoinformationaboutthecurrentcommand,suchastheexecutinguser,Subject,thevdb,thesessionid,etc.ThisCommandContextparametershouldnotbedeclaredinthefunctionmetadata.
SampleUDFcode
packageorg.something;
publicclassTempConv
{
/**
*ConvertsthegivenCelsiustemperaturetoFahrenheit,andreturnsthe
*value.
*@paramdoubleCelsiusTemp
*@returnFahrenheit
*/
publicstaticDoublecelsiusToFahrenheit(DoubledoubleCelsiusTemp)
{
if(doubleCelsiusTemp==null)
{
returnnull;
}
return(doubleCelsiusTemp)*9/5+32;
}
}
SampleUDAFcode
packageorg.something;
publicstaticclassSumAllimplementsUserDefinedAggregate<Integer>{
privatebooleanisNull=true;
privateintresult;
publicvoidaddInput(Integer...vals){
isNull=false;
for(inti:vals){
result+=i;
}
}
@Override
publicIntegergetResult(org.teiid.CommandContextcommandContext){
if(isNull){
returnnull;
}
returnresult;
}
@Override
publicvoidreset(){
isNull=true;
result=0;
}
}
SampleCommandContextUsage
packageorg.something;
publicclassSessionInfo
{
/**
*@paramcontext
SupportforUser-DefinedFunctions(Non-Pushdown)
263
*@returnthecreatedTimestamp
*/
publicstaticTimestampsessionCreated(CommandContextcontext)
{
returnnewTimestamp(context.getSession().getCreatedTime());
}
}
ThecorrespondingUDFwouldbedeclaredasTimestampsessionCreated().
PostCodeActivities
AftercodingthefunctionsyoushouldcompiletheJavacodeintoaJavaArchive(JAR)file.
ZipDeployment
TheJARfilemaybeplacedinyourVDBunderthe"/lib"directory.ItwillautomaticallybeusedfortheVDBclassloaderclasspathwhendeployed.
ASModule
CreateaWildFlymodulewiththeJARfileunder<jboss-as>/modulesdirectoryanddefinethemoduleonthe-vdb.xmlfileasshownbelowexample
<vdbname="{vdb-name}"version="1">
<propertyname="lib"value="{module-name}"></property>
...
</vdb>
Thelibpropertyvaluemaycontainaspacedelimitedlistofmodulenamesifmorethanonedependencyisneeded.
SupportforUser-DefinedFunctions(Non-Pushdown)
264
ArchetypeTemplateUDFProjectOnewaytostartdevelopingacustomuserdefinedfunction(UDF)istocreateaprojectusingtheTeiidUDFarchetypetemplate.Whentheprojectiscreatedfromthetemplate,itwillcreateamavenprojectthatcontainsanexamplejavaclassandtheassemblyresourcesforpackagingasamoduleoraCLIscriptforconfiguringviajboss-cli.
Note Theprojectwillbecreatedasanindependentprojectandhasnoparentmavendependencies.It’sdesignedtobebuiltindependentofbuildingTeiid.
Youhave2optionsforcreatingaUDFproject;inEclipsebycreatinganewmavenprojectfromthearchetypeorbyusingthecommandlinetogeneratetheproject.
CreateProjectinEclipseTocreateaJavaprojectinEclipsefromanarchetype,performthefollowing:
OpentheJAVAperspective
FromthemenuselectFile–>New—>Other
Inthetree,expandMavenandselectMavenProject,pressNext
Onthe"SelectprojectnameandLocation"window,youcanacceptthedefaults,pressNext
Onthe"SelectanArchetype"window,selectConfigurebutton
Addtheremotecatalog:https://repository.jboss.org/nexus/content/repositories/releases/thenclickOKtoreturn
Enter"teiid"inthefiltertoseetheTeiidarchetypes.
Selecttheudf-archetypev9.0.0,thenpressNext
Enteralltheinformation(i.e.,GroupID,ArtifactID,method-name,method-args,return-typeetc.)neededtogeneratetheproject,thenclickFinish
Theprojectwillbecreatedandnameaccordingtothe*ArtifactID*.
CreateProjectusingCommandLine
Tocreateacustomtranslatorprojectfromthecommandline,youcanusethefollowingtemplatecommand:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/rel
eases/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=udf-archetype\
-DarchetypeVersion=9.0.0\
-DgroupId=${groupId} \
-DartifactId=${udf-artifact-id} \
-Dpackage=${packageName}\
-Dversion=0.0.1-SNAPSHOT\
-Dudf-name=${functionName}\
-Dmethod-name=${methodName}\
-Dmethod-args=${methodArguments}\
-Dreturn-type=${returnType}
where:
SupportforUser-DefinedFunctions(Non-Pushdown)
265
-DarchetypeGroupId-isthegroupIDforthearchetypetousetogenerate
-DarchetypeArtifactId-istheartifactIDforthearchetypetousetogenerate
-DarchetypeVersion -istheversionforthearchetypetousetogenerate
-DgroupId -(userdefined)groupIDforthenewudfprojectpom.xml
-DartifactId -(userdefined)artifactIDforthenewudfprojectpom.xml
-Dpackage -(userdefined)thepackagestructurewherethejava,moduleandresourcefileswill
becreated
-Dversion -(userdefined)theversionthatthenewconnectorprojectpom.xmlwillbe
-Dudf-name -(userdefined)thenametogivethenewuserdefinedfunction,willbecometheClass
Name
-Dmethod-name-(userdefined)thenameofthemethodthatwillbeconfiguredinthemodelprocedure
-Dmethod-args-(userdefined)theargumentsthemethodwillaccept.'Typename[,Typename[,...]]
Example:'Stringarg0'or'Stringarg0,integerarg1'
-Dreturn-type-(userdefined)thedatatypeofthevaluereturnedbythemethod
Thefollowingisanexampletoexecute:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/r
eleases/\\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=udf-archetype\
-DarchetypeVersion=9.0.0\
-DgroupId=org.teiid.udf\
-DartifactId=udf-myFunction\
-Dpackage=org.teiid.udf\
-Dversion=0.0.1-SNAPSHOT\
-Dudf-name=myFunction\
-Dmethod-name=myFunction\
-Dmethod-args='Stringarg1'\
-Dreturn-type=String
Whenexecuted,youwillbeaskedtoconfirmtheproperties
[INFO]Archetyperepositorynotdefined.Usingtheonefrom[org.jboss.teiid.arche-types:udf-archetype:9.0.0]f
oundincataloglocal
[INFO]Usingproperty:groupId=org.teiid.udf
[INFO]Usingproperty:artifactId=udf-myFunction
[INFO]Usingproperty:version=0.0.1-SNAPSHOT
[INFO]Usingproperty:package=org.teiid.udf
[INFO]Usingproperty:method-args=Stringarg1
[INFO]Usingproperty:method-name=myFunction
[INFO]Usingproperty:return-type=String
[INFO]Usingproperty:udf-name=myFunction
Confirmpropertiesconfiguration:
groupId:org.teiid.udf
artifactId:udf-myFunction
version:0.0.1-SNAPSHOT
package:org.teiid.udf
method-args:Stringarg1
method-name:myFunction
return-type:String
udf-name:myFunction
Y::y
typeY(yes)andpressenter,andthecreationofthetranslatorprojectwillbedone
Uponcreation,adirectorybasedonthe*artifactId*willbecreated,thatwillcontaintheproject.'cd'intothatdirectoryandexecuteatestbuildtoconfirmtheprojectwascreatedcorrectly:
mvncleaninstall
Thisshouldbuildsuccessfully,andnowyouarereadytostartaddingyourcustomcode.
SupportforUser-DefinedFunctions(Non-Pushdown)
266
SupportforUser-DefinedFunctions(Non-Pushdown)
267
AdminAPIInmostcircumstancestheadminoperationswillbeperformedthroughtheadminconsoleorAdminShelltooling,butitisalsopossibletoinvokeadminfunctionalitydirectlyinJavathroughtheAdminAPI.
AllclassesfortheAdminAPIareintheclientjarundertheorg.teiid.adminapipackage.
Connecting
AnAdminAPIconnection,whichisrepresentedbytheorg.teiid.adminapi.Admininterface,isobtainedthroughtheorg.teiid.adminapi.AdminFactory.createAdminmethods.AdminFactoryisasingletonintheteiid-jboss-adminjar,seeAdminFactory.getInstance().TheAdmininstanceautomaticallytestsitsconnectionandreconnectstoaserverintheeventofafailure.Theclosemethodshouldbecalledtoterminatetheconnection.
SeeyourJBossinstallationfortheappropriateadminport-thedefaultportis9999.
AdminMethodsAdminmethodsexistformonitoring,serveradministration,andconfigurationpurposes.Notethattheobjectsreturnedbythemonitoringmethods,suchasgetRequests,areread-onlyandcannotbeusedtochangeserverstate.SeetheJavaDocsforallofthedetails
AdminAPI
268
CustomLoggingTheTeiidsystemprovidesawealthofinformationusinglogging.Tocontrollogginglevel,contexts,andloglocations,youshouldbefamiliarwithcontainer’sstandalone.xmlordomain.xmlconfigurationfileandcheckout"logging"subsystem.RefertotheAdministrator’sGuideformoredetailsaboutdifferentTeiidcontextsavailable.
Ifyouwantacustomloghandler,followthedirectionstowriteacustomjava.util.logging.Handler.IfyoudevelopacustomloggingHandler,theimplementationclassalongshouldbeplacedasajarin"org.jboss.teiid"moduleanddefineitsnameinthemodule.xmlfileaspartofthemodulealongwithanydependenciesitmayneed.Seebelow.
CommandLoggingAPI
Ifyouwanttobuildacustomhandlerforcommandloggingthatwillhaveaccesstojava.util.loggingLogRecordstothe"COMMAND_LOG"context,thehandlerwillreceiveainstanceofLogRecordmessage,thisobjectwillcontainaparameteroftypeorg.teiid.logging.CommandLogMessage.TherelevantTeiidclassesaredefinedintheteiid-api-9.2.0.Final.jar.TheCommmdLogMessageincludesinformationaboutvdb,session,commandsql,etc.CommandLogMessagesareloggedattheINFO(userqueries),DEBUG(sourcequeries),andTRACE(queryplan)levels.
SampleCommandLogMessageUsage
packageorg.something;
importjava.util.logging.Handler;
importjava.util.logging.LogRecord;
publicclassCommandHandlerextendsHandler{
@Override
publicvoidpublish(LogRecordrecord){
CommandLogMessagemsg=(CommandLogMessage)record.getParameters()[0];
//logtoadatabase,triggeranemail,etc.
}
@Override
publicvoidflush(){
}
@Override
publicvoidclose()throwsSecurityException{
}
}
AuditLoggingAPI
Ifyouwanttobuildacustomhandlerforcommandloggingthatwillhaveaccesstojava.util.loggingLogRecordstothe"AUDIT_LOG"context,thehandlerwillreceiveainstanceofLogRecordmessage,thisobjectwillcontainaparameteroftypeorg.teiid.logging.AuditMessage.TheAuditMessageincludesinformationaboutuser,theaction,andthetarget(s)oftheaction.TherelevantTeiidclassesaredefinedintheteiid-api-9.2.0.Final.jar.AuditMessagesareloggedattheDEBUGlevel.AuditMessagesareusedforbothdatarolevalidationandforlogon/logoffevents.OnlylogoneventswillcontainLogonInfo.
SampleAuditMessageUsage
packageorg.something;
importjava.util.logging.Handler;
importjava.util.logging.LogRecord;
publicclassAuditHandlerextendsHandler{
@Override
CustomLogging
269
publicvoidpublish(LogRecordrecord){
AuditMessagemsg=(AuditMessage)record.getParameters()[0];
//logtoadatabase,triggeranemail,etc.
}
@Override
publicvoidflush(){
}
@Override
publicvoidclose()throwsSecurityException{
}
}
ConfigurationNowthatyouhavedevelopedacustomhandlerclass,nowpackageimplementationinJarfile,thencopythisJarfileinto<jboss-as7>/modules/org/jboss/teiid/mainfolder,andeditmodule.xmlfileinthesamedirectoryandadd
<resource-rootpath="{your-jar-name}.jar"/>
thenusetheclitoupdatetheloggingconfiguration,suchasshownwiththeauditcommandscriptsinthebin/scriptsdirectoryoreditstandalone-teiid.xmlordomain.xmlfilebylocatingthe"logging"subsystemandaddthefollowingentries:
<custom-handlername="COMMAND"class="org.teiid.logging.CommandHandler"
module="org.jboss.teiid">
</custom-handler>
..otherentries
<loggercategory="org.teiid.COMMAND_LOG">
<levelname="DEBUG"/>
<handlers>
<handlername="COMMAND"/>
</handlers>
</logger>
ChangetheaboveconfigurationaccordinglyforAuditHandler,ifyouareworkingwithAuditMessages.
CustomLogging
270
RuntimeUpdatesTeiidsupportsseveralmechanismsforupdatingtheruntimesystem.
DataUpdates
DatachangeeventsareusedbyTeiidtoinvalidateresultsetcacheentries.Resultsetcacheentriesaretrackedbythetablesthatcontributedtotheirresults.BydefaultTeiidwillcaptureinternaldataeventsagainstphysicalsourcesanddistributethemacrossthecluster.Thisapproachhasseverallimitations.FirstupdatesarescopedonlytotheiroriginatingVDB/version.SecondupdatesmadeoutsideofTeiidarenotcaptured.ToincreasedataconsistencyexternalchangedatacapturetoolscanbeusedtosendeventstoTeiid.FromwithinaTeiidclustertheorg.teiid.events.EventDistributorFactoryandorg.teiid.events.EventDistributorcanbeusedtodistributechangeevents.TheEventDistributorFactorycanbelookedupbyitsname"teiid/event-distributor-factory".SeeProgrammaticControlforadataModificationexample.
Whenexternallycapturingallupdateevents,"detect-change-events"propertyintheteiidsubsystemincanbesettofalse,tonotduplicatechangeevents.Bydefault,thispropertyissettotrue.
Note Usingtheorg.teiid.events.EventDistributorinterfaceyoucanalsoupdateruntimemetadata.PleasechecktheAPI.
TheuseoftheotherEventDistributormethodstomanuallydistributeothereventsisnotalwaysnecessary.ChecktheSystemProceduresforSQLbasedupdates.
RuntimeMetadataUpdates
RuntimeupdatesviasystemproceduresandDDLstatementsarebydefaultephemeral.Theyareeffectiveacrosstheclusteronlyforthecurrentlyrunningvdbs.Withthenextvdbstartthevalueswillreverttowhateverisstoredinthevdb.Updatesmaybemadepersistentthoughbyconfiguringaorg.teiid.metadata.MetadataRepository.AninstanceofaMetadataRepositorycanbeinstalledviaVDBfile.InDesignerbasedVDB,youcaneditthevdb.xmlfileintheMETA-INFdirectoryoruseVDBfileasbelow.
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="VIRTUAL">
<metadatatype="{jboss-as-module-name}"></metadata>
</model>
</vdb>
Intheabovecodefragment,replacethe{jboss-as-module-name}withaWildFlymodulenamethathaslibrarythatimplementstheorg.teiid.metadata.MetadataRepositoryinterfaceanddefinesfile"META-INF/services/org.teiid.metadata.MetadataRepository"withnameoftheimplementationfile.
TheMetadataRepositoryrepositoryinstancemayfullyimplementasmanyofthemethodsasneededandreturnnullfromanyunneededgetter.
Itisnotrecommendedtodirectlymanipulateorg.teiid.metadata.AbstractMetadataRecordinstances.SystemproceduresandDDLstatementsshouldbeusedinsteadsincetheeffectswillbedistributedthroughtheclusterandwillnotintroduceinconsistencies.
org.teiid.metadata.AbstractMetadataRecordobjectspassedtotheMetadataRepositoryhavenotyetbeenmodified.IftheMetadataRepositorycannotpersisttheupdate,thenaRuntimeExceptionshouldbethrowntopreventtheupdatefrombeingappliedbytheruntimeengine.
RuntimeUpdates
271
TheMetadataRepositorycanbeaccessedbymultiplethreadsbothduringloadoratruntimewiththroughDDLstatements.Yourimplementationshouldhandleanyneededsynchronization.
CostingUpdates
SeetheReferenceforthesystemproceduresSYSADMIN.setColumnStatsandSYSADMIN.setTableStats.TomakecostingupdatespersistentMetadataRepositoryimplementationsshouldbeprovidedfor:
TableStatsgetTableStats(StringvdbName,StringvdbVersion,Tabletable);
voidsetTableStats(StringvdbName,StringvdbVersion,Tabletable,TableStatstableStats);
ColumnStatsgetColumnStats(StringvdbName,StringvdbVersion,Columncolumn);
voidsetColumnStats(StringvdbName,StringvdbVersion,Columncolumn,ColumnStatscolumnStats);
SchemaUpdates
SeetheReferenceforsupportedDDLstatements.Tomakeschemaupdatespersistentimplementationsshouldbeprovidedfor:
StringgetViewDefinition(StringvdbName,StringvdbVersion,Tabletable);
voidsetViewDefinition(StringvdbName,StringvdbVersion,Tabletable,StringviewDefinition);
StringgetInsteadOfTriggerDefinition(StringvdbName,StringvdbVersion,Tabletable,Table.TriggerEventtrigger
Operation);
voidsetInsteadOfTriggerDefinition(StringvdbName,StringvdbVersion,Tabletable,Table.TriggerEventtriggerOp
eration,StringtriggerDefinition);
booleanisInsteadOfTriggerEnabled(StringvdbName,StringvdbVersion,Tabletable,Table.TriggerEventtriggerOpe
ration);
voidsetInsteadOfTriggerEnabled(StringvdbName,StringvdbVersion,Tabletable,Table.TriggerEventtriggerOpera
tion,booleanenabled);
StringgetProcedureDefinition(StringvdbName,StringvdbVersion,Procedureprocedure);
voidsetProcedureDefinition(StringvdbName,StringvdbVersion,Procedureprocedure,StringprocedureDefinition)
;
LinkedHashMap<String,String>getProperties(StringvdbName,StringvdbVersion,AbstractMetadataRecordrecord);
voidsetProperty(StringvdbName,StringvdbVersion,AbstractMetadataRecordrecord,Stringname,Stringvalue);
RuntimeUpdates
272
CustomMetadataRepositoryIfaboveprovidedmetadatafacilitiesarenotsufficientthenadevelopercanextendtheMetadataRepositoryclassprovidedintheorg.teiid.apijartoplug-intheirownmetadatafacilitiesintotheTeiidengine.Forexample,ausercanwriteametadatafacilitythatisbasedonreadingdatafromadatabaseoraJCRrepository.SeeSettingupthebuildenvironmenttostartdevelopment.ForExample:
SampleJavaCode
importorg.teiid.metadata.MetadataRepository;
...
packagecom.something;
publicclassCustomMetadataRepositoryextendsMetadataRepository{
@Override
publicvoidloadMetadata(MetadataFactoryfactory,ExecutionFactoryexecutionFactory,ObjectconnectionFacto
ry)
throwsTranslatorException{
/*Provideimplementationandfillthedetailsinfactory*/
...
}
}
ThenbuildaJARarchivewithaboveimplementationclassandcreatefileanamedorg.teiid.metadata.MetadataRepositoryintheMETA-INF/servicesdirectorywithcontents:
com.something.CustomMetadataRepository
OncetheJARfilehasbeenbuilt,itneedstobedeployedintheWildFlyasamoduleunder<jboss-as>/modulesdirectory.Followthebelowstepstocreateamodule.
Createadirectory<jboss-as>/modules/com/something/main
Underthisdirectorycreatea"module.xml"filethatlookslike
Samplemodule.xmlfile
<?xmlversion="1.0"encoding="UTF-8"?>
<modulexmlns="urn:jboss:module:1.0"name="com.something">
<resources>
<resource-rootpath="something.jar"/>
</resources>
<dependencies>
<modulename="javax.api"/>
<modulename="javax.resource.api"/>
<modulename="org.jboss.teiid.common-core"/>
<modulename="org.jboss.teiid.teiid-api"/>
</dependencies>
</module>
Copythejarfileunderthissamedirectory.Makesureyouaddanyadditionaldependenciesifrequiredbyyourimplementationclassunderdependencies.
Restarttheserver
ThebelowXMLfragmentshowshowtoconfiguretheVDBwiththecustommetadatarepositorycreated
Samplevdb.xmlfile
CustomMetadataRepository
273
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="{metadata-repo-module}"></metadata>
</model>
</vdb>
NowwhenthisVDBgetsdeployed,itwillcalltheCustomMetadataRepositoryinstanceformetadataofthemodel.UsingthisyoucandefinemetadataforsinglemodelorforthewholeVDBpragmatically.Becarefulaboutholdingstateandsynchronizationinyourrepositoryinstance.
DevelopmentConsiderations
MetadataRepositoryinstancesarecreatedonapervdbbasisandmaybecalledconcurrentlyfortheloadofmultiplemodels.
SeetheMetadataFactoryandtheorg.teiid.metadatapackagejavadocsformetadataconstructionmethodsandobjects.ForexampleifyouuseyourownDDL,thencalltheMetadataFactory.parse(Reader)method.IfyouneedaccesstofilesinaVDBzipdeployment,thenusetheMetadataFactory.getVDBResourcesmethod.
UsetheMetadataFactory.addPermissionandaddMetadataFactory.addColumnPermissionmethodtograntpermissionsonthegivenmetadataobjectstothenamedroles.Therolesshouldbedeclaredinyourvdb.xml,whichisalsowheretheyaretypicallytiedtocontainerroles.
CustomMetadataRepository
274
PreParserIfitisdesirabletomanipulateincomingqueriespriortobeinghandledbyTeiidlogic,thenacustompre-parsercanbeinstalled.UsethePreParserinterfaceprovidedintheorg.teiid.apijartoplug-inapre-parserfortheTeiidengine.SeeSettingupthebuildenvironmenttostartdevelopment.ForExample:
SampleJavaCode
importorg.teiid.PreParser;
...
packagecom.something;
publicclassCustomPreParserimplementsPreParser{
@Override
publicStringpreParse(Stringcommand,CommandContextcontext){
//manipulatethecommand
}
}
ThenbuildaJARarchivewithaboveimplementationclassandcreateafilenamedorg.teiid.PreParserinMETA-INF/servicesdirectorywithcontents:
com.something.CustomPreParser
Thethejarhasbeenbuilt,thisneedstobedeployedintheWildFlyasamoduleunder<jboss-as>/modulesdirectory.Followthebelowstepstocreateamodule.
Createadirectory<jboss-as>/modules/com/something/main
Underthisdirectorycreateamodule.xmlfilethatlookslike
Samplemodule.xmlfile
<?xmlversion="1.0"encoding="UTF-8"?>
<modulexmlns="urn:jboss:module:1.0"name="com.something">
<resources>
<resource-rootpath="something.jar"/>
</resources>
<dependencies>
<modulename="javax.api"/>
<modulename="javax.resource.api"/>
<modulename="org.jboss.teiid.common-core"/>
<modulename="org.jboss.teiid.teiid-api"/>
</dependencies>
</module>
Copythejarfileunderthissamedirectory.Makesureyouaddanyadditionaldependenciesifrequiredbyyourimplementationclassunderdependencies.
Usethecliormodifytheconfigurationtosetthepreparser-moduleintheTeiidsubsystemconfigurationtotheappropriatemodulename.
Restarttheserver
DevelopmentConsiderations
PreParser
275
Changingtheincomingquerytoadifferenttypeofstatementisnotrecommendedasareanymodificationstothenumberortypesofprojectedsymbols.
PreParser
276
ArchetypeTemplatePreParserProjectOnewaytostartdevelopingacustompreparseristocreateaprojectusingtheTeiidarchetypetemplate.Whentheprojectiscreatedfromthetemplate,itwillcontainanexampleclassandresourcesforyoutobeginaddingyourcustomlogic.Additionally,themavendependenciesaredefinedinthepom.xmlsothatyoucanbegincompilingtheclasses.
Note Theprojectwillbecreatedasanindependentprojectandhasnoparentmavendependencies.It’sdesignedtobebuiltindependentofbuildingTeiid.
Youhave2optionsforcreatingatranslatorproject;inEclipsebycreatinganewmavenprojectfromthearchetypeorbyusingthecommandlinetogeneratetheproject.
CreateProjectinEclipseTocreateaJavaprojectinEclipsefromanarchetype,performthefollowing:
OpentheJAVAperspective
FromthemenuselectFile–>New—>Other
Inthetree,expandMavenandselectMavenProject,pressNext
Onthe"SelectprojectnameandLocation"window,youcanacceptthedefaults,pressNext
Onthe"SelectanArchetype"window,selectConfigurebutton
Addtheremotecatalog:https://repository.jboss.org/nexus/content/repositories/releases/thenclickOKtoreturn
Enter"teiid"inthefiltertoseetheTeiidarchetypes.
Selectthepreparser-archetype,thenpressNext
Enteralltheinformation(i.e.,GroupID,ArtifactID,etc.)neededtogeneratetheproject,thenclickFinish
Theprojectwillbecreatedandnameaccordingtothe*ArtifactID*.
CreateProjectusingCommandLine
Tocreateacustompreparserprojectfromthecommandline,youcanusethefollowingtemplatecommand:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/rel
eases/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=preparser-archetype\
-DarchetypeVersion=9.0.0\
-DgroupId=${groupId} \
-DartifactId=${udf-artifact-id} \
-Dpackage=${packageName}\
-Dversion=0.0.1-SNAPSHOT\
-Dclass-name=${className}\
-Dteiid-version${teiidVersion}
where:
-DarchetypeGroupId-isthegroupIDforthearchetypetousetogenerate
-DarchetypeArtifactId-istheartifactIDforthearchetypetousetogenerate
ArchethypeTemplatePreParserProject
277
-DarchetypeVersion -istheversionforthearchetypetousetogenerate
-DgroupId -(userdefined)groupIDforthenewpreparserprojectpom.xml
-DartifactId -(userdefined)artifactIDforthenewpreparserprojectpom.xml
-Dpackage -(userdefined)thepackagestructurewherethejava,moduleandresourcefileswill
becreated
-Dversion -(userdefined)theversionthatthenewpreparserprojectpom.xmlwillbe
-Dclass-name -(userdefined)theclassnametogivethenewuserpreparser,willbecometheClass
Name
-Dteiid-version-Optional,defaultsto9.0.0.Final
EXAMPLE
thisisanexampleofthetemplatethatcanberun:
mvnarchetype:generate-DarchethypeRepository=https://repository.jboss.org/nexus/content/repositories/rel
eases/\
-DarchetypeGroupId=org.jboss.teiid.arche-types\
-DarchetypeArtifactId=preparser-archetype\
-DarchetypeVersion=9.0.0\
-DgroupId=org.teiid.preparser\
-DartifactId=preparser-myParser\
-Dpackage=org.teiid.preparser\
-Dversion=0.0.1-SNAPSHOT\
-Dclass-name=myPreParser
Whenexecuted,youwillbeaskedtoconfirmthepackageproperty
[INFO]Usingproperty:groupId=org.teiid.preparser
[INFO]Usingproperty:artifactId=preparser-myParser
[INFO]Usingproperty:version=0.0.1-SNAPSHOT
[INFO]Usingproperty:package=org.teiid.preparser
[INFO]Usingproperty:class-name=myPreParser
[INFO]Usingproperty:teiid-version=9.0.0.Final
Confirmpropertiesconfiguration:
groupId:org.teiid.preparser
artifactId:preparser-myParser
version:0.0.1-SNAPSHOT
package:org.teiid.preparser
class-name:myPreParser
teiid-version:9.0.0.Final
Y::y
typeY(yes)andpressenter,andthecreationofthepreparserprojectwillbedone
Uponcreation,adirectorybasedonthe*artifactId*willbecreated,thatwillcontaintheproject.'cd'intothatdirectoryandexecuteatestbuildtoconfirmtheprojectwascreatedcorrectly:
mvncleaninstall
Thisshouldbuildsuccessfully,andnowyouarereadytostartaddingyourcustomcode.
ArchethypeTemplatePreParserProject
278
EmbeddedGuideEmbeddedisalight-weightversionofTeiidforuseinanyJava7+JRE.WildFlynoranyapplicationserverisnotrequired.Thisfeature/kitarestillevolving.PleaseconsultthesourceexamplesandevenunittestsutilizingtheEmbeddedServerforamorecompleteguideastoitsuse.
TableofContentsConfigurationTheClasspath
EmbeddedUsingMavenOSGIMavenVulnerableLibraries
VDBDeploymentAccessfromclientapplicationsSecurity
ExampleTransactionsAdminApiLoggingOtherDifferencesBetweenTeiidEmbeddedandanASDeployment
Configuration
TheprimarywaytoconfigureTeiidEmbeddediswiththeEmbeddedConfigurationclass.ItisprovidedtotheEmbeddedServeratstart-upanddictatesmuchofthebehavioroftheembeddedinstance.FromtheretherunningserverinstancemayhavetranslatorsandVDBsdeployedasneeded.AdditionalmodificationstotheEmbeddedConfigurationaftertheserverisstartedwillnothaveaneffect.
InmanycasesanEmbeddedConfigurationinstancecanjustbeinstantiatedandpassedtotheEmbeddedServerwithouttheneedtosetadditionalproperties.Manyproperties,includingthoseusedtoconfiguretheBufferManager,willbegivenasimilarnametotheirserversidecounterpart-forexamplesetProcessorBatchSize.
ImportantMostofthedefaultconfigurationvaluesformemoryandthreadsassumethatthereisonlyoneTeiidinstanceinthevm.IfyouareusingmultipleTeiidEmbeddedinstancesinthesamevm,thenmemoryandthreadresourcesshouldbeconfiguredmanually.
TheClasspath
EmbeddedUsingMaven
YourapplicationisresponsibleforhavingtheappropriateclasspathtoutilizeTeiidembedded.TypicallyyouwillwantalltransitivedependenciesfromreferencedTeiidartifactstobeincluded.Optionaldependencies,suchasHibernatecore4.1.6orcompatible,willbeneededforspecificfeatures-suchasutilizingtheJDBCtranslatorsupportfordependentjoinsusingtemptables.
SomeoftheTeiidtransitivedependencieshaveknownvulnerabilities.WildFly/Teiidaddressesthisbyintroducingmanageddependencyoverrides.ItisrecommendedthatyouincludetheseoverridesinyourusageofTeiidEmbeddedbyimportingtheTeiidparentpominyourdepedencymanagementsection:
EmbeddedGuide
279
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-parent</artifactId>
<version>${version.teiid}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
OSGI
AllTeiidjarscanalsobedeployedasbundlesinaOSGIcontainerlikeKaraf.IfyouareworkingwithKaraf,afeature.xmlfileisavailableinmavenrepoforyourconvenience.Usagepatternisbelow
features:addurlmvn:org.jboss.teiid/teiid/9.1.0.Final/xml/karaf-features
features:install-vteiid
Maven
IfyouaretryingrunTeiddEmbeddedwithMavenbasedprojectandusingmaventopullartifacts,theruntime,admin,connector,translatordependenciesarenecessaryasbelow
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-runtime</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-admin</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.teiid.connectors</groupId>
<artifactId>translator-SOURCE</artifactId>
</dependency>
<dependency>
<groupId>org.jboss.teiid.connectors</groupId>
<artifactId>connector-SOURCE</artifactId>
</dependency>
VulnerableLibraries
PriortoTeiid8.13/8.12.2whenusingtheremoteJDBCtransport,Teiidembeddedcouldbesusceptibletodeserializationexploitsifitalsoincludedmostversionsofcommon-collections,olderversionofgroovy,orspringintheclasspath-seealsothispostingformoredetailsontheaffectedlibraries.
VDBDeployment
VDBsmaybedeployedinseveralwaysinEmbedded.
VDBMetadataAPI
VDBdeploymentcanbedonedirectlythroughVDBmetadataobjectsthataretheunderpinningofvdb.xmldeployment.Models(schemas)aredeployedasasettoformanamedvdb-seetheEmbeddedServer.deployVDBmethod.
EmbeddedGuide
280
XMLDeployment
Similartoaserverbased-vdb.xmldeploymentanInputStreammaybegiventoavdb.xmlfile-seetheEmbeddedServer.deployVDB(InputStream)method.
ZipDeployment
Similartoaserverbased.vdbdeploymentaURLmaybegiventoazipfile-seetheEmbeddedServer.deployVDBZipmethod.TheuseoftheziplibfordependencyloadingisnotenabledinEmbedded.
SeeVDBGuideandMetadataRepositoriesformoreonatypicalvdbfileandzipstructures.TeiidDesigner7andlaterVDBsarealsosupported,butaresubjecttoallofthelimitations/differenceshighlightedinthisguide.
Translators
TranslatorsinstancescanbescopedtoaVDBinASusingdeclarationsinavdb.xmlfile,howevernamedinstancesinembeddedarescopedtotheentireEmbeddedServerandmustberegisteredviatheEmbeddedServer.addTranslatormethods.NotethattherearethreeaddTranslatormethods:
addTranslator(Class<?extendsExecutionFactory>clazz)-AddsadefaultinstanceoftheExecutionFactory,usingthedefaultnameeitherfromtheTranslatorannotationortheclassname.
addTranslator(Stringname,ExecutionFactory<?,?>ef)-Addsapre-initialized(ExecutionFactory.start()musthavealreadybeencalled)instanceoftheExecutionFactory,usingthegiventranslatorname.Theinstancewillbesharedforallusage.
addTranslator(Stringname,Stringtype,Map<String,String>properties)-Addsadefinitionofanoverridetranslator-thisisfunctionallyequivalenttousingavdb.xmltranslatoroverride.
Anewserverinstancedoesnotassumeanytranslatorsaredeployedanddoesnotperformanysortoflibraryscanningtofindtranslators.
Sources
TheEmbeddedServerwillstillattempttolookupthegivenJNDIconnectionfactorynamesviaJNDI.Inmostnon-containerenvironmentsitislikelythatnosuchbindingsexist.InthiscasetheEmbeddedServerinstancemusthaveConnectionFactoryProviderinstancesmanuallyregistered,eitherusingtheEmbeddedServer.addConnectionFactorymethod,ortheEmbeddedServer.addConnectionFactoryProvidermethodtoimplementConnectionFactoryProviderregistering.NotethattheEmbeddedServerdoesnothavebuilt-inpoolinglogic,sotomakebetteruseofastandardjava.sql.DataSourceortoenableproperuseofjavax.sql.XADataSourceyoumustfirstconfiguretheinstanceviaathird-partyconnectionpool.
Example-Deployment
EmbeddedServeres=newEmbeddedServer();
EmbeddedConfigurationec=newEmbeddedConfiguration();
//setanyconfigurationproperties
ec.setUseDisk(false);
es.start(ec);
//exampleofaddingatranslatorbypre-initializedExecutionFactoryandgiventranslatorname
H2ExecutionFactoryef=newH2ExecutionFactory()
ef.setSupportsDirectQueryProcedure(true);
ef.start();
es.addTranslator("translator-h2",ef);
//addaConnectionFactorywithathird-partyconnectionpool
DataSourceds=EmbeddedHelper.newDataSource("org.h2.Driver","jdbc:h2:mem://localhost/~/account","sa","sa");
es.addConnectionFactory("java:/accounts-ds",ds);
//addavdb
//physicalmodel
ModelMetaDatammd=newModelMetaData();
EmbeddedGuide
281
mmd.setName("my-schema");
mmd.addSourceMapping("my-schema","translator-h2","java:/accounts-ds");
//virtualmodel
ModelMetaDatammd1=newModelMetaData();
mmd1.setName("virt");
mmd1.setModelType(Type.VIRTUAL);
mmd1.setSchemaSourceType("ddl");
mmd1.setSchemaText("createview\"my-view\"OPTIONS(UPDATABLE'true')asselect*from\"my-table\"");
es.deployVDB("test",mmd,mmd1);
SecuredDataSources
IfSourcerelatedsecurityauthentication,forexample,ifyouwantconnect/federate/integrateTwittersuppliedrestsource,asecurityauthenticationisanecessary,thefollowingstepscanusetoexecutesecurityauthentication:
1. refertoSecureEmbeddedwithPicketBoxstartsectiontodevelopaSubjectFactory,
2. initializeaConnectionManagerwithironjacamarlibaries,setSubjectFactorytoConnectionManager
3. usethefollowingmethodtocreateConnectionFactory
Example-SecuredDataSources
WSManagedConnectionFactorymcf=newWSManagedConnectionFactory();
NoTxConnectionManagerImplcm=newNoTxConnectionManagerImpl();
cm.setSecurityDomain(securityDomain);
cm.setSubjectFactory(newEmbeddedSecuritySubjectFactory(authConf))
ObjectconnectionFactory=mcf.createConnectionFactory(cm);
server.addConnectionFactory("java:/twitterDS",connectionFactory);
twitter-as-a-datasourceisacompletedexample.
AccessfromclientapplicationsTypicallywhenTeiidisdeployedasEmbeddedServer,andifyourenduserapplicationisalsodeployedinthesamevirtualmachineastheTeiidEmbedded,youcanuseLocalJDBCConnection,toaccesstoyourvirtualdatabase.Forexample:
Example-LocalJDBCConnection
EmbeddedServeres=...
Driverdriver=es.getDriver();
Connectionconn=driver.connect("jdbc:teiid:<vdb-name>",null);
//doworkwithconn;createstatementandexecuteit
conn.close();
Thisisthemostefficientmethodasitdoesnotimposeanyserializationofobjects.
IfyourclientapplicationisdeployedinremoteVM,oryourclientapplicationisnotaJAVAbasedapplicationthenaccessestotheTeiidEmbeddedisnotpossiblethroughabovemechanism.Inthosesituations,youneedtoopenasocketbasedconnectionfromremoteclientapplicationtotheEmbeddedTeiidServer.Bydefault,whenyoustarttheEmbeddedTeiidSeveritdoesnotaddanycapabilitiestoacceptremoteJDBC/ODBCbasedconnections.IfyouwouldliketoexposethefunctionalitytoacceptremoteJDBC/ODBCconnectionrequests,thenconfigurenecessarytransportsduringtheinitializationoftheTeiidEmbeddedServer.TheexamplebelowshowsasamplecodetoenableaODBCtransport
Example-RemoteODBCtransport
EmbeddedServeres=newEmbeddedServer()
SocketConfigurations=newSocketConfiguration();
s.setBindAddress("<host-name>");
EmbeddedGuide
282
s.setPortNumber(35432);
s.setProtocol(WireProtocol.pg);
EmbeddedConfigurationconfig=newEmbeddedConfiguration();
config.addTransport(s);
es.start(config);
Example-SSLtransport
EmbeddedServerserver=newEmbeddedServer();
...
EmbeddedConfigurationconfig=newEmbeddedConfiguration();
SocketConfigurationsocketConfiguration=newSocketConfiguration();
SSLConfigurationsslConfiguration=newSSLConfiguration();
//Settingsshownwiththeirdefaultvalues
//sslConfiguration.setMode(SSLConfiguration.ENABLED);
//sslConfiguration.setAuthenticationMode(SSLConfiguration.ONEWAY);
//sslConfiguration.setSslProtocol(SocketUtil.DEFAULT_PROTOCOL);
//sslConfiguration.setKeymanagementAlgorithm(KeyManagerFactory.getDefaultAlgorithm());
//optionallyrestricttheciphersuites
//sslConfiguration.setEnabledCipherSuites("SSL_RSA_WITH_RC4_128_MD5,SSL_RSA_WITH_RC4_128_SHA");
//fortheserverkey
sslConfiguration.setKeystoreFilename("ssl-example.keystore");
sslConfiguration.setKeystorePassword("redhat");
sslConfiguration.setKeystoreType("JKS");
sslConfiguration.setKeystoreKeyAlias("teiid");
sslConfiguration.setKeystoreKeyPassword("redhat");
//fortwowaysslsetatruststoreforclientcerts
//sslConfiguration.setTruststoreFilename("ssl-example.truststore");
//sslConfiguration.setTruststorePassword("redhat");
socketConfiguration.setSSLConfiguration(sslConfiguration);
config.addTransport(socketConfiguration);
server.start(config);
ifyouwanttoaddaJDBCtransport,followtheinstructionsabove,howeversettheprotocoltoWireProtocol.teiidandchooseadifferentportnumber.Oncetheaboveserverisrunning,youcanusesameinstructionsasTeiidServertoaccessEmbeddedTeiidServerfromremoteclientapplication.NotethatyoucanaddmultipletransportstosingleEmbeddedServerinstance,toexposedifferenttransports.
SecurityTheprimaryinterfaceforTeiidembedded’ssecurityistheorg.teiid.security.SecurityHelperintheenginejar.TheSecurityHelperinstanceisassociatedwithwiththeEmbeddedServerviaEmbeddedConfiguration.setSecurityHelper.IfnoSecurityHelperisset,thennoauthenticationwillbeperformed.ASecurityHelpercontrolsauthenticationandassociatesasecuritycontextwithathread.Howasecuritycontextisobtainedcandependuponthesecuritydomainname.Thedefaultsecuritydomainnameisteiid-securityandcanbechangedviaEmbeddedConfiguration.setSecurityDomain.TheeffectivesecuritydomainmayalsobeconfiguredviaatransportoftheVDB.
SeetheJBossSecurityHelpersourceforanexampleofexpectedmechanics.
YoucanjustreturnnullfromnegotiateGssLoginunlessyouwanttoallGSSauthenticationsfromJDBC/ODBC.
Example
EmbeddedGuide
283
embedded-portfolio-securitydemonstrateshowtoimplementsecurityauthenticationinTeiidEmbedded:
EmbeddedSecurityHelperistheimplementationoforg.teiid.security.SecurityHelper
users.propertiesandroles.propertiesinclasspathusertopredefineusersandroles
application-policy’snameinauthentication.confshouldmatchtosecuritydomain(EmbeddedConfiguration.setSecurityDomain)
Transactions
TransactionprocessingrequiressettingtheTransactionManagerintheEmbeddedConfigurationusedtostarttheEmbeddedServer.Aclientfacingjavax.sql.DataSourceisnotprovidedforembedded.Howevertheusageofprovidedjava.sql.Drivershouldbesufficientastheembeddedserverisbydefaultabletodetectthreadboundtransactionsandappropriatelypropagatethetransactiontothreadslaunchedaspartofrequestprocessing.Theusageoflocalconnectionsisalsopermitted.
AdminApi
EmbeddedprovidesatheAdmininterfaceviatheEmbeddedServer.getAdminmethod.Notallmethodsareimplementedforembedded-forexamplethosethatdealwithdatasources.AlsothedeploymethodmayonlydeployVDBxmlartifacts.
Logging
TeiidbydefaultuseJBossLogging,whichwillutilizeJUL(JavaUtilLogging)orothercommonloggingframeworksdependingupontheirpresenceintheclasspath.RefertoLogginginTeiidEmbeddedfordetails.
TheinternalinterfaceforTeiidembedded’sloggingisorg.teiid.logging.Loggerinteiid-apijar.TheLoggerinstanceisassociatedwiththeorg.teiid.logging.LogManagerviastaticmethodLogManager.setLogListener().YoumayalternativelychoosetodirectlysetaLoggerofyourchoice.
OtherDifferencesBetweenTeiidEmbeddedandanASDeployment
ThereisnodefaultJDBC/ODBCsockettransportinembedded.YouareexpectedtoobtainaDriverconnectionviatheEmbeddedServer.getDrivermethod.IfyouwantremoteJDBC/ODBCtransportseeaboveonhowtoaddatransport.
AMetadataRepositoryisscopedtoaVDBinAS,butisscopedtotheentireEmbeddedServerinstanceandmustberegisteredviatheEmbeddedServer.addMetadataRepositorymethod.
MDCloggingvaluesarenotavailableasJavalogginglackstheconceptofamappeddiagnosticcontext.
Translatoroverridesinvdb.xmlfilesisnotsupported.
Thelegacyfunctionmodelisnotsupported.
EmbeddedGuide
284
LogginginTeiidEmbeddedTeiid’sLogManagerisaninterfacetoasingleloggingframeworkthatiseasilyaccessiblebyanycomponent.UsingtheLogManager,acomponentcanquicklysubmitalogmessage,andcanrelyupontheLogManagertodetermine
whetherthatmessageistoberecordedordiscarded
wheretosendanyrecordedmessages
JBossLoggingJBossLoggingisusedbydefault.TheJBossLoggingjarisalreadyinthekitandyoujustneedtoensurethejboss-logginglibraryisinyourclasspath.IfyouuseMaven,addthedependencyasshownbelow:
<dependency>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-loggging</artifactId>
</dependency>
BridgingwithJBossLoggingJBossLogManagerisareplacementfortheJDKloggingsystemLogManagerthatfixesorworksaroundmanyseriousproblemsinthedefaultimplementation.TouseJBossLogManagerwithJBossLogging,theonlyneedtodoisaddjboss-logmanagerlibrarytoclasspath.IfuseMaventopulldependencies,addthedependencyasshownbelow:
<dependency>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logmanager</artifactId>
</dependency>
TeiidEmbeddedLoggingisaexampleforLoggingwithJBossLogManager.
Asamplelogging.propertiesforTeiidEmbedded:
loggers=sun.rmi,com.arjuna
logger.level=TRACE
logger.handlers=FILE,CONSOLE
logger.sun.rmi.level=WARN
logger.sun.rmi.useParentHandlers=true
logger.com.arjuna.level=WARN
logger.com.arjuna.useParentHandlers=true
handler.CONSOLE=org.jboss.logmanager.handlers.ConsoleHandler
handler.CONSOLE.level=INFO
handler.CONSOLE.formatter=COLOR-PATTERN
handler.CONSOLE.properties=autoFlush,target,enabled
handler.CONSOLE.autoFlush=true
LogginginTeiidEmbedded
285
handler.CONSOLE.target=SYSTEM_OUT
handler.CONSOLE.enabled=true
handler.FILE=org.jboss.logmanager.handlers.PeriodicRotatingFileHandler
handler.FILE.formatter=PATTERN
handler.FILE.properties=append,autoFlush,enabled,suffix,fileName
handler.FILE.constructorProperties=fileName,append
handler.FILE.append=true
handler.FILE.autoFlush=true
handler.FILE.enabled=true
handler.FILE.suffix=.yyyy-MM-dd
handler.FILE.fileName=target/teiid-embedded.log
formatter.PATTERN=org.jboss.logmanager.formatters.PatternFormatter
formatter.PATTERN.properties=pattern
formatter.PATTERN.pattern=%d{yyyy-MM-ddHH\:mm\:ss,SSS}%-5p\[%c\](%t)%s%e%n
formatter.COLOR-PATTERN=org.jboss.logmanager.formatters.PatternFormatter
formatter.COLOR-PATTERN.properties=pattern
formatter.COLOR-PATTERN.pattern=%K{level}%d{HH\:mm\:ss,SSS}%-5p\[%c\](%t)%s%e%n
BridgingwithLog4jTobridgeJBossLoggingwithLog4j,theonlyneedtodoishavea1.xlog4jjarinyourclasspath.
IfyoursystemuseLog4jasloggingframework,withaboveJBossLogManagerbridgeLog4jfunctionalityandstepsinBridgingwithJBossLogging,it’seasytosetuploggingframeworkconsistentbetweenTeiidEmbeddedandyoursystem.
LogginginTeiidEmbedded
286
SecureEmbeddedwithPicketBoxSecureEmbeddedwithPicketBox.
TableofContentsStepsofimplementaJAASauthenticationHowtodevelopaSecurityHelperEmbeddedSecuritywithUsersRolesLoginModuleEmbeddedSecuritywithLdapExtLoginModule
StepsofimplementaJAASauthentication
PicketBoxisaJavaSecurityFrameworkthatbuildontopofJAAS.PicketBoxisconfiguredviaaschemaformattedSecurityConfigurationFile(security-config_5_0.xsd)andprovidesvariousLoginModuleImplementations(UsersRolesLoginModule,LdapExtLoginModule,DatabaseServerLoginModule,etc).Thefollowingare5keystepstoexecuteaauthentication:
//1.establishtheJAASConfigurationwithpicketboxauthenticationxmlfile
SecurityFactory.prepare();
//2.loadpicketboxauthenticationxmlfile
PicketBoxConfigurationconfig=newPicketBoxConfiguration();
config.load(SampleMain.class.getClassLoader().getResourceAsStream("picketbox/authentication.conf"));
//3.getAuthenticationManager
AuthenticationManagerauthManager=SecurityFactory.getAuthenticationManager(securityDomain);
//4.executeauthentication
authManager.isValid(userPrincipal,credString,subject);
//5.releaseresource
SecurityFactory.release();
TeiidEmbeddedexposes2methodsforsecurityauthentication:
EmbeddedConfiguration.setSecurityHelper()-associatedwithaorg.teiid.security.SecurityHelperintheenginejar.IfnoSecurityHelperisset,thennoauthenticationwillbeperformed.
EmbeddedConfiguration.setSecurityDomain()-associatedwithaapplication-policy’snameinSecurityConfigurationfile.IfnoSecurityDomainisset,thenadefaultteiid-securitywillbeused.
EmbeddedSecurityHelperisasampleimplementationofSecurityHelper,authentication.confisasampleSecurityConfigurationfile.
HowtodevelopaSecurityHelper
Add’teiid-engine-VERSION.jar’toclasspathisnecessary.Ifyouareusingthemaventopullartifacts,theenginedependencycanaddedasbelow,
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-engine</artifactId>
</dependency>
SecureEmbeddedwithPicketBox
287
ThekeytodevelopaSecurityHelperisimplementtheauthenticate()method.PicketBox’s5keystepstoexecuteanauthenticationwhichdepictedinStepsofimplementaJAASauthenticationisshownintheexamplebelow:
@Override
publicSecurityContextauthenticate(StringsecurityDomain,StringbaseUserName,Credentialscredentials,String
applicationName)throwsLoginException{
SecurityFactory.prepare();
try{
PicketBoxConfigurationconfig=newPicketBoxConfiguration();
config.load(this.getClass().getClassLoader().getResourceAsStream("picketbox/authentication.conf"));
AuthenticationManagerauthManager=SecurityFactory.getAuthenticationManager(securityDomain);
if(authManager!=null){
finalPrincipaluserPrincipal=newSimplePrincipal(baseUserName);
finalSubjectsubject=newSubject();
finalStringcredString=credentials==null?null:newString(credentials.getCredentialsAsCharArray()
);
finalStringdomain=securityDomain;
booleanisValid=authManager.isValid(userPrincipal,credString,subject);
if(isValid){
SecurityContextsecurityContext=AccessController.doPrivileged(newPrivilegedAction<SecurityCo
ntext>(){
@Override
publicSecurityContextrun(){
SecurityContextsc;
try{
sc=SecurityContextFactory.createSecurityContext(userPrincipal,credString,subjec
t,domain);
}catch(Exceptione){
thrownewRuntimeException(e);
}
returnsc;
}});
returnsecurityContext;
}
}
}finally{
SecurityFactory.release();
}
thrownewLoginException("Theusername"+baseUserName+"and/orpasswordcouldnotbeauthenticatedby
securitydomain"+securityDomain+".");
}
YoucanjustreturnnullfromnegotiateGssLoginunlessyouwanttoallGSSauthenticationsfromJDBC/ODBC.
EmbeddedSecuritywithUsersRolesLoginModuleAddthefollowingcontenttoPicketBoxSecurityConfigurationfile:
<application-policyname="teiid-security">
<authentication>
<login-modulecode="org.jboss.security.auth.spi.UsersRolesLoginModule"flag="required"></login-modu
le>
</authentication>
</application-policy>
Toprepareusers/rolesbyaddusers.propertiesandroles.propertiestoclasspath.Asampleofusers.properties
testUser=password
SecureEmbeddedwithPicketBox
288
Asampleofroles.properties
testUser=user
TostartEmbeddedServerwithUsersRolesLoginModulebasedsecurityauthenticationvia:
EmbeddedServerserver=
...
EmbeddedConfigurationconfig=newEmbeddedConfiguration();
config.setSecurityDomain("teiid-security-file");
config.setSecurityHelper(newEmbeddedSecurityHelper());
server.start(config);
EmbeddedSecuritywithLdapExtLoginModuleAddthefollowingcontenttothePicketBoxSecurityConfigurationFile:
<application-policyname="teiid-security-ldap">
<authentication>
<login-modulecode="org.jboss.security.auth.spi.LdapExtLoginModule"flag="required">
<module-optionname="java.naming.factory.initial">com.sun.jndi.ldap.LdapCtxFactory</module-option>
<module-optionname="java.naming.provider.url">ldap://HOST:389</module-option>
<module-optionname="java.naming.security.authentication">simple</module-option>
<module-optionname="bindDN">cn=Manager,dc=example,dc=com</module-option>
<module-optionname="bindCredential">redhat</module-option>
<module-optionname="baseCtxDN">ou=Customers,dc=example,dc=com</module-option>
<module-optionname="baseFilter">(uid={0})</module-option>
<module-optionname="rolesCtxDN">ou=Roles,dc=example,dc=com</module-option>
<module-optionname="roleFilter">(uniqueMember={1})</module-option>
<module-optionname="roleAttributeID">cn</module-option>
</login-module>
</authentication>
</application-policy>
Todefinesecurityusers/rolesrefertoyourLDAPVendorsdocumentation.Forexample,ifyouuseOpenLDAP,thenwiththeldiffilecustomer-security.ldif,execute
ldapadd-x-D"cn=Manager,dc=example,dc=com"-wredhat-fcustomer-security.ldif
tosetupusers/roles.
Tip module-optionssettinglikeurl,bindDN,bindCredential,baseCtxDN,rolesCtxDNshouldmatchtoyourLDAPserversetting.
TostartEmbeddedServerwithLdapExtLoginModulebasedsecurityauthenticationvia:
EmbeddedServerserver=
...
EmbeddedConfigurationconfig=newEmbeddedConfiguration();
config.setSecurityDomain("teiid-security-ldap");
config.setSecurityHelper(newEmbeddedSecurityHelper());
server.start(config);
SecureEmbeddedwithPicketBox
289
SecureEmbeddedwithPicketBox
290
ReferenceGuideTeiidoffersahighlyscalableandhighperformancesolutiontoinformationintegration.ByallowingintegratedandenricheddatatobeconsumedrelationallyorasXMLovermultipleprotocols,Teiidsimplifiesdataaccessfordevelopersandconsumingapplications.
Commercialdevelopmentsupport,productionsupport,andtrainingforTeiidisavailablethroughJBossInc.TeiidisaProfessionalOpenSourceprojectandacriticalcomponentoftheJBossEnterpriseDataServicesPlatform.
BeforeonecandelveintoTeiiditisveryimportanttolearnfewbasicconstructsofTeiid,likewhatisVDB?whatisModel?etc.Forthatpleasereadtheshortintroductionherehttp://teiid.jboss.org/basics/
ReferenceGuide
291
DataSourcesTeiidprovidesthemeans(i.e.,TranslatorsandJEEconnectors)toaccessavarietyoftypesofdatasources.
Thetypesofdatasourcesthatarecurrentlyaccessibleare:DatabasesWebServicesODataBigData/NoSQL/SearchEngines/JCRandOtherSourcesEnterpriseSystemsObjectSourcesLDAPFilesSpreadsheets
Databases
SeeJDBCTranslatorsforaccessto:
Oracle
PostgreSQL
MySQL/MariaDB
DB2
MicrosoftSQLServer
Sybase
SybaseIQ
MicrosoftAccess
Derby
H2
HSQL
Ingres
Informix
MetaMatrix
Teradata
Vertica
GenericANSISQL-fortypicalJDBC/ODBCsources
SimpleSQL-foranyJDBC/ODBCsource
WebServices
DataSources
292
SeeWebServicesTranslatorforaccessto:
SOAP
REST
ArbitraryHTTP(S)
OData
SeetheODataTranslator
BigData/NoSQL/SearchEngines/JCRandOtherSources
ActianVector
AmazonSimpleDB
ApacheAccumulo
ApacheCassandraDB
ApacheSOLR
ApacheSpark
Greenplum
Hive/Hadoop/AmazonElasticMapReduce
Impala/Hadoop/AmazonElasticMapReduce
ModeShapeJCRRepository
MongoDB
MondrianOLAP
Netezzadatawarehouseappliance
Phoenix/HBase
PrestoDB
Redshift
EnterpriseSystems
OSISoftPI
SalesForce
SAPGateway
SAPHana
Teiid
ObjectSources
DataSources
293
JDG/InfinispanLibraryMode
JDG/InfinispanHotRodMode
IntersystemsCacheObjectDatabase
JPAsources
LDAP
SeetheLDAPTranslatorforaccessto:
RedHatDirectoryServer
ActiveDirectory
Files
SeetheFileTranslatorforusewith:
Delimited/Fixedwidth
XML
SpreadsheetsExcel
GoogleSpreadsheet
Thisrepresentsdatasourcesthathavebeenvalidatedtoworkusingtheavailabletranslatorsandconnectors.However,thisdoesnotprecludeanewdatasourcefromworking.Itcanbeaseasyasextendinganexistingtranslator,tocreatinganewtranslatorusingtheTranslatorDevelopmentextensions.
TakealookatthelistofTranslatorsthatareusedasthebridgebetweenTeiidandtheexternalsystem.
DataSources
294
VirtualDatabasesAvirtualdatabase(orVDB)isametadatacontainerforcomponentsusedtointegratedatafrommultipledatasources,sothattheycanbeaccessedinanintegratedmannerthroughasingle,uniformAPI.
AVDBtypicallycontainsmultipleschemacomponents(alsocalledasmodels),andeachschemacontainsthemetadata(tables,procedures,functions).Therearetwo(2)differenttypesofschemas
SourceSchema(alsocalledPhysicalorForeignschema),whichrepresentsanexternal/remotedatasourceslikeRelationaldatabase(Oracle,DB2,MySQL..),Files(CSV,Excel..),Web-Services(SOAP,REST)etc.
VirtualSchema.Thisisaviewlayerorlogicalschemalayer,thatisdefinedusingschemaelementsfromForeignSchemas.Forexample,creatingaviewtableusingmultipleforeigntablesfromdifferentsources,thushidingthecomplexitiesofdefinitionoftheviewfromuser.
Oneimportantthingtonoteis,aVDBONLYcontainsmetadata,NEVERcopies/hastheactualdata.AnyusecaseinvolvingTeiidMUSThaveaVDBtobeginwith.So,itisveryimportanttolearnhowaVDBcanbedesigned/developed.
BelowisanexampleVDB,thatisusingasingleforeignschemacomponentdefiningaconnectiontoPostgreSQLdatabase.
Example:1
<vdbname="my-example"version="1">
<modelname="test"type="PHYSICAL">
<propertyname="importer.schemaPattern"value="public"/>
<propertyname="importer.useFullSchemaName"value="false"/>
<propertyname="importer.tableTypes"value="TABLE,VIEW"/>
<sourcename="pqsql"translator-name="postgresql"connection-jndi-name="java:/postgres-ds"/>
</model>
</vdb>
AnothervariationoftheVDBusingcompletelyDDLandusingSQL-MEDspecification.
Example:2
CREATEDATABASEmy_exampleVERSION'1.0.0';
USEDATABASEmy_exampleVERSION'1.0.0'
CREATEFOREIGNDATAWRAPPERpostgresql;
CREATESERVERpgsqlTYPE'postgresql-9.4-1201.jdbc41.jar'
VERSION'one'FOREIGNDATAWRAPPERpostgresql
OPTIONS(
"jndi-name"'java:/postgres-ds'
);
CREATESCHEMAtestSERVERpgsql;
VirtualDatabases
295
IMPORTFOREIGNSCHEMApublicFROMSERVERpgsqlINTOtest
OPTIONS(
importer.useFullSchemaNamefalse,
importer.tableTypes'TABLE,VIEW'
);
BothformatsdefinethesameVDB.
Thereislottobeexplainedfromaboveexamples,inthefollowingsections,wewillgointodetailabouteachofthoselines.BeforethatweneedtolearnaboutfurtherfractionsintheSourceSchemacomponent.
ExternalDataSourcesA"sourceschema"componentinVDBasshowninaboveexampleisacollectionschemaelementsastables,proceduresandfunctionsthatrepresentanexternaldatasource’smetadatalocally.Howeverintheaboveexample,itdidnotdefineanysuchschemaelements,howeverdetailsofconnectiontotheexternaldatasourcewereprovidedthrough"jndi-name",whichisanamedconnectionreferencetoaexternaldatasource.
ForthepurposesofTeiid,connectingandissuingqueriestofetchthemetadatafromtheseexternaldatasources,Teiiddefines/providestwotypesofresources.
ResourceAdapter
Aresourceadapter(alsocalledasSERVER)isconnectionobjecttotheexternaldatasource.InthecaseofrelationaldatabasethiscanbeachievedthroughaJDBCconnection,orinthecaseofaFilethismaybeareferencetofile’slocation.Theresource-adapterprovidesaunifiedinterfacetodefineaconnectionintheTeiid.Aresourceadapteralsoprovideswaytonativelyissuecommandsandgatherresults.Teiidprovidesvarietyofresourceadaptorstomanydifferentsystemsoronecanbedevelopedfornew/customdatasource.Aresourceadaptersconnectionisrepresentedaboveasthe"jndi-name".
AsVDBdeveloperyouneedtoknow,howtoconfigurethesesourcesintheTeiid.InWildFlyServerthesearedefinedasJCAcomponents.InTeiidembedded,thedeveloperhastodefinetheconnectionstothesesourcesprogrammatically.CheckoutAdministrator’sGuideonhowtoconfiguretheseinWildFly,orembeddedexamples,ifyouareworkingwithTeiidEmbedded.
Translator
ATranslator(alsocalledDATAWRAPPER)isacomponentthatprovidesanabstractionlayerbetweenTeiidQueryEngineandphysicaldatasource,thatknowshowtoconvertTeiidissuedquerycommandsintosourcespecificcommandsandexecutethemusingtheResourceAdapter.ItalsohavesmartstoconverttheresultdatathatcamefromthephysicalsourceintoaformthatTeiidQueryengineisexpecting.Forexample,whenworkingwithaweb-servicetranslator,aSQLprocedureexecutedatTeiidlayermaybeconvertedtoaHTTPbasedcallthroughatranslator,andresponseJSONcouldbeconvertedtotabularresults.
Teiidprovidesvarioustranslatorsaspartofthesystem,oronecanbedevelopedusingtheprovidedjavalibraries.ForlistofavailableTranslatorsseeTranslators
Important InaVDB,asourceschemamustbeconfiguredwithacorrectTranslatorandavalidresourceadapter,tomakethesystemwork.
VirtualDatabases
296
DevelopingaVirtualDatabaseTherearefewdifferentwaysaVirtualDatabasecanbedeveloped.Eachmethodhasadvantagesanddisadvantages.
AVDBisdevelopedasfileartifact,whichcandeployedintoaTeiidServer.ThisfileartifactcontainsthemetadataabouttheVDB,orcontainsthedetailstofetchthemetadatafromsourcedatasources.Theseartifactscanbesharedandmovedbetweendifferentservers.
vdb.xml:Inthisfileformat,youcanusecombinationofXMLelementsandDDLelementstodefinethemetadata.
vdb.ddl:Inthisfileformat,youcanusestrictlyDDLusingSQL-MED(withfewcustomextensions)todefinethemetadata.Thiscanbeviewedasnextversiontothevdb.xml.
Designer:DesignerprovidesagraphicalUI,thatusercanusetodesignaVDB.Usingthis,usercaninteractivelydesigntables,views,procedures,functionsetc.Designertypicallygeneratesa.vdb(zip)fileartifact,howeveritcanalsoexportthisas-vdb.xmlfile.Supporttoexportasvdb.ddliscomingsoon.
vdb.xmlandvdb.ddlmaybedeployedasstandalonefiles.Asastandalonefile,theVDBfilenamepatternmustadhereto"-vdb.xxx"fortheTeiidVDBdeployertorecognizethisfile.
Theymayalsobecontainedina.vdbzipfilealongwithotherrelevantfiles,suchasjars,additionalddl,andstaticfileresources.
Important Itisimportanttonotethat,themetadatarepresentedbytheVDBformatsisEXACTLYsameinalldifferentways.Infact,youcanconvertaVDBfromonetypetotheother.
StepstofollowindevelopingaVDBThiswillwalkthroughdevelopingaDDLbasedVDB.
Step1:PickNameandVersion
Pickthenameandversionofthevirtualdatabaseyouwanttocreate.Frompreviousexamplethisrepresents
CREATEDATABASEmy_exampleVERSION'1.0.0';
USEDATABASEmy_exampleVERSION'1.0.0';
Step2:ConfiguringaSource(s)
Whenworkingwithexternalsources,therearefewextrastepsneedtobefollowed,asnotallthesoftwarecomponentsrequiredfortheconnectionnorconfigurationareautomaticallyprovidedbyTeiid.
Step2A:FindtheTranslator
FirstfindoutifthesupportforthesourceisprovidedinTeiid.LookatTeiiddocumentationandsupportedtranslators.Pickthenamesoftranslator(s)youwillbeusing.Frompreviousexamplethisrepresents
CREATEFOREIGNDATAWRAPPERpostgresql;
here"postgresql"isourtranslatorname,asexampleassumeswearegoingtoqueryaPostgreSQLdatabase.
Step2B:FindthemoduletoconnecttoExternalSource
DevelopingaVirtualDatabase
297
TypicallyallreleationaldatabasesareconnectedusingtheirJDBCdrivers.FindoutiftheexternalsourcehasaJDBCdriver?ifthissourcehasJDBCdriver,thenacquirethedriverjarfile.
Oncethedriverisacquired,thenmakesurethisdriverisType4driver,andthendeploythisdriverintoTeiidserverusingeitherweb-consoleapplicationorCLIadmin-console.ThebelowexampleshowsdeployingtheOracledriverinTeiidServerbasedonWildFlyusingCLIadmin-console.IfdriverisnotType4,itcanbestillused,butmoresetupisneeded.
</wildfly/bin>$./jboss-cli.sh--connect
[standalone@localhost:9990/]deploy/path/to/ojdbc6.jar
ifthesourcedoesnothaveJDBCdriverandhasresource-adapterprovidedbyTeiid,thendriverforitisalreadyavailableinTeiidserver.Nofurtheractionrequiredforthis.
Step2C:CreateaConnectiontoExternalSource
Basedonabovedriverorresourceadapteraconnectiontotheexternalsourceneedtobecreated.Therearemanymethodstocreateadatasourceconnection.
TeiidServer(chooseonemethodfrombelow)
Editthewildfly/standalone/configuration/standalone-teiid.xmlfileandaddrespectivedatasourceorresourceadapterconfiguration.Theexamplesofthesetemplatesareprovidedinwildfly/docs/teiid/datasourcesdirectory.
UseTeiidWeb-consoleandfollowthedirectionstocreateadatasourceorresource-adapter.
UseCLIadmin-consoleandexecutethescript.Thesamplescriptsaregiveninwildfly/docs/teiid/datasourcesdirectory.Also,checkoutdocumentationatAdministrator’sGuideformoredetails.
TeiidEmbedded
Createtheconnectionprogrammatically,bysupplyingyourownlibrariestoconnecttothesource.
Frompreviousexamplethisrepresents
CREATESERVERpgsqlTYPE'postgresql-9.4-1201.jdbc41.jar'
VERSION'one'FOREIGNDATAWRAPPERpostgresql
OPTIONS(
"jndi-name"'java:/postgres-ds'
);
Warning ThisprobablyismostchallengingstepintermsofunderstandingTeiid,makesureyoufollowbeforegoingfurtherintonextsteps.
Step3:CreateSourceSchema
Nowthataccesstheexternalsourcesisdefined,"sourceschema"ormodelsasshownbeforeneedstobecreatedandmetadataneedstobedefined.
Frompreviousexamplethisrepresents
CREATESCHEMAtestSERVERpgsql;
SETSCHEMAtest;
SETSCHEMAstatementsetsthecontextinwhichfollowingDDLstatementstofallin.
DevelopingaVirtualDatabase
298
Schemacomponentisdefined,butithasnometadata.i.etables,proceduresorfunctions.Thesecanbedefinedoneoftwowaysforasourcemodel,eitherimportingthemetadatadirectlyfromthesourcesystemitself,ordefiningtheDDLmanuallyinlineinthisfile.
Step3A:ImportMetadata
UsingthedatasourceconnectionscreatedinStep2,importthemetadataupondeploymentoftheVDB.Notethatthiscapabilityisslightlydifferentforeachsource,astowhatandhow/whatkindofmetadatais.Checkindividualsource’stranslatordocumentationformoreinformation.Frompreviousexamplethisrepresents
IMPORTFOREIGNSCHEMApublicFROMSERVERpgsqlINTOtest
OPTIONS(
importer.useFullSchemaNamefalse,
importer.tableTypes'TABLE,VIEW'
);
Theaboveimportstatementissayingthat,importthe"public"schemafromexternaldatasourcedefinedby"pgsql"intolocal"test"schemainTeiid.ItalsofurtherconfigurestoonlyfetchTABLE,VIEWtypes,anddonotusefullyqualifiedschemanamesintheimportedmetadata.Eachtranslator/sourcehasmanyoftheseconfigurationoptionsyoucanusetofilter/refineyourselections,formoreinformationconsultthetranslatordocumentsatTranslatorsforeverysourceyouaretryingtoconnectto.
Step3B:DefineMetadatausingDDL
Insteadofimportingthemetadata,youcanmanuallydefinethetablesandproceduresinlinetodefinethemetadata.ThiswillbefurtherexplainedinnextsectionsdetailoneveryDDLstatementsupported.Forexample,youcandefineatablelike
CREATEFOREIGNTABLECUSTOMER(
SSNchar(10)PRIMARYKEY,
FIRSTNAMEstring(64),
LASTNAMEstring(64),
ST_ADDRESSstring(256),
APT_NUMBERstring(32),
CITYstring(64),
STATEstring(32),
ZIPCODEstring(10)
);
Warning
Pleasenotethatwhenmetadataisdefinedinthismanner,thesourcesystemmustalsohaverepresentativeschematosupportanyqueriesresultingfromthismetadata.TeiidCANNOTautomaticallycreatethisstructureinyourdatasource.Forexample,withabovetabledefinition,ifyouareconnectingOracledatabase,theOracledatabasemusthavetheexistingtablewithmatchingnames.TeiidcannotcreatethistableinOracleforyou.
RepeatthisStep2&Step3,foralltheexternaldatasourcestobeincludedinthisVDB
Step5:CreateVirtualViews
Nowusingtheabovesource’smetadata,definetheabstract/logicalmetadatalayerusingTeiid’sDDLsyntax.i.e.createVIEWS,PROCEDURESetctomeettheneedsofyourbusinesslayer.Forexample(pseudocode):
CREATEVIRTUALSCHEMAreports;
DevelopingaVirtualDatabase
299
CREATEVIEWSalesByRegion(
quarterdate,
amountdecimal,
regionvarchar(50)
)AS
SELECT...FROMSalesJOINRegiononx=yWHERE...
RepeatthisstepasneededanynumberofVirtualViewsyouneed.YoucanrefertoViewtablesinoneviewfromothers.
Step6:DeploytheVDB
OncetheVDBiscompleted,thenthisVDBneedstobedeployedtotheTeiidServer.(thisisexactlysameasyoudeployingaWARfileforexample).OnecanuseTeiidweb-consoleorCLIadmin-consoletodothisjob.Forexamplebelowclicanbeused
deploymy-vdb.ddl
Step7:ClientAccess
OncetheVDBisavailableontheTeiidServerinACTIVEstatus,thisVDBcanbeaccessedfromanyJDBC/ODBCconnectionbasedapplications.YoucanuseBItoolssuchasTableau,BusinessObjects,QuickView,PentahobycreatingaconnectiontothisVDB.YoucanalsoaccesstheVDBusingODataV4protocolwithoutanyfurthercoding.
NomatterhowyouaredevelopingtheVDB,whetheryouareusingthetoolingornot,theabovearestepstobefollowedtobuildasuccessfulVDB.
vdb.xmlThevdb-deployer.xsdschemaforthisxmlfileformatisavailableintheschemafolderunderthedocswiththeTeiiddistribution.
Seealsolink:xml_deployment_mode.adoc
VDBZipDeployment
Formorecomplicatedscenariosyouarenotlimitedtojustanxml/ddlfiledeployment.AfullzipfilesimilartoaDesignerVDBmayalsobedeployed.Inavdbzipdeployment:
Thedeploymentmustendwiththeextension.vdb
Thevdbxmlfilemustbezipunder/META-INF/vdb.xml
Ifa/libfolderexistsanyjarsfoundunderneathwillautomaticallybeaddedtothevdbclasspath.
ForbackwardscompatibilitywithDesignerVDBs,ifany.INDEXfileexiststhedefaultmetadatarepositorywillbeassumedtobeINDEX.
FileswithintheVDBzipareaccessiblebyaCustomMetadataRepositoryusingtheMetadataFactory.getVDBResources()method,whichreturnsamapofallVDBResourcesintheVDBkeyedbyabsolutepathrelativetothevdbroot.TheresourcesarealsoavailableatruntimeviatheSYSADMIN.VDBResourcestable.
Thebuilt-inDDL-FILEmetadatarepositorytypemaybeusedtodefineDDL-basedmetadatainotherfileswithintheziparchive.Thisimprovesthememoryfootprintofthevdbmetadataandthemaintainabilityofthemetadata.
DevelopingaVirtualDatabase
300
ExampleVDBZipStructure
/META-INF
vdb.xml
/ddl
schema1.ddl
/lib
some-udf.jar
Intheaboveexampleavdb.xmlcoulduseaDDL-FILEmetadatatypeforschema1:
<modelname="schema1"...
<metadatatype="DDL-FILE">/ddl/schema1.ddl</metadata>
</model>
Thecontentsinsideschema1.ddlcanincludeDDLforSchemaObjects
DevelopingaVirtualDatabase
301
DDLVDBAVirtualDatabase(VDB)cancreatedthroughDDLstatements.TeiidsupportsSQL-MEDspecificationtoconfiguretheforeigndatasources.
ADDLfilecapturesinformationabouttheVDB,thesourcesitintegrate,andpreferencesforimportingmetadata.TheformatoftheDDLfilecanbeanyelementsindocumentedhere.TheDDLfilemaybedeployedasasinglefile,orinaziparchive.SeeDevelopingaVirtualDatabaseforadiscussionofthe.vdbzippackaging.
DDLFileDeployment
YoucansimplycreateaSOME-NAME-vdb.ddlfile.
Important TheVDBnamepatternmustadhereto"-vdb.ddl"fortheTeiidVDBdeployertorecognizethisfilewhendeployedinTeiidServer.
ExampleVDBDDLTemplate
CREATEDATABASEmy_exampleVERSION'1.0.0';
USEDATABASEmy_exampleVERSION'1.0.0'
CREATEFOREIGNDATAWRAPPERpostgresql;
CREATESERVERpgsqlTYPE'postgresql-9.4-1201.jdbc41.jar'
VERSION'one'FOREIGNDATAWRAPPERpostgresql
OPTIONS(
"jndi-name"'java:/postgres-ds'
);
CREATESCHEMAtestSERVERpgsql;
IMPORTFOREIGNSCHEMApublicFROMSERVERpgsqlINTOtest
OPTIONS(
importer.useFullSchemaNamefalse,
importer.tableTypes'TABLE,VIEW'
);
DDLFileFormat
Step1:CreateaDatabase
EveryVDBfilemuststartwithdatabasedefinitionwhereitspecifiesthenameandversionofthedatabase.Thecreatesyntaxfordatabaseis
CREATEDATABASE{db-name}[VERSION{version-string}]OPTIONS(<options-clause>)
<options-clause>::=
<key><value>[,<key>,<value>]*
Anexamplestatement
CREATEDATABASEmy_exampleVERSION'1'OPTIONS("cache-metadata"true);
youcanaddadditionalOPTIONSatlatertimeusingbelowsyntax.
DDLVDB
302
ALTERDATABASE<name>OPTIONS(ADD|SET|DROP<key-value>)
ForlistdatabasescopedpropertiesseeVDBproperties
AswelearnedabouttheVDBcomponentsearlierintheguide,weneedtofirstcreatetranslators,thenconnectionstodatasources,andthenusingthesewecangathermetadataaboutthesesources.Thereisnolimitonhowmanytranslators,ordatasourcesorschemasyoucreatetobuildVDB.
Step2A:CreateaTranslatorTranslatorisanadaptertotheforeigndatasource,thecreationoftranslatorinthecontextoftheVDBcreatesareferencetothesoftwaremodulethatisavailableintheTeiidsystem.Forlistofavailabletranslators,youshouldlookupinTeiidsystem.Someoftheexamplesofavailabletranslatormodulesinclude
oracle
mysql
postgresql
mongodb
CREATEFOREIGN(DATAWRAPPER|TRANSLATOR){translator-name}
[TYPE{base-translator-type}]
OPTIONS(<options-clause>)
<options-clause>::=
<key><value>[,<key>,<value>]*
OptionalTYPEisusedtocreate"override"translator.TheOPTIONSclauseisusedtoprovidethe"execution-properties"ofaspecifictranslatordefinedineitherin{translator-name}or{base-translator-name}.ThesenamesMUSTmatchwithavailableTranslatorsinthesystem.link:Translators.adoc[Translators}documentsalltheavailabletranslators.
Example1:Examplecreatingtranslator
CREATEFOREIGNDATAWRAPPERpostgresql;
ForallavailabletranslatorsseeTranslators
1. Example2:ExamplecreatingOverrideTranslator
CREATEFOREIGNDATAWRAPPERoracle-overrideTYPEoracleOPTIONS(useBindVariables
false);
Theaboveexamplecreatesatranslatoroverridewithanexampleshowingturningoffthepreparedstatements.
Additionalmanagementsupporttoalter,deleteatranslator
ALTER(DATAWRAPPER|TRANSLATOR){translator-name}OPTIONS(ADD|SET|DROP<key-
value>);
DROPFOREIGN[<DATA><WRAPPER>|<TRANSLATOR>]{translator-name}
DDLVDB
303
Step2C:CreateaConnectionToExternalSource
Beforeyoucancreateaconnectiontothedatasource,youmusteitherhaveaJDBCdriver(Type4)thatcanconnecttothedatasource,orTeiidsystemmusthaveprovidedaresourceadapter(RAR)filetoenableconnectiontothedatasource.IfyouareusingtheJDBCdriverfilethisshouldhavealreadybeendeployedtotheTeiidsystem,ormadeitavailableontheclasspathinthecaseoftheTeiidEmbedded.ThisisStep2Bfromearlier,thereiscurrentlynoDDLmechanismtodeploytheexternaldrivers.
Nowtocreateconnectiontotheexternaldatasource.Oneneedstoknowthenameofdeployment.ForJDBCdrivers,itistypicallyJARnamewithoutpath.Forresourceadapters,itisthenameoftheresource-adapter.Stepalsoassociatestheconnectioncreatedwiththetranslatortobeusedincommunicatingwiththissource.
CREATESERVER{source-name}TYPE'{source-type}'
[VERSION'{version}']FOREIGNDATAWRAPPER{translator-name}
OPTIONS(<options-clause>)
<options-clause>::=
<key><value>[,<key>,<value>]*
Name Description
source-name Namegiventothesource’sconnection.
source-type ForJDBCconnection,thedrivernameorresource-adaptername.
translator-name Nameofthetranslatortobeusedwiththisserver.
options Allconnectionpropertiesfortheconnection.
ForallavailabletranslatorsseeTranslators
Example3:creatingadatasourceconnectiontoPostgresdatabase
CREATESERVERpgsqlTYPE'postgresql-9.4-1201.jdbc41.jar'
FOREIGNDATAWRAPPERpostgresql
OPTIONS(
"jndi-name"'java:/postgres-ds'
);
ThebelowarethetypicalpropertiesthatneedtobeconfiguredforaJDBCconnection
Name Description
jndi-name Jndinameofthedatasource
Note Anyadditionalpropertiestocreateadata-sourceinWildFlycanalsousedhereinOPTIONSclause.
ImportantIfthedatasourceisalreadyexistsintheconfiguration,thensupplyonlyprovidejndi-nameproperty(youcanomitallotherproperties),thenabovecommandwillcreateanewconnection,butwilluseexistingconfigurationinthesystem.
Thebelowshowsanexampleconnectionwithresourceadapter.
Example4:creatingadatasourceconnectionto"file"resourceadapter.
CREATESERVERmarketdataTYPE'file'
DDLVDB
304
FOREIGNDATAWRAPPERfile
OPTIONS(
ParentDirectory'/path/to/marketdata'
);
Forallavailabledatasourcesseedatasources
Additionalmanagementsupporttoalter/deleteaconnection.
ALTERSERVER{source-name}OPTIONS(ADD|SET|DROP<key-value>);
DROPSERVER{source-name};
Warning ALTERcanbeusedtochangeproperties,butduetoabuginWildFlythisfeaturecurrentlydoesnotwork.
NowthatwehavetheTranslatorsandConnectionscreated,thenextstepistocreateSCHEMAsandworkwithmetadata.
Step3&5:CreateSCHEMAinVDBBeforemetadataaboutdatasourcesorabstractionlayerscanbecreated,acontainerforthismetadataneedstobecreated.InrelationaldatabaseconceptsthisiscalledSchema,andthisalsoworksasanamespaceinwhichmetadataelementslikeTABLES,VIEWSandPROCEDURESexist.ThebelowDDLshowshowtocreateaSCHEMAelement.
CREATE[VIRTUAL]SCHEMA{schema-name}
[SERVER{server-name}(<COMMA>{server-name})*]
OPTIONS(<options-clause>)
<options-clause>::=
<key><value>[,<key>,<value>]*
TheuseofVIRTUALkeyworddefinesifthisschemais"VirtualSchema".IntheabsenceoftheVIRTUALkeyword,thisSchemaelementrepresentsa"SourceSchema".RefertoVDBGuideaboutdifferenttypesofSchematypes.
Important
IftheSchemaisdefinedas"SourceSchema",thenSERVERconfigurationmustbeprovided,tobeabletodeterminethedatasourceconnectiontobeusedwhenexecutingqueriesthatbelongtothisSchema.ProvidingmultipleServernamesconfigurethisSchemaas"multi-source"model.SeeMultisourceModelsformoreinformation.
ThebelowarethetypicalpropertiesthatneedtobeconfiguredforaSchemaintheOPTIONSclause.
Name Description
VISIBILITY IsSchemavisibleduringmetadatainterrogation
Example5:ShowingtocreateasourceschemaforPostgreSQLserverfromexampleabove
CREATESCHEMAtestSERVERpgsql;
Additionalmanagementsupporttoalter/deleteaschemacanbedonethroughfollowingcommands.
ALTER[VIRTUAL]SCHEMA{schema-name}OPTIONS(ADD|SET|DROP<key-value>);
DROPSCHEMA{schema-name};
DDLVDB
305
WhenworkingwithexistingVDB,orwheneditingmultipleSchema’sinaworkingVDB,oneneedstosetupcontextofSchemahe/shewantstomodify,assomeoftheDDLcommandsarecontextsensitiveastowhichSchematheybelong.TobeabletoestablishthecontextthatyouareworkingwithcertainSchemausefollowingcommand.
SETSCHEMA{schema-name};
thenyouwillbeadd/drop/alteranyotherSchemaelements.
Step4:ImportingSchemaIfyouaredesigningasourceschema,youcanaddtheTABLES,PROCEDURESmanuallytorepresentthedatasource,howeverincertainsituationsthiscanbetedious,orcomplicated.Forexample,ifyouneedtorepresent100sofexistingtablesfromyourOracledatabaseinTeiid?OrifyouareworkingwithMongoDB,howareyougoingtomapadocumentstructureintoaTABLE?Forthispurpose,Teiidprovidesanimportmetadatacommand,thatcanimport/createmetadatathatrepresentsthesource.Thefollowingcommandcanbeusedforthatpurposewithmostofthesources(LDAPsourceisonlyexception,notprovidingimport)
IMPORTFOREIGNSCHEMA{foreign-schema-name}[<importqualifications>]
FROMSERVER{server-name}
INTO{schema-name}
OPTIONS(<options-clause>)
<importqualifications>::=
LIMITTO(<tablenamelist>)|EXCEPT(<tablenamelist>)
<options-clause>::=
<key><value>[,<key>,<value>]*
foreign-schema-name:Nameofschemainexternaldatasourcetoimport.Typicallymostdatabasesaretiedtoaschemaname,like"public","dbo"ornameofthedatabase.Ifyouareworkingwithnon-relationalsource,youcanprovideadummyvaluehere.server-name:nameoftheservercreatedabovetoimportmetadatafrom.schema-name:Nameschemaelementcreatedabovetoimportmetadatainto.importqualifications:usingthisyoucanlimityourimportoftheTablesfromforeigndatasourcespecifiedtothislist.options-clause:The"importer"propertiesthatcanbeusedtorefinetheimportprocessbehaviorofthemetadata.EachTranslatordefinesasetof"importer"propertieswiththeirdocumentationorthroughextensionproperties.
ThebelowexampleshowsimportingmetadatafromaPostgreSQLusingserverexampleabove.
Example6
--importfromnativedatabase
IMPORTFOREIGNSCHEMApublic
LIMITTOcustomers,orders
FROMSERVERpgsql
INTOtest
--inarchivebasedvdbs(.vdb)youcanprovideeachschemainaseparatefileand
pulltheminmainvdb.ddlfileas
IMPORTFOREIGNSCHEMApublic
FROMREPOSITORYDDL-FILE
INTOtestOPTIONS("ddl-file"'/path/to/schema.ddl')
DDLVDB
306
Tip TheexampleIMPORTSCHEMAcanbeusedwithanycustomMetadataRepository,intheREPOSITORY{DDL-FILE},DDL-FILErepresentsaparticulartypeofrepository.
Theabovecommandimportspublic.customers,public.orderstablesusingpgsql’sconnectionintoaVDBschematest.
ImportinganotherVirtualDatabase(VDBReuse)
IfyouliketoimportanotherVDBthatiscreatedintothecurrentVDB,thefollowingcommandcnbeusedtoimportallthemetadata
IMPORTDATABASE{vdb-name}VERSION{version}[WITHACCESSCONTROL]
SpecifyingtheWITHACCESSCONTROLalsoimportsanyDataRolesdefinedintheotherdatabase.
DataRoles
Dataroles,alsocalledentitlements,aresetsofpermissionsdefinedperVDBthatdictatedataaccess(create,read,update,delete).Datarolesuseafine-grainedpermissionsystemthatTeiidwillenforceatruntimeandprovideauditlogentriesforaccessviolations.ToreadmoreaboutDataRolesandPermissionsseeDataRolesandPermissions
HerewewillshowDDLsupporttocreatetheseDataRolesandcorrespondingpermissions.
BNFforCreateDataRole
CREATEROLE{data-role}
[WITHJAASROLE{enterprise-role}(,{enterprise-role})*]
[WITHANYAUTHENTICATED]
data-role:DatarolereferencedintheVDBenterprise-role:Enterpriserole(s)thatthisdata-rolerepresentsWITHANYAUTHENTICATED:Whenpresent,thisdata-roleisgiventoanyuserwhoisvalidauthenticateduser.
Example:CreateDataRole
CREATEROLEreadWriteWITHJASSROLEdeveloper,analyst;
CREATEROLEreadOnlyWITHANYAUTHENTICATED;
NowtoSettingPermissions,seePermissionsformoredetails.
BNFforGrantcommand
GRANT(<permission-types>(,<permission-types>)*)
ON(<grant-resource>)
TO{data-role}
<permission-types>::=
SELECT|INSERT|UPDATE|DELETE|
EXECUTE|LANGUAGE|ALTER|DROP|
ALLPRIVILEGES|TEMPORARYTABLES)
<grant-resource>::=
DATABASE|
TABLE{schema-name}.{table-name}[<condition>]|
PROCEDURE{schema-name}.{procedure-name}[<condition>]|
SCHEMA{schema-name}|
COLUMN{schema-name}.{table-name}.{column-name}|
[MASK[ORDER\d]{expression}]
[<condition>]
<condition>::=
DDLVDB
307
CONDITION[CONSTRAINT]{expression}
permission-types:Typesofpermissionstobegranted
grant-resource:ThisisSchemaelementintheVDBonwhichthisgrantappliesto.ItcananythingbetweenaDATABASEandCOLUMN
schema-name:Nameoftheschemathisresourcebelongsto
table-name:NameoftheTable/View
procedure-name:ProcedureName
column-name:Nameofthecolumn
condition:Whenpresent,the{expression}isappendedtotheWHEREclauseofthequery
expression:anyvalidsqlexpression,thiscanincludecolumnsfromreferencedresource
CONSTRAINT:WhenthisissuppliedalongwithCONDITION,the{expression}isalsoappliedduringtheINSERT/UPDATEqueries.BydefaultCONDITIONonlyappliesSELECTqueries.AlsoCONSTRAINTdoesNOTapplytoVIEWsonlyFOREIGNTABLES.
Caution LANGUAGEpermissioncanonlybeappliedonDATABASE
Example:GiveRead,write,updatepermissiononsingletabletouserwithenterpriserole"role1"
CREATEROLERoleAWITHJAASROLErole1;
GRANTINSERT,READ,UPDATEONTABLEtest.CustomerTORoleA;
Example:Giveallpermissionstouserwith"admin"enterpriserole
CREATEROLEeverythingWITHJAASROLEadmin;
GRANTALLPRIVILEGESONDATABASETOeverything;
Example:UseofCONDITION,alluserscanseeonlyOrderstablecontentsamount<1000
CREATEROLEbase-roleWITHANYAUTHENTICATED;
GRANTREADONTABLEtest.OrdersCONDITION'amount<1000'TObase-role;
Example:UseofCONDITION,overridepreviousexampletomoreprivilegeduser
GRANTREADONTABLEtest.OrdersCONDITION'amount<1000andamount>=1000'TORoleA;
Example:Restrictingrows,ROWBASEDSECURITY
GRANTREADONTABLEtest.CustomerOrdersCONDITIONCONSTRAINT'name=user()'TORoleA;
Intheaboveexample,user()functionreturnsthecurrentlyloggedinuserid,ifthatmatchestothenamecolumn,onlythoserowswillbereturned.TherearefunctionslikehasRole('x')thatcanbeusedtoo.
Example:ColumnMasking,mask"amountforallusers"
GRANTREADONCOLUMNtest.Order.amount
MASK'xxxx'
TObase-role;
Example:ColumnMasking,mask"amountforalluserswhenamount>1000"
GRANTREADONCOLUMNtest.Order.amount
DDLVDB
308
MASK'CASEWHENamount>1000THEN'xxxx'END'
TObase-role;
Example:ColumnMasking,mask"amountforallusers"exceptthecallinguserisequaltotheuser()
GRANTREADONCOLUMNtest.Order.amount
MASK'xxxx'
CONDITION'customerid<>user()'
TObase-role;
DDLVDB
309
XMLVDBXMLbasedmetadatamaybedeployedinasinglexmlfiledeploymentorazipfilecontainingatleastthexmlfile.Thecontentsofthexmlfilewillbesimilareitherway.SeeDevelopingaVirtualDatabaseforadiscussionofthe.vdbzippackaging.TheXMLmayembeddedorreferenceDDL.
XMLFileDeployment
YoucansimplycreateaSOME-NAME-vdb.xmlfile.TheXMLfilecapturesinformationabouttheVDB,thesourcesitintegrate,andpreferencesforimportingmetadata.TheformatoftheXMLfileneedtoadheretovdb-deployer.xmlfile,whichisavailableintheschemafolderunderthedocswiththeTeiiddistribution.
Important TheVDBnamepatternmustadhereto"-vdb.xml"fortheTeiidVDBdeployertorecognizethisfilewhendeployedinTeiidServer.
Tip ifyouhaveexistingVDBincombinationofXML&DDLformat,youcanmigratetoallDDLversionusingthe"teiid-convert-vdb.bat"or"teiid-convert-vdb.sh"utilityinthe"bin"directoryoftheinstallation.
XMLFileFormatExampleVDBXMLTemplate
<vdbname="${name}"version="${version}">
<!--Optionaldescription-->
<description>...</description>
<!--Optionalconnection-type-->
<connection-type>...</connection-type>
<!--VDBproperties-->
<propertyname="${property-name}"value="${property-value}"/>
<!--UDFdefinedinanASmodule,seeDevelopersGuide-->
<propertyname="lib"value="{module-name}"></property>
<import-vdbname="..."version="..."import-data-policies="true|false"/>
<!--defineamodelfragmentforeachdatasource-->
<modelvisible="true"name="${model-name}"type="${model-type}">
<propertyname="..."value="..."/>
<sourcename="${source-name}"translator-name="${translator-name}"
connection-jndi-name="${deployed-jndi-name}">
<metadatatype="${repository-type}">rawtext</metadata>
<!--additionalmetadata
<metadatatype="${repository-type}">rawtext</metadata>
-->
</model>
<!--defineamodelwithmultiplesources-seeMulti-SourceModels-->
<modelname="${model-name}"path="/Test/Customers.xmi">
<propertyname="multisource"value="true"/>
...
<sourcename="${source-name}"
UsingXML&DDL
310
translator-name="${translator-name}"connection-jndi-name="${deployed-jndi-name}"/>
<source.../>
<source.../>
</model>
<!--seeReferenceGuide-DataRoles-->
<data-rolename="${role-name}">
<description>${role-description}</description>
….
</data-role>
<!--createtranslatorinstancesthatoverridedefaultproperties-->
<translatorname="${translator-name}"type="${translator-type}"/>
<propertyname="..."value="..."/>
</translator>
</vdb>
NotePropertySubstitution-Ifa-vdb.xmlfilehasdefinedpropertyvalueslike$\{my.property.name.value},thesecanbereplacedbyactualvaluesthataredefinedthroughJAVAsystemproperties.TodefinesystempropertiesonaWildFlyserver,pleaseconsultWildFlydocumentation.
WarningYoumaychoosetolocallynamevdbartifactsasyouwish,buttheruntimenamesofdeployedVDBartifactsmusteitherbe*.vdbforazipfileor\*-vdb.xmlforanxmlfile.FailuretonamethedeploymentproperlywillresultinadeploymentfailureastheTeiidsubsystemwillnotknowhowtoproperlyhandletheartifact.
VDBElement
Attributes
name
ThenameoftheVDB.TheVDBnamereferencedthroughthedriverordatasourceduringtheconnectiontime.
version
TheversionoftheVDB.ProvidesanexplicitversioningmechanismtotheVDBname-seeVDBVersioning.
DescriptionElement
OptionaltextelementtodescribetheVDB.
ConnectionTypeElement
DetermineshowclientscanconnecttotheVDB.CanbeoneofBY_VERSION,ANY,orNONE.DefaultstoBY_VERSION.SeeVDBVersioning.
PropertiesElement
seeVDBPropertiesforpropertiesthatcanbesetatVDBlevel.
import-vdbElement
VDBsmayreuseotherVDBsdeployedinthesameserverinstancebyusingan"import-vdb"declarationinthevdb.xmlfile.AnimportedVDBcanhaveit’stablesandproceduresreferencedbyviewsandproceduresintheimportingVDBasiftheyarepartoftheVDB.ImportedVDBsarerequiredtoexistbeforeanimportingVDBmaystart.IfanimportedVDBisundeployed,thenanyimportingVDBwillbestopped.+
AnimportedVDBincludesallofitsmodelsandmaynotconflictwithanymodel,datapolicy,orsourcealreadydefinedintheimportingVDB.OnceaVDBisimporteditismostlyoperationallyindependentfromthebaseVDB.OnlycostrelatedmetadatamaybeupdatedforanobjectfromanimportedVDBinthescopeoftheimportingVDB.Allotherupdatesmustbemadethrough
UsingXML&DDL
311
theoriginalVDB,buttheywillbevisibleinallimportedVDBs.EvenmaterializedviewsareseparatelymaintainedforanimportedVDBinthescopeofeachimportingVDB.
ExamplereuseVDBXML
<vdbname="reuse"version="1">
<import-vdbname="common"version="1"import-data-policies="false"/>
<modelvisible="true"type="VIRTUAL"name="new-model">
<metadatatype="DDL"><![CDATA[
CREATEVIEWx(
yvarchar
)AS
select*fromold-model.tbl;
]]>
</metadata>
</model>
</vdb>
Attributes
name
ThenameoftheVDBtobeimported.
version
TheversionoftheVDBtobeimported(shouldbeanpositiveinteger).
import-data-policies
Optionalattributetoindicatewhetherthedatapoliciesshouldbeimportedaswell.Defaultsto"true".
ModelElement
Attributes
name
Thenameofthemodelisusedasatoplevelschemanameforallofthemetadataimportedfromtheconnector.ThenameshouldbeuniqueamongallModelsintheVDBandshouldnotcontainthe'.'character.
visible
Bydefaultthisvalueissetto"true",whenthevalueissetto"false",thismodelwillnotbevisibletowhenJDBCmetadataqueries.Usuallyitisusedtohideamodelfromclientapplicationsthatshouldnotdirectlyissuequeriesagainstit.However,thisdoesnotprohibiteitherclientapplicationorotherviewmodelsusingthismodel,iftheyknewtheschemaforthismodel.
PropertyElements
AllpropertiesareavailableasextensionmetadataonthecorrespondingSchemaobjectthatisaccessibleviathemetadataAPI.
cache-metadata
Canbe"true"or"false".defaultsto"false"for-vdb.xmldeploymentsotherwise"true".If"false",Teiidwillobtainmetadataonceforeverylaunchofthevdb."true"willsaveafilecontainingthemetadataintothePROFILE/data/teiiddirectoryCanbeusedtooverridethevdblevelcache-metadataproperty.
teiid_rel:DETERMINISM
Canbeoneof:DETERMINISMNONDETERMINISTICCOMMAND_DETERMINISTICSESSION_DETERMINISTICUSER_DETERMINISTICVDB_DETERMINISTICDETERMINISTIC
UsingXML&DDL
312
Willinfluencethecachescopeforresultsetcacheentriesformedfromaccessingthismodel.AlternativelythescopemaybeinfluencedthroughtheTranslatorAPIorviatable/procedureextensionmetadata.
SourceElement
Asourceisanamedbindingofatranslatorandconnectionsourcetoamodel.
name
Thenameofthesourcetouseforthismodel.Thiscanbeanynameyoulike,butwilltypicallybethesameasthemodelname.Havinganamedifferentthanthemodelnameisonlyusefulinmulti-sourcescenarios.Inmulti-source,thesourcenamesunderagivenmodelmustbeunique.Ifyouhavethesamesourceboundtomultiplemodelsitmayhavethesamenameforeach.Anexceptionwillberaisedifthesamesourcenameisusedfordifferentsources.
translator-name
ThenameortypeoftheTeiidTranslatortouse.Possiblevaluesincludethebuilt-intypes(ws,file,ldap,oracle,sqlserver,db2,derby,etc.)andtranslatorsdefinedinthetranslatorssection.
connection-jndi-name
TheJNDInameofthissource’sconnectionfactory.ThereshouldbeacorrespondingdatasourcethatdefinestheconnectionfactoryintheJBossAS.CheckoutthedeployingVDBdependenciessectionforinfo.YoualsoneedtodefinetheseconnectionfactoriesbeforeyoucandeploytheVDB.
PropertyElements
importer.<propertyname>
Propertytobeusedbytheconnectorimporterforthemodelforpurposesimportingmetadata.Seepossiblepropertyname/valuesintheTranslatorspecificsection.Notethatusingthesepropertiesyoucannarroworwidenthedataelementsavailableforintegration.
MetadataElement
Theoptionalmetadataelementdefinesthemetadatarepositorytypeandoptionalrawmetadatatobeconsumedbythemetadatarepository.
type
Themetadatarepositorytype.DefaultstoINDEXforDesignerVDBsandNATIVEfornon-DesignerVDBsourcemodels.Forallotherdeployments/modelsavaluemustbespecified.Built-intypesincludeDDL,NATIVE,INDEX,andDDL-FILE.Theusageoftherawtextvarieswiththebytype.NATIVEandINDEX(onlyforDesignerVDBs)metadatarepositoriesdonotusetherawtext.TherawtextforDDLisexpectedtobebeaseriesofDDLstatementsthatdefinetheschema.Notethat,since<model>elementmeansschema,youonlyuselinke:DDL_Metadata.adoc[SchemaObjectDDL].RestofDDLstatementscanNOTbeusedintheartifactmode,asthoseconstructsaredefinedbytheXMLfile.Like<Model>elementissimilarto"CREATESCHEMA…".DuetobackwardscompatibilityTeiidsupportsbothmodesasbothhavetheiradvantages.
DDL-FILE(usedonlywithzipdeployments)issimilartoDDL,exceptthattherawtextspecifiesanabsolutepathrelativetothevdbrootofthelocationofafilecontainingtheDDL.SeeMetadataRepositoriesformoreinformationandexamples
TranslatorElement
Attributes
name
ThenameofthetheTranslator.Referencedbythesourceelement.
type
UsingXML&DDL
313
ThebasetypeoftheTranslator.Canbeoneofthebuilt-intypes(ws,file,ldap,oracle,sqlserver,db2,derby,etc.).
PropertyElements
Setavaluethatoverridesatranslatordefaultproperty.Seepossiblepropertyname/valuesintheTranslatorspecificsection.
VDBReuseVDBsmayreuseotherVDBsdeployedinthesameserverinstancebyusingan"import-vdb"declaration.AnimportedVDBcanhaveit’stablesandproceduresreferencedbyviewsandproceduresintheimportingVDBasiftheyarepartoftheVDB.ImportedVDBsarerequiredtoexistbeforeanimportingVDBmaystart.IfanimportedVDBisundeployed,thenanyimportingVDBwillbestopped.
AnimportedVDBincludesallofitsmodelsandmaynotconflictwithanymodel,datapolicy,orsourcealreadydefinedintheimportingVDB.OnceaVDBisimporteditismostlyoperationallyindependentfromthebaseVDB.OnlycostrelatedmetadatamaybeupdatedforanobjectfromanimportedVDBinthescopeoftheimportingVDB.AllotherupdatesmustbemadethroughtheoriginalVDB,buttheywillbevisibleinallimportedVDBs.EvenmaterializedviewsareseparatelymaintainedforanimportedVDBinthescopeofeachimportingVDB.
ExamplereuseVDBXML
<vdbname="reuse"version="1">
<propertyname="imported-model.visible"value="false"/>
<import-vdbname="common"version="1"import-data-policies="false"/>
<modelvisible="true"type="VIRTUAL"name="new-model">
<metadatatype="DDL"><![CDATA[
CREATEVIEWx(
yvarchar
)AS
select*fromimported-model.tbl;
]]>
</metadata>
</model>
</vdb>
IntheaboveexamplethereuseVDBwillhaveaccesstoallofthemodelsdefinedinthecommonVDBandaddsinthe"new-model".Thevisibilityofimportedmodelsmaybeoverriddenviabooleanvdbpropertiesusingthekeymodel.visible-shownaboveasimported-model.visiblewithavalueoffalse.
UsingXML&DDL
314
VirtualDatabaseRelatedProperties
PropertiesDATABASELevel
cache-metadata
Canbe"true"or"false".defaultsto"false"for-vdb.xmldeploymentsotherwise"true".If"false",Teiidwillobtainmetadataonceforeverylaunchofthevdb."true"willsaveafilecontainingthemetadataintothePROFILE/data/teiiddirectory
query-timeout
SetsthedefaultquerytimeoutinmillisecondsforqueriesexecutedagainstthisVDB.0indicatesthattheserverdefaultquerytimeoutshouldbeused.Defaultsto0.Willhavenoeffectiftheserverdefaultquerytimeoutissettoalesservalue.Notethatclientscanstillsettheirowntimeoutsthatwillbemanagedontheclientside.
lib
Settoalistofmodulesforthevdbclasspathforuserdefinedfunctionloading.SeealsoSupportforUser-DefinedFunctions(Non-Pushdown).
security-domain
SettothesecuritydomaintouseifaspecificsecuritydomainisapplicabletotheVDB.Otherwisethesecuritydomainlistfromthetransportwillbeused.
<propertyname="security-domain"value="custom-security"/>
Note Anadminneedstoconfigureamatching"custom-security"loginmoduleinstandalone-teiid.xmlconfigurationfilebeforetheVDBisdeployed.
connection.XXX
ForusebytheODBCtransportandODatatosetdefaultconnection/executionproperties.SeeDriverConnectionforallproperties.Notethesearesetontheconnectionafterithasbeenestablished.
<propertyname="connection.partialResultsMode"value="true"/>
authentication-type
Authenticationtypeofconfiguredsecuritydomain.Allowedvaluescurrentlyare(GSS,USERPASSWORD).Thedefaultissetonthetransport(typicallyUSERPASSWORD).
password-pattern
Regularexpressionmatchedagainsttheconnectinguser’snamethatdeterminesifUSERPASSWORDauthenticationisused.password-patternTakesprecedenceofoverauthentication-type.Thedefaultisauthentication-type.
gss-pattern
Regularexpressionmatchedagainsttheconnectinguser’snamethatdeterminesifGSSauthenticationisused.gss-patternTakesprecedenceofoverpassword-pattern.Thedefaultispassword-pattern.
model.visible
Usedtooverridethevisibilityofimportedvdbmodels,wheremodelisthenameoftheimportedmodel.
VDBProperties
315
include-pg-metadata
Bydefault,PGmetadataisalwaysaddedtoVDBunlessSystemPropertiessetpropertyorg.teiid.addPGMetadatatofalse.ThispropertyenablesaddingPGmetadataperVDB.PleasenotethatifyouareusingODBCtoaccessyourVDB,theVDBmustincludePGmetadata.
lazy-invalidate
BydefaultTTLexpirationwillbeinvalidating-seeInternalMaterialization.Settinglazy-invalidatetotruewillmakettlrefreshesnon-invalidating.
deployment-name
Effectivelyreserved.Willbesetatdeploytimebytheservertothenameoftheserverdeployment.
PropertiesSchema/ModelLevel
visible
MarkstheSchemaisvisiblewhenvalueistrue.visiblefalgissettofalse,theSchema’smetadataishiddenfromanymetadatarequests.HowevernotethatthisdoesnotprohibittheuserfromissuingthequeriesagainstthisSchema,inordertocontrolthequerieslookintoDataRoles.
multisource
MarkstheSchemaasmulti-sourcemode,wherethedataexistsinpartitionsinmultipledifferentsources.ItisassumedthatmetadataoftheSchemaacrossallthedatasourcesisexactlysame.
multisource.columnName
Inamulti-sourceschemaallthetableswillbeimplicitlyaddedwithadditionalcolumntodesignatethepartitioncolumnaboutidentityofthatsource.Thispropertydefinesthenameofthatcolumn,thetypewillbealways'String'.
multisource.addColumn
Thisflagtoindicate,toaddtheimplicitpartitioncolumntoallthetablesinthisSchema.truevalueaddsthecolumn.Defaultisfalse.
allowed-languages
Theallowed-languagespropertyenablesthelanguagesuseforanypurposeinthevdb,whiletheallow-languagepermissionallowsthelanguagetobeusedbyuserswithRoleA.
VDBProperties
316
DDLMetadataforSchemaObjectsTheDDLforschemaobjectsiscommontobothXMLandDDLVDBs.
TableofContentsDataTypesCreatingaForeignTable
DefiningTableCONSTRAINTSALTERTABLE
CreateViewALTERTABLEINSTEADOFTRIGGERSOnVIEW(UpdateVIEW)AFTERTRIGGERSOnSourceTables
CreateProcedure/FunctionExtensionMetadata
Built-inNamespacePrefixes
DataTypes
TheBNFforDataTypesrefertoDataTypes
CreatingaForeignTable
AFOREIGNtableistablethatisdefinedonsourceschemathatrepresentsarealrelationaltableinsourcedatabaseslikeOracle,SQLServeretc.Forrelationaldatabases,TeiidhascapabilitytoautomaticallyretrievethedatabaseschemainformationuponthedeploymentoftheVDB,ifoneliketoautoimporttheexistingschema.However,usercanusebelowFOREIGNtablesemantics,whentheywouldliketoexplicitlydefinetablesonPHYSICALschemaorrepresentnon-relationaldataasrelationalincustomtranslators.
BNFforCreateTable
CREATEFOREIGNTABLE{table-name}(
<table-element>(,<table-element>)*
[<constraint>(,<constraint>)*
)OPTIONS(<options-clause>)
<table-element>::=
{column-name}<data-type><element-attr><options-clause>
<data-type>::=
varchar|boolean|integer|double|date|timestamp..(seeDataTypes)
<element-attr>::=
[AUTO_INCREMENT][NOTNULL][PRIMARYKEY][UNIQUE][INDEX][DEFAULT{expr}]
<constraint>::=
CONSTRAINT{constraint-name}(
PRIMARYKEY<columns>|
FOREIGNKEY(<columns>)REFERENCEStbl(<columns>)
UNIQUE<columns>|
ACCESSPATTERN<columns>
INDEX<columns>
<columns>::=
SchemaObjectDDL
317
({column-name}[,{column-name}])
<options-clause>::=
<key><value>[,<key>,<value>]*
ForvalidatingBNFforcreatetablerefertoCREATETABLE
Example7:CreateForeignTable(CreatedonPHYSICALmodel)
CREATEFOREIGNTABLECustomer(
idintegerPRIMARYKEY,
firstnamevarchar(25),
lastnamevarchar(25),
dobtimestamp);
CREATEFOREIGNTABLEOrder(
idintegerPRIMARYKEY,
customeridintegerOPTIONS(ANNOTATION'Customerprimarykey'),
saledatedate,
amountdecimal(25,4),
CONSTRAINTCUSTOMER_FKFOREGINKEY(customerid)REFERENCESCustomer(id)
)OPTIONS(UPDATABLEtrue,ANNOTATION'OrdersTable');
TABLEOPTIONS:(thebelowarewellknownoptions,anyotherspropertiesdefinedwillbeconsideredasextensionmetadata)
Property DataTypeorAllowedValues Description
UUID string UniqueidentifierforView
CARDINALITY intCostinginformation.Numberofrowsinthetable.Usedforplanningpurposes
UPDATABLE 'TRUE'|'FALSE' Definesiftheviewisallowedtoupdateornot
ANNOTATION string Descriptionoftheview
DETERMINISM
NONDETERMINISTIC,COMMAND_DETERMINISTIC,SESSION_DETERMINISTIC,USER_DETERMINISTIC,VDB_DETERMINISTIC,DETERMINISTIC
Onlycheckedonsourcetables
SchemaObjectDDL
318
COLUMNOPTIONS:(thebelowarewellknownoptions,anyotherspropertiesdefinedwillbeconsideredasextensionmetadata)
Property DataTypeorAllowedValues Description
UUID stringAuniqueidentifierforthecolumn
NAMEINSOURCE string
IfthisisacolumnnameontheFOREIGNtable,thisvaluerepresentsnameofthecolumninsourcedatabase,ifomittedthecolumnnameisusedwhenqueryingfordataagainstthesource
CASE_SENSITIVE 'TRUE'|'FALSE'
SELECTABLE 'TRUE'|'FALSE'
TRUEwhenthiscolumnisavailableforselectionfromtheuserquery
UPDATABLE 'TRUE'|'FALSE'
Definesifthecolumnisupdatable.Defaultstotrueiftheview/tableisupdatable.
SIGNED 'TRUE'|'FALSE'
CURRENCY 'TRUE'|'FALSE'
FIXED_LENGTH 'TRUE'|'FALSE'
SEARCHABLE 'SEARCHABLE'|'UNSEARCHABLE'|'LIKE_ONLY'|'ALL_EXCEPT_LIKE'
columnsearchability,usuallydictatedbythedatatype
MIN_VALUE
MAX_VALUE
SchemaObjectDDL
319
CHAR_OCTET_LENGTH integer
ANNOTATION string
NATIVE_TYPE string
RADIX integer
NULL_VALUE_COUNT long
costinginformation.NumberofNULLSinthiscolumn
DISTINCT_VALUES long
costinginformation.Numberofdistinctvaluesinthiscolumn
ColumnsmayalsobemarkedasNOTNULL,auto_increment,andwithaDEFAULTvalue.Currentlyonlystringvaluesaresupportedasthedefaultvalue.Tohavethestringinterpretedasanexpressionusetheextensionpropertyteiid_rel:default_handlingsettoexpression.
Acolumnoftypebigdecimal/decimal/numericcanbedeclaredwithoutaprecision/scalewhichwilldefaulttoaninternalmaximumforprecisionwithhalfscale,orwithaprecisionwhichwilldefaulttoascaleof0.
DefiningTableCONSTRAINTS
Constraintscanbedefinedontable/viewtodefineindexesandrelationshipstoothertables/views.ThisinformationisusedbytheTeiidoptimizertoplanqueriesorusetheindexesinmaterializationtablestooptimizetheaccesstothedata.
CONSTRAINTSaresameasonecandefineonRDBMS.
ExampleofCONSTRAINTs
CREATEFOREIGNTABLEOrders(
namevarchar(50),
saledatedate,
amountdecimal,
CONSTRAINTCUSTOMER_FKFOREGINKEY(customerid)REFERENCESCustomer(id)
ACCESSPATTERN(name),
PRIMARYKEY...
UNIQUE...
INDEX...
SchemaObjectDDL
320
ALTERTABLE
TheBNFforALTERtable,refertoALTERTABLE
UsingtheALTERCOMMAND,onecanAdd,Change,Deletecolumns,and/ormodifyanyOPTIONSvalues.Someexamplesbelow.
--addcolumntothetable
ALTERFOREIGNTABLE"Customer"ADDCOLUMNaddressvarchar(50)OPTIONS(SELECTABLEtrue);
--removecolumntothetable
ALTERFOREIGNTABLE"Customer"DROPCOLUMNaddress;
--addingoptionspropertyonthetable
ALTERFOREIGNTABLE"Customer"OPTIONS(ADDCARDINALITY10000);
--Changingoptionspropertyonthetable
ALTERFOREIGNTABLE"Customer"OPTIONS(SETCARDINALITY9999);
--Changingoptionspropertyonthetable'scolumn
ALTERFOREIGNTABLE"Customer"ALTERCOLUMN"name"OPTIONS(SETUPDATABLEFALSE)
--Changingtable'scolumntypetointeger
ALTERFOREIGNTABLE"Customer"ALTERCOLUMN"id"TYPEbigdecimal;
--Changingtable'scolumncolumnname
ALTERFOREIGNTABLE"Customer"RENAMECOLUMN"id"TO"customer_id";
CreateViewAviewisavirtualtable.Aviewcontainsrowsandcolumns,likearealtable.Thecolumnsinaviewarecolumnsfromoneormorerealtablesfromthesourceorotherviewmodels.Theycanalsobeexpressionsmadeupmultiplecolumns,oraggregatedcolumns.Whencolumndefinitionsarenotdefinedontheviewtable,theywillbederivedfromtheprojectedcolumnsoftheview’sselecttransformationthatisdefinedaftertheASkeyword.
Youcanaddfunctions,JOINstatementsandWHEREclausestoaviewdataasifthedatawerecomingfromonesingletable.
BNFforCreateTable
CREATEVIEW{table-name}AS{transformation_query}
OPTIONS(<options-clause>)
<options-clause>::=
<key><value>[,<key>,<value>]*
VIEWOPTIONS:(ThesepropertiesareinadditiontopropertiesdefinedintheCREATETABLE)
Property DataTypeorAllowedValues Description
MATERIALIZED 'TRUE'|'FALSE' Definesifatableismaterialized
SchemaObjectDDL
321
MATERIALIZED_TABLE 'table.name'
Ifthisviewisbeingmaterializedtoaexternaldatabase,thisdefinesthenameofthetablethatisbeingmaterializedto
Example:CreateViewTable(CreatedonVIRTUALschema)
CREATEVIEWCustomerOrders
AS
SELECTconcat(c.firstname,c.lastname)asname,
o.saledateassaledate,
o.amountasamount
FROMCustomerCJOINOrderoONc.id=o.customerid;
ImportantNotethatthecolumnsareimplicitlydefinedbythetransformationquery(SELECTstatement),theycanalsodefinedinlinebutiftheyaredefinedtheycanbeonlyalteredtomodifytheirproperties,youcannotADDorDROPnewcolumns.
ALTERTABLE
TheBNFforALTERVIEW,refertoALTERTABLE
UsingtheALTERCOMMANDyoucanchangethetransformationqueryoftheVIEW.YouareNOTallowedtoAlterthecolumninformation.Alsothetransformationquerymustbevalid
ALTERVIEWCustomerOrders
AS
SELECTconcat(c.firstname,c.lastname)asname,
o.saledateassaledate,
o.amountasamount
FROMCustomerCJOINOrderoONc.id=o.customerid
WHEREsaledate<TIMESTAMPADD(now(),-1,SQL_TSI_MONTH)
INSTEADOFTRIGGERSOnVIEW(UpdateVIEW)
AviewcomprisingmultiplebasetablesmustuseanINSTEADOFtriggertosupportinserts,updatesanddeletesthatreferencedatainthetables.Basedontheselecttransformation’scomplexitysometimesINSTEADOFTRIGGERSareautomaticallyprovidedfortheuserwhen"UPDATABLE"OPTIONontheVIEWissetto"TRUE".However,usingtheCREATETRIGGERmechanismusercanprovide/overridethedefaultbehavior.
Example:DefineinsteadoftriggeronViewforINSERT
CREATETRIGGERONCustomerOrdersINSTEADOFINSERTAS
FOREACHROW
BEGINATOMIC
INSERTINTOCustomer(...)VALUES(NEW.name...);
INSERTINTOOrders(...)VALUES(NEW.value...);
END
SchemaObjectDDL
322
ForUpdate
Example:DefineinsteadoftriggeronViewforUPDATE
CREATETRIGGERONCustomerOrdersINSTEADOFUPDATEAS
FOREACHROW
BEGINATOMIC
IF(CHANGING.saledate)
BEGIN
UPDATECustomerSETsaledate=NEW.saledate;
UPDATEINTOOrders(...)VALUES(NEW.value...);
END
END
Whileupdatingyouhaveaccesstopreviousandnewvaluesofthecolumns.FormoredetailedexplanationoftheseupdateprocedurespleaserefertoUpdateProcedures
AFTERTRIGGERSOnSourceTables
Asourcetablecanhaveanynumberofuniquelynamedtriggersregisteredtohandlechangeeventsthatarereportedbyachangedatacapturesystem.
SimilartoviewtriggersAFTERinsertprovidesaccesstonewvaluesviatheNEWgroup,AFTERdeleteprovidesaccesstooldvaluesviatheOLDgroup,andAFTERupdateprovidesaccesstoboth.
Example:DefineAFTERtriggeronCustomer
CREATETRIGGERONCustomerAFTERINSERTAS
FOREACHROW
BEGINATOMIC
INSERTINTOCustomerOrders(CustomerName,CustomerID)VALUES(NEW.Name,NEW.ID);
END
Youwilltypicallydefineahandlerforeachoperation-INSERT/UPDATE/DELTE.
FormoredetailedexplanationoftheseupdateprocedurespleaserefertoUpdateProcedures
CreateProcedure/Function
Usingthebelowsyntax,usercandefinea
SourceProcedure("CREATEFOREIGNPROCEDURE")-astoredprocedureinsource
SourceFunction("CREATEFOREIGNFUNCTION")-Afunctionthatissupportedbythesource,whereTeiidwillpushdowntosourceinsteadofevaluatinginTeiidengine
VirtualProcedure("CREATEVIRTUALPROCEDURE")-Similartostoredprocedure,howeverthisisdefinedusingtheTeiid’sProcedurelanguageandevaluatedintheTeiid’sengine.
Function/UDF("CREATEVIRTUALFUNCTION")-Auserdefinedfunction,thatcanbedefinedusingtheTeiidprocedurelanguageorcanhavetheimplementationdefinedusingaJAVAClass.
SchemaObjectDDL
323
Seethefullgrammarforcreatefunction/procedureintheBNFforSQLGrammar.
VariableArgumentSupport
InsteadofusingjustanINparameter,thelastnonoptionalparametercanbedeclaredVARIADICtoindicatethatitcanberepeated0ormoretimeswhentheprocedureiscalled
Example:Varargprocedure
CREATEFOREIGNPROCEDUREproc(xinteger,VARIADICzinteger)
RETURNS(xstring);
FUNCTIONOPTIONS:(thebelowarewellknownoptions,anyotherspropertiesdefinedwillbeconsideredasextensionmetadata)
Property DataTypeorAllowedValues Description
UUID string uniqueIdentifier
NAMEINSOURCE
Ifthisissourcefunction/procedurethenameinthephysicalsource,ifdifferentfromthelogicalnamegivenabove
ANNOTATION string Descriptionofthefunction/procedure
CATEGORY string FunctionCategory
DETERMINISM
NONDETERMINISTIC,COMMAND_DETERMINISTIC,SESSION_DETERMINISTIC,USER_DETERMINISTIC,VDB_DETERMINISTIC,DETERMINISTIC
Notusedonvirtualprocedures
NULL-ON-NULL 'TRUE'|'FALSE'
SchemaObjectDDL
324
JAVA_CLASS string JavaClassthatdefinesthemethodincaseofUDF
JAVA_METHOD stringTheJavamethodnameontheabovedefinedjavaclassfortheUDFimplementation
VARARGS 'TRUE'|'FALSE'
Indicatesthatthelastargumentofthefunctioncanberepeated0toanynumberoftimes.defaultfalse.ItismorepropertouseaVARIADICparameter.
AGGREGATE 'TRUE'|'FALSE'
Indicatesthefunctionisauserdefinedaggregatefunction.Propertiesspecifictoaggregatesarelistedbelow.
NotethatNULL-ON-NULL,VARARGS,andalloftheAGGREGATEpropertiesarealsovalidrelationalextensionmetadatapropertiesthatcanbeusedonsourceproceduresmarkedasfunctions.SeealsoSourceSupportedFunctionsforcreatingFOREIGNfunctionsthataresupportedbyasource.
AGGREGATEFUNCTIONOPTIONS:
Property DataTypeorAllowedValues Description
ANALYTIC 'TRUE'|'FALSE' indicatestheaggregatefunctionmustbewindowed.defaultfalse.
ALLOWS-ORDERBY 'TRUE'|'FALSE'indicatestheaggregatefunctionsupportsanORDERBYclause.defaultfalse
ALLOWS-DISTINCT 'TRUE'|'FALSE'indicatestheaggregatefunctionsupportstheDISTINCTkeyword.defaultfalse
DECOMPOSABLE 'TRUE'|'FALSE'
indicatesthesingleargumentaggregatefunctioncanbedecomposedasagg(agg(x))oversubsetsofdata.defaultfalse
USES-DISTINCT-ROWS 'TRUE'|'FALSE'indicatestheaggregatefunctioneffectivelyusesdistinctrowsratherthanallrows.defaultfalse
NotethatvirtualfunctionsdefinedusingtheTeiidprocedurelanguagecannotbeaggregatefunctions.
NoteProvidingtheJARlibraries-IfyouhavedefinedaUDF(virtual)functionwithoutaTeiidproceduredeinition,thenitmustbeaccompaniedbyitsimplementationinJava.ToconfiguretheJavalibraryasdependencytotheVDB,seeSupportforUser-DefinedFunctions
PROCEDUREOPTIONS:(thebelowarewellknownoptions,anyotherspropertiesdefinedwillbeconsideredasextensionmetadata)
Property DataTypeorAllowedValues Description
UUID string UniqueIdentifier
SchemaObjectDDL
325
NAMEINSOURCE string Inthecaseofsource
ANNOTATION string Descriptionoftheprocedure
UPDATECOUNT int
ifthisprocedureupdatestheunderlyingsources,whatistheupdatecount,whenupdatecountis>1theXAprotocolforexecutionisenforced
Example:DefinevirtualProcedure
CREATEVIRTUALPROCEDURECustomerActivity(customeridinteger)
RETURNS(namevarchar(25),activitydatedate,amountdecimal)
AS
BEGIN
...
END
ReadmoreinformationaboutvirtualproceduresatVirtualProcedures,andtheseproceduresarewrittenusingProcedureLanguage
Example:DefineVirtualFunction
CREATEVIRTUALFUNCTIONCustomerRank(customeridinteger)
RETURNSintegerAS
BEGIN
...
END
ProcedurecolumnsmayalsobemarkedasNOTNULL,orwithaDEFAULTvalue.Currentlyonlystringvaluesaresupportedasthedefaultvalue.Tohavethestringinterpretedasanexpressionusetheextensionpropertyteiid_rel:default_handlingsettoexpression.OnasourceprocedureifyouwanttheparametertobedefaultableinthesourceprocedureandnotsupplyadefaultvalueinTeiid,thentheparametermustbenullableandusetheextensionpropertyteiid_rel:default_handlingsettoomit.
TherecanonlybeasingleRESULTparameteranditmustbeanoutparameter.ARESULTparameteristhesameashavingasinglenon-tableRETURNStype.Ifbotharedeclaredtheyareexpectedtomatchotherwiseanexceptionisthrown.Oneisnomorecorrectthantheother."RETURNStype"isshorterhandsyntaxespeciallyforfunctions,whiletheparameterformisusefulforadditionalmetadata(explicitname,extensionmetadata,alsodefiningareturnstable,etc.)
RelationalExtensionOPTIONS:
Property DataTypeorAllowedValues Description
native-query ParameterizedString
Appliestobothfunctionsandprocedures.Thereplacementforthefunctionsyntaxratherthanthestandardprefixformwithparens.SeealsoTranslators#native
non-prepared boolean
AppliestoJDBCproceduresusingthenative-queryoption.IftrueaPreparedStatementwillnotbeusedtoexecutethenativequery.
Example:NativeQuery
CREATEFOREIGNFUNCTIONfunc(xinteger,yinteger)
RETURNSintegerOPTIONS("teiid_rel:native-query"'$1<<$2');
SchemaObjectDDL
326
Example:SequenceNativeQuery
CREATEFOREIGNFUNCTIONseq_nextval()
RETURNSinteger
OPTIONS("teiid_rel:native-query"'seq.nextval');
Tip UntilTeiidprovideshigher-levelmetadatasupportforsequences,asourcefunctionrepresentationisthebestfittoexposesequencefunctionality.
ExtensionMetadata
WhendefiningtheextensionmetadatainthecaseofCustomTranslators,thepropertiesontables/views/procedures/columnscandefinenamespaceforthepropertiessuchthattheywillnotcollidewiththeTeiidspecificproperties.ThepropertyshouldbeprefixedwithaliasoftheNamespace.Prefixesstartingwithteiid_arereservedforusebyTeiid.
ExampleofNamespace
SETNAMESPACE'http://custom.uri'ASfoo
CREATEVIEWMyView(...)
OPTIONS("foo:mycustom-prop"'anyvalue')
Built-inNamespacePrefixes
Prefix URI Description
teiid_rel http://www.teiid.org/ext/relational/2012Relationalextensions.Usesincludefunctionandnativequerymetadata
teiid_sf http://www.teiid.org/translator/salesforce/2012 Salesforceextensions.
teiid_mongo http://www.teiid.org/translator/mongodb/2013 MongoDBExtensions
teiid_odata http://www.jboss.org/teiiddesigner/ext/odata/2012 ODataExtensions
teiid_accumulo http://www.teiid.org/translator/accumulo/2013 Accumuloextensions
teiid_excel http://www.teiid.org/translator/excel/2014 ExcelExtensions
teiid_ldap http://www.teiid.org/translator/ldap/2015 LDAPExtensions
teiid_rest http://teiid.org/rest RESTExtensions
teiid_pi http://www.teiid.org/translator/pi/2016 PIDatabaseExtensions
SchemaObjectDDL
327
SchemaObjectDDL
328
MultisourceModelsMultisourcemodelscanbeusedtoquicklyaccessdatainmultiplesourceswithhomogeneousmetadata.Whenyouhavemultipleinstancesusingidenticalschema(horizontalsharding),Teiidcanhelpyougatherdataacrossalltheinstances,using"multisource"models.Inthisscenario,insteadofcreating/importingamodelforeverydatasource,onesourcemodelisdefinedtorepresentstheschemaandisconfiguredwithmultipledata"sources"underneathit.Duringruntimewhenaqueryissuedagainstthismodel,thequeryengineanalyzestheinformationandgatherstherequireddatafromallsourcesconfiguredandgatherstheresultsandprovidesinasingleresult.Sinceallsourcesutilizethesamephysicalmetadata,thisfeatureismostappropriateforaccessingthesamesourcetypewithmultipleinstances.
Configuration
Tomarkamodelasmultisource,themodelpropertymultisourcecanbesettotrueormorethanonesourcecanbelistedforthemodelinthe"vdb.xml"file.Hereisacodeexampleshowingavdbwithsinglemodelwithmultiplesourcesdefined.
<vdbname="vdbname"version="1">
<modelvisible="true"type="PHYSICAL"name="Customers"path="/Test/Customers.xmi">
<propertyname="multisource"value="true"/>
<!--optionalproperties
<propertyname="multisource.columnName"value="somename"/>
<propertyname="multisource.addColumn"value="true"/>
-->
<sourcename="chicago"
translator-name="oracle"connection-jndi-name="chicago-customers"/>
<sourcename="newyork"
translator-name="oracle"connection-jndi-name="newyork-customers"/>
<sourcename="la"
translator-name="oracle"connection-jndi-name="la-customers"/>
</model>
</vdb>
NOTECurrentlythetoolingsupportformanagingthemultisourcefeatureislimited,soifyouneedtousethisfeaturebuildtheVDBasusualintheTeiidDesignerandtheneditthe"vdb.xml"fileintheVDBarchiveusingaTexteditortoaddtheadditionalsourcesasdefinedabove.Youmustdeployaseparatedatasourceforeachsourcedefinedinthexmlfile.
Intheaboveexample,theVDBhasasinglemodelcalledCustomers,thathasmultiplesources(chicago,newyork,andla)thatdefinedifferentinstancesofdata.
TheMultisourceColumn
Whenamodelismarkedasmultisource,theenginewilladdoruseanexistingcolumnoneachtabletorepresentthesourcenamevalues.Intheabovevdb.xmlthecolumnwouldreturnchicago,la,newyorkforeachoftherespectivesources.ThenameofthecolumndefaultstoSOURCE_NAME,butisconfigurablebysettingthemodelpropertymultisource.columnName.Ifacolumnalreadyexistsonthetable(oranINprocedureparameter)withthesamename,theenginewillassumethatitshouldrepresentthemultisourcecolumnanditwillnotbeusedtoretrievephysicaldata.Ifthemultisourcecolumnisnotpresent,thegeneratedcolumnwillbetreatedasapseudocolumnwhichisnotselectableviawildcards(*nortbl.*).
Thisallowsquerieslikethefollowing:
select*fromtablewhereSOURCE_NAME='newyork'
updatetablecolumn=valuewhereSOURCE_NAME='chicago'
deletefromtablewherecolumn=xandSOURCE_NAME='la'
insertintotable(column,SOURCE_NAME)VALUES('value','newyork')
MultiSourceModels
329
TheMulti-SourceColumninSystemMetadata
Thepseudocolumnisbydefaultnotpresentinyouractualmetadata;itisnotaddedonsourcetables/procedureswhenyouimportthemetadata.Ifyouwouldliketousethemultisourcecolumninyourtransformationstocontrolwhichsourcesareaccessedorupdatedand/orwantthecolumnreportedviametadatafacilities,thereareseveraloptions:
IfdirectlyusingDDL,thepseduo-columnwillalreadybeavailabletotransformations,butwillnotbepresentinyourSystemmetadatabydefault.IfusingDDLandwanttobeselective(ratherthanusingthemultisource.addColumnproperty),youcanmanuallyaddthecolumnviaDDL.
WitheitherVDBtypetomakethemultisourcecolumnpresentinthesystemmetadata,youmaysetthemodelpropertymultisource.addColumntotrueonamultisourcemodel.Ifthetablehasacolumnortheprocedurehasaparameteralreadywithamatchingname,thenanadditionalcolumnwillnotbeadded.Avariadicprocedurecanstillhaveasourceparameteradded,butitcanonlybespecifiedwhenusingnamedparameters.CareshouldbetakenthoughwhenusingthispropertyinDesignerasanytransformationlogic(views/procedures)thatyouhavedefinedwillnothavebeenawareofthemultisourcecolumnandmayfailvalidationuponserverdeployment.
IfusingDesigner,youcanmanuallyaddthemultisourcecolumn.
PlanningandExecutionTheplannerlogicallytreatsamultisourcetableasifitwereaviewcontainingtheunionalloftherespectivesourcetables.Morecomplexpartitioningscenarios,suchasheterogeneoussourcesorlistpartitioningwillrequiretheuseofaFederatedOptimizations#PartitionedUnion.
Mostofthefederatedoptimizationsavailableoverunionsarestillapplicableinmultisourcemode.Thisincludesaggregationpushdown/decomposition,limitpushdown,joinpartitioning,etc.
Youcanadd/removesourcesfrommultisourcemodelsatruntimewiththeadminaddSourceandremoveSourceoptions.Theprocessingofamultisourceplanwilldeterminethesetofmultisourcetargetswhentheaccessnodeisopened.Iftheplanisreusedandthesourceschangesincethelastexecution,themultisourceaccesswillberegenerated.Ifasourceisaddedafterarelevantmultisourcequerystarts,itwillnotbeintheresults.Ifasourceisremovedafterarelevantmultisourcequerystarts,itwillbetreatedasanullsourcewhichshouldinmostsituationsallowthequerytocompletenormally.
ThattheSHOWPLANoutputwillvaryuponwhenitisobtained.IfyougettheSHOWPLANoutputpriortoexecution,themultisourceaccesswillappearasasingleaccessnode.AfterexecutiontheSHOWPLANoutputwillshowthesetofsourcesaccessedasindividualnodes.
SELECTs,UPDATEs,DELETEs
AmultisourcequeryagainstaSELECT/UPDATE/DELETEmayaffectanysubsetofthesourcesbasedupontheevaluationoftheWHEREclause.
Themultisourcecolumnmaynotbetargetedinanupdatechangeset.
ThesumoftheupdatecountsforUPDATEs/DELETEswillbereturnedastheresultantupdatecount.
Whenrunningunderatransactioninamodethatdetectstheneedforatransactionandmultipleupdatesmayperformedoratransactionalreadisrequiredandmultiplesourcesmaybereadfrom,atransactionwillbestartedtoenlisteachsource.
INSERTs
AmultisourceINSERTmustusethesource_namecolumnasaninsertcolumntospecifywhichsourceshouldbetargetedbytheINSERT.OnlyanINSERTusingtheVALUESclauseissupported.
MultiSourceModels
330
StoredProcedures
Aphysicalstoredproceduresrequirestheadditionofastringinparametermatchingthemultisourcecolumnnametospecifywhichsourcetheprocedureisexecutedon.Iftheparameterisnotpresentanddefaultstoanullvalue,thentheprocedurewillbeexecutedoneachsource.ItisnotpossibletoexecuteproceduresthatarerequiredtoreturnIN/OUT,OUT,orRETURNparametersvaluesonmorethan1source.
ExampleDDL
CREATEFOREIGNPROCEDUREPROC(arg1INSTRINGNOTNULL,arg2INSTRING,SOURCE_NAMEINSTRING)
ExampleCallsAgainstASingleSource
CALLPROC(arg1=>'x',SOURCE_NAME=>'sourceA')
EXECPROC('x','y','sourceB')
ExampleCallsAgainstAllSources
CALLPROC(arg1=>'x')
EXECPROC('x','y')
MultiSourceModels
331
MetadataRepositoriesTraditionallythemetadataforaVirtualDatabaseisbuiltbyTeiidDesignerandsuppliedtoTeiidenginethroughaVDBarchivefile.ThisVDBfilecontains.INDEXmetadatafiles.BydefaulttheyareloadedbyaMetadataRepositorywiththenameINDEX.Otherbuilt-inmetadatarepositoriesincludethefollowing:
NATIVE
Thisisonlyapplicableonsourcemodels(andisalsothedefault),whenusedthemetadataforthemodelisretrievedfromthesourcedatabaseitself.
Samplevdb.xmlfile
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="NATIVE"></metadata>
</model>
</vdb>
DDL
Samplevdb.xmlfile
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="DDL">
**DDLHere**
</metadata>
</model>
</vdb>
Thisisapplicabletobothsourceandviewmodels.SeeDDLMetadataformoreinformationonhowtousethisfeature.
FILE
Samplevdb.xmlfile
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="DDL-FILE">/accounts.ddl</metadata>
</model>
</vdb>
DDLisapplicabletobothsourceandviewmodelsinzipVDBdeployments.SeeDDLMetadataformoreinformationonhowtousethisfeature.
ChainingRepositories
MetadataRepositories
332
Whendefiningthemetadatatypeforamodel,multiplemetadataelementscanbeused.Alltherepositoryinstancesdefinedareconsultedintheorderconfiguredtogatherthemetadataforthegivenmodel.Forexample:
Samplevdb.xmlfile
<vdbname="{vdb-name}"version="1">
<modelname="{model-name}"type="PHYSICAL">
<sourcename="AccountsDB"translator-name="oracle"connection-jndi-name="java:/oracleDS"/>
<metadatatype="NATIVE"/>
<metadatatype="DDL">
**DDLHere**
</metadata>
</model>
</vdb>
Note Fortheabovemodel,NATIVEimporterisfirstused,thenDDLimporterusedtoaddadditionalmetadatatoNATIVEimportedmetadata.
Custom
SeeCustomMetadataRepository
MetadataRepositories
333
RESTServiceThroughVDBWithhelpofDDLMetadatavarietyofmetadatacanbedefinedonVDBschemamodels.Thismetadataisnotlimitedtojustdefiningthetables,proceduresandfunctions.ThecapabilitiesofsourcesystemsoranyextensionstometadatacanalsobedefinedontheschemaobjectsusingtheOPTIONSclause.OnesuchextensionpropertiesthatTeiiddefinesistoexposeTeiidproceduresasRESTbasedservices.
ExposeTeiidProcedureasRestService
OnecandefinebelowRESTbasedpropertiesonaTeiidvirtualprocedure,andwhentheVDBisdeployedtheTeiidVDBdeployerwillanalyzethemetadataanddeployaRESTserviceautomatically.WhentheVDBun-deployedtheRESTservicealsodeployed.
PropertyName Description IsRequired AllowedValues
METHOD HTTPMethodtouse Yes GET|POST|PUT|DELETE
URI URIofprocedure Yes ex:/procedure
PRODUCES Typeofcontentproducedbytheservice no xml|json|plain|anytext
CHARSET
WhenprocedurereturnsBlob,andcontenttypetextbased,thischaractersettousedtoconvertthedata
no US-ASCII|UTF-8
TheabovepropertiesmustbedefinedwithNAMESPACE`http://teiid.org/rest'onthemetadata.HereisanexampleVDBthatdefinestheRESTbasedservice.
ExampleVDBwithRESTbasedmetadataproperties
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="sample"version="1">
<propertyname="{http://teiid.org/rest}auto-generate"value="true"/>
<modelname="PM1">
<sourcename="text-connector"translator-name="loopback"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLEG1(e1string,e2integer);
CREATEFOREIGNTABLEG2(e1string,e2integer);
]]></metadata>
</model>
<modelname="View"type="VIRTUAL">
<metadatatype="DDL"><![CDATA[
SETNAMESPACE'http://teiid.org/rest'ASREST;
CREATEVIRTUALPROCEDUREg1Table(INp1integer)RETURNSTABLE(xml_outxml)OPTIONS(UPDATECOUNT0,
"REST:METHOD"'GET',"REST:URI"'g1/{p1}')
AS
BEGIN
SELECTXMLELEMENT(NAME"rows",XMLATTRIBUTES(g1Table.p1asp1),XMLAGG(XMLELEMENT(NAME"row",
XMLFOREST(e1,e2))))ASxml_outFROMPM1.G1;
END
]]></metadata>
</model>
</vdb>
RESTServiceThroughVDB
334
Note
<propertyname="\{http://teiid.org/rest[http://teiid.org/rest]}auto-generate"value="true"/>,canbeusedtocontrolthegenerationoftheRESTbasedWARbasedontheVDB.ThispropertyalongwithatleastoneprocedurewithRESTbasedextensionmetadataisrequiredtogenerateaRESTWARfile.Also,theprocedureneedstoreturnresultsetwithsinglecolumnofeitherXML,Clob,BloborString.WhenPRODUCESpropertyisnotdefined,thispropertyisderivedfromtheresultcolumnthatisprojectedout.
whentheaboveVDBisdeployedintheWildFly+Teiidserver,andiftheVDBisvalidandafterthemetadataisloadedthenaRESTwargeneratedautomaticallyanddeployedintothelocalWildFlyserver.TheRESTVDBisdeployedwith"{vdb-name}_{vdb-version}"context.Themodelnameisprependedtourioftheservicecall.Forexampletheprocedureinaboveexamplecanbeaccessedas
http://{host}:8080/sample_1/view/g1/123
where"sample_1"iscontext,"view"ismodelname,"g1"isURI,and123isparameter{p1}fromURI.IfyoudefinedaprocedurethatreturnsaXMLcontent,thenRESTservicecallshouldbecalledwith"accepts"HTTPheaderof"application/xml".Also,ifyoudefinedaprocedurethatreturnsaJSONcontentandPRODUCESpropertyisdefined"json"thenHTTPclientcallshouldincludethe"accepts"headerof"application/json".Inthesituationswhere"accepts"headerismissing,andonlyoneprocedureisdefinedwithuniquepath,thatprocedurewillbeinvoked.IftherearemultipleprocedureswithsameURIpath,forexampleonegeneratingXMLandanothergeneratingJSONcontentthen"accepts"headerdirectstheRESTengineastowhichprocedureshouldbeinvokedtogettheresults.Awrong"accepts"headerwillresultinerror.
"GETMethods"
WhendesigningtheproceduresthatwillbeinvokedthroughGETbasedcall,theinputparametersforprocedurescanbedefinedinthePATHoftheURI,asthe{p1}exampleabove,ortheycanalsobedefinedasqueryparameter,orcombinationofboth.Forexample
http://{host}:8080/sample_1/view/g1?p1=123
http://{host}:8080/sample_1/view/g1/123?p2=foo
MakesurethatthenumberofparametersdefinedontheURIandquerymatchtotheparametersdefinedonproceduredefinition.Ifyoudefinedadefaultvalueforaparameterontheprocedure,andthatparametergoingtobepassedinqueryparameteronURLthenyouhavechoicetoomitthatqueryparameter,ifyoudefinedasPATHyoumustsupplyavalueforit.
"POSTmethods"
'POST'methodsMUSTnotbedefinedwithURIwithPATHSforparametersasinGEToperations,theprocedureparametersareautomaticallyaddedas@FormParamannotationsonthegeneratedprocedure.AclientinvokingthisservicemustuseFORMtopostthevaluesfortheparameters.TheFORMfieldnamesMUSTmatchthenamesoftheprocedureparametersnames.
IfanyoneoftheprocedureparametersareBLOB,CLOBorXMLtype,thenPOSToperationcanbeonlyinvokedusing"multipart/form-data"RFC-2388protocol.ThisallowsusertouploadlargebinaryorXMLfilesefficientlytoTeiidusingstreaming".
"VARBINARYtype"
IfaparametertotheprocedureisVARBINARYtypethenthevalueoftheparametermustbeproperlyBASE64encoded,irrespectiveoftheHTTPmethodusedtoexecutetheprocedure.IfthisVARBINARYhaslargecontent,thenconsiderusingBLOB.
SecurityonGeneratedServices
RESTServiceThroughVDB
335
BydefaultallthegeneratedRestbasedservicesaresecuredusing"HTTPBasic"withsecuritydomain"teiid-security"andwithsecurityrole"rest".However,thesepropertiescanbecustomizedbydefiningthetheninvdb.xmlfile.
Examplevdb.xmlfilesecurityspecification
<vdbname="sample"version="1">
<propertyname="{http://teiid.org/rest}auto-generate"value="true"/>
<propertyname="{http://teiid.org/rest}security-type"value="HttpBasic"/>
<propertyname="{http://teiid.org/rest}security-domain"value="teiid-security"/>
<propertyname="{http://teiid.org/rest}security-role"value="example-role"/>
...
</vdb>
_auto-generate-willautomaticallygeneratetheWARfileforthedeployedVDB
security-type-definesthesecuritytype.allowedvaluesare"HttpBasic"or"none".Ifomittedwilldefaultto"HttpBasic"
security-domain-definesJAASsecuritydomaintobeusedwithHttpBasic.Ifomittedwilldefaultto"teiid-security"
security-role-securityrolethatHttpBasicwillusetoauthorizetheusers.Ifomittedthevaluewilldefaultto"rest"
Note rest-security-itisourintentiontoprovideothertypesofsecuritieslikeKerberosandOAuth2infuturereleases.
SpecialAd-HocRestServices
Apartfromtheexplicitlydefinedprocedurebasedrestservices,thegeneratedjax-rswarfilewillalsoimplicitlyincludeaspecialrestbasedserviceunderURI"/query"thatcantakeanyXMLorJSONproducingSQLasparameterandexposetheresultsofthatqueryasresultoftheservice.Thisserviceisdefinedwith"POST",acceptingaFormParameternamed"sql".Forexample,afteryoudeploytheVDBdefinedinaboveexample,youcanissueaHTTPPOSTcallas
http://localhost:8080/sample_1/view/query
sql=SELECTXMLELEMENT(NAME"rows",XMLAGG(XMLELEMENT(NAME"row",XMLFOREST(e1,e2))))ASxml_outFROMPM1.G1
AsampleHTTPRequestfromJavacanbemadelikebelow
publicstaticStringhttpCall(Stringurl,Stringmethod,Stringparams)throwsException{
StringBufferbuff=newStringBuffer();
HttpURLConnectionconnection=(HttpURLConnection)newURL(url).openConnection();
connection.setRequestMethod(method);
connection.setDoOutput(true);
if(method.equalsIgnoreCase("post")){
OutputStreamWriterwr=newOutputStreamWriter(connection.getOutputStream());
wr.write(params);
wr.flush();
}
BufferedReaderserverResponse=newBufferedReader(newInputStreamReader(connection.getInputStream()));
Stringline;
while((line=serverResponse.readLine())!=null){
buff.append(line);
}
returnbuff.toString();
}
publicstaticvoidmain(String[]args)throwsException{
Stringparams=URLEncoder.encode("sql","UTF-8")+"="+URLEncoder.encode("SELECTXMLELEMENT(NAME"ro
ws",XMLAGG(XMLELEMENT(NAME"row",XMLFOREST(e1,e2))))ASxml_outFROMPM1.G1","UTF-8");
httpCall("http://localhost:8080/sample_1/view/query","POST",params);
}
RESTServiceThroughVDB
336
RESTServiceThroughVDB
337
SQLSupportTeiidprovidesnearlyallofthefunctionalityofSQL-92DML.SQL-99andlaterfeaturesareconstantlybeingaddedbaseduponcommunityneed.ThefollowingdoesnotattempttocoverSQLexhaustively,butratherhighlightsSQL’susagewithinTeiid.SeetheBNFforSQLGrammarfortheexactformofSQLacceptedbyTeiid.
SQLSupport
338
IdentifiersSQLcommandscontainreferencestotablesandcolumns.Thesereferencesareintheformofidentifiers,whichuniquelyidentifythetablesandcolumnsinthecontextofthecommand.Allqueriesareprocessedinthecontextofavirtualdatabase,orVDB.Becauseinformationcanbefederatedacrossmultiplesources,tablesandcolumnsmustbescopedinsomemannertoavoidconflicts.Thisscopingisprovidedbyschemas,whichcontaintheinformationforeachdatasourceorsetofviews.
Fully-qualifiedtableandcolumnnamesareofthefollowingform,wheretheseparate`parts'oftheidentifieraredelimitedbyperiods.
TABLE:<schema_name>.<table_spec>
COLUMN:<schema_name>.<table_spec>.<column_name>
SyntaxRules:
Identifierscanconsistofalphanumericcharacters,ortheunderscore(_)character,andmustbeginwithanalphabeticcharacter.AnyUnicodecharactermaybeusedinanidentifier.
Identifiersindoublequotescanhaveanycontents.Thedoublequotecharactercanit’sbeescapedwithanadditionaldoublequote.e.g."some","id"
Becausedifferentdatasourcesorganizetablesindifferentways,someprependingcatalogorschemaoruserinformation,Teiidallowstablespecificationtobeadot-delimitedconstruct.
NoteWhenatablespecificationcontainsadotresolvingwillallowforthematchofapartialnameagainstanynumberoftheendsegmentsinthename.e.g.atablewiththefully-qualifiednamevdbname."sourcescema.sourcetable"wouldmatchthepartialnamesourcetable.
Columns,schemas,andaliasesidentifierscannotcontainadot.
Identifiers,evenwhenquoted,arenotcase-sensitiveinTeiid.
Someexamplesofvalidfully-qualifiedtableidentifiersare:
MySchema.Portfolios
"MySchema.Portfolios"
MySchema.MyCatalog.dbo.Authors
Someexamplesofvalidfully-qualifiedcolumnidentifiersare:
MySchema.Portfolios.portfolioID
"MySchema.Portfolios"."portfolioID"
MySchema.MyCatalog.dbo.Authors.lastName
Fully-qualifiedidentifierscanalwaysbeusedinSQLcommands.Partially-orunqualifiedformscanalsobeused,aslongastheresultingnamesareunambiguousinthecontextofthecommand.Differentformsofqualificationcanbemixedinthesamequery.
ReservedWordsTeiid’sreservedwordsincludethestandardSQL2003Foundation,SQL/MED,andSQL/XMLreservedwords,aswellasTeiidspecificwordssuchasBIGINTEGER,BIGDECIMAL,orMAKEDEP.SeetheBNFforSQLGrammarReservedKeywordsandReservedKeywordsForFutureUsesectionsforallreservedwords.
Identifiers
339
Identifiers
340
ExpressionsIdentifiers,literals,andfunctionscanbecombinedintoexpressions.Expressionscanbeusedalmostanywhereinaquery–SELECT,FROM(ifspecifyingjoincriteria),WHERE,GROUPBY,HAVING,orORDERBY.
Teiidsupportsthefollowingtypesofexpressions:
ColumnIdentifiers
Literals
ScalarFunctions
AggregateFunctions
WindowFunctions
CaseandSearchedCase
ScalarSubqueries
ParameterReferences
Criteria
Arrays
ColumnIdentifiers
ColumnidentifiersareusedtospecifytheoutputcolumnsinSELECTstatements,thecolumnsandtheirvaluesforINSERTandUPDATEstatements,andcriteriausedinWHEREandFROMclauses.TheyarealsousedinGROUPBY,HAVING,andORDERBYclauses.ThesyntaxforcolumnidentifierswasdefinedintheIdentifierssectionabove.
LiteralsLiteralvaluesrepresentfixedvalues.Thesecananyofthe'standard'datatypes.
SyntaxRules:
Integervalueswillbeassignedanintegraldatatypebigenoughtoholdthevalue(integer,long,orbiginteger).
Floatingpointvalueswillalwaysbeparsedasadouble.
Thekeyword'null'isusedtorepresentanabsentorunknownvalueandisinherentlyuntyped.Inmanycases,anullliteralvaluewillbeassignedanimpliedtypebasedoncontext.Forexample,inthefunction'5+null',thenullvaluewillbeassignedthetype'integer'tomatchthetypeofthevalue'5'.AnullliteralusedintheSELECTclauseofaquerywithnoimpliedcontextwillbeassignedtotype'string'.
Someexamplesofsimpleliteralvaluesare:
'abc'
escapedsingletick
'isn"ttrue'
Expressions
341
5
scientificnotation
-37.75e01
exactnumerictypeBigDecimal
100.0
true
false
unicodecharacter
'\u0027'
binary
X'0F0A'
Date/TimeLiteralscanuseeitherJDBCEscapedLiteralSyntax:
DateLiteral
{d'...'}
TimeLiteral
{t'...'}
TimestampLiteral
{ts'...'}
OrtheANSIkeywordsyntax:
DateLiteral
DATE'...'
TimeLiteral
TIME'...'
TimestampLiteral
TIMESTAMP'...'
Eitherwaythestringliteralvalueportionoftheexpressionisexpectedtofollowthedefinedformat-"yyyy-MM-dd"fordate,"hh:mm:ss"fortime,and"yyyy-MM-dd[hh:mm:ss[.fff…]]"fortimestamp.
Expressions
342
AggregateFunctions
AggregatefunctionstakesetsofvaluesfromagroupproducedbyanexplicitorimplicitGROUPBYandreturnasinglescalarvaluecomputedfromthegroup.
Teiidsupportsthefollowingaggregatefunctions:
COUNT(*)–countthenumberofvalues(includingnullsandduplicates)inagroup.Returnsaninteger-anexceptionwillbethrownifalargercountiscomputed.
COUNT(x)–countthenumberofvalues(excludingnulls)inagroup.Returnsaninteger-anexceptionwillbethrownifalargercountiscomputed.
SUM(x)–sumofthevalues(excludingnulls)inagroup
AVG(x)–averageofthevalues(excludingnulls)inagroup
MIN(x)–minimumvalueinagroup(excludingnull)
MAX(x)–maximumvalueinagroup(excludingnull)
ANY(x)/SOME(x)–returnsTRUEifanyvalueinthegroupisTRUE(excludingnull)
EVERY(x)–returnsTRUEifeveryvalueinthegroupisTRUE(excludingnull)
VAR_POP(x)–biasedvariance(excludingnull)logicallyequals(sum(x^2)-sum(x)^2/count(x))/count(x);returnsadouble;nullifcount=0
VAR_SAMP(x)–samplevariance(excludingnull)logicallyequals(sum(x^2)-sum(x)^2/count(x))/(count(x)-1);returnsadouble;nullifcount<2
STDDEV_POP(x)–standarddeviation(excludingnull)logicallyequalsSQRT(VAR_POP(x))
STDDEV_SAMP(x)–samplestandarddeviation(excludingnull)logicallyequalsSQRT(VAR_SAMP(x))
TEXTAGG(expression[asname],…[DELIMITERchar][QUOTEchar|NOQUOTE][HEADER][ENCODINGid][ORDERBY…])–CSVtextaggregationofallexpressionsineachrowofagroup.WhenDELIMITERisnotspecified,bydefaultcomma(,)isusedasdelimiter.Allnon-nullvalueswillbequoted.Doublequotes(")isthedefaultquotecharacter.UseQUOTEtospecifyadifferentvalue,orNOQUOTEfornovaluequoting.IfHEADERisspecified,theresultcontainstheheaderrowasthefirstline-theheaderlinewillbepresenteveniftherearenorowsinagroup.Thisaggregationreturnsablob.
TEXTAGG(col1,col2asnameDELIMITER'|'HEADERORDERBYcol1)
XMLAGG(xml_expr[ORDERBY…])–xmlconcatenationofallxmlexpressionsinagroup(excludingnull).TheORDERBYclausecannotreferencealiasnamesorusepositionalordering.
JSONARRAY_AGG(x[ORDERBY…])–createsaJSONarrayresultasaClobincludingnullvalue.TheORDERBYclausecannotreferencealiasnamesorusepositionalordering.SeealsotheJSONArrayfunction.
integervalueexample
jsonArray_Agg(col1orderbycol1nullsfirst)
couldreturn
[null,null,1,2,3]
Expressions
343
STRING_AGG(x,delim)–createsalobresultsfromtheconcatenationofxusingthedelimiterdelim.Ifeitherargumentisnull,novalueisconcatenated.Bothargumentsareexpectedtobecharacter(string/clob)orbinary(varbinary,blob)andtheresultwillbecloborblobrespectively.DISTINCTandORDERBYareallowedinSTRING_AGG.
stringaggexample
string_agg(col1,','ORDERBYcol1ASC)
couldreturn
'a,b,c'
ARRAY_AGG(x[ORDERBY…])–createsanarraywithabasetypematchingtheexpressionx.TheORDERBYclausecannotreferencealiasnamesorusepositionalordering.
agg([DISTINCT|ALL]arg…[ORDERBY…])–auserdefinedaggregatefunction
SyntaxRules:
Someaggregatefunctionsmaycontainakeyword'DISTINCT'beforetheexpression,indicatingthatduplicateexpressionvaluesshouldbeignored.DISTINCTisnotallowedinCOUNT(*)andisnotmeaningfulinMINorMAX(resultwouldbeunchanged),soitcanbeusedinCOUNT,SUM,andAVG.
AggregatefunctionscannotbeusedinFROM,GROUPBY,orWHEREclauseswithoutaninterveningqueryexpression.
Aggregatefunctionscannotbenestedwithinanotheraggregatefunctionwithoutaninterveningqueryexpression.
Aggregatefunctionsmaybenestedinsideotherfunctions.
AnyaggregatefunctionmaytakeanoptionalFILTERclauseoftheform
FILTER(WHEREcondition)
Theconditionmaybeanybooleanvalueexpressionthatdoesnotcontainasubqueryoracorrelatedvariable.Thefilterwilllogicallybeevaluatedforeachrowpriortothegroupingoperation.Iffalsetheaggregatefunctionwillnotaccumulateavalueforthegivenrow.
Formoreinformationonaggregates,seethesectionsonGROUPBYorHAVING.
WindowFunctionsTeiidsupportsANSISQL2003windowfunctions.Awindowfunctionallowsanaggregatefunctiontobeappliedtoasubsetoftheresultset,withouttheneedforaGROUPBYclause.Awindowfunctionissimilartoanaggregatefunction,butrequirestheuseofanOVERclauseorwindowspecification.
Usage:
aggregate|rankingOVER([PARTITIONBY...]][ORDERBY...])
aggregatecanbeanyAggregateFunctions.RankingcanbeoneofROW_NUMBER(),RANK(),DENSE_RANK().
SyntaxRules:
WindowfunctionscanonlyappearintheSELECTandORDERBYclausesofaqueryexpression.
Windowfunctionscannotbenestedinoneanother.
Expressions
344
Partitioningandorderbyexpressionscannotcontainsubqueriesorouterreferences.
Theranking(ROW_NUMBER,RANK,DENSE_RANK)functionsrequiretheuseofthewindowspecificationORDERBYclause.
AnXMLAGGorJSONARRAY_AGGORDERBYclausecannotbeusedwhenwindowed.
ThewindowspecificationORDERBYclausecannotreferencealiasnamesorusepositionalordering.
WindowedaggregatesmaynotuseDISTINCTifthewindowspecificationisordered.
AnalyticalFunctionDefinitionsROW_NUMBER()–functionalthesameasCOUNT(*)withthesamewindowspecification.Assignsanumbertoeachrowinapartitionstartingat1.
RANK()–Assignsanumbertoeachuniqueorderingvaluewithineachpartitionstartingat1,suchthatthenextrankisequaltothecountofpriorrows.
DENSE_RANK()–Assignsanumbertoeachuniqueorderingvaluewithineachpartitionstartingat1,suchthatthenextrankissequential.
Allvaluesareintegers-anexceptionwillbethrownifalargervalueisneeded.
ProcessingWindowfunctionsarelogicallyprocessedjustbeforecreatingtheoutputfromtheSELECTclause.WindowfunctionscanusenestedaggregatesifaGROUPBYclauseispresent.Theisnoguaranteedaffectontheoutputorderingfromthepresenceofwindowfunctions.TheSELECTstatementmusthaveanORDERBYclausetohaveapredictableordering.
Teiidwillprocessallwindowfunctionswiththesamewindowspecificationtogether.IngeneralafullpassovertherowvaluescomingintotheSELECTclausewillberequiredforeachuniquewindowspecification.ForeachwindowspecificationthevalueswillbegroupedaccordingtothePARTITIONBYclause.IfnoPARTITIONBYclauseisspecified,thentheentireinputistreatedasasinglepartition.Theoutputvalueisdeterminedbaseduponthecurrentrowvalue,it’speers(thatisrowsthatarethesamewithrespecttotheirordering),andallpriorrowvaluesbaseduponorderinginthepartition.TheROW_NUMBERfunctionwillassignauniquevaluetoeveryrowregardlessofthenumberofpeers.
ExampleWindowedResults
SELECTname,salary,max(salary)over(partitionbyname)asmax_sal,
rank()over(orderbysalary)asrank,dense_rank()over(orderbysalary)asdense_rank,
row_number()over(orderbysalary)asrow_numFROMemployees
name salary max_sal rank dense_rank row_num
John 100000 100000 2 2 2
Henry 50000 50000 5 4 5
John 60000 100000 3 3 3
Suzie 60000 150000 3 3 4
Suzie 150000 150000 1 1 1
Expressions
345
CaseandSearchedCase
TeiidsupportstwoformsoftheCASEexpressionwhichallowsconditionallogicinascalarexpression.
Supportedforms:
CASE<expr>(WHEN<expr>THEN<expr>)+[ELSEexpr]END
CASE(WHEN<criteria>THEN<expr>)+[ELSEexpr]END
Eachformallowsforanoutputbasedonconditionallogic.ThefirstformstartswithaninitialexpressionandevaluatesWHENexpressionsuntilthevaluesmatch,andoutputstheTHENexpression.IfnoWHENismatched,theELSEexpressionisoutput.IfnoWHENismatchedandnoELSEisspecified,anullliteralvalueisoutput.Thesecondform(thesearchedcaseexpression)searchestheWHENclauses,whichspecifyanarbitrarycriteriatoevaluate.Ifanycriteriaevaluatestotrue,theTHENexpressionisevaluatedandoutput.IfnoWHENistrue,theELSEisevaluatedorNULLisoutputifnoneexists.
ScalarSubqueries
SubqueriescanbeusedtoproduceasinglescalarvalueintheSELECT,WHERE,orHAVINGclausesonly.AscalarsubquerymusthaveasinglecolumnintheSELECTclauseandshouldreturneither0or1row.Ifnorowsarereturned,nullwillbereturnedasthescalarsubqueryvalue.Forothertypesofsubqueries,seetheSubqueriessection.
ParameterReferences
Parametersarespecifiedusinga'?'symbol.ParametersmayonlybeusedwithPreparedStatementorCallableStatementsinJDBC.Eachparameterislinkedtoavaluespecifiedby1-basedindexintheJDBCAPI.
Arrays
ArrayvaluesmaybeconstructedusingparenthesisaroundanexpressionlistwithanoptionaltrailingcommaorwithanexplicitARRAYconstructor
emptyarrays
()
(,)
ARRAY[]
singleelementarray
(expr,)
ARRAY[expr]
Note Atrailingcommaisrequiredfortheparsertorecognizeasingleelementexpressionasanarraywithparenthesis,ratherthanasimplenestedexpression.
generalarraysyntax
(expr,expr...[,])
ARRAY[expr,...]
Ifalloftheelementsinthearrayhavethesametype,thearraywillhaveamatchingbasetype.Iftheelementtypesdifferthearraybasetypewillbeobject.
Expressions
346
Anarrayelementreferencetakestheformof:
array_expr[index_expr]
index_exprmustresolvetoanintegervalue.Thissyntaxiseffectivelythesameasthearray_getsystemfunctionandexpects1-basedindexing.
OperatorPrecedenceTeiidparsesandevaluatesoperatorswithhigherprecedencebeforethosewithlowerprecedence.Operatorwithequalprecedenceareleftassociative.Operatorprecedencelistedfromhightolow:
Operator Description
[] arrayelementreference
+,- positive/negativevalueexpression
*,/ multiplication/division
+,- addition/subtraction
||\ concat
criteria seeCriteria
Expressions
347
CriteriaCriteriamaybe:
Predicatesthatevaluatetotrueorfalse
Logicalcriteriathatcombinescriteria(AND,OR,NOT)
Avalueexpressionwithtypeboolean
Usage:
criteriaAND|ORcriteria
NOTcriteria
(criteria)
expression(=|<>|!=|<|>|<=|>=)(expression|((ANY|ALL|SOME)subquery|(array_expression)))
expression[NOT]ISNULL
expression[NOT]IN(expression[,expression]*)|subquery
expression[NOT]LIKEpattern[ESCAPEchar]
Matchesthestringexpressionagainstthegivenstringpattern.Thepatternmaycontain%tomatchanynumberofcharactersand_tomatchanysinglecharacter.Theescapecharactercanbeusedtoescapethematchcharacters%and_.
expression[NOT]SIMILARTOpattern[ESCAPEchar]
SIMILARTOisacrossbetweenLIKEandstandardregularexpressionsyntax.%and_arestillused,ratherthan.*and.respectively.
Note
TeiiddoesnotexhaustivelyvalidateSIMILARTOpatternvalues.Ratherthepatternisconvertedtoanequivalentregularexpression.CareshouldbetakennottorelyongeneralregularexpressionfeatureswhenusingSIMILARTO.Ifadditionalfeaturesareneeded,thenLIKE_REGEXshouldbeused.Usageofanon-literalpatternisdiscouragedaspushdownsupportislimited.
expression[NOT]LIKE_REGEXpattern
LIKE_REGEXallowsforstandardregularexpressionsyntaxtobeusedformatching.ThisdiffersfromSIMILARTOandLIKEinthattheescapecharacterisnolongerused(\isalreadythestandardescapemechanisminregularexpressionsand%and_havenospecialmeaning.TheruntimeengineusestheJREimplementationofregularexpressions-seethejava.util.regex.Patternclassfordetails.
Note
TeiiddoesnotexhaustivelyvalidateLIKE_REGEXpatternvalues.ItispossibletouseJREonlyregularexpressionfeaturesthatarenotspecifiedbytheSQLspecification.Additionalnotallsourcessupportthesameregularexpressionflavororextensions.Careshouldbetakeninpushdownsituationstoensurethatthepattern
Criteria
348
usedwillhavesamemeaninginTeiidandacrossallapplicablesources.
EXISTS(subquery)
expression[NOT]BETWEENminExpressionANDmaxExpression
TeiidconvertsBETWEENintotheequivalentformexpression>=minExpressionANDexpression⇐maxExpression
expression
Whereexpressionhastypeboolean.
SyntaxRules:
Theprecedenceorderingfromlowesttohighestiscomparison,NOT,AND,OR
Criterianestedbyparenthesiswillbelogicallyevaluatedpriortoevaluatingtheparentcriteria.
Someexamplesofvalidcriteriaare:
(balance>2500.0)
100*(50-x)/(25-y)>z
concat(areaCode,concat(’-`,phone))LIKE’314%1'
ComparingnullValues
Tip Nullvaluesrepresentanunknownvalue.Comparisonwithanullvaluewillevaluateto`unknown',whichcanneverbetrueevenif`not'isused.
CriteriaPrecedence
Teiidparsesandevaluatesconditionswithhigherprecedencebeforethosewithlowerprecedence.Conditionswithequalprecedenceareleftassociative.Conditionprecedencelistedfromhightolow:
Condition Description
sqloperators SeeExpressions
EXISTS,LIKE,SIMILARTO,LIKE_REGEX,BETWEEN,IN,ISNULL,<,⇐,>,>=,=,<> comparison
NOT negation
AND conjunction
OR disjunction
Notehoweverthattopreventlookaheadstheparserdoesnotacceptallpossiblecriteriasequences.Forexample"a=bisnull"isnotaccepted,sincebytheleftassociativeparsingwefirstrecognize"a=",thenlookforacommonvalueexpression."bisnull"isnotavalidcommonvalueexpression.Thusnestingmustbeused,forexample"(a=b)isnull".SeeBNFforSQLGrammarforallparsingrules.
Criteria
349
Criteria
350
ScalarFunctionsTeiidprovidesanextensivesetofbuilt-inscalarfunctions.SeealsoSQLSupportandDatatypes.Inaddition,TeiidprovidesthecapabilityforuserdefinedfunctionsorUDFs.SeetheDevelopersGuideforaddingUDFs.OnceaddedUDFsmaybecalledjustlikeanyotherfunction.
ScalarFunctions
351
NumericFunctionsNumericfunctionsreturnnumericvalues(integer,long,float,double,biginteger,bigdecimal).Theygenerallytakenumericvaluesasinputs,thoughsometakestrings.
Function Definition DatatypeConstraint
+-*/ Standardnumericoperatorsxin\{integer,long,float,double,biginteger,bigdecimal},returntypeissameasx[a]
ABS(x) Absolutevalueofx Seestandardnumericoperatorsabove
ACOS(x) Arccosineofx xin\{double,bigdecimal},returntypeisdouble
ASIN(x) Arcsineofx xin\{double,bigdecimal},returntypeisdouble
ATAN(x) Arctangentofx xin\{double,bigdecimal},returntypeisdouble
ATAN2(x,y) Arctangentofxandy x,yin\{double,bigdecimal},returntypeisdouble
CEILING(x) Ceilingofx xin\{double,float},returntypeisdouble
COS(x) Cosineofx xin\{double,bigdecimal},returntypeisdouble
COT(x) Cotangentofx xin\{double,bigdecimal},returntypeisdouble
DEGREES(x) Convertxdegreestoradians xin\{double,bigdecimal},returntypeisdouble
EXP(x) e^x xin\{double,float},returntypeisdouble
FLOOR(x) Floorofx xin\{double,float},returntypeisdouble
FORMATBIGDECIMAL(x,y) Formatsxusingformaty xisbigdecimal,yisstring,returnsstring
FORMATBIGINTEGER(x,y) Formatsxusingformaty xisbiginteger,yisstring,returnsstring
FORMATDOUBLE(x,y) Formatsxusingformaty xisdouble,yisstring,returnsstring
FORMATFLOAT(x,y) Formatsxusingformaty xisfloat,yisstring,returnsstring
ScalarFunctions
352
FORMATINTEGER(x,y) Formatsxusingformaty xisinteger,yisstring,returnsstring
FORMATLONG(x,y) Formatsxusingformaty xislong,yisstring,returnsstring
LOG(x) Naturallogofx(basee) xin\{double,float},returntypeisdouble
LOG10(x) Logofx(base10) xin\{double,float},returntypeisdouble
MOD(x,y) Modulus(remainderofx/y)xin\{integer,long,float,double,biginteger,bigdecimal},returntypeissameasx
PARSEBIGDECIMAL(x,y) Parsesxusingformaty x,yarestrings,returnsbigdecimal
PARSEBIGINTEGER(x,y) Parsesxusingformaty x,yarestrings,returnsbiginteger
PARSEDOUBLE(x,y) Parsesxusingformaty x,yarestrings,returnsdouble
PARSEFLOAT(x,y) Parsesxusingformaty x,yarestrings,returnsfloat
PARSEINTEGER(x,y) Parsesxusingformaty x,yarestrings,returnsinteger
PARSELONG(x,y) Parsesxusingformaty x,yarestrings,returnslong
PI() ValueofPi returnisdouble
POWER(x,y) xtotheypowerxin\{double,bigdecimal,biginteger},returnisthesametypeasx
RADIANS(x) Convertxradianstodegrees xin\{double,bigdecimal},returntypeisdouble
RAND()
Returnsarandomnumber,usinggeneratorestablishedsofarinthequeryorinitializingwithsystemclockifnecessary.
Returnsdouble.
RAND(x) Returnsarandomnumber,usingnewgeneratorseededwithx. xisinteger,returnsdouble.
ROUND(x,y)Roundxtoyplaces;negativevaluesofyindicateplacestotheleftofthedecimalpoint
xin\{integer,float,double,bigdecimal}yisinteger,returnissametypeasx
SIGN(x) 1ifx>0,0ifx=0,-1ifx<0xin\{integer,long,float,double,biginteger,bigdecimal},returntypeisinteger
SIN(x) Sinevalueofx xin\{double,bigdecimal},returntypeisdouble
SQRT(x) Squarerootofx xin\{long,double,bigdecimal},returntypeisdouble
ScalarFunctions
353
TAN(x) Tangentofx xin\{double,bigdecimal},returntypeisdouble
BITAND(x,y) BitwiseANDofxandy x,yin{integer},returntypeisinteger
BITOR(x,y) BitwiseORofxandy x,yin{integer},returntypeisinteger
BITXOR(x,y) BitwiseXORofxandy x,yin{integer},returntypeisinteger
BITNOT(x) BitwiseNOTofx xin{integer},returntypeisinteger
[a]Theprecisionandscaleofnon-bigdecimalarithmeticfunctionfunctionsresultsmatchesthatofJava.TheresultsofbigdecimaloperationsmatchJava,exceptfordivision,whichusesapreferredscaleofmax(16,dividend.scale+divisor.precision+1),whichthenhastrailingzerosremovedbysettingthescaletomax(dividend.scale,normalizedscale)
ParsingNumericDatatypesfromStringsTeiidoffersasetoffunctionsyoucanusetoparsenumbersfromstrings.Foreachstring,youneedtoprovidetheformattingofthestring.Thesefunctionsusetheconventionestablishedbythejava.text.DecimalFormatclasstodefinetheformatsyoucanusewiththesefunctions.YoucanlearnmoreabouthowthisclassdefinesnumericstringformatsbyvisitingtheSunJavaWebsiteatthefollowingURLforSunJava.
Forexample,youcouldusethesefunctioncalls,withtheformattingstringthatadherestothejava.text.DecimalFormatconvention,toparsestringsandreturnthedatatypeyouneed:
InputString FunctionCalltoFormatString OutputValue OutputDatatype
'$25.30' parseDouble(cost,'$,0.00;($,0.00)') 25.3 double
'25%' parseFloat(percent,',#0%') 25 float
'2,534.1' parseFloat(total,',0.;-,0.') 2534.1 float
'1.234E3' parseLong(amt,'0.###E0') 1234 long
'1,234,567' parseInteger(total,',0;-,0') 1234567 integer
FormattingNumericDatatypesasStringsTeiidoffersasetoffunctionsyoucanusetoconvertnumericdatatypesintostrings.Foreachstring,youneedtoprovidetheformatting.Thesefunctionsusetheconventionestablishedwithinthejava.text.DecimalFormatclasstodefinetheformatsyoucanusewiththesefunctions.YoucanlearnmoreabouthowthisclassdefinesnumericstringformatsbyvisitingtheSunJavaWebsiteatthefollowingURLforSunJava.
Forexample,youcouldusethesefunctioncalls,withtheformattingstringthatadherestothejava.text.DecimalFormatconvention,toformatthenumericdatatypesintostrings:
ScalarFunctions
354
InputValue InputDatatype FunctionCalltoFormatString
OutputString
25.3 double formatDouble(cost,'$,0.00;($,0.00)') '$25.30'
25 float formatFloat(percent,',#0%') '25%'
2534.1 float formatFloat(total,',0.;-,0.') '2,534.1'
1234 long formatLong(amt,'0.###E0') '1.234E3'
1234567 integer formatInteger(total,',0;-,0') '1,234,567'
ScalarFunctions
355
StringFunctionsStringfunctionsgenerallytakestringsasinputsandreturnstringsasoutputs.
Unlessspecified,alloftheargumentsandreturntypesinthefollowingtablearestringsandallindexesare1-based.The0indexisconsideredtobebeforethestartofthestring.
Function Definition DatatypeConstraint
x||y Concatenationoperator x,yin\{string,clob},returntypeisstringorclob
ASCII(x)ProvideASCIIvalueoftheleftmostcharacterinx.Theemptystringwillasinputwillreturnnull.
returntypeisinteger
CHR(x)CHAR(x) ProvidethecharacterforASCIIvaluex[a] xin{integer}
CONCAT(x,y)ConcatenatesxandywithANSIsemantics.Ifxand/oryisnull,returnsnull.
x,yin{string}
CONCAT2(x,y)
Concatenatesxandywithnon-ANSInullsemantics.Ifxandyisnull,returnsnull.Ifonlyxoryisnull,returnstheothervalue.
x,yin{string}
ENDSWITH(x,y) Checksifyendswithx.Ifxoryisnull,returnsnull. x,yin{string},returnsboolean
INITCAP(x)Makefirstletterofeachwordinstringxcapitalandallotherslowercase
xin{string}
INSERT(str1,start,length,str2) Insertstring2intostring1 str1in{string},startin{integer},lengthin{integer},str2in{string}
LCASE(x) Lowercaseofx xin{string}
LEFT(x,y) Getleftycharactersofx xin{string},yin{integer},returnstring
LENGTH(x) Lengthofx returntypeisinteger
LOCATE(x,y) Findpositionofxinystartingatbeginningofy
xin{string},yin{string},returninteger
LOCATE(x,y,z) Findpositionofxinystartingatz xin{string},yin{string},zin{integer},returninteger
LPAD(x,y) Padinputstringxwithspacesonthelefttothelengthofy
xin{string},yin{integer},returnstring
LPAD(x,y,z) Padinputstringxonthelefttothelengthofyusingcharacterz
xin{string},yin{string},zin{character},returnstring
ScalarFunctions
356
LTRIM(x) Lefttrimxofblankchars xin{string},returnstring
QUERYSTRING(path[,expr[ASname]…])
Returnsaproperlyencodedquerystringappendedtothegivenpath.Nullvaluedexpressionsareomitted,andanullpathistreatedas".Namesareoptionalforcolumnreferenceexpressions.e.g.QUERYSTRING('path','value'as
"&x",'&'asy,nullasz)
returns'path?
%26x=value&y=%20%26%20'
path,exprin{string}.nameisanidentifier
REPEAT(str1,instances) Repeatstring1aspecifiednumberoftimes
str1in{string},instancesin{integer}returnstring
RIGHT(x,y) Getrightycharactersofx xin{string},yin{integer},returnstring
RPAD(inputstringx,padlengthy) Padinputstringxwithspacesontherighttothelengthofy
xin{string},yin{integer},returnstring
RPAD(x,y,z) Padinputstringxontherighttothelengthofyusingcharacterz
xin{string},yin{string},zin{character},returnstring
RTRIM(x) Righttrimxofblankchars xisstring,returnstring
SPACE(x) Repeatthespacecharacterxnumberoftimes xisinteger,returnstring
SUBSTRING(x,y)SUBSTRING(xFROMy)
[b]Getsubstringfromx,frompositionytotheendofx yin{integer}
SUBSTRING(x,y,z)SUBSTRING(xFROMyFORz)
[b]Getsubstringfromxfrompositionywithlengthz y,zin{integer}
TRANSLATE(x,y,z)Translatestringxbyreplacingeachcharacterinywiththecharacterinzatthesameposition
xin{string}
TRIM([[LEADING|TRAILING|BOTH][x]FROM]y)
Trimtheleading,trailing,orbothendsofastringyofcharacterx.IfLEADING/TRAILING/BOTHisnotspecified,BOTHisused.Ifnotrimcharacterxisspecficedthentheblankspace’isused.
xin{character},yin{string}
UCASE(x) Uppercaseofx xin{string}
UNESCAPE(x)
Unescapedversionofx.Possibleescapesequencesare\b-backspace,\t-tab,\n-linefeed,\f-formfeed,\r-carriagereturn.\uXXXX,whereXisahexvalue,canbeusedtospecifyanyunicodecharacter.\XXX,whereXisanoctaldigit,canbeusedtospecifyanoctalbytevalue.Ifanyothercharacterappearsafteranescapecharacter,that
xin{string}
ScalarFunctions
357
characterwillappearintheoutputandtheescapecharacterwillbeignored.
[a]Non-ASCIIrangecharactersorintegersusedinthesefunctionsmayproducedifferentresultsorexceptionsdependingonwherethefunctionisevaluated(Teiidvs.source).Teiid’susesJavadefaultinttocharandchartointconversions,whichoperatesoverUTF16values.
[b]Thesubstringfunctiondependinguponthesourcedoesnothaveconsistentbehaviorwithrespecttonegativefrom/lengthargumentsnoroutofboundsfrom/lengtharguments.ThedefaultTeiidbehavioris:
returnanullvaluewhenthefromvalueisoutofboundsorthelengthislessthan0
azerofromindexiseffectivethesameas1.
anegativefromindexisfirstcountedfromtheendofthestring.
Somesourceshowevercanreturnanemptystringinsteadofnullandsomesourcesdonotsupportnegativeindexing.Ifanyoftheseinconsistenciesimpactyou,thenpleaseloganissue.
EncodingFunctions
TO_CHARS
Returnaclobfromtheblobwiththegivenencoding.
TO_CHARS(x,encoding[,wellformed])
BASE64,HEX,andthebuilt-inJavaCharsetnamesarevalidvaluesfortheencoding[b].xisablob,encodingisastring,wellformedisaboolean,andreturnsaclob.Thetwoargumentformdefaultstowellformed=true.Ifwellformedisfalse,theconversionfunctionwillimmediatelyvalidatetheresultsuchthatanunmappablecharacterormalformedinputwillraiseanexception.
TO_BYTES
Returnablobfromtheclobwiththegivenencoding.
TO_BYTES(x,encoding[,wellformed])
BASE64,HEX,andthebuiltinJavaCharsetnamesarevalidvaluesfortheencoding[b].xinaclob,encodingisastring,wellformedisabooleanandreturnsablob.Thetwoargumentformdefaultstowellformed=true.Ifwellformedisfalse,theconversionfunctionwillimmediatelyvalidatetheresultsuchthatanunmappablecharacterormalformedinputwillraiseanexception.Ifwellformedistrue,thenunmappablecharacterswillbereplacedbythedefaultreplacementcharacterforthecharacterset.Binaryformats,suchasBASE64andHEX,willbecheckedforcorrectnessregardlessofthewellformedparameter.
[b]SeetheCharsetJavaDocformoreonsupportedCharsetnames.
ReplacementFunctions
REPLACE
Replacealloccurrencesofagivenstringwithanother.
ScalarFunctions
358
REPLACE(x,y,z)
Replacealloccurrencesofywithzinx.x,y,zarestringsandthereturnvalueisastring.
REGEXP_REPLACE
Replaceoneoralloccurrencesofagivenpatternwithanotherstring.
REGEXP_REPLACE(str,pattern,sub[,flags])
Replaceoneormoreoccurrencesofpatternwithsubinstr.Allargumentsarestringsandthereturnvalueisastring.
ThepatternparameterisexpectedtobeavalidJavaRegularExpression
Theflagsargumentcanbeanyconcatenationofanyofthevalidflagswiththefollowingmeanings:
flag name meaning
g global Replacealloccurrences,notjustthefirst
m multiline Matchovermultiplelines
i caseinsensitive Matchwithoutcasesensitivity
Usage:
Thefollowingwillreturn"xxbyeWxx"usingtheglobalandcaseinsensitiveoptions.
Exampleregexp_replace
regexp_replace('GoodbyeWorld','[g-o].','x','gi')
ScalarFunctions
359
Date_TimeFunctionsDateandtimefunctionsreturnoroperateondates,times,ortimestamps.
ParseandformatDate/Timefunctionsusetheconventionestablishedwithinthejava.text.SimpleDateFormatclasstodefinetheformatsyoucanusewiththesefunctions.YoucanlearnmoreabouthowthisclassdefinesformatsbyvisitingtheJavadocsforSimpleDateFormat.
Function Definition DatatypeConstraint
CURDATE() Returncurrentdate returnsdate
CURTIME() Returncurrenttime returnstime
NOW() Returncurrenttimestamp(dateandtime) returnstimestamp
DAYNAME(x) Returnnameofdayinthedefaultlocale
xin\{date,timestamp},returnsstring
DAYOFMONTH(x) Returndayofmonthxin\{date,timestamp},returnsinteger
DAYOFWEEK(x) Returndayofweek(Sunday=1,Saturday=7)
xin\{date,timestamp},returnsinteger
DAYOFYEAR(x) Returndaynumberinyearxin\{date,timestamp},returnsinteger
EXTRACT(YEAR|MONTH|DAY|HOUR|MINUTE|SECONDFROMx)
Returnthegivenfieldvaluefromthedatevaluex.ProducesthesameresultastheassoceatedYEAR,MONTH,DAYOFMONTH,HOUR,MINUTE,SECONDfunctions.TheSQLspecificationalsoallowsforTIMEZONE_HOURandTIMEZONE_MINUTEasextractiontargets.InTeiidalldatevaluesareinthetimezoneoftheserver.
xin\{date,time,timestamp},returnsinteger
FORMATDATE(x,y) Formatdatexusingformaty xisdate,yisstring,returnsstring
FORMATTIME(x,y) Formattimexusingformaty xistime,yisstring,returnsstring
FORMATTIMESTAMP(x,y) Formattimestampxusingformaty
xistimestamp,yisstring,returnsstring
ScalarFunctions
360
FROM_UNIXTIME(unix_timestamp)ReturntheUnixtimestamp(inseconds)asaTimestampvalue
Unixtimestamp(inseconds)
HOUR(x) Returnhour(inmilitary24-hourformat)
xin\{time,timestamp},returnsinteger
MINUTE(x) Returnminutexin\{time,timestamp},returnsinteger
MODIFYTIMEZONE(timestamp,startTimeZone,endTimeZone)
Returnsatimestampbasedupontheincomingtimestampadjustedforthedifferentialbetweenthestartandendtimezones.i.e.iftheserverisinGMT-6,thenmodifytimezone(\{ts'2006-01-1004:00:00.0'},'GMT-7','GMT-8')willreturnthetimestamp\{ts'2006-01-1005:00:00.0'}asreadinGMT-6.Thevaluehasbeenadjusted1houraheadtocompensateforthedifferencebetweenGMT-7andGMT-8.
startTimeZoneandendTimeZonearestrings,returnsatimestamp
MODIFYTIMEZONE(timestamp,endTimeZone)
Returnatimestampinthesamemannerasmodifytimezone(timestamp,startTimeZone,endTimeZone),butwillassumethatthestartTimeZoneisthesameastheserverprocess.
Timestampisatimestamp;endTimeZoneisastring,returnsatimestamp
MONTH(x) Returnmonthxin\{date,timestamp},returnsinteger
MONTHNAME(x) Returnnameofmonthinthedefaultlocale
xin\{date,timestamp},returnsstring
PARSEDATE(x,y) Parsedatefromxusingformaty
x,yin{string},returnsdate
PARSETIME(x,y) Parsetimefromxusingformaty
x,yin{string},returnstime
PARSETIMESTAMP(x,y) Parsetimestampfromxusingformaty
x,yin{string},returnstimestamp
QUARTER(x) Returnquarterxin\{date,timestamp},returnsinteger
SECOND(x) Returnsecondsxin\{time,timestamp},returnsinteger
ScalarFunctions
361
TIMESTAMPCREATE(date,time) Createatimestampfromadateandtime
datein{date},timein{time},returnstimestamp
WEEK(x)Returnweekinyear1-53,seealsoSystemPropertiesforcustomization
xin\{date,timestamp},returnsinteger
YEAR(x) Returnfour-digityearxin\{date,timestamp},returnsinteger
Timestampadd/Timestampdiff
Timestampadd
Addaspecifiedintervalamounttothetimestamp.
Syntax
TIMESTAMPADD(interval,count,timestamp)
Arguments
Name Description
interval
Adatetimeintervalunit,canbeoneofthefollowingkeywords:
SQL_TSI_FRAC_SECOND-fractionalseconds(billionthsofasecond)
SQL_TSI_SECOND-seconds
SQL_TSI_MINUTE-minutes
SQL_TSI_HOUR-hours
SQL_TSI_DAY-days
SQL_TSI_WEEK-weeksusingSundayasthefirstday
SQL_TSI_MONTH-months
SQL_TSI_QUARTER-quarters(3months)wherethefirstquarterismonths1-3,etc.
SQL_TSI_YEAR-years
count Anintegerrepresentthedatetimeneedaddtotimestamp
timestamp Adatetimeexpression.
Example
SELECTTIMESTAMPADD(SQL_TSI_MONTH,12,'2016-10-10')
SELECTTIMESTAMPADD(SQL_TSI_SECOND,12,'2016-10-1023:59:59')
Timestampdiff
Calculatesthenumberofdatepartintervalscrossedbetweenthetwotimestampsreturnalongvalue.
Syntax
ScalarFunctions
362
TIMESTAMPDIFF(interval,startTime,endTime)
Arguments
Name Description
interval Adatetimeintervalunit,thesameaskeywordsusedbyTimestampadd.
startTime Adatetimeexpression.
endTime Adatetimeexpression.
Example
SELECTTIMESTAMPDIFF(SQL_TSI_MONTH,'2000-01-02','2016-10-10')
SELECTTIMESTAMPDIFF(SQL_TSI_SECOND,'2000-01-0200:00:00','2016-10-1023:59:59')
SELECTTIMESTAMPDIFF(SQL_TSI_FRAC_SECOND,'2000-01-0200:00:00.0','2016-10-1023:59:59.999999')
NoteIf(endTime>startTime),anon-negativenumberwillbereturned.If(endTime<startTime),anon-positivenumberwillbereturned.Thedatepartdifferencedifferenceiscountedregardlessofhowclosethetimestampsare.Forexample,'2000-01-0200:00:00.0'isstillconsidered1houraheadof'2000-01-0123:59:59.999999'.
CompatibilityIssues
Timestampdifftypicallyreturnsaninteger,howeverTeiid’sversionreturnsalong.Youmayreceiveanexceptionifyouexpectavalueoutoftheintegerrangefromapusheddowntimestampdiff.
Teiid’simplementationoftimestampdiffin8.2andpriorversionsreturnedvaluesbaseduponthenumberofwholecanonicalintervalapproximations(365daysinayear,91daysinaquarter,30daysinamonth,etc.)crossed.Forexamplethedifferenceinmonthsbetween2013-03-24and2013-04-01was0,butbaseduponthedatepartscrossedis1.SeeSystemPropertiesforbackwardscompatibility.
ParsingDateDatatypesfromStringsTeiiddoesnotimplicitlyconvertstringsthatcontaindatespresentedindifferentformats,suchas'19970101'and'31/1/1996'todate-relateddatatypes.Youcan,however,usetheparseDate,parseTime,andparseTimestampfunctions,describedinthenextsection,toexplicitlyconvertstringswithadifferentformattotheappropriatedatatype.Thesefunctionsusetheconventionestablishedwithinthejava.text.SimpleDateFormatclasstodefinetheformatsyoucanusewiththesefunctions.YoucanlearnmoreabouthowthisclassdefinesdateandtimestringformatsbyvisitingtheJavadocsforSimpleDateFormat.NotethattheformatstringswillbelocalespecifictoyourJavadefaultlocale.
Forexample,youcouldusethesefunctioncalls,withtheformattingstringthatadherestothejava.text.SimpleDateFormatconvention,toparsestringsandreturnthedatatypeyouneed:
String FunctionCallToParseString
'1997010' parseDate(myDateString,'yyyyMMdd')
'31/1/1996' parseDate(myDateString,'dd''/''MM''/''yyyy')
'22:08:56CST' parseTime(myTime,'HH:mm:ssz')
'03.24.2003at06:14:32' parseTimestamp(myTimestamp,'MM.dd.yyyy''at''hh:mm:ss')
ScalarFunctions
363
SpecifyingTimeZones
Timezonescanbespecifiedinseveralformats.CommonabbreviationssuchasESTfor"EasternStandardTime"areallowedbutdiscouraged,astheycanbeambiguous.Unambiguoustimezonesaredefinedintheformcontinentorocean/largestcity.Forexample,America/New_York,America/Buenos_Aires,orEurope/London.Additionally,youcanspecifyacustomtimezonebyGMToffset:GMT[+/-]HH:MM.
Forexample:GMT-05:00
ScalarFunctions
364
TypeConversionFunctionsWithinyourqueries,youcanconvertbetweendatatypesusingtheCONVERTorCASTkeyword.SeealsoTypeConversions
Function Definition
CONVERT(x,type) Convertxtotype,wheretypeisaTeiidBaseType
CAST(xAStype) Convertxtotype,wheretypeisaTeiidBaseType
Thesefunctionsareidenticalotherthansyntax;CASTisthestandardSQLsyntax,CONVERTisthestandardJDBC/ODBCsyntax.
Important Optionsthatarespecifiedonthetype,suchaslength,precision,scale,etc.,areeffectivelyignored-theruntimeissimplyconvertingfromoneobjecttypetoanother.
ScalarFunctions
365
ChoiceFunctionsChoicefunctionsprovideawaytoselectfromtwovaluesbasedonsomecharacteristicofoneofthevalues.
Function Definition DatatypeConstraint
COALESCE(x,y+) Returnsthefirstnon-nullparameter xandally’scanbeanycompatibletypes
IFNULL(x,y) Ifxisnull,returny;elsereturnx x,y,andthereturntypemustbethesametypebutcanbeanytype
NVL(x,y) Ifxisnull,returny;elsereturnx x,y,andthereturntypemustbethesametypebutcanbeanytype
NULLIF(param1,param2) Equivalenttocasewhen(param1=param2)thennullelseparam1
param1andparam2mustbecompatablecomparabletypes
IFNULLandNVLarealiasesofeachother.Theyarethesamefunction.
ScalarFunctions
366
DecodeFunctionsDecodefunctionsallowyoutohavetheTeiidServerexaminethecontentsofacolumninaresultsetandalter,ordecode,thevaluesothatyourapplicationcanbetterusetheresults.
Function Definition DatatypeConstraint
DECODESTRING(x,y[,z])
Decodecolumnxusingstringofvaluepairsywithoptionaldelimiterzandreturnthedecodedcolumnasastring.Ifadelimiterisnotspecified,isused.yhastheformateSearchDelimResultDelimSearchDelimResult[DelimDefault]ReturnsDefaultifspecifiedorxiftherearenomatches.Deprecated.UseaCASEexpressioninstead.
allstring
DECODEINTEGER(x,y[,z])
Decodecolumnxusingstringofvaluepairsywithoptionaldelimiterzandreturnthedecodedcolumnasaninteger.Ifadelimiterisnotspecified,isused.yhastheformateSearchDelimResultDelimSearchDelimResult[DelimDefault]ReturnsDefaultifspecifiedorxiftherearenomatches.Deprecated.UseaCASEexpressioninstead.
allstringparameters,returninteger
Withineachfunctioncall,youincludethefollowingarguments:
1. xistheinputvalueforthedecodeoperation.Thiswillgenerallybeacolumnname.
2. yistheliteralstringthatcontainsadelimitedsetofinputvaluesandoutputvalues.
3. zisanoptionalparameteronthesemethodsthatallowsyoutospecifywhatdelimiterthestringspecifiedinyuses.
Forexample,yourapplicationmightqueryatablecalledPARTSthatcontainsacolumncalledIS_IN_STOCKwhichcontainsaBooleanvaluethatyouneedtochangeintoanintegerforyourapplicationtoprocess.Inthiscase,youcanusetheDECODEINTEGERfunctiontochangetheBooleanvaluestointegers:
SELECTDECODEINTEGER(IS_IN_STOCK,'false,0,true,1')FROMPartsSupplier.PARTS;
WhentheTeiidSystemencountersthevaluefalseintheresultset,itreplacesthevaluewith0.
If,insteadofusingintegers,yourapplicationrequiresstringvalues,youcanusetheDECODESTRINGfunctiontoreturnthestringvaluesyouneed:
SELECTDECODESTRING(IS_IN_STOCK,'false,no,true,yes,null')FROMPartsSupplier.PARTS;
Inadditiontotwoinput/outputvaluepairs,thissamplequeryprovidesavaluetouseifthecolumndoesnotcontainanyoftheprecedinginputvalues.IftherowintheIS_IN_STOCKcolumndoesnotcontaintrueorfalse,theTeiidServerinsertsanullintotheresultset.
WhenyouusetheseDECODEfunctions,youcanprovideasmanyinput/outputvaluepairsifyouwantwithinthestring.Bydefault,theTeiidSystemexpectsacommadelimiter,butyoucanaddathirdparametertothefunctioncalltospecifyadifferentdelimiter:
SELECTDECODESTRING(IS_IN_STOCK,'false:no:true:yes:null',':')FROMPartsSupplier.PARTS;
YoucanusekeywordnullintheDECODEstringaseitheraninputvalueoranoutputvaluetorepresentanullvalue.However,ifyouneedtousetheliteralstringnullasaninputoroutputvalue(whichmeansthewordnullappearsinthecolumnandnotanullvalue)youcanputthewordinquotes:"null".
ScalarFunctions
367
SELECTDECODESTRING(IS_IN_STOCK,'null,no,"null",no,nil,no,false,no,true,yes')FROMPartsSupplier.PARTS;
IftheDECODEfunctiondoesnotfindamatchingoutputvalueinthecolumnandyouhavenotspecifiedadefaultvalue,theDECODEfunctionwillreturntheoriginalvaluetheTeiidServerfoundinthatcolumn.
ScalarFunctions
368
LookupFunctionTheLookupfunctionprovidesawaytospeedupaccesstovaluesfromareferencetable.TheLookupfunctionautomaticallycachesallkeyandreturncolumnpairsdeclaredinthefunctionforthereferencedtable.Subsequentlookupsagainstthesametableusingthesamekeyandreturncolumnswillusethecachedvalues.Thiscachingacceleratesresponsetimetoqueriesthatuselookuptables,alsoknowninbusinessterminologyascodeorreferencetables.
LOOKUP(codeTable,returnColumn,keyColumn,keyValue)
InthelookuptablecodeTable,findtherowwherekeyColumnhasthevaluekeyValueandreturntheassociatedreturnColumnvalueornull,ifnomatchingkeyValueisfound.codeTablemustbeastringliteralthatisthefully-qualifiednameofthetargettable.returnColumnandkeyColumnmustalsobestringliteralsandmatchcorrespondingcolumnnamesinthecodeTable.ThekeyValuecanbeanyexpressionthatmustmatchthedatatypeofthekeyColumn.ThereturndatatypematchesthatofreturnColumn.
CountryCodeLookup
lookup('ISOCountryCodes','CountryCode','CountryName','UnitedStates')
AnISOCountryCodestableisusedtotranslateacountrynametoanISOcountrycode.Onecolumn,CountryName,representsthekeyColumn.Asecondcolumn,CountryCode,representsthereturnColumn,containingtheISOcodeofthecountry.Hence,theusageofthelookupfunctionherewillprovideaCountryName,shownaboveas`UnitedStates',andexpectaCountryCodevalueinresponse.
WhenyoucallthisfunctionforanycombinationofcodeTable,returnColumn,andkeyColumnforthefirsttime,theTeiidSystemcachestheresult.TheTeiidSystemusesthiscacheforallqueries,inallsessions,thatlateraccessthislookuptable.Youshouldgenerallynotusethelookupfunctionfordatathatissubjecttoupdatesormaybesession/userspecific-includingrowbasedsecurityandcolumnmaskingeffects.SeetheCachingGuideformoreonthecachingaspectsoftheLookupfunction.
ThekeyColumnisexpectedtocontainuniquevaluesforitscorrespondingcodeTable.IfthekeyColumncontainsduplicatevalues,anexceptionwillbethrown.
ScalarFunctions
369
SystemFunctionsSystemfunctionsprovideaccesstoinformationintheTeiidsystemfromwithinaquery.
TableofContentsCOMMANDPAYLOADENVNODE_IDSESSION_IDUSERCURRENT_DATABASETEIID_SESSION_GETTEIID_SESSION_SET
COMMANDPAYLOAD
Retrieveastringfromthecommandpayloadornullifnocommandpayloadwasspecified.ThecommandpayloadissetbytheTeiidStatement.setPayloadmethodontheTeiidJDBCAPIextensionsonaper-querybasis.
COMMANDPAYLOAD([key])
Ifthekeyparameterisprovided,thecommandpayloadobjectiscasttoajava.util.Propertiesobjectandthecorrespondingpropertyvalueforthekeyisreturned.IfthekeyisnotspecifiedthereturnvalueisthecommandpayloadobjecttoStringvalue.
key,returnvaluearestrings
ENVRetrieveasystemenvironmentproperty.
ENV(key)
Topreventuntrustedaccesstosystemproperties,thisfunctionisnotenabledbydefault.TheENVfunctionneedstobeenabledviaCLI
/subsystem=teiid:write-attribute(name=allow-env-function,value=true)
oreditthestandalone-teiid.xmlfileandaddfollowingtothe"teiid"subsystem
<allow-env-function>true</allow-env-function>
thencallusingENV('KEY'),whichreturnsvalueasstring.Ex:ENV('PATH')
NODE_IDRetrievethenodeid-typicallytheSystempropertyvaluefor"jboss.node.name"whichwillnotbesetforTeiidembedded.
NODE_ID()
returnvalueisstring.
ScalarFunctions
370
SESSION_ID
Retrievethestringformofthecurrentsessionid.
SESSION_ID()
returnvalueisstring.
USER
Retrievethenameoftheuserexecutingthequery.
USER([includeSecurityDomain])
includeSecurityDomainisaboolean.returnvalueisstring.
IfincludeSecurityDomainisomittedortrue,thentheusernamewillbereturnedwith@security-domainappended.
CURRENT_DATABASE
Retrievethecatalognameofthedatabase.TheVDBnameisalwaysthecatalogname.
CURRENT_DATABASE()
returnvalueisstring.
TEIID_SESSION_GET
Retrievethesessionvariable.
TEIID_SESSION_GET(name)
nameisastringandthereturnvalueisanobject.
Anullnamewillreturnanullvalue.TypicallyyouwillusetheagetwrappedinaCASTtoconverttothedesiredtype.
TEIID_SESSION_SETSetthesessionvariable.
TEIID_SESSION_SET(name,value)
nameisastring,valueisanobject,andthereturnvalueisanobject.
Thepreviousvalueforthekeyornullwillbereturned.Asethasnoeffectonthecurrenttransactionandisnotaffectedbycommit/rollback.
ScalarFunctions
371
XMLFunctionsXMLfunctionsprovidefunctionalityforworkingwithXMLdata.SeealsotheJSONTOXMLfunction.
TableofContentsSampleDataForExamplesXMLCASTXMLCOMMENTXMLCONCATXMLELEMENTXMLFORESTXMLAGGXMLPARSEXMLPIXMLQUERYXMLEXISTSXMLSERIALIZEXMLTEXTXSLTRANSFORMXPATHVALUEExamples
GeneratinghierarchicalXMLfromflatdatastructure
SampleDataForExamples
ExamplesprovidedwithXMLfunctionsusethefollowingtablestructure
TABLECustomer(
CustomerIdintegerPRIMARYKEY,
CustomerNamevarchar(25),
ContactNamevarchar(25)
Addressvarchar(50),
Cityvarchar(25),
PostalCodevarchar(25),
Countryvarchar(25),
);
withData
CustomerID CustomerName ContactName Address City PostalCode Country
87 WartianHerkku PirkkoKoskitalo
Torikatu38 Oulu 90110 Finland
88 WellingtonImportadora PaulaParente
RuadoMercado,12
Resende 08737-363 Brazil
89 WhiteCloverMarkets KarlJablonski
305-14thAve.S.Suite3B
Seattle 98128 USA
ScalarFunctions
372
XMLCAST
CasttoorfromXML.
XMLCAST(expressionAStype)
ExpressionortypemustbeXML.Thereturnvaluewillbetypedastype.ThisisthesamefunctionalityasXMLTABLEusestoconvertvaluestothedesiredruntimetype-withtheexceptionthatarraytypetargetsarenotsupportedwithXMLCAST.
XMLCOMMENT
Returnsanxmlcomment.
XMLCOMMENT(comment)
Commentisastring.Returnvalueisxml.
XMLCONCATReturnsanXMLwiththeconcatenationofthegivenxmltypes.
XMLCONCAT(content[,content]*)
Contentisxml.Returnvalueisxml.
Ifavalueisnull,itwillbeignored.Ifallvaluesarenull,nullisreturned.
ConcatenatetwoormoreXMLfragments
SELECTXMLCONCAT(
XMLELEMENT("name",CustomerName),
XMLPARSE(CONTENT'<a>b</a>'WELLFORMED)
)
FROMCustomerc
WHEREc.CustomerID=87;
==========================================================
<name>WartianHerkku</name><a>b</a>
XMLELEMENTReturnsanXMLelementwiththegivennameandcontent.
XMLELEMENT([NAME]name[,<NSP>][,<ATTR>][,content]*)
ATTR:=XMLATTRIBUTES(exp[ASname][,exp[ASname]]*)
NSP:=XMLNAMESPACES((uriASprefix|DEFAULTuri|NODEFAULT))+
Ifthecontentvalueisofatypeotherthanxml,itwillbeescapedwhenaddedtotheparentelement.Nullcontentvaluesareignored.WhitespaceinXMLorthestringvaluesofthecontentispreserved,butnowhitespaceisaddedbetweencontentvalues.
ScalarFunctions
373
XMLNAMESPACESisusedprovidenamespaceinformation.NODEFAULTisequivalenttodefiningthedefaultnamespacetothenulluri-xmlns="".OnlyoneDEFAULTorNODEFAULTnamespaceitemmaybespecified.Thenamespaceprefixesxmlnsandxmlarereserved.
Ifaattributenameisnotsupplied,theexpressionmustbeacolumnreference,inwhichcasetheattributenamewillbethecolumnname.Nullattributevaluesareignored.
Name,prefixareidentifiers.uriisastringliteral.contentcanbeanytype.Returnvalueisxml.Thereturnvalueisvalidforuseinplaceswhereadocumentisexpected.
SimpleExample
SELECTXMLELEMENT("name",CustomerName)
FROMCustomerc
WHEREc.CustomerID=87;
==========================================================
<name>WartianHerkku</name>
MultipleColumns
SELECTXMLELEMENT("customer",
XMLELEMENT("name",c.CustomerName),
XMLELEMENT("contact",c.ContactName))
FROMCustomerc
WHEREc.CustomerID=87;
==========================================================
<customer><name>WartianHerkku</name><contact>PirkkoKoskitalo</contact></customer>
ColumnsasAttributes
SELECTXMLELEMENT("customer",
XMLELEMENT("name",c.CustomerName,
XMLATTRIBUTES(
"contact"asc.ContactName,
"id"asc.CustomerID
)
)
)
FROMCustomerc
WHEREc.CustomerID=87;
==========================================================
<customer><namecontact="PirkkoKoskitalo"id="87">WartianHerkku</name></customer>
XMLFOREST
ReturnsanconcatenationofXMLelementsforeachcontentitem.
XMLFOREST(content[ASname][,<NSP>][,content[ASname]]*)
SeeXMLELEMENTforthedefinitionofNSP-XMLNAMESPACES
Nameisanidentifier.Contentcanbeanytype.Returnvalueisxml.
Ifanameisnotsuppliedforacontentitem,theexpressionmustbeacolumnreference,inwhichcasetheelementnamewillbeapartiallyescapedversionofthecolumnname.
ScalarFunctions
374
YoucanuseXMLFORRESTtosimplifythedeclarationofmultipleXMLELEMENTS,XMLFORESTfunctionallowsyoutoprocessmultiplecolumnsatonce
Example
SELECTXMLELEMENT("customer",
XMLFOREST(
c.CustomerNameAS"name",
c.ContactNameAS"contact"
))
FROMCustomerc
WHEREc.CustomerID=87;
==========================================================
<customer><name>WartianHerkku</name><contact>PirkkoKoskitalo</contact></customer>
XMLAGG
XMLAGGisanaggregatefunction,thattakesacollectionofXMLelementsandreturnsanaggregatedXMLdocument.
XMLAGG(xml)
FromaboveexampleinXMLElement,eachrowintheCustomertabletablewillgeneraterowofXMLiftherearemultiplerowsmatchingthecriteria.ThatwillgenerateavalidXML,butitwillnotbewellformed,becauseitlackstherootelement.XMLAGGcanusedtocorrectthat
Example
SELECTXMLELEMENT("customers",
XMLAGG(
XMLELEMENT("customer",
XMLFOREST(
c.CustomerNameAS"name",
c.ContactNameAS"contact"
)))
FROMCustomerc
==========================================================
<customers>
<customer><name>WartianHerkku</name><contact>PirkkoKoskitalo</contact></customer>
<customer><name>WellingtonImportadora</name><contact>PaulaParente</contact></customer>
<customer><name>WhiteCloverMarkets</name><contact>KarlJablonski</contact></customer>
</customers>
XMLPARSEReturnsanXMLtyperepresentationofthestringvalueexpression.
XMLPARSE((DOCUMENT|CONTENT)expr[WELLFORMED])
exprin\{string,clob,blob,varbinary}.Returnvalueisxml.
IfDOCUMENTisspecifiedthentheexpressionmusthaveasinglerootelementandmayormaynotcontainanXMLdeclaration.
IfWELLFORMEDisspecifiedthenvalidationisskipped;thisisespeciallyusefulforCLOBandBLOBknowntoalreadybevalid.
ScalarFunctions
375
SELECTXMLPARSE(CONTENT'<customer><name>WartianHerkku</name><contact>PirkkoKoskitalo</contact></customer>'W
ELLFORMED);
WillreturnaSQLXMLwithcontents
===============================================================
<customer><name>WartianHerkku</name><contact>PirkkoKoskitalo</contact></customer>
XMLPIReturnsanxmlprocessinginstruction.
XMLPI([NAME]name[,content])
Nameisanidentifier.Contentisastring.Returnvalueisxml.
XMLQUERY
ReturnstheXMLresultfromevaluatingthegivenxquery.
XMLQUERY([<NSP>]xquery[<PASSING>][(NULL|EMPTY)ONEMPTY]]
PASSING:=PASSINGexp[ASname][,exp[ASname]]*
SeeXMLELEMENTforthedefinitionofNSP-XMLNAMESPACES
Namespacesmayalsobedirectlydeclaredinthexqueryprolog.
TheoptionalPASSINGclauseisusedtoprovidethecontextitem,whichdoesnothaveaname,andnamedglobalvariablevalues.Ifthexqueryusesacontextitemandnoneisprovided,thenanexceptionwillberaised.OnlyonecontextitemmaybespecifiedandshouldbeanXMLtype.Allnon-contextnon-XMLpassingvalueswillbeconvertedtoanappropriateXMLtype.Nullwillbereturnedifthecontextitemevaluatestonull.
TheONEMPTYclauseisusedtospecifytheresultwhentheevalutedsequenceisempty.EMPTYONEMPTY,thedefault,returnsanemptyXMLresult.NULLONEMPTYreturnsanullresult.
xqueryinstring.Returnvalueisxml.
XMLQUERYispartoftheSQL/XML2006specification.
SeealsoFROMClause#XMLTABLE
Note SeealsoXQueryOptimization
XMLEXISTSReturnstrueifanon-emptysequencewouldbereturnedbyevaluatingthegivenxquery.
XMLEXISTS([<NSP>]xquery[<PASSING>]]
PASSING:=PASSINGexp[ASname][,exp[ASname]]*
SeeXMLELEMENTforthedefinitionofNSP-XMLNAMESPACES
ScalarFunctions
376
Namespacesmayalsobedirectlydeclaredinthexqueryprolog.
TheoptionalPASSINGclauseisusedtoprovidethecontextitem,whichdoesnothaveaname,andnamedglobalvariablevalues.Ifthexqueryusesacontextitemandnoneisprovided,thenanexceptionwillberaised.OnlyonecontextitemmaybespecifiedandshouldbeanXMLtype.Allnon-contextnon-XMLpassingvalueswillbeconvertedtoanappropriateXMLtype.Null/Unknownwillbereturnedifthecontextitemevaluatestonull.
xqueryinstring.Returnvalueisboolean.
XMLEXISTSispartoftheSQL/XML2006specification.
Note SeealsoXQueryOptimization
XMLSERIALIZEReturnsacharactertyperepresentationofthexmlexpression.
XMLSERIALIZE([(DOCUMENT|CONTENT)]xml[ASdatatype][ENCODINGenc][VERSIONver][(INCLUDING|EXCLUDING)XMLDECL
ARATION])
Returnvaluematchesdatatype.Ifnodatatypeisspecified,thenclobwillbeassumed.
Thetypemaybecharacter(string,varchar,clob)orbinary(blob,varbinar).CONTENTisthedefault.IfDOCUMENTisspecifiedandthexmlisnotavaliddocumentorfragment,thenanexceptionisraised.
Theencodingencisspecifiedasanidentifier.Acharacterserializationmaynotspecifyanencoding.Theversionverisspecifiedasastringliteral.IfaparticularXMLDECLARATIONisnotspecified,thentheresultwillhaveadeclarationonlyifperforminganonUTF-8/UTF-16ornonversion1.0documentserializationortheunderlyingxmlhasandeclaration.IfCONTENTisbeingserialized,thenthedeclarationwillbeomittedifthevalueisnotadocumentorelement.
SeethefollowingexamplethatproducesaBLOBofXMLinUTF-16includingtheappropriatebyteordermarkofFEFFandXMLdeclaration.
SampleBinarySerialization
XMLSERIALIZE(DOCUMENTvalueASBLOBENCODING"UTF-16"INCLUDINGXMLDECLARATION)
XMLTEXTReturnsxmltext.
XMLTEXT(text)
textisastring.Returnvalueisxml.
XSLTRANSFORM
AppliesanXSLstylesheettothegivendocument.
XSLTRANSFORM(doc,xsl)
Doc,xslin\{string,clob,xml}.Returnvalueisaclob.
ScalarFunctions
377
Ifeitherargumentisnull,theresultisnull.
XPATHVALUE
AppliestheXPATHexpressiontothedocumentandreturnsastringvalueforthefirstmatchingresult.FormorecontrolovertheresultsandXQuery,usetheXMLQUERYfunction.
XPATHVALUE(doc,xpath)
Docin\{string,clob,blob,xml}.xpathisstring.Returnvalueisastring.
Matchinganon-textnodewillstillproduceastringresult,whichincludesalldescendanttextnodes.Ifasingleelementismatchedthatismarkedwithxsi:nil,thennullwillbereturned.
Whentheinputdocumentutilizesnamespaces,itissometimesnecessarytospecifyXPATHthatignoresnamespaces:
SampleXMLforxpathValueIgnoringNamespaces
<?xmlversion="1.0"?>
<ns1:returnxmlns:ns1="http://com.test.ws/exampleWebService">Hello<x>World</x></return>
Function:
SamplexpathValueIgnoringNamespaces
xpathValue(value,'/*[local-name()="return"]')
ResultsinHelloWorld
Examples
GeneratinghierarchicalXMLfromflatdatastructure
Withfollowingtableanditscontents
Table{
xstring,
yinteger
}
datalike['a',1],['a',2],['b',3],['b',4],ifyouwantgenerateaXMLthatlookslike
<root>
<x>
a
<y>1</y>
<y>2</y>
</x>
<x>
b
<y>3</y>
<y>4</y>
</x>
</root>
usetheSQLstatementinTeiidasbelow
ScalarFunctions
378
selectxmlelement(name"root",xmlagg(p))
from(selectxmlelement(name"x",x,xmlagg(xmlelement(name"y",y))aspfromtblgroupbyx))asv
anotherusefullinkofexamplescanbefoundhere
ScalarFunctions
379
JSONFunctionsJSONfunctionsprovidefunctionalityforworkingwithJSON(JavaScriptObjectNotation)data.
TableofContentsJSONTOXMLJSONARRAYJSONOBJECTJSONPARSEConversiontoJSON
JSONTOXML
ReturnsanxmldocumentfromJSON.
JSONTOXML(rootElementName,json)
rootElementNameisastring,jsonisin\{clob,blob}.Returnvalueisxml.
TheappropriateUTFencoding(8,16LE.16BE,32LE,32BE)willbedetectedforJSONblobs.Ifanotherencodingisused,seetheto_charsfunction.
Theresultisalwaysawell-formedXMLdocument.
ThemappingtoXMLusesthefollowingrules:
ThecurrentelementnameisinitiallytherootElementName,andbecomestheobjectvaluenameastheJSONstructureistraversed.
Allelementnamesmustbevalidxml1.1names.InvalidnamesarefullyescapedaccordingtotheSQLXMLspecification.
Eachobjectorprimitivevaluewillbeenclosedinanelementwiththecurrentname.
Unlessanarrayvalueistheroot,itwillnotbeenclosedinanadditionalelement.
Nullvalueswillberepresentedbyanemptyelementwiththeattributexsi:nil="true"
Booleanandnumericalvalueelementswillhavetheattributexsi:typesettobooleananddecimalrespectively.
JSON:
SampleJSONtoXMLforjsonToXml(’person’,x)
{"firstName":"John","children":["Randy","Judy"]}
XML:
SampleJSONtoXMLforjsonToXml(’person’,x)
<?xmlversion="1.0"?>
<person>
<firstName>John</firstName>
<children>Randy</children>
<children>Judy<children>
</person>
ScalarFunctions
380
JSON:
SampleJSONtoXMLforjsonToXml('person',x)witharootarray
[{"firstName":"George"},{"firstName":"Jerry"}]
XML(Noticethereisanextra"person"wrappingelementtokeeptheXMLwell-formed):
SampleJSONtoXMLforjsonToXml(’person’,x)witharootarray
<?xmlversion="1.0"?>
<person>
<person>
<firstName>George</firstName>
</person>
<person>
<firstName>Jerry</firstName>
</person>
</person>
JSON:
SampleJSONtoXMLforjsonToXml(’root’,x)withaninvalidname
{"/invalid":"abc"}
XML:
SampleJSONtoXMLforjsonToXml(’root’,x)withaninvalidname
<?xmlversion="1.0"?>
<root>
<_u002F_invalid>abc</_u002F_invalid>
</root>
JSONARRAY
ReturnsaJSONarray.
JSONARRAY(value...)
valueisanyobjectconvertabletoaJSONvalue.ReturnvalueisaclobmarkedasbeingvalidJSON.
Nullvalueswillbeincludedintheresultasnullliterals.
mixedvalueexample
jsonArray('a"b',1,null,false,{d'2010-11-21'})
Wouldreturn
["a\"b",1,null,false,"2010-11-21"]
JSONOBJECT
ReturnsaJSONobject.
ScalarFunctions
381
JSONARRAY(value[asname]...)
valueisanyobjectconvertabletoaJSONvalue.ReturnvalueisaclobmarkedasbeingvalidJSON.
Nullvalueswillbeincludedintheresultasnullliterals.
Ifanameisnotsuppliedandtheexpressionisacolumnreference,thecolumnnamewillbeusedotherwiseexprNwillbeusedwhereNisthe1-basedindexofthevalueintheJSONARRAYexpression.
mixedvalueexample
jsonObject('a"b'asval,1,nullas"null")
Wouldreturn
{"val":"a\"b","expr2":1,"null":null}
JSONPARSEValidatesandreturnsaJSONresult.
JSONPARSE(value,wellformed)
valueisblobwithanappropriateJSONbinaryencoding(UTF-8,UTF-16,orUTF-32)oraclob.wellformedisabooleanindicatingthatvalidationshouldbeskipped.ReturnvalueisaclobmarkedasbeingvalidJSON.
Anullforeitherinputwillreturnnull.
jsonparseofasimpleliteralvalue
jsonParse('"a"',true)
ConversiontoJSON
Astraight-forwardspecificationcompliantconversionisusedforconvertingvaluesintotheirappropriateJSONdocumentform.
nullvaluesareincludedasthenullliteral.
valuesparsedasJSONorreturnedfromaJSONconstructionfunction(JSONPARSE,JSONARRAY,JSONARRAY_AGG)willbedirectlyappendedintoaJSONresult.
booleanvaluesareincludedastrue/falseliterals
numericvaluesareincludedastheirdefaultstringconversion-insomecircumstancesifnotanumberor+-infinityresultsareallowed,invalidjsonmaybeobtained.
stringvaluesareincludedintheirescaped/quotedform.
binaryvaluesarenotimplicitlyconvertabletoJSONvaluesandrequireaspecificpriortoinclusioninJSON.
allothervalueswillbeincludedastheirstringconversionintheappropriateescaped/quotedform.
ScalarFunctions
382
ScalarFunctions
383
SecurityFunctionsSecurityfunctionsprovidetheabilitytointeractwiththesecuritysystemortohash/encryptvalues.
TableofContentsHASROLEMD5SHA1SHA2_256SHA2_512AES_ENCRYPTAES_DECRYPT
HASROLE
WhetherthecurrentcallerhastheTeiiddataroleroleName.
hasRole([roleType,]roleName)
roleNamemustbeastring,thereturntypeisboolean.
Thetwoargumentformisprovidedforbackwardscompatibility.roleTypeisastringandmustbe`data'.
Rolenamesarecase-sensitiveandonlymatchTeiidDataRoles.JAASroles/groupsnamesarenotvalidforthisfunction-unlessthereiscorrespondingdatarolewiththesamename.
MD5ComputestheMD5hashofthevalue.
MD5(value)
valuemustbeastringorvarbinary,thereturntypeisvarbinary.StringvaluesarefirstconvertedtotheirUTF-8byterepresentation.
SHA1ComputestheSHA-1hashofthevalue.
SHA1(value)
valuemustbeastringorvarbinary,thereturntypeisvarbinary.StringvaluesarefirstconvertedtotheirUTF-8byterepresentation.
SHA2_256
ComputestheSHA-2256bithashofthevalue.
ScalarFunctions
384
SHA2_256(value)
valuemustbeastringorvarbinary,thereturntypeisvarbinary.StringvaluesarefirstconvertedtotheirUTF-8byterepresentation.
SHA2_512ComputestheSHA-2512bithashofthevalue.
SHA2_512(value)
valuemustbeastringorvarbinary,thereturntypeisvarbinary.StringvaluesarefirstconvertedtotheirUTF-8byterepresentation.
AES_ENCRYPT
aes_encrypt(data,key)
AES_ENCRYPT()allowencryptionofdatausingtheofficialAES(AdvancedEncryptionStandard)algorithm,16bytes(128bit)keylength,andAES/CBC/PKCS5Paddingcipheralgorithmwithanexplicitinitializationvector.
TheAES_ENCRYPT()willreturnaBinaryTypeencrypteddata.TheargumentdataisaBinaryTypedatathatneedtoencrypt,theargumentkeyisaBinaryTypeusedinencryption.
AES_DECRYPT
aes_decrypt(data,key)
AES_DECRYPT()allowdecryptionofdatausingtheofficialAES(AdvancedEncryptionStandard)algorithm,16bytes(128bit)keylength,andAES/CBC/PKCS5Paddingcipheralgorithmexpectinganexplicitinitializationvector.
TheAES_DECRYPT()willreturnaBinaryTypedecrypteddata.TheargumentdataisaBinaryTypedatathatneedtodecrypt,theargumentkeyisaBinaryTypeusedindecryption.
ScalarFunctions
385
SpatialFunctionsSpatialfunctionsprovidefunctionalityforworkingwithgeospatialdata.TeiidreliesontheJTSTopologySuitetoprovidepartialsupportfortheOpenGISSimpleFeaturesSpecificationForSQLRevision1.1.PleaserefertothespecificationortoPostGISformoredetailsaboutparticularfunctions.
MostGeometrysupportislimitedtotwodimensionsduetotheWKBandWKTformats.
Note Geometrysupportisstillevolving.TheremaybeminordifferencesbetweenTeiidandpushdownresultsthatwillneedtobefurtherrefined.
TableofContentsConversionFunctions
ST_GeomFromTextST_GeomFromWKB/ST_GeomFromBinaryST_GeomFromEWKBST_GeomFromTextST_GeomFromGeoJSONST_GeomFromGMLST_AsTextST_AsBinaryST_AsEWKBST_AsGeoJSONST_AsGMLST_AsEWKTST_AsKML
Operators&&
RelationshipFunctionsST_ContainsST_CrossesST_DisjointST_DistanceST_DWithinST_EqualsST_IntersectsST_OrderingEqualsST_OverlapsST_RelateST_TouchesST_Within
AttributesandTestsST_AreaST_CoordDimST_DimensionST_EndPointST_ExteriorRingST_GeometryNST_GeometryTypeST_HasArc
ScalarFunctions
386
ST_InteriorRingNST_IsClosedST_IsEmptyST_IsRingST_IsSimpleST_IsValidST_LengthST_NumGeometriesST_NumInteriorRingsST_NunPointsST_PointOnSurfaceST_PerimeterST_PointNST_SRIDST_SetSRIDST_StartPointST_XST_YST_Z
Misc.FunctionsST_BoundaryST_BufferST_CentroidST_ConvexHullST_CurveToLineST_DifferenceST_EnvelopeST_Force_2DST_IntersectionST_SimplifyST_SimplifyPreserveTopologyST_SnapToGridST_SymDifferenceST_TransformST_Union
AggregateFunctionsST_Extent
ConstructionFunctionsST_PointST_Polygon
ConversionFunctions
ST_GeomFromText
ReturnsageometryfromaClobinWKTformat.
ST_GeomFromText(text[,srid])
textisaclob,sridisanoptionalinteger.Returnvalueisageometry.
ScalarFunctions
387
ST_GeomFromWKB/ST_GeomFromBinary
ReturnsageometryfromablobinWKBformat.
ST_GeomFromWKB(bin[,srid])
binisablob,sridisanoptionalinteger.Returnvalueisageometry.
ST_GeomFromEWKB
ReturnsageometryfromablobinEWKBformat.
ST_GeomFromEWKB(bin)
binisablob.Returnvalueisageometry.Only2dimensionsaresupported.
ST_GeomFromText
ReturnsageometryfromaClobinEWKTformat.
ST_GeomFromEWKT(text)
textisaclob.Returnvalueisageometry.Only2dimensionsaresupported.
ST_GeomFromGeoJSON
ReturnsageometryfromaClobinGeoJSONformat.
ST_GeomFromGeoJson(text[,srid])
textisaclob,sridisanoptionalinteger.Returnvalueisageometry.
ST_GeomFromGML
ReturnsageometryfromaClobinGML2format.
ST_GeomFromGML(text[,srid])
textisaclob,sridisanoptionalinteger.Returnvalueisageometry.
ST_AsText
ST_AsText(geom)
geomisageometry.ReturnvalueisclobinWKTformat.
ST_AsBinary
ST_AsBinary(geom)
ScalarFunctions
388
geomisageometry.ReturnvalueisablobinWKBformat.
ST_AsEWKB
ST_AsEWKB(geom)
geomisageometry.ReturnvalueisblobinEWKBformat.
ST_AsGeoJSON
ST_AsGeoJSON(geom)
geomisageometry.ReturnvalueisaclobwiththeGeoJSONvalue.
ST_AsGML
ST_AsGML(geom)
geomisageometry.ReturnvalueisaclobwiththeGML2value.
ST_AsEWKT
ST_AsEWKT(geom)
geomisageometry.ReturnvalueisaclobwiththeEWKTvalue.TheEWKTvalueistheWKTvaluewiththeSRIDprefix.
ST_AsKML
ST_AsKML(geom)
geomisageometry.ReturnvalueisaclobwiththeKMLvalue.TheKMLvalueiseffectivelyasimplifiedGMLvalueandprojectedintoSRID4326.
Operators
&&
Returnstrueiftheboundingboxesofgeom1andgeom2intersect.
geom1&&geom2
geom1,geom2aregeometries.Returnvalueisaboolean.
RelationshipFunctions
ST_Contains
ScalarFunctions
389
Returnstrueifgeom1containsgeom2containsanother.
ST_Contains(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Crosses
Returnstrueifthegeometriescross.
ST_Crosses(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Disjoint
Returnstrueifthegeometriesaredisjoint.
ST_Disjoint(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Distance
Returnsthedistancebetweentwogeometries.
ST_Distance(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisadouble.
ST_DWithin
Returnstrueifthegeometriesarewithinagivendistanceofoneanother.
ST_DWithin(geom1,geom2,dist)
geom1,geom2aregeometries.distisadouble.Returnvalueisaboolean.
ST_Equals
Returnstrueifthetwogeometriesarespatiallyequal-thepointsandordermaydiffer,butneithergeometryliesoutsideoftheother.
ST_Equals(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Intersects
Returnstrueifthegeometriesintersect.
ScalarFunctions
390
ST_Intersects(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_OrderingEquals
Returnstrueifgeom1andgeom2havethesamestructureandthesameorderingofpoints.
ST_OrderingEquals(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Overlaps
Returnstrueifthegeometriesoverlap.
ST_Overlaps(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Relate
Testorreturntheintersectionofgeom1andgeom2.
ST_Relate(geom1,geom2,pattern)
geom1,geom2aregeometries.patternisaninecharacterDE-9IMpatternstring.Returnvalueisaboolean.
ST_Relate(geom1,geom2)
geom1,geom2aregeometries.ReturnvalueistheninecharacterDE-9IMintersectionstring.
ST_Touches
Returnstrueifthegeometriestouch.
ST_Touches(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
ST_Within
Returnstrueifgeom1iscompletelyinsidegeom2.
ST_Within(geom1,geom2)
geom1,geom2aregeometries.Returnvalueisaboolean.
AttributesandTests
ScalarFunctions
391
ST_Area
Returnstheareaofgeom.
ST_Area(geom)
geomisageometry.Returnvalueisadouble.
ST_CoordDim
Returnsthecoordinatedimensionsofgeom.
ST_CoordDim(geom)
geomisageometry.Returnvalueisanintegerbetween0and3.
ST_Dimension
Returnsthedimensionofgeom.
ST_Dimension(geom)
geomisageometry.Returnvalueisanintegerbetween0and3.
ST_EndPoint
ReturnstheendPointoftheLineStringgeom.ReturnsnullifgeomisnotaLineString.
ST_EndPoint(geom)
geomisageometry.Returnvalueisageometry.
ST_ExteriorRing
ReturnstheexteriorringorshellLineStringofthePolygongeom.ReturnsnullifgeomisnotaPolygon.
ST_ExteriorRing(geom)
geomisageometry.Returnvalueisageometry.
ST_GeometryN
Returnsthenthgeometryatthegiven1-basedindexingeom.Returnsnullifageometryatthegivenindexdoesnotexist.Noncollectiontypesreturnthemselvesatthefirstindex.
ST_GeometryN(geom,index)
geomisageometry.indexisaninteger.Returnvalueisageometry.
ST_GeometryType
ScalarFunctions
392
ReturnsthetypenameofgeomasST_name.WherenamewillbeLineString,Polygon,Pointetc.
ST_GeometryType(geom)
geomisageometry.Returnvalueisastring.
ST_HasArc
Testifthegeometryhasacircularstring.Willcurrentlyonlyreportfalseascurvedgeometrytypesarenotsupported.
ST_HasArc(geom)
geomisageometry.Returnvalueisageometry.
ST_InteriorRingN
ReturnsthenthinteriorringLinearStringgeometryatthegiven1-basedindexingeom.ReturnsnullifageometryatthegivenindexdoesnotexistorifgeomisnotaPolygon.
ST_InteriorRingN(geom,index)
geomisageometry.indexisaninteger.Returnvalueisageometry.
ST_IsClosed
ReturnstrueifLineStringgeomisclosed.ReturnsfalseifgeomisnotaLineString
ST_IsClosed(geom)
geomisageometry.Returnvalueisaboolean.
ST_IsEmpty
Returnstrueifthesetofpointsisempty.
ST_IsEmpty(geom)
geomisageometry.Returnvalueisaboolean.
ST_IsRing
ReturnstrueiftheLineStringgeomisaring.ReturnsfalseifgeomisnotaLineString.
ST_IsRing(geom)
geomisageometry.Returnvalueisaboolean.
ST_IsSimple
Returnstrueifthegeomissimple.
ScalarFunctions
393
ST_IsSimple(geom)
geomisageometry.Returnvalueisaboolean.
ST_IsValid
Returnstrueifthegeomisvalid.
ST_IsValid(geom)
geomisageometry.Returnvalueisaboolean.
ST_Length
Returnsthelengthofa(Multi)LineStringotherwise0.
ST_Length(geom)
geomisageometry.Returnvalueisadouble.
ST_NumGeometries
Returnsthenumberofgeometriesingeom.Willreturn1ifnotageometrycollection.
ST_NumGeometries(geom)
geomisageometry.Returnvalueisaninteger.
ST_NumInteriorRings
ReturnsthenumberofinteriorringsinthePolygongeom.ReturnsnullifgeomisnotaPolygon.
ST_NumInteriorRings(geom)
geomisageometry.Returnvalueisaninteger.
ST_NunPoints
ReturnsthenumberofPointsingeom.
ST_NunPoints(geom)
geomisageometry.Returnvalueisaninteger.
ST_PointOnSurface
ReturnsaPointthatisguarenteedtobeonthesurfaceofgeom.
ST_PointOnSurface(geom)
geomisageometry.ReturnvalueisaPointgeometry.
ScalarFunctions
394
ST_Perimeter
Returnstheperimeterofthe(Multi)Polygongeom.Willreturn0ifgeomisnota(Multi)Polygon
ST_Perimeter(geom)
geomisageometry.Returnvalueisadouble.
ST_PointN
ReturnsthenthPointatthegiven1-basedindexingeom.ReturnsnullifaPointatthegivenindexdoesnotexistorifgeomisnotaLineString.
ST_PointN(geom,index)
geomisageometry.indexisaninteger.Returnvalueisageometry.
ST_SRID
ReturnstheSRIDforthegeometry.
ST_SRID(geom)
geomisageometry.Returnvalueisaninteger.A0valueratherthannullwillbereturnedforanunknownSRIDonanon-nullgeometry.
ST_SetSRID
SettheSRIDforthegivengeometry.
ST_SetSRID(geom,srid)
geomisageometry.sridisaninteger.Returnvalueisageometry.OnlytheSRIDmetadataofthegeometryismodified.
ST_StartPoint
ReturnsthestartPointoftheLineStringgeom.ReturnsnullifgeomisnotaLineString.
ST_StartPoint(geom)
geomisageometry.Returnvalueisageometry.
ST_X
ReturnstheXordinatevalue,ornullifthePointisempty.ThrowsanexceptioniftheGeometryisnotaPoint.
ST_X(geom)
geomisageometry.Returnvalueisadouble.
ST_Y
ScalarFunctions
395
ReturnstheYordinatevalue,ornullifthePointisempty.ThrowsanexceptioniftheGeometryisnotaPoint.
ST_Y(geom)
geomisageometry.Returnvalueisadouble.
ST_Z
ReturnstheZordinatevalue,ornullifthePointisempty.ThrowsanexceptioniftheGeometryisnotaPoint.Willtypicallyreturnnullas3dimensionsarenotfullysupported.
ST_Z(geom)
geomisageometry.Returnvalueisadouble.
Misc.Functions
ST_Boundary
Computestheboundaryofthegivengeometry.
ST_Boundary(geom)
geomisageometry.Returnvalueisageometry.
ST_Buffer
Computesthegeometrythathaspointswithinthegivendistanceofgeom.
ST_Buffer(geom,distance)
geomisageometry.distanceisadouble.Returnvalueisageometry.
ST_Centroid
ComputesthegeometriccenterPointofgeom.
ST_Centroid(geom)
geomisageometry.Returnvalueisageometry.
ST_ConvexHull
ReturnthesmallestconvexPolygonthatcontainsallofthepointsingeom.
ST_ConvexHull(geom)
geomisageometry.Returnvalueisageometry.
ST_CurveToLine
ScalarFunctions
396
ConvertsaCircularString/CurvedPolygontoaLineString/Polygon.NotcurrentlyimplementedinTeiid.
ST_CurveToLine(geom)
geomisageometry.Returnvalueisageometry.
ST_Difference
Computestheclosureofthepointsetofthepointscontainedingeom1thatarenotingeom2
ST_Difference(geom1,geom2)
geom1,geom2aregeometry.Returnvalueisageometry.
ST_Envelope
Computesthe2Dboundingboxofthegivengeometry.
ST_Envelope(geom)
geomisageometry.Returnvalueisageometry.
ST_Force_2D
Removesthezcoordinatevalueifpresent.
ST_Force_2D(geom)
geomisageometry.Returnvalueisageometry.
ST_Intersection
Computesthepointsetintersectionofthepointscontainedingeom1andingeom2
ST_Intersection(geom1,geom2)
geom1,geom2aregeometry.Returnvalueisageometry.
ST_Simplify
SimplifiesaGeometryusingtheDouglas-Peuckeralgorithm,butmayoversimplifytoaninvalidoremptygeometry.
ST_Simplify(geom,distanceTolerance)
geomisageometry.distanceToleranceisadouble.Returnvalueisageometry.
ST_SimplifyPreserveTopology
SimplifiesaGeometryusingtheDouglas-Peuckeralgorithm.Willalwaysreturnavalidgeometry.
ST_SimplifyPreserveTopology(geom,distanceTolerance)
ScalarFunctions
397
geomisageometry.distanceToleranceisadouble.Returnvalueisageometry.
ST_SnapToGrid
Snapsallpointsinthegeometrytogridofgivensize.
ST_SnapToGrid(geom,size)
geomisageometry.sizeisadouble.Returnvalueisageometry.
ST_SymDifference
Returnthepartofgeom1thatdoesnotintersectwithgeom2andviceversa.
ST_SymDifference(geom1,geom2)
geom1,geom2aregeometry.Returnvalueisageometry.
ST_Transform
Transformsthegeometryvaluefromonecoordinatesystemtoanother.
ST_Transform(geom,srid)
geomisageometry.sridisaninteger.Returnvalueisageometry.ThesridvalueandthesridofthegeometryvaluemustexistintheSPATIAL_REF_SYSview.
ST_Union
Returnageometrythatrepresentsthepointsetcontainingallofgeom1andgeom2.
ST_Union(geom1,geom2)
geom1,geom2aregeometry.Returnvalueisageometry.
AggregateFunctions
ST_Extent
Computesthe2Dboundingboxaroundallofthegeometryvalues.Allvaluesshouldhavethesamesrid.
ST_Extent(geom)
geomisageometry.Returnvalueisageometry.
ConstructionFunctions
ST_Point
RetunsthePointforthegivencooridinates.
ScalarFunctions
398
ST_Point(x,y)
xandyaredoubles.ReturnvalueisaPointgeometry.
ST_Polygon
RetunsthePolygonwiththegivenshellandsrid.
ST_Polygon(geom,srid)
geomisalinearringgeometryandsridisaninteger.ReturnvalueisaPolygongeometry.
ScalarFunctions
399
MiscellaneousFunctionsDocumentsadditionalfunctionsandthosecontributedbyotherprojects.
TableofContentsArrayfunctions
array_getarray_length
OtherFunctionsuuid
DataQualityFunctionsosdq.randomosdq.digitosdq.whitespaceIndexosdq.validCreditCardosdq.validSSNosdq.validPhoneosdq.validEmailosdq.cosineDistanceosdq.jaccardDistanceosdq.jaroWinklerDistanceosdq.levenshteinDistance
Arrayfunctions
array_get
Returnstheobjectvalueatagivenarrayindex.
array_get(array,index)
arrayistheobjecttype,indexmustbeaninteger,andthereturntypeisobject.
1-basedindexingisused.Theactualarrayvalueshouldbeajava.sql.Arrayorjavaarraytype.Anullwillbereturnedifeitherargumentisnulloriftheindexisoutofbounds.
array_length
Returnsthelengthforagivenarray
array_length(array)
arrayistheobjecttype,andthereturntypeisinteger.
Theactualarrayvalueshouldbeajava.sql.Arrayorjavaarraytype.Anexceptionwillbethrownifthearrayvalueisthewrongtype.
OtherFunctions
ScalarFunctions
400
uuid
Retunsauniversallyuniqueidentifier.
uuid()
thereturntypeisstring.
Generatesatype4(pseudorandomlygenerated)UUIDusingacryptographicallystrongrandomnumbergenerator.TheformatisXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXXwhereeachXisahexdigit.
DataQualityFunctions
DataQualityfunctionsarecontributedbytheODDQProject.Thefunctionsareprefixedwith'osdq.',butmaybecalledwithouttheprefix.
osdq.random
Returnstherandomizedstring.Forexample,jbossteiidmayrandomizetojtidssoibe.
random(sourceValue)
ThesourceValueisthestringthatneedtorandomize.
osdq.digit
Returnsdigitcharactersofthestring.Forexample,a1b2c3d4willbecome1234
digit(sourceValue)
ThesourceValueisthestringthatneedtodigit.
osdq.whitespaceIndex
Returnstheindexofthefirstwhitespace,Forexample,jbossteiidwillreturn5.
whitespaceIndex(sourceValue)
ThesourceValueisthestringthatneedtofindwhitespaceindex.
osdq.validCreditCard
CheckwhetheraCreditCardnumberisavalidCreditCardnumber,returntrueifmatchescreditcardlogicandchecksum.
validCreditCard(cc)
TheccistheCreditCardnumberstringthatneedtocheck.
osdq.validSSN
CheckwhetheraSSNnumberisavalidSSNnumber,returntrueifmatchesssnlogic.
ScalarFunctions
401
validSSN(ssn)
ThessnistheSSNnumberstringthatneedtocheck.
osdq.validPhone
Checkwhetheraphonenumberisavalidphonenumber,returntrueifmatchesphonelogicthatmorethan8characterlessthan12character,can’tstartwith000.
validPhone(phone)
Thephoneisthephonenumberstringneedtocheck.
osdq.validEmail
Checkwhetheraemailaddressisavalidemailaddress,returntrueifvalid.
validEmail(email)
Theemailistheemailadressstringthatneedtocheck.
osdq.cosineDistance
ReturnsthefloatdistancebetweentwostringwhichbaseonCosineSimilarityalgorithm.
cosineDistance(a,b)
Theaandbarestringsthatneedtocalculatethedistance.
osdq.jaccardDistance
ReturnsthefloatdistancebetweentwostringwhichbaseonJaccardsimilarityalgorithm.
jaccardDistance(a,b)
Theaandbarestringsthatneedtocalculatethedistance.
osdq.jaroWinklerDistance
ReturnsthefloatdistancebetweentwostringwhichbaseonJaro-Winkleralgorithm.
jaroWinklerDistance(a,b)
Theaandbarestringsthatneedtocalculatethedistance.
osdq.levenshteinDistance
ReturnsthefloatdistancebetweentwostringwhichbaseonLevenshteinalgorithm.
levenshteinDistance(a,b)
ScalarFunctions
402
Theaandbarestringsthatneedtocalculatethedistance.
ScalarFunctions
403
NondeterministicFunctionHandlingTeiidcategorizesfunctionsbyvaryingdegreesofdeterminism.Whenafunctionisevaluatedandtowhatextenttheresultcanbecachedarebaseduponitsdeterminismlevel.
1. Deterministic-thefunctionwillalwaysreturnthesameresultforthegiveninputs.Deterministicfunctionsareevaluatedbytheengineassoonasallinputvaluesareknown,whichmayoccurassoonastherewritephase.Somefunctions,suchasthelookupfunction,arenottrulydeterministic,butistreatedassuchforperformance.Allfunctionsnotcategorizedbelowareconsidereddeterministic.
2. UserDeterministic-thefunctionwillreturnthesameresultforthegiveninputsforthesameuser.ThisincludesthehasRoleanduserfunctions.Userdeterministicfunctionsareevaluatedbytheengineassoonasallinputvaluesareknown,whichmayoccurassoonastherewritephase.Ifauserdeterministicfunctionisevaluatedduringthecreationofapreparedprocessingplan,thentheresultingplanwillbecachedonlyfortheuser.
3. SessionDeterministic-thefunctionwillreturnthesameresultforthegiveninputsunderthesameusersession.Thiscategoryincludestheenvfunction.Sessiondeterministicfunctionsareevaluatedbytheengineassoonasallinputvaluesareknown,whichmayoccurassoonastherewritephase.Ifasessiondeterministicfunctionisevaluatedduringthecreationofapreparedprocessingplan,thentheresultingplanwillbecachedonlyfortheuser’ssession.
4. CommandDeterministic-theresultoffunctionevaluationisonlydeterministicwithinthescopeoftheusercommand.Thiscategoryincludethecurdate,curtime,now,andcommandpayloadfunctions.Commanddeterministicfunctionsaredelayedinevaluationuntilprocessingtoensurethatevenpreparedplansutilizingthesefunctionswillbeexecutedwithrelevantvalues.Commanddeterministicfunctionevaluationwilloccurpriortopushdown-howevermultipleoccurrencesofthesamecommanddeterministictimefunctionarenotguaranteedtoevaluatetothesamevalue.
5. Nondeterministic-theresultoffunctionevaluationisfullynondeterministic.ThiscategoryincludestherandfunctionandUDFsmarkedasnondeterministic.Nondeterministicfunctionsaredelayedinevaluationuntilprocessingwithapreferenceforpushdown.Ifthefunctionisnotpusheddown,thenitmaybeevaluatedforeveryrowinit’sexecutioncontext(forexampleifthefunctionisusedintheselectclause).
Note Uncorrelatedsubquerieswillbetreatedasdeterministicregardlessofthefunctionsusedwithinthem.
ScalarFunctions
404
DMLCommandsTeiidsupportsSQLforissuingqueriesandfordefiningviewtransformations;seealsoProcedureLanguageforhowSQLisusedinvirtualproceduresandupdateprocedures.NearlyallthesefeaturesfollowstandardSQLsyntaxandfunctionality,soanySQLreferencecanbeusedformoreinformation.
Thereare4basiccommandsformanipulatingdatainSQL,correspondingtotheCRUDcreate,read,update,anddeleteoperations:INSERT,SELECT,UPDATE,andDELETE.AMERGEstatementactsasacombinationofINSERTandUPDATE.
Inaddition,procedurescanbeexecutedusingtheEXECUTEcommand,throughaProceduralRelationalCommand,oranAnonymousProcedureBlock.
SELECTCommandTheSELECTcommandisusedtoretrieverecordsanynumberofrelations.
ASELECTcommandhasanumberofclauses:
WITH…
SELECT…
[FROM…]
[WHERE…]
[GROUPBY…]
[HAVING…]
[ORDERBY…]
[(LIMIT…)|([OFFSET…][FETCH…])]
[OPTION…]
AlloftheseclausesotherthanOPTIONaredefinedbytheSQLspecification.Thespecificationalsospecifiestheorderthattheseclauseswillbelogicallyprocessed.Belowistheprocessingorderwhereeachstagepassesasetofrowstothefollowingstage.Notethatthisprocessingmodelislogicalanddoesnotrepresentthewayanyactualdatabaseengineperformstheprocessing,althoughitisausefulmodelforunderstandingquestionsaboutSQL.
WITHstage-gathersallrowsfromallwithitemsintheorderlisted.Subsequentwithitemsandthemainquerycanreferencetheawithitemasifitisatable.
FROMstage-gathersallrowsfromalltablesinvolvedinthequeryandlogicallyjoinsthemwithaCartesianproduct,producingasinglelargetablewithallcolumnsfromalltables.Joinsandjoincriteriaarethenappliedtofilterrowsthatdonotmatchthejoinstructure.
WHEREstage-appliesacriteriatoeveryoutputrowfromtheFROMstage,furtherreducingthenumberofrows.
GROUPBYstage-groupssetsofrowswithmatchingvaluesinthegroupbycolumns.
HAVINGstage-appliescriteriatoeachgroupofrows.Criteriacanonlybeappliedtocolumnsthatwillhaveconstantvalueswithinagroup(thoseinthegroupingcolumnsoraggregatefunctionsappliedacrossthegroup).
DMLCommands
405
SELECTstage-specifiesthecolumnexpressionsthatshouldbereturnedfromthequery.Expressionsareevaluated,includingaggregatefunctionsbasedonthegroupsofrows,whichwillnolongerexistafterthispoint.Theoutputcolumnsarenamedusingeithercolumnaliasesoranimplicitnamedeterminedbytheengine.IfSELECTDISTINCTisspecified,duplicateremovalwillbeperformedontherowsbeingreturnedfromtheSELECTstage.
ORDERBYstage-sortstherowsreturnedfromtheSELECTstageasdesired.Supportssortingonmultiplecolumnsinspecifiedorder,ascendingordescending.TheoutputcolumnswillbeidenticaltothosecolumnsreturnedfromtheSELECTstageandwillhavethesamename.
LIMITstage-returnsonlythespecifiedrows(withskipandlimitvalues).
ThismodelcanbeusedtounderstandmanyquestionsaboutSQL.Forexample,columnsaliasedintheSELECTclausecanonlybereferencedbyaliasintheORDERBYclause.Withoutknowledgeoftheprocessingmodel,thiscanbesomewhatconfusing.Seeninlightofthemodel,itisclearthattheORDERBYstageistheonlystageoccurringaftertheSELECTstage,whichiswherethecolumnsarenamed.BecausetheWHEREclauseisprocessedbeforetheSELECT,thecolumnshavenotyetbeennamedandthealiasesarenotyetknown.
Tip TheexplicittablesyntaxTABLExmaybeusedasashortcutforSELECT*FROMx.
VALUESCommandTheVALUEScommandisusedtoconstructasimpletable.
ExampleSyntax
VALUES(value,...)
VALUES(value,...),(valueX,...)...
AVALUEScommandwithasinglevaluesetisequivalentto"SELECTvalue,….".AVALUEScommandwithmultiplevaluessetsisequivalenttoaUNIONALLofsimpleSELECTs-"SELECTvalue,….UNIONALLSELECTvalueX,…".
UpdateCommands
Updatecommandscanreportintegerupdatecounts.Ifalargernumberorrowsisupdated,thenthemaxintegervaluewillbereported(2^31-1).
INSERTCommand
TheINSERTcommandisusedtoaddarecordtoatable.
ExampleSyntax
INSERTINTOtable(column,...)VALUES(value,...)
INSERTINTOtable(column,...)query
UPDATECommand
DMLCommands
406
TheUPDATEcommandisusedtomodifyrecordsinatable.Theoperationmayresultin1ormorerecordsbeingupdated,orinnorecordsbeingupdatedifnonematchthecriteria.
ExampleSyntax
UPDATEtableSET(column=value,...)[WHEREcriteria]
DELETECommand
TheDELETEcommandisusedtoremoverecordsfromatable.Theoperationmayresultin1ormorerecordsbeingdeleted,orinnorecordsbeingdeletedifnonematchthecriteria.
ExampleSyntax
DELETEFROMtable[WHEREcriteria]
UPSERT/MERGECommandTheUPSERT(orMERGE)isusedtoaddand/orupdaterecords.TheTeiidspecific(non-ANSI)UPSERTissimplyamodifiedINSERTstatementthatrequiresthetargettabletohaveaprimarykeyandforthetargetcolumnstocovertheprimarykey.TheUPSERToperationwillthenchecktheexistenceofeachrowpriortoINSERTandinsteadperformanUPDATEiftherowalreadyexists.
ExampleSyntax
UPSERTINTOtable(column,...)VALUES(value,...)
UPSERTINTOtable(column,...)query
Note UPSERTPushdown-IfUPSERTstatementisnotpushedtothesource,itwillbebrokendownintotherespectiveinsert/updateoperations,whichrequiresXAsupportonthetargetsystemtoguaranteeatomicity.
EXECUTECommand
TheEXECUTEcommandisusedtoexecuteaprocedure,suchasavirtualprocedureorastoredprocedure.Proceduresmayhavezeroormorescalarinputparameters.Thereturnvaluefromaprocedureisaresultsetorthesetofinout/out/returnscalars.NotethatEXECorCALLcanbeusedasashortformofthiscommand.
ExampleSyntax
EXECUTEproc()
CALLproc(value,...)
NamedParameterSyntax
EXECUTEproc(name1=>value1,name4=>param4,...)
DMLCommands
407
SyntaxRules:
Thedefaultorderofparameterspecificationisthesameashowtheyaredefinedintheproceduredefinition.
Youcanspecifytheparametersinanyorderbyname.Parametersthatarehavedefaultvaluesand/orarenullableinthemetadata,canbeomittedfromthenamedparametercallandwillhavetheappropriatevaluepassedatruntime.
Positionalparametersthatarehavedefaultvaluesand/orarenullableinthemetadata,canbeomittedfromtheendoftheparameterlistandwillhavetheappropriatevaluepassedatruntime.
Iftheproceduredoesnotreturnaresultset,thevaluesfromtheRETURN,OUT,andIN_OUTparameterswillbereturnedasasinglerowwhenusedasaninlineviewquery.
AVARIADICparametermayberepeated0ormoretimesasthelastpositionalargument.
ProceduralRelationalCommand
ProceduralrelationalcommandsusethesyntaxofaSELECTtoemulateanEXEC.InaproceduralrelationalcommandaproceduregroupnamesisusedinaFROMclauseinplaceofatable.Thatprocedurewillbeexecutedinplaceofanormaltableaccessifallofthenecessaryinputvaluescanbefoundincriteriaagainsttheprocedure.Eachcombinationofinputvaluesfoundinthecriteriaresultsinanexecutionoftheprocedure.
ExampleSyntax
select*fromproc
selectoutput_param1,output_param2fromprocwhereinput_param1='x'
selectoutput_param1,output_param2fromproc,tablewhereinput_param1=table.col1andinput_param2=table.c
ol2
SyntaxRules:
Theprocedureasatableprojectsthesamecolumnsasanexecwiththeadditionoftheinputparameters.Forproceduresthatdonotreturnaresultset,IN_OUTcolumnswillbeprojectedastwocolumns,onethatrepresentstheoutputvalueandonenamed\{columnname}_INthatrepresentstheinputoftheparameter.
Inputvaluesarepassedviacriteria.Valuescanbepassedby'=','isnull',or'in'predicates.Disjunctsarenotallowed.Itisalsonotpossibletopassthevalueofanon-comparablecolumnthroughanequalitypredicate.
TheprocedureviewautomaticallyhasanaccesspatternonitsINandIN_OUTparameterswhichallowsittobeplannedcorrectlyasadependentjoinwhennecessaryorfailwhensufficientcriteriacannotbefound.
Procedurescontainingduplicatenamesbetweentheparameters(IN,IN_OUT,OUT,RETURN)andresultsetcolumnscannotbeusedinaproceduralrelationalcommand.
DefaultvaluesforIN,IN_OUTparametersarenotusedifthereisnocriteriapresentforagiveninput.Defaultvaluesareonlyvalidfornamedproceduresyntax.
MultipleExecution
Theusageof'in'orjoincriteriacanresultintheprocedurebeingexecutedmultipletimes.
AlternativeSyntax
Noneofissueslistedinthesyntaxrulesaboveexistifanestedtablereferenceisused.
DMLCommands
408
AnonymousProcedureBlock
AProcedureLanguageblockmaybeexecutedasausercommand.Thisisadvantageousinsituationswhenavirtualproceduredoesn’texists,butasetofprocessingcanbecaredoutontheserversidetogether.
ExampleSyntax
begininsertintopm1.g1(e1,e2)select?,?;selectrowcount;end;
SyntaxRules:
Inparametersaresupportedwithprepared/callablestatementparametersasshownabovewitha?parameter.
outparametersarenotyetsupported-considerusingsessionvariablesasaworkaroundasneeded.
areturnparameterisnotsupported.
asingleresultwillbereturnedifanyofthestatementsreturnsaresultset.Allreturnableresultsetsmusthaveamatchingnumberofcolumnsandtypes.UsetheWITHOUTRETURNclausetoindicatethatastatementisnotintendedtoaresultsetasneeded.
DMLCommands
409
SetOperationsTeiidsupportstheUNION,UNIONALL,INTERSECT,EXCEPTsetoperationasawayofcombiningtheresultsofqueryexpressions.
Usage:
queryExpression(UNION|INTERSECT|EXCEPT)[ALL]queryExpression[ORDERBY...]
SyntaxRules:
Theoutputcolumnswillbenamedbytheoutputcolumnsofthefirstsetoperationbranch.
EachSELECTmusthavethesamenumberofoutputcolumnsandcompatibledatatypesforeachrelativecolumn.Datatypeconversionwillbeperformedifdatatypesareinconsistentandimplicitconversionsexist.
IfUNION,INTERSECT,orEXCEPTisspecifiedwithoutall,thentheoutputcolumnsmustbecomparabletypes.
INTERSECTALL,andEXCEPTALLarecurrentlynotsupported.
DMLCommands
410
SubqueriesAsubqueryisaSQLqueryembeddedwithinanotherSQLquery.Thequerycontainingthesubqueryistheouterquery.
Supportedsubquerytypes:
Scalarsubquery-asubquerythatreturnsonlyasinglecolumnwithasinglevalue.Scalarsubqueriesareatypeofexpressionandcanbeusedwheresinglevaluedexpressionsareexpected.
Correlatedsubquery-asubquerythatcontainsacolumnreferencetofromtheouterquery.
Uncorrelatedsubquery-asubquerythatcontainsnoreferencestotheoutersub-query.
InlineviewsSubqueriesintheFROMclauseoftheouterquery(alsoknownas"inlineviews")canreturnanynumberofrowsandcolumns.Thistypeofsubquerymustalwaysbegivenanalias.Aninlineviewisnearlyidenticaltoatraditionalview.SeealsoWITHClause.
ExampleSubqueryinFROMClause(InlineView)
SELECTaFROM(SELECTY.b,Y.cFROMYWHEREY.d='3')ASXWHEREa=X.cANDb=X.b
Subqueriescanappearanywherewhereanexpressionorcriteriaisexpected.Subqueriesaresupportedinquantifiedcriteria,theEXISTSpredicate,theINpredicate,andasScalarSubqueries.
ExampleSubqueryinWHEREUsingEXISTS
SELECTaFROMXWHEREEXISTS(SELECT1FROMYWHEREc=X.a)
ExampleQuantifiedComparisonSubqueries
SELECTaFROMXWHEREa>=ANY(SELECTbFROMYWHEREc=3)
SELECTaFROMXWHEREa<SOME(SELECTbFROMYWHEREc=4)
SELECTaFROMXWHEREa=ALL(SELECTbFROMYWHEREc=2)
ExampleINSubquery
SELECTaFROMXWHEREaIN(SELECTbFROMYWHEREc=3)
SeealsoSubqueryOptimization.
DMLCommands
411
WITHClauseTeiidsupportsnon-recursivecommontableexpressionsviatheWITHclause.WITHclauseitemsmaybereferencedastablesinsubsequentwithclauseitemsandinthemainquery.TheWITHclausecanbethoughtofasprovidingqueryscopedtemporarytables.
Usage:
WITHname[(column,...)]AS[/*+no_inline|materialize*/](queryexpression)...
SyntaxRules:
Alloftheprojectedcolumnnamesmustbeunique.Iftheyarenotunique,thenthecolumnnamelistmustbeprovided.
IfthecolumnsoftheWITHclauseitemaredeclared,thentheymustmatchthenumberofcolumnsprojectedbythequeryexpression.
Eachwithclauseitemmusthaveauniquename.
Theoptionalno_inlinehintindicatestotheoptimizerthatthequeryexpressionshouldnotbesubstitutedasaninlineviewwherereferenced.Itispossiblewithno_inlineformultipleevaluationsofthecommontableasneededbysourcequeries.
TheoptionalmaterializehintrequiresthatthecommontablebecreatedasatemporarytableinTeiid.Thisforcesasingleevaluationofthecommontable.
Note TheWITHclauseisalsosubjecttooptimizationandit’sentriesmaynotbeprocessediftheyarenotneededinthesubsequentquery.
Examples:
WITHn(x)AS(selectcolfromtbl)selectxfromn,nasn1
WITHn(x)AS/*+no_inline*/(selectcolfromtbl)selectxfromn,nasn1
RecursiveCommonTableExpressions
Arecursivecommontableexpressionisaspecialformofacommontableexpressionthatisallowedtorefertoitselftobuildthefullcommontableresultinarecursiveoriterativefashion.
Usage:
WITHname[(column,...)]AS(anchorqueryexpressionUNION[ALL]recursivequeryexpression)...
Therecursivequeryexpressionisallowedtorefertothecommontablebyname.ProcessingflowswithTheanchorqueryexpressionexecutedfirst.Theresultswillbeaddedtothecommontableandwillbereferencedfortheexecutionoftherecursivequeryexpression.Theprocesswillberepeatedagainstthenewresultsuntiltherearenomoreintermediateresults.
Note Anonterminatingrecursivecommontableexpressioncanleadtoexcessiveprocessing.
Topreventrunawayprocessingofarecursivecommontableexpression,processingisbydefaultlimitedto10000iterations.Recursivecommontableexpressionsthatarepusheddownarenotsubjecttothislimit,butmaybesubjecttoothersourcespecificlimits.Thelimitcanbemodifiedbysettingthesessionvariableteiid.maxRecusiontoalargerintegervalue.Oncethemaxhas
DMLCommands
412
beenexceededanexceptionwillbethrown.
Example:
SELECTteiid_session_set('teiid.maxRecursion',25);
WITHn(x)AS(values('a')UNIONselectchr(ascii(x)+1)fromnwherex<'z')select*fromn
Thiswillfailtoprocessastherecursionlimitwillbereachedbeforeprocessingcompletes.
DMLCommands
413
SELECTClauseSQLqueriesthatstartwiththeSELECTkeywordandareoftenreferredtoas"SELECTstatements".TeiidsupportsmostofthestandardSQLqueryconstructs.
Usage:
SELECT[DISTINCT|ALL]((expression[[AS]name])|(groupidentifier.STAR))*|STAR...
SyntaxRules:
AliasedexpressionsareonlyusedastheoutputcolumnnamesandintheORDERBYclause.Theycannotbeusedinotherclausesofthequery.
DISTINCTmayonlybespecifiediftheSELECTsymbolsarecomparable.
DMLCommands
414
FROMClauseTheFROMclausespecifiesthetargettable(s)forSELECT,UPDATE,andDELETEstatements.
ExampleSyntax:
FROMtable[[AS]alias]
FROMtable1[INNER|LEFTOUTER|RIGHTOUTER|FULLOUTER]JOINtable2ONjoin-criteria
FROMtable1CROSSJOINtable2
FROM(subquery)[AS]alias
FROMTABLE(subquery)[AS]alias
FROMtable1JOIN/*+MAKEDEP*/table2ONjoin-criteria
FROMtable1JOIN/*+MAKENOTDEP*/table2ONjoin-criteria
FROM/*+MAKEIND*/table1JOINtable2ONjoin-criteria
FROM/*+NO_UNNEST*/vw1JOINtable2ONjoin-criteria
FROMtable1leftouterjoin/*+optional*/table2ONjoin-criteria
FROMTEXTTABLE…
FROMXMLTABLE…
FROMARRAYTABLE…
FROMOBJECTTABLE…
FROM(SELECT…
FromClauseHintsFromclausehintsaretypicallyspecifiedinacommentblockprecedingtheaffectedclause.MAKEDEPandMAKENOTDEPmayalsoappearafterinnon-commentformaftertheaffectedclause.Ifmultiplehintsapplytothatclause,thehintsshouldbeplacedinthesamecommentblock.
ExampleHint
FROM/*+MAKEDEPPRESERVE*/(tbl1innerjointbl2innerjointbl3ontbl2.col1=tbl3.col1ontbl1.col1=tbl2
.col1),tbl3WHEREtbl1.col1=tbl2.col1
DependentJoins
MAKEIND,MAKEDEP,andMAKENOTDEParehintsusedtocontroldependentjoinbehavior.Theyshouldonlybeusedinsituationswheretheoptimizerdoesnotchoosethemostoptimalplanbaseduponquerystructure,metadata,andcostinginformation.Thehintsmayappearinacommentthatproceedsthefromclause.Thehintscanbespecifiedagainstanyfromclause,notjustanamedtable.
MAKEIND-treatthisclauseastheindependent(feeder)sideofadependentjoinifpossible.
MAKEDEP-treatthisclauseasthedependent(filtered)sideofadependentjoinifpossible.
DMLCommands
415
MAKENOTDEP-donottreatthisclauseasthedependent(filtered)sideofajoin.
MAKEDEPandMAKEINDsupportoptionalmaxandjoinarguments:
MAKEDEP(JOIN)meansthattheentirejoinshouldbepushed
MAKEDEP(NOJOIN)meansthattheentirejoinshouldnotbepushed
MAKEDEP(MAX:val)meaningthatthedependentjoinshouldonlybeperformediftherearelessthanthemaxnumberofvaluesfromtheindependentside.
OtherHints
NO_UNNESTcanbespecifiedagainstasubqueryfromclauseorviewtoinstructtheplannertonotmergethenestedSQLinthesurroundingquery-alsoknownasviewflattening.ThishintonlyappliestoTeiidplanningandisnotpassedtosourcequeries.NO_UNNESTmayappearinacommentthatproceedsthefromclause.
ThePRESERVEhintcanbeusedagainstanANSIjointreetopreservethestructureofthejoinratherthanallowingtheTeiidoptimizertoreorderthejoin.ThisissimilarinfunctiontotheOracleORDEREDorMySQLSTRAIGHT_JOINhints.
ExamplePRESERVEHint
FROM/*+PRESERVE*/(tbl1innerjointbl2innerjointbl3ontbl2.col1=tbl3.col1ontbl1.col1=tbl2.col1)
NestedTableReference
NestedtablesmayappearintheFROMclausewiththeTABLEkeyword.Theyareanalternativetousingaviewwithnormaljoinsemantics.ThecolumnsprojectedfromthecommandcontainedinthenestedtablemaybeusedjustasanyoftheotherFROMclauseprojectedcolumnsinjoincriteria,thewhereclause,etc.
AnestedtablemayhavecorrelatedreferencestoprecedingFROMclausecolumnreferencesaslongasINNERandLEFTOUTERjoinsareused.Thisisespeciallyusefulincaseswherethennestedexpressionisaprocedureorfunctioncall.
Validexample:
select*fromt1,TABLE(callproc(t1.x))t2
Invalidexample,sincet1appearsafterthenestedtableinthefromclause:
select*fromTABLE(callproc(t1.x))t2,t1
Note MultipleExecution-Theusageofacorrelatednestedtablemayresultinmultipleexecutionsofthetableexpression-onceforeachcorrelatedrow.
DMLCommands
416
XMLTABLETheXMLTABLEfunctionusesXQuerytoproducetabularoutput.TheXMLTABLEfunctionisimplicitlyanestedtableandmaybecorrelatedtoprecedingFROMclauseentries.XMLTABLEispartoftheSQL/XML2006specification.
Usage:
XMLTABLE([<NSP>,]xquery-expression[<PASSING>][COLUMNS<COLUMN>,...)]ASname
COLUMN:=name(FORORDINALITY|(datatype[DEFAULTexpression][PATHstring]))
SeeXMLELEMENTforthedefinitionofNSP-XMLNAMESPACES.
SeeXMLQUERYforthedefinitionofPASSING.
SeealsoXMLQUERY
Note SeealsoXQueryOptimization
Parameters
TheoptionalXMLNAMESPACESclausespecifiesthenamepacesforuseintheXQueryandCOLUMNpathexpressions.
Thexquery-expressionshouldbeavalidXQuery.EachsequenceitemreturnedbythexquerywillbeusedtocreatearowofvaluesasdefinedbytheCOLUMNSclause.
IfCOLUMNSisnotspecified,thenthatisthesameashavingtheCOLUMNSclause:"COLUMNSOBJECT_VALUEXMLPATH'."',whichreturnstheentireitemasanXMLvalue.
AFORORDINALITYcolumnistypedasintegerandwillreturnthe1-baseditemnumberasitsvalue.
Eachnon-ordinalitycolumnspecifiesatypeandoptionallyaPATHandaDEFAULTexpression.
IfPATHisnotspecified,thenthepathwillbethesameasthecolumnname.
SyntaxRules:
Only1FORORDINALITYcolumnmaybespecified.
Thecolumnsnamesmustnotcontainduplicates.
Theblobdatatypeissupported,butthereisonlybuilt-insupportforxs:hexBinaryvalues.Forxs:base64Binary,useaworkaroundofaPATHthatusestheexplicitvalueconstructor"xs:base64Binary(<path>)".
Thecolumnexpressionmustevaluatetoasinglevalueifanon-arraytypeisexpected.
Ifanarraytypeisspecifiedthenanarraywillbereturnedunlesstherearenoelementsinthesequence,inwhichcaseanullvalueisreturned.
Anemptyelementisnotavalidnullvalueasitiseffectivelytheemptystring.Thexsi:nilattributeshouldbeusedtodefineconveyanullvaluedelement.
Examples
Useofpassing,returns1row[1]:
select*fromxmltable('/a'PASSINGxmlparse(document'<aid="1"/>')COLUMNSidintegerPATH'@id')x
DMLCommands
417
Asanestedtable:
selectx.*fromt,xmltable('/x/y'PASSINGt.docCOLUMNSfirststring,secondFORORDINALITY)x
Invalidmulti-value:
select*fromxmltable('/a'PASSINGxmlparse(document'<a><bid="1"/><bid="2"/></a>')COLUMNSidintegerPATH
'b/@id')x
Arraymulti-value:
select*fromxmltable('/a'PASSINGxmlparse(document'<a><bid="1"/><bid="2"/></a>')COLUMNSidinteger[]PATH
'b/@id')x
Nilelement.Withoutthenilattributeanexceptionwouldbethrownconvertingbtoanintegervalue.
select*fromxmltable('/a'PASSINGxmlparse(document'<axmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
><bxsi:nil="true"/></a>')COLUMNSidintegerPATH'b')x
DMLCommands
418
ARRAYTABLETheARRAYTABLEfunctionprocessesanarrayinputtoproducetabularoutput.Thefunctionitselfdefineswhatcolumnsitprojects.TheARRAYTABLEfunctionisimplicitlyanestedtableandmaybecorrelatedtoprecedingFROMclauseentries.
Usage:
ARRAYTABLE(expressionCOLUMNS<COLUMN>,...)ASname
COLUMN:=namedatatype
Parameters
expression-thearraytoprocess,whichshouldbeajava.sql.Arrayorjavaarrayvalue.
SyntaxRules:
Thecolumnsnamesmustbenotcontainduplicates.
Examples
Asanestedtable:
selectx.*from(callsource.invokeMDX('somequery'))r,arraytable(r.tupleCOLUMNSfirststring,secondbigdec
imal)x
ARRAYTABLEiseffectivelyashortcutforusingtheMiscellaneousFunctions#array_getfunctioninanestedtable.Forexample
ARRAYTABLE(valCOLUMNScol1string,col2integer)ASX
isthesameas
TABLE(SELECTcast(array_get(val,1)ASstring)AScol1,cast(array_get(val,2)ASinteger)AScol2)ASX
*Prev*FROMClause
Topofpage
Frontpage
*Next*OBJECTTABLE
DMLCommands
419
OBJECTTABLETheOBJECTTABLEfunctionprocessesanobjectinputtoproducetabularoutput.Thefunctionitselfdefineswhatcolumnsitprojects.TheOBJECTTABLEfunctionisimplicitlyanestedtableandmaybecorrelatedtoprecedingFROMclauseentries.
Usage:
OBJECTTABLE([LANGUAGElang]rowScript[PASSINGvalASname...]COLUMNScolNamecolTypecolScript[DEFAULTdefa
ultExpr]...)ASid
Parameters
lang-anoptionalstringliteralthatisthecasesensitivelanguagenameofthescriptstobeprocessed.ThescriptenginemustbeavailableviaaJSR-223ScriptEngineManagerlookup.Insomeinstancesthismaymeanmakingadditionalmodulesavailabletoyourvdb,whichcanbedoneviathesameprocessasaddingmodules/librariesforUDFs.IfaLANGUAGEisnotspecified,thedefaultof'teiid_script'(seebelow)willbeused.
name-anidentifierthatwillbindthevalexpressionvalueintothescriptcontext.
rowScriptisastringliteralspecifyingthescripttocreatetherowvalues.foreachnon-nullitemtheIteratorproducesthecolumnswillbeevaluated.
colName/colTypearetheid/datatypeofthecolumn,whichcanoptionallybedefaultedwiththeDEFAULTclauseexpressiondefaultExpr.
colScriptisastringliteralspecifyingthescriptthatevaluatestothecolumnvalue.
SyntaxRules:
Thecolumnsnamesmustbenotcontainduplicates.
Teiidwillplaceseveralspecialvariablesinthescriptexecutioncontext.TheCommandContextisavailableasteiid_context.AdditionallythecolScriptsmayaccessteiid_rowandteiid_row_number.teiid_rowisthecurrentrowobjectproducedbytherowscript.teiid_row_numberisthecurrent1-basedrownumber.
rowScriptisevaluatedtoanIterator.IftheresultsisalreadyanIterator,itisuseddirectly.IftheevaluationresultisanIteratable,Array,orArraytype,thenanIteratorwillbeobtained.AnyotherObjectwillbetreatedasanIteratorofasingleitem).Inallcasesnullrowvalueswillbeskipped.
Note WhilethereisnorestrictionwhatcanbeusedasaPASSINGvariablenamesyoushouldchoosenamesthatcanbereferencedasidentifiersinthetargetlanguage.
Examples
Accessingspecialvariables:
SELECTx.*FROMOBJECTTABLE('teiid_context'COLUMNS"user"string'teiid_row.userName',row_numberinteger'tei
id_row_number')ASx
Theresultwouldbearowwithtwocolumnscontainingtheusernameand1respectively.
Note
DuetotheirmostlyunrestrictedaccesstoJavafunctionality,usageoflanguagesotherthanteiid_scriptisrestrictedbydefault.AVDBmustdeclareallallowablelanguagesbynameintheallowed-languagesVDBPropertiesusingacommaseparatedlist.Thenamesarecasesensitivenamesandshouldbeseparatedwithoutwhitespace.WithoutthispropertyitisnotpossibletouseOBJECTTABLEevenfromwithinviewdefinitionsthatarenotsubjecttonormalpermissionchecks.
DMLCommands
420
DatarolesarealsosecuredwithDataRolesusingthelanguagepermission.
teiid_script
teiid_scriptisasimplescriptingexpressionlanguagethatallowsaccesstopassingandspecialvariablesaswellasanynon-void0-argumentmethodsonobjectsandindexedvaluesonarrays/lists.Ateiid_scriptexpressionbeginsbyreferencingthepassingorspecialvariable.Thenanynumberof.'accessorsmaybechainedtoevaluatetheexpressiontoadifferentvalue.Methodsmaybeaccessedbytheirpropertynames,forexamplefooratherthangetFoo.Iftheobjectbotha`getFoo()
andfoo()method,thentheaccessorfooreferencesfo()andgetFooshouldbeusedtocallthegetter.Anarrayorlistindexmaybeaccessedusinga1-basedpositiveintegralvalue-usingthesame'.'accessorsyntax.Thesamelogicasthesystemfunctionarray_getisusedmeaningthatnullwillbereturnedratherthanexceptioniftheindexisoutofbounds.
teiid_scriptiseffectivelydynamicallytypedastypingisperformedatruntime.Ifaaccessordoesnotexistontheobjectorifthemethodisnotaccessible,thenanexceptionwillberaised.Ifatanypointintheaccessorchainevaluatestoanullvalue,thennullwillbereturned.
Examples
TogettheVDBdescriptionstring:
teiid_context.session.vdb.description
TogetthefirstcharacteroftheVDBdescriptionstring:
teiid_context.session.vdb.description.toCharArray.1
DMLCommands
421
TEXTTABLETheTEXTTABLEfunctionprocessescharacterinputtoproducetabularoutput.Itsupportsbothfixedanddelimitedfileformatparsing.Thefunctionitselfdefineswhatcolumnsitprojects.TheTEXTTABLEfunctionisimplicitlyanestedtableandmaybecorrelatedtoprecedingFROMclauseentries.
Usage:
TEXTTABLE(expression[SELECTORstring]COLUMNS<COLUMN>,...[NOROWDELIMITER|ROWDELIMITERchar][DELIMITER
char][(QUOTE|ESCAPE)char][HEADER[integer]][SKIPinteger][NOTRIM])ASname
Where<COLUMN>
COLUMN:=name(FORORDINALITY|([HEADERstring]datatype[WIDTHinteger[NOTRIM]][SELECTORstringinteger]))
Parameters
expression-thetextcontenttoprocess,whichshouldbeconvertibletoCLOB.
SELECTORisusedwithfilescontainingmultipletypesofrows(example:orderheader,detail,summary).ATEXTTABLESELECTORspecifieswhichlinestoincludeintheoutput.Matchinglinesmustbeginwiththeselectorstring.Theselectorincolumndelimitedfilesmustbefollowedbythecolumndelimiter.
IfaTEXTTABLESELECTORisspecified,aSELECTORmayalsobespecifiedforcolumnvalues.AcolumnSELECTORargumentwillselectthenearestprecedingtextlinewiththegivenSELECTORprefixandselectthevalueatthegiven1-basedintegerposition(whichincludestheselectoritself).Ifnosuchtextlineorpositionwithagivenlineexists,anullvaluewillbeproduced.AcolumnSELECTORisnotvalidwithfixedwidthparsing.
NOROWDELIMITERindicatesthatfixedparsingshouldnotassumethepresenceofnewlinerowdelimiters.
ROWDELIMITERsetstherowdelimiter/newlinetoanalternatecharacter.Defaultstothenewlinecharacter-withbuiltinhandlingfortreatingcarriagereturnnewlineasasinglecharacter.IfROWDELIMITERisspecified,carriagereturnwillbegivennospecialtreatment.
DELIMITERsetsthefielddelimitercharactertouse.Defaultsto','.
QUOTEsetsthequote,orqualifier,characterusedtowrapfieldvalues.Defaultsto'"'.
ESCAPEsetstheescapecharactertouseifnoquotingcharacterisinuse.Thisisusedinsituationswherethedelimiterornewlinecharactersareescapedwithaprecedingcharacter,e.g.\,
HEADERspecifiesthetextlinenumber(countingeverynewline)onwhichthecolumnnamesoccur.IftheHEADERoptionforacolumnisspecified,thenthatwillbeusedastheexpectedheadername.Alllinespriortotheheaderwillbeskipped.IfHEADERisspecified,thentheheaderlinewillbeusedtodeterminetheTEXTTABLEcolumnpositionbycase-insensitivenamematching.Thisisespeciallyusefulinsituationswhereonlyasubsetofthecolumnsareneeded.IftheHEADERvalueisnotspecified,itdefaultsto1.IfHEADERisnotspecified,thencolumnsareexpectedtomatchpositionallywiththetextcontents.
SKIPspecifiesthenumberoftextlines(countingeverynewline)toskipbeforeparsingthecontents.HEADERmaystillbespecifiedwithSKIP.
AFORORDINALITYcolumnistypedasintegerandwillreturnthe1-baseditemnumberasitsvalue.
DMLCommands
422
WIDTHindicatesthefixed-widthlengthofacolumnincharacters-notbytes.WiththedefaultROWDELIMITER,aCRNLsequencecountsasasinglecharacter.
NOTRIMspecifiedontheTEXTTABLE,itwillaffectallcolumnandheadervalues.IfNOTRIMisspecifiedonacolumn,thenthefixedorunqualifiedtextvaluenotbetrimmedofleadingandtrailingwhitespace.
SyntaxRules:
Ifwidthisspecifiedforonecolumnitmustbespecifiedforallcolumnsandbeanon-negativeinteger.
Ifwidthisspecified,thenfixedwidthparsingisusedESCAPE,QUOTE,columnSELECTOR,norHEADERshouldnotbespecified.
Ifwidthisnotspecified,thenNOROWDELIMITERcannotbeused.
Thecolumnsnamesmustnotcontainduplicates.
TheQUOTE,DELIMITER,andROWDELIMITERmustallbedifferentcharacters.
Examples
UseoftheHEADERparameter,returns1row['b']:
SELECT*FROMTEXTTABLE(UNESCAPE('col1,col2,col3\na,b,c')COLUMNScol2stringHEADER)x
Useoffixedwidth,returns2rows['a','b','c'],['d','e','f']:
SELECT*FROMTEXTTABLE(UNESCAPE('abc\ndef')COLUMNScol1stringwidth1,col2stringwidth1,col3stringwidt
h1)x
Useoffixedwidthwithoutarowdelimiter,returns3rows['a'],['b'],['c']:
SELECT*FROMTEXTTABLE('abc'COLUMNScol1stringwidth1NOROWDELIMITER)x
UseofESCAPEparameter,returns1row['a,','b']:
SELECT*FROMTEXTTABLE('a:,,b'COLUMNScol1string,col2stringESCAPE':')x
Asanestedtable:
SELECTx.*FROMt,TEXTTABLE(t.clobcolumnCOLUMNSfirststring,seconddateSKIP1)x
UseofSELECTORs,returns2rows['c','d','b'],['c','f','b']:
SELECT*FROMTEXTTABLE('a,b\nc,d\nc,f'SELECTOR'c'COLUMNScol1string,col2stringcol3stringSELECTOR'a'2
)x
DMLCommands
423
WHEREClauseTheWHEREclausedefinesthecriteriatolimittherecordsaffectedbySELECT,UPDATE,andDELETEstatements.
ThegeneralformoftheWHEREis:
WHERECriteria
DMLCommands
424
GROUPBYClauseTheGROUPBYclausedenotesthatrowsshouldbegroupedaccordingtothespecifiedexpressionvalues.Onerowwillbereturnedforeachgroup,afteroptionallyfilteringthoseaggregaterowsbasedonaHAVINGclause.
ThegeneralformoftheGROUPBYis:
GROUPBYexpression[,expression]*
GROUPBYROLLUP(expression[,expression]*)
SyntaxRules:
ColumnreferencesinthegroupbycannotbemadetoaliasnamesintheSELECTclause.
Expressionsusedinthegroupbymustappearintheselectclause.
ColumnreferencesandexpressionsintheSELECT/HAVING/ORDERBYclausesthatarenotusedinthegroupbyclausemustappearinaggregatefunctions.
IfanaggregatefunctionisusedintheSELECTclauseandnoGROUPBYisspecified,animplicitGROUPBYwillbeperformedwiththeentireresultsetasasinglegroup.Inthiscase,everycolumnintheSELECTmustbeanaggregatefunctionasnoothercolumnvaluewillbefixedacrosstheentiregroup.
Thegroupbycolumnsmustbeofacomparabletype.
Rollups
Justlikenormalgrouping,rollupprocessinglogicallyoccursbeforetheHAVINGclauseisprocessed.AROLLUPofexpressionswillproducethesameoutputasaregulargroupingwiththeadditionofaggregatevaluescomputedathigheraggregationlevels.ForNexpressionsintheROLLUP,aggregateswillbeprovidedover(),(expr1),(expr1,expr2),etc.upto(expr1,…exprN-1)withtheothergroupingexpressionsintheoutputasnullvalues.Forexamplewiththenormalaggregationquery
SELECTcountry,city,sum(amount)fromsalesgroupbycountry,city
returning:
country city sum(amount)
US St.Louis 10000
US Raleigh 150000
US Denver 20000
UK Birmingham 50000
UK London 75000
Therollupquery
DMLCommands
425
SELECTcountry,city,sum(amount)fromsalesgroupbyrollup(country,city)
wouldreturn:
country city sum(amount)
US St.Louis 10000
US Raleigh 150000
US Denver 20000
US <null> 180000
UK Birmingham 50000
UK London 75000
UK <null> 125000
<null> <null> 305000
Note NotallsourcessupportROLLUPsandsomeoptimizationscomparedtonormalaggregateprocessingmaybeinhibitedbytheuseofaROLLUP.
Teiid’ssupportforROLLUPismorelimitedthantheSQLspecification.InfuturereleasessupportforCUBE,groupingsets,andmorethanasingleextendedgroupingelementmaybesupported.
DMLCommands
426
HAVINGClauseTheHAVINGclauseoperatesexactlyasaWHEREclausealthoughitoperatesontheoutputofaGROUPBY.ItsupportsthesamesyntaxastheWHEREclause.
SyntaxRules:
Expressionsusedinthegroupbyclausemusteithercontainanaggregatefunction:COUNT,AVG,SUM,MIN,MAX.orbeoneofthegroupingexpressions.
DMLCommands
427
ORDERBYClauseTheORDERBYclausespecifieshowrecordsshouldbesorted.TheoptionsareASC(ascending)andDESC(descending).
Usage:
ORDERBYexpression[ASC|DESC][NULLS(FIRST|LAST)],...
SyntaxRules:
Sortcolumnsmaybespecifiedpositionallybya1-basedpositionalinteger,bySELECTclausealiasname,bySELECTclauseexpression,orbyanunrelatedexpression.
ColumnreferencesmayappearintheSELECTclauseastheexpressionforanaliasedcolumnormayreferencecolumnsfromtablesintheFROMclause.IfthecolumnreferenceisnotintheSELECTclausethequerymustnotbeasetoperation,specifySELECTDISTINCT,orcontainaGROUPBYclause.
Unrelatedexpressions,expressionsnotappearingasanaliasedexpressionintheselectclause,areallowedintheorderbyclauseofanon-setQUERY.Thecolumnsreferencedintheexpressionmustcomefromthefromclausetablereferences.Thecolumnreferencescannotbetoaliasnamesorpositional.
TheORDERBYcolumnsmustbeofacomparabletype.
IfanORDERBYisusedinaninlinevieworviewdefinitionwithoutalimitclause,itwillberemovedbytheTeiidoptimizer.
IfNULLSFIRST/LASTisspecified,thennullsareguaranteedtobesortedeitherfirstorlast.Ifthenullorderingisnotspecified,thenresultswilltypicallybesortedwithnullsaslowvalues,whichisTeiid’sinternaldefaultsortingbehavior.Howevernotallsourcesreturnresultswithnulsssortedaslowvaluesbydefault,andTeiidmayreturnresultswithdifferentnullorderings.
Warning TheuseofpositionalorderingisnolongersupportedbytheANSISQLstandardandisadeprecatedfeatureinTeiid.Itispreferabletousealiasnamesintheorderbyclause.
DMLCommands
428
LIMITClauseTheLIMITclausespecifiesalimitonthenumberofrecordsreturnedfromtheSELECTcommand.Anoptionaloffset(thenumberofrowstoskip)canbespecified.TheLIMITclausecanalsobespecifiedusingtheSQL2008OFFSET/FETCHFIRSTclauses.IfanORDERBYisalsospecified,itwillbeappliedbeforetheOFFSET/LIMITareapplied.IfanORDERBYisnotspecifiedthereisgenerallynoguaranteewhatsubsetofrowswillbereturned.
Usage:
LIMIT[offset,]limit
[OFFSEToffsetROW|ROWS][FETCHFIRST|NEXT[limit]ROW|ROWSONLY]
SyntaxRules:
Thelimit/offsetexpressionsmustbeanon-negativeintegeroraparameterreference(?).Anoffsetof0isignored.Alimitof0willreturnnorows.
ThetermsFIRST/NEXTareinterchangeableaswellasROW/ROWS.
ThelimitclausemaytakeanoptionalprecedingNON_STRICThinttoindicatethatpushoperationsshouldnotbeinhibitedeveniftheresultswillnotbeconsistentwiththelogicalapplicationofthelimit.Thehintisonlyneededonunorderedlimits,e.g."SELECT*FROMVW/*+NON_STRICT*/LIMIT2".
Examples:
LIMIT100-returnsthefirst100records(rows1-100)
LIMIT500,100-skips500recordsandreturnsthenext100records(rows501-600)
OFFSET500ROWS-skips500records
OFFSET500ROWSFETCHNEXT100ROWSONLY-skips500recordsandreturnsthenext100records(rows501-600)
FETCHFIRSTROWONLY-returnsonlythefirstrecord
DMLCommands
429
INTOClause
Warning UsageoftheINTOClauseforinsertingintoatablehasbeenbeendeprecated.AnINSERTwithaquerycommandshouldbeusedinstead.
WhentheintoclauseisspecifiedwithaSELECT,theresultsofthequeryareinsertedintothespecifiedtable.Thisisoftenusedtoinsertrecordsintoatemporarytable.TheINTOclauseimmediatelyprecedestheFROMclause.
Usage:
INTOtableFROM...
SyntaxRules:
TheINTOclauseislogicallyappliedlastinprocessing,aftertheORDERBYandLIMITclauses.
Teiid’ssupportforSELECTINTOissimilartoMSSQLServer.ThetargetoftheINTOclauseisatablewheretheresultoftherestselectcommandwillbeinserted.SELECTINTOshouldnotbeusedUNIONquery.
DMLCommands
430
OPTIONClauseTheOPTIONkeyworddenotesoptionstheusercanpassinwiththecommand.TheseoptionsareTeiidspecificandnotcoveredbyanySQLspecification.
Usage:
OPTIONoption(,option)*
Supportedoptions:
MAKEDEPtable(,table)*-specifiessourcetablesthatshouldbemadedependentinthejoin
MAKEINDtable(,table)*-specifiessourcetablesthatshouldbemadedependentinthejoin
MAKENOTDEPtable(,table)*-preventsadependentjoinfrombeingused
NOCACHE[table(,table)*]-preventscachefrombeingusedforalltablesorforthegiventables
Examples:
OPTIONMAKEDEPtable1
OPTIONNOCACHE
AlltablesspecifiedintheOPTIONclauseshouldbefullyqualified,howeverthenamemaymatcheitheranaliasnameorthefullyqualifiedname.
Themakedepandmakeindhintscantakeoptionalargumentstocontrolthedependentjoin.Theextendedhintformis:
MAKEDEPtbl([max:val][[no]join])
tbl(JOIN)meansthattheentirejoinshouldbepushed
tbl(NOJOIN)meansthattheentirejoinshouldnotbepushed
tbl(MAX:val)meaningthatthedependentjoinshouldonlybeperformediftherearelessthanthemaxnumberofvaluesfromtheindependentside.
Tip PreviousversionsofTeiidacceptedthePLANONLY,DEBUG,andSHOWPLANoptionarguments.ThesearenolongeracceptedintheOPTIONclause.PleaseseetheClientDevelopersGuideforreplacementstothoseoptions.
NoteMAKEDEPandMAKENOTDEPhintsmaytaketablenamesintheformof@view1.view2…table.Forexamplewithaninlineview"select*from(select*fromtbl1,tbl2wheretbl1.c1=tbl2.c2)asv1optionmakedep@v1.tbl1"thehintwillnowbeunderstoodasapplyingunderthev1view.
DMLCommands
431
DDLCommandsTeiidsupportsasubsetofDDLatruntimetocreate/droptemporarytablesandtomanipulateprocedureandviewdefinitions.Itisnotcurrentlypossibletoarbitrarilydrop/createnon-temporarymetadataentries.SeeDDLMetadataforDDLusedwithinaVDBtodefineschemas.
Note AMetadataRepositorymustbeconfiguredtomakeanon-temporarymetadataupdatepersistent.SeetheDevelopersGuideRuntimeMetadataUpdatessectionformore.
DDLCommands
432
TempTablesTeiidsupportscreatingtemporary(or"temp")tables.Temptablesaredynamicallycreated,butaretreatedasanyotherphysicaltable.
TableofContentsLocalTemporaryTables
ExampleGlobalTemporaryTablesGlobalandLocalTemporaryTableFeaturesForeignTemporaryTables
LocalTemporaryTables
LocaltemporarytablescanbedefinedimplicitlybyreferencingtheminaINSERTstatementorexplicitlywithaCREATETABLEstatement.Implicitlycreatedtemptablesmusthaveanamethatstartswith#.
ExplicitCreationsyntax
CREATELOCALTEMPORARYTABLEname(columntype[NOTNULL],...[PRIMARYKEY(column,...)])[ONCOMMITPRESERVE
ROWS]
UsetheSERIALdatatypetospecifyaNOTNULLandauto-incrementingINTEGERcolumn.ThestartingvalueofaSERIALcolumnis1.
ImplicitCreationsyntax
INSERTINTO#name(column,...)VALUES(value,...)
INSERTINTO#name[(column,...)]selectc1,c2fromt
If#xdoesn’texist,itwillbedefinedusingthegivencolumnnamesandtypesfromthevalueexpressions,orthetargetcolumnnames(innotsupplied,thecolumnnameswillmatchthederivedcolumnnamesfromthequery),andthetypesfromthequeryderivedcolumns.
Note
Teiid’sinterpretationoflocalisdifferentthantheSQLspecificationandotherdatabasevendors.Localmeansthatthescopeoftemptablewillbeeithertothesessionortheblockofavirtualprocedurethatcreatesit.Uponexitingtheblockortheterminationofthesessionthetableisdropped.Sessionandanyothertemporarytablescreatedincallingproceduresarenotvisibletocalledprocedures.Ifatemporarytableofthesamenameiscreatedinacalledprocedureanewinstanceiscreated.
Dropsyntax
DROPTABLEname
Example
Thefollowingexampleisaseriesofstatementsthatloadsatemporarytablewithdatafrom2sources,andwithamanuallyinsertedrecord,andthenusesthattemptableinasubsequentquery.
CREATELOCALTEMPORARYTABLETEMP(ainteger,binteger,cinteger);
SELECT*INTOtempFROMSrc1;
SELECT*INTOtempFROMSrc2;
INSERTINTOtempVALUES(1,2,3);
DDLCommands
433
SELECTa,b,cFROMSrc3,tempWHERESrc3.a=temp.b;
SeeVirtualProceduresformoreonlocaltemporarytableusage.
GlobalTemporaryTables
GlobaltemporarytablesarecreatedinTeiidDesignerorviathemetadatasuppliedtoTeiidatdeploytime.Unlikelocaltemporarytables,theycannotbecreatedatruntime.Aglobaltemporarytablesshareacommondefinitionviaaschemaentry,buteachsessionhasanewinstanceofthetemporarytablecreateduponit’sfirstuse.Thetableisthendroppedwhenthesessionends.Thereisnoexplicitdropsupport.Acommonuseforaglobaltemporarytableistopassresultsintoandoutofprocedures.
Creationsyntax
CREATEGLOBALTEMPORARYTABLEname(columntype[NOTNULL],...[PRIMARYKEY(column,...)])OPTIONS(UPDATABLE
'true')
IftheSERIALdatatypeisused,theneachsession’sinstanceoftheglobaltemporarytablewilluseit’sownsequence.
SeetheCREATETABLEDDLstatementforallsyntaxoptions.
CurrentlyUPDATABLEmustbeexplicitlyspecifiedforthetemporarytabletobeupdated.
GlobalandLocalTemporaryTableFeaturesPrimaryKeySupport:
Allkeycolumnsmustbecomparable.
Useofaprimarykeycreatesaclusteredindexthatsupportssearchimprovementsforcomparison,in,like,andorderby.
Nullisanallowableprimarykeyvalue,buttheremustbeonly1rowthathasanallnullkey.
TransactionSupport:
TemptablessupportaREAD_UNCOMMITEDtransactionisolationlevel.Therearenolockingmechanismsavailabletosupporthigherisolationlevelsandtheresultofarollbackmaybeinconsistentacrossmultipletransactions.Ifconcurrenttransactionsarenotassociatedwiththesamelocaltemporarytableorsession,thenthetransactionisolationleveliseffectivelySERIALIZABLE.Ifyouwantfullconsistencywithlocaltemporarytables,thenonlyuseaconnectionwith1transactionatatime.Thismodeofoperationisensuredbyconnectionpoolingthattracksconnectionsbytransaction.
Limitations:
WiththeCREATETABLEsyntaxonlybasictabledefinition(columnname,type,andnullableinformation)andanoptionalprimarykeyaresupported.Forglobaltemporarytablesadditionalmetadatainthecreatestatementiseffectivelyignoredwhencreatingthetemporarytableinstance-butmaystillbeutilizedbyplanningsimilartoanyothertableentry.
SimilartoPostgreSQL,TeiiddefaultstoONCOMMITPRESERVEROWS.NootherONCOMMITactionissupportedatthistime.
The"dropbehavior"optionisnotsupportedinthedropstatement.
Temptablesarenotfail-oversafe.
Non-inlinedlobvalues(xml,clob,blob)aretrackedbyreferenceratherthanbyvalueinatemporarytable.Lobvaluesfromexternalsourcesthatareinsertedinatemporarytablemaybecomeunreadablewhentheassociatedstatementorconnectionisclosed.
DDLCommands
434
ForeignTemporaryTables
UnlikeTeiidlocalorglobaltemporarytables,aforeigntemporarytableisareferencetoasourcetablethatiscreatedatruntimeratherthanduringthemetadataload.
Aforeigntemporarytablerequiresexplicitcreationsyntax:
CREATEFOREIGNTEMPORARYTABLEname...ONschema
WherethetablecreationbodysyntaxisthesameasastandardCREATEFOREIGNTABLEDDLstatement.Ingeneral,usageofDDLOPTIONclausesmayberequiredtoproperlyaccessthesourcetable,includingsettingthenameinsource,updatability,nativetypes,etc.
Theschemanamemustspecifyanexistingschema/modelintheVDB.Thetablewillbeaccessedasifitisonthatsource,howeverwithinTeiidthetemporarytablewillstillbescopedthesameasanon-foreigntemporarytable.ThismeansthattheforeigntemporarytablewillnotbelongtoaTeiidschemaandwillbescopedtothesessionorprocedureblockwherecreated.
TheDROPsyntaxforaforeigntemporarytableisthesameasforanon-foreigntemporarytable.
Note NeitheraCREATEnoracorrespondingDROPofaforeigntemporarytableissueapushdowncommand,ratherthismechanismsimplyexposesasourcetableforusewithinTeiidonatemporarybasis.
TherearetwousagescenariosforaFOREIGNTEMPORARYTABLE.Thefirstistodynamicallyaccessadditionaltablesonthesource.TheotheristoreplacetheusageofaTeiidlocaltemporarytableforperformancereasons.Theusagepatternforthelattercasewouldlooklike:
//-createthesourcetable
source.native("CREATEGLOBALTEMPORARYTABLEnameIFNOTEXISTS...ONCOMMITDELETEROWS");
//-bringthetableintoTeiid
CREATEFOREIGNTEMPORARYTABLEname...OPTIONS(UPDATABLEtrue)
//-usethetable
...
//-forgetthetable
DROPTABLEname
NotetheusageofthenativeproceduretopasssourcespecificCREATEddltothesource.TeiiddoesnotcurrentlyattempttopushdownasourcecreationofatemporarytablebasedupontheCREATEstatement.Someothermechanism,suchasthenativeprocedureshownabove,mustbeusedtofirstcreatethetable.Alsonotethetableisexplicitlymarkedasupdatable,sinceDDLdefinedtablesarenotupdatablebydefault.
Thesource’shandlingoftemporarytablesmustalsobeunderstoodtomakethisworkasintended.SourcesthatusethesameGLOBALtabledefinitionforallsessionswhilescopingthedatatobesessionspecific(suchasOracle)orsourcesthatsupportsessionscopedtemporarytables(suchasPostgreSQL)willworkifaccessedunderatransaction.Atransactionisnecessarybecause:
thesourceoncommitbehavior(mostlikelyDELETEROWSorDROP)willensureclean-up.KeepinmindthataTeiiddropdoesnotissueasourcecommandandisnotguaranteedtooccur(insomeexceptioncases,lossofdbconnectivity,hardshutdown,etc.).
thesourcepoolwhenusingtrackconnectionsbytransactionwillensurethatmultipleusesofthatsourcebyTeiidwillusethesameconnection/sessionandthusthesametemporarytableanddata.
Tip SinceTeiiddoesnotyetsupporttheONCOMMITclauseit’simportanttoconsiderthatthesourcetableONCOMMITbehaviorwilllikelybedifferentthatthedefault,PRESERVEROWS,forTeiidlocaltemporarytables.
DDLCommands
435
DDLCommands
436
AlterViewUsage:
ALTERVIEWnameASqueryExpression
SyntaxRules:
Thealterqueryexpressionmaybeprefixedwithacachehintformaterializedviewdefinitions.Thehintwilltakeeffectthenexttimethematerializedviewtableisloaded.
DDLCommands
437
AlterProcedureUsage:
ALTERPROCEDUREnameASblock
SyntaxRules:
ThealterblockshouldnotincludeCREATEVIRTUALPROCEDURE
Thealterblockmaybeprefixedwithacachehintforcachedprocedures.
DDLCommands
438
AlterTriggerUsage:
ALTERTRIGGERONnameINSTEADOFINSERT|UPDATE|DELETE(ASFOREACHROWblock)|(ENABLED|DISABLED)
SyntaxRules:
Thetarget,name,mustbeanupdatableview.
Triggersarenotyettrueschemaobjects.Theyarescopedonlytotheirviewandhavenoname.
AnUpdateProcedures(Triggers)mustalreadyexistforthegiventriggerevent.
Note IfthedefaultinherentupdateischoseninTeiidDesigner,anySQLassociatedwithupdate(showninagreyedouttextbox)isnotpartoftheVDBandcannotbeenabledwithanaltertriggerstatement.
DDLCommands
439
XMLSELECTCommandNOTE:XMLDocumentsModelsaredeprecated.YoushoulduseSQL/XMLfunctionsorODataaccesstoformcomplexdocumentsinstead.
ComplexXMLdocumentscanbedynamicallyconstructedbyTeiidusingXMLDocumentModels.Adocumentmodelisgenerallycreatedfromaschema.ThedocumentmodelisboundtorelevantSQLstatementsthroughmappingclasses.SeetheDesignerguideformoreoncreatingdocumentmodels.
XMLdocumentsmayalsocreatedviaXQuerywiththeXMLQueryfunctionorwithvariousotherSQL/XMLfunctions.
QueryingXMLdocumentsissimilartoqueryingrelationaltables.AnidiomaticSQLvariantwithspecialscalarfunctionsgivescontroloverwhichpartsofagivendocumenttoreturn.
XMLSELECTCommand
440
QueryStructureAvalidXMLSELECTCommandagainstadocumentmodelisoftheformSELECT…FROM…WHERE…ORDERBY….TheuseofanyotherSELECTcommandclauseisnotallowed.
ThefullyqualifiednameforanXMLelementis:
"model"."documentname".[pathtoelement]."elementname"
.
Thefullyqualifiednameforanattributeis:
"model"."documentname".[pathtoelement]."elementname".[@]"attributename"
Partiallyqualifiednamesforelementsandattributescanbeusedaslongasthepartialnameisunique.
FROMClause
Specifiesthedocumenttogenerate.Documentnamesresembleothervirtualgroups-"model"."documentname".
SyntaxRules:
Thefrommayonlycontainoneunaryclausespecifyingthedesireddocument.
SELECTClause
TheselectclausedetermineswhichpartsoftheXMLdocumentaregeneratedforoutput.
ExampleSyntax:
select*frommodel.doc
selectmodel.doc.root.parent.element.*frommodel.doc
selectelement,element1.@attributefrommodel.doc
SyntaxRules:
SELECT*andSELECT"xml"areequivalentandspecifythateveryelementandattributeofthedocumentshouldbeoutput.
TheSELECTclauseofanXMLQuerymayonlycontain*,"xml",orelementandattributereferencesfromthespecifieddocument.Anyotherexpressionsarenotallowed.
IftheSELECTclausecontainsanelementorattributereference(otherthan*or"xml")thenonlythespecifiedelements,attributes,andtheirancestorelementswillbeinthegenerateddocument.
element.*specifiesthattheelement,it’sattribute,andallchildcontentshouldbeoutput.
WHEREClause
XMLSELECTCommand
441
Thewhereclausespecifieshowtofiltercontentfromthegenerateddocumentbaseduponvaluescontainedintheunderlyingmappingclasses.MostpredicatesarevalidinanXMLSELECTCommand,howevercombiningvaluereferencesfromdifferentpartsofthedocumentmaynotalwaysbeallowed.
Criteriaislogicallyappliedtoacontextwhichisdirectlyrelatedtoamappingclass.Startingwiththerootmappingclass,thereisarootcontextthatdescribesallofthetoplevelrepeatedelementsthatwillbeintheoutputdocument.Criteriaappliedtotherootoranyothercontextwillchangetherelatedmappingclassquerytoapplytheaffectsofthecriteria,whichcanincludecheckingvaluesfromanyofthedescendantmappingclasses.
ExampleSyntax:
selectelement,element1.@attributefrommodel.docwhereelement1.@attribute=1
selectelement,element1.@attributefrommodel.docwherecontext(element1,element1.@attribute)=1
SyntaxRules:
Eachcriteriaconjunctmustrefertoasinglecontextandcanbecriteriathatappliestoamappingclass,containarowlimitfunction,orcontainrowlimitexceptionfunction.
Criteriathatappliestoamappingclassisassociatedtothatmappingclassviathecontextfunction.Theabsenceofacontextfunctionimpliesthecriteriaappliestotherootcontext.
Atagivencontextthecriteriacanspanmultiplemappingclassesprovidedthatallmappingclassesinvolvedareeitherparentsofthecontext,thecontextitself,oradescendantofthecontext.
NoteSiblingRootMappingClasses-Impliedrootcontextusercriteriaagainstadocumentmodelwithsiblingrootmappingclassesisnotgenerallysemanticallycorrect.Itisappliedasifeachoftheconjunctsisappliedtoonlyasinglerootmappingclass.Thisbehavioristhesameaspriorreleasesbutmaybefixedinafuturerelease.
XMLSELECTCommandSpecificFunctions
XMLSELECTCommandfunctionsareresemblescalarfunctions,butactashintsintheWHEREclause.ThesefunctionsareonlyvalidinanXMLSELECTCommand.
ContextFunction
CONTEXT(arg1,arg2)
Selectthecontextforthecontainingconjunct.
SyntaxRules:
Contextfunctionsapplytothewholeconjunct.
Thefirstargumentmustbeanelementorattributereferencefromthemappingclasswhosecontextthecriteriaconjunctwillapplyto.
Thesecondparameteristhereturnvalueforthefunction.
RowlimitFunction
ROWLIMIT(arg)
XMLSELECTCommand
442
Limitstherowsprocessedforthegivencontext.
SyntaxRules:
Thefirstargumentmustbeanelementorattributereferencefromthemappingclasswhosecontexttherowlimitapplies.
Therowlimitfunctionmustbeusedinequalitycomparisoncriteriawiththerighthandexpressionequaltoanpositiveintegernumberorrowstolimit.
Onlyonerowlimitorrowlimitexceptionmayapplytoagivencontext.
RowlimitexceptionFunction
Limitstherowsprocessedforthegivencontextandthrowsanexceptionifthegivennumberofrowsisexceeded.
ROWLIMITEXCEPTION(arg)
SyntaxRules:
Thefirstargumentmustbeanelementorattributereferencefromthemappingclasswhosecontexttherowlimitexceptionapplies.
Therowlimitexceptionfunctionmustbeusedinequalitycomparisoncriteriawiththerighthandexpressionequaltoanpositiveintegernumberorrowstolimit.
Onlyonerowlimitorrowlimitexceptionmayapplytoagivencontext.
ORDERBYClauseTheXMLSELECTCommandORDERBYClausespecifiesorderingforthereferencedmappingclassqueries.
SyntaxRules:
Eachorderbyitemmustbeanelementorattributereferencetiedaoutputvaluefromamappingclass.
Theorderortheorderbyitemsistherelativeordertheywillbeappliedtotheirrespectivemappingclasses.
XMLSELECTCommand
443
DocumentGenerationDocumentgenerationstartswiththerootmappingclassandproceedsiterativelyandhierarchicallyoverallofthechildmappingclasses.Thiscanresultinalargenumberofqueryexecutions.Forexampleifadocumenthasarootmappingclasswith3childmappingclasses.Thenforeachrowselectedbytherootmappingclassaftertheapplicationoftherootcontextcriteria,eachofthechildmappingclassesquerieswillalsobeexecuted.
Note
DocumentCorrectness-BydefaultXMLgeneratedbyXMLdocumentsarenotcheckedforcorrectnessvs.therelevantschema.Itispossiblethatthemappingclassqueries,theusageofspecificSELECTorWHEREclausevalueswillgeneratedadocumentthatisnotvalidwithrespecttotheschema.SeeDocumentValidationonhowtoensurecorrectness.
Siblingorcousinelementsdefinedbythesamemappingclassthatdonothaveacommonparentinthatmappingclasswillbetreatedasindependentmappingclassesduringplanningandexecution.Thisallowsforamoredocumentcentricapproachtoapplyingcriteriaandorderbystomappingclasses.
DocumentValidationTheexecutionpropertyXMLValidationshouldbesetto'true'toindicatethatgenerateddocumentsshouldbecheckedforcorrectness.Correctnesscheckingwillnotpreventinvaliddocumentsfrombeinggenerated,sincecorrectnessischeckedaftergenerationandnotduring.
XMLSELECTCommand
444
Procedures
Procedures
445
ProcedureLanguageTeiidsupportsaprocedurallanguagefordefiningVirtualProcedures.Thesearesimilartostoredproceduresinrelationaldatabasemanagementsystems.YoucanusethislanguagetodefinethetransformationlogicfordecomposingINSERT,UPDATE,andDELETEcommandsagainstviews;theseareknownasUpdateProcedures(Triggers).
TableofContentsCommandStatementDynamicSQLCommandDeclarationStatementAssignmentStatementSpecialVariablesCompoundStatement
ExceptionHandlingIfStatementLoopStatementWhileStatementContinueStatementBreakStatementLeaveStatementReturnStatementErrorStatementRaiseStatementExceptionExpression
CommandStatement
AcommandstatementexecutesaDMLCommand,suchasSELECT,INSERT,UPDATE,DELETE,EXECUTE,oraDDLstatement,dynamicSQL,etc.
Usage:
command[(WITH|WITHOUT)RETURN];
ExampleCommandStatements
SELECT*FROMMySchema.MyTableWHEREColA>100WITHOUTRETURN;
INSERTINTOMySchema.MyTable(ColA,ColB)VALUES(50,'hi');
SyntaxRules:
EXECUTEcommandstatementsmayaccessIN/OUT,OUT,andRETURNparameters.Toaccessthereturnvaluethestatementwillhavetheformvar=EXECproc….ToaccessOUTorIN/OUTvaluesnamedparametersyntaxmustbeused.Forexample,EXECproc(in_param⇒'1',out_param⇒var)willassignthevalueoftheoutparametertothevariablevar.Itisexpectedthatthedatatypeofparameterwillbeimplicitlyconvertabletothedatatypeofthevariable.
TheRETURNclausedeterminesiftheresultofthecommandisreturnablefromtheprocedure.WITHRETURNisthedefault.Ifthecommanddoesnotreturnaresultsetortheproceduredoesnotreturnaresultset,theRETURNclauseisignored.IfWITHRETURNisspecified,theresultsetofthecommandmustmatchtheexpectedresultsetoftheprocedure.OnlythelastsuccessfullyexecutedstatementexecutedWITHRETURNwillbereturnedastheprocedureresultset.Iftherearenoreturnableresultsetsandtheproceduredeclaresthataresultsetwillbereturned,thenanemptyresultsetisreturned.
Procedures
446
DynamicSQLCommand
DynamicSQLallowsfortheexecutionofanarbitrarySQLcommandinavirtualprocedure.DynamicSQLisusefulinsituationswheretheexactcommandformisnotknownpriortoexecution.
Usage:
EXECUTEIMMEDIATE<sqlexpression>AS<variable><type>[,<variable><type>]*[INTO<variable>][USING<variab
le>=<expression>[,<variable>=<expression>]*][UPDATE<literal>]
SyntaxRules:
Thesqlexpressionmustbeaclob/stringvaluelessthan262144characters.
The"AS"clauseisusedtodefinetheprojectedsymbolsnamesandtypesreturnedbytheexecutedSQLstring.The"AS"clausesymbolswillbematchedpositionallywiththesymbolsreturnedbytheexecutedSQLstring.Non-convertibletypesortoofewcolumnsreturnedbytheexecutedSQLstringwillresultinanerror.
The"INTO"clausewillprojectthedynamicSQLintothespecifiedtemptable.Withthe"INTO"clausespecified,thedynamiccommandwillactuallyexecuteastatementthatbehaveslikeanINSERTwithaQUERYEXPRESSION.IfthedynamicSQLcommandcreatesatemporarytablewiththe"INTO"clause,thenthe"AS"clauseisrequiredtodefinethetable’smetadata.
The"USING"clauseallowsthedynamicSQLstringtocontainvariablereferencesthatareboundatruntimetospecifiedvalues.ThisallowsforsomeindependenceoftheSQLstringfromthesurroundingprocedurevariablenamesandinputnames.Inthedynamiccommand"USING"clause,eachvariableisspecifiedbyshortnameonly.HoweverinthedynamicSQLthe"USING"variablemustbefullyqualifiedto"DVAR.".The"USING"clauseisonlyforvaluesthatwillbeusedinthedynamicSQLaslegalexpressions.Itisnotpossibletousethe"USING"clausetoreplacetablenames,keywords,etc.Thismakesusingsymbolsequivalentinpowertonormalbind(?)expressionsinpreparedstatements.The"USING"clausehelpsreducetheamountofstringmanipulationneeded.IfareferenceismadetoaUSINGsymbolintheSQLstringthatisnotboundtoavalueinthe"USING"clause,anexceptionwilloccur.
The"UPDATE"clauseisusedtospecifytheUpdatingModelCount.Acceptedvaluesare(0,1,*).0isthedefaultvalueiftheclauseisnotspecified.
ExampleDynamicSQL
...
/*Typicallycomplexcriteriawouldbeformedbaseduponinputstotheprocedure.
Inthissimpleexamplethecriteriaisreferencestheusingclausetoisolate
theSQLstringfromreferencingavaluefromtheproceduredirectly*/
DECLAREstringcriteria='Customer.Accounts.Last=DVARS.LastName';
/*NowwecreatethedesiredSQLstring*/
DECLAREstringsql_string='SELECTID,First||""||LastASName,BirthdateFROMCustomer.AccountsWHERE'
||criteria;
/*TheexecutionoftheSQLstringwillcreatethe#temptablewiththecolumns(ID,Name,Birthdate).
NotethatwealsohavetheUSINGclausetobindavaluetoLastName,whichisreferencedinthecriteria.*/
EXECUTEIMMEDIATEsql_stringASIDinteger,Namestring,BirthdatedateINTO#tempUSINGLastName='somename';
/*ThetemptablecannowbeusedwiththevaluesfromtheDynamicSQL*/
loopon(SELCTIDfrom#temp)asmyCursor
...
HereisanexampleshowingamorecomplexapproachtobuildingcriteriaforthedynamicSQLstring.Inshort,thevirtualprocedureAccountAccess.GetAccountshasinputsID,LastName,andbday.IfavalueisspecifiedforIDitwillbetheonlyvalueusedinthedynamicSQLcriteria.OtherwiseifavalueisspecifiedforLastNametheprocedurewilldetectifthevalueisasearch
Procedures
447
string.IfbdayisspecifiedinadditiontoLastName,itwillbeusedtoformcompoundcriteriawithLastName.
ExampleDynamicSQLwithUSINGclauseanddynamicallybuiltcriteriastring
...
DECLAREstringcrit=null;
IF(AccountAccess.GetAccounts.IDISNOTNULL)
crit='(Customer.Accounts.ID=DVARS.ID)';
ELSEIF(AccountAccess.GetAccounts.LastNameISNOTNULL)
BEGIN
IF(AccountAccess.GetAccounts.LastName=='%')
ERROR"Lastnamecannotbe%";
ELSEIF(LOCATE('%',AccountAccess.GetAccounts.LastName)<0)
crit='(Customer.Accounts.Last=DVARS.LastName)';
ELSE
crit='(Customer.Accounts.LastLIKEDVARS.LastName)';
IF(AccountAccess.GetAccounts.bdayISNOTNULL)
crit='('||crit||'and(Customer.Accounts.Birthdate=DVARS.BirthDay))';
END
ELSE
ERROR"IDorLastNamemustbespecified.";
EXECUTEIMMEDIATE'SELECTID,First||""||LastASName,BirthdateFROMCustomer.AccountsWHERE'||critUS
INGID=AccountAccess.GetAccounts.ID,LastName=AccountAccess.GetAccounts.LastName,BirthDay=AccountAccess.GetAcc
ounts.Bday;
...
KnownLimitationsandWork-Arounds
TheuseofdynamicSQLcommandresultsinanassignmentstatementrequirestheuseofatemptable.
ExampleAssignment
EXECUTEIMMEDIATE<expression>ASxstringINTO#temp;
DECLAREstringVARIABLES.RESULT=(SELECTxFROM#temp);
Theconstructionofappropriatecriteriawillbecumbersomeifpartsofthecriteriaarenotpresent.Forexampleif"criteria"werealreadyNULL,thenthefollowingexampleresultsin"criteria"remainingNULL.
ExampleDangerousNULLhandling
...
criteria='('||criteria||'and(Customer.Accounts.Birthdate=DVARS.BirthDay))';
ThepreferredapproachisfortheusertoensurethecriteriaisnotNULLprioritsusage.Ifthisisnotpossible,agoodapproachistospecifyadefaultasshowninthefollowingexample.
ExampleNULLhandling
...
criteria='('||nvl(criteria,'(1=1)')||'and(Customer.Accounts.Birthdate=DVARS.BirthDay))';
IfthedynamicSQLisanUPDATE,DELETE,orINSERTcommand,therowcountofthestatementcanbeobtainedfromtherowcountvariable.
ExamplewithASandINTOclauses
/*Executeanupdate*/
EXECUTEIMMEDIATE<expression>;
Unlessusedinotherpartsoftheprocedure,tablesinthedynamiccommandwillnotbeseenassourcesintheDesigner.
Procedures
448
Whenusingthe"AS"clauseonlythetypeinformationwillbeavailabletotheDesigner.ResultSetcolumnsgeneratedfromthe"AS"clausethenwillhaveadefaultsetofpropertiesforlength,precision,etc.
DeclarationStatement
Adeclarationstatementdeclaresavariableanditstype.Afteryoudeclareavariable,youcanuseitinthatblockwithintheprocedureandanysub-blocks.Avariableisinitializedtonullbydefault,butcanalsobeassignedthevalueofanexpressionaspartofthedeclarationstatement.
Usage:
DECLARE<type>[VARIABLES.]<name>[=<expression>];
ExampleSyntax
declareintegerx;
declarestringVARIABLES.myvar='value';
SyntaxRules:
Youcannotredeclareavariablewithaduplicatenameinasub-block
TheVARIABLESgroupisalwaysimpliedevenifitisnotspecified.
TheassignmentvaluefollowsthesamerulesasforanAssignmentStatement.
Inadditiontothestandardtypes,youmayspecifyEXCEPTIONifdeclaringanexceptionvariable.
AssignmentStatement
Anassignmentstatementassignsavaluetoavariablebyevaluatinganexpression.
Usage:
<variablereference>=<expression>;
ExampleSyntax
myString='Thankyou';
VARIABLES.x=(SELECTColumn1FROMMySchema.MyTable);
Validvariablesforassignmentincludeanyinscopevariablethathasbeendeclaredwithadeclarationstatement,ortheprocedurein_outandoutparameters.In_outandoutparameterscanbeaccessedastheirfullyqualifiedname.
ExampleOutParameter
CREATEVIRTUALPROCEDUREproc(OUTSTRINGx,INOUTSTRINGy)AS
BEGIN
proc.x='somevalue'||proc.y;
y='somenewvalue';
END
SpecialVariables
Procedures
449
VARIABLES.ROWCOUNTintegervariablewillcontainthenumbersofrowsaffectedbythelastinsert/update/deletecommandstatementexecuted.InsertsthatareprocessedbydynamicsqlwithanintoclausewillalsoupdatetheROWCOUNT.
Usage:
SampleUsage
...
UPDATEFOOSETX=1WHEREY=2;
DECLAREINTEGERUPDATED=VARIABLES.ROWCOUNT;
...
Non-updatecommandstatements(WITHorWITHOUTRETURN)willresettheROWCOUNTto0.
Note ToensureyouaregettingtheappropriateROWCOUNTvalue,savetheROWCOUNTtoavariableimmediatelyafterthecommandstatement.
CompoundStatement
Acompoundstatementorblocklogicallygroupsaseriesofstatements.Temporarytablesandvariablescreatedinacompoundstatementarelocalonlytothatblockaredestroyedwhenexitingtheblock.
Usage:
[label:]BEGIN[[NOT]ATOMIC]
statement*
[EXCEPTIONex
statement*
]
END
Note WhenablockisexpectedbyaIF,LOOP,WHILE,etc.asinglestatementisalsoacceptedbytheparser.EventhoughtheblockBEGIN/ENDarenotexpected,thestatementwillexecuteasifwrappedinaBEGIN/ENDpair.
SyntaxRules
IFNOTATOMICornoATOMICclauseisspecified,theblockwillbeexecutednon-atomically.
IFATOMICtheblockmustexecuteatomically.Ifatransactionisalreadyassociatedwiththethread,noadditionalactionwillbetaken-savepointsand/orsub-transactionsarenotcurrentlyused.Ifthehigherleveltransactionisusedandtheblockdoesnotcomplete-regardlessofthepresenceofexceptionhandlingthetransactionwillbemarkedasrollbackonly.Otherwiseatransactionwillbeassociatedwiththeexecutionoftheblock.Uponsuccessfulcompletionoftheblockthetransactionwillbecommitted.
Thelabelmustnotbethesameasanyotherlabelusedinstatementscontainingthisone.
Variableassignmentsandtheimplicitresultcursorareunaffectedbyrollbacks.Ifablockdoesnotcompletesuccessfullyitsassignmentswillstilltakeaffect.
ExceptionHandling
IftheEXCEPTIONclauseisusedwithinacompoundstatement,anyprocessingexceptionemittedfromstatementswillbecaughtwiththeflowofexecutiontransferringtoEXCEPTIONstatements.Anyblockleveltransactionstartedbythisblockwillcommitiftheexceptionhandlersuccessfullycompletes.Ifanotherexceptionortheoriginalexceptionisemittedfromtheexceptionhandlerthetransactionwillrollback.AnytemporarytablesorvariablesspecifictotheBLOCKwillnotbeavailabletotheexceptionhandlerstatements.
Procedures
450
Note Onlyprocessingexceptions,whicharetypicallycausedbyerrorsoriginatingatthesourcesorwithfunctionexecution,arecaught.Alow-levelinternalTeiiderrororJavaRuntimeExceptionwillnotbecaught.
ToaidintheprocessingofacaughtexceptiontheEXCEPTIONclausespecifiesagroupnamethatexposesthesignificantfieldsoftheexception.Theexceptiongroupwillcontain:
Variable Type Description
STATE string TheSQLState
ERRORCODE integer
Theerrororvendorcode.InthecaseofTeiidinternalexceptionsthiswillbetheintegersuffixoftheTEIIDxxxxcode
TEIIDCODE string ThefullTeiideventcode.TypicallyTEIIDxxxx.
EXCEPTION object Theexceptionbeingcaught,willbeaninstanceofTeiidSQLException
CHAIN object Thechainedexceptionorcauseofthecurrentexception
Note TeiiddoesnotyetfullycomplywiththeANSISQLspecificationonSQLStateusage.ForTeiiderrorswithoutanunderlyingSQLExceptioncause,itisbesttousetheTeiidcode.
Theexceptiongroupnamemaynotbethesameasanyhigherlevelexceptiongrouporloopcursorname.
ExampleExceptionGroupHandling
BEGIN
DECLAREEXCEPTIONe=SQLEXCEPTION'thisisbad'SQLSTATE'xxxxx';
RAISEvariables.e;
EXCEPTIONe
IF(e.state='xxxxx')
//inthistrivialexample,we'llalwayshitthisbranchandjustlogtheexception
RAISESQLWARNINGe.exception;
ELSE
RAISEe.exception;
END
IfStatementAnIFstatementevaluatesaconditionandexecuteseitheroneoftwostatementsdependingontheresult.YoucannestIFstatementstocreatecomplexbranchinglogic.AdependentELSEstatementwillexecuteitsstatementonlyiftheIFstatementevaluatestofalse.
Usage:
IF(criteria)
block
[ELSE
block]
END
ExampleIfStatement
Procedures
451
IF(var1='NorthAmerica')
BEGIN
...statement...
ENDELSE
BEGIN
...statement...
END
ThecriteriamaybeanyvalidbooleanexpressionoranISDISTINCTFROMpredicate.ISDISTINCTFROMusesthesyntax:
rowValIS[NOT]DISTINCTFROMrowValOther
WhererowValandrowValOtherarereferencestorowvaluegroup.Thiswouldtypicallybeusedininsteadofupdatetriggersonviewstoquicklydetermineiftherowvaluesarechanging:
ExampleISDISTINCTFROMIfStatement
IF("new"ISDISTINCTFROM"old")
BEGIN
...statement...
END
ISDISTINCTFROMconsidersnullvaluesequivalentandneverproducesanUNKNOWNvalue.
Tip NULLvaluesshouldbeconsideredinthecriteriaofanIFstatement.ISNULLcriteriacanbeusedtodetectthepresenceofaNULLvalue.
LoopStatement
ALOOPstatementisaniterativecontrolconstructthatisusedtocursorthrougharesultset.
Usage:
[label:]LOOPON<selectstatement>AS<cursorname>
statement
SyntaxRules
Thelabelmustnotbethesameasanyotherlabelusedinstatementscontainingthisone.
WhileStatement
AWHILEstatementisaniterativecontrolconstructthatisusedtoexecuteastatementrepeatedlywheneveraspecifiedconditionismet.
Usage:
[label:]WHILE<criteria>
statement
SyntaxRules
Thelabelmustnotbethesameasanyotherlabelusedinstatementscontainingthisone.
Procedures
452
ContinueStatement
ACONTINUEstatementisusedinsideaLOOPorWHILEconstructtocontinuewiththenextloopbyskippingovertherestofthestatementsintheloop.ItmustbeusedinsideaLOOPorWHILEstatement.
Usage:
CONTINUE[label];
SyntaxRules
Ifthelabelisspecified,itmustexistonacontainingLOOPorWHILEstatement.
Ifnolabelisspecified,thestatementwillaffecttheclosestcontainingLOOPorWHILEstatement.
BreakStatement
ABREAKstatementisusedinsideaLOOPorWHILEconstructtobreakfromtheloop.ItmustbeusedinsideaLOOPorWHILEstatement.
Usage:
BREAK[label];
SyntaxRules
Ifthelabelisspecified,itmustexistonacontainingLOOPorWHILEstatement.
Ifnolabelisspecified,thestatementwillaffecttheclosestcontainingLOOPorWHILEstatement.
LeaveStatement
ALEAVEstatementisusedinsideacompound,LOOP,orWHILEconstructtoleavetothespecifiedlevel.
Usage:
LEAVElabel;
SyntaxRules
Thelabelmustexistonacontainingcompoundstatement,LOOP,orWHILEstatement.
ReturnStatementAReturnstatementgracefullyexitstheprocedureandoptionallyreturnsavalue.
Usage:
RETURN[expression];
SyntaxRules
Procedures
453
Ifanexpressionisspecified,theproceduremusthaveareturnparameterandthevaluemustbeimplicitlyconvertabletotheexpectedtype.
Eveniftheprocedurehasareturnvalue,itisnotrequiredtospecifyareturnvalueinaRETURNstatement.
ErrorStatement
AnERRORstatementdeclaresthattheprocedurehasenteredanerrorstateandshouldabort.Thisstatementwillalsorollbackthecurrenttransaction,ifoneexists.AnyvalidexpressioncanbespecifiedaftertheERRORkeyword.
Usage:
ERRORmessage;
ExampleErrorStatement
ERROR'Invalidinputvalue:'||nvl(Acct.GetBalance.AcctID,'null');
AnERRORstatementisequivalentto:
RAISESQLEXCEPTIONmessage;
RaiseStatementARAISEstatementisusedtoraiseanexceptionorwarning.Whenraisinganexception,thisstatementwillalsorollbackthecurrenttransaction,ifoneexists.
Usage:
RAISE[SQLWARNING]exception;
Whereexceptionmaybeavariablereferencetoanexceptionoranexceptionexpression.
SyntaxRules
IfSQLWARNINGisspecified,theexceptionwillbesenttotheclientasawarningandtheprocedurewillcontinuetoexecute.
Anullwarningwillbeignored.Anullnon-warningexceptionwillstillcauseanexceptiontoberaised.
ExampleRaiseStatement
RAISESQLWARNINGSQLEXCEPTION'invalid'SQLSTATE'05000';
ExceptionExpression
Anexceptionexpressioncreatesanexceptionthatcanberaisedorusedasawarning.
Usage:
SQLEXCEPTIONmessage[SQLSTATEstate[,code]]CHAINexception
SyntaxRules
Procedures
454
Anyofthevaluesmaybenull;
messageandstatearestringexpressionsspecifyingtheexceptionmessageandSQLstaterespectively.TeiiddoesnotyetfullycomplywiththeANSISQLspecificationonSQLstateusage,butyouareallowedtosetanySQLstateyouchoose.
codeisanintegerexpressionspecifyingthevendorcode
exceptionmustbeavariablereferencetoanexceptionoranexceptionexpressionandwillbechainedtotheresultingexceptionasitsparent.
Procedures
455
VirtualProceduresVirtualproceduresaredefinedusingtheTeiidprocedurallanguage.Avirtualprocedurehaszeroormoreinput/inout/outparameters,anoptionalreturnparameter,andanoptionalresultset.VirtualproceduressupporttheabilitytoexecutequeriesandotherSQLcommands,definetemporarytables,adddatatotemporarytables,walkthroughresultsets,useloops,anduseconditionallogic.
TableofContentsVirtualProcedureDefinitionProcedureParametersExampleVirtualProceduresExecutingVirtualProceduresLimitations
VirtualProcedureDefinition
InDesigner:
[CREATEVIRTUALPROCEDURE]statement
InDDL:DDLMetadata#CreateProcedure/Function
Withinthebodyoftheprocedure,anyvalidstatementmaybeused.
Thereisnoexplicitcursoringorvaluereturningstatement,ratherthelastunnamedcommandstatementexecutedintheprocedurethatreturnsaresultsetwillbereturnedastheresult.Theoutputofthatstatementmustmatchtheexpectedresultsetandparametersoftheprocedure.
ProcedureParametersVirtualprocedurescantakezeroormoreIN/INOUTparametersandmayalsohaveanynumberofOUTparametersandanoptionalRETURNparameter.Eachinputhasthefollowinginformationthatisusedduringruntimeprocessing:
Name-Thenameoftheinputparameter
Datatype-Thedesign-timetypeoftheinputparameter
Defaultvalue-Thedefaultvalueiftheinputparameterisnotspecified
Nullable-NO_NULLS,NULLABLE,NULLABLE_UNKNOWN;parameterisoptionalifnullable,andisnotrequiredtobelistedwhenusingnamedparametersyntax
Youreferenceaparameterinavirtualprocedurebyusingthefully-qualifiednameoftheparam(orlessifunambiguous).Forexample,MySchema.MyProc.Param1.
ExampleofReferencinganInputParameterandAssigninganOutParameterforGetBalanceProcedure
BEGIN
MySchema.GetBalance.RetVal=UPPER(MySchema.GetBalance.AcctID);
SELECTBalanceFROMMySchema.AcctsWHEREMySchema.Accts.AccountID=MySchema.GetBalance.AcctID;
END
Procedures
456
IfanINOUTparameterisnotassignedanyvalueinaprocedureitwillremainthevalueitwasassignedforinput.AnyOUT/RETURNparameternotassignedavaluewillremaintheasthedefaultNULLvalue.TheINOUT/OUT/RETURNoutputvaluesarevalidatedagainsttheNOTNULLmetadataoftheparameter.
ExampleVirtualProcedures
ThisexampleisaLOOPthatwalksthroughacursoredtableandusesCONTINUEandBREAK.
VirtualProcedureUsingLOOP,CONTINUE,BREAK
BEGIN
DECLAREdoubletotal;
DECLAREintegertransactions;
LOOPON(SELECTamt,typeFROMCashTxnTable)AStxncursor
BEGIN
IF(txncursor.type<>'Sale')
BEGIN
CONTINUE;
ENDELSE
BEGIN
total=(total+txncursor.amt);
transactions=(transactions+1);
IF(transactions=100)
BEGIN
BREAK;
END
END
END
SELECTtotal,(total/transactions)ASavg_transaction;
END
ThisexampleisusesconditionallogictodeterminewhichoftwoSELECTstatementstoexecute.
VirtualProcedurewithConditionalSELECT
BEGIN
DECLAREstringVARIABLES.SORTDIRECTION;
VARIABLES.SORTDIRECTION=PartsVirtual.OrderedQtyProc.SORTMODE;
IF(ucase(VARIABLES.SORTDIRECTION)='ASC')
BEGIN
SELECT*FROMPartsVirtual.SupplierInfoWHEREQUANTITY>PartsVirtual.OrderedQtyProc.QTYINORDERBYPartsVi
rtual.SupplierInfo.PART_ID;
ENDELSE
BEGIN
SELECT*FROMPartsVirtual.SupplierInfoWHEREQUANTITY>PartsVirtual.OrderedQtyProc.QTYINORDERBYPartsVi
rtual.SupplierInfo.PART_IDDESC;
END
END
ExecutingVirtualProceduresYouexecuteproceduresusingtheSQLEXECUTEcommand.Iftheprocedurehasdefinedinputs,youspecifythoseinasequentiallist,orusing"name=value"syntax.Youmustusethenameoftheinputparameter,scopedbythefullprocedurenameiftheparameternameisambiguousinthecontextofothercolumnsorvariablesintheprocedure.
AvirtualprocedurecallwillreturnaresultsetjustlikeanySELECT,soyoucanusethisinmanyplacesyoucanuseaSELECT.Typicallyyou’llusethefollowingsyntax:
SELECT*FROM(EXEC...)ASx
Procedures
457
Limitations
Teiidvirtualproceduresmayonlyreturn1resultset.Ifyouneedtopassinaresultsetorpassoutmultipleresultset,thenconsiderusingglobaltemporarytables.
Procedures
458
Triggers
ViewTriggers
Viewsareabstractionsabovephysicalsources.Theytypicallyunionorjoininformationfrommultipletables,oftenfrommultipledatasourcesorotherviews.Teiidcanperformupdateoperationsagainstviews.Updatecommands-INSERT,UPDATE,orDELETE-againstaviewrequirelogictodefinehowthetablesandviewsintegratedbytheviewareaffectedbyeachtypeofcommand.Thistransformationlogic,alsoreferredtoasatrigger,isinvokedwhenanupdatecommandisissuedagainstaview.Updateproceduresdefinethelogicforhowauser’supdatecommandagainstaviewshouldbedecomposedintotheindividualcommandstobeexecutedagainsttheunderlyingphysicalsources.SimilartoVirtualProcedures,updateprocedureshavetheabilitytoexecutequeriesorothercommands,definetemporarytables,adddatatotemporarytables,walkthroughresultsets,useloops,anduseconditionallogic.
TeiidsupportsINSTEADOFtriggersonviewssimilartotraditionaldatabases.Theremayonlybe1FOREACHROWprocedureforeachINSERT,UPDATE,orDELETEoperationagainstaview.
Usage:
CREATETRIGGERONview_nameINSTEADOFINSERT|UPDATE|DELETEAS
FOREACHROW
...
UpdateProcedureProcessing
1. TheuserapplicationsubmitstheSQLcommandthroughoneofSOAP,JDBC,orODBC.
2. TheviewthisSQLcommandisexecutedagainstisdetected.
3. ThecorrectprocedureischosendependinguponwhetherthecommandisanINSERT,UPDATE,orDELETE.
4. Theprocedureisexecuted.TheprocedureitselfcancontainSQLcommandsofitsownwhichcanbeofdifferenttypesthanthecommandsubmittedbytheuserapplicationthatinvokedtheprocedure.
5. Commands,asdescribedintheprocedure,areissuedtotheindividualphysicaldatasourcesorotherviews.
6. Avaluerepresentingthenumberofrowschangedisreturnedtothecallingapplication.
SourceTriggers
TeiidsupportsAFTERtriggersonsourcetables,whicharecalledbyeventsfromaCDC(changedatacapture)system.
Usage:
CREATETRIGGERONsource_tableAFTERINSERT|UPDATE|DELETEAS
FOREACHROW
...
ForEachRow
Procedures
459
OnlytheFOREACHROWconstructissupportedasatriggerhandler.AFOREACHROWtriggerprocedurewillevaluateitsblockforeachrowoftheview/sourceaffectedbytheassociatedchange.ForUPDATEandDELETEstatementsthiswillbeeveryrowthatpassestheWHEREcondition.ForINSERTstatementstherewillbe1newrowforeachsetofvaluesfromtheVALUESorqueryexpression.Foraviewtherowsupdatedisreportedasthisnumberregardlessoftheaffectoftheunderlyingprocedurelogic.
Definition
Usage:
FOREACHROW
BEGINATOMIC
...
END
TheBEGINandENDkeywordsareusedtodenoteblockboundaries.Withinthebodyoftheprocedure,anyvalidstatementmaybeused.
Tip Theuseoftheatomickeywordiscurrentlyoptionalforbackwardcompatibility,butunlikeanormalblock,thedefaultforinsteadoftriggersisatomic.
SpecialVariables
Youcanuseanumberofspecialvariableswhendefiningyourupdateprocedure.
NEWVariables
Everyattributeontheview/tablewhoseUPDATEandINSERTtransformationsyouaredefininghasanequivalentvariablenamedNEW.<column_name>
WhenanINSERToranUPDATEcommandisexecutedortheeventisreceived,thesevariablesareinitializedtothevaluesintheINSERTVALUESclauseortheUPDATESETclauserespectively.
InanUPDATEprocedure,thedefaultvalueofthesevariables,iftheyarenotsetbythecommand,istheoldvalue.InanINSERTprocedure,thedefaultvalueofthesevariablesisthedefaultvalueofthevirtualtableattributes.SeeCHANGINGVariablesfordistinguishingdefaultsfrompassedvalues.
OLDVariables
Everyattributeontheview/tablewhoseUPDATEandDELETEtransformationsyouaredefininghasanequivalentvariablenamedOLD.<column_name>
WhenaDELETEorUPDATEcommandisexecutedortheeventisreceived,thesevariablesareinitializedtothecurrentvaluesoftherowbeingdeletedorupdatedrespectively.
CHANGINGVariables
Everyattributeontheview/tablewhoseUPDATEandINSERTtransformationsyouaredefininghasanequivalentvariablenamedCHANGING.<column_name>
WhenanINSERToranUPDATEcommandisexecutedorantheeventisreceived,thesevariablesareinitializedtotrueorfalsedependingonwhethertheINPUTvariablewassetbythecommand.ACHANGINGvariableiscommonlyusedtodifferentiatebetweenadefaultinsertvalueandonespecifiedintheuserquery.
Forexample,foraviewwithcolumnsA,B,C:
Procedures
460
IfUserExecutes… Then…
INSERTINTOVT(A,B)VALUES(0,1)
CHANGING.A=true,CHANGING.B=true,CHANGING.C=false
UPDATEVTSETC=2CHANGING.A=false,CHANGING.B=false,CHANGING.C=true
Procedures
461
CommentsTeiidsupportsmulti-linecommentsenclosedwith/**/:
/*comment
comment
comment...*/
Andsinglelinecomments:
SELECT...--comment
Commentnestingissupported.
Comments
462
Datatypes
Datatypes
463
SupportedTypesTeiidsupportsacoresetofruntimetypes.Runtimetypescanbedifferentthansemantictypesdefinedintypefieldsatdesigntime.Theruntimetypecanalsobespecifiedatdesigntimeoritwillbeautomaticallychosenastheclosestbasetypetothesemantictype.
Table1.TeiidRuntimeTypes
Type Description JavaRuntimeClass JDBCType ODBCType
stringorvarchar
variablelengthcharacterstringwithamaximumlengthof4000.
java.lang.String VARCHAR VARCHAR
varbinary
variablelengthbinarystringwithanominalmaximumlengthof8192.
byte[][1] VARBINARY VARBINARY
char asingleUnicodecharacter java.lang.Character CHAR CHAR
boolean
asinglebit,orBoolean,thatcanbetrue,false,ornull(unknown)
java.lang.Boolean BIT SMALLINT
byteortinyint numeric,integraltype,signed8-bit java.lang.Byte TINYINT SMALLINT
shortorsmallint numeric,integraltype,signed16-bit java.lang.Short SMALLINT SMALLINT
integerorserial
numeric,integraltype,signed32-bit.Theserialtypealsoimpliesnotnullandhasanauto-incrementingvaluethatstartsat1.serialtypesarenotautomaticallyUNIQUE.
java.lang.Integer INTEGER INTEGER
longorbigint numeric,integraltype,signed64-bit java.lang.Long BIGINT NUMERIC
biginteger
numeric,integraltype,arbitraryprecisionofupto1000digits
java.math.BigInteger NUMERIC NUMERIC
floatorreal
numeric,floatingpointtype,32-bitIEEE754floating-pointnumbers
java.lang.Float REAL FLOAT
SupportedTypes
464
doublenumeric,floatingpointtype,64-bitIEEE754floating-pointnumbers
java.lang.Double DOUBLE DOUBLE
bigdecimalordecimal
numeric,floatingpointtype,arbitraryprecisionofupto1000digits.
java.math.BigDecimal NUMERIC NUMERIC
date
datetime,representingasingleday(year,month,day)
java.sql.Date DATE DATE
time
datetime,representingasingletime(hours,minutes,seconds,milliseconds)
java.sql.Time TIME TIME
timestamp
datetime,representingasingledateandtime(year,month,day,hours,minutes,seconds,milliseconds,nanoseconds)
java.sql.Timestamp TIMESTAMP TIMESTAMP
object
anyarbitraryJavaobject,mustimplementjava.lang.Serializable
Any JAVA_OBJECT VARCHAR
blobbinarylargeobject,representingastreamofbytes
java.sql.Blob[2] BLOB VARCHAR
clobcharacterlargeobject,representingastreamofcharacters
java.sql.Clob[3] CLOB VARCHAR
xml XMLdocument java.sql.SQLXML[4] JAVA_OBJECT VARCHAR
geometry GeospatialObject java.sql.Blob[5] BLOB BLOB
Note Evenifatypeisdeclaredwithalength,precision,orscaleargument,thoserestrictionsareeffectivelyignoredbytheruntimesystem,butmaybeenforced/reportedattheedgebyOData,ODBC,JDBC.
ReferenceLink
1. Theruntimetypeisorg.teiid.core.types.BinaryType.TranslatorswillneedtoexplicitlyhandleBinaryTypevalues.UDFswillinsteadhaveabyte[]valuepassed.
2. Theconcretetypeisexpectedtobeorg.teiid.core.types.BlobType
3. Theconcretetypeisexpectedtobeorg.teiid.core.types.ClobType
4. Theconcretetypeisexpectedtobeorg.teiid.core.types.XMLType
5. Theconcretetypeisexpectedtobeorg.teiid.core.types.GeometryType
SupportedTypes
465
Arrays
Warning Teiid’ssupportforarraysisanewfeatureasofthe8.5release.Supportwillberefinedandenhancedinsubsequentreleases.
Anarrayofanytypeisdesignatedbyadding[]foreacharraydimensiontothetypedeclaration.
Examplearraytypes:
string[]
integer[][]
Note Teiidarrayhandlingistypicallyinmemory.Itisnotadvisabletorelyontheusageoflargearrayvalues.Alsoarraysoflobsarenotwellsupportedandwilltypicallynotbehandledcorrectlywhenserialized.
SupportedTypes
466
TypeConversionsDatatypesmaybeconvertedfromoneformtoanothereitherexplicitlyorimplicitly.Implicitconversionsautomaticallyoccurincriteriaandexpressionstoeasedevelopment.ExplicitdatatypeconversionsrequiretheuseoftheCONVERTfunctionorCASTkeyword.
TypeConversionConsiderations:
AnytypemaybeimplicitlyconvertedtotheOBJECTtype.
TheOBJECTtypemaybeexplicitlyconvertedtoanyothertype.
TheNULLvaluemaybeconvertedtoanytype.
Anyvalidimplicitconversionisalsoavalidexplicitconversion.
Situationsinvolvingliteralvaluesthatwouldnormallyrequireexplicitconversionsmayhavetheexplicitconversionappliedimplicitlyifnolossofinformationoccurs.
IfwidenComparisonToStringisfalse(thedefault),whenTeiiddetectsthatanexplicitconversioncannotbeappliedimplicitlyincriteria,thenanexceptionwillberaised.IfwidenComparisonToStringistrue,thendependinguponthecomparisonawideningconversionwillbeappliedorthecriteriawillbetreatedasfalse.
Forexample:
SELECT*FROMmy.tableWHEREcreated_by='notadate'
WithwidenComparisonToStringasfalseandcreated_byistypedasdate,ratherthanconvertingnotadatetoadatevalue,anexceptionwillberaised.
Arrayconversionsarecurrentlyonlyvalidtoandfromcompatibleobjectarrays.Itisnotyetpossibleforexampletocastfrominteger[]tolong[].
Explicitconversionsthatarenotallowedbetweentwotypeswillresultinanexceptionbeforeexecution.Allowedexplicitconversionsmaystillfailduringprocessingiftheruntimevaluesarenotactuallyconvertable.
Warning
TheTeiidconversionsoffloat/double/bigdecimal/timestamptostringrelyontheJDBC/Javadefinedoutputformats.Pushdownbehaviorattemptstomimictheseresults,butmayvarydependingupontheactualsourcetypeandconversionlogic.Careshouldbetakentonotassumethestringformincriteriaorotherplaceswhereavariationmaycausedifferentresults.
Table1.TypeConversions
SourceType ValidImplicitTargetTypes ValidExplicitTargetTypes
string clobchar,boolean,byte,short,integer,long,biginteger,float,double,bigdecimal,xml
char string
boolean string,byte,short,integer,long,biginteger,float,double,bigdecimal
byte string,short,integer,long,biginteger,float,double,bigdecimal boolean
short string,integer,long,biginteger,float, boolean,byte
[1]
TypeConversions
467
short string,integer,long,biginteger,float,double,bigdecimal
boolean,byte
integer string,long,biginteger,double,bigdecimal boolean,byte,short,float
long string,biginteger,bigdecimal,float,double
boolean,byte,short,integer,float,double
biginteger string,bigdecimalfloat ,double boolean,byte,short,integer,long,float,double
bigdecimal string,float ,double boolean,byte,short,integer,long,biginteger,float,double
float string,bigdecimal,double boolean,byte,short,integer,long,biginteger
double string,bigdecimal,float boolean,byte,short,integer,long,biginteger,float
date string,timestamp
time string,timestamp
timestamp string date,time
clob string
xml string
1.stringtoxmlisequivalenttoXMLPARSE(DOCUMENTexp)-SeealsoXMLFunctions#XMLPARSE2.implicitconversiontofloat/doubleonlyoccursforliteralvalues3.xmltostringisequivalenttoXMLSERIALIZE(expASSTRING)-seealsoXMLFunctions#XMLSERIALIZE
[2] [2]
[2][2]
[2] [2]
[2]
[3]
TypeConversions
468
SpecialConversionCases
ConversionofStringLiterals
TeiidautomaticallyconvertsstringliteralswithinaSQLstatementtotheirimpliedtypes.Thistypicallyoccursinacriteriacomparisonwhereanexpressionwithadifferentdatatypeiscomparedtoaliteralstring:
SELECT*FROMmy.tableWHEREcreated_by='2016-01-02'
Hereifthecreated_bycolumnhasthedatatypeofdate,Teiidautomaticallyconvertsthestringliteraltoadatedatatypeaswell.
ConvertingtoBooleanTeiidcanautomaticallyconvertliteralstringsandnumerictypevaluestoBooleanvaluesasfollows:
Type LiteralValue BooleanValue
String 'false' false
'unknown' null
other true
Numeric 0 false
other true
Date/Time/TimestampTypeConversions
Teiidcanimplicitlyconvertproperlyformattedliteralstringstotheirassociateddate-relateddatatypesasfollows:
StringLiteralFormat PossibleImplicitConversionType
yyyy-mm-dd DATE
hh:mm:ss TIME
yyyy-mm-dd[hh:mm:ss.[fff…]] TIMESTAMP
TheformatsabovearethoseexpectedbytheJDBCdatetypes.TouseotherformatsseethefunctionsPARSEDATE,PARSETIME,PARSETIMESTAMP.
SpecialConversionCases
469
EscapedLiteralSyntaxRatherthanrelyingonimplicitconversion,datatypevaluesmaybeexpresseddirectlyinSQLusingescapesyntaxtodefinethetype.Notethatthesuppliedstringvaluemustmatchtheexpectedformatexactlyoranexceptionwilloccur.
Datatype EscapedSyntax
DATE \{d'yyyy-mm-dd'}
TIME \{t'hh-mm-ss'}
TIMESTAMP \{ts'yyyy-mm-dd[hh:mm:ss.[fff…]]'}
EscapedLiteralSyntax
470
UpdatableViewsAnyviewmaybemarkedasupdatable.InmanycircumstancestheviewdefinitionmayallowtheviewtobeinherentlyupdatablewithouttheneedtomanuallydefineatriggertohandleINSERT/UPDATE/DELETEoperations.
Aninherentlyupdatableviewcannotbedefinedwithaquerythathas:
Asetoperation(INTERSECT,EXCEPT,UNION).
SELECTDISTINCT
Aggregation(aggregatefunctions,GROUPBY,HAVING)
ALIMITclause
AUNIONALLcandefineaninherentlyupdatableviewonlyifeachoftheUNIONbranchesisitselfinherentlyupdatable.AviewdefinedbyaUNIONALLcansupportinherentINSERTsifitisaFederatedOptimizations#PartitionedUnionandtheINSERTspecifiesvaluesthatbelongtoasinglepartition.
AnyviewcolumnthatisnotmappeddirectlytoacolumnisnotupdatableandcannotbetargetedbyanUPDATEsetclauseorbeanINSERTcolumn.
IfaviewisdefinedbyajoinqueryorhasaWITHclauseitmaystillbeinherentlyupdatable.Howeverinthesesituationstherearefurtherrestrictionsandtheresultingqueryplanmayexecutemultiplestatements.Foranon-simplequerytobeupdatable,itisrequired:
AnINSERT/UPDATEcanonlymodifyasingleKey-preservedTable.
ToallowDELETEoperationstheremustbeonlyasingleKey-preservedTable.
IfthedefaulthandlingisnotavailableoryouwishtohaveanalternativeimplementationofanINSERT/UPDATE/DELETE,thenyoumayuseUpdateProcedures(Triggers)todefineprocedurestohandletherespectiveoperations.
UpdatableViews
471
preservedTableAkey-preservedtablehasaprimaryoruniquekeythatwouldremainuniqueifitwereprojectedintotheresultofthequery.NotethatitisnotactuallyrequiredforaviewtoreferencethekeycolumnsintheSELECTclause.Thequeryenginecandetectakeypreservedtablebyanalyzingthejoinstructure.Theenginewillensurethatajoinofakey-preservedtablemustbeagainstoneofitsforeignkeys.
preservedTable
472
TransactionSupportTeiidutilizesXAtransactionsforparticipatinginglobaltransactionsandfordemarcatingitslocalandcommandscopedtransactions.JBossTransactionsisusedbyTeiidasitstransactionmanager.SeethisdocumentationfortheadvancedfeaturesprovidedbyJBossTransactions.
Table1.TeiidTransactionScopes
Scope Description
Command
Treatstheusercommandasifallsourcecommandsareexecutedwithinthescopeofthesametransaction.TheAutoCommitTxnexecutionpropertycontrolsthebehaviorofcommandleveltransactions.
Local Thetransactionboundaryislocaldefinedbyasingleclientsession.
Global TeiidparticipatesinaglobaltransactionasanXAResource.
ThedefaulttransactionisolationlevelforTeiidisREAD_COMMITTED.
TransactionSupport
473
AutoCommitTxnExecutionPropertySinceuserlevelcommandsmayexecutemultiplesourcecommands,userscanspecifytheAutoCommitTxnexecutionpropertytocontrolthetransactionalbehaviorofausercommandwhennotinalocalorglobaltransaction.
Table1.AutoCommitTxnSettings
Setting Description
OFFDonotwrapeachcommandinatransaction.Individualsourcecommandsmaycommitorrollbackregardlessofthesuccessorfailureoftheoverallcommand.
ON Wrapeachcommandinatransaction.Thismodeisthesafest,butmayintroduceperformanceoverhead.
DETECTThisisthedefaultsetting.Willautomaticallywrapcommandsinatransaction,butonlyifthecommandseemstobetransactionallyunsafe.
TheconceptofcommandsafetywithrespecttoatransactionisdeterminedbyTeiidbaseduponcommandtype,thetransactionisolationlevel,andavailablemetadata.Awrappingtransactionisnotneededif:
Ifausercommandisfullypushedtothesource.
IftheusercommandisaSELECT(includingXML)andthetransactionisolationisnotREPEATABLE_READnorSERIALIABLE.
IftheusercommandisastoredprocedureandthetransactionisolationisnotREPEATABLE_READnorSERIALIABLEandtheUpdatingModelCountiszero.
Theupdatecountmaybesetonallproceduresaspartoftheproceduremetadatainthemodel.
AutoCommitTxnExecutionProperty
474
UpdatingModelCountTheterm"updatingmodelcount"referstothenumberoftimesanymodelisupdatedduringtheexecutionofacommand.Itisusedtodeterminewhetheratransaction,ofanyscope,isrequiredtosafelyexecutethecommand.
Table1.UpdatingModelCountSettings
Count Description
0 Noupdatesareperformedbythiscommand.
1
Indicatesthatonlyonemodelisupdatedbythiscommand(anditssubcommands).Alsothesuccessorfailureofthatupdatecorrespondstothesuccessoffailureofthecommand.Itshouldnotbepossiblefortheupdatetosucceedwhilethecommandfails.Executionisnotconsideredtransactionallyunsafe.
*Anynumbergreaterthan1indicatesthatexecutionistransactionallyunsafeandanXAtransactionwillberequired.
UpdatingModelCount
475
JDBCandTransactions
JDBCAPIFunctionality
ThetransactionscopesabovemaptotheseJDBCmodes:
Command-ConnectionautoCommitpropertysettotrue.
Local-ConnectionautoCommitpropertysettofalse.ThetransactioniscommittedbysettingautoCommittotrueorcallingjava.sql.Connection.commit.Thetransactioncanberolledbackbyacalltojava.sql.Connection.rollback
Global-theXAResourceinterfaceprovidedbyanXAConnectionisusedtocontrolthetransaction.NotethatXAConnectionsareavailableonlyifTeiidisconsumedthroughitsXADataSource,org.teiid.jdbc.TeiidDataSource.JEEcontainersordataaccessAPIstypicallycontrolXAtransactionsonbehalfofapplicationcode.
J2EEUsageModels
J2EEprovidesthreewaystomanagetransactionsforbeans:
Client-controlled–theclientofabeanbeginsandendsatransactionexplicitly.
Bean-managed–thebeanitselfbeginsandendsatransactionexplicitly.
Container-managed–theappservercontainerbeginsandendsatransactionautomatically.
Inanyofthesecases,transactionsmaybeeitherlocalorXAtransactions,dependingonhowthecodeanddescriptorsarewritten.Somekindsofbeans(statefulsessionbeansandentitybeans)arenotrequiredbythespectosupportnon-transactionalsources,althoughthespecdoesallowanappservertooptionallysupportthiswiththecautionthatthisisnotportableorpredictable.Generallyspeaking,tosupportmosttypicalEJBactivitiesinaportablefashionrequiressomekindoftransactionsupport.
JDBCandTransactions
476
TransactionalBehaviorwithWildFlyDataSourceTypesWildFlyallowscreationofdifferenttypesofdatasources,basedontheirtransactionalcapabilities.ThetypeofdatasourceyoucreateforyourVDB’ssourcesalsodictatesifthatdatasourcewillbeparticipatingthedistributedtransactionornot,irrespectiveofthetransactionscopeyouselectedfromabove.Herearedifferenttypesofdatasources
xa-datasource:CapableofparticipatinginthedistributedtransactionusingXA.ThisisrecommendedtypebeusedwithanyTeiidsources.
local-datasource:DoesnotparticipateinXA,unlessthisistheonlysourcethatislocal-datasourcethatisparticipatingamongotherxa-datasourcesinthecurrentdistributedtransaction.Thistechniqueiscalledlastcommitoptimization.However,ifyouhavemorethenonelocal-datasourcesparticipatinginatransaction,thenthetransactionmanagerwillendupwith"Couldnotenlistintransactiononenteringmeta-awareobject!;"exception.
no-tx-datasource:Doesnotparticipateindistributedtransactionatall.InthescopeofTeiidcommandovermultiplesources,youcanincludethistypeofdatasourceinthesamedistributedtransactioncontext,howeverthissourcewillbeitwillnotbesubjecttoanytransactionalparticipation.Anychangesdoneonthissourceaspartofthetransactionscope,cannotberolledback.IfyouhavethreedifferentsourcesA,B,CandtheyarebeingusedinTeiid.Herearesomevariationsonhowtheybehavewithdifferenttypesofdatasources.Thesuffixes"xa","local","no-tx"definedifferenttypeofsourcesused.
A-xaB-xa,C-xa:Canparticipateinalltransactionalscopes.Norestrictions.
A-xa,B-xa,c-local:Canparticipateinalltransactionalscopes.Notethatthereisonlyonesinglesourceis"local".ItisassumedthatintheGlobalscope,thethirdpartydatasource,otherthanTeiidDatasourceisalsoXA.
A-xa,B-xa,C-no-tx:Canparticipateinalltransactionalscopes.Note"C"isnotareallyboundbyanytransactionalcontract.AandBaretheonlyparticipentsinXAtransaction.
A-xa,B-local,C-no-tx:Canparticipateinalltransactionalscopes.Note"C"isnotareallyboundbyanytransactionalcontract,andthereisonlysingle"local"source.
Ifanytwoormoresourcesare"local":TheycanonlyparticipateinCommandmodewith"autoCommitTxn=OFF".Otherwisewillendwithexceptionas"Couldnotenlistintransactiononenteringmeta-awareobject!;"exception,asitisnotpossibletodoaXAtransactionwith"local"datasources.
A-no-tx,B-no-tx,C-no-tx:Canparticipateinalltransactionscopes,butnoneofthesourceswillbeboundbytransactionalterms.ThisisequivalenttonotusingtransactionsorsettingCommandmodewith"autoCommitTxn=OFF".
Note
TeiidDesignercreates"local"datasourcebydefaultwhichisnotoptimalfortheXAtransactions.TeiidwouldlikethistobecreatingaXAdatasources,howeverwithcurrentlimitationswithDTPthatfeatureiscurrentlynotavailable.TocreateXAdatasource,lookintheWildFly"doc"directoryforexampletemplates,orusethe"admin-console"tocreatetheXAdatasources.
IfyourdatasourceisnotXA,andnottheonlylocalsourceandcannotuse"no-tx",thenyoucanlookintoextendingthesourcetoimplementthecompensatingXAimplementation.i.e.defineyourownresourcemanagerforyoursourceandmanagethetransactionthewayyouwantittobehave.NotethatthiscouldbecomplicatedifnotimpossibleifyoursourcenativelydoesnotsupportdistributedXAprotocol.Insummay
UseXAdatasourceifpossible
Useno-txdatasourceifapplicable
UseautoCommitTxn=OFF,andletgodistributedtransactions,thoughnotrecommended
WriteacompensatingXAbasedimplementation.
Table1.TeiidTransactionParticipation
TransactionalBehaviorwithJBossDataSourceTypes
477
Teiid-Tx-Scope XAsource LocalSource No-TxSource
Local(Auto-commit=false) always OnlyIfSingleSource never
Global always OnlyIfSingleSource never
Auto-commit=true,AutoCommitTxn=ON,orDETECTandtxnstarted
always OnlyIfSingleSource never
Auto-commit=true,AutoCommitTxn=OFF never never never
TransactionalBehaviorwithJBossDataSourceTypes
478
LimitationsandWorkaroundsTheclientsettingoftransactionisolationlevelisnotpropogatedtotheconnectors.ThetransactionisolationlevelcanbesetoneachXAconnector,howeverthisisolationlevelisfixedandcannotbechangedatruntimeforspecificconnections/commands.
LimitationsandWorkarounds
479
DataRolesDataroles,alsocalledentitlements,aresetsofpermissionsdefinedperVDBthatdictatedataaccess(create,read,update,delete).Datarolesuseafine-grainedpermissionsystemthatTeiidwillenforceatruntimeandprovideauditlogentriesforaccessviolations-seeLoggingandCustomLoggingformore.
Priortoapplyingdataroles,youshouldconsiderrestrictingsourcesystemaccessthroughthefundamentaldesignofyourVDB.Foremost,Teiidcanonlyaccesssourceentriesthatarerepresentedinimportedmetadata.YoushouldnarrowimportedmetadatatoonlywhatisnecessaryforusebyyourVDB.WhenusingTeiidDesigner,youmaythengofurtherandmodifytheimportedmetadataatagranularleveltoremovespecificcolumns,marktablesasnon-updatable,etc.
IfdatarolevalidationisenabledanddatarolesaredefinedinaVDB,thenaccesspermissionswillbeenforcedbytheTeiidServer.Theuseofdatarolesmaybedisabledsystemwidebyremovingthesettingfortheteiidsubsystempolicy-decider-module.Datarolesalsohavebuilt-insystemfunctionsthatcanbeusedforrow-basedandotherauthorizationchecks.
Tip
UnlikepreviousversionsofTeiiddataroleswillonlybecheckedifpresentinaVDB.AVDBdeployedwithoutdatarolesisopenforusebyanyauthenticateduser.Ifyouwanttoensuresomeattempthasbeenmadeatsecuringaccess,thensetthedata-roles-requiredconfigurationelementtotrueviatheCLIorinthestandalone.xmlontheteiidsubsystem.
DataRoles
480
Permissions
UserQueryPermissions
CREATE,READ,UPDATE,DELETE(CRUD)permissionscanbesetforanyresourcepathinaVDB.Aresourcepathcanbeasspecificasthefullyqualifiednameofacolumnorasgeneralatoplevelmodel(schema)name.Permissionsgrantedtoaparticularpathapplytoitandanyresourcepathsthatsharethesamepartialname.Forexample,grantingreadto"model"willalsograntreadto"model.table","model.table.column",etc.Allowingordenyingaparticularactionisdeterminedbysearchingforpermissionsfromthemosttoleastspecificresourcepaths.Thefirstpermissionfoundwithaspecificallowordenywillbeused.Thusitispossibletosetverygeneralpermissionsathigh-levelresourcepathnamesandtooverrideonlyasnecessaryatmorespecificresourcepaths.
Permissiongrantsareonlyneededforresourcesthataroleneedsaccessto.Permissionsarealsoonlyappliedtothecolumns/tables/proceduresintheuserquery-nottoeveryresourceaccessedtransitivelythroughviewandproceduredefinitions.Itisimportantthereforetoensurethatpermissiongrantsareappliedconsistentlyacrossmodelsthataccessthesameresources.
NoteUnlikepreviousversionsofTeiid,non-visiblemodelsareaccessiblebyuserqueries.Torestrictuseraccessatamodellevel,atleastonedataroleshouldbecreatedtoenabledatarolechecking.Inturnthatrolecanbemappedtoanyauthenticateduserandshouldnotgrantpermissionstomodelsthatshouldbeinaccessible.
PermissionsarenotapplicabletotheSYSandpg_catalogschemas.Thesemetadatareportingschemasarealwaysaccessibleregardlessoftheuser.TheSYSADMINschemahowevermayneedpermissionsasapplicable.
ToprocessaSELECTstatementorastoredprocedureexecution,theuseraccountrequiresthefollowingaccessrights:
READ-ontheTable(s)beingaccessedortheprocedurebeingcalled.
READ-oneverycolumnreferenced.
ToprocessanINSERTstatement,theuseraccountrequiresthefollowingaccessrights:
CREATE-ontheTablebeinginsertedinto.
CREATE-oneverycolumnbeinginsertedonthatTable.
ToprocessanUPDATEstatement,theuseraccountrequiresthefollowingaccessrights:
UPDATE-ontheTablebeingupdated.
UPDATE-oneverycolumnbeingupdatedonthatTable.
READ-oneverycolumnreferencedinthecriteria.
ToprocessaDELETEstatement,theuseraccountrequiresthefollowingaccessrights:
DELETE-ontheTablebeingdeleted.
READ-oneverycolumnreferencedinthecriteria.
ToprocessaEXEC/CALLstatement,theuseraccountrequiresthefollowingaccessrights:
EXECUTE(orREAD)-ontheProcedurebeingexecuted.
Toprocessanyfunction,theuseraccountrequiresthefollowingaccessrights:
EXECUTE(orREAD)-ontheFunctionbeingcalled.
ToprocessanyALTERorCREATETRIGGERstatement,theuseraccountrequiresthefollowingaccessrights:
Permissions
481
ALTER-onthevieworprocedurethatiseffected.INSTEADOFTriggers(updateprocedures)arenotyettreatedasfullschemaobjectsandareinsteadtreatedasattributesoftheview.
ToprocessanyOBJECTTABLEfunction,theuseraccountrequiresthefollowingaccessrights:
LANGUAGE-specifyingthelanguagenamethatisallowed.
ToprocessanystatementagainstaTeiidtemporarytablerequiresthefollowingaccessrights:
allow-create-temporary-tablesattributeonanyapplicablerole
CREATE,READ,UPDATE,DELETE-againstthetargetmodel/schemaasneededforoperationsagainstaFOREIGNtemporarytable.
RowandColumnBasedSecurity
AlthoughspecifiedinasimilarwaytouserqueryCRUDpermissions,row-basedandcolumn-basedpermissionsmaybeusedtogetherorseparatelytocontrolatamoregranularandconsistentlevelthedatareturnedtousers.SeealsoXMLDefinitionforexamplesofspecifyingdataroleswithrowandcolumnbasedsecurity.
Row-BasedSecurity
Apermissionagainstafullyqualifiedtable/view/proceduremayalsospecifyacondition.UnliketheallowCRUDactionsdefinedabove,aconditionisalwaysapplied-notjustattheuserquerylevel.TheconditioncanbeanyvalidSQLreferencingthecolumnsofthetable/view/procedure.Procedureresultsetcolumnsmaybereferencedasproc.col.Theconditionwillactasarow-basedfilterandasacheckedconstraintforinsert/updateoperations.
HowRow-BasedConditionsAreApplied
Aconditionisappliedconjunctivelytoupdate/delete/selectwhereclausesagainsttheaffectedresource.Thosequerieswillthereforeonlyeverbeeffectiveagainstthesubsetofrowsthatpassthecondition,i.e."SELECT*FROMTBLWHEREblahANDcondition".Theconditionwillbepresentregardlessofhowthetable/viewisusedinthequery,whetherviaaunion,join,etc.
Insertsandupdatesagainstphysicaltablesaffectedbyaconditionarefurthervalidatedsothattheinsert/changevaluesmustpassthecondition(evaluatetotrue)fortheinsert/updatetosucceed-thisiseffectivelythesameaSQLconstraint.Thiswillhappenforallstylesofinsert/update-insertwithqueryexpression,bulkinsert/update,etc.Inserts/updatesagainstviewsarenotcheckedwithregardstotheconstraint.Youmaydisabletheinsert/updateconstraintcheckbysettingtheconditionconstraintflagtofalse.Thisistypicallyonlyneededincircumstanceswhentheconditioncannotalwaysbeevaluated.Howeverdisablingtheconditionasaconstraintsimplydropstheconditionfromconsiderationwhenlogicallyevaluatingtheconstraint.Anyotherconditionconstraintswillstillbeevaluated.
Acrossmultipleapplicablerolesifmorethanoneconditionappliestothesameresource,theconditionswillbeaccumulateddisjunctivelyviaOR,i.e."(condition1)OR(condition2)…".Thereforegrantingapermissionwiththecondition"true"willallowusersinthatroletoseeallrowsofthegivenresource.
ConsiderationsWhenUsingConditions
Non-pushdownconditionsmayadverselyimpactperformance,sincetheirevaluationmayinhibitpushdownofqueryconstructsontopoftheaffectedresource.Multipleconditionsagainstthesameresourceshouldgenerallybeavoidedasanynon-pushdownconditionwillcausetheentireORofconditionstonotbepusheddown.Insomecircumstancestheinsertionofpermissionconditionsmayrequirethattheplanbealteredwiththeadditionofaninlineview,whichcanresultinadverseperformanceagainstsourcesthatdonotsupportinlineviews.
Permissions
482
Pushdownofmulti-rowinsert/updateoperationswillbeinhibitedsincetheconditionmustbecheckedforeachrow.
Inadditiontomanagingpermissionconditionsonaper-rolebasis,anotherapproachistoaddconditionpermissionswouldinananyauthenticatedrolesuchthattheconditionsaregeneralizedforallusers/rolesusingthehasRole,user,andothersuchsecurityfunctions.Theadvantageofthelatterapproachisthatthereiseffectivelyastaticrow-basedpolicyineffectsuchthatallqueryplanscanstillbesharedbetweenusers.
HandlingofnullvaluesisuptotheimplementerofthedataroleandmayrequireISNULLcheckstoensurethatnullvaluesareallowedwhenacolumnisnullable.
Limitations
Conditionsonsourcetablesthatactascheckconstraintsmustcurrentlynotcontaincorrelatedsubqueries.
Conditionsmaynotcontainaggregateorwindowedfunctions.
Tablesandproceduresreferencedviasubquerieswillstillhaverow-basedfiltersandcolumnmaskingappliedtothem.
Note Row-basedfilterconditionsareenforcedevenformaterializedviewloads.
Youshouldensurethattablesconsumedtoproducematerializedviewsdonothaverow-basedfilterconditionsonthemthatcouldaffectthematerializedviewresults.
ColumnMasking
Apermissionagainstafullyqualifiedtable/view/procedurecolumnmayalsospecifyamaskandoptionallyacondition.Whenthequeryissubmittedtherolesareconsultedandtherelevantmask/conditioninformationarecombinedtoformasearchedcaseexpressiontomaskthevaluesthatwouldhavebeenreturnedbytheaccess.UnliketheCRUDallowactionsdefinedabove,theresultingmaskingeffectisalwaysapplied-notjustattheuserquerylevel.TheconditionandexpressioncanbeanyvalidSQLreferencingthecolumnsofthetable/view/procedure.Procedureresultsetcolumnsmaybereferencedasproc.col.
HowColumnMasksAreApplied
ColumnmaskingisappliedonlyagainstSELECTs.Columnmaskingisappliedlogicallyaftertheaffectofrowbasedsecurity.Howeversincebothviewsandsourcetablesmayhaverowandcolumnbasedsecurity,theactualviewlevelmaskingmaytakeplaceontopofsourcelevelmasking.Iftheconditionisspecifiedalongwiththemask,thentheeffectivemaskexpressioneffectsonlyasubsetoftherows:"CASEWHENconditionTHENmaskELSEcolumn".OtherwisetheconditionisassumedtobeTRUE,meaningthatthemaskappliestoallrows.
Ifmultiplerolesspecifyamaskagainstacolumn,themaskorderargumentwilldeterminetheirprecedencefromhighesttolowestaspartofalargersearchedcaseexpression.Forexampleamaskwiththedefaultorderof0andamaskwithanorderof1wouldbecombinedas"CASEWHENcondition1THENmask1WHENcondition0THENmask0ELSEcolumn".
ConsiderationsWhenUsingMasking
Non-pushdownmaskingconditions/expressionsmayadverselyimpactperformance,sincetheirevaluationmayinhibitpushdownofqueryconstructsontopoftheaffectedresource.Insomecircumstancestheinsertionofmaskingmayrequirethattheplanbealteredwiththeadditionofaninlineview,whichcanresultinadverseperformanceagainstsourcesthatdonotsupportinlineviews.
Inadditiontomanagingmaskingonaper-rolebasiswiththeuseoftheordervalue,anotherapproachistospecifymaskinginasingleanyauthenticatedrolesuchthattheconditions/expressionsaregeneralizedforallusers/rolesusingthehasRole,user,andothersuchsecurityfunctions.Theadvantageofthelatterapproachisthatthereiseffectivelyastaticmaskingpolicyineffectsuchthatallqueryplanscanstillbesharedbetweenusers.
Permissions
483
Limitations
Intheeventthattwomaskshavethesameordervalue,itisnotwelldefinedwhatordertheyareappliedin.
Masksortheirconditionsmaynotcontainaggregateorwindowedfunctions.
Tablesandproceduresreferencedviasubquerieswillstillhaverow-basedfiltersandcolumnmaskingappliedtothem.
Note Maskingisenforcedevenformaterializedviewloads.
Youshouldensurethattablesconsumedtoproducematerializedviewsdonothavemaskingonthemthatcouldaffectthematerializedviewresults.
Permissions
484
RoleMappingEachTeiiddatarolecanbemappedtoanynumberofcontainerrolesoranyauthenticateduser.YoumaycontrolrolemembershipthroughwhateversystemtheTeiidsecuritydomainloginmodulesareassociatedwith.ThekitincludesexamplefilesforusewiththeUsersRolesLoginModule-seeteiid-security-roles.properties.
IfyouhaveanalternativesecuritydomainthataVDBshoulduse,thensettheVDBpropertysecurity-domaintotherelevantsecuritydomain.
Itispossibleforausertohaveanynumberofcontainerroles,whichinturnimplyasubsetofTeiiddataroles.EachapplicableTeiiddatarolecontributescumulativelytothepermissionsoftheuser.Noonerolesupersedesornegatesthepermissionsoftheotherdataroles.
RoleMapping
485
XMLDefinitionDatarolesaredefinedinsidethevdb.xmlfile(insidethe.vdbZiparchiveunderMETA-INF/vdb.xml)ifyouusedDesigner.The"vdb.xml"fileischeckedagainsttheschemafilevdb-deployer.xsd,whichcanbefoundinthekitunderdocs/teiid/schema.Thisexamplewillshowasample"vdb.xml"filewithfewsimpledataroles.
Forexample,ifaVDBdefinesatable"TableA"inschema"modelName"withcolumns(column1,column2)-notethatthecolumntypesdonotmatter.Andwewishtodefinethreeroles"RoleA","RoleB",and"admin"withfollowingpermissions:
1. RoleAhaspermissionstoread,writeaccesstoTableA,butcannotdelete.
2. RoleBhaspermissionsthatonlyallowreadaccesstoTableA.column1
3. adminhasallpermissions
vdb.xmldefiningRoleA,RoleB,andAdmin
<?xmlversion="1.0"encoding="UTF-8"?>
<vdbname="sample"version="1">
<modelname="modelName">
<sourcename="source-name"translator-name="oracle"connection-jndi-name="java:myDS"/>
</model>
<data-rolename="RoleA">
<description>Allowall,exceptDelete</description>
<permission>
<resource-name>modelName.TableA</resource-name>
<allow-create>true</allow-create>
<allow-read>true</allow-read>
<allow-update>true</allow-update>
</permission>
<mapped-role-name>role1</mapped-role-name>
</data-role>
<data-rolename="RoleB">
<description>Allowreadonly</description>
<permission>
<resource-name>modelName.TableA</resource-name>
<allow-read>true</allow-read>
</permission>
<permission>
<resource-name>modelName.TableA.colum2</resource-name>
<allow-read>false</allow-read>
</permission>
<mapped-role-name>role2</mapped-role-name>
</data-role>
<data-rolename="admin"grant-all="true">
<description>Adminrole</description>
<mapped-role-name>admin-group</mapped-role-name>
</data-role>
</vdb>
XMLDefinition
486
TheaboveXMLdefinedthreedataroles,"RoleA"whichallowseverythingexceptdeleteonthetable,"RoleB"thatallowsonlyreadoperationonthetable,andthe"admin"rolewithallpermissions.SinceTeiidusesdenybydefault,thereisnoexplicitdata-roleentryneededfor"RoleB".NotethatexplicitcolumnpermissionsarenotneededforRoleA,sincetheparentresourcepath,modelName.TableA,permissionsstillapply.RoleBhowevermustexplicitlydisallowreadtocolumn2.
The"mapped-role-name"definesthecontainerJAASrolesthatareassignedthedatarole.ForassigningrolestoyourusersintheWildFly,checkouttheinstructionsfortheselectedLoginModule.Checkthe"AdminGuide"forconfiguringLoginModules.
Usingthegrant-alloptionprovideseverypermissiononoverobjectinthevdb.Whenimportingavdbanditsroles,grant-allappliesonlytoresourcesfromtheimportedvdb.
AdditionalRoleAttributes
Youmayalsochoosetoallowanyauthenticatedusertohaveadatarolebysettingtheany-authenticatedattributevaluetotrueondata-roleelement.
The"allow-create-temporary-tables"data-rolebooleanattributeisusedtoexplicitlyenableordisabletemporarytableusagefortherole.Ifitisleftunspecified,thenthevaluewillbedefaultedtofalse.
TempTableRoleforAnyAuthenticated
<data-rolename="role"any-authenticated="true"allow-create-temporary-tables="true">
<description>TempTableRoleforAnyAuthenticated</description>
<permission>
...
</permission>
</data-role>
LanguageAccess
Thefollowingshowsavdbxmlthatallowstheuseofthejavascriptlanguage.Theallowed-languagespropertyenablesthelanguagesuseforanypurposeinthevdb,whiletheallow-languagepermissionallowsthelanguagetobeusedbyuserswithRoleA.
vdb.xmlallowingJavaScriptaccess
<?xmlversion="1.0"encoding="UTF-8"?>
<vdbname="sample"version="1">
<propertyname="allowed-languages"value="javascript"/>
<modelname="modelName">
<sourcename="source-name"translator-name="oracle"connection-jndi-name="java:myDS"/>
</model>
<data-rolename="RoleA">
<description>Readandjavascriptaccess.</description>
<permission>
<resource-name>modelName</resource-name>
<allow-read>true</allow-read>
</permission>
<permission>
<resource-name>javascript</resource-name>
<allow-language>true</allow-language>
</permission>
<mapped-role-name>role1</mapped-role-name>
</data-role>
XMLDefinition
487
</vdb>
Row-BasedSecurity
Thefollowingshowsavdbxmlutilizingaconditiontorestrictaccess.Theconditionactsasbothafilterandconstraint.EventhoughRoleAopensupread/insertaccesstomodelName.tblName,thebase-roleconditionwillensurethatonlyvaluesofcolumn1matchingthecurrentusercanbereadorinserted.Notethatheretheconstraintenforcementhasbeendisabled.
vdb.xmlallowingconditionalaccess
<?xmlversion="1.0"encoding="UTF-8"?>
<vdbname="sample"version="1">
<modelname="modelName">
<sourcename="source-name"translator-name="oracle"connection-jndi-name="java:myDS"/>
</model>
<data-rolename="base-role"any-authenticated="true">
<description>Conditionalaccess</description>
<permission>
<resource-name>modelName.tblName</resource-name>
<conditionconstraint="false">column1=user()</condition>
</permission>
</data-role>
<data-rolename="RoleA">
<description>Read/Insertaccess.</description>
<permission>
<resource-name>modelName.tblName</resource-name>
<allow-read>true</allow-read>
<allow-create>true</allow-create>
</permission>
<mapped-role-name>role1</mapped-role-name>
</data-role>
</vdb>
ColumnMasking
Thefollowingshowsavdbxmlutilizingcolumnmasking.HeretheRoleAcolumn1masktakesprecedenceoverthebase-rolemask,butonlyforasubsetoftherowsasspecifiedbythecondition.ForuserswithoutRoleA,accesstocolumn1willeffectivelybereplacedwith"CASEWHENcolumn1=user()THENcolumn1END",whileforuserswithRoleA,accesstocolumn1willeffectivelybereplacedwith"CASEWHENcolumn2=’x’THENcolumn1WHENTRUETHENCASEWHENcolumn1=user()THENcolumn1ENDEND".
vdb.xmlwithcolumnmasking
<?xmlversion="1.0"encoding="UTF-8"?>
<vdbname="sample"version="1">
<modelname="modelName">
<sourcename="source-name"translator-name="oracle"connection-jndi-name="java:myDS"/>
</model>
<data-rolename="base-role"any-authenticated="true">
<description>Masking</description>
<permission>
<resource-name>modelName.tblName.column1</resource-name>
<mask>CASEWHENcolumn1=user()THENcolumn1END</mask>
XMLDefinition
488
</permission>
</data-role>
<data-rolename="RoleA">
<description>Read/Insertaccess.</description>
<permission>
<resource-name>modelName.tblName</resource-name>
<allow-read>true</allow-read>
<allow-create>true</allow-create>
</permission>
<permission>
<resource-name>modelName.tblName.column1</resource-name>
<condition>column2='x'</condition>
<maskorder="1">column1</mask>
</permission>
<mapped-role-name>role1</mapped-role-name>
</data-role>
</vdb>
XMLDefinition
489
CustomizingSeetheDeveloper’sGuidechaptersonCustomAuthorizationValidatorsand[Teiid:LoginModules]fordetailsonusinganalternativeauthorizationscheme.
Customizing
490
SystemSchemaThebuilt-inSYSandSYSADMINschemasprovidemetadatatablesandproceduresagainstthecurrentVDB.
BydefaultasystemschemaforODBCmetadatapg_catalogisalsoexposed-howeverthatshouldbeconsideredforgeneraluse.
MetadataVisibility
TheSYSsystemschematablesandproceduresarealwaysvisible/accessible.
UnlikeTeiid8.xandpriorreleaseswhenDataRolesareinusetable/viewsandproceduremetadataentrieswillnotbevisibleiftheuserisnotentitledtousetheobject.Tables/views/columnsrequiretheREADpermissionandproceduresrequiretheEXECUTEpermission.Allcolumnsofakeymustbeaccessiblefortheentrytobevisible.
Note Ifthereisanycachingofsystemmetadatawhendatarolesareinuse,thenvisibilityneedstobeconsidered.
SystemSchema
491
SYSSchemaSystemschemaforpublicinformationandactions.
TableofContentsTables/Views
SYS.ColumnsSYS.DataTypesSYS.KeyColumnsSYS.KeysSYS.ProcedureParamsSYS.ProceduresSYS.FunctionParamsSYS.FunctionsSYS.PropertiesSYS.ReferenceKeyColumnsSYS.SchemasSYS.TablesSYS.VirtualDatabasesSYS.spatial_sys_refSYS.GEOMETRY_COLUMNS
ProceduresSYS.getXMLSchemasSYS.ArrayIterate
Tables/Views
SYS.Columns
Thistablesuppliesinformationaboutalltheelements(columns,tags,attributes,etc)inthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
TableName string Tablename
Name string Elementname(notqualified)
Position integer Positioningroup(1-based)
NameInSource string Nameofelementinsource
DataType string Teiidruntimedatatypename
Scale integer Numberofdigitsafterthedecimalpoint
SYS
492
ElementLength integer Elementlength(mostlyusedforstrings)
sLengthFixed boolean Whetherthelengthisfixedorvariable
SupportsSelect boolean ElementcanbeusedinSELECT
SupportsUpdates boolean Valuescanbeinsertedorupdatedintheelement
IsCaseSensitive boolean Elementiscase-sensitive
IsSigned boolean Elementissignednumericvalue
IsCurrency boolean Elementrepresentsmonetaryvalue
IsAutoIncremented boolean Elementisauto-incrementedinthesource
NullType string Nullability:"Nullable","NoNulls","Unknown"
MinRange string Minimumvalue
MaxRange string Maximumvalue
DistinctCount integer Distinctvaluecount,-1canindicateunknown
NullCount integer Nullvaluecount,-1canindicateunknown
SearchType stringSearchability:"Searchable","AllExceptLike","LikeOnly",Unsearchable"
Format string Formatofstringvalue
DefaultValue string Defaultvalue
JavaClass string Javaclassthatwillbereturned
Precision integer Numberofdigitsinnumericvalue
CharOctetLength integer Measureofreturnvaluesize
Radix integer Radixfornumericvalues
GroupUpperName string Upper-casefullgroupname
UpperName string Upper-caseelementname
UID string ElementuniqueID
SYS
493
Description string Description
TableUID string ParentTableuniqueID
SYS.DataTypes
Thistablesuppliesinformationondatatypes.
ColumnName Type Description
Name string Teiiddesign-timetypename
IsStandard boolean Alwaysfalse
IsPhysical boolean Alwaysfalse
TypeName string Design-timetypename(sameasName)
JavaClass string Javaclassreturnedforthistype
Scale integer Maxscaleofthistype
TypeLength integer Maxlengthofthistype
NullType string Nullability:"Nullable","NoNulls","Unknown"
IsSigned boolean Issignednumeric?
IsAutoIncremented boolean Isauto-incremented?
IsCaseSensitive boolean Iscase-sensitive?
Precision integer Maxprecisionofthistype
Radix integer Radixofthistype
SearchType stringSearchability:"Searchable","AllExceptLike","LikeOnly","Unsearchable"
UID string DatatypeuniqueID
RuntimeType string Teiidruntimedatatypename
BaseType string Basetype
Description string Descriptionoftype
SYS.KeyColumns
SYS
494
Thistablesuppliesinformationaboutthecolumnsreferencedbyakey.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
TableName string Tablename
Name string Elementname
KeyName string Keyname
KeyType string Keytype:"Primary","Foreign","Unique",etc
RefKeyUID string ReferencedkeyUID
UID string KeyUID
Position integer Positioninkey
TableUID string ParentTableuniqueID
SYS.Keys
Thistablesuppliesinformationaboutprimary,foreign,anduniquekeys.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
TableName string Tablename
Name string Keyname
Description string Description
NameInSource string Nameofkeyinsourcesystem
Type string Typeofkey:"Primary","Foreign","Unique",etc
IsIndexed boolean Trueifkeyisindexed
RefKeyUID string ReferencedkeyUID(ifforeignkey)
UID string KeyuniqueID
SYS
495
SYS.ProcedureParams
Thissuppliesinformationonprocedureparameters.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
ProcedureName string Procedurename
Name string Parametername
DataType string Teiidruntimedatatypename
Position integer Positioninprocedureargs
Type string Parameterdirection:"In","Out","InOut","ResultSet","ReturnValue"
Optional boolean Parameterisoptional
Precision integer Precisionofparameter
TypeLength integer Lengthofparametervalue
Scale integer Scaleofparameter
Radix integer Radixofparameter
NullType string Nullability:"Nullable","NoNulls","Unknown"
Description string Descriptionofparameter
SYS.Procedures
Thistablesuppliesinformationabouttheproceduresinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Procedurename
NameInSource string Procedurenameinsourcesystem
ReturnsResults boolean Returnsaresultset
UID string ProcedureUID
SYS
496
Description string Description
SchemaUID string ParentSchemauniqueID
SYS.FunctionParams
Thissuppliesinformationonfunctionparameters.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
FunctionName string Functionname
FunctionUID string FunctionUID
Name string Parametername
DataType string Teiidruntimedatatypename
Position integer Positioninprocedureargs
Type string Parameterdirection:"In","Out","InOut","ResultSet","ReturnValue"
Precision integer Precisionofparameter
TypeLength integer Lengthofparametervalue
Scale integer Scaleofparameter
Radix integer Radixofparameter
NullType string Nullability:"Nullable","NoNulls","Unknown"
Description string Descriptionofparameter
SYS.Functions
Thistablesuppliesinformationaboutthefunctionsinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Functionname
NameInSource string
SYS
497
NameInSource string
UID string FunctionUID
Description string Description
IsVarArgs boolean Doesthefunctionacceptvariablearguments
SYS.Properties
Thistablesuppliesuser-definedpropertiesonallobjectsbasedonmetamodelextensions.Normally,thistableisemptyifnometamodelextensionsarebeingused.
ColumnName Type Description
Name string Extensionpropertyname
Value string Extensionpropertyvalue
UID string KeyuniqueID
ClobValue clob ClobValue
SYS.ReferenceKeyColumns
Thistablesuppliesinformatonaboutcolumn’skeyreference.
ColumnName Type Description
PKTABLE_CAT string VDBName
PKTABLE_SCHEM string SchemaName
PKTABLE_NAME string Table/ViewName
PKCOLUMN_NAME string ColumnName
FKTABLE_CAT string VDBName
FKTABLE_SCHEM string SchemaName
FKTABLE_NAME string Table/ViewName
FKCOLUMN_NAME string ColumnName
KEY_SEQ short KeySequence
UPDATE_RULE integer UpdateRule
DELETE_RULE integer DeleteRule
SYS
498
FK_NAME string FKName
PK_NAME string PKNmae
DEFERRABILITY integer
SYS.Schemas
Thistablesuppliesinformationaboutalltheschemasinthevirtualdatabase,includingthesystemschemaitself(System).
ColumnName Type Description
VDBName string VDBname
Name string Schemaname
IsPhysical boolean Trueifthisrepresentsasource
UID string UniqueID
Description string Description
PrimaryMetamodelURI stringURIfortheprimarymetamodeldescribingthemodelusedforthisschema
SYS.Tables
Thistablesuppliesinformationaboutallthegroups(tables,views,documents,etc)inthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Shortgroupname
Type string Tabletype(Table,View,Document,…)
NameInSource string Nameofthisgroupinthesource
IsPhysical boolean Trueifthisisasourcetable
SupportsUpdates boolean Trueifgroupcanbeupdated
UID string GroupuniqueID
Cardinality integer Approximatenumberofrowsinthegroup
Description string Description
SYS
499
IsSystem boolean Trueifinsystemtable
SchemaUID string ParentSchemauniqueID
SYS.VirtualDatabases
Thistablesuppliesinformationaboutthecurrentlyconnectedvirtualdatabase,ofwhichthereisalwaysexactlyone(inthecontextofaconnection).
ColumnName Type Description
Name string ThenameoftheVDB
Version string TheversionoftheVDB
SYS.spatial_sys_ref
SeealsothePostGISDocumentation
ColumnName Type Description
srid integer SpatialReferenceIdentifier
auth_name string Nameofthestandardorstandardsbody
auth_srid integer SRIDfortheauth_nameauthority
srtext string Well-KnownTextrepresentation
proj4text string ForusewiththeProj4library
SYS.GEOMETRY_COLUMNS
SeealsothePostGISDocumentation
ColumnName Type Description
F_TABLE_CATALOG string catalogname
F_TABLE_SCHEMA string schemaname
F_TABLE_NAME string tablename
F_GEOMETRY_COLUMN string columnname
COORD_DIMENSION integer Numberofcoordinatedimensions
SRID integer SpatialReferenceIdentifier
TYPE string Geometrytypename
SYS
500
Note:Thecoord_dimensionandsridpropertiesaredeterminedfromthe{http://www.teiid.org/translator/spatial/2015}coord_dimensionand{http://www.teiid.org/translator/spatial/2015}sridextensionpropertiesonthecolumn.Whenpossiblethesevalueswillbesetautomaticallybytherelevantimporter.Iftheyarenotset,theywillbereportedas2and0respectively.Ifclientlogicexpectsactualvalues,suchasintegrationwithGeoServer,thenyoumayneedtosetthesevaluesmanually.
Procedures
SYS.getXMLSchemas
Returnsaresultsetwithasinglecolumn,schema,containingtheschemasasxml.
SYS.getXMLSchemas(INdocumentstringNOTNULL)RETURNSTABLE(schemaxml)
SYS.ArrayIterate
Returnsaresultsetwithasinglecolumnwitharowforeachvalueinthearray.
SYS.ArrayIterate(INvalobject[])RETURNSTABLE(colobject)
ExampleArrayIterate
selectarray_get(cast(x.colasstring[]),2)from(execarrayiterate((('a','b'),('c','d'))))x
Thiswillproducetworows-'b',and'd'.
SYS
501
SYSADMINSchemaSystemschemaforadministrativeinformationandactions.
TableofContentsTables/Views
SYSADMIN.UsageSYSADMIN.MatViewsSYSADMIN.VDBResourcesSYSADMIN.TriggersSYSADMIN.ViewsSYSADMIN.StoredProcedures
ProceduresSYSADMIN.isLoggableSYSADMIN.logMsgSYSADMIN.refreshMatViewSYSADMIN.refreshMatViewRowSYSADMIN.refreshMatViewRowsSYSADMIN.setColumnStatsSYSADMIN.setPropertySYSADMIN.setTableStatsSYSADMIN.matViewStatusSYSADMIN.loadMatViewSYSADMIN.updateMatView
Tables/Views
SYSADMIN.Usage
Thistablesuppliesinformationabouthowviews/proceduresaredefined.
ColumnName Type Description
VDBName string VDBname
UID string ObjectUID
object_type string Typeofobject(StoredProcedure,View,etc.)
Name string ObjectNameorparentnameifElementNameisnotnull
ElementName string Nameofcolumnorparameter
Uses_UID string UsedobjectUID
Uses_object_type string Usedobjecttype
Uses_SchemaName string Usedobjectschema
SYSADMIN
502
Uses_Name string UsedobjectnameorparentnameifUses_ElementNameisnotnull
Uses_ElementName string Usedcolumnorparametername
ExampleSYSADMIN.Usage
SELECT*FROMSYSADMIN.Usage
Recursivecommontablequeriescanbeusedtodeterminetransitiverelationships.
ExampleFindingAllIncomingUsage
withim_usingas(
select0aslevel,uid,Uses_UID,Uses_Name,Uses_Object_Type,Uses_ElementName
fromusagewhereuid=(selectuidfromsys.tableswherename='tablename'andschemaName='schemaname')
unionall
selectlevel+1,usage.uid,usage.Uses_UID,usage.Uses_Name,usage.Uses_Object_Type,usage.Uses_ElementNam
e
fromusage,im_usingwherelevel<10andusage.uid=im_using.Uses_UID)select*fromim_using
ExampleFindingAllOutgoingUsage
withuses_meas(
select0aslevel,uid,Uses_UID,Name,Object_Type,ElementName
fromusagewhereuses_uid=(selectuidfromsys.tableswherename='tablename'andschemaName='schemana
me')
unionall
selectlevel+1,usage.uid,usage.Uses_UID,usage.Name,usage.Object_Type,usage.ElementName
fromusage,uses_mewherelevel<10andusage.uses_uid=uses_me.UID)select*fromuses_me
SYSADMIN.MatViews
Thistablesuppliesinformationaboutallthematerailizedviewsinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Shortgroupname
TargetSchemaName stringNameofthematerializedtableschema.Willbenullforinternalmaterialization.
TargetName string Nameofthematerializedtable
Valid booleanTrueifmaterializedtableiscurrentlyvalid.Willbenullforexternalmaterialization.
LoadState boolean
Theloadstate,canbeoneofNEEDS_LOADING,LOADING,LOADED,FAILED_LOAD.Willbenullforexternalmaterialization.
SYSADMIN
503
Updated timestamp Thetimestampofthelastfullrefresh.Willbenullforexternalmaterialization.
Cardinality integerThenumberofrowsinthematerializedviewtable.Willbenullforexternalmaterialization.
Valid,LoadState,Updated,andCardinalitymaybecheckedforexternalmaterializedviewswiththeSYSADMIN.matViewStatusprocedure.
ExampleSYSADMIN.MatViews
SELECT*FROMSYSADMIN.MatViews
SYSADMIN.VDBResources
ThistableprovidesthecurrentVDBcontents.
ColumnName Type Description
resourcePath string Thepathtothecontents.
contents blob Thecontentsasablob.
ExampleSYSADMIN.VDBResources
SELECT*FROMSYSADMIN.VDBResources
SYSADMIN.Triggers
ThistableprovidestheTriggersinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
TableName string Tablename
Name string Triggername
TriggerType string TriggerType
TriggerEvent string TriggeringEvent
Status string IsEnabled
Body clob TriggerAction(FOREACHROW…)
TableUID string TableUniqueID
ExampleSYSADMIN.Triggers
SYSADMIN
504
SELECT*FROMSYSADMIN.Triggers
SYSADMIN.Views
ThistableprovidestheViewsinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Viewname
Body clob ViewDefinitionBody(SELECT…)
UID string TableUniqueID
ExampleSYSADMIN.Views
SELECT*FROMSYSADMIN.Views
SYSADMIN.StoredProcedures
ThistableprovidestheStoredProceduresinthevirtualdatabase.
ColumnName Type Description
VDBName string VDBname
SchemaName string SchemaName
Name string Procedurename
Body clob ProcedureDefinitionBody(BEGIN…)
UID string UniqueID
ExampleSYSADMIN.StoredProcedures
SELECT*FROMSYSADMIN.StoredProcedures
Procedures
SYSADMIN.isLoggable
Testsifloggingisenabledatthegivenlevelandcontext.
SYSADMIN.isLoggable(OUTloggablebooleanNOTNULLRESULT,INlevelstringNOTNULLDEFAULT'DEBUG',INcontext
stringNOTNULLDEFAULT'org.teiid.PROCESSOR')
SYSADMIN
505
Returnstrueifloggingisenabled.levelcanbeoneofthelog4jlevels:OFF,FATAL,ERROR,WARN,INFO,DEBUG,TRACE.leveldefaultsto'DEBUG'andcontextdefaultsto'org.teiid.PROCESSOR'
ExampleisLoggable
IF((CALLSYSADMIN.isLoggable(context=>'org.something'))
BEGIN
DECLARESTRINGmsg;
//logictobuildthemessage...
CALLSYSADMIN.logMsg(msg=>msg,context=>'org.something')
END
SYSADMIN.logMsg
Logamessagetotheunderlyingloggingsystem.
SYSADMIN.logMsg(OUTloggedbooleanNOTNULLRESULT,INlevelstringNOTNULLDEFAULT'DEBUG',INcontextstring
NOTNULLDEFAULT'org.teiid.PROCESSOR',INmsgobjectNOTNULL)
Returnstrueifthemessagewaslogged.levelcanbeoneofthelog4jlevels:OFF,FATAL,ERROR,WARN,INFO,DEBUG,TRACE.leveldefaultsto'DEBUG'andcontextdefaultsto'org.teiid.PROCESSOR'
ExamplelogMsg
CALLSYSADMIN.logMsg(msg=>'somedebug',context=>'org.something')
Thiswilllogthemessage'somedebug'atthedefaultlevelDEBUGtothecontextorg.something.
SYSADMIN.refreshMatView
Fullrefresh/loadofaninternalmaterializedview.ReturnsintegerRowsUpdated.-1indicatesaloadisinprogress,otherwisethecardinalityofthetableisreturned.SeetheCachingGuideformore.
SeealsoSYSADMIN.loadMatView
SYSADMIN.refreshMatView(OUTRowsUpdatedintegerNOTNULLRESULT,INViewNamestringNOTNULL,INInvalidateboo
leanNOTNULLDEFAULT'false')
SYSADMIN.refreshMatViewRow
Refreshesarowinaninternalmaterializedview.
ReturnsintegerRowsUpdated.-1indicatesthematerializedtableiscurrentlyinvalid.0indicatesthatthespecifiedrowdidnotexistinthelivedataqueryorinthematerializedtable.SeetheCachingGuideformore.
SYSADMIN.CREATEFOREIGNPROCEDURErefreshMatViewRow(OUTRowsUpdatedintegerNOTNULLRESULT,INViewNamestring
NOTNULL,INKeyobjectNOTNULL,VARIADICKeyOtherobject)
ExampleofSYSADMIN.refreshMatViewRow
ThematerializedviewSAMPLEMATVIEWhas3rowsundertheTestMatModelasbelow:
SYSADMIN
506
Assumingtheprimarykeyonlycontainsonecolumn,id,updatethesecondrow:
EXECSYSADMIN.refreshMatViewRow('TestMat.SAMPLEMATVIEW','101')
Assumingtheprimarykeycontainsmorecolumns,aandb,updatethesecondrow:
EXECSYSADMIN.refreshMatViewRow('TestMat.SAMPLEMATVIEW','101','a1','b1')
SYSADMIN.refreshMatViewRows
Refreshesrowsinaninternalmaterializedview.
ReturnsintegerRowsUpdated.-1indicatesthematerializedtableiscurrentlyinvalid.AnyrowthatdoesnotexistinthelivedataqueryorinthematerializedtablewillnotcounttowardtheRowsUpdated.SeetheCachingGuideformore.
SYSADMIN.refreshMatViewRows(OUTRowsUpdatedintegerNOTNULLRESULT,INViewNamestringNOTNULL,VARIADICKey
object[]NOTNULL)
ExampleofSYSADMIN.refreshMatViewRows
ContinuingusetheSAMPLEMATVIEWinExampleofSYSADMIN.refreshMatViewRow.Assumingtheprimarykeyonlycontainsonecolumn,id,updateallrows:
EXECSYSADMIN.refreshMatViewRows('TestMat.SAMPLEMATVIEW',('100',),('101',),('102',))
Assumingtheprimarykeycomtainmorecolumns,id,aandbcomposeoftheprimarykey,updateallrows:
EXECSYSADMIN.refreshMatViewRows('TestMat.SAMPLEMATVIEW',('100','a0','b0'),('101','a1','b1'),('102','a2
','b2'))
SYSADMIN.setColumnStats
Setstatisticsforthegivencolumn.
SYSADMIN.setColumnStats(INtableNamestringNOTNULL,INcolumnNamestringNOTNULL,INdistinctCountlong,IN
nullCountlong,INmaxstring,INminstring)
Allstatvaluesarenullable.Passinganullstatvaluewillleavecorrespondingmetadatavalueunchanged.
SYSADMIN.setProperty
Setanextensionmetadatapropertyforthegivenrecord.ExtensionmetadataistypicallyusedbyTranslators.
SYSADMIN.setProperty(OUTOldValueclobNOTNULLRESULT,INUIDstringNOTNULL,INNamestringNOTNULL,IN"Va
lue"clob)
SYSADMIN
507
Settingavaluetonullwillremovetheproperty.
ExamplePropertySet
CALLSYSADMIN.setProperty(uid=>(SELECTuidFROMTABLESWHEREname='tab'),name=>'somename',value=>'somevalue'
)
Thiswillsettheproperty'somename'='somevalue'ontabletab.
Note Theuseofthisprocedurewillnottriggerreplanningofassociatedpreparedplans.
Propertiesfrombuilt-inteiid_*namespacescanbesetusingthetheshortform-namespace:keyform.
SYSADMIN.setTableStats
Setstatisticsforthegiventable.
SYSADMIN.setTableStats(INtableNamestringNOTNULL,INcardinalitylongNOTNULL)
NoteSYSADMIN.setColumnStats,SYSADMIN.setProperty,SYSADMIN.setTableStatsareMetadataProcedures.AMetadataRepositorymustbeconfiguredtomakeanon-temporarymetadataupdatepersistent.SeetheDeveloper’sGuideRuntimeMetadataUpdatessectionformore.
SYSADMIN.matViewStatus
matViewStatusisusedtoretrieveMaterializedviews'statusviaschemaNameandviewName.
ReturnstableswhichcontainsTargetSchemaName,TargetName,Valid,LoadState,Updated,Cardinality,LoadNumber,OnErrorAction.
SYSADMIN.matViewStatus(INschemaNamestringNOTNULL,INviewNamestringNOTNULL)RETURNSTABLE(TargetSchemaN
amevarchar(50),TargetNamevarchar(50),Validboolean,LoadStatevarchar(25),Updatedtimestamp,Cardinalityl
ong,LoadNumberlong,OnErrorActionvarchar(25))
SYSADMIN.loadMatView
loadMatViewisusedtoperformacompleterefreshofaninternalorexternalmaterializedtable.
ReturnsintegerRowsInserted.-1indicatesthematerializedtableiscurrentlyloading.And-3indicatestherewasanexceptionwhenperformingtheload.SeetheCachingGuideformore.
SYSADMIN.loadMatView(INschemaNamestringNOTNULL,INviewNamestringNOTNULL,INinvalidatebooleanNOTNULL
DEFAULT'false')RETURNSinteger
ExampleloadMatView
execSYSADMIN.loadMatView(schemaName=>'TestMat',viewname=>'SAMPLEMATVIEW',invalidate=>'true')
SYSADMIN.updateMatView
TheupdateMatViewprocedureisusedtoupdateasubsetofaninternalorexternalmaterializedtablebasedontherefreshcriteria.
SYSADMIN
508
ReturnsintegerRowsUpdated.-1indicatesthematerializedtableiscurrentlyinvalid.And-3indicatestherewasanexceptionwhenperformingtheupdate.SeetheCachingGuideformore.
SYSADMIN.updateMatView(INschemaNamestringNOTNULL,INviewNamestringNOTNULL,INrefreshCriteriastring)R
ETURNSinteger
SYSADMIN.updateMatView
ContinuingusetheSAMPLEMATVIEWinExampleofSYSADMIN.refreshMatViewRow.Updateviewrows:
EXECSYSADMIN.updateMatView('TestMat','SAMPLEMATVIEW','id=''101''ANDa=''a1''')
SYSADMIN
509
TranslatorsTheTeiidConnectorArchitecture(TCA)providesTeiidwitharobustmechanismforintegratingwithexternalsystems.TheTCAdefinesacommonclientinterfacebetweenTeiidandanexternalsystemthatincludesmetadataastowhatSQLconstructsaresupportedforpushdownandtheabilitytoimportmetadatafromtheexternalsystem.
ATranslatoristheheartoftheTCAandactsasthebridgelogicbetweenTeiidandanexternalsystem,whichismostcommonlyaccessedthroughaJCAresourceadapter.RefertotheTeiidDevelopersGuidefordetailsondevelopingcustomTranslatorsandJCAresourceadaptersforusewithTeiid.
Tip TheTCAisnotthesameastheJCA,theJavaEEConnectorArchitecture,althoughtheTCAisdesignedforusewithJCAresourceadapters.
Tip TheimportcapabilitiesofTeiidTranslatorscanbeutilizedinTeiidDesignerviatheTeiidConnectionImporter.
ATranslatoristypicallypairedwithaparticularJCAresourceadapter.Ininstanceswherepooling,environmentdependentconfigurationmanagement,advancedsecurityhandling,etc.arenotneeded,thenaJCAresourceadapterisnotneeded.TheconfigurationofJCAConnectionFactoriesforneededresourceadaptersisnotpartofthisguide,pleaseseetheTeiidAdministratorGuideandthekitexamplesforconfiguringresourceadaptersforuseinWildFly.
Translatorscanhaveanumberofconfigurableproperties.Thesearebrokendownintoexecutionproperties,whichdetermineaspectsofhowdataisretrieved,andimportsettings,whichdeterminewhatmetadataisreadforimport.
Theexecutionpropertiesforatranslatortypicallyhavereasonabledefaults.Forspecifictranslatortypes,e.g.theDerbytranslator,baseexecutionpropertiesarealreadytunedtomatchthesource.Inmostcasestheuserwillnotneedtoadjusttheirvalues.
Table1.BaseExecutionProperties-sharedbyalltranslators
Name Description Default
Immutable Settotruetoindicatethatthesourceneverchanges. false
RequiresCriteriaSettotruetoindicatethatsourceSELECT/UPDATE/DELETEqueriesrequireawhereclause.
false
SupportsOrderBy SettotruetoindicatethattheORDERBYclauseissupported. false
SupportsOuterJoins SettotruetoindicatethatOUTERJOINsaresupported. false
SupportsFullOuterJoinsIfouterjoinsaresupported,trueindicatesthatFULLOUTERJOINsaresupported.
false
SupportsInnerJoins SettotruetoindicatethatINNERJOINsaresupported. false
SupportedJoinCriteria
Ifjoinsaresupported,defineswhatcriteriamaybeusedasthejoincriteria.Maybeoneof(ANY,THETA,EQUI,orKEY).
ANY
Translators
510
MaxInCriteriaSize Ifincriteriaaresupported,defineswhatthemaximumnumberofinentriesareperpredicate.-1indicatesnolimit.
-1
MaxDependentInPredicates
Ifincriteriaaresupported,defineswhatthemaximumnumberofpredicatesthatcanbeusedforadependentjoin.Valueslessthan1indicatetouseonlyoneinpredicateperdependentvaluepushed(whichmatchesthepre-7.4behavior).
-1
DirectQueryProcedureName
ifthedirectqueryprocedureissupportedonthetranslator,thispropertyindicatesthenameoftheprocedure.
native
SupportsDirectQueryProcedureSettotruetoindicatethetranslatorsupportsthedirectexecutionofcommands
false
ThreadBoundSettotruetoindicatethetranslator’sExecutionsshouldbeprocessedbyonlyasinglethread
false
CopyLobs
Iftrue,thenreturnedlobs(clob,blob,sql/xml)willbecopiedbytheengineinamemorysafemanner.Usethisoptionifthesourcedoesnotsupportmemorysafelobsoryouwanttodisconnectlobsfromthesourceconnection.
false
TransactionSupport
Thehighestleveloftransactionsupport.UsedbytheengineasahinttodetermineifatransactionisneededforautoCommitTxn=DETECTmode.CanbeoneofXA,NONE,orLOCAL.IfXA,thenaccessunderatransactionwillbeserialized.
XA
Note Onlyasubsetofthesupportsmetadatacanbesetthroughexecutionproperties.Ifmorecontrolisneeded,pleaseconsulttheDeveloper’sGuide.
Therearenobaseimportersettings.
OverrideExecutionPropertiesForallthetranslatorstooverrideExecutionPropertiescanbeconfiguredinthevdb.xmlfile.
ExampleOverridingofTranslatorProperty
<modelname="ora">
<sourcename="ora"translator-name="oracle-override"connection-jndi-name="java:/oracle"/>
</model>
<translatorname="oracle-override"type="oracle">
<propertyname="RequiresCriteria"value="true"/>
Translators
511
</translator>
TheaboveXMLfragmentisoverridingtheoracletranslatorandalteringthebehaviorofRequiresCriteriapropertytotrue.NotethatthemodifiedtranslatorisonlyavailableinthescopeofthisVDB.
SeealsoVDBDefinition.
ParameterizableNativeQueriesInsomesituationstheteiid_rel:native-querypropertyandnativeproceduresacceptparameterizablestringsthatcanpositionallyreferenceINparameters.Aparameterreferencehastheform$integer,i.e.$1Notethat1basedindexingisusedandthatonlyINparametersmaybereferenced.Dollar-signintegeristhereforereserved,butmaybeescapedwithanother$,i.e.$$1.Thevaluewillbeboundasapreparedvalueoraliteralisasourcespecificmanner.Thenativequerymustreturnaresultsetthatmatchestheexpectationofthecallingprocedure.
Forexamplethenative-queryselectcfromgwherec1=$1andc2='$$1'resultsinaJDBCsourcequeryofselectcfromgwherec1=?andc2='$1',where?willbereplacedwiththeactualvalueboundtoparameter1.
Translators
512
AmazonSimpleDBTranslatorTheAmazonSimpleDBTranslator,knownbythetypenamesimpledb,exposesqueryingfunctionalitytoAmazonSimpleDBDataSources.
Note
"AmazonSimpleDB"-AmazonSimpleDBisawebserviceforrunningqueriesonstructureddatainrealtime.ThisserviceworksincloseconjunctionwithAmazonSimpleStorageService(AmazonS3)andAmazonElasticComputeCloud(AmazonEC2),collectivelyprovidingtheabilitytostore,processandquerydatasetsinthecloud.Theseservicesaredesignedtomakeweb-scalecomputingeasierandmorecost-effectivefordevelopers.Readmoreaboutitathttp://aws.amazon.com/simpledb/
ThistranslatorprovidesaneasywayconnecttoAmazonSimpleDBandprovidesrelationalwayusingSQLtoaddrecordsfromdirectlyfromuserorfromothersourcesthatareintegratedwithTeiid.Italsogivesabilitytoread/update/deleteexistingrecordsfromSimpleDBstore.
UsageAmazonSimpleDBishostedkey/valuestorewhereasinglekeycancontainhostmultipleattributename/valuepairswherewherevaluecanalsobeamulti-value.Thedatastructurecanberepresentedby
Basedonabovedatastructure,whenyouimportthemetadatafromSimpleDBintoTeiid,theconstructsarealignedasbelow
SimpleDBName SQL(Teiid)
Domain Table
ItemName Column(ItemName)PrimaryKey
attribute-singlevalue Column-StringDatatype
attribute-multivalue Column-StringArrayDatatype
Sinceallattributesarebydefaultareconsideredasstringdatatypes,columnsaredefinedwithstringdatatype.However,duringmodelingoftheschemainDesigner,onecanusevariousotherdatatypessupportedthroughTeiidtodefineadatatypeofcolumn,thatuserwishestoexposeas.
NoteIfyoudidmodifydatatypebeotherthanstringbased,becautionedanddonotusethosecolumnsincomparisonqueries,asSimpleDBdoesonlylexicographicalmatching.Toavoidit,setthe"SearchType"onthatcolumnto"UnSearchable".
AnExampleVDBthatshowsSimpleDBtranslatorcanbedefinedas
AmazonSimpleDBTranslator
513
<vdbname="myvdb"version="1">
<modelname="simpledb">
<sourcename="node"translator-name="simpledb"connection-jndi-name="java:/simpledbDS"/>
</model>
<vdb>
ThetranslatordoesNOTprovideaconnectiontotheSimpleDB.Forthatpurpose,TeiidhasaJCAadapterthatprovidesaconnectiontoSimpleDBusingAmazonSDKJavalibraries.Todefinesuchconnector,seeAmazonSimpleDBDataSourcesorseeanexamplein"<jboss-as>/docs/teiid/datasources/simpledb"
IfyouareusingDesignerTooling,tocreateVDB
Create/useaTeiidDesignerModelproject
Use"TeiidConnection>>SourceModel"importer,createSimpleDBDataSourceusingdatasourcecreationwizardandusesimpledbastranslatorintheimporter.Thetable(s)iscreatedinasourcemodelbythetimeyoufinishwiththisimporter,ifthedataisalreadydefinedonAmazonSimpleDB.
CreateaVDBanddeployintoTeiidServeranduseeitherjdbc,odbc,odataetctoquery.
PropertiesTheAmazonSimpleDBTranslatorcurrentlyhasnoimportorexecutionproperties.
CapabilitiesTheAmazonSimpleDBTranslatorsupportsSELECTstatementswitharestrictivesetofcapabilitiesincluding:comparisonpredicates,INpredicates,LIMITandORDERBY.Insert,update,deletearealsosupported.
QueriesonAttributeswithMultipleValues
Attributeswithmultiplevalueswilldefinedasstringarraytype.SothiscolumnistreatedSQLArraytype.ThebelowtableshowsSimpleDBwayofqueryingtoTeiidwaytoquery.Thequeriesarebasedonhttp://docs.aws.amazon.com/AmazonSimpleDB/latest/DeveloperGuide/RangeValueQueriesSelect.html
SimpleDBQuery TeiidQuery
select*frommydomainwhereRating='4stars'orRating='**' select*frommydomainwhereRating=('4stars','**')
select*frommydomainwhereKeyword='Book'andKeyword='Hardcover'
select*frommydomainwhereintersection(Keyword,'Book','Hardcover')
select*frommydomainwhereevery(Rating)='**' select*frommydomainwhereevery(Rating)='**'
WithInsert/Update/DeleteyouwritepreparestatementsoryoucanwriteSQLlike
INSERTINTOmydomain(ItemName,title,author,year,pages,keyword,rating)values('0385333498','TheSirens
ofTitan','KurtVonnegut',('1959'),('Book',Paperback'),('*****','5stars','Excellent'))
DirectQuerySupport
Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthe
AmazonSimpleDBTranslator
514
source.Toenablethisfeature,overridetheexecutionpropertycalledSupportsDirectQueryProceduretotrue.
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyiscallednative.OverridetheexecutionpropertyDirectQueryProcedureNametochangeittoanothername.
TheSimpleDBtranslatorprovidesaproceduretoexecuteanyad-hocsimpledbquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.Directquerysupportedfor"select"basedcalls.
SELECTX.*
FROMsimpledb_source.native('SELECTfirstname,lastnameFROMusers')n,ARRAYTABLE(n.tupleCOLUMNSfirstname
string,lastnamestring)ASX
JCAResourceAdapter
TheTeiidspecificAmazonSimpleDBResourceAdaptershouldbeusedwiththistranslator.SeeAmazonSimpleDBDataSourcesforconnectingtoSimpleDB.
AmazonSimpleDBTranslator
515
ApacheAccumuloTranslatorTheApacheAccumuloTranslator,knownbythetypenameaccumulo,exposesqueryingfunctionalitytoAccumuloDataSources.ApacheAccumuloisasorted,distributedkeyvaluestorewithrobust,scalable,highperformancedatastorageandretrievalsystem.ThistranslatorprovidesaneasywayconnecttoAccumulosystemandprovidesrelationalwayusingSQLtoaddrecordsfromdirectlyfromuserorfromothersourcesthatareintegratedwithTeiid.Italsogivesabilitytoread/update/deleteexistingrecordsfromAccumulostore.Teiidhascapabilitytopass-inloggedinuser’srolesasvisibilitypropertiestorestrictthedataaccess.
Tip "versions"-ThedevelopmentwasdoneusingAccumulo1.5.0,Hadoop2.2.0andZookeeper3.4.5
Note ThisdocumentassumesthatuserisfamiliarwithAccumulosourceandhasbasicunderstandingofhowTeiidworks.ThisdocumentonlycontainsdetailsaboutAccumulotranslator.
IntendedUsecasesTheusageAccumulotranslatorcanbehighlydependentonuser’susecase(s).Herearesomecommonscenarios.
AccumulosourcecanbeusedinTeiid,tocontinuallyadd/updatethedocumentsintheAccumulosystemfromothersourcesautomatically.
AccessAccumulothroughSQLinterface.
Makeuseofcelllevelsecuritythroughenterpriseroles.
AccumulotranslatorcanbeusedasanindexingsystemtogatherdatafromotherenterprisesourcessuchasRDBMS,WebService,SalesForceetc,allinsingleclientcalltransparentlywithoutanycoding.
Usage
ApacheAccumuloisdistributedkeyvaluestorewithuniquedatamodel.Itallowstogroupitskey-valuepairsinacollectioncalled"table".Thekeystructureisdefinedas
Basedonaboveinformation,onecandefineaschemarepresentingAccumulotablestructuresinTeiidusingDDLorusingTeiidDesignerwithhelpofmetadataextensionpropertiesdefinedbelow.Sincenodatatypeinformationisdefinedonthecolumns,bydefaultallcolumnsareconsideredasstringdatatypes.However,duringmodelingoftheschema,onecanusevariousotherdatatypessupportedthroughTeiidtodefineadatatypeofcolumn,thatuserwishestoexposeas.
OncethisschemaisdefinedandexposedthroughVDBinaTeiiddatabase,andAccumuloDataSourcesiscreated,theusercanissue"INSERT/UPDATE/DELETE"basedSQLcallstoinsert/update/deleterecordsintotheAccumulo,andissue"SELECT"basedcallstoretrieverecordsfromAccumulo.YoucanusefullrangeofSQLwithTeiidsystemintegratingothersourcesalongwithAccumulosource.
ApacheAccumuloTranslator
516
Bydefault,Accumulotablestructureisflatcannotdefinerelationshipsamongtables.So,aSQLJOINisperformedinTeiidlayerratherthanpushedtosourceevenifbothtablesoneithersideoftheJOINresideintheAccumulo.CurrentlyanycriteriabasedonEQUALITYand/orCOMPARISONusingcomplexAND/ORclausesarehandledbyAccumulotranslatorandwillbeproperlyexecutedatsource.
AnExampleVDBthatshowsAccumulotranslatorcanbedefinedas
<vdbname="myvdb"version="1">
<modelname="accumulo">
<sourcename="node-one"translator-name="accumulo"connection-jndi-name="java:/accumuloDS"/>
</model>
<vdb>
ThetranslatordoesNOTprovideaconnectiontotheAccumulo.Forthatpurpose,TeiidhasaJCAadapterthatprovidesaconnectiontoAccumulousingAccumuloJavalibraries.Todefinesuchconnector,seeAccumuloDataSourcesorseeanexamplein"<jboss-as>/docs/teiid/datasources/accumulo"
IfyouareusingDesignerTooling,tocreateVDB
Create/useaTeiidDesignerModelproject
Use"TeiidConnection>>SourceModel"importer,createAccumuloDataSourceusingdatasourcecreationwizardanduseaccumuloastranslatorintheimporter.Thetableiscreatedinasourcemodelbythetimeyoufinishwiththisimporter.
CreateaVDBanddeployintoTeiidServeranduseeitherjdbc,odbc,odataetctoquery.
PropertiesAccumulotranslatoriscapableoftraversingthroughAccumulotablestructuresandbuildametadatastructureforTeiidtranslator.TheschemaimportercanunderstandsimpletablesbytraversingasingleROWIDofdata,thenlooksforalltheuniquekeys,basedonititcomesupwithatabularstructureforAccumulobasedtable.Usingthefollowingimportproperties,youcanfurtherrefinetheimportbehavior.
ImportProperties
PropertyName Description Required Default
ColumnNamePattern Howthecolumnnameshouldbeformed false {CF}_{CQ}
ValueInWherethevalueforcolumnisdefinedCQorVALUE
false {VALUE}
Note{CQ},{CF},{ROWID}areexpressionsthatyoucanusetodefineabovepropertiesinanypattern,andrespectivevaluesofColumnQualifer,ColumnFamiliyorROWIDwillbereplacedatimporttime.ROWIDoftheAccumulotable,isautomaticallycreatedasROWIDcolumn,andwillbedefinedasPrimaryKeyonthetable.
YoucanalsodefinethemetadatafortheAccumulobasedmodel,usingDDLorusingtheTeiidDesigner.Whendoingsuchexercise,theAccumuloTranslatorcurrentlydefinesfollowingextendedmetadatapropertiestobedefinedonitsTeiidschemamodeltoguidethetranslatortomakeproperdecisions.ThefollowingpropertiesaredescribedunderNAMESPACE"http://www.teiid.org/translator/accumulo/2013",foruserconveniencethisnamespacehasaliasnameteiid_accumulodefindinTeiid.Todefineaextensionpropertyuseexpressionlike"teiid_accumulo:{property-name}value".AllthepropertiesbelowareintendedtobeusedasOPTIONpropertiesonCOLUMNS.SeeDDLMetadataformoreinformationondefiningDDLbasedmetadata.
ApacheAccumuloTranslator
517
ExtensionMetadataProperties
PropertyName Description Required Default
CF ColumnFamily true none
CQ ColumnQualifier false empty
VALUE-INValueofcolumndefinedin.Possiblevalues(VALUE,CQ)
false VALUE
HowtouseaboveProperties
Sayforexampleyouhaveatablecalled"User"inyourAccumuloinstance,anddoingascanreturnedfollowingdata
root@teiid>tableUser
root@teiidUser>scan
1name:age[]43
1name:firstname[]John
1name:lastname[]Does
2name:age[]10
2name:firstname[]Jane
2name:lastname[]Smith
3name:age[]13
3name:firstname[]Mike
3name:lastname[]Davis
IfyouusedthedefaultimporterfromtheAccumulotranslator(liketheVDBdefinedabove),thetablegeneratedwillbelikebelow
CREATEFOREIGNTABLE"User"(
rowidstringOPTIONS(UPDATABLEFALSE,SEARCHABLE'All_Except_Like'),
name_agestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"'age'
,"teiid_accumulo:VALUE-IN"'{VALUE}'),
name_firstnamestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"
'firstname',"teiid_accumulo:VALUE-IN"'{VALUE}'),
name_lastnamestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"
'lastname',"teiid_accumulo:VALUE-IN"'{VALUE}'),
CONSTRAINTPK0PRIMARYKEY(rowid)
)OPTIONS(UPDATABLETRUE);
Youcanuse"ImportProperty"as"ColumnNamePattern"as"{CQ}"willgeneratethefollowing(notethenamesofthecolumn)
CREATEFOREIGNTABLE"User"(
rowidstringOPTIONS(UPDATABLEFALSE,SEARCHABLE'All_Except_Like'),
agestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"'age',"t
eiid_accumulo:VALUE-IN"'{VALUE}'),
firstnamestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"'fi
rstname',"teiid_accumulo:VALUE-IN"'{VALUE}'),
lastnamestringOPTIONS(SEARCHABLE'All_Except_Like',"teiid_accumulo:CF"'name',"teiid_accumulo:CQ"'las
tname',"teiid_accumulo:VALUE-IN"'{VALUE}'),
CONSTRAINTPK0PRIMARYKEY(rowid)
)OPTIONS(UPDATABLETRUE);
ApacheAccumuloTranslator
518
respectivelyifthecolumnnameisdefinedbyColumnFamily,youcanuse"ColumnNamePattern"as"{CF}",andifthevalueforthatcolumnexistsintheColumnQualifierthenyoucanuse"ValueIn"as"{CQ}".Usingimportpropertiesyoucandictatehowthetableshouldbemodeled.
Ifyoudidnotusebuiltinimport,andwouldliketomanuallydesignthetableinDesignerlikebelow
ThenyoumustmakesureyousupplytheExtensionMetadataPropertiesdefinedaboveontheUsertable’scolumnsfromAccumuloextendedmetadata(InDesigner,rightclickonModel,andselect"ModelExtensionDefinitions"andselectAccumulo.ForexampleonFirstNamecolumn,youwouldsupply
teiid_accumulo:CFname
teiid_accumulo:CQfirstname
teiid_accumulo:VALUE-INVALUE
andrepeatforeachandeverycolumn,sothatTeiidknowshowtocommunicatecorrectlywithAccumulo.
JCAResourceAdapterTheTeiidspecificAccumuloResourceAdaptershouldbeusedwiththistranslator.SeeAccumuloDataSourcesforconnectingtoaAccumuloSource.
NativeQueriesCurrentlythisfeatureisnotapplicable.BasedonuserdemandTeiidcouldexposeawayforusertosubmitaMAP-REDUCEjob.
DirectQueryProcedureThisfeatureisnotapplicableforthistranslator.
ApacheAccumuloTranslator
519
ApacheSOLRTranslatorTheApacheSOLRTranslator,knownbythetypenamesolr,exposesqueryingfunctionalitytoSolrDataSources.ApacheSolrisasearchenginebuiltontopofApacheLuceneforindexingandsearching.ThistranslatorprovidesaneasywayconnecttoexistingoranewSolrsearchsystem,andprovideswaytoadddocuments/recordsfromdirectlyfromuserorfromothersourcesthatareintegratedwithTeiid.Italsogivesabilitytoread/update/deleteexistingdocumentsfromSolrSearchsystem.
Properties
TheSolrTranslatorcurrentlyhasnoimportorexecutionproperties.Itdoesnotdefineanyextensionmetadata.
IntendedUsecases
TheusageSolrtranslatorcanbehighlydependentonuser’susecase(s).Herearesomecommonscenarios.
SolrsourcecanbeusedinTeiid,tocontinuallyadd/updatethedocumentsinthesearchsystemfromothersourcesautomatically.
IfthesearchfieldsarestoredinSolrsystem,thiscanbeusedasverylowlatencydataretrievalforservinghightrafficapplications.
Solrtranslatorcanbeusedasafastfulltextsearch.TheSolrdocumentcancontainonlytheindexinformation,thentheresultsasaninvertedindextogathertargetfulldocumentsfromtheotherenterprisesourcessuchasRDBMS,WebService,SalesForceetc,allinsingleclientcalltransparentlywithoutanycoding.
Usage
Solrsearchsystemprovidessearchesbasedonindexedsearchfields.EachSolrinstanceistypicallyconfiguredwithasinglecorethatdefinesmultiplefieldswithdifferenttypeinformation.Teiidmetadataqueryingmechanismisequippedwith"Luke"basedqueries,thatatdeploytimeoftheVDBusethismechanismtoretrieveallthestored/indexedfields.CurrentlyTeiiddoesNOTsupportdynamicfieldsandnon-storedfields.Basedonretrievedfields,Solrtranslatorexposesasingletablethatcontainsallthefields.Ifafieldismulti-valuebased,it’stypeisrepresentedasArraytype.
OncethistableisexposedthroughVDBinaTeiiddatabase,andSolrDataSourcesiscreated,theusercanissue"INSERT/UPDATE/DELETE"basedSQLcallstoinsert/update/deletedocumentsintotheSolr,andissue"SELECT"basedcallstoretrievedocumentsfromSolr.YoucanusefullrangeofSQLwithTeiidsystemintegratingothersourcesalongwithSolrsource.
TheSolrTranslatorsupportsSELECTstatementswitharestrictivesetofcapabilitiesincluding:comparisonpredicates,INpredicates,LIMITandOrderBy.
AnExampleVDBthatshowsSolrtranslatorcanbedefinedas
<vdbname="search"version="1">
<modelname="solr">
<sourcename="node-one"translator-name="solr"connection-jndi-name="java:/solrDS"/>
</model>
<vdb>
ApacheSOLRTranslator
520
ThetranslatordoesNOTprovideaconnectiontotheSolr.Forthatpurpose,TeiidhasaJCAadapterthatprovidesaconnectiontoSolrusingtheSolrJJavalibrary.Todefinesuchconnector,seeSolrDataSourcesorseeanexamplein"<jboss-as>/docs/teiid/datasources/solr"
IfyouareusingDesignerTooling,tocreateVDBthen
Create/useaTeiidDesignerModelproject
Use"TeiidConnection>>SourceModel"importer,createSolrDataSourceusingdatasourcecreationwizardandusesolrastranslatorintheimporter.Thesearchtableiscreatedinasourcemodelbythetimeyoufinishwiththisimporter.
CreateaVDBanddeployintoTeiidServeranduseeitherjdbc,odbc,odataetctoquery.
JCAResourceAdapterTheTeiidspecificSolrResourceAdaptershouldbeusedwiththistranslator.SeeSolrDataSourcesforconnectingtoaSolrSearchEngine.
NativeQueries
ThisfeatureisnotapplicableforSolrtranslator.
DirectQueryProcedure
ThisfeatureisnotavailableforSolrtranslatorcurrently.
ApacheSOLRTranslator
521
CassandraTranslatorTheCassandraTranslator,knownbythetypenamecassandra,exposesqueryingfunctionalitytoCassandraDataSources.ThetranslatortranslatesTeiidpushdowncommandsintoCassandraCQL.
Properties
TheCassandraTranslatorcurrentlyhasnoimportorexecutionproperties.
Usage
TheCassandraTranslatorsupportsINSERT/UPDATE/DELETE/SELECTstatementswitharestrictivesetofcapabilitiesincluding:count(*),comparisonpredicates,INpredicates,andLIMIT.Onlyindexedcolumnsaresearchable.Consideracustomextensionorcreateanenhancementrequestshouldyourusagerequireadditionalcapabilities.
IfyouareusingDesignerTooling,tocreateVDBthen:
Create/useaTeiidDesignerModelproject
Use"TeiidConnection>>SourceModel"importer,createanewJBossDataSourceconnectionprofile,specifyingtheJNDInameforresourceadapterconfiguredhttps://docs.jboss.org/author/display/TEIIDCassandra+Data+Sources[datasource]andusecassandraastranslatortype.Thesourcemodelwillbecreatedwhenyoufinishwiththisimporter.
CreateaVDBanddeployintoTeiidServeranduseeitherjdbc,odbc,odataetctoquery.
Cassandraupdatesalwaysreturnanupdatecountof1perupdateregardlessofthenumberofrowsaffected.
Cassandrainsertsarefunctionallyupserts,thatisifagivenrowexistsitwillbeupdatedratherthancausinganexception.
JCAResourceAdapter
TheTeiidspecificCassandraResourceAdaptershouldbeusedwiththistranslator.SeeCassandraDataSourcesforconnectingtoaCassandracluster.
NativeQueries
Cassandrasourceproceduresmaybecreatedusingtheteiid_rel:native-queryextension-seeParameterizableNativeQueries.Theprocedurewillinvokethenative-querysimilartoadirectprocedurecallwiththebenefitsthatthequeryispredeterminedandthatresultcolumntypesareknown,ratherthanrequiringtheuseofARRAYTABLEorsimilarfunctionality.
DirectQueryProcedure
Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyiscallednative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheCassandratranslatorprovidesaproceduretoexecuteanyad-hocCQLquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.
ExampleCQLDirectQuery
SELECTX.*
FROMcassandra_source.native('SELECTfirstname,lastnameFROMusersWHEREbirth_year=$1ANDcountry=$2AL
CassandraTranslator
522
LOWFILTERING',1981,'US')n,
ARRAYTABLE(n.tupleCOLUMNSfirstnamestring,lastnamestring)ASX
CassandraTranslator
523
DelegatingTranslators
Translator"delegator"
Atranslatorbyname"delegator"isavailableincoreTeiidinstallation,thatcanbeusedtomodifythecapabilitiesofaexistingtranslator.Oftentimesfordebuggingpurposesorinspecialsituations,onemayrequiretoeitherturnon/offcertaincapabilityoftranslator.Forexample,assumeHivedatabaseintheirlatestversionsupportingtheORDERBYconstruct,howeverTeiid’scurrentversionoftheHivetranslatordoesnothavethiscapability,youcanusethe"delegator"translatortoturnONthe"ORDERBY"supportwithoutactuallywritinganycode.Sometimesyoumaywanttodothereverse,youwantturnoffcertaincapabilitytoproduceabetterplan.Inthesesituations,youcanusethistranslator.
Tousethistranslator,youneedtodefinethistranslatorintheVDB,asashowninthebelowVDB.Thebelowexampleoverridingthe"hive"translatorandturningofftheORDERBYsupport.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="myvdb"version="1">
<modelname="mymodel">
<sourcename="source"translator-name="hive-delegator"connection-jndi-name="java:hive-ds"/>
</model>
<!--thebelowitiscalledtranslatoroverriding,whereyoucansetdifferentproperties-->
<translatorname="hive-delegator"type="delegator"/>
<propertyname="delegateName"value="hive"/>
<propertyname="supportsOrderBy"value="false"/>
</translator>
</vdb>
Youcanoverrideany/allthetranslatorcapabilitiesdefinedhereTranslatorCapabilitiesasExecutionPropertiestooverride.Exampleof"supportsOrderBy"isshowninaboveexample.
Extendingthe"delegator"translator
Youmaycreateadelegatingtranslatorbyextendingtheorg.teiid.translator.BaseDelegatingExecutionFactory.Onceyourclassesarethenpackagedasacustomtranslator,youwillbeabletowireanothertranslatorinstanceintoyourdelegatingtranslatoratruntimeinordertointerceptallofthecallstothedelegate.Thisbaseclassdoesnotprovideanyfunctionalityonitsown,otherthandelegation.Thedifferenceherefromprevious"delegator"translatoris,youcanhardcodethecapabilitiesinsteadofdefiningasconfigurationinsidethe-vdb.xml,aswellasoverridemethodstoprovidealternatebehavior.
ExecutionProperties
Name Description Default
delegateName Translatorinstancenametodelegateto n/a
Letssayyouarecurrentlyusing"oracle"translatorinyourVDB,youwanttointerceptthecallsgoingthroughthistranslator,thenyoufirstwriteacustomdelegatingtranslatorlike
@Translator(name="interceptor",description="interceptor")
publicclassInterceptorExecutionFactoryextendsorg.teiid.translator.BaseDelegatingExecutionFactory{
@Override
publicvoidgetMetadata(MetadataFactorymetadataFactory,Cconn)throwsTranslatorException{
DelegatingTranslators
524
//dointerceptingcodehere..
//Ifyouwantcalltheoriginaldelegate,donotcallifdonotneedto.
//butifyoudidnotcallthedelegatefullfillthemethodcontract
super.getMetadata(metadataFactory,conn);
//domoreinterceptingcodehere..
}
}
NowdeploythistranslatorinTeiidengine.Theninyour-vdb.xmlor.vdbfiledefinelikebelow.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="myvdb"version="1">
<modelname="mymodel">
<sourcename="source"translator-name="oracle-interceptor"connection-jndi-name="java:oracle-ds"/>
</model>
<!--thebelowitiscalledtranslatoroverriding,whereyoucansetdifferentproperties-->
<translatorname="orcle-interceptor"type="interceptor"/>
<propertyname="delegateName"value="oracle"/>
</translator>
</vdb>
Wehavedefineda"translator"overridecalled"oracle-interceptor",whichisbasedonthecustomtranslator"interceptor"fromabove,andsuppliedthetranslatoritneedstodelegateto"oracle"asitsdelegateName.Then,weusedthisoverridetranslator"oracle-interceptor"inyourVDB.NowanycallsgoingintothisVDBmodel’stranslatorwillbeinterceptedbyYOURcodetodowhateveryouwanttodo.
DelegatingTranslators
525
FileTranslatorThefiletranslator,knownbythetypenamefile,exposesstoredprocedurestoleveragefilesystemresourcesexposedbytheFileDataSourceandtheFTPDataSource.ItwillcommonlybeusedwiththeTEXTTABLEorXMLTABLEtablefunctionstouseCSVorXMLformatteddata.
ExecutionProperties
Name Description Default
EncodingTheencodingthatshouldbeusedforCLOBsreturnedbythegetTextFilesprocedure
Thesystemdefaultencoding
ExceptionIfFileNotFoundThrowanexceptioningetFilesorgetTextFilesifthespecifiedfile/directorydoesnotexist.
true(falsepriorto8.2)
Usage
getFiles
RetrieveallfilesasBLOBswithanoptionalextensionatthegivenpath.
callgetFiles('path/*.ext')
Ifthepathisadirectory,thenallfilesinthedirectorywillbereturned.Ifthepathmatchesasinglefile,itwillbereturned.
The'*'characterwillbetreatedasawildcardtomatchanynumberofcharactersinthepathname-0ormatchingfileswillbereturned.
If'*'isnotusedandthepathdoesn’texistandExceptionIfFileNotFoundistrue,thenanexceptionwillberaised.
getTextFiles
RetrieveallfilesasCLOB(s)withtheanoptionalextensionatthegivenpath.
callgetTextFiles('path/*.ext')
AllthesamefilesagetFileswillberetrieved,theonlydifferenceisthattheresultswillbeCLOBvaluesusingtheencodingexecutionpropertyasthecharacterset.
saveFile
SavetheCLOB,BLOB,orXMLvaluetogivenpath
callsaveFile('path',value)
deleteFile
Deletethefileatthegivenpath
FileTranslator
526
calldeleteFile('path')
Thepathshouldreferenceanexistingfile.IfthefiledoesnotexistandExceptionIfFileNotFoundistrue,thenanexceptionwillbethrown.Orifthefilecannotbedeletedanexceptionwillbethrown.
NOTE Nativequeries-NativeordirectqueryexecutionisnotsupportedontheFileTranslator.
JCAResourceAdapterTheresourceadapterforthistranslatorprovidedthrough"FileDataSource",RefertoAdminGuideforconfigurationinformation.
FileTranslator
527
GoogleSpreadsheetTranslatorThegoogle-spreadsheettranslatorisusedtoconnecttoaGoogleSpreadsheet.TousetheGoogleSpreadsheetTranslatoryouneedtoconfigureanddeploytheGoogleJCAconnector-seetheAdminGuide.
Thequeryapproachexpectsthedataintheworksheettobeinaspecificformat.Namely:
Anycolumnthathasdataisqueryable.
Anycolumnwithanemptycellhasthevalueretrievedasnull.Howeverdifferentiatingbetweennullstringandemptystringvaluesmaynotalwaysbepossibleasgoogletreatstheminterchanably.Wherepossiblethetranslatormayprovideawarningorthrowanexceptioniftheremaybeaconfusionofnullvs.emptystrings.
Ifthefirstrowispresentandcontainsstringvalues,thenitwillbeassumedtorepresentthecolumnlabels.
Ifyouareusingthedefaultnativemetadataimport,themetadataforyourGoogleaccount(worksheetsandinformationaboutcolumnsinworksheets)areloadedupontranslatorstartup.Ifyoumakeanychangesindatatypes,itisadvisabletorestartyourvdb.
Thetranslatorsupportsqueriesagainstasinglesheet.Itsupportsordering,aggregation,basicpredicates,andmostofthefunctionssupportedbythespreadsheetquerylanguage.
Therearenogoogle-spreadsheetimportersettings,butitcanprovidemetadataforVDBs.
JCAResourceAdapterTheTeiidspecificGoogleSpreadsheetDataSourcesResourceAdaptershouldbeusedwiththistranslator.
NativeQueries
Googlespreadsheetsourceproceduresmaybecreatedusingtheteiid_rel:native-queryextension-seeParameterizableNativeQueries.Theprocedurewillinvokethenative-querysimilartoannativeprocedurecallwiththebenefitsthatthequeryispredeterminedandthatresultcolumntypesareknown,ratherthanrequiringtheuseofARRAYTABLEorsimilarfunctionality.SeetheSelectformatbelow.
DirectQueryProcedure
Note Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyiscallednative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheGooglespreadsheettranslatorprovidesaproceduretoexecuteanyad-hocquerydirectlyagainstthesourcewithoutanyTeiidparsingorresolving.Sincethemetadataofthisprocedure’sexecutionresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.Teiidexposesthisprocedurewithasimplequerystructureasbelow:
Select
SelectExample
SELECTx.*FROM(callgoogle_source.native('worksheet=People;query=SELECTA,B,C'))w,
GoogleSpreadsheetTranslator
528
ARRAYTABLE(w.tupleCOLUMNS"id"string,"type"string,"name"String)ASx
thefirstargumenttakessemi-colon(;)separatednamevaluepairsoffollowingpropertiestoexecutetheprocedure:
Property Description Required
worksheet Googlespreadsheetname yes
query spreadsheetquery yes
limit numberrowstofetch no
offset offsetofrowstofetchfromlimitorbeginning no
GoogleSpreadsheetTranslator
529
InfinispanHotRodTranslatorTheInfinispanHotRodTranslator,knownbythetypeispn-hotrod,canreadthejavaobjectsfromaremoteJDGCacheviatheHotRodclientusingtheGoogleProtobufforserialization.ThiswillenableTeiidtoquerytheremotecacheusingJDGDSL.ThistranslatorextendstheObjectTranslatorandusesitforthecoreprocessingofreadingandwritingobjects.
IfyouareusingJDGinlibrarymode,seetheInfinispanLibraryModeTranslatorforthistypeofconfiguration.
Usage
Retrieveobjectsfromacacheandtransformintorowsandcolumns.
Supportsperformingwritestothecache
Useforexternalmaterializationtoimprovequeryperformance
SupportedCapabilities
Thefollowingaretheconnectorcapabilities:
CompareCriteria-EQ
CompareCriteriaOrdered-LT,GT,LE,GE-supportforSupportsComapareCriteriaOrderedwillbecontrolledbytheversionofJDGbeingaccessed.AnyJDGversion6.5andpriorwillhavethissettofalseduetoanissuewithJDG.
And/OrCriteria
InCriteria
LikeCriteria
OrderBy
INSERT,UPDATE,DELETE(non-transactional)
ThefollowingwillnotbepusheddowntoJDGforprocessing,butwillbedonewithinTeiid:
Not(NE,<>)
IsNull
Limitations:
supportfor'Not'hasbeendisabled,becauseJDGconsiders<columnwhennull><>1tobetrueandSQLdoesnot.
booleandatatype:JDGwillthrowanexceptionifnovalueisspecifiedontheinsertorwhennodefaultvalueisdefinedintheprotobufdefinitionfile.
chardatatype:isnotasupportedtypeintheProtobufdatatypes(https://developers.google.com/protocol-buffers/docs/proto#scalar).WouldeitherhavetohandleconversionintheprotobufmarshallerorcreateaTeiidviewwiththedatatypeaschar.
1-to-Many,currentlyonlysupportsCollectionorArray,notMap’s'
XAtransactionsarenotsupportedbyJDGwhenusingHotRodclient.
Tobedone:
InfinispanHotRodTranslator
530
supportdeletingcontainingclass(s)(i.e.,person-→phones,deleteaphone)
supportupdatingcontainingclass(s)
Installation
Theispn-hotrodtranslatorisnotconfigured,out-of-the-box,inthestandalone-teiid.xmlconfiguration.Toconfigurethetranslator,runtheadd-infinispan-hotrod-translator.cliscript.Thisscriptcanalsobefoundintheteiid-jboss-dist.zipkit,underdocs/teiid/datasources/infinispan.Seethejdg-remote-cachequickstartforanexample.Note,thisassumesyouwillbeinstallingaseparateJDGserver,whichwillbeaccessedviatheHotRodClientinstalledintotheTeiidserver.
Metadata
OptionsforDefining
Thereareseveraloptionstodefiningthemetadatarepresentingyourobjectinthecache.
"Recommended"UsetheTeiidConnectionImporterinTeiidDesignertocreatethephysicalsourcemodelbasedonyourobjectcache.Thetablecolumnswillbecreatedfromthegoogleprotobufdefinition,thatcorrespondstoaregisteredclass.
UseTeiidDesignertomanuallycreatethephysicalsourcemodelbasedonyourobjectcacheusingthebelowDefinitionRequirements.
AsimpleVDBthatonlydefinesthedatasourcetouse.Example:
<modelname="People"type="Physical">
<propertyname="importer.useFullSchemaName"value="false"/>
<sourcename="infinispan-hotrod-connector"translator-name="ispn-hotrod"connection-jndi-name="java:/in
finispanRemoteDSL"/>
</model>
Themetadatawillberesolvedbyreverseengineeringthedefinedobjectinthecache.ThiscanbeusefulwhenusingtheTeiidDesignerTeiidConnectionImporterforbuildingthephysicalsourcemodel(s).
YoucanalsodefinethemetadatausingDDL.SeeObjectTranslatorforanexample.
DefinitionRequirements
seeObjectTranslatorMetadatasectionforbasedefinitionrequirements.
ColumnswillbeidentifiedasSEARCHABLEifeithertheprotobufdefinitionforacolumnindicatesitsindexedorthepojoclasshastheattribute/methodannotated.
A1-to-*relationshipclassmusthaveaforeignkeytomaptotherootclass/table,wherethenameinsourcefortheforeignkeyisthenameoftherootclassmethodtoaccessthosechildobjects.Note,thisistheclassmethod,notareferenceinthegoogleprotobufdefinition.
Acontainer/childclasswillhaveattributeswheretheNIScontainaperiod.Example:phone.number.ThisisbecausethismapstotogoogleprotobufdefinitionandwhatisexpectedbytheDSLquerytoreferenceattributesofainnerclass.
ExternalMaterialization
InfinispanHotRodTranslator
531
Thistranslatorsupportsusingthecacheforexternalmaterialization.However,therearespecificconfigurationsettingsthatarerequiredontheJDGHotRodResourceAdapterandatthetranslator.
Note
Creatingorselectingaviewtomaterialize:
Unfortunatelyforprotobufs,complexdatatypesdon’tconverttoC/C++.Sowhenyouarecreatingaviewtouseformaterialization,itisrecommendedtostickwithprimitivedatatypesbecauseprotobufferdoesn’tsupportbigdecimaldirectly.Sothereare3options:
1. useallprimitivedatatypes
2. implementamarshallerthatwillhandletheconversion,whichmeansthe.protofilewillalsoneedtobecreated(seeJDGforcreationoffiles)
3. createaviewthatwillconvertthebigdecimaltoastring,thenmaterializethatview.
NativeQueries
ExternalmaterializationisenabledbytheuseofnativequeriesintheBEFORE_LOAD_SCRIPTandAFTER_LOAD_SCRIPT.Atranslatoroverridewillneedtobesettoenablenativequeries:SupportsNativeQueries=true
Thefollowingmaterializationpropertiesmustbedefined:
Script Nativequery Description
teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT truncatecache Totruncatethecacheidentifiedasthestagingcache
teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT swapcachenames
Toswapthealiasesforthecaches,sothattheprimarycachepointstotherecentlyloadedcache
ThefollowingisanexampleoffordefiningtheloadscriptsinDDL:
..
"teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT"'executeStockMatCache.native(''truncatecache'');',
"teiid_rel:MATVIEW_LOAD_SCRIPT"'insertintoStockMatCache.Stock(productId,symbol,price,companyName)SELECT
A.ID,S.symbol,S.price,A.COMPANY_NAMEFROMStocks.StockPricesASS,Accounts.PRODUCTASAWHERES.symbol=
A.SYMBOL',
"teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT"'executeStockMatCache.native(''swapcachenames'');',
NativequeriesareusedtosimulatehowitsdoneusingRDBMSandrenamingtables,becauseInfinispandoesn’tcurrentlysupportrenamingacache.Sothenativequerieswilltriggertheclearingofthe"staging"cache,andtheswappingofthecachealiases.
DirectQueryProcedure
Additionally,theexecutionofnativequeriesisdonethruthesupportofdirectqueryprocedures.Theproceduretobeexecutediscallednative.
WarningThisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionproperty[OverrideExecutionProperties]calledSupportsDirectQueryProceduretotrue.
MetadataRequirements
IfyoumanuallymodelthecachetableinTeiidDesigner,thenyouwillneedtoaddthepropertyextensionfordefiningtheproperty"primary_table".ThefollowingisaDDLexample:
InfinispanHotRodTranslator
532
SETNAMESPACE'http://www.teiid.org/translator/object/2016'ASn0;
CREATEFOREIGNTABLETrade(
....
CONSTRAINTPK_TRADEIDPRIMARYKEY(tradeId)
)OPTIONS(UPDATABLETRUE);
CREATEFOREIGNTABLEST_Trade(
....
)OPTIONS(NAMEINSOURCE'Trade',UPDATABLETRUE,"n0:primary_table"'ObjectSchema.Trade');
JCAResourceAdapterSeeJDGHotRodDataSourcesresourceadapterforthistranslator.ItcanbeconfiguredtolookupthecachecontainerviaJNDI,serverlist,orhotrodproperties.
InfinispanHotRodTranslator
533
InfinispanLibraryModeTranslatorTheInfinispanLibraryModetranslator,knownbythenameofispn-lib-mode,isabridgeforreadingandwritingjavaobjectsto/fromanJDGCacherunninginlibrarymode.ThistranslatorextendstheObjectTranslatorandusesitforthecoreprocessingofreadingandwritingobjects.TheInfinispanLibraryModeTranslatoriswrittensothatitcancontrolhowthecacheissearchedandanycapabilitiesthatareneededtocontrolthatbehavior.
IfyouareusingJDGtoaccessaremotecache,seetheInfinispanHotRodTranslatorforthistypeofconfiguration.
Usage
Retrieveobjectsfromacacheandtransformintorowsandcolumns.
Performwritestothecache
Useforexternalmaterializationtoimprovequeryperformance
SearchingOptions
Supportsthefollowingoptionsforsearching:
DSLsearching(JDG6.3+)
Hibernate/Lucenesearching(nowdeprecated)
Keysearching(whennoindexingisused).
Seethejdg-local-cachequickstartforanexample.
SupportedCapabilities
ThefollowingaretheconnectorcapabilitieswhenKeySearchingisused:
SELECTcommand
CompareCriteria-onlyEQ
InCriteria
ThefollowingaretheconnectorcapabilitieswhenDSLSearchingisenabled:
SELECTcommand
CompareCriteria-EQ,NE
CompareCriteria-GT,LT,GE,andLEcanbeenabled,seeExecutionProperties
And/OrCriteria
InCriteria
LikeCriteria
OrCriteria
IsNullcheck-seeExecutionProperties
Not(negation)-example:NotIn,NotLike
InfinispanLibraryModeTranslator
534
INSERT,UPDATE,DELETE
Installation
Theispn-lib-modetranslatorisnotconfigured,out-of-the-box,inthestandalone-teiid.xmlconfiguration.Toconfigurethetranslator,runtheadd-infinispan-libmode-translator.cliscript.Thisscriptcanalsobefoundintheteiid-jboss-dist.zipkit,underdocs/teiid/datasources/infinispan.
ExecutionProperties
Name Description Required Default
SupportsLuceneSearching(Deprecated)
SettingtotrueassumesyourobjectsareannotatedandHibernate/Lucenewillbeusedtosearchthecache
No false
SupportsDSLSearchingSettingtotrueassumesyourareusingJDGv6.3orbetterandyourcachehasindexingenabled
No true
SupportsIsNullCriteriaSettingtotrueassumeshttps://issues.jboss.org/browse/TEIID-3539hasbeenresolved
No false
SupportsCompareCriteriaOrderedSettingtotrueassumeshttps://issues.jboss.org/browse/TEIID-3627hasbeenresolved
No false
SupportsNotCriteriaSettingtotrueassumeshttps://issues.jboss.org/browse/TEIID-3573hasbeenresolved
No false
SupportsNotCriteriadefaultstofalsebecausethetestingofcolumn<>1returnstruewhenthecolumnisnull,whichisn’tcorrectbySQLstandards-seehttps://issues.jboss.org/browse/TEIID-3573.Thereisanenhancementcomingthatwillenableadding"columnISNOTNULL"whencolumn<>1typecriteriaisdetected.
SupportsIsNullCriteriadefaultstofalsebecauseInfinispan/JDGhasanissuewithissuingaISNULLcheckonanumerictypeattribute-seehttps://issues.jboss.org/browse/TEIID-3539.Setthistotrueifyouneedthischeckandcancontrolwhichnon-numericcolumnsthatthiswillbeissuedagainst.
SupportsCompareCriteriaOrdereddefaultstofalsebecausetheInfinispan/JDGhasanissuewithstringcomparisons-seehttps://issues.jboss.org/browse/TEIID-3627.
ConfiguringPojoclass
Thepojoclassistheobjectthatwillbeusedtostorethedatainthecache.Itshouldbebuiltaccordingly:
Totakeadvantageofthecachebeingindexedenabled,shouldannotatetheclass.SeeJDGdocumentationathttps://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Data_Grid/6.6/html-single/Infinispan_Query_Guide/index.html#chap-Annotating_Objects_and_Querying
Theclassshouldbepackagedintoajarsothatitcanbedeployedasamodule
Exampleclass
@Indexed
publicclassStock{
InfinispanLibraryModeTranslator
535
@NumericField@Field(index=Index.YES,store=Store.YES,analyze=Analyze.NO)
publicintproductId;
@Field(index=Index.YES,store=Store.YES,analyze=Analyze.NO)
publicBigDecimalprice;
@Field(index=Index.YES,store=Store.YES,analyze=Analyze.NO)
publicStringsymbol;
@Field(index=Index.YES,store=Store.YES,analyze=Analyze.NO)
publicStringcompanyName;
publicintgetProductId(){
returnthis.productId;
}
publicvoidsetProductId(intid){
this.productId=id;
}
publicBigDecimalgetPrice(){
returnthis.price;
}
publicvoidsetPrice(BigDecimalprice){
this.price=price;
}
publicvoidsetPrice(doubleprice){
this.price=newBigDecimal(price);
}
publicStringgetSymbol(){
returnthis.symbol;
}
publicvoidsetSymbol(Stringsymbol){
this.symbol=symbol;
}
publicStringgetCompanyName(){
returncompanyName;
}
publicvoidsetCompanyName(Stringname){
this.companyName=name;
}
}
Toconfiguretheuseofthepojo,dothefollowing:
Deploythepojojarasamoduleinthejboss-asserver.Thendefinethe"lib"propertyinthe-vdb.xmlandassignthecorrectmodulename.Thiscanbedoneusingthefollowingtemplate:
<propertyname="lib"value="{pojo_module_name}"></property>
TheJDGcommonsmodule,org.infinispan.commons,slot="jdg-6.6"orslotforversioninstalled,needstohavethepojodependencyadded:
<modulename="{pojo_module_name}"export="true"/>
Metadata
OptionsforDefining
InfinispanLibraryModeTranslator
536
Therearecoupleoptionstodefiningthemetadatarepresentingyourobjectinthecache.
"Recommended"UsetheTeiidConnectionImporterinTeiidDesignertocreatethephysicalsourcemodelbasedonyourobjectcache.Thetablecolumnswillbecreatedfromthegoogleprotobufdefinition,thatcorrespondstoaregisteredclass.
UseTeiidDesignertomanuallycreatethephysicalsourcemodelbasedonyourobjectcacheusingthebelowDefinitionRequirements.
ThefollowingisaVDBexamplesimilartoquickstart(seegithubathttps://github.com/teiid/teiid-quickstarts/tree/master/jdg-local-cache).ItdefinesthephysicalsourceandwillusethetranslatornativeimporterlogictohavethemetadatareverseengineeredfromtheStockclass,seeabove,tobeexposedasthesourcetable.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="Stocks"version="1">
<description>ShowshowtocallanInfinispanlocalcache</description>
<propertyname="cache-metadata"value="true"/>
<propertyname="lib"value="com.client.quickstart.pojos"></property>
<modelname="StockCache"type="Physical">
<propertyname="importer.useFullSchemaName"value="false"/>
<sourcename="StockSource"translator-name="infinispan1"connection-jndi-name="java:/infinispanLocal"/>
</model>
<translatorname="infinispan1"type="ispn-lib-mode">
<propertyname="SupportsDSLSearching"value="true"/>
</translator>
</vdb>
Themetadatawillberesolvedbyreverseengineeringthedefinedobjectinthecache.ThiscanbeusefulwhenusingtheTeiidDesignerTeiidConnectionImporterforbuildingthephysicalsourcemodel(s).
YoucanalsodefinethemetadatausingDDL.SeeObjectTranslatorforanexample.
DefinitionRequirements
seeObjectTranslatorMetadatasectionforbasedefinitionrequirements.
AllcolumnsthatarenottheprimarykeynorcoveredbyaluceneindexshouldbemarkedasSEARCHABLE'Unsearchable'.
ExternalMaterializationThistranslatorsupportsusingthecacheforexternalmaterialization.However,therearespecificconfigurationchangesthatarerequiredattheInfinispanDataSourcesresource-adapterandatthetranslator.Foranexample,seethejdglocalquickstart.
NativeQueries
ExternalmaterializationisenabledbytheuseofnativequeriesintheBEFORE_LOAD_SCRIPTandAFTER_LOAD_SCRIPT.Atranslatoroverridewillneedtobesettoenablenativequeries:SupportsNativeQueries=true
Thefollowingmaterializationpropertiesmustbedefined:
Script Nativequery Description
teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT truncatecache Totruncatethecacheidentifiedasthestagingcache
InfinispanLibraryModeTranslator
537
teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT swapcachenames
Toswapthealiasesforthecaches,sothattheprimarycachepointstotherecentlyloadedcache
ThefollowingisanexampleofdefiningthematerializationloadscriptsinDDL:
..
"teiid_rel:MATVIEW_BEFORE_LOAD_SCRIPT"'executeStockMatCache.native(''truncatecache'');',
"teiid_rel:MATVIEW_LOAD_SCRIPT"'insertintoStockMatCache.Stock(productId,symbol,price,companyName)SELECT
A.ID,S.symbol,S.price,A.COMPANY_NAMEFROMStocks.StockPricesASS,Accounts.PRODUCTASAWHERES.symbol=
A.SYMBOL',
"teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT"'executeStockMatCache.native(''swapcachenames'');',
NativequeriesareusedtosimulatehowitsdoneusingRDBMSandrenamingtables,becauseInfinispandoesn’tcurrentlysupportrenamingacache.Sothenativequerieswilltriggertheclearingofthe"staging"cache,andtheswappingofthecachealiases.
DirectQueryProcedure
Additionally,theexecutionofnativequeriesisdonethruthesupportofdirectqueryprocedures.Theproceduretobeexecutediscallednative.
WarningThisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,[overridetheexecutionproperty|Translators#OverrideExecutionProperties]calledSupportsDirectQueryProceduretotrue.
MetadataRequirements
IfyoumanuallymodelthecachetableinTeiidDesigner,thenyouwillneedtoaddthepropertyextensionfordefiningtheproperty"primary_table".ThefollowingisaDDLexample:
SETNAMESPACE'http://www.teiid.org/translator/object/2016'ASn0;
CREATEFOREIGNTABLETrade(
....
CONSTRAINTPK_TRADEIDPRIMARYKEY(tradeId)
)OPTIONS(UPDATABLETRUE);
CREATEFOREIGNTABLEST_Trade(
....
)OPTIONS(NAMEINSOURCE'Trade',UPDATABLETRUE,"n0:primary_table"'ObjectSchema.Trade');
JCAResourceAdapter
SeeJDGLibraryModeDataSourcesresourceadapterforthistranslator.ItcanbeconfiguredtolookupthecachecontainerviaJNDIorcreated(i.e.,ConfigurationFileName).
InfinispanLibraryModeTranslator
538
JDBCTranslatorsTheJDBCtranslatorsbridgebetweenSQLsemanticanddatatypedifferencesbetweenTeiidandatargetRDBMS.Teiidhasarangeofspecifictranslatorsthattargetthemostpopularopensourceandproprietarydatabases.
TableofContentsUsageExecutionProperties-sharedbyallJDBCTranslatorsImporterProperties-sharedbyallJDBCTranslators
NativeQueriesDirectQueryProcedure
JCAResourceAdapter
Usage
UsageofaJDBCsourceisstraight-forward.UsingTeiidSQL,thesourcemaybequeriedasifthetablesandprocedureswerelocaltotheTeiidsystem.
Ifyouareusingarelationaldatasource,oradatasourcethathasaJDBCdriver,andyoudonotfindaspecifictranslatoravailableforthatdatasourcetype,thenstartwiththeJDBCANSITranslator.TheJDBCANSITranslatorshouldenableyoutoperformtheSQLbasics.Iftherespecificdatasourcecapabilitiesthatarenotavailable,thenconsiderusingtheTranslatorDevelopmenttocreatewhatyouneed.OrlogaTeiidJirawithyourrequirements.
ExecutionProperties-sharedbyallJDBCTranslators
Name Description Default
DatabaseTimeZoneThetimezoneofthedatabase.Usedwhenfetchingsdate,time,ortimestampvalues.
Thesystemdefaulttimezone
DatabaseVersionThespecificdatabaseversion.Usedtofurthertunepushdownsupport.
ThebasesupportedversionorderivedfromtheDatabaseMetadata.getDatabaseProductVersionstring.AutomaticdetectionrequiresaConnection.Iftherearecircumstanceswhereyouaregettinganexceptionfromcapabilitiesbeingunavailable(mostlikelyduetoanissueobtainingaConnection),thensetDatabaseVersionproperty.UsetheJDBCExecutionFactory.usesDatabaseVersion()methodtocontrolwhetheryourtranslatorrequiresaconnectiontodeterminecapabilities.
TrimStrings
truetotrimtrailingwhitespacefromfixedlengthcharacterstrings.NotethatTeiidonlyhasastring,orvarchar,typethattreatstrailingwhitespaceasmeaningful.
false
JDBCTranslators
539
UseBindVariables
truetoindicatethatPreparedStatementsshouldbeusedandthatliteralvaluesinthesourcequeryshouldbereplacewithbindvariables.IffalseonlyLOBvalueswilltriggertheuseofPreparedStatements.
true
UseCommentsInSourceQuery
Thiswillembedaleadingcommentwithsession/requestidinthesourceSQLforinformationalpurposes.CanbecustomizedwiththeCommentFormatproperty.
false
CommentFormat
MessageFormatstringtobeusedifUseCommentsInSourceQueryisenabled.Availableproperties:0-sessionidstring,1-parentrequestidstring,2-requestpartidstring,3-executioncountidstring,4-usernamestring,5-vdbnamestring,6-vdbversioninteger,7-istransactionalboolean
/teiidsessionid:{0},requestid:{1}.{2}/
MaxPreparedInsertBatchSize Themaxsizeofapreparedinsertbatch. 2048
StructRetrieval
StructretrievalmodecanbeoneofOBJECT-getObjectvaluereturned,COPY-returnedasaSerialStruct,ARRAY-returnedasanArray)
OBJECT
EnableDependentJoins
Forsourcesthatsupporttemporarytables(DB2,Derby,H2,HSQL2.0+,MySQL5.0+,Oracle,PostgreSQL,SQLServer,Sybase)allowdependentjoinpushdown
false
ImporterProperties-sharedbyallJDBCTranslatorsWhenspecifyingtheimporterproperty,itmustbeprefixedwith"importer.".Example:importer.tableTypes
Name Description Default
catalog SeeDatabaseMetaData.getTables[1] null
schemaPattern SeeDatabaseMetaData.getTables[1] null
tableNamePattern SeeDatabaseMetaData.getTables[1] null
procedureNamePattern SeeDatabaseMetaData.getProcedures[1] null
JDBCTranslators
540
tableTypes Commaseparatedlist-withoutspaces-ofimportedtabletypes.SeeDatabaseMetaData.getTables[1]
null
excludeTables
Acase-insensitiveregularexpressionthatwhenmatchedagainstafullyqualifiedtablename[2]willexcludeitfromimport.Appliedaftertablenamesareretrieved.Useanegativelook-ahead(?!<inclusionpattern>).*toactasaninclusionfilter.
null
excludeProcedures
Acase-insensitiveregularexpressionthatwhenmatchedagainstafullyqualifiedprocedurename[2]willexcludeitfromimport.Appliedafterprocedurenamesareretrieved.Useanegativelook-ahead(?!<inclusionpattern>).*toactasaninclusionfilter.
null
useFullSchemaName
Whenfalse,directstheimportertodropthesourcecatalog/schemafromtheTeiidobjectname,sothattheTeiidfullyqualifiednamewillbeintheformof<modelname>.<tablename>-Note:whenfalsethismayleadtoobjectswithduplicatenameswhenimportingfrommultipleschemas,whichresultsinanexception.Thisoptiondoesnotaffectthenameinsourceproperty.
true
importKeystruetoimportprimaryandforeignkeys-NOTEforeignkeystotablesthatarenotimportedwillbeignored
true
autoCreateUniqueConstraints truetocreateauniqueconstraintifoneisnotfoundforaforeignkeys true
importIndexes truetoimportindex/uniquekey/cardinalityinformation false
importApproximateIndexestruetoimportapproximateindexinformation.SeeDatabaseMetaData.getIndexInfo[1]
true
importProcedures
truetoimportproceduresandprocedurecolumns-Notethatitisnotalwayspossibletoimportprocedureresultsetcolumnsduetodatabaselimitations.Itisalsonotcurrentlypossibletoimportoverloadedprocedures.
false
widenUnsignedTypes
truetoconvertunsignedtypestothenextwidesttype.ForexampleSQLServerreportstinyintasanunsignedtype.Withthisoptionenabled,tinyintwouldbeimportedasashortinsteadofabyte.
true
JDBCTranslators
541
quoteNameInSource falsewilloverridethedefaultanddirectTeiidtocreatesourcequeriesusingunquotedidentifiers.
true
useProcedureSpecificName
truewillallowtheimportofoverloadedprocedures(whichwillnormallyresultinaduplicateprocedureerror)byusingtheuniqueprocedurspecificnameastheTeiidname.ThisoptionwillonlyworkwithJDBC4.0compatibledriversthatreportspecificnames.
false
useCatalogName
truewilluseanynon-null/non-emptycatalognameaspartofthenameinsource,e.g."catalog"."schema"."table"."column",andintheTeiidruntimenameifuseFullSchemaNameisalsotrue.falsewillnotusethecatalognameineitherthenameinsourceortheTeiidruntimename.Shouldbesettofalseforsourcesthatdonotfullysupportacatalogconcept,butreturnanon-nullcatalognameintheirmetadata-suchasHSQL.
true
useQualifiedName
truewillusenamequalificationforboththeTeiidnameandnameinsourceasdictatedbytheuseCatalogNameanduseFullSchemaNameproperties.SettofalsetodisableallqualificationforboththeTeiidnameandthenameinsource,whicheffectivelyignorestheuseCatalogNameanduseFullSchemaNameproperties.Note:whenfalsethismayleadtoobjectswithduplicatenameswhenimportingfrommultipleschemas,whichresultsinanexception.
true
useAnyIndexCardinality
truewillusethemaximumcardinalityreturnedfromDatabaseMetaData.getIndexInfo.importKeysorimportIndexesneedstobeenabledforthissettingtohaveaneffect.Thisallowsforbetterstatsgatheringfromsourcesthatdon’tsupportreturningastatisticalindex.
false
importStatistics
truewillusedatabasedependentlogictodeterminethecardinalityifnoneisdetermined.Notyetsupportedbyalldatabasetypes-currentlyonlysupportedbyOracleandMySQL.
false
importRowIdAsBinary truewillimportRowIdcolumnsasvarbinaryvalues. false
JDBCTranslators
542
[1]JavaDocforDatabaseMetaData[2]Thefullyqualifiednameforexclusionisbaseduponthesettingsofthetranslatorandtheparticularsofthedatabase.Alloftheapplicablenamepartsusedbythetranslatorsettings(seeuseQualifiedNameanduseCatalogName)includingcatalog,schema,tablewillbecombinedascatalogName.schemaName.tableNamewithnoquoting.ForexampleOracledoesnotreportacatalog,sothenameusedwithdefaultsettingsforcomparisonwouldbejustschemaName.tableName.
WarningThedefaultimportsettingswillcrawlallavailablemetadata.Thisimportprocessistimeconsumingandfullmetadataimportisnotneededinmostsituations.Mostcommonlyyou’llwanttolimittheimportbyatleastschemaPatternandtableTypes.
Exampleimportersettingstoonlyimporttablesandviewsfrommy-schema.SeealsoVDBGuide
<model...
<propertyname="importer.tableTypes"value="TABLE,VIEW"/>
<propertyname="importer.schemaPattern"value="my-schema"/>
...
</model>
NativeQueries
Physicaltables,functions,andproceduresmayoptionallyhavenativequeriesassociatedwiththem.Novalidationofthenativequeryisperformed,itissimplyusedinastraight-forwardmannertogeneratethesourceSQL.Foraphysicaltablesettingtheteiid_rel:native-queryextensionmetadatawillexecutethenativequeryasaninlineviewinthesourcequery.Thisfeatureshouldonlybeusedagainstsourcesthatsupportinlineviews.Thenativequeryisusedasisandisnottreatedasaparameterizedstring.ForexampleonaphysicaltableywithnameInSource="x"andteiid_rel:native-query="selectcfromg",theTeiidsourcequery"SELECTcFROMy"wouldgeneratetheSQLquery"SELECTcFROM(selectcfromg)asx".NotethatthecolumnnamesinthenativequerymustmatchthenameInSourceofthephysicaltablecolumnsfortheresultingSQLtobevalid.
Forphysicalproceduresyoumayalsosettheteiid_rel:native-queryextensionmetadatatoadesiredquerystringwiththeaddedabilitytopositionallyreferenceINparameters-seeParameterizableNativeQueries.Theteiid_rel:non-preparedextensionmetadatapropertymaybesettofalsetoturnoffparameterbinding.NotethisoptionshouldbeusedwithcautionasinboundmayallowforSQLinjectionattacksifnotproperlyvalidated.Thenativequerydoesnotneedtocallastoredprocedure.AnySQLthatreturnsaresultsetpositionallymatchingtheresultsetexpectedbythephysicalstoredproceduremetadatawillwork.Forexampleonastoredprocedurexwithteiid_rel:native-query="selectcfromgwherec1=$1andc2=`$$1"',theTeiidsourcequery"CALLx(?)"wouldgeneratetheSQLquery"selectcfromgwherec1=?andc2=`$1"'.Notethat?inthisexamplewillbereplacedwiththeactualvalueboundtoparameter1.
DirectQueryProcedure
Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyisnative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheJDBCtranslatorprovidesaproceduretoexecuteanyad-hocSQLquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.
SelectExample
SELECTx.*FROM(calljdbc_source.native('select*fromg1'))w,
ARRAYTABLE(w.tupleCOLUMNS"e1"integer,"e2"string)ASx
InsertExample
JDBCTranslators
543
SELECTx.*FROM(calljdbc_source.native('insertintog1(e1,e2)values(?,?)',112,'foo'))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
UpdateExample
SELECTx.*FROM(calljdbc_source.native('updateg1sete2=?wheree1=?','blah',112))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
DeleteExample
SELECTx.*FROM(calljdbc_source.native('deletefromg1wheree1=?',112))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
JCAResourceAdapterTheresourceadapterforthistranslatorprovidedthroughdatasourceinWildFly,SeetoAdminGuidesectionWildFlyDataSourcesforconfiguration.
JDBCTranslators
544
ActianVectorTranslator(actian-vector)AlsoseecommonJDBCTranslatorInformation
TheActianVectorTranslator,knownbythetypenameactian-vector,isforuseActianVectorinHadoop.
DownloadtheJDBCdriverathttp://esd.actian.com/platform.NotetheportnumberinconnectionURLis"AH7"whichmapsto16967.
JDBCTranslators
545
ApacheHBaseTranslator(phoenix)AlsoseecommonJDBCTranslatorInformation
TheApachePhoenixTranslator,knownbythetypenamephoenix,exposesqueryingfunctionalitytoHBaseTables.ApachePhoenixisaJDBCSQLinterfaceforHBase-seePhoenixDataSourcesthatisrequiredforthistranslatorasitpushesdowncommandsintoPhoenixSQL.
Thetranslatorisalsoknownbythedeprecatednamehbase.ThenamechangereflectsthatthetranslatorisspecifictophoenixandtheremaybeothertranslatorsintroducedinthefurturethatalsoconnecttoHBase.
TheDatabaseTimezonepropertyshouldnotbeusedwiththistranslator.
TheHBaseTranslatordoesn’tsupportJoins.PhoenixusestheHBaseTableRowIDasthePrimaryKey,whichmapto.ThisTranslatorisdevelopedwithPhoenix4.3+forHBase0.98.1+.
NoteThetranslatorimplementsINSERT/UPDATEthroughthePhoenixUPSERToperation.ThismeansyoucanseedifferentbehaviorthanwithstandardINSERT/UPDATE-suchasrepeatedinsertswillnotthrowaduplicatekeyexception,butwillinsteadupdatetherowinquestion.
Note DuetoPhoenixdriverlimitationstheimporterwillnotlookforuniqueconstraintsanddefaultstonotimportingforeignkeys.
Note
ThetranslatorsupportsoffsetandotherfeaturesstartingwithPhoenix4.8.ThePhoenixdriverhardcodestheserverversioninPhoenixDatabaseMetaData,anddoesnototherwiseprovideawaytodetecttheserverversionatruntime.Ifanewerdriverisusedwithanolderserver,pleasesetthedatabaseversiontranslatorpropertymanually.
WarningThePhoenixdriverdoesnothaverobusthandlingoftimevalues.Ifyourtimevaluesarenormalizedtouseadatecomponentof1970-01-01,thenthedefaulthandlingwillworkcorrectly.Ifnot,thenthetimecolumnshouldbemodeledastimestampinstead.
JDBCTranslators
546
ClouderaImpalaTranslator(impala)AlsoseecommonJDBCTranslatorInformation
TheCouderaImpalaTranslator,knownbythetypenameimpala,isforusewithClouderaImpala1.2.1orlater.
Impalahaslimitedsupportfordatatypes.Itisdoesnothavenativesupportfortime/date/xmlorLOBs.Theselimitationsarereflectedinthetranslatorcapabilities.ATeiidviewcanusethesetypes,howeverthetransformationwouldneedtospecifythenecessaryconversions.Notethatinthosesituations,theevaluationswillbedoneintheTeiidengine.
TheDatabaseTimeZonetranslatorpropertyshouldnotbeused.
ImpalaonlysupportsEQUIjoin,sousinganyotherjoinstypesonitssourcetableswillresultininefficientqueries.
Towritecriteriabasedonpartitionedcolumns,modeledthemonsourcetable,butdonotincludetheminselectioncolumns.
Note ImpalaHiveimporterdoesnothaveconceptofcatalogorsourceschema,nordoesitimportkeys,procedures,indexes,etc.
Impalaspecificimporterproperties:
useDatabaseMetaData-Settotruetousethenormalimportlogicwiththeoptiontoimportindexinformationdisabled.Defaultstofalse.
Note IfuseDatabaseMetaDataisfalsethetypicalJDBCDatabaseMetaDatacallsarenotusedsonotallofthecommonJDBCimporterpropertiesareapplicabletoImpala.YoumaystilluseexcludeTablesregardless.
Note
SomeversionsofImpalarequirestheuseofaLIMITwhenperforminganORDERBY.IfnodefaultisconfiguredinImpala,thenanexceptioncanoccurwhenaTeiidquerywithanORDERBYbutnoLIMITisissued.YoushouldsetanImpalawidedefault,orconfiguretheconnectionpooltouseanewconnectionsqlstringtoissueaSETDEFAULT_ORDER_BY_LIMITstatement.SeetheClouderadocsformoreonlimitoptions-suchascontrollingwhathappenswhenthelimitisexceeded.
Note TheImpalaJDBCdriverseemstohaveissueswithPreparedStatementsandstatementparsingingeneralthatmayrequiredisablinguseBindVariables-seelink:https://issues.jboss.org/browse/TEIID-4610
JDBCTranslators
547
DB2Translator(db2)AlsoseecommonJDBCTranslatorInformation
TheDB2Translator,knownbythetypenamedb2,ifforusewithDB28orlaterandDB2fori5.4orlater.
DB2specificexecutionproperties:
DB2ForI-indicatesthatthetheDB2instanceisDB2fori.Defaultstofalse.
supportsCommonTableExpressions-indicatesthattheDB2instancesupportsCommonTableExpressions.Defaultstotrue.Someolderversions,orinstancesrunninginaconversionmode,ofDB2lackfullcommontableexpressionsupportandmayneedsupportdisabled.
JDBCTranslators
548
DerbyTranslator(derby)AlsoseecommonJDBCTranslatorInformation
TheDerbyTranslator,knownbythetypenamederby,isforusewithDerby10.1orlater.
JDBCTranslators
549
GreenplumTranslator(greenplum)AlsoseecommonJDBCTranslatorInformation
TheGreenplumTranslator,knownbythetypenamegreenplum,isforusewiththeGreenplumdatabase.ThistranslatorisanextensionofthePostgreSQLTranslatorandinheritsitsoptions.
JDBCTranslators
550
H2Translator(h2)AlsoseecommonJDBCTranslatorInformation
TheH2Translator,knownbythetypenameh2,isforusewithH2version1.1orlater.
JDBCTranslators
551
HiveTranslator(hive)AlsoseecommonJDBCTranslatorInformation
TheHiveTranslator,knownbythetypenamehive,isforusewithHivev.10andSparkSQLv1.0andlater.
Capabilities
Hivehaslimitedsupportfordatatypes.Itdoesnothavenativesupportfortime/xmlorLOBs.Theselimitationsarereflectedinthetranslatorcapabilities.ATeiidviewcanusethesetypes,howeverthetransformationwouldneedtospecifythenecessaryconversions.Notethatinthosesituations,theevaluationswillbedoneinTeiidengine.
TheDatabaseTimeZonetranslatorpropertyshouldnotbeused.
HiveonlysupportsEQUIjoin,sousinganyotherjoinstypesonitssourcetableswillresultininefficientqueries.
Towritecriteriabasedonpartitionedcolumns,modeledthemonsourcetable,butdonotincludetheminselectioncolumns.
Note TheHiveimporterdoesnothaveconceptofcatalogorsourceschema,nordoesitimportkeys,procedures,indexes,etc.
ImportProperties
trimColumnNames-ForHive0.11.0andlaterthetheDESCRIBEcommandmetadataisinappropriatelyreturnedwithpadding,settotruetostriptrimwhitespacefromcolumnnames.Defaultstofalse.
useDatabaseMetaData-ForHive0.13.0andlaterthenormalJDBCDatabaseMetaDatafacilitiesaresufficienttoperformanimport.Settotruetousethenormalimportlogicwiththeoptiontoimportindexinformationdisabled.Defaultstofalse.Whentrue,trimColumnNameshasnoeffect.
Note IffalsethetypicalJDBCDatabaseMetaDatacallsarenotusedsonotallofthecommonJDBCimporterpropertiesareapplicabletoHive.YoumaystilluseexcludeTablesregardless.
"DatabaseName"
WhenthedatabasenameusedintheHiveisdifferentthan"default",themetadataretrievalandexecutionofqueriesdoesnotworkasexpectedinTeiid,asHiveJDBCdriverseemstobeimplicitlyconnecting(testedwith<0.12)to"default"database,thusignoringthedatabasenamementionedonconnectionURL.ThiscanworkaroundintheTeiidinWildFlyenvironmentbysettingthefollowingindatasourceconfiguration.
<new-connection-sql>use{database-name}</new-connection-sql>
Thisisfixedin>0.13versionHiveDriver.Seehttps://issues.apache.org/jira/browse/HIVE-4256
JDBCTranslators
552
HSQLTranslator(hsql)AlsoseecommonJDBCTranslatorInformation
TheHSQLTranslator,knownbythetypenamehsql,isforusewithHSQLDB1.7orlater.
JDBCTranslators
553
InformixTranslator(informix)AlsoseecommonJDBCTranslatorInformation
TheInformixTranslator,knownbythetypenameinformix,isforusewithanyInformixversion.
KnownIssues
TEIID-3808-TheInformixdriverhandlingoftimezoneinformationisinconsistent-evenifthedatabaseTimezonetranslatorpropertyisset.ConsiderensuringthattheInformixserverandtheapplicationserverareinthesametimezone.
JDBCTranslators
554
IngresTranslators(ingres/ingres93)AlsoseecommonJDBCTranslatorInformation
TheIngrestranslationissupportedby2translators.
ingres
TheIngresTranslator,knownbythetypenameingres,isforusewithIngres2006orlater.
ingres93
TheIngres93Translator,knownbythetypenameingres93,isforusewithIngres9.3orlater.
JDBCTranslators
555
IntersystemsCacheTranslator(intersystems-cache)AlsoseecommonJDBCTranslatorInformation
TheIntersystemCacheTranslator,knownbythetypenameintersystems-cache,isforusewithIntersystemsCacheObjectdatabase(onlyrelationalaspectofit).
JDBCTranslators
556
JDBCANSITranslator(jdbc-ansi)AlsoseecommonJDBCTranslatorInformation
TheJDBCANSItranslator,knownbythetypenamejdbc-ansi,declaressupportformostSQLconstructssupportedbyTeiid,exceptforrowlimit/offsetandEXCEPT/INTERSECT.TranslatessourceSQLintoANSIcompliantsyntax.Thistranslatorshouldbeusedwhenanothermorespecifictypeisnotavailable.IfsourceexceptionsarisefromunsupportedSQL,thenconsiderusingtheJDBCSimpleTranslatortofurtherrestrictcapabilities,orcreateaCustomTranslator/createanenhancementrequest.
JDBCTranslators
557
JDBCSimpleTranslator(jdbc-simple)AlsoseecommonJDBCTranslatorInformation
TheJDBCSimpletranslator,knownbythetypenamejdbc-simple,isthesameasjdbc-ansi,exceptdisablessupportfornearlyallpushdownconstructsformaximumcompatibility.
JDBCTranslators
558
MetaMatrixTranslator(metamatrix)AlsoseecommonJDBCTranslatorInformation
TheMetaMatrixTranslator,knownbythetypenamemetamatrix,isforusewithMetaMatrix5.5.0orlater.
JDBCTranslators
559
MicrosoftAccessTranslatorsAlsoseecommonJDBCTranslatorInformation
access
TheMicrosoftAccessTranslatorknownbythetypenameaccessisforusewithMicrosoftAccess2003orlaterviatheJDBC-ODBCbridge.
IfyouareusingthedefaultnativemetadataimportortheTeiidconnectionimportertheimporterdefaultstoimportKeys=falseandexcludeTables=.[.]MSys.toavoidissueswiththemetadataprovidedbytheJDBCODBCbridge.YoumayneedtoadjustthesevaluesifyouuseadifferentJDBCdriver.
ucanaccess
TheMicrosoftAccessTranslatorknownbythetypenameucanaccessisforusewithMicrosoftAccess2003orlaterviathefortheUCanAccessdriver.
JDBCTranslators
560
MicrosoftSQLServerTranslator(sqlserver)AlsoseecommonJDBCTranslatorInformation
TheMicrosoftSQLServerTranslator,knownbythetypenamesqlserver,isforusewithSQLServer2000orlater.ASQLServerJDBCdriverversion2.0orlater(orcompatiblee.g.JTDS1.2orlater)shouldbeused.TheSQLServerDatabaseVersionpropertymaybesetto2000,2005,2008,or2012,butotherwiseexpectsastandardversionnumber-e.g."10.0".
SequenceSupport
WithTeiid8.5+sequenceoperationsmaybemodeledassourcefunctions.
Example:SequenceNativeQuery
CREATEFOREIGNFUNCTIONseq_nextval()returnsintegerOPTIONS("teiid_rel:native-query"'NEXTVALUEFORseq');
ExecutionProperties
SQLServerspecificexecutionproperties:
JtdsDriver-indicatesthattheopensourceJTDSdriverisbeingused.Defaultstofalse.
JDBCTranslators
561
ModeShapeTranslator(modeshape)AlsoseecommonJDBCTranslatorInformation
TheModeShapeTranslator,knownbythetypenamemodeshape,isforusewithModeshape2.2.1orlater.
Usage
ThePATH,NAME,LOCALNODENAME,DEPTH,andSCOREfunctionsshouldbeaccessedaspseudo-columns,e.g."nt:base"."jcr:path".
TeiidUFDs(prefixedbyJCR_)areavailableforCONTIANS,ISCHILDNODE,ISDESCENDENT,ISSAMENODE,REFERENCE-seetheJCRFunctions.xmi.IfaselectornameisneededinaJCRfunction,youshouldusethepseudo-column"jcr:path",e.g.JCR_ISCHILDNODE(foo.jcr_path,'x/y')wouldbecomeISCHILDNODE(foo,`x/y')intheModeShapequery.
Anadditionalpseudo-column"mode:properties"shouldbeimportedbysettingtheModeShapeJDBCconnectionpropertyteiidsupport=true.Thecolumn"mode:properties"shouldbeusedbytheJCR_REFERENCEandotherfunctionsthatexpecta.*selectorname,e.g.JCR_REFERENCE(nt_base.jcr_properties)wouldbecomeREFERENCE("nt:base".*)intheModeShapequery.
JDBCTranslators
562
MySQLTranslator(mysql/mysql5)AlsoseecommonJDBCTranslatorInformation
MySQL/MariaDBtranslationissupportedby2translators.
mysql
TheMysqltranslator,knownbythetypenamemysql,isforusewithMySQLversion4.x.
mysql5
TheMysql5translator,knownbythetypenamemysql5,isforusewithMySQLversion5orlater.
AlsosupportscompatibleMySQLderivativesincludingMariaDB.
UsageTheMySQLTranslatorsexpectthedatabaseorsessiontobeusingANSImode.IfthedatabaseisnotusingANSImode,aninitializationqueryshouldbeusedonthepooltosetANSImode:
setSESSIONsql_mode='ANSI'
Ifyoumaydealwithnulltimestampvalues,thensettheconnectionpropertyzeroDateTimeBehavior=convertToNull.Otherwiseyou’llgetconversionerrorsinTeiidthat`0000-00-0000:00:00'cannotbeconvertedtoatimestamp.
Warning Ifretrievinglargeresultsets,youshouldconsidersettingtheconnectionpropertyuseCursorFetch=true,otherwiseMySQLwillfullyfetchresultsetsintomemoryontheTeiidinstance.
Note
MySQLreportsTINYINT(1)columnsasaJDBCBITtype-howeverthevaluerangeisnotactuallyrestrictedandmaycauseissuesifforexampleyouarerelyingon-1beingrecognizedasatruevalue.Ifnotusingthenativeimporter,youshouldchangeaffectedsourceBOOLEANcolumnstohaveanativetypeof"TINYINT(1)"ratherthanBITsothatthetranslatorcanappropriatelyhandlethebooleanconversion.
JDBCTranslators
563
NetezzaTranslator(netezza)AlsoseecommonJDBCTranslatorInformation
TheNetezzaTranslator,knownbythetypenamenetezza,isforusewithanyNetezzaversion.
Usage
ThecurrentNetezzavendorsuppliedJDBCdriverperformspoorlywithsingletransactionalupdates.Asisgenerallythecasewhenpossibleusebatchedupdates.
ExecutionProperties
Netezzaspecificexecutionproperties:
SqlExtensionsInstalled-indicatesthatSQLExtensions,includingsupportforREGEXP_LIKE,areinstalled.AllotherREGEXPfunctionsarethenavailableaspushdownfunctions.Defaultstofalse.
JDBCTranslators
564
OracleTranslator(oracle)AlsoseecommonJDBCTranslatorInformation
TheOracleTranslator,knownbythetypenameoracle,isforusewithOracle9iorlater.
ImporterProperties
useGeometryType-UsetheTeiidGeometytypewhenimportingcolumnswithasourcetypeofSDO_GEOMETRY.Defaultstofalse.
useIntegralTypes-Useintegraltypesratherthandecimalwhenthescaleis0.Defaultstofalse.
ExecutionPropertiesOracleSuppliedDriver-indicatesthattheOraclesupplieddriver(typicallyprefixedbyojdbc)isbeingused.Defaultstotrue.SettofalsewhenusingDataDirectorotherOracleJDBCdrivers.
OracleSpecificMetadata
Sequences
SequencesmaybeusedwiththeOracletranslator.AsequencemaybemodeledasatablewithanameinsourceofDUALandcolumnswiththenameinsourcesetto<sequencename>.[nextval|currval]
Teiid8.4andPriorOracleSequenceDDL
CREATEFOREIGNTABLEseq(nextvalintegerOPTIONS(NAMEINSOURCE'seq.nextval'),currvalintegeroptions(NAMEIN
SOURCE'seq.currval'))OPTIONS(NAMEINSOURCE'DUAL')
WithTeiid8.5it’snolongernecessarytorelyonatablerepresentationandOraclespecifichandlingforsequences.SeeDDLMetadataforrepresentingcurrvalandnextvalassourcefunctions.
8.5Example:SequenceNativeQuery
CREATEFOREIGNFUNCTIONseq_nextval()returnsintegerOPTIONS("teiid_rel:native-query"'seq.nextval');
Youcanalsouseasequenceasthedefaultvalueforinsertcolumnsbysettingthecolumntoautoincrementandthenameinsourceto<elementname>:SEQUENCE=<sequencename>.<sequencevalue>.
Rownum
ArownumcolumncanalsoaddedtoanyOraclephysicaltabletosupporttherownumpseudo-column.Arownumcolumnshouldhaveanameinsourceofrownum.TheserownumcolumnsdonothavethesamesemanticsastheOraclerownumconstructsocaremustbetakenintheirusage.
OutParameterResultSet
Outparametersforproceduresmayalsobeusedtoreturnaresultset,ifthisisnotrepresentedcorrectlybytheautomaticimportyouneedtomanuallycreatearesultsetandrepresenttheoutputparameterwithnativetype"REFCURSOR".
JDBCTranslators
565
DDLforoutparameterresultset
createforeignprocedureproc(inxinteger,outyobjectoptions(native_type'REFCURSOR'))
returnstable(ainteger,bstring)
GeoSpatialfunctionsupport
Oracletranslatorsupportsgeospatialfunctions.Thesupportedfunctionsare:
Relate=sdo_relate
CREATEFOREIGNFUNCTIONsdo_relate(arg1string,arg2string,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_relate(arg1Object,arg2Object,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_relate(arg1string,arg2Object,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_relate(arg1Object,arg2string,arg3string)RETURNSstring;
Nearest_Neighbor=sdo_nn
CREATEFOREIGNFUNCTIONsdo_nn(arg1string,arg2Object,arg3string,arg4integer)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_nn(arg1Object,arg2Object,arg3string,arg4integer)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_nn(arg1Object,arg2string,arg3string,arg4integer)RETURNSstring;
Within_Distance=sdo_within_distance
CREATEFOREIGNFUNCTIONsdo_within_distance(arg1Object,arg2Object,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_within_distance(arg1string,arg2Object,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_within_distance(arg1Object,arg2string,arg3string)RETURNSstring;
Nearest_Neigher_Distance=sdo_nn_distance
CREATEFOREIGNFUNCTIONsdo_nn_distance(arginteger)RETURNSinteger;
Filter=sdo_filter
CREATEFOREIGNFUNCTIONsdo_filter(arg1Object,arg2string,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_filter(arg1Object,arg2Object,arg3string)RETURNSstring;
CREATEFOREIGNFUNCTIONsdo_filter(arg1string,arg2object,arg3string)RETURNSstring;
JDBCTranslators
566
OSISoftPITranslator(osisoft-pi)AlsoseecommonJDBCTranslatorInformation
TheOSISoftTranslator,knownbythetypenameosisoft-pi,isforusewithOSIsoftPIOLEDBEnterprise.ThistranslatorusestheJDBCdriverprovidedbytheOSISoft.ThedriverisnotprovidedwithTeiidinstall,thisneedsbedownloadedfromOSISoftandinstalledcorrectlyonTeiidserveraccordingtoOSISoftdocumentationPI-JDBC-2016-Administrator-Guide.pdforlatestdocument.
InstallonLinux
MakesureyouhaveOpenSSLlibrariesinstalled,andyouhavefollowing"export"addedcorrectlyinyourshellenvironmentvariables.Otherwiseyoucanalsoaddin<WildFly>/bin/standalone.shfileor<WildFly>/bin/domain.shfile.
exportPI_RDSA_LIB=/<path>/pipc/jdbc/lib/libRdsaWrapper-1.5b.so
exportPI_RDSA_LIB64=/<path>/pipc/jdbc/lib/libRdsaWrapper64-1.5b.so
PleasealsonotetoexecutefromLinux,youalsoneedinstall'gSoap'library,asPIJDBCdriverusesSOAPoverHTTPStocommunicatewithPIserver.
InstallonWindows
FollowtheinstallationprogramprovidedbyOSISoftforinstallingtheJDBCdrivers.Makesureyouhavethefollowingenvironmentvariablesconfigured.
PI_RDSA_LIBC:\ProgramFiles(x86)\PIPC\JDBC\RDSAWrapper.dll
PI_RDSA_LIB64C:\ProgramFiles\PIPC\JDBC\RDSAWrapper64.dll
InstallingtheJDBCdriverforTeiid(sameforbothLinuxandWindows)
Thencopythemoduledirectoryfrom<WildFly>/teiid/datasources/osisoft-pi/modulesdirectoryinto_<WilfFly>/modulesdirectory.Thenfindthe"PIJDBCDriver.jar"filefromtheinstallationdirectory,andcopyitto_<WildFly>/module/system/layers/dv/com/osisoft/main"directory.Thenaddthedriverdefinitiontothestandalone.xmlfilebyeditingthefileandaddingsomethingsimilartobelow
<drivers>
<drivername="osisoft-pi"module="com.osisoft">
<driver-class>com.osisoft.jdbc.Driver</driver-class>
</driver>
</drivers>
ThatcompletestheconfigurationofthePIdriverintheTeiid.WestillhavenotcreatedaconnectiontothePIserver.youcanstarttheservernow.
CreatingaDataSourcetoPI
YoucanexecutefollowingsimilarCLIscripttocreateadatasource
JDBCTranslators
567
/subsystem=datasources/data-source=pi-ds:add(jndi-name=java:/pi-ds,driver-
name=osisoft-pi,connection-url=jdbc:pioledbent://<DACServer>/DataSource=<AF
Server>;IntegratedSecurity=SSPI,user-name=user,password=mypass)
/subsystem=datasources/data-source=pi-ds:enable
thiswillcreatefollowingXMLinstandalone.xmlordomain.xml(youcanalsodirectlyeditthesefilesandaddmanually)
<datasourcejndi-name="java:/pi-ds"pool-name="pi-ds">
<connection-url>jdbc:pioledbent://<DACServer>/DataSource=<AFServer>;
IntegratedSecurity=SSPI</connection-url>
<driver>osisoft-pi</driver>
<pool>
<prefill>false</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<user-name>user</user-name>
<password>mypass</password>
</security>
</datasource>
NowyouhavefullyconfiguredtheTeiidwithPIdatabaseconnection.YoucancreateVDBthatcanusethisconnectiontoissuethequeries.
UsageYoucandevelopaVDBlikefollowstofetchmetadatafromPIandgiveyouaccesstoexecutingqueriesagainstPI.
pi-vdb.xml
<vdbname="pi"version="1">
<modelname="AF">
<propertyname="importer.importProcedures"value="true"/>
<sourceconnection-jndi-name="java:/pi-ds"name="pi-connector"translator-
name="osisoft-pi"/>
</model>
</vdb>
DeploythisfileintoTeiidusingCLIorusingmanagementconsole
deploypi-vdb.xml
OncethemetadataisloadedandVDBisactiveyoucanuseTeiidJDBC/ODBCdriverorODatatoconnecttotheVDBandissuequeries.
PITranslatorCapabilities
JDBCTranslators
568
PItranslatorisextensionofjdbc-ansitranslator,soalltheSQLANSIqueriesaresupported.PItranslatoralsosupportsLATERALjoinwithTableValuedFunctions(TVF).AnexampleTeiidquerylookslike
{code}SELECTEH.Name,BT."Time",BT."NumberofComputers",BT."Temperature"FROMSample.Asset.ElementHierarchyEHLEFTJOINLATERAL(exec"TransposeArchive_BuildingTemplate"(EH.ElementID,TIMESTAMPADD(SQL_TSI_HOUR,-1,now()),now()))BTon1=1WHEREEH.ElementIDIN(SELECTElementIDFROMSample.Asset.ElementHierarchyWHEREPath='\DataCenter\'){code}
NoteANSISQLsemanticsrequireaONclause,butCROSSAPPLYorOUTERAPPLYdonohaveaONclause,soforthisreasonuserneedtopassinadummyONclauselikeON(1=1),whichwillbeignoredwhenconvertedtoAPPLYclausewhichwillbepusheddown.
Bydefaultthistranslatorturnsoffthe"importer.ImportKeys"tofalse.
NoteThePIdatatype,"GUID"willneedtobemodeledas"String"andmustdefinetheNATIVE_TYPEoncolumnas"guid",thenTeiidtranslatorwillappropriatelyconvertthedatabackforthwiththePIdatasource’snativeguidtypewithappropriatetypecastingfromstring.
JDBCTranslators
569
PostgreSQLTranslator(postgresql)AlsoseecommonJDBCTranslatorInformation
ThePostgreSQLTranslator,knownbythetypenamepostgresql,isforusewith8.0orlaterclientsand7.1orlaterserver.
ExecutionProperties
PostgreSQLspecificexecutionproperties:
PostGisVersion-indicatethePostGISversioninuse.Defaultsto0meaningPostGISisnotinstalled.Willbesetautomaticallyifthedatabaseversionisnotset.
ProjSupported-booleanindicatingifProjissupportforPostGis.Willbesetautomaticallyifthedatabaseversionisnotset.
Note SomedriverversionsofPostgreSQLwillnotassociatecolumnsto"INDEX"typetablesandfailduringimportiftheimporter.tableTypepropertyisleftasthedefault.
JDBCTranslators
570
PrestoDBTranslator(prestodb)AlsoseecommonJDBCTranslatorInformation
ThePrestoDBtranslator,knownbythetypenameprestodb,exposesqueryingfunctionalitytoPrestoDBDataSources.Indataintegrationrespect,PrestoDBhasverysimilarcapabilitiesofTeiid,howeveritgoesbeyondintermsofdistributedqueryexecutionwithmultipleworkernodes.Teiid’sexecutionmodelislimitedtosingleexecutionnodeandfocusesmoreonpushingthequerydowntosources.CurrentlyTeiidhasmuchmorecompletequerysupportandmanyenterprisefeatures.
Capabilities
ThePrestoDBtranslatorsupportsonlySELECTstatementswitharestrictivesetofcapabilities.Thistranslatorisdevelopedwith0.85versionofPrestoDBandcapabilitiesaredesignedforthisversion.WithnewversionsofPrestoDBTeiidwilladjustthecapabilitiesofthistranslator.SincePrestoDBexposesarelationalmodel,theusageofthisisnodifferentthananyRDBMSsourcelikeOracle,DB2etc.ForconfiguringthePrestoDBconsultthePrestoDBdocumentation.
Tip PrestoDBnotsupportmultiplecolumnsintheORDERBYwithinJOINsituationswell,thetranslatorpropertysupportsOrderBycanusetodisableOrderbyinsomespecificsituations.
Tip SomeversionsofPrestoDBnotsupportnullasvalidvaluesinsubquerywell,tocheckthePrestoDBwhethersupportnullasvalidvaluesinsubqueryifyouhitreleatederror.
Tip PrestoDBnotsupporttransaction,defineano-tx-datasourceisrecommend.
Note
EverycatalogsinPrestoDBhasainformation_schemaschemabydefault,Ifconfiguremultiplecatalogs,itshouldbeconsidertouseimportoptionstofiltertheschemas/tables,toavoidDuplicateTableerrorcauseVDBdeployfailed.Forinstance,setcatalogtoaspecificcatalognametomatchthecatalognameasitisstoredintheprestodb,setschemaPatterntoaregularexpressiontofilterschemasbymatchingresult,setexcludeTablestoaregularexpressiontofiltertablesbymatchingresult.
NotePrestoDBJDBCdriverusesJoda-Timelibrarytoworkwithtime/date/timestamp.Ifyouneedtocustomizeserver’stimezone(e.g.setting-Duser.timezoneviaJAVA_OPTS),youcannotuseGMT/…IDasJoda-Timedoesnotregonizeit.However,youcanuseequivalentEtc/…ID.FormoredetailsseeJoda-Timetimezones.
JDBCTranslators
571
RedshiftTranslator(redshift)AlsoseecommonJDBCTranslatorInformation
TheRedshiftTranslator,knownbythetypenameredshift,isforusewiththeRedshiftdatabase.ThistranslatorisanextensionofthePostgreSQLTranslatorandinheritsitsoptions.
JDBCTranslators
572
SAPHanaTranslator(hana)AlsoseecommonJDBCTranslatorInformation
TheSAPHanaTranslator,knownbythenameofhana,isforusewithSAPHana.
KnownIssues
TEIID-3805-ThepushdownofthesubstringfunctionisinconsistentwiththeTeiidsubstringfunctionwhenthefromindexexceedsthelengthofthestring.SAPHanawillreturnanemptystring,whileTeiidproducesanullvalue.
JDBCTranslators
573
SybaseIQTranslator(sybaseiq)AlsoseecommonJDBCTranslatorInformation
TheSybaseIQTranslator,knownbythetypenamesybaseiq,isforusewithSybaseIQversion15.1orlater.
JDBCTranslators
574
SybaseTranslator(sybase)TheSybaseTranslator,knownbythetypenamesybase,isforusewithSybaseversion12.5orlater.
Ifusingthethedefaultnativeimportandnoimportpropertiesarespecified(notrecommended,seeimportpropertiesbelow),thenexceptionscanbethrownretrievingsystemtableinformation.YoushouldspecifyaschemaPatternoruseexcludeTablestoexcludesystemtablesifthisoccurs.
Ifthenameinsourcemetadatacontainsquotedidentifiers(suchasrequiredbyreservedwordsorwordscontainingcharactersthatwouldnototherwisebeallowed)andyouareusingajconnectSybasedriver,youmustfirstconfiguretheconnectionpooltoenablequoted_identifier:
DriverURLwithSQLINITSTRING
jdbc:sybase:Tds:host.at.some.domain:5000/db_name?SQLINITSTRING=setquoted_identifieron
IfyouareajconnectSybasedriverandwilltargetthesourcefordependentjoins,youshouldallowthetranslatortosendmorevaluesbysettingtheJCONNECT_VERSION.Otherwiseyouwillgetexceptionswithstatementsthathavemorethan481bindvalues:
DriverURLwithJCONNECT_VERSION
jdbc:sybase:Tds:host.at.some.domain:5000/db_name?SQLINITSTRING=setquoted_identifieron&JCONNECT_VERSION=6
Sybasespecificexecutionproperties:
JtdsDriver-indicatesthattheopensourceJTDSdriverisbeingused.Defaultstofalse.
JDBCTranslators
575
TeiidTranslator(teiid)AlsoseecommonJDBCTranslatorInformation
TheTeiidTranslator,knownbythetypenameteiid,isforusewithTeiid6.0orlater.
JDBCTranslators
576
TeradataTranslator(teradata)AlsoseecommonJDBCTranslatorInformation
TheTeradataTranslator,knownbythetypenameteradata,isforusewithTeradataV2R5.1orlater.
WithTeradatadriverversion15date,time,andtimestampvaluesbydefaultwillbeadjustedfortheTeiidservertimezone.Toremovethisadjustment,setthetranslatorDatabaseTimezonepropertytoGMTorwhatevertheTeradataserverdefaultsto.
JDBCTranslators
577
VerticaTranslator(vertica)AlsoseecommonJDBCTranslatorInformation
TheVerticaTranslator,knownbythetypenamevertica,isforusewithVertica6orlater.
JDBCTranslators
578
JPATranslatorTheJPAtranslator,knownbythetypenamejpa2,canreverseaJPAobjectmodelintoarelationalmodel,whichcanthenbeintegratedwithotherrelationalornon-relationalsources.ForinformationonJPApersistenceinaWildFly,seeJPAReferenceGuide.
Properties
TheJPATranslatorcurrentlyhasnoimportorexecutionproperties.
NativeQueriesJPAsourceproceduresmaybecreatedusingtheteiid_rel:native-queryextension-seeParameterizableNativeQueries.Theprocedurewillinvokethenative-querysimilartoannativeprocedurecallwiththebenefitsthatthequeryispredeterminedandthatresultcolumntypesareknown,ratherthanrequiringtheuseofARRAYTABLEorsimilarfunctionality.Seethequerysyntaxbelow.
DirectQueryProcedure
Note Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyisnative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheJPAtranslatorprovidesaproceduretoexecuteanyad-hocJPA-QLquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasobjectarray.UsercanuseARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.Teiidexposesthisprocedurewithasimplequerystructureasbelow
Select
SelectExample
SELECTx.*FROM(calljpa_source.native('search;FROMAccount'))w,
ARRAYTABLE(w.tupleCOLUMNS"id"string,"type"string,"name"String)ASx
fromtheabovecode,the"search"keywordfollowedbyaquerystatement-seeParameterizableNativeQueriestosubstituteparametervalues.
Delete
DeleteExample
SELECTx.*FROM(calljpa_source.native('delete;<jpa-ql>'))w,
ARRAYTABLE(w.tupleCOLUMNS"updatecount"integer)ASx
formtheabovecode,the"delete"keywordfollowedbyJPA-QLfordeleteoperation.
Update
JPATranslator
579
CreateExample
SELECTx.*FROM
(calljpa_source.native('update;<jpa-ql>'))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
formtheabovecode,the"update"keywordmustbefollowedbyJPA-QLfortheupdatestatement.
Create
UpdateExample
SELECTx.*FROM
(calljpa_source.native('create;',<entity>))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
Createoperationneedstosend"create"wordasmarkerandsendtheentityasathefirstparameter.
JPATranslator
580
LDAPTranslatorTheLDAPtranslatorisimplementedbytheorg.teiid.translator.ldap.LDAPExecutionFactoryclassandknownbythetranslatortypenameldap.TheLDAPtranslatorexposesanLDAPdirectorytreerelationallywithpushdownsupportforfilteringviacriteria.ThisistypicallycoupledwiththeLDAPresourceadapter.
Note TheresourceadapterforthistranslatorisprovidedbyconfiguringtheldapdatasourceintheJBossEAPinstance.
ExecutionProperties
Name Description Default
SearchDerfaultBaseDN DefaultBaseDNforLDAPSearches null
SearchDefaultScope
DefaultScopeforLDAPSearches.CanbeoneofSUBTREE_SCOPE,OBJECT_SCOPE,ONELEVEL_SCOPE.
ONELEVEL_SCOPE
RestrictToObjectClass RestrictSearchestoobjectClassnamedintheNamefieldforatable false
UsePaginationUseaPagedResultsControltopagethroughlargeresults.Thisisnotsupportedbyalldirectoryservers.
false
ExceptionOnSizeLimitExceeded
SettotruetothrowanexceptionwhenaSizeLimitExceededExceptionisreceivedandaLIMITisnotproperlyenforced.
false
Therearenoimportsettingsfortheldaptranslator;italsodoesnotprovidemetadata.
MetadataOptions
SEARCHABLE'equality_only'
Foropenldap,apacheds,andotherldapserversdnattributeshavesearchrestrictions,suchthatonlyequalitypredicatesaresupported.UseSEARCHABLEequality_onlytoindicatesthatonlyequalitypredicatesshouldbepusheddown.Anyotherpredicatewouldneedevaluatedintheengine.Forexample
colstringOPTIONS(SEARCHABLE'equality_only',...)
teiid_ldap:rdn_type
Usedonacolumnwithadnvaluetoindicatetherdntoextract.Iftheentrysuffixdoesnotmatchthisrdntype,thennorowwillbeproduced.Forexample
colstringOPTIONS("teiid_ldap:rdn_type"'cn',...)
teiid_ldap:dn_prefix
LDAPTranslator
581
Usedonacolumnifrdn_typeisspecifiedtoindicatesthatthevaluesshouldmatchthisprefix,norowwillbeproducedforanon-matchingentry.Forexample
colstringOPTIONS("teiid_ldap:rdn_type"'cn',"teiid_ldap:dn_prefix"'ou=groups,dc=example,dc=com',...)
MultivaluedAttributeSupport
Ifoneofthemethodsbelowisnotusedandtheattributeismappedtoanon-arraytype,thenanyvaluemaybereturnedonareadoperation.Alsoinsert/update/deletesupportwillnotbemulti-valueaware.
Concatenation
Stringcolumnswithadefaultvalueof"multivalued-concat"willconcatenateallattributevaluestogetherinalphabeticalorderusinga?delimiter.Insert/updatewillfunctionasexpectedifallapplicablevaluesaresuppliedintheconcatenatedformat.
Arraysupport
Multipleattributevaluesmayalsosupportedasanarraytype.Thearraytypemappingalsoallowsforinsert/updateoperations.
ForexamplehereisddlwithobjectClassanduniqueMemberasarrays:
createforeigntableldap_groups(objectClassstring[],DNstring,namestringoptions(nameinsource'cn'),uni
queMemberstring[])options(nameinsource'ou=groups,dc=teiid,dc=org',updatabletrue)
ThearrayvaluescanberetrievedwithaSELECT.Anexampleinsertwitharrayvaluescouldlooklike:
insertintoldap_groups(objectClass,DN,name,uniqueMember)values(('top','groupOfUniqueNames'),'cn=a,ou=g
roups,dc=teiid,dc=org','a',('cn=SamSmith,ou=people,dc=teiid,dc=org',))
Unwrap
Whenamultivaluedattributerepresentsanassociationbetweenentities,it’spossibletouseextensionmetadatapropertiestorepresentitasa1-to-manyormany-to-manyrelationship.
Examplemany-to-manyDDL:
CREATEforeigntableusers(usernamestringprimarykeyoptions(nameinsource'cn'),surnamestringoptions(na
meinsource'sn'),...)options(nameinsource'ou=users,dc=example,dc=com');
CREATEforeigntablegroups(groupnamestringprimarykeyoptions(nameinsource'cn'),descriptionstring,...)
options(nameinsource'ou=groups,dc=example,dc=com');
CREATEforeigntablemembership(usernamestringoptions(nameinsource'cn'),groupnameoptions(nameinsource'
memberOf',SEARCHABLE'equality_only',"teiid_rel:partial_filter"true,"teiid_ldap:unwrap"true,"teiid_ldap:d
n_prefix"'ou=groups,dc=example,dc=com',"teiid_ldap:rdn_type"'cn'),foreignkey(username)referencesusers(
username),foreignkey(groupname)referencesgroups(groupname))options(nameinsource'ou=users,dc=example,dc
=com');
Theresultfrom"select*frommembership"willthenproduce1rowforeachmemberOfandthekeyvaluewillbebaseduponthecnrdnvalueratherthanthefulldn.Alsoqueriesthatjoinbetweenusersandmembershipwillbepushedasasinglequery.
Iftheunwrapattributeismissingortherearenovalues,thenasinglerowwithanullvaluewillbeproduced.
NativeQueries
LDAPTranslator
582
LDAPproceduresmayoptionallyhavenativequeriesassociatedwiththem-seeParameterizableNativeQueries.Theoperationprefix(select;,insert;,update;,delete;-seebelowformore)mustbepresentinthenative-query,butitwillnotbeissuedaspartofthequerytothe
ExampleDDLforanLDAPnativeprocedure
CREATEFOREIGNPROCEDUREproc(arg1integer,arg2string)OPTIONS("teiid_rel:native-query"'search;context-nam
e=corporate;filter=(&(objectCategory=person)(objectClass=user)(!cn=$2));count-limit=5;timeout=$1;search-scope=O
NELEVEL_SCOPE;attributes=uid,cn')returns(col1string,col2string);
Parametervalueswillhavereservedcharactersescaped,butareotherwisedirectlysubstitutedintothequery.
DirectQueryProcedure
Note Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyisnative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheLDAPtranslatorprovidesaproceduretoexecuteanyad-hocLDAPquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.
Search
SearchExample
SELECTx.*FROM(callpm1.native('search;context-name=corporate;filter=(objectClass=*);count-limit=5;timeout=6;
search-scope=ONELEVEL_SCOPE;attributes=uid,cn'))w,
ARRAYTABLE(w.tupleCOLUMNS"uid"string,"cn"string)ASx
fromtheabovecode,the"search"keywordfollowedbybelowproperties.Eachpropertymustbedelimitedbysemi-colon(;)Ifapropertycontainsasemi-colon(;),itshouldbeescapedbyanothersemi-colon-seealsoParameterizableNativeQueriesandthenative-queryprocedureexampleabove.
Name Description Required
context-name LDAPContextname Yes
filter querytofiltertherecordsinthecontext No
count-limit limitthenumberofresults.sameasusingLIMIT No
timeout Timeoutthequeryifnotfinishedingivenmilliseconds No
search-scope
LDAPsearchscope,oneofSUBTREE_SCOPE,OBJECT_SCOPE,ONELEVEL_SCOPE
No
attributes attributestoretrieve Yes
LDAPTranslator
583
Delete
DeleteExample
SELECTx.*FROM(callpm1.native('delete;uid=doe,ou=people,o=teiid.org'))w,
ARRAYTABLE(w.tupleCOLUMNS"updatecount"integer)ASx
formtheabovecode,the"delete"keywordfollowedthe"DN"string.Allthestringcontentsafterthe"delete;"usedasDN.
CreateorUpdate
CreateExample
SELECTx.*FROM
(callpm1.native('create;uid=doe,ou=people,o=teiid.org;attributes=one,two,three','one',2,3.0))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
formtheabovecode,the"create"keywordfollowedthe"DN"string.Allthestringcontentsafterthe"create;"isusedasDN.Italsotakesonepropertycalled"attributes"whichiscommaseparatedlistofattributes.Thevaluesforeachattributeisspecifiedasseparateargumenttothe"native"procedure.
Updateissimilarto"create".
UpdateExample
SELECTx.*FROM
(callpm1.native('update;uid=doe,ou=people,o=teiid.org;attributes=one,two,three','one',2,3.0))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
LDAPConnectorCapabilitiesSupport
LDAPdoesnotprovidethesamesetoffunctionalityasarelationaldatabase.TheLDAPConnectorsupportsmanystandardSQLconstructs,andperformsthejoboftranslatingthoseconstructsintoanequivalentLDAPsearchstatement.Forexample,theSQLstatement:
SELECTfirstname,lastname,guid
FROMpublic_views.people
WHERE
(lastname='Jones'andfirstnameIN('Michael','John'))
OR
guid>600000
usesanumberofSQLconstructs,including:
SELECTclausesupport
selectindividualelementsupport(firstname,lastname,guid)
FROMsupport
WHEREclausecriteriasupport
nestedcriteriasupport
AND,ORsupport
Comparecriteria(Greater-than)support
INsupport
LDAPTranslator
584
TheLDAPConnectorexecutesLDAPsearchesbypushingdowntheequivalentLDAPsearchfilterwheneverpossible,basedonthesupportedcapabilities.TeiidautomaticallyprovidesadditionaldatabasefunctionalitywhentheLDAPConnectordoesnotexplicitlyprovidesupportforagivenSQLconstruct.Inthesecases,theSQLconstructcannotbepusheddowntothedatasource,soitwillbeevaluatedinTeiid,inordertoensurethattheoperationisperformed.IncaseswherecertainSQLcapabilitiescannotbepusheddowntoLDAP,Teiidpushesdownthecapabilitiesthataresupported,andfetchesasetofdatafromLDAP.Teiidthenevaluatestheadditionalcapabilities,creatingasubsetoftheoriginaldataset.Finally,Teiidwillpasstheresulttotheclient.Itisusefultobeawareofunsupportedcapabilities,inordertoavoidfetchinglargedatasetsfromLDAPwhenpossible.
LDAPConnectorCapabilitiesSupportList
ThefollowingcapabilitiesaresupportedintheLDAPConnector,andwillbeevaluatedbyLDAP:
SELECTqueries
SELECTelementpushdown(forexample,individualattributeselection)
ANDcriteria
Comparecriteria(e.g.<,⇐,>,>=,=,!=)
INcriteria
LIKEcriteria.
ORcriteria
INSERT,UPDATE,DELETEstatements(mustmeetModelingrequirements)
DuetothenatureoftheLDAPsource,thefollowingcapabilityisnotsupported:
SELECTqueries
ThefollowingcapabilitiesarenotsupportedintheLDAPConnector,andwillbeevaluatedbyTeiidafterdataisfetchedbytheconnector:
Functions
Aggregates
BETWEENCriteria
CaseExpressions
AliasedGroups
CorrelatedSubqueries
EXISTSCriteria
Joins
Inlineviews
ISNULLcriteria
NOTcriteria
ORDERBY
Quantifiedcomparecriteria
RowOffset
SearchedCaseExpressions
LDAPTranslator
585
SelectDistinct
SelectLiterals
UNION
XATransactions
Usage
ldap-as-a-datasourcequickstartdemonstratesusingtheldapTranslatortoaccessdatainOpenLDAPServer.Thenameofthetranslatortouseinvdb.xmlis"translator-ldap",forexample:
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="ldapVDB"version="1">
<modelname="HRModel">
<sourcename="local"translator-name="translator-ldap"
connection-jndi-name="java:/ldapDS"/>
</model>
</vdb>
ThetranslatordoesnotprovideaconnectiontotheOpenLDAP.Forthatpurpose,TeiidhasaJCAadapterthatprovidesaconnectiontoOpenLDAPusingtheJavaNamingAPI.Todefinesuchconnector,usethefollowingXMLfragmentinstandalone-teiid.xml.Seeaexamplein"<jboss-as>/docs/teiid/datasources/ldap"
<resource-adapterid="ldapQS">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.ldap"/>
<connection-definitions>
<connection-definition
class-name="org.teiid.resource.adapter.ldap.LDAPManagedConnectionFactory"
jndi-name="java:/ldapDS"enabled="true"use-java-context="true"
pool-name="ldapDS">
<config-propertyname="LdapAdminUserPassword">
redhat
</config-property>
<config-propertyname="LdapAdminUserDN">
cn=Manager,dc=example,dc=com
</config-property>
<config-propertyname="LdapUrl">
ldap://localhost:389
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
Theabovedefinesthetranslatorandconnector.FormorewaystocreatetheconnectorseeLDAPDataSources,LDAPtranslatorcanderivethemetadatabasedonexistingUsers/GroupsinLDAPServer,userneedtodefinethemetadata.Forexample,youcandefineaschemausingDDL:
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="ldapVDB"version="1">
<modelname="HRModel">
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLEHR_Group(
DNstringoptions(nameinsource`dn'),
SNstringoptions(nameinsource`sn'),
UIDstringoptions(nameinsource`uid'),
MAILstringoptions(nameinsource`mail'),
NAMEstringoptions(nameinsource`cn')
)OPTIONS(nameinsource`ou=HR,dc=example,dc=com',updatabletrue);
]]>
LDAPTranslator
586
</metadata>
</model>
</vdb>
whenSELECToperationbelowexecutedagainsttableusingTeiidwillretrieveUsers/GroupsinLDAPServer:
SELECT*FROMHR_Group
LDAPAttributeDatatypeSupport
LDAPproviderscurrentlyreturnattributevaluetypesofjava.lang.Stringandbyte[],anddonotsupporttheabilitytoreturnanyotherattributevaluetype.TheLDAPConnectorcurrentlysupportsattributevaluetypesofjava.lang.Stringonly.Therefore,allattributesaremodeledusingtheStringdatatypeinTeiidDesigner.ConversionfunctionsthatareavailableinTeiidallowyoutousemodelsthatconvertaStringvaluefromLDAPintoadifferentdatatype.Someconversionsmaybeappliedimplicitly,anddonotrequiretheuseofanyconversionfunctions.Otherconversionsmustbeappliedexplicitly,viatheuseofCONVERTfunctions.SincetheCONVERTfunctionsarenotsupportedbytheunderlyingLDAPsystem,theywillbeevaluatedinTeiid.Therefore,ifanycriteriaisevaluatedagainstaconverteddatatype,thatevaluationcannotbepushedtothedatasource,sincethenativetypeisString.
WhenconvertingfromStringtoothertypes,beawarethatcriteriaagainstthatnewdatatypewillnotbepusheddowntotheLDAPdatasource.Thismaydecreaseperformanceforcertainqueries.
Asanalternative,thedatatypecanremainastringandtheclientapplicationcanmaketheconversion,ortheclientapplicationcancircumventanyLDAPsupports⇐and>=,buthasnoequivalentfor<or>.Inordertosupport<or>pushdowntothesource,theLDAPConnectorwilltranslate<to⇐,anditwilltranslate>to>=.WhenusingtheLDAPConnector,beawarethatstrictly-less-thanandstrictly-greater-thancomparisonswillbehavedifferentlythanexpected.Itisadvisabletouse⇐and>=forqueriesagainstanLDAPbaseddatasource,sincethishasadirectmappingtocomparisonoperatorsinLDAP.
LDAP:TestingYourConnector
YoumustdefineLDAPConnectorpropertiesaccuratelyortheTeiidserverwillreturnunexpectedresults,ornoneatall.AsyoudeploytheconnectorinConsole,improperconfigurationcanleadtoproblemswhenyouattempttostartyourconnector.YoucantestyourLDAPConnectorinTeiidDesignerpriortoConsoledeploymentbysubmittingqueriesatmodelingtimeforverification.
LDAP:ConsoleDeploymentIssues
TheConsoleshowsanExceptionThatSaysErrorSynchronizingtheServer,IfyoureceiveanexceptionwhenyousynchronizetheserverandyourLDAPConnectoristheonlyservicethatdoesnotstart,itmeansthattherewasaproblemstartingtheconnector.Verifywhetheryouhavecorrectlytypedinyourconnectorpropertiestoresolvethisissue.
JCAResourceAdapter
Theresourceadapterforthistranslatorprovidedthrough"LDAPDataSource",RefertoAdminGuideforconfiguration.
LDAPTranslator
587
LoopbackTranslatorTheLoopbacktranslator,knownbythetypenameloopback,providesaquicktestingsolution.ItsupportsallSQLconstructsandreturnsdefaultresults,withsomeconfigurablebehavior.
ExecutionProperties
Name Description Default
ThrowError truetoalwaysthrowanerror false
RowCount Rowsreturnedfornon-updatequeries. 1
WaitTime Waitrandomlyuptothisnumberofmillisecondswitheachsourcequery. 0
PollIntervalInMilli
ifpositiveresultswillbeasynchronouslyreturned-thatisaDataNotAvailableExceptionwillbethrowninitiallyandtheenginewillwaitthepollintervalbeforepollingfortheresults.
-1
DelegateName settothenameofthetranslatortomimicthecapabilitiesof
YoucanalsousetheLoopbacktranslatortomimichowarealsourcequerywouldbeformedforagiventranslator(althoughloopbackwillstillreturndummydatathatmaynotbeusefulforyoursituation).Toenablethisbehavior,settheDelegateNamepropertytothenameofthetranslatoryouwishtomimic.Forexampletodisableallcapabilities,settheDelegateNamepropertyto"jdbc-simple".
JCAResourceAdapter
Asourceconnectionisnotrequiredforthistranslator.
LoopbackTranslator
588
MicrosoftExcelTranslatorTheMicrosoftExcelTranslator,knownbythetypenameexcel,exposesqueryingfunctionalitytoExceldocumentsusingFileDataSources.MicrosoftExcelisapopularspreadsheetsoftwarethatisusedbyalltheorganizationsacrosstheglobeforsimplereportingpurposes.ThistranslatorprovidesaneasywayreadaExcelspreadsheetandprovidecontentsofthespreadsheetinthetabularformthatcanbeintegratedwithothersourcesinTeiid.
Note "DoesitonlyworkonWindows?"-No,itworksonallplatforms,includingWindowsandLinux.ThistranslatorusesApachePOIlibrariestoaccesstheExceldocumentswhichareplatformindependent.
Usage
ThebelowtabledescribeshowExceltranslatorinterpretsthedatainExceldocumentintorelationalterms.
ExcelTerm RelationalTerm
Workbook schema
Sheet Table
Row Rowofdata
Cell ColumnDefinitionorDataofacolumn
Exceltranslatorsupports"sourcemetadata"feature,wheregivenExcelworkbook,itcanintrospectandbuildtheschemabasedontheSheetsdefinedinsideit.Thereareoptionsavailableforyouguide,tobeabletodetectheadercolumnsanddatacolumnsinaworksheettodefinethecorrectmetadataofatable.
VDBExample
ThebelowshowsanexampleofaVDB,thatshowsaexposingaExcelDocument.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="excelvdb"version="1">
<modelname="excel">
<propertyname="importer.headerRowNumber"value="1"/>
<propertyname="importer.ExcelFileName"value="names.xls"/>
<sourcename="connector"translator-name="excel"connection-jndi-name="java:/fileDS"/>
</model>
</vdb>
"connection-jndi-name"inaboverepresentsconnectiontoExceldocument.TheExceltranslatordoesNOTprovideaconnectiontotheExcelDocument.Forthatpurpose,TeiidusesFileJCAadapterthatprovidesaconnectiontoExcel.Todefinesuchconnector,seeFileDataSourcesorseeanexamplein"<jboss-as>/docs/teiid/datasources/file".Onceyouconfigurebothoftheabove,youcandeploythemtoTeiidServerandaccesstheExcelDocumentusingJDBC/ODBC/ODataprotocol.
DesignerVDB
IfyouareusingDesignerTooling,tocreateExcelbasedVDB
MicrosoftExcelTranslator
589
Create/useaTeiidDesignerModelproject
Use"TeiidConnection>>SourceModel"importer,createFileDataSourceusingdatasourcecreationwizardanduseexcelastranslatorintheimporter.BasedontheExceldocumentrelevantrelationaltableswillbecreated.
CreateaVDBanddeployintoTeiidServerandandaccesstheExcelDocumentusingJDBC/ODBC/ODataprotocol.
Note"HeadersinDocument?"-IfyouhaveheadersintheExceldocument,youcanguidetheimportprocesstoselectthecellheadersasthecolumnnamesinthetablecreationprocess.See"ImportProperties"sectionbelowondefiningthe"import"properties.
ImportProperties
ImportpropertiesguidetheschemagenerationpartduringthedeploymentoftheVDB.Thiscanbeusedinanativeimportorwhileusing"TeiidConnection>>SourceModel"inTeiidDesigner.
PropertyName Description Default
importer.excelFileName
DefinesthenameoftheExcelDocumenttoimportmetadata.Thiscanbedefinedasafilepattern(*.xls),howeverwhendefinedaspatternallfilesmustbeofsameformatandthetranslatorwillchooseanarbitraryfiletoimportmetadatafrom.UsefilepatterntoreaddatafrommultipleExceldocumentsinthesamedirectory,insinglefilecasechoosetheabsolutename.
required
importer.headerRowNumber Definesthecellheaderinformationtobeusedascolumnnames
optional,defaultisfirstdatarowofsheet
importer.dataRowNumber Definestherownumberwherethedatarowsstart
optional,defaultisfirstdatarowofsheet
Itishighlyrecommendedthatyoudefinealltheaboveimporterproperties,suchthatinformationinsidetheExcelDocumentiscorrectlyinterpreted.
Note
Purelynumericalcellsinacolumncontaincontainingmixedtypeswillhaveastringformmatchingtheirdecimalrepresentation,thusintegralvalueswillhave.0appended.Ifyouneedtheexacttextrepresentation,thencellmustbeastringvaluewhichcanbeforcedbyputtingasinglequote'infrontofthenumerictextofthecell,orbyputtingasinglespaceinfrontofthenumerictext.
TranslatorExtensionProperties
CurrentlytherearenoTranslatorExtensionpropertiesdefinedforthistranslator.
MetadataExtensionProperties
MetadataExtensionPropertiesarethepropertiesthataredefinedontheschemaartifactslikeTable,Column,Procedureetc,todescribehowthetranslatorneedstointeractorinterpretwithsourcesystems.Allthepropertiesaredefinedwithnamespace'http://www.teiid.org/translator/excel/2014[http://www.teiid.org/translator/excel/2014\]',whichalsohasarecognizedalias'teiid_excel'.
PropertyName SchemaItemPropertyBelongsTo Description Mandatory
MicrosoftExcelTranslator
590
FILE Table
DefinesExcelDocumentnameornamepattern(*.xls).Filepatterncanbeusedtoreaddatafrommultiplefiles.
Yes
FIRST_DATA_ROW_NUMBER Table
Definestherownumberwhererecordsstartinthesheet(appliestoeverysheet)
optional
CELL_NUMBER ColumnofTableDefinescellnumbertouseforreadingdataofparticularcolumn
Yes
ThebelowshowsanexampletablethatisdefinedusingtheExtensionMetadataProperties.
<?xmlversion="1.0"encoding="UTF-8"standalone="yes"?>
<vdbname="excelvdb"version="1">
<modelname="excel">
<sourcename="connector"translator-name="excel"connection-jndi-name="java:/fileDS"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLEPerson(
ROW_IDintegerOPTIONS(SEARCHABLE'All_Except_Like',"teiid_excel:CELL_NUMBER"'ROW_ID'),
FirstNamestringOPTIONS(SEARCHABLE'Unsearchable',"teiid_excel:CELL_NUMBER"'1'),
LastNamestringOPTIONS(SEARCHABLE'Unsearchable',"teiid_excel:CELL_NUMBER"'2'),
AgeintegerOPTIONS(SEARCHABLE'Unsearchable',"teiid_excel:CELL_NUMBER"'3'),
CONSTRAINTPK0PRIMARYKEY(ROW_ID)
)OPTIONS("NAMEINSOURCE"'Sheet1',"teiid_excel:FILE"'names.xlsx',"teiid_excel:FIRST_DATA_ROW_NU
MBER"'2')
]]></metadata>
</model>
</vdb>
Note
"ExtendedcapabilitiesusingROW_IDcolumn"-Ifyoudefinecolumn,thathasextensionmetadataproperty"CELL_NUMBER"withvalue"ROW_ID",thenthatcolumnvaluecontainstherowinformationfromExceldocument.YoucanmarkthiscolumnasPrimaryKey.YoucanusethiscolumninSELECTstatementswitharestrictivesetofcapabilitiesincluding:comparisonpredicates,INpredicatesandLIMIT.Allothercolumnscannotbeusedaspredicatesinaquery.
Tip
Userdoesnothavetodependupon"sourcemetadata"import,orDesignertoolimporttocreatetheschemarepresentedbyExceldocument,theycanmanuallycreateasourcetableandaddtheappropriateextensionpropertiestomakeafullyfunctionalmodel.Ifyouintrospecttheschemamodelcreatedbytheimport,itwouldlooklikeabove.
TIP:CurrentlyExceltranslatordoesnotsupportupdates.
JCAResourceAdapterTheTeiidspecificExcelResourceAdapterdoesnotexist,usershoulduseFileJCAadapterwiththistranslator.SeeFileDataSourcesforopeningaFilebasedconnection.
NativeQueries
Note ThisfeatureisnotapplicableforExceltranslator.
DirectQueryProcedure
MicrosoftExcelTranslator
591
Note ThisfeatureisnotapplicableforExceltranslator.
MicrosoftExcelTranslator
592
MongoDBTranslatorTheMongoDBtranslator,knownbythetypenamemongodb,providesarelationalviewofdatathatresidesinaMongoDBdatabase.ThistranslatoriscapableofconvertingTeiidSQLqueriesintoMongoDBbasedqueries.ItsupportsafullrangeofSELECT,INSERT,UPDATEandDELETEcalls.
MongoDBisadocumentbased"schema-less"databasewithitownquerylanguage-itdoesnotmapperfectlywithrelationalconceptsortheSQLquerylanguage.MoreandmoresystemsareusingaMongoDBkindofNOSQLstoreforscalabilityandperformance.Forexample,applicationslikestoringauditlogsormanagingwebsitedatafitswellwithMongoDB,anddoesnotrequireusingastructuraldatabaselikeOracle,Postgresetc.MongoDBusesJSONdocumentsasitsprimarystorageunit,andthosedocumentscanhaveadditionalembeddeddocumentsinsidetheparentdocument.Byusingembeddeddocumentsitco-locatestherelatedinformationtoachievede-normalizationthattypicallyrequireseitherduplicationofdataorjoinstoachievequeryinginarelationaldatabase.
TomakeMongoDBworkwithTeiidthechallengefortheMongoDBtranslatoris"HowtodesignaMongoDBstorethatcanachievethebalancebetweenrelationalanddocumentbasedstorage?"Inouropiniontheadvantagesof"schema-less"designaregreatatdevelopmenttime,notmuchatruntimeexceptinfewspecialsituations."Schema-less"canalsobeaproblemwithmigrationofapplicationversionsandtheabilitytoqueryandmakeuseofreturnedinformationeffectively.
SinceitishardandmaybeimpossibleincertainsituationstoderiveaschemabasedonexistingtheMongoDBcollection(s),Teiidapproachestheprobleminreversecomparedtoothertranslators.WhenworkingwithMongoDB,TeiidrequirestheusertodefinetheMongoDBschemaupfrontusingTeiidmetadata.SinceTeiidonlyallowsrelationalschemaasitsmetadata,theuserneedstodefinetheirMongoDBschemainrelationaltermsusingtables,procedures,andfunctions.ForthepurposesofMongoDB,theTeiidmetadatahasbeenextendedtosupportextensionpropertiesthatcanbedefinedonthetabletoconvertitintoaMongoDBbaseddocument.Theseextensionpropertiesletusersdefine,howaMongoDBdocumentisstructuredandstored.Basedontherelationships(primary-key,foreign-key)definedonatable,andthecardinality(ONE-to-ONE,ONE-to-MANY,MANY-to-ONE)relationsbetweentablesaremappedsuchthatrelatedinformationcanbeembeddedalongwiththeparentdocumentforco-location(seethede-normalizationcommentabove).Thusarelationalschemabaseddesign,butdocumentbasedstorageinMongoDB.CurrentlydirectmappingofMANY-to-MANYisnotsupported.
Whoistheprimaryaudienceforthistranslator?
Theabovemaynotsatisfyeveryuser’sneeds.ThedocumentstructureinMongoDBcanbemorecomplexthanwhatTeiidcancurrentlydefine.WehopethiswilleventuallycatchupinfutureversionsofTeiid.Thisiscurrentlydesignedfor:
1. Usersthatareusingrelationaldatabasesandwouldliketomove/migratetheirdatatoMongoDBtotakeadvantagesscalingandperformancewithoutmodifyingtheenduserapplicationscurrentlyrunning.
2. UsersthatarestartingoutwithMongoDBanddonothaveexperiencewithMongoDB,butareseasonedSQLdevelopers.ThisprovidesalowbarrierofentrycomparedtousingMongoDBdirectlyasanapplicationdeveloper.
3. IntegrateotherenterprisedatasourceswithMongoDBbaseddata.
Usage
Thenameofthetranslatortouseinvdb.xmlis"mongodb".Forexample:
<vdbname="nothwind"version="1">
<modelname="northwind">
<sourcename="local"translator-name="mongodb"connection-jndi-name="java:/mongoDS"/>
</model>
MongoDBTranslator
593
<vdb>
ThetranslatordoesnotprovideaconnectiontotheMongoDB.Forthatpurpose,TeiidhasaJCAadapterthatprovidesaconnectiontoMongoDBusingtheMongoDBJavaDriver.Todefinesuchconnector,usethefollowingXMLfragmentinstandalone-teiid.xml.Seeaexamplein"<jboss-as>/docs/teiid/datasources/mongodb"
<resource-adapters>
<resource-adapterid="mongodb">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.mongodb"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.mongodb.MongoDBManagedConnectionF
actory"
jndi-name="java:/mongoDS"
enabled="true"
use-java-context="true"
pool-name="teiid-mongodb-ds">
<!--MongoDBserverlist(host:port[;host:port...])-->
<config-propertyname="RemoteServerList">localhost:27017</config-property>
<!--DatabaseNameintheMongoDB-->
<config-propertyname="Database">test</config-property>
<!--
Uncommentthesepropertiestosupplyusernameandpassword
<config-propertyname="Username">user</config-property>
<config-propertyname="Password">user</config-property>
-->
</connection-definition>
</connection-definitions>
</resource-adapter>
</resource-adapters>
Theabovedefinesthetranslatorandconnector.FormorewaystocreatetheconnectorseeMongoDBDataSources.MongoDBtranslatorcanderivethemetadatabasedonexistingdocumentcollectionsinsomescenarios,howeverwhenworkingwithcomplexdocumentstheinterpretationofmetadatamaybeinaccurate,inthosesituationstheuserMUSTdefinethemetadata.Forexample,youcandefineaschemausingDDL:
<vdbname="nothwind"version="1">
<modelname="northwind">
<sourcename="local"translator-name="mongodb"connection-jndi-name="java:/mongoDS"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLECustomer(
customer_idinteger,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
]]></metadata>
</model>
<vdb>
whenINSERToperationbelowexecutedagainsttableusingTeiid,
INSERTINTOCustomer(customer_id,FirstName,LastName)VALUES(1,'John','Doe');
MongoDBtranslatorwillcreateabelowdocumentintheMongoDB
{
_id:ObjectID("509a8fb2f3f4948bd2f983a0"),
customer_id:1,
FirstName:"John",
LastName:"Doe"
MongoDBTranslator
594
}
IfaPRIMARYKEYisdefinedonthetableas
CREATEFOREIGNTABLECustomer(
customer_idintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
thenthatcolumnnameisautomaticallyusedas"_id"fieldintheMongoDBcollection,thendocumentstructureisstoredintheMongoDBas
{
_id:1,
FirstName:"John",
LastName:"Doe"
}
IfyoudefinedthecompositePRIMARYKEYonCustomertableas
CREATEFOREIGNTABLECustomer(
customer_idinteger,
FirstNamevarchar(25),
LastNamevarchar(25),
PRIMARYKEY(FirstName,LastName)
)OPTIONS(UPDATABLE'TRUE');
thedocumentstructurewillbe
{
_id:{
FirstName:"John",
LastName:"Doe"
},
customer_id:1,
}
DataTypes
MongoDBtranslatorsupportsautomaticmappingofTeiiddatatypesintoMongoDBdatatypes,includingthesupportforBlobs,ClobsandXML.TheLOBsupportisbasedonGridFSinMongoDB.Arraysareintheformof
{
_id:1,
FirstName:"John",
LastName:"Doe"
Score:[89,"ninety",91.0]
}
aresupported.Usercangetindividualitemsinthearrayusingfunctionarray_get,orcantransformthearrayintotabularstructureusingARRATTABLE.
Note Notethateventhoughembeddeddocumentscanalsobeinarrays,thehandlingofembeddeddocumentsisdifferentfromarraywithscalarvalues.
MongoDBTranslator
595
RegularExpressions,MongoDB::Code,MongoDB::MinKey,MongoDB::MaxKey,MongoDB::OIDisnotcurrentlysupported.
ExtensionMetadataPropertiesToBuildComplexDocuments
UsingtheaboveDDLoranyothermetadatafacility,ausercanmapatableinarelationalstoreintoadocumentinMongoDB,howevertomakeeffectiveuseofMongoDB,youneedtobeabletobuildcomplexdocuments,thatcanco-locaterelatedinformation,sothatdatacanqueriedinasingleMongoDBquery.Otherwise,sinceMongoDBdoesnotsupportjoinrelationshipslikerelationaldatabase,youneedtoissuemultiplequeriestoretrieveandjoindatamanually.ThepowerofMongoDBcomesfromits"embedded"documentsanditssupportofcomplexdatatypeslikearraysanduseoftheaggregationframeworktobeabletoquerythem.Thistranslatorprovideswaytoachievethatgoals.
WhenyoudonotdefinethecomplexembeddeddocumentsinMongoDB,Teiidcanstepinforjoinprocessingandprovidethatfunctionality,howeverifyouwanttomakeuseofthepowerofMongoDBitselfinqueryingthedataandavoidbringingtheunnecessarydataandimproveperformance,youneedtolookintobuildingthesecomplexdocuments.
MongoDBtranslatordefinestwoadditionalmetadatapropertiesalongwithotherTeiidmetadatapropertiestoaidinbuildingthecomplex"embedded"documents.YoucanusethefollowingmetadatapropertiesinyourDDL.
teiid_mongo:EMBEDDABLE-Meansthatdatadefinedinthistableisallowedtobeincludedasan"embeddable"documentinanyparentdocument.Theparentdocumentisreferencedbytheforeignkeyrelationships.Inthisscenario,TeiidmaintainsmorethanonecopyofthedatainMongoDBstore,oneinitsowncollectionandalsoacopyineachoftheparenttablesthathaverelationshiptothistable.Youcanevennestembeddabletableinsideanotherembeddabletablewithsomelimitations.Usethispropertyontable,wheretablecanexist,encompassallitsrelationsonitsown.Forexample,a"Category"tablethatdefinesa"Product"’scategoryisindependentofProduct,whichcanbeembeddablein"Products"table.
teiid_mongo:MERGE-Meansthatdataofthistableismergedwiththedefinedparenttable.Thereisonlyasinglecopyofthedatathatisembeddedintheparentdocument.Parentdocumentisdefinedusingtheforeignkeyrelationships.
UsingtheabovepropertiesandFOREIGNKEYrelationships,wewillillustratehowtobuildcomplexdocumentsinMongoDB.
NoteUsage-Pleasenoteagiventablecancontaineitherthe"teiid_mongo:EMBEDDABLE"propertyorthe"teiid_mongo:MERGE"propertydefiningthetypeofnestinginMongoDB.Atableisnotallowedtohavebothproperties.
2-ONEMapping
IfyourcurrentDDLstructurerepresentingONE-2-ONErelationshipislike
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEAddress(
CustomerIdinteger,
Streetvarchar(50),
Cityvarchar(25),
Statevarchar(25),
Zipcodevarchar(6),
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE');
bydefault,thiswillproducetwodifferentcollectionsinMongoDB,likewithsampledataitwilllooklike
Customer
{
MongoDBTranslator
596
_id:1,
FirstName:"John",
LastName:"Doe"
}
Address
{
_id:ObjectID("..."),
CustomerId:1,
Street:"123Lane"
City:"NewYork",
State:"NY"
Zipcode:"12345"
}
YoucanenhancethestorageinMongoDBtoasinglecollectionbyusing"teiid_mongo:MERGE’extensionpropertyonthetable’sOPTIONSclause
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEAddress(
CustomerIdintegerPRIMARYKEY,
Streetvarchar(50),
Cityvarchar(25),
Statevarchar(25),
Zipcodevarchar(6),
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE',"teiid_mongo:MERGE"'Customer');
thiswillproducesinglecollectioninMongoDB,like
Customer
{
_id:1,
FirstName:"John",
LastName:"Doe",
Address:
{
Street:"123Lane",
City:"NewYork",
State:"NY",
Zipcode:"12345"
}
}
WiththeabovebothtablesaremergedintoasinglecollectionthatcanbequeriedtogetherusingtheJOINclauseintheSQLcommand.Sincetheexistenceofchild/additionalrecordhasnomeaningwithoutparenttableusingthe"teiid_mongo:MERGE"extensionpropertyisrightchoiceinthissituation.
Note NotethattheForeignKeydefinedonchildtable,mustrefertoPrimaryKeysonbothparentandchildtablestoformaOne-2-Onerelationship.
2-MANYMapping.
Typicallytherecanbemorethantwo(2)tablesinvolvedinthisrelationship.IfMANYsideisonlyassociatedsingletable,thenuse"teiid_mongo:MERGE"propertyonMANYsideoftableanddefineONEastheparent.Ifassociatedwithmorethansingletablethenuse"teiid_mongo:EMBEDDABLE".
MongoDBTranslator
597
ForexampleifyouhaveDDLlike
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
CustomerIdinteger,
OrderDatedate,
Statusinteger,
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE');
intheaboveaSingleCustomercanhaveMANYOrders.TherearetwooptionstodefinethehowwestoretheMongoDBdocument.Ifinyourschema,theCustomertable’sCustomerIdisonlyreferencedinOrdertable(i.e.CustomerinformationusedforonlyOrderpurposes),youcanuse
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
CustomerIdinteger,
OrderDatedate,
Statusinteger,
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE',"teiid_mongo:MERGE"'Customer');
thatwillproduceasingledocumentforCustomertablelike
{
_id:1,
FirstName:"John",
LastName:"Doe",
Order:
[
{
_id:100,
OrderDate:ISODate("2000-01-01T06:00:00Z")
Status:2
},
{
_id:101,
OrderDate:ISODate("2001-03-06T06:00:00Z")
Status:5
}
...
]
}
IfCustomertableisreferencedinmoretablesotherthanOrdertable,thenuse"teiid_mongo:EMBEDDABLE"property
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE',"teiid_mongo:EMBEDDABLE"'TRUE');
MongoDBTranslator
598
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
CustomerIdinteger,
OrderDatedate,
Statusinteger,
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEComments(
CommentIDintegerPRIMARYKEY,
CustomerIdinteger,
Commentvarchar(140),
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)
)OPTIONS(UPDATABLE'TRUE');
ThiscreatesthreedifferentcollectionsinMongoDB.
Customer
{
_id:1,
FirstName:"John",
LastName:"Doe"
}
Order
{
_id:100,
CustomerId:1,
OrderDate:ISODate("2000-01-01T06:00:00Z")
Status:2
Customer:
{
FirstName:"John",
LastName:"Doe"
}
}
Comment
{
_id:12,
CustomerId:1,
Comment:"Thisworks!!!"
Customer:
{
FirstName:"John",
LastName:"Doe"
}
}
HereasyoucanseetheCustomertablecontentsareembeddedalongwithothertable’sdatawheretheywerereferenced.ThiscreatesduplicateddatawheremultipleoftheseembeddeddocumentsaremanagedautomaticallyintheMongoDBtranslator.
Note
AlltheSELECT,INSERT,DELETEoperationsthataregeneratedagainstthetableswith"teiid_mongo:EMBEDDABLE"propertyareatomic,exceptforUPDATES,astherecanbemultipleoperationsinvolvedtoupdateallthecopies.SincetherearenotransactionsinMongoDB,TeiidplanstoprovideautomaticcompensatingtransactionframeworkaroundthisinfuturereleasesTEIID-2957.
MANY-2-ONEMapping.
ThisissameasONE-2-MANY,seeabovetodefinerelationships.
Note Aparenttablecanhavemultiple"embedded"andaswellas"merge"documentsinsideit,itnotlimitedsoeitheroneorother.However,pleasenotethatMongoDBimposesdocumentsizeislimitedcannotexceed16MB.
MongoDBTranslator
599
MANY-2-MANYMapping.
Thiscanalsomappedwithcombinationof"teiid_mongo:MERGE"and"teiid_mongo:EMBEDDABLE"properties(partially).ForexampleifDDLlookslike
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
OrderDatedate,
Statusinteger
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEOrderDetail(
OrderIDinteger,
ProductIDinteger,
PRIMARYKEY(OrderID,ProductID),
FOREIGNKEY(OrderID)REFERENCESOrder(OrderID),
FOREIGNKEY(ProductID)REFERENCESProduct(ProductID)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEProducts(
ProductIDintegerPRIMARYKEY,
ProductNamevarchar(40)
)OPTIONS(UPDATABLE'TRUE');
youmodifytheDDLlikebelow,tohave
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
OrderDatedate,
Statusinteger
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEOrderDetail(
OrderIDinteger,
ProductIDinteger,
PRIMARYKEY(OrderID,ProductID),
FOREIGNKEY(OrderID)REFERENCESOrder(OrderID),
FOREIGNKEY(ProductID)REFERENCESProduct(ProductID)
)OPTIONS(UPDATABLE'TRUE',"teiid_mongo:MERGE"'Order');
CREATEFOREIGNTABLEProducts(
ProductIDintegerPRIMARYKEY,
ProductNamevarchar(40)
)OPTIONS(UPDATABLE'TRUE',"teiid_mongo:EMBEDDABLE"'TRUE');
Thatwillproduceadocumentlike
{
_id:10248,
OrderDate:ISODate("1996-07-04T05:00:00Z"),
Status:5
OrderDetails:[
{
_id:{
OrderID:10248,
ProductID:11
Products:{
ProductID:11
ProductName:"Hammer"
}
}
},
{
MongoDBTranslator
600
_id:{
OrderID:10248,
ProductID:14
Products:{
ProductID:14
ProductName:"ScrewDriver"
}
}
}
]
}
Products
{
{
ProductID:11
ProductName:"Hammer"
}
{
ProductID:14
ProductName:"ScrewDriver"
}
}
Limitations
CurrentlynestedembeddingofdocumentshaslimitedsupportduetocapabilitiesofhandlingnestedarraysislimitedintheMongoDB.Nestingof"EMBEDDALBLE"propertywithmultiplelevelsisOK,howevermorethantwolevelswithMERGEisnotrecommended.Also,youneedtobecautionaboutnotexceedingthedocumentsizeof16MBforsinglerow,sodeepnestingisnotrecommended.
JOINSbetweenrelatedtables,MUSThaveusedeitherof"EMBEDDABLE"or"MERGE"property,otherwisethequerywillresultinerror.InorderforTeiidtocorrectlyplanandsupporttheJOINS,inthecasethatanytwotablesareNOTembeddedineachother,useallow-joins=falsepropertyontheForeignKeythatrepresentstherelation.Forexample:
CREATEFOREIGNTABLECustomer(
CustomerIdintegerPRIMARYKEY,
FirstNamevarchar(25),
LastNamevarchar(25)
)OPTIONS(UPDATABLE'TRUE');
CREATEFOREIGNTABLEOrder(
OrderIDintegerPRIMARYKEY,
CustomerIdinteger,
OrderDatedate,
Statusinteger,
FOREIGNKEY(CustomerId)REFERENCESCustomer(CustomerId)OPTIONS(allow-join'FALSE')
)OPTIONS(UPDATABLE'TRUE');
withtheexampleabove,Teiidwillcreatetwocollections,howeverwhenuserissuesquerysuchas
SELECTOrderID,LastNameFROMOrderJOINCustomerONOrder.CustomerId=Customer.CustomerId;
insteadofresultinginerror,theJOINprocessingwillhappenintheTeiidengine,withouttheabovepropertyitwillresultinanerror.
WhenyouuseabovepropertiesandcarefullydesigntheMongoDBdocumentstructure,Teiidtranslatorcanintelligentlycollatedatabasedontheirco-locationandtakeadvantageofitwhilequerying.
GeoSpatialfunctionsupport
MongoDBTranslator
601
MongoDBtranslatorsupportsgeospatialqueryoperatorsinthe"WHERE"clause,whenthedataisstoredintheGeoJSonformatintheMongoDBDocument.Thesupportedfunctionsare
CREATEFOREIGNFUNCTIONgeoIntersects(columnRefstring,typestring,coordinatesdouble[][])RETURNSboolean;
CREATEFOREIGNFUNCTIONgeoWithin(ccolumnRefstring,typestring,coordinatesdouble[][])RETURNSboolean;
CREATEFOREIGNFUNCTIONnear(ccolumnRefstring,coordinatesdouble[],maxdistanceinteger)RETURNSboolean;
CREATEFOREIGNFUNCTIONnearSphere(ccolumnRefstring,coordinatesdouble[],maxdistanceinteger)RETURNSboole
an;
CREATEFOREIGNFUNCTIONgeoPolygonIntersects(refstring,northdouble,eastdouble,westdouble,southdouble)
RETURNSboolean;
CREATEFOREIGNFUNCTIONgeoPolygonWithin(refstring,northdouble,eastdouble,westdouble,southdouble)RET
URNSboolean;
asamplequerylookslike
SELECTlocFROMmapswheremongo.geoWithin(loc,'LineString',((cast(1.0asdouble),cast(2.0asdouble)),(cast
(1.0asdouble),cast(2.0asdouble))))
Samefunctionsusingbuilt-inGeometrytype(theabovefunctionswillbedeprecatedandremovedinfutureversions)
CREATEFOREIGNFUNCTIONgeoIntersects(columnRefstring,geogeometry)RETURNS
boolean;
CREATEFOREIGNFUNCTIONgeoWithin(ccolumnRefstring,geogeometry)RETURNS
boolean;
CREATEFOREIGNFUNCTIONnear(ccolumnRefstring,geogeometry,maxdistanceinteger)
RETURNSboolean;
CREATEFOREIGNFUNCTIONnearSphere(ccolumnRefstring,geogeometry,maxdistance
integer)RETURNSboolean;
CREATEFOREIGNFUNCTIONgeoPolygonIntersects(refstring,geogeometry)RETURNS
boolean;
CREATEFOREIGNFUNCTIONgeoPolygonWithin(refstring,geogeometry)RETURNS
boolean;
asamplequerylookslike
SELECTlocFROMmapswheremongo.geoWithin(loc,ST_GeomFromGeoJSON('{"coordinates":[[1,2],[3,4]],"type":"Polygo
n"}'))
Therearevarious"st_geom.."methodsareavailableintheGeoSpatialfunctionlibraryinTeiid.
CapabilitiesMongoDBtranslatordesignedontopoftheMongoDBaggregationframework,useofMongoDBversionthatsupportsthisframeworkismandatory.ApartfromSELECTqueries,thistranslatoralsosupportsINSERT,UPDATEandDELETEqueries.
Thistranslatorsupports
grouping
matching
sorting
MongoDBTranslator
602
filtering
limit
supportforLOBsusingGridFS
Compositeprimaryandforeignkeys.
Note example-Forafullexampleseehttps://github.com/teiid/teiid/blob/master/connectors/translator-mongodb/src/test/resources/northwind.ddl
NativeQueries
MongoDBsourceproceduresmaybecreatedusingtheteiid_rel:native-queryextension-seeParameterizableNativeQueries.Theprocedurewillinvokethenative-querysimilartoadirectprocedurecallwiththebenefitsthatthequeryispredeterminedandthatresultcolumntypesareknown,ratherthanrequiringtheuseofARRAYTABLEorsimilarfunctionality.
DirectQueryProcedure
Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyiscallednative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheMongoDBtranslatorprovidesaproceduretoexecuteanyad-hocaggregatequerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarraycontainingsingleblobatarraylocationone(1).ThisblobcontainstheJSONdocument.XMLTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.
ExampleMongoDBDirectQuery
selectx.*fromTABLE(callnative('city;{$match:{"city":"FREEDOM"}}'))t,
xmltable('/city'PASSINGJSONTOXML('city',cast(array_get(t.tuple,1)asBLOB))COLUMNScitystring,
statestring)x
Intheaboveexample,acollectioncalled"city"islookedupwithfilterthatmatchesthe"city"namewith"FREEDOM",using"native"procedureandthenusingthenestedtablesfeaturetheoutputispassedtoaXMLTABLEconstruct,wheretheoutputfromtheprocedureissenttoaJSONTOXMLfunctiontoconstructaXMLthentheresultsofthatareexposedintabularform.
ThedirectqueryMUSTbeintheformat
"collectionName;{$pipelineinstr}+"
FromTeiid8.10,MongoDBtranslatoralsoallowstoexecuteShelltypejavascriptcommandslikeremove,drop,createIndex.Forthisthecommandneedstobeinformat
"$ShellCmd;collectionName;operationName;{$instr}+"
andexamplelookslike
"$ShellCmd;MyTable;remove;{qty:{$gt:20}}"
MongoDBTranslator
603
MongoDBTranslator
604
ObjectTranslatorTheObjecttranslator,knownbythenameofmap-cache,isabridgeforreadingandwritingjavaobjectsfromexternalsources(i.e.,Mapcache)anddeliveringthemtotheengineforprocessing.Andtoassistinprovidingthatbridge,theOBJECTTABLEfunctionisanalternaitivemeansfortransformingcomplexjavaobjectsintorowsandcolumns.
SearchCapabilities
SupportsalocalcachethatisoftypeMapandituseskeysearchingonthemaptofindobjects.
SupportedCapabilities
ThefollowingaretheconnectorcapabilitieswhenKeySearchingisused:
SELECTcommand
CompareCriteria-onlyEQ
InCriteria
Insert,UpdateandDelete
UsageRetrieveobjectsfromacacheandtransformintorowsandcolumns.
Performinserts,updatesanddeletestothecache
Properties
Objecttranslatoriscapableofreturningtotheclientapplicationthejavaobjectstoredinthecache.Toenablethetranslatortoautomaticallyincludetheobjectasacolumninthemetadata,usethefollowingImportProperties.
ImportProperties
PropertyName Description Required Default
ClassObjectColumn
Iftrue,andwhenthetranslatorprovidesthemetadata,acolumnofobjectdatatypewillbecreatedthatrepresentsthestoredobjectinthecache
false false
Metadata
RequirementsforDefiningTableforRootClass
Tablelevelrequirements
ObjectTranslator
605
Thetablefortherootclass,musthaveaprimarykeydefined,whichmustmaptoanattributeintheclass.
Thetablename(orthenameinsource)mustmatchthenameofthePojoclassname.Thisishowthetableismatchedtotheregisteredclassinthecache.
Note Theprimarykeydatatypefortheattributeintheclassmustmatchthecachekeydatatype.
Columnlevelrequirements
Theclassobjectcolumnthatrepresentsthecachedobjectshouldhaveanameinsourceof'this'.Allothercolumnswillhavetheirnameinsource(whichdefaultstothecolumnname)interpretedasthepathtothecolumnvaluefromtheprimaryobject.
AllcolumnsthatarenottheprimarykeynorcoveredbyaluceneindexshouldbemarkedasSEARCHABLE'Unsearchable'.
Attributesdefinedasrepeatable(i.e.,collections,arrays,ormap)oracontainerclass,canbesupportedas1-to-*relationship,andwillhavecorrespondingregisteredclass(iftheyaretobesearched).
RequirementsforDefiningChildTableinaRelationship
Arelationshipneedstobedefinediftheprimaryclasscontainsacomplexobjecttypethatyouwanttoquery.
Supportedrelationships
1-to-1-musthavegetter/setterforasingleobject
1-to-many-musthavegetter/setterforaCollection,MaporArraytype.
Tablelevelrequirements
Thetablename(orthenameinsource)mustmatchthenameofthePojoclassstoringthechildinformation.Thisishowthetableismatchedtotheregisteredclass(s)inthecache.
Aprimarykeyisrequiredifupdatesaretobeperformed
Thechildtablemusthaveaforeignkeythatmapstotheparenttableprimarykey.
Thenameinsourcefortheforeignkeyisthenameoftheparentclassmethodtoaccessthechildobjects.
ExampleforDefiningMetadata
ThefollowingisanexampleofaPersonthatcanhavePhones.ThisdemonstrateshowtodefinetheprimaryclassandarelationshipusingDDL.
publicclassPerson{
publicStringname;
publicintid;
publicStringemail;
publicList<PhoneNumber>phones;
publicStringgetName(){
returnname;
}
publicvoidsetName(Stringname){
this.name=name;
}
ObjectTranslator
606
publicintgetId(){
returnid;
}
publicvoidsetId(intid){
this.id=id;
}
publicStringgetEmail(){
returnemail;
}
publicvoidsetEmail(Stringemail){
this.email=email;
}
publicList<PhoneNumber>getPhones(){
returnphones;
}
publicvoidsetPhones(List<PhoneNumber>phones){
this.phones=phones;
}
}
publicclassPhoneNumber{
privateStringnumber;
privateStringtype;
publicStringgetNumber(){
returnnumber;
}
publicvoidsetNumber(Stringnumber){
this.number=number;
}
publicStringgetType(){
returntype;
}
publicvoidsetType(Stringtype){
this.type=type;
}
}
Note,thisalsoshowsacontainerclass,PhoneNumber,asanexampleoftheforeignkeythat’sdefinestherelationship.
<vdbname="PersonVDB"version="1">
<modelname="PersonModel"visible="false">
<sourcename="objsource"translator-name="map-cache"connection-jndi-name="java:cache-jndi"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLEPerson(
PersonObjectobjectOPTIONS(NAMEINSOURCE'this',SELECTABLEFALSE,UPDATABLEFALSE,SE
ARCHABLE'Unsearchable',NATIVE_TYPE'org.jboss.as.quickstarts.datagrid.hotrod.query.domain.Person'),
idintegerNOTNULLOPTIONS(NAMEINSOURCE'id',SEARCHABLE'Searchable',NATIVE_TYPE'i
nt'),
namestringOPTIONS(NAMEINSOURCE'name',SEARCHABLE'Searchable',NATIVE_TYPE'java.la
ng.String'),
emailstringOPTIONS(NAMEINSOURCE'email',SEARCHABLE'Searchable',NATIVE_TYPE'java.
lang.String'),
CONSTRAINTPK_IDPRIMARYKEY(id)
)OPTIONS(UPDATABLETRUE);
ObjectTranslator
607
CREATEFOREIGNTABLEPhoneNumber(
idintegerNOTNULLOPTIONS(NAMEINSOURCE'id',SELECTABLEFALSE,UPDATABLEFALSE,SEAR
CHABLE'Searchable',NATIVE_TYPE'int'),
numberstringOPTIONS(NAMEINSOURCE'phone.number',SEARCHABLE'Searchable',NATIVE_TYP
E'java.lang.String'),
typestringOPTIONS(NAMEINSOURCE'phone.type',SEARCHABLE'Unsearchable',NATIVE_TYPE
'java.lang.Enum'),
CONSTRAINTFK_PERSONFOREIGNKEY(id)REFERENCESPerson(id)OPTIONS(NAMEINSOURCE'phon
es')
)OPTIONS(UPDATABLETRUE);
]]></metadata>
</model>
</vdb>
ThismetadatacouldalsobedefinedbyusingtheTeiidDesigner.
JCAResourceAdapter
ObjectTranslator
608
ODataTranslatorTheODatatranslator,knownbythetypename"odata"exposestheODataV2andV3datasourcesandusestheTeiidWSresourceadapterformakingwebservicecalls.ThistranslatorisextensionofWebServicesTranslator.
Note
WhatisOdata-TheOpenDataProtocol(OData)Webprotocolisforqueryingandupdatingdatathatprovidesawaytounlockyourdataandfreeitfromsilosthatexistinapplicationstoday.ODatadoesthisbyapplyingandbuildinguponWebtechnologiessuchasHTTP,AtomPublishingProtocol(AtomPub)andJSONtoprovideaccesstoinformationfromavarietyofapplications,services,andstores.ODataisbeingusedtoexposeandaccessinformationfromavarietyofsourcesincluding,butnotlimitedto,relationaldatabases,filesystems,contentmanagementsystemsandtraditionalWebsites.
UsingthisspecificationfromOASISgroup,withthehelpfromtheOData4Jframework,TeiidmapsODataentitiesintorelationalschema.TeiidsupportsreadingofCSDL(ConceptualSchemaDefinitionLanguage)fromtheODataendpointprovidedandconvertstheODataschemaintorelationalschema.ThebelowtableshowsthemappingselectionsinODataTranslatorfromCSDLdocument
OData MappedtoRelationalEntity
EntitySet Table
FunctionImport Procedure
AssosiationSet ForeignKeysontheTable*
ComplexType ignored**
AManytoManyassociationwillresultinalinktablethatcannotbeselectedfrom,butcanbeusedforjoinpurposes.
WhenusedinFunctions,animplicittableisexposed.Whenusedtodefineaembeddedtable,allthecolumnswillbein-lined
AllCRUDoperationswillbeappropriatelymappedtotheresultingentitybasedontheSQLsubmittedtotheODatatranslator.
Usage
UsageofaODatasourceissimilaraJDBCtranslator.Themetadataimportissupportedthroughthetranslator,oncethemetadataisimportedfromsourcesystemandexposedinrelationalterms,thenthissourcecanbequeriedasiftheEntitySetsandFunctionImportswerelocaltotheTeiidsystem.
ExecutionProperties
Name Description Default
DatabaseTimeZoneThetimezoneofthedatabase.Usedwhenfetchingsdate,time,ortimestampvalues
Thesystemdefaulttimezone
SupportsOdataCount Supports$count true
SupportsOdataFilter Supports$filter true
SupportsOdataOrderBy Supports$orderby true
ODataTranslator
609
SupportsOdataSkip Supports$skip true
SupportsOdataTop Supports$top true
ImporterProperties
Name Description Default
schemaNamespace Namespaceoftheschematoimport null
entityContainer EntityContainerNametoimport defaultcontainer
ExampleimportersettingstoonlyimporttablesandviewsfromNetflixCatalog.
<propertyname="importer.schemaNamespace"value="System.Data.Objects"/>
<propertyname="importer.schemaPattern"value="NetflixCatalog"/>
Note
ODataServerisnotfullycompatible-Sometimesit’spossiblethattheodataserveryouarequeryingdoesnotfullyimplementallODataspecificationfeatures.IfyourODataimplementationdoesnotsupportacertainfeature,thenturnoffthecorrespondingcapabilityusing"executionProperties",sothatTeiidwillnotpushdowninvalidqueriestothetranslator.Forexample,toturnoff$filteryouaddfollowingtoyourvdb.xml
<translatorname="odata-override"type="odata">
<propertyname="SupportsOdataFilter"value="false"/>
</translator>
thenuse"odata-override"asthetranslatornameonyoursourcemodel.
TipNativeQueries-NativeordirectqueryexecutionisnotsupportedthroughODatatranslator.However,usercanuseWebServicesTranslator’sinvokehttpmethoddirectlytoissueaRestbasedcallandparseresultsusingSQLXML.
Tip WanttouseasServer?-TeiidcannotonlyconsumeODatabaseddatasources,butitcanexposeanydatasourceasanOdatabasedwebservice.FormoreinformationseeODataSupport.
JCAResourceAdapterTheresourceadapterforthistranslatorisaWebServiceDataSource.
ODataTranslator
610
ODataV4TranslatorTheODataV4translator,knownbythetypename"odata4"exposestheODataVersion4datasourcesandusestheTeiidWSresourceadapterformakingwebservicecalls.ThistranslatorisextensionofWebServicesTranslator.
Note
WhatisOdata-TheOpenDataProtocol(OData)Webprotocolisforqueryingandupdatingdatathatprovidesawaytounlockyourdataandfreeitfromsilosthatexistinapplicationstoday.ODatadoesthisbyapplyingandbuildinguponWebtechnologiessuchasHTTP,AtomPublishingProtocol(AtomPub)andJSONtoprovideaccesstoinformationfromavarietyofapplications,services,andstores.ODataisbeingusedtoexposeandaccessinformationfromavarietyofsourcesincluding,butnotlimitedto,relationaldatabases,filesystems,contentmanagementsystemsandtraditionalWebsites.
UsingthisspecificationfromOASISgroup,withthehelpfromtheOlingoframework,TeiidmapsODataV4CSDL(ConceptualSchemaDefinitionLanguage)documentfromtheODataendpointprovidedandconvertstheODatametadataintoTeiid’srelationalschema.ThebelowtableshowsthemappingselectionsinODataV4TranslatorfromCSDLdocument
OData MappedtoRelationalEntity
EntitySet Table
EntityType Tablesee[1]
ComplexType Tablesee[2]
FunctionImport Procedure[3]
ActionImport Procedure[3]
NavigationProperties Table[4]
[1]OnlyiftheEntityTypeisexposedastheEntitySetintheEntityContainer[2]OnlyifthecomplextypeisusedaspropertyintheexposedEntitySet.Thistablewillbedesignedaschildtablewithforeignkey[1to1]or[1tomany]relationshiptotheparent[3]IfthereturntypeisEntityTypeorComplexType,theprocedureisdesignedtoreturnatable[4]Navigationpropertiesareexposedastables.Thetablewillbecreatedwithforeignkeyrelationshiptotheparent.
AllCRUDoperationswillbeappropriatelymappedtotheresultingentitybasedontheSQLsubmittedtotheODatatranslator.
Usage
UsageofaODatasourceissimilaraJDBCtranslator.Themetadataimportissupportedthroughthetranslator,oncethemetadataisimportedfromsourcesystemandexposedinrelationalterms,thenthissourcecanbequeriedasiftheEntitySets,FunctionImportsandActionImportswerelocaltotheTeiidsystem.
ThebelowissampleVDBthatcanreadmetadataservicefromTripPinserviceonhttp://odata.orgsite.
<vdbname="trippin"version="1">
<modelname="trippin">
<sourcename="odata4"translator-name="odata4"connection-jndi-name="java:/tripDS"/>
</model>
</vdb>
Therequiredresource-adapterconfigurationwilllooklike
ODataV4Translator
611
<resource-adapterid="trippin">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.webservice"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory"jndi-name=
"java:/tripDS"enabled="true"use-java-context="true"pool-name="teiid-trip-ds">
<config-propertyname="EndPoint">
http://services.odata.org/V4/(S(va3tkzikqbtgu1ist44bbft5))/TripPinServiceRW
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
Onceyouconfigureaboveresource-adapteranddeploytheVDBsuccessfully,thenyoucanconnecttotheVDBdeployedusingTeiidJDBCdriverandissueSQLstatementslike
SELECT*FROMtrippin.People;
SELECT*FROMtrippin.PeopleWHEREUserName='russelwhyte';
SELECT*FROMtrippin.PeoplepINNERJOINtrippin.People_FriendspfONp.UserName=pf.People_UserName;(notet
hatPeople_UserNameisimplicitlyaddedbyTeiidmetadata)
EXECGetNearestAirport(lat,lon);
ConfigurationofTranslator
ExecutionProperties
Executionpropertiesextend/limitthefunctionalityofthetranslatorbasedonthephysicalsourcecapabilities.Sometimesdefaultpropertiesneedtoadjustedforproperexecutionofthetranslator.
ExecutionProperties
Name Description Default
SupportsOdataCount Supports$count true
SupportsOdataFilter Supports$filter true
SupportsOdataOrderBy Supports$orderby true
SupportsOdataSkip Supports$skip true
SupportsOdataTop Supports$top true
SupportsUpdates SupportsINSERT/UPDATE/DELETE true
Sometimesit’spossiblethattheodataserveryouarequeryingdoesnotfullyimplementallODataspecificationfeatures.IfyourODataimplementationdoesnotsupportacertainfeature,thenturnoffthecorrespondingcapabilityusing"executionProperties",sothatTeiidwillnotpushdowninvalidqueriestothetranslator.Forexample,toturnoff$filteryouaddfollowingtoyourvdb.xml
<translatorname="odata-override"type="odata">
<propertyname="SupportsOdataFilter"value="false"/>
</translator>
thenuse"odata-override"asthetranslatornameonyoursourcemodel.
ODataV4Translator
612
ImporterProperties
Importerpropertiesdefinethebehaviorofthetranslatorduringthemetadataimportfromthephysicalsource.
ImporterProperties
Name Description Default
schemaNamespace Namespaceoftheschematoimport null
ExampleimportersettingstoonlyimporttablesandviewsfromTrippinserviceexposedonodata.org
<propertyname="importer.schemaNamespace"value="Microsoft.OData.SampleService.Models.TripPin"/>
Youcanleavethispropertyundefined,asifitdoesnotfindoneconfiguredthetranslatorwillselectthedefaultnameoftheEntityContainer.
JCAResourceAdapter
TheresourceadapterforthistranslatorisaWebServiceDataSource.
NoteNativeQueries-NativeordirectqueryexecutionisnotsupportedthroughODatatranslator.However,usercanuseWebServicesTranslator’sinvokehttpmethoddirectlytoissueaRestbasedcallandparseresultsusingSQLXML.
Note WanttouseasODataServer?-TeiidcannotonlyconsumeODatabaseddatasources,butitcanexposeanydatasourceasanODatabasedwebservice.FormoreinformationseeODataSupport.
ODataV4Translator
613
SwaggerTranslatorTheSwaggertranslator,knownbythetypename"swagger"exposestheSwaggerdatasourcesinrealationalconceptsandusestheTeiidWSresourceadapterformakingwebservicecalls.
Note
WhatisSwagger-http://swagger.io/[OpenAPISpecification(Swagger)]SwaggerisasimpleyetpowerfulrepresentationofyourRESTfulAPI.WiththelargestecosystemofAPItoolingontheplanet,thousandsofdevelopersaresupportingSwaggerinalmosteverymodernprogramminglanguageanddeploymentenvironment.WithaSwagger-enabledAPI,yougetinteractivedocumentation,clientSDKgenerationanddiscoverability.
StartingJanuary1st2016theSwaggerSpecificationhasbeendonatedtotheOpenAPIInitiative(OAI)andhasbeenrenamedtotheOpenAPISpecification.
UsageUsageofaSwaggersourceissimilaranyothertranslatorinTeiid.Themetadataimportissupportedthroughthetranslator,themetadataisimportedfromsourcesystem’sswagger.jsonfileandthenAPIfromthisfileisexposedasstoredproceduresinTeiid,thensourcesystemcanbequeriedbyexecutingthesestoredproceduresinTeiidsystem.
ThebelowissampleVDBthatcanreadmetadatafromPetstorereferenceserviceonhttp://petstore.swagger.io/site.
<vdbname="petstore"version="1">
<modelvisible="true"name="m">
<sourcename="s"translator-name="swagger"connection-jndi-name="java:/swagger"/>
</model>
</vdb>
Therequiredresource-adapterconfigurationwilllooklike
<resource-adapterid="swagger">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.webservice"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory"jndi-name=
"java:/swagger"enabled="true"use-java-context="true"pool-name="teiid-swagger-ds">
<config-propertyname="EndPoint">
http://petstore.swagger.io/v2
</config-property>
</connection-definition>
</connection-definitions>
</resource-adapter>
Onceyouconfigureaboveresource-adapteranddeploytheVDBsuccessfully,thenyoucanconnecttotheVDBdeployedusingTeiidJDBCdriverandissueSQLstatementslike
EXECfindPetsByStatus(('sold',))
EXECgetPetById(1461159803)
EXECdeletePet('',1461159803)
ConfigurationofTranslator
ExecutionProperties
SwaggerTranslator
614
Executionpropertiesextend/limitthefunctionalityofthetranslatorbasedonthephysicalsourcecapabilities.Sometimesdefaultpropertiesneedtoadjustedforproperexecutionofthetranslator.
ExecutionProperties
none
ImporterProperties
Importerpropertiesdefinethebehaviorofthetranslatorduringthemetadataimportfromthephysicalsource.
ImporterProperties
Name Description Default
useDefaultHost
UsedefaulthostspecifiedintheSwaggerfile;Defaultstotrue,whenfalseusestheendpointintheresource-adapter
true
preferredSchemePreferredSchemetousewhenSwaggerfilesupportsmultipleinvocationschemeslikehttp,https
null
preferredProducesPreferredAcceptMIMEtypeheader,thisshouldbeoneoftheSwagger'produces'types;
application/json
preferredConsumesPreferredContent-TypeMIMEtypeheader,thisshouldbeoneoftheSwagger'consumer'types;
application/json
Exampleimportersettingstoavoidcallinghostdefinedontheswagger.jsonfile
<propertyname="importer.useDefaultHost"value="false"/>
JCAResourceAdapter
TheresourceadapterforthistranslatorisaWebServiceDataSource.
NoteNativeQueries-NativeordirectqueryexecutionisnotsupportedthroughSwaggertranslator.However,usercanuseWebServicesTranslator’sinvokehttpmethoddirectlytoissueaRestbasedcallandparseresultsusingSQLXML.
Limitations
"application/xml"mimetypeinboth"Accept"and"Content-Type"iscurrentlynotsupported
File,Mapproeprtiesarecurrentlynotsupported,thusanymulti-partpayloadsarenotsupported
Securitymetadataiscurrentlynotsupported
Custompropertiesthatstartwith"x-"arenotsupported.
Schemawith"allof","multipleof","items"fromJSONschemaarenotsupported
SwaggerTranslator
615
SwaggerTranslator
616
OLAPTranslatorTheOLAPServicestranslator,knownbythetypenameolap,exposesstoredproceduresforcallinganalysissevicesbackedbyaOLAPserverusingMDXquerylanguage.Thistranslatorexposesastoredprocedure,invokeMDX,thatreturnsaresultsetcontainingtuplearrayvaluesforagivenMDXquery.invokeMDXwillcommonlybeusedwiththeARRAYTABLEtablefunctiontoextracttheresults.
SincetheCubemetadataexposedbytheOLAPserversandrelationaldatabasemetadataaresodifferent,thereisnosinglewaytomapthemetadatafromonetoother.ItisbesttoqueryOLAPsystemusingitsownnativeMDXlanguagethrough.MDXqueriesmybedefinedstaticallyorbuiltdynamicallyinTeiid’sabstractionlayers.
Usage
Theolaptranslatorexposesonelowlevelprocedureforaccessingolapservices.
InvokeMDXProcedure
invokeMdxreturnsaresultsetofthetuplesasarrayvalues.
ProcedureinvokeMdx(mdxinSTRING,paramsVARIADICOBJECT)returnstable(tupleobject)
ThemdxparameterisaMDXquerytobeexecutedontheOLAPserver.
Theresultsofthequerywillbereturnedsuchthateachrowontherowaxiswillbepackedintoanarrayvaluethatwillfirstcontaineachhierarchymembernameontherowaxistheneachmeasurevaluefromthecolumnaxis.
TheuseofDataRolesshouldbeconsideredtopreventarbitraryMDXfrombeingsubmittedtotheinvokeMDXprocedure.
NativeQueries
OLAPsourceproceduresmaybecreatedusingtheteiid_rel:native-queryextension-seeParameterizableNativeQueries.
Theparametervaluesubstitutiondirectlyinsertsboolean,andnumbervalues,andtreatsallothervaluesasstringliterals.
Theprocedurewillinvokethenative-querysimilartoaninvokeMdxcallwiththebenefitsthatthequeryispredeterminedandthatresultcolumntypesareknown,ratherthanrequiringtheuseofARRAYTABLEorsimilarfunctionality.
DirectQueryProcedureTheinvokeMdxprocedureisthedirectqueryprocedurefortheOLAPtranslator.Itmaybedisabledorhaveit’snamechangedviathecommondirectquerytranslatorpropertiesjustlikeanyothersource.Acalltothedirectqueryprocedurewithoutanyparameterswillnotattempttoparsethemdxqueryforparameterization.Ifparametersareused,thevaluesubstitutiondirectlyinsertsboolean,andnumbervalues,andtreatsallothervaluesasstringliterals.
JCAResourceAdapter
OLAPTranslator
617
TheresourceadapterforthistranslatorprovidedthroughdatasourceinWildFly,RefertoAdminGuidefor"JDBCDataSources"configurationsection.TwosamplexmlfilesareprovidedforaccessingOLAPserversintheteiid-examplessection.OneisMondrianspecific,whenMondrianserverisdeployedinthesameWildFlyasTeiid(mondrian-ds.xml).ToaccessanyotherOLAPserversusingXMLAinterface,thedatasourceforthemcanbecreatedusingthemexampletemplateolap-xmla-ds.xml
NoteDuetoaclassloadingchangewithMondrian3.6andlater,aworkaroundisneededtousealaterdriver-TEIID-4617Theolaptranslatormodule.xmlundermodules/system/layers/dv/org/jboss/teiid/translator/olap/main/needstohaveadependencytotheMondriandrivermodule.
OLAPTranslator
618
SalesforceTranslatorsTheSalesforcetranslatorsupportstheSELECT,DELETE,INSERT,UPSERT,andUPDATEoperationsagainstaSalesforce.comaccount.ItisdesignedforusewiththeTeiidSalesforceresourceadapter.
SalesforceAPIVersionSupport
salesforce
Thetranslator,knownbythetypenamesalesforce,providesSalesforceAPI22.0support.ThetranslatormustbeusedwiththecorrespondingSalesforceresourceadapterofthesameAPIversion.SalesforceAPIversion22.0supporthasbeendeprecated.
salesforce-34
Thetranslator,knownbythetypenameofsalesforce-34,providesSalesforceAPI34.0support.ThetranslatormustbeusedwiththecorrespondingSalesforceresourceadapterofthesameAPIversion.
OtherAPIVersions
IfyouneedconnectivitytoanAPIversionotherthanwhatisbuiltin,pleaseutilizetheprojecthttps://github.com/teiid/salesforcetogeneratenewresourceadapter/translatorpair.
ExecutionProperties
Name Description Default
ModelAuditFeilds AuditModelFields false
MaxBulkInsertBatchSize BatchSizetousetoinsertbulkinserts 2048
SupportsGroupBy
SupportsGroupByPushdown.SettofalsetohaveTeiidprocessgroupbyaggregations,suchasthosereturningmorethan2000rowswhicherrorinSOQL.
true
TheSalesforcetranslatorcanimportmetadata.
ImportProperties
PropertyName Description Required Default
NormalizeNames
Iftheimportershouldattempttomodifytheobject/fieldnamessothattheycanbeusedunquoted.
false true
SalesforceTranslators
619
excludeTables
Acase-insensitiveregularexpressionthatwhenmatchedagainstatablenamewillexcludeitfromimport.Appliedaftertablenamesareretrieved.Useanegativelook-ahead(?!<inclusionpattern>).*toactasaninclusionfilter.
false n/a
includeTables
Acase-insensitiveregularexpressionthatwhenmatchedagainstatablenamewillbeincludedduringimport.Appliedaftertablenamesareretrievedfromsource.
false n/a
importStatstics
RetrievescardinalitiesduringimportusingtheRESTAPIexplainplanfeature.
false false
NOTEWhenboth includeTablesandexcludeTablespatternsarepresentduringtheimport,the_includeTablespatternmatchedfirst,thentheexcludePatternswillbeapplied.
TIPTeiidtranslatorusesSalesforcePartnerAPIlibrariestoaccesstheSalesforcesystem.BydefaultTeiidusesPartnerAPIversion"22".MetadataretrievalusingthetranslatorandexecutionofqueriesagainstSalesforceusingthismetadataarestronglydependentupontheAPIversionthathasbeenused.ItisrequiredthatyouusesameAPIversionsforbothMetadataretrievalandqueryexecution.ForthisreasonifyouhaveolderVDBSthenyoueitherneedtore-importthemetadataorkeepusingthedefaultversion.ForanyreasonyouneedtouseadifferentversionofAPIthan"22",thenyoucanprovidethenewtranslatorandresource-adaptercombinationforthatAPI.Foryourconvenienceweprovidedatemplateprojecthttps://github.com/teiid/salesforcethatyoucanbuildanddeployintoTeiidsystemforthenewversionoftheAPI.IfyouareusingTeiidEmbedded,thenyoucouldsimplychoosetoreplacethekittedsalesforcejarswithalaterversion.Uptoversionv34isknowntobecompatible.
ExtensionMetadataProperties
Salesforceisnotrelationaldatabase,howeverTeiidprovideswaystomapSaleforcedataintorelationalconstructslikeTablesandProcedures.YoucandefineaforeignTableusingDDLinTeiidVDB,whichmapstoSalesforce’sSObject.Atruntime,tointerpretthistablebacktoaSObject,Teiiddecoratesortagsthistabledefinitionwithadditionalmetadata.Forexample,atableisdefinedas
CREATEFOREIGNTABLEPricebook2(
Idstring,
Namestring,
IsActiveboolean,
IsStandardboolean,
Descriptionstring,
IsDeletedboolean)
OPTIONS(
UPDATABLE'TRUE',
"teiid_sf:SupportsQuery"'TRUE');
IntheabovethepropertyinOPTIONSclausewithproperty"teiid_sf:SupportsQuery"annotatingthatthistablessupportsSELECTcommands.ThebelowarelistofmetadataextensionpropertiesthatcanbeusedonSalesforceschema.
PropertyName Description Required Default AppliesTo
SalesforceTranslators
620
SupportsQueryThetablesupportsSELECTcommands
false true Table
SupportsRetrieve
ThetablesupportsretrievalofresultsasresultofSELECTcommands
false true Table
SQLProcessing
Salesforcedoesnotprovidethesamesetoffunctionalityasarelationaldatabase.Forexample,Salesforcedoesnotsupportarbitraryjoinsbetweentables.However,workingincombinationwiththeTeiidQueryPlanner,theSalesforceconnectorsupportsnearlyalloftheSQLsyntaxsupportedbytheTeiid.
TheSalesforceConnectorexecutesSQLcommandsby"pushingdown"thecommandtoSalesforcewheneverpossible,basedonthesupportedcapabilities.TeiidwillautomaticallyprovideadditionaldatabasefunctionalitywhentheSalesforceConnectordoesnotexplicitlyprovidesupportforagivenSQLconstruct.IncaseswherecertainSQLcapabilitiescannotbepusheddowntoSalesforce,Teiidwillpushdownthecapabilitiesthataresupported,andfetchasetofdatafromSalesforce.Then,Teiidwillevaluatetheadditionalcapabilities,creatingasubsetoftheoriginaldataset.Finally,Teiidwillpasstheresulttotheclient.
IfyouareissuingquerieswithagroupbyclauseandreceiveanerrorforsalesforcerelatedtoqueryMorenotbeingsupported,youmayeitheraddlimitsorsettheexecutionpropertySupportsGroupBytofalse.
SELECTarray_agg(Reports)FROMSupervisorwhereDivision='customersupport';
NeitherSalesforcenortheSalesforceConnectorsupportthearray_agg()scalar,buttheydosupportCompareCriteriaEquals,sothequerythatispassedtoSalesforcebytheconnectorwillbetransformedtothisquery.
SELECTReportsFROMSupervisorwhereDivision='customersupport';
Thearray_agg()functionwillbeappliedbytheTeiidQueryEnginetotheresultsetreturnedbytheconnector.
InsomecasesmultiplecallstotheSalesforceapplicationwillbemadetosupporttheSQLpassedtotheconnector.
DELETEFromCaseWHEREStatus='Closed';
TheAPIinSalesforcetodeleteobjectsonlysupportsdeletingbyID.InordertoaccomplishthistheSalesforceconnectorwillfirstexecuteaquerytogettheIDsofthecorrectobjects,andthendeletethoseobjects.SotheaboveDELETEcommandwillresultinthefollowingtwocommands.
SELECTIDFromCaseWHEREStatus='Closed';
DELETEFromCasewhereIDIN(<resultofquery>);
NOTETheSalesforceAPIDELETEcallisnotexpressedinSQL,buttheaboveisanequivalentSQLexpression.
It’susefultobeawareofunsupportedcapabilities,inordertoavoidfetchinglargedatasetsfromSalesforceandmakingyouqueriesasperformantaspossible.SeeallSupportedCapabilities.
SelectingfromMulti-SelectPicklists
SalesforceTranslators
621
Amulti-selectpicklistisafieldtypeinSalesforcethatcancontainmultiplevaluesinasinglefield.QuerycriteriaoperatorsforfieldsofthistypeinSOQLarelimitedtoEQ,NE,includesandexcludes.ThefullSalesforcedocumentationforselectingfrommullti-selectpicklistscanbefoundatthefollowinglinkQueryingMulit-selectPicklists
TeiidSQLdoesnotsupporttheincludesorexcludesoperators,buttheSalesforceconnectorprovidesuserdefinedfunctiondefinitionsfortheseoperatorsthatprovidedequivalentfunctionalityforfieldsoftypemulti-select.Thedefinitionforthefunctionsis:
booleanincludes(Columncolumn,Stringparam)
booleanexcludes(Columncolumn,Stringparam)
Forexample,takeasinglemulti-selectpicklistcolumncalledStatusthatcontainsallofthesevalues.
current
working
critical
Forthatcolumn,allofthebelowarevalidqueries:
SELECT*FROMIssueWHEREtrue=includes(Status,'current,working');
SELECT*FROMIssueWHEREtrue=excludes(Status,'current,working');
SELECT*FROMIssueWHEREtrue=includes(Status,'current;working,critical');
EQandNEcriteriawillpasstoSalesforceassupplied.Forexample,thesequerieswillnotbemodifiedbytheconnector.
SELECT*FROMIssueWHEREStatus='current';
SELECT*FROMIssueWHEREStatus='current;critical';
SELECT*FROMIssueWHEREStatus!='current;working';
SelectingAllObjects
TheSalesforceconnectorsupportsthecallingthequeryAlloperationfromtheSalesforceAPI.ThequeryAlloperationisequivalenttothequeryoperationwiththeexceptionthatitreturnsdataaboutallcurrentanddeletedobjectsinthesystem.
TheconnectordeterminesifitwillcallthequeryorqueryAlloperationviareferencetotheisDeletedpropertypresentoneachSalesforceobject,andmodeledasacolumnoneachtablegeneratedbytheimporter.BydefaultthisvalueissettoFalsewhenthemodelisgeneratedandthustheconnectorcallsquery.UsersarefreetochangethevalueinthemodeltoTrue,changingthedefaultbehaviouroftheconnectortobequeryAll.
ThebehaviorisdifferentifisDeletedisusedasaparameterinthequery.IftheisDeletedcolumnisusedasaparameterinthequery,andthevalueis'true'theconnectorwillcallqueryAll.
select*fromContactwhereisDeleted=true;
IftheisDeletedcolumnisusedasaparameterinthequery,andthevalueis'false'theconnectorperformthedefaultbehaviorwillcallquery.
select*fromContactwhereisDeleted=false;
SelectingUpdatedObjects
IftheoptionisselectedwhenimportingmetadatafromSalesforce,aGetUpdatedprocedureisgeneratedinthemodelwiththefollowingstructure:
SalesforceTranslators
622
GetUpdated(ObjectNameINstring,
StartDateINdatetime,
EndDateINdatetime,
LatestDateCoveredOUTdatetime)
returns
IDstring
SeethedescriptionoftheGetUpdatedoperationintheSalesforcedocumentationforusagedetails.
SelectingDeletedObjects
IftheoptionisselectedwhenimportingmetadatafromSalesforce,aGetDeletedprocedureisgeneratedinthemodelwiththefollowingstructure:
GetDeleted(ObjectNameINstring,
StartDateINdatetime,
EndDateINdatetime,
EarliestDateAvailableOUTdatetime,
LatestDateCoveredOUTdatetime)
returns
IDstring,
DeletedDatedatetime
SeethedescriptionoftheGetDeletedoperationintheSalesforcedocumentationforusagedetails.
RelationshipQueries
Salesforcedoesnotsupportjoinslikearelationaldatabase,butitdoeshavesupportforqueriesthatincludeparent-to-childorchild-to-parentrelationshipsbetweenobjects.ThesearetermedRelationshipQueries.TheSalesForceconnectorsupportsRelationshipQueriesthroughOuterJoinsyntax.
SELECTAccount.name,Contact.NamefromContactLEFTOUTERJOINAccount
onContact.Accountid=Account.id
ThisqueryshowsthecorrectsyntaxtoqueryaSalesForcemodelwithtoproducearelationshipqueryfromchildtoparent.ItresolvestothefollowingquerytoSalesForce.
SELECTContact.Account.Name,Contact.NameFROMContact
selectContact.Name,Account.NamefromAccountLeftouterJoinContact
onContact.Accountid=Account.id
ThisqueryshowsthecorrectsyntaxtoqueryaSalesForcemodelwithtoproducearelationshipqueryfromparenttochild.ItresolvestothefollowingquerytoSalesForce.
SELECTAccount.Name,(SELECTContact.NameFROM
Account.Contacts)FROMAccount
SeethedescriptionoftheRelationshipQueriesoperationintheSalesForcedocumentationforlimitations.
BulkInsertQueries
SalesforceTranslators
623
SalesForcetranslatoralsosupportsbulkinsertstatementsusingJDBCbatchsemanticsorSELECTINTOsemantics.ThebatchsizeisdeterminedbytheexecutionpropertyMaxBulkInsertBatchSize,whichcanbeoverriddeninthevdb.xmlfile.Thedefaultvalueofthebatchis2048.ThebulkinsertfeatureusestheasyncRESTbasedAPIexposedbySalesforceforexecutionforbetterperformance.
BulkSelects
Whenqueringlargetables(typicallyover10,000,000records)orifexperiencingtimeoutswithjustresultbatching,TeiidcanissuequeriestoSalesforceusingthebulkAPI.Whenusingabulkselect,PKchunkingwillbeenabledifsupportedbythequery.
Theuseofthebulkapirequiresasourcehintinthequery:
SELECT/*+shsalesforce:'bulk'*/Name...FROMAccount
Wheresalesforceisthesourcenameofthetargetsource.
Thedefaultchunksizeof100,000recordswillbeused.
Note:thisfeatureisonlysupportedbySalsforceAPIequaltogreaterthan28,sincethedefault"salesforce"translatorusesversion22ofAPI,itrecommendedtouse"salesforce-34"versionofthetranslatortousethisfeature.
SupportedCapabilitiesThefollowingarethecapabilitiessupportedbytheSalesforceConnector.TheseSQLconstructswillbepusheddowntoSalesforce.
SELECTcommand
INSERTCommand
UPDATECommand
DELETECommand
NotCriteria
OrCriteria
CompareCriteriaEquals
CompareCriteriaOrdered
IsNullCritiera
InCriteria
LikeCriteria-SupportedforStringfieldsonly.
RowLimit
BasicAggregates
OuterJoinswithjoincriteriaKEY
NativeQueries
SalesforceTranslators
624
Salesforceproceduresmayoptionallyhavenativequeriesassociatedwiththem-seeParameterizableNativeQueries.Theoperationprefix(select;,insert;,update;,delete;-seebelowformore)mustbepresentinthenative-query,butitwillnotbeissuedaspartofthequerytothesource.
ExampleDDLforaSFnativeprocedure
CREATEFOREIGNPROCEDUREproc(arg1integer,arg2string)OPTIONS("teiid_rel:native-query"'search;SELECT...
complexSOQL...WHEREcol1=$1andcol2=$2')returns(col1string,col2string,col3timestamp);
DirectQueryProcedure
Thisfeatureisturnedoffbydefaultbecauseofthesecurityriskthisexposestoexecuteanycommandagainstthesource.Toenablethisfeature,overridetheexecutionpropertycalled_SupportsDirectQueryProceduretotrue.
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyisnative.Overridetheexecutionproperty_DirectQueryProcedureNametochangeittoanothername.
TheSalesforcetranslatorprovidesaproceduretoexecuteanyad-hocSOQLquerydirectlyagainstthesourcewithoutTeiidparsingorresolving.Sincethemetadataofthisprocedure’sresultsarenotknowntoTeiid,theyarereturnedasanobjectarray.ARRAYTABLEcanbeusedconstructtabularoutputforconsumptionbyclientapplications.Teiidexposesthisprocedurewithasimplequerystructureasfollows:
Select
SelectExample
SELECTx.*FROM(callsf_source.native('search;SELECTAccount.Id,Account.Type,Account.NameFROMAccount'))w,
ARRAYTABLE(w.tupleCOLUMNS"id"string,"type"string,"name"String)ASx
fromtheabovecode,the"search"keywordfollowedbyaquerystatement.
Note TheSOQListreatedasaparameterizednativequerysothatparametervaluesmaybeinsertedinthequerystringproperly-seeParameterizableNativeQueries
TheresultsreturnedbysearchmaycontaintheobjectIdasthefirstcolumnvalueregardlessofwhetheritwasselected.Alsoqueriesthatselectcolumnsfrommultipleobjecttypeswillnotbecorrect.
Delete
DeleteExample
SELECTx.*FROM(callsf_source.native('delete;','id1','id2'))w,
ARRAYTABLE(w.tupleCOLUMNS"updatecount"integer)ASx
formtheabovecode,the"delete;"keywordfollowedbytheidstodeleteasvarargs.
CreateorUpdate
CreateExample
SELECTx.*FROM
(callsf_source.native('create;type=table;attributes=one,two,three','one',2,3.0))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
SalesforceTranslators
625
formtheabovecode,the"create"or"update"keywordmustbefollowedbythefollowingproperties.Attributesmustbematchedpositionallybytheprocedurevariables-thusintheexampleattributetwowillbesetto2.
PropertyName Description Required
type TableName Yes
attributes commaseparatedlistofnamesofthecolumns no
Thevaluesforeachattributeisspecifiedasseparateargumenttothe"native"procedure.
Updateissimilartocreate,withonemoreextrapropertycalled"id",whichdefinesidentifierfortherecord.
UpdateExample
SELECTx.*FROM
(callsf_source.native('update;id=pk;type=table;attributes=one,two,three','one',2,3.0))w,
ARRAYTABLE(w.tupleCOLUMNS"update_count"integer)ASx
Tip Bydefaultthenameoftheprocedurethatexecutesthequeriesdirectlyiscallednative,howeverusercan+setoverrideexecutionpropertyvdb.xmlfiletochangeit.
JCAResourceAdapterTheresourceadapterforthistranslatorisprovidedthroughSalesforceDataSources.RefertoAdminGuideforconfiguration.
SalesforceTranslators
626
SAPGatewayTranslatorTheSAPGatewayTranslator,knownbythetypenamesap-gateway,providesatranslatorforaccessingtheSAPGatewayusingtheODataprotocol.ThistranslatorisextensionofODataTranslatorandusesTeiidWSresourceadapterformakingwebservicecalls.ThistranslatorunderstandsthemostoftheSAPspecificODataextensionstothemetadatadefinedinthedocumentSAPAnnotationsforODataVersion2.0
WhenthemetadataisimportedfromSAPGateway,theTeiidmodelsarecreatedtoaccordinglyforSAPspecificEntitySetandPropertyannotationsdefinedindocumentabove.
Thefollowing"executionproperties"aresupportedinthistranslator
ExecutionProperties
Name Description Default
DatabaseTimeZoneThetimezoneofthedatabase.Usedwhenfetchingsdate,time,ortimestampvalues
Thesystemdefaulttimezone
SupportsOdataCount Supports$count true
SupportsOdataFilter Supports$filter true
SupportsOdataOrderBy Supports$orderby true
SupportsOdataSkip Supports$skip true
SupportsOdataTop Supports$top true
BasedonhowyouimplementedyourSAPGatewayservice,ifcanchoosetoturnoffsomeofthefeaturesabove.
Note
Usingpagable,topablemetadataextensions?-Ifmetadataonyourservicedefined"pagable"and/or"topable"as"false"onanytable,youmustturnoff"SupportsOdataTop"and"SupportsOdataSkip"execution-propertiesinyourtranslator,sothatyouwillnotendupwithwrongresults.SAPmetadatahascapabilitytocontroltheseinafinegrainedfashionanyonEnitySet,howeverTeiidcanonlycontroltheseattranslatorlevel.
Note
SAPExamples-Sampleexamplesdefinedathttp://scn.sap.com/docs/DOC-31221,wefoundtobelackinginfullmetadataincertainexamples.Forexample,"filterable"clauseneverdefinedonsomeproperties,butifyousendarequest$filteritwillsilentlyignoreit.YoucanverifythisbehaviorbydirectlyexecutingtheRESTserviceusingawebbrowserwithrespectivequery.So,Makesureyouhaveimplementedyourservicecorrectly,oryoucanturnoffcertainfeaturesinthistranslatorbyusing"executionproperties"override.SeeanexampleinODataTranslator
SAPGatewayTranslator
627
WebServicesTranslatorTheWebServicestranslator,knownbythetypenamews,exposesstoredproceduresforcallingwebservicesbackedbyaTeiidWSresourceadapter.TheWSresourceadaptermayoptionallybeconfiguredtopointataspecificWSDL.ResultsfromthistranslatorwillcommonlybeusedwiththeTEXTTABLEorXMLTABLEtablefunctionstouseCSVorXMLformateddata.
ExecutionProperties
Name Description WhenUsed Default
DefaultBinding
Thebindingthatshouldbeusedifoneisnotspecified.CanbeoneofHTTP,SOAP11,orSOAP12
invoke* SOAP12
DefaultServiceMode
Thedefaultservicemode.ForSOAP,MESSAGEmodeindicatesthattherequestwillcontaintheentireSOAPenvelopeandnotjustthecontentsoftheSOAPbody.CanbeoneofMESSAGEorPAYLOAD
invoke*orWSDLcall PAYLOAD
XMLParamName
UsedwiththeHTTPbinding(typicallywiththeGETmethod)toindicatethattherequestdocumentshouldbepartofthequerystring.
invoke* null-unused
NoteSettingtheproperbindingvalueonthetranslatorisrecommendedasitremovestheneedforcallerstopassanexplicitvalue.IfyourserviceisactuallyusesSOAP11,butthebindingusedSOAP12youwillreceiveexecutionfailures.
Therearenowsimportersettings,butitcanprovidemetadataforVDBs.IftheconnectionisconfiguredtopointataspecificWSDL,thetranslatorwillimportallSOAPoperationsunderthespecifiedserviceandportasprocedures.
ImporterPropertiesWhenspecifyingtheimporterproperty,itmustbeprefixedwith"importer.".Example:importer.tableTypes
Name Description Default
importWSDL ImportthemetadatafromtheWSDLURLconfiguredinresource-adapter true
Usage
TheWStranslatorexposeslowlevelproceduresforaccessingwebservices.Seealsothetwitterexampleinthekit.
InvokeProcedure
WebServicesTranslator
628
Invokeallowsformultiplebinding,orprotocolmodes,includingHTTP,SOAP11,andSOAP12.
Procedureinvoke(bindinginSTRING,actioninSTRING,requestinXML,endpointinSTRING,streaminBOOLEAN)re
turnsXML
Thebindingmaybeoneofnull(tousethedefault)HTTP,SOAP11,orSOAP12.ActionwithaSOAPbindingindicatestheSOAPActionvalue.ActionwithaHTTPbindingindicatestheHTTPmethod(GET,POST,etc.),whichdefaultstoPOST.
Anullvalueforthebindingorendpointwillusethedefaultvalue.ThedefaultendpointisspecifiedintheWSresourceadapterconfiguration.TheendpointURLmaybeabsoluteorrelative.Ifit’srelativethenitwillbecombinedwiththedefaultendpoint.
Sincemultipleparametersarenotrequiredtohavevalues,itisoftenmorecleartocalltheinvokeprocedurewithnamedparametersyntax.
callinvoke(binding=>'HTTP',action=>'GET')
TherequestXMLshouldbeavalidXMLdocumentorrootelement.
InvokeHTTPProcedure
invokeHttpcanreturnthebytecontentsofanHTTP(S)call.
ProcedureinvokeHttp(actioninSTRING,requestinOBJECT,endpointinSTRING,streaminBOOLEAN,contentTypeou
tSTRING,headersinCLOB)returnsBLOB
ActionindicatestheHTTPmethod(GET,POST,etc.),whichdefaultstoPOST.
Anullvalueforendpointwillusethedefaultvalue.ThedefaultendpointisspecifiedintheWSresourceadapterconfiguration.TheendpointURLmaybeabsoluteorrelative.Ifit’srelativethenitwillbecombinedwiththedefaultendpoint.
Sincemultipleparametersarenotrequiredtohavevalues,itisoftenmorecleartocalltheinvokeprocedurewithnamedparametersyntax.
callinvokeHttp(action=>'GET')
TherequestcanbeoneofSQLXML,STRING,BLOB,orCLOB.TherequestwillbesentasthePOSTpayloadinbyteform.ForSTRING/CLOBvaluesthiswilldefaulttotheUTF-8encoding.Tocontrolthebyteencoding,seetheto_bytesfunction.
TheoptionalheadersparametercanbeusedtospecifytherequestheadervaluesasaJSONvalue.TheJSONvalueshouldbeaJSONobjectwithprimitiveorlistofprimitivevalues.
callinvokeHttp(...headers=>jsonObject('application/json'as"Content-Type",jsonArray('gzip','deflate')as"
Accept-Encoding"))
Recommendationsforsettingheadersparameter:
Content-TypemaybenecessaryifHTTPPOST/PUTmethodisinvoked
AcceptisnecessaryifyouwanttocontrolreturnMediaType
WSDLbasedProcedures
WebServicesTranslator
629
Theproceduresabovegiveyouanonymouswaytoexecuteanywebservicemethodsbysupplyinganendpoint,withthismechanismyoucanaltertheendpointdefinedinWSDLwithadifferentendpoint.However,ifyouhaveaccesstotheWSDL,thenyoucanconfiguretheWSDLURLintheweb-serviceresource-adapter’sconnectionconfiguration,WebServicetranslatorcanparsetheWSDLandprovidethemethodsunderconfiguredportaspre-builtproceduresasitsmetadata.Ifyouareusingthedefaultnativemetadataimport,youwillseetheproceduresinyourwebservice’ssourcemodel.
Note Nativequeries-NativequeriesoradirectqueryexecutionprocedureisnotsupportedontheWebServicesTranslator.
StreamingConsiderations
Ifthestreamparameterissettotrue,thentheresultinglobvaluemayonlybeusedasingletime.Ifstreamisnullorfalse,thentheenginemayneedtosaveacopyoftheresultforrepeateduse.Caremustbeusedassomeoperations,suchascastingorXMLPARSEmayperformvalidationwhichresultsinthestreambeingconsumed.
JCAResourceAdapter
TheresourceadapterforthistranslatorisaWebServiceDataSource.
Note WS-Security-CurrentlyyoucanonlyuseWSDLbasedProceduresparticipateinWS-Security,whenresource-adapterisconfiguredwithcorrectCXFconfiguration.
WebServicesTranslator
630
FederatedPlanningTeiidatitscoreisafederatedrelationalqueryengine.ThisqueryengineallowsyoutotreatallofyourdatasourcesasonevirtualdatabaseandaccesstheminasingleSQLquery.Thisallowsyoutofocusonbuildingyourapplication,notonhand-codingjoins,andotherrelationaloperations,betweendatasources.
ChildPages
PlanningOverview
QueryPlanner
QueryPlans
FederatedOptimizations
SubqueryOptimization
XQueryOptimization
FederatedFailureModes
ConformedTables
FederatedPlanning
631
PlanningOverviewWhenthequeryenginereceivesanincomingSQLqueryitperformsthefollowingoperations:
1. Parsing-validatesyntaxandconverttointernalform
2. Resolving-linkallidentifierstometadataandfunctionstothefunctionlibrary
3. Validating-validateSQLsemanticsbasedonmetadatareferencesandtypesignatures
4. Rewriting-rewriteSQLtosimplifyexpressionsandcriteria
5. Logicalplanoptimization-therewrittencanonicalSQLisconvertedintoalogicalplanforin-depthoptimization.TheTeiidoptimizerispredominantlyrule-based.Baseduponthequerystructureandhintsacertainrulesetwillbeapplied.Theserulesmaytriggerinturntriggertheexecutionofmorerules.Withinseveralrules,Teiidalsotakesadvantageofcostinginformation.ThelogicalplanoptimizationstepscanbeseenbyusingSETSHOWPLANDEBUGclause,asamplestepsaredescribedinQueryPlanner#ReadingaDebugPlan.Moredetailsaboutlogicalplan’snodesandrule-basedoptimizationrefertoQueryPlanner.
6. Processingplanconversion-thelogicplanisconvertedintoanexecutableformwherethenodesarerepresentativeofbasicprocessingoperations.ThefinalprocessingplanisdisplayedastheQueryPlans.
Thelogicalqueryplanisatreeofoperationsusedtotransformdatainsourcetablestotheexpectedresultset.Inthetree,dataflowsfromthebottom(tables)tothetop(output).Theprimarylogicaloperationsareselect(selectorfilterrowsbasedonacriteria),project(projectorcomputecolumnvalues),join,source(retrievedatafromatable),sort(ORDERBY),duplicateremoval(SELECTDISTINCT),group(GROUPBY),andunion(UNION).
Forexample,considerthefollowingquerythatretrievesallengineeringemployeesbornsince1970.
Examplequery
SELECTe.title,e.lastnameFROMEmployeesASeJOINDepartmentsASdONe.dept_id=d.dept_idWHEREyear(e.birt
hday)>=1970ANDd.dept_name='Engineering'
Logically,thedatafromtheEmployeesandDepartmentstablesareretrieved,thenjoined,thenfilteredasspecified,andfinallytheoutputcolumnsareprojected.Thecanonicalqueryplanthuslookslikethis:
PlanningOverview
632
Dataflowsfromthetablesatthebottomupwardsthroughthejoin,throughtheselect,andfinallythroughtheprojecttoproducethefinalresults.Thedatapassedbetweeneachnodeislogicallyaresultsetwithcolumnsandrows.
Ofcourse,thisiswhathappenslogically,nothowtheplanisactuallyexecuted.Startingfromthisinitialplan,thequeryplannerperformstransformationsonthequeryplantreetoproduceanequivalentplanthatretrievesthesameresultsfaster.Bothafederatedqueryplannerandarelationaldatabaseplannerdealwiththesameconceptsandmanyofthesameplantransformations.Inthisexample,thecriteriaontheDepartmentsandEmployeestableswillbepusheddownthetreetofiltertheresultsasearlyaspossible.
Inbothcases,thegoalistoretrievethequeryresultsinthefastestpossibletime.However,therelationaldatabaseplannerdoesthisprimarilybyoptimizingtheaccesspathsinpullingdatafromstorage.
Incontrast,afederatedqueryplannerislessconcernedaboutstorageaccessbecauseitistypicallypushingthatburdentothedatasource.Themostimportantconsiderationforafederatedqueryplannerisminimizingdatatransfer.
PlanningOverview
633
QueryPlannerCanonicalPlanandAllNodes
NodeProperties
AccessProperties
SetoperationProperties
JoinProperties
ProjectProperties
SelectProperties
SortProperties
SourceProperties
GroupProperties
TupleLimitProperties
GeneralandCostingProperties
Rules
Foreachsub-commandintheusercommandanappropriatekindofsub-plannerisused(relational,XML,procedure,etc).
Eachplannerhasthreeprimaryphases:
1. Generatecanonicalplan
2. Optimization
3. Plantoprocessconverter-convertsplandatastructureintoaprocessingform
RelationalPlanner
Arelationalprocessingplaniscreatedbytheoptimizerafterthelogicalplanismanipulatedbyaseriesofrules.Theapplicationofrulesisdeterminedbothbythequerystructureandbytherulesthemselves.Thenodestructureofthedebugplanresemblesthatoftheprocessingplan,butthenodetypesmorelogicallyrepresentSQLoperations.
CanonicalPlanandAllNodes
AsPlanningOverview,auserSQLstatementafterParsing,Resolving,Validating,Rewriting,itbeconvertedintoacanonicalplanform.ThecanonicalplanformmostcloselyresemblestheinitialSQLstructure.ASQLselectqueryhasthefollowingpossibleclauses(allbutSELECTareoptional):WITH,SELECT,FROM,WHERE,GROUPBY,HAVING,ORDERBY,LIMIT.Theseclausesarelogicallyexecutedinthefollowingorder:
1. WITH(createcommontableexpressions)-handledbyaspecializedPROJECTNODE
2. FROM(readandjoinalldatafromtables)-SOURCEnodeforeachfromclauseitem,Joinnode(if>1table)
3. WHERE(filterrows)-SELECTnode
4. GROUPBY(grouprowsintocollapsedrows)-GROUPnode
QueryPlanner
634
5. HAVING(filtergroupedrows)-SELECTnode
6. SELECT(evaluateexpressionsandreturnonlyrequestedrows)-PROJECTnodeandDUP_REMOVEnode(forSELECTDISTINCT)
7. INTO-specializedPROJECTwithaSOURCEchild
8. ORDERBY(sortrows)-SORTnode
9. LIMIT(limitresultsettoacertainrangeofresults)-LIMITnode
Forexample,aSQLstatementsuchasSELECTmax(pm1.g1.e1)FROMpm1.g1WHEREe2=1createsalogicalplan:
Project(groups=[anon_grp0],props={PROJECT_COLS=[anon_grp0.agg0ASexpr1]})
Group(groups=[anon_grp0],props={SYMBOL_MAP={anon_grp0.agg0=MAX(pm1.G1.E1)}})
Select(groups=[pm1.G1],props={SELECT_CRITERIA=pm1.G1.E2=1})
Source(groups=[pm1.G1])
HeretheSourcecorrespondstotheFROMclause,theSelectcorrespondstotheWHEREclause,theGroupcorrespondstotheimpliedgroupingtocreatethemaxaggregate,andtheProjectcorrespondstotheSELECTclause.
Note Theaffectofgroupinggenerateswhatiseffectivelyaninlineview,anon_grp0,tohandletheprojectionofvaluescreatedbythegrouping.
Table1.NodeTypes
TypeName Description
ACCESS asourceaccessorplanexecution.
DUP_REMOVE removesduplicaterows
JOIN ajoin(LEFTOUTER,FULLOUTER,INNER,CROSS,SEMI,etc.)
QueryPlanner
635
PROJECT aprojectionoftuplevalues
SELECT afilteringoftuples
SORT anorderingoperation,whichmaybeinsertedtoprocessotheroperationssuchasjoins
SOURCE anylogicalsourceoftuplesincludinganinlineview,asourceaccess,XMLTABLE,etc.
GROUP agroupingoperation
SET_OP asetoperation(UNION/INTERSECT/EXCEPT)
NULL asourceofnotuples
TUPLE_LIMIT rowoffset/limit
NodeProperties
Eachnodehasasetofapplicablepropertiesthataretypicallyshownonthenode.
AccessProperties
Table2.AccessProperties
PropertyName Description
ATOMIC_REQUEST Thefinalformofasourcerequest
MODEL_ID Themetadataobjectforthetargetmodel/schema
PROCEDURE_CRITERIA/PROCEDURE_INPUTS/PROCEDURE_DEFAULTS Usedinplanningprocedurealrelationalqueries
IS_MULTI_SOURCE settotruewhenthenoderepresentsamulti-sourceaccess
SOURCE_NAME usedtotrackthemulti-sourcesourcename
CONFORMED_SOURCEStracksthesetofconformedsourceswhentheconformedextensionmetadataisused
SUB_PLAN/SUB_PLANS usedinmulti-sourceplanning
SetoperationProperties
Table3.SetoperationProperties
PropertyName Description
SET_OPERATION/USE_ALL definesthesetoperation(UNION/INTERSECT/EXCEPT)andifallrowsordistinctrowsareused.
QueryPlanner
636
JoinProperties
Table4.JoinProperties
PropertyName Description
JOIN_CRITERIA alljoinpredicates
JOIN_TYPE typeofjoin(INNER,LEFTOUTER,etc.)
JOIN_STRATEGY thealgorithmtouse(nestedloop,merge,etc.)
LEFT_EXPRESSIONS theexpressionsinequi-joinpredicatesthatoriginatefromtheleftsideofthejoin
RIGHT_EXPRESSIONS theexpressionsinequi-joinpredicatesthatoriginatefromtherightsideofthejoin
DEPENDENT_VALUE_SOURCE setifadependentjoinisused
NON_EQUI_JOIN_CRITERIA non-equijoinpredicates
SORT_LEFT iftheleftsideneedssortedforjoinprocessing
SORT_RIGHT iftherightsideneedssortedforjoinprocessing
IS_OPTIONAL ifthejoinisoptional
IS_LEFT_DISTINCT iftheleftsideisdistinctwithrespecttotheequijoinpredicates
IS_RIGHT_DISTINCT iftherightsideisdistinctwithrespecttotheequijoinpredicates
IS_SEMI_DEP ifthedependentjoinrepresentsasemi-join
PRESERVE ifthepreservehintispreservingthejoinorder
ProjectProperties
Table5.ProjectProperties
PropertyName Description
PROJECT_COLS theexpressionsprojected
INTO_GROUP thegrouptargetedifthisisaselectintoorinsertwithaqueryexpression
HAS_WINDOW_FUNCTIONS trueifwindowfunctionsareused
CONSTRAINT theconstraintthatmustbemetifthevaluesarebeingprojectedintoagroup
UPSERT Iftheinsertisanupsert
QueryPlanner
637
SelectProperties
Table6.SelectProperties
PropertyName Description
SELECT_CRITERIA thefilter
IS_HAVING ifthefilterisappliedaftergrouping
IS_PHANTOM trueifthenodeismarkedforremoval,buttemporarilyleftintheplan.
IS_TEMPORARY inferredcriteriathatmaynotbeusedinthefinalplan
IS_COPIED ifthecriteriahasalreadybeenprocessedbyrulecopycriteria
IS_PUSHED ifthecriteriaispushedasfaraspossible
IS_DEPENDENT_SET ifthecriteriaisthefilterofadependentjoin
SortProperties
Table7.SortProperties
PropertyName Description
SORT_ORDER theorderbythatdefinesthesort
UNRELATED_SORT iftheorderingincludesavaluethatisnotbeingprojected
IS_DUP_REMOVAL ifthesortshouldalsoperformduplicateremovalovertheentireprojection
SourceProperties
Table8.SourceProperties
PropertyName Description
SYMBOL_MAP themappingfromthecolumnsabovethesourcetotheprojectedexpressions.AlsopresentonGroupnodes
PARTITION_INFO thepartitioningoftheunionbranches
VIRTUAL_COMMAND ifthesourcerepresentsanvieworinlineview,thequerythatdefinedtheview
MAKE_DEP hintinformation
PROCESSOR_PLAN theprocessorplanofanon-relationalsource(typicallyfromtheNESTED_COMMAND)
NESTED_COMMAND thenon-relationalcommand
QueryPlanner
638
TABLE_FUNCTION thetablefunction(XMLTABLE,OBJECTTABLE,etc.)definingthesource
CORRELATED_REFERENCES thecorrelatedreferencesforthenodesbelowthesource
MAKE_NOT_DEP ifmakenotdepisset
INLINE_VIEW Ifthesourcenoderepresentsaninlineview
NO_UNNEST iftheno_unnesthintisset
MAKE_IND ifthemakeindhintisset
SOURCE_HINT thesourcehint.SeeFederatedOptimizations.
ACCESS_PATTERNS accesspatternsyettobesatisfied
ACCESS_PATTERN_USED satisfiedaccesspatterns
REQUIRED_ACCESS_PATTERN_GROUPS groupsneededtosatisfytheaccesspatterns.Usedinjoinplanning.
Note Manysourcepropertiesalsobecomepresentonassociatedaccessnodes.
GroupProperties
Table9.GroupProperties
PropertyName Description
GROUP_COLS thegroupingcolumns
ROLLUP ifthegroupingincludesarollup
TupleLimitProperties
Table10.TupleLimitProperties
PropertyName Description
MAX_TUPLE_LIMIT expressionthatevaluatestothemaxnumberoftuplesgenerated
OFFSET_TUPLE_COUNT Expressionthatevaluatestothetupleoffsetofthestartingtuple
IS_IMPLICIT_LIMIT ifthelimitiscreatedbytherewriteraspartofasubqueryoptimization
IS_NON_STRICT iftheunorderedlimitshouldnotbeenforcedstrictly
GeneralandCostingProperties
Table11.GeneralandCostingProperties
QueryPlanner
639
PropertyName Description
OUTPUT_COLS theoutputcolumnsforthenode.Istypicallysetafterruleassignoutputelements.
EST_SET_SIZErepresentstheestimatedsetsizethisnodewouldproduceforasiblingnodeastheindependentnodeinadependentjoinscenario
EST_DEP_CARDINALITYvaluethatrepresentstheestimatedcardinality(amountofrows)producedbythisnodeasthedependentnodeinadependentjoinscenario
EST_DEP_JOIN_COST valuethatrepresentstheestimatedcostofadependentjoin(thejoinstrategyforthiscouldbeNestedLooporMerge)
EST_JOIN_COST valuethatrepresentstheestimatedcostofamergejoin(thejoinstrategyforthiscouldbeNestedLooporMerge)
EST_CARDINALITY representstheestimatedcardinality(amountofrows)producedbythisnode
EST_COL_STATS columnstatisticsincludingnumberofnullvalues,distinctvaluecount,etc.
EST_SELECTIVITY representstheselectivityofacriterianode
Rules
Relationaloptimizationisbaseduponruleexecutionthatevolvestheinitialplanintotheexecutionplan.Thereareasetofpre-definedrulesthataredynamicallyassembledintoarulestackforeveryquery.Therulestackisassembledbasedonthecontentsoftheuser’squeryandtheviews/proceduresaccessed.Forexample,iftherearenoviewlayers,thenruleMergeVirtual,whichmergesviewlayerstogether,isnotneededandwillnotbeaddedtothestack.Thisallowstherulestacktoreflectthecomplexityofthequery.
Logicallytheplannodedatastructurerepresentsatreeofnodeswherethesourcedatacomesupfromtheleafnodes(typicallyAccessnodesinthefinalplan),flowsupthroughthetreeandproducestheuser’sresultsoutthetop.Thenodesintheplanstructurecanhavebidirectionallinks,dynamicproperties,andallowanynumberofchildnodes.Processingplansincontrasttypicallyhavefixedproperties.
Planrulemanipulatetheplantree,fireotherrules,anddrivetheoptimizationprocess.Eachruleisdesignedtoperformanarrowsetoftasks.Somerulescanberunmultipletimes.Somerulesrequireaspecificsetofprecursorstorunproperly.
AccessPatternValidation-ensuresthatallaccesspatternshavebeensatisfied
ApplySecurity-appliesrowandcolumnlevelsecurity
AssignOutputSymbol-thisrulewalkstopdownthrougheverynodeandcalculatestheoutputcolumnsforeachnode.Columnsthatarenotneededaredroppedateverynode,whichisknownasprojectionminimization.Thisisdonebykeepingtrackofboththecolumnsneededtofeedtheparentnodeandalsokeepingtrackofcolumnsthatare"created"atacertainnode.
CalculateCost-addscostinginformationtotheplan
ChooseDependent-thisrulelooksateachjoinnodeanddetermineswhetherthejoinshouldbemadedependentandinwhichdirection.Cardinality,thenumberofdistinctvalues,andprimarykeyinformationareusedinseveralformulastodeterminewhetheradependentjoinislikelytobeworthwhile.Thedependentjoindiffersinperformanceideallybecausea
QueryPlanner
640
fewernumberofvalueswillbereturnedfromthedependentside.Also,wemustconsiderthenumberofvaluespassedfromindependenttodependentside.IfthatsetislargerthanthemaxnumberofvaluesinanINcriteriaonthedependentside,thenwemustbreakthequeryintoasetofqueriesandcombinetheirresults.Executingeachqueryintheconnectorhassomeoverheadandthatistakenintoaccount.Withoutcostinginformationalotofcommoncaseswheretheonlycriteriaspecifiedisonanon-unique(butstronglylimiting)fieldaremissed.Ajoiniseligibletobedependentif:
thereisatleastoneequi-joincriterion,i.e.tablea.col=tableb.col
thejoinisnotafullouterjoinandthedependentsideofthejoinisontheinnersideofthejoin
Thejoinwillbemadedependentifoneofthefollowingconditions,listedinprecedenceorder,holds:
Thereisanunsatisfiedaccesspatternthatcanbesatisfiedwiththedependentjoincriteria
Thepotentialdependentsideofthejoinismarkedwithanoptionmakedep
(4.3.2)ifcostingwasenabled,theestimatedcostforthedependentjoin(5.0+possiblyineachdirectioninthecaseofinnerjoins)iscomputedandcomparedtonotperformingthedependentjoin.Ifthecostswerealldetermined(whichrequiresallrelevanttablecardinality,columnndv,andpossiblynnvvaluestobepopulated)thelowestischosen.
Ifkeymetadatainformationindicatesthatthepotentialdependentsideisnot"small"andtheothersideis"notsmall"or(5.0.1)thepotentialdependentsideistheinnersideofaleftouterjoin.
Dependentjoinisthekeyoptimizationweusetoefficientlyprocessmulti-sourcejoins.InsteadofreadingallofsourceAandallofsourceBandjoiningthemonA.x=B.x,wereadallofAthenbuildasetofA.xthatarepassedasacriteriawhenqueryingB.IncaseswhereAissmallandBislarge,thiscandrasticallyreducethedataretrievedfromB,thusgreatlyspeedingtheoverallquery.
ChooseJoinStrategy-choosethejoinstrategybaseduponthecostandattributesofthejoin.
CleanCriteria-removesphantomcriteria
CollapseSource-takesallofthenodesbelowanaccessnodeandcreatesaSQLqueryrepresentation
CopyCriteria-thisrulecopiescriteriaoveranequalitycriteriathatispresentinthecriteriaofajoin.Sincetheequalitydefinesanequivalence,thisisavalidwaytocreateanewcriteriathatmaylimitresultsontheothersideofthejoin(especiallyinthecaseofamulti-sourcejoin).
DecomposeJoin-thisruleperfomrsapartition-wisejoinoptimizationonjoinsofFederatedOptimizations#PartitionedUnion.Thedecisiontodecomposeisbasedupondetectingthateachsideofthejoinisapartitionedunion(notethatnon-ansijoinsofmorethan2tablesmaycausetheoptimizationtonotdetecttheappropriatejoin).Therulecurrentlyonlylooksforsituationswhereatmost1partitionmatchesfromeachside.
ImplementJoinStrategy-addsnecessarysortandothernodestoprocessthechosenjoinstrategy
MergeCriteria-combinesselectnodesandcanconvertsubqueriestosemi-joins
MergeVirtual-removesviewandinlineviewlayers
PlaceAccess-placesaccessnodesundersourcenodes.Anaccessnoderepresentsthepointatwhicheverythingbelowtheaccessnodegetspushedtothesourceorisaplaninvocation.Laterrulesfocusoneitherpushingundertheaccessorpullingtheaccessnodeupthetreetomovemoreworkdowntothesources.ThisruleisalsoresponsibleforplacingFederatedOptimizations#AccessPatterns.
PlanJoins-thisruleattemptstofindanoptimalorderingofthejoinsperformedintheplan,whileensuringthatFederatedOptimizations#AccessPatternsdependenciesaremet.Thisrulehasthreemainsteps.Firstitmustdetermineanorderingofjoinsthatsatisfytheaccesspatternspresent.Seconditwillheuristicallycreatejoinsthatcanbepushedtothesource(ifasetofjoinsarepushedtothesource,wewillnotattempttocreateanoptimalorderingwithinthatset.Morethanlikelyitwillbe
QueryPlanner
641
senttothesourceinthenon-ANSImulti-joinsyntaxandwillbeoptimizedbythedatabase).Thirditwillusecostinginformationtodeterminethebestleft-linearorderingofjoinsperformedintheprocessingengine.Thisthirdstepwilldoanexhaustivesearchfor7orlessjoinsourcesandisheuristicallydrivenbyjoinselectivityfor8ormoresources.
PlanOuterJoins-reordersouterjoinsaspermittedtoimprovepushdown.
PlanProcedures-plansproceduresthatappearinproceduralrelationalqueries
PlanSorts-optimizationsaroundsorting,suchascombiningsortoperationsormovingprojection
PlanUnions-reordersunionchildrenformorepushdown
PlanAggregates-performsaggregatedecompositionoverajoinorunion
PushLimit-pushestheaffectofalimitnodefurtherintotheplan
PushNon-JoinCriteria-thisrulewillpushpredicatesoutofanonclauseifitisnotnecessaryforthecorrectnessofthejoin.
PushSelectCriteria-pushedselectnodesasfaraspossiblethroughunions,joins,andviewslayerstowardtheaccessnodes.Inmostcasesmovementdownthetreeisgoodasthiswillfilterrowsearlierintheplan.WecurrentlydonotundothedecisionsmadebyPushSelectCriteria.Howeverinsituationswherecriteriacannotbeevaluatedbythesource,thiscanleadtosuboptimalplans.
Oneofthemostimportantoptimizationrelatedtopushingcriteria,ishowthecriteriawillbepushedthroughjoin.Considerthefollowingplantreethatrepresentsasubtreeoftheplanforthequeryselect*fromAinnerjoinbon(A.x=B.x)whereB.y=3
SELECT(B.y=3)
|
JOIN-InnerJoinon(A.x=B.x)
/\
SRC(A)SRC(B)
Note SELECTnodesrepresentcriteria,andSRCstandsforSOURCE.
Itisalwaysvalidforinnerjoinandcrossjoinstopush(singlesource)criteriathatareabovethejoin,belowthejoin.Thisallowsforcriteriaoriginatingintheuserquerytoeventuallybepresentinsourcequeriesbelowthejoins.Thisresultcanberepresentedvisuallyas:
JOIN-InnerJoinon(A.x=B.x)
/\
/SELECT(B.y=3)
||
SRC(A)SRC(B)
Thesameoptimizationisvalidforcriteriaspecifiedagainsttheoutersideofanouterjoin.Forexample:
SELECT(B.y=3)
|
JOIN-RightOuterJoinon(A.x=B.x)
/\
SRC(A)SRC(B)
Becomes
JOIN-RightOuterJoinon(A.x=B.x)
/\
/SELECT(B.y=3)
||
SRC(A)SRC(B)
QueryPlanner
642
Howevercriteriaspecifiedagainsttheinnersideofanouterjoinneedsspecialconsideration.Theabovescenariowithaleftorfullouterjoinisnotthesame.Forexample:
SELECT(B.y=3)
|
JOIN-LeftOuterJoinon(A.x=B.x)
/\
SRC(A)SRC(B)
Canbecome(availableonlyafter5.0.2):
JOIN-InnerJoinon(A.x=B.x)
/\
/SELECT(B.y=3)
||
SRC(A)SRC(B)
Sincethecriterionisnotdependentuponthenullvaluesthatmaybepopulatedfromtheinnersideofthejoin,thecriterioniseligibletobepushedbelowthejoin–butonlyifthejointypeisalsochangedtoaninnerjoin.Ontheotherhand,criteriathataredependentuponthepresenceofnullvaluesCANNOTbemoved.Forexample:
SELECT(B.yisnull)
|
JOIN-LeftOuterJoinon(A.x=B.x)
/\
SRC(A)SRC(B)
Thisplantreemusthavethecriteriaremainabovethejoin,sincetheouterjoinmaybeintroducingnullvaluesitself.
RaiseAccess-thisruleattemptstoraisetheAccessnodesasfaruptheplanaspossible.Thisismostlydonebylookingatthesource’scapabilitiesanddeterminingwhethertheoperationscanbeachievedinthesourceornot.
RaiseNull-raisesnullnodes.Raisinganullnoderemovestheneedtoconsideranypartoftheoldplanthatwasbelowthenullnode.
RemoveOptionalJoins-removesjoinsthataremarkedasordeterminedtobeoptional
SubstituteExpressions-usedonlywhenafunctionbasedindexispresent
ValidateWhereAll-ensurescriteriaisusedwhenrequiredbythesource
CostCalculations
Thecostofnodeoperationsisprimarilydeterminedbyanestimateofthenumberofrows(alsoreferredtoascardinality)thatwillbeprocessedbyit.Theoptimizerwilltypicallycomputecardinalitiesfromthebottomupoftheplan(orsubplan)atseveralpointsintimewithplanning-oncegenerallywithrulecalculatecost,andthenspecificallyforjoinplanningandotherdecisions.Thecostcalculationismainlydirectedbythestatisticssetonphysicaltables(cardinality,NNV,NDV,etc.)andisalsoinfluencedbythepresenceofconstraints(unique,primarykey,index,etc.).Ifthereisasituationthatseemslikeasub-optimalplanisbeingchosen,youshouldfirstensurethatatleastrepresentativetablecardinalitiesaresetonthephysicaltablesinvolved.
ReadingaDebugPlan
Aseachrelationalsubplanisoptimized,theplanwillshowwhatisbeingoptimizedandit’scanonicalform:
OPTIMIZE:
SELECTe1FROM(SELECTe1FROMpm1.g1)ASx
QueryPlanner
643
----------------------------------------------------------------------------
GENERATECANONICAL:
SELECTe1FROM(SELECTe1FROMpm1.g1)ASx
CANONICALPLAN:
Project(groups=[x],props={PROJECT_COLS=[e1]})
Source(groups=[x],props={NESTED_COMMAND=SELECTe1FROMpm1.g1,SYMBOL_MAP={x.e1=e1}})
Project(groups=[pm1.g1],props={PROJECT_COLS=[e1]})
Source(groups=[pm1.g1])
Withmorecomplicateduserqueries,suchasaprocedureinvocationoronecontainingsubqueries,thesubplansmaybenestedwithintheoverallplan.Eachplanendsbyshowingthefinalprocessingplan:
----------------------------------------------------------------------------
OPTIMIZATIONCOMPLETE:
PROCESSORPLAN:
AccessNode(0)output=[e1]SELECTg_0.e1FROMpm1.g1ASg_0
Theaffectofrulescanbeseenbythestateoftheplantreebeforeandaftertherulefires.Forexample,thedebuglogbelowshowstheapplicationofrulemergevirtual,whichwillremovethe"x"inlineviewlayer:
EXECUTINGAssignOutputElements
AFTER:
Project(groups=[x],props={PROJECT_COLS=[e1],OUTPUT_COLS=[e1]})
Source(groups=[x],props={NESTED_COMMAND=SELECTe1FROMpm1.g1,SYMBOL_MAP={x.e1=e1},OUTPUT_COLS=[e1]})
Project(groups=[pm1.g1],props={PROJECT_COLS=[e1],OUTPUT_COLS=[e1]})
Access(groups=[pm1.g1],props={SOURCE_HINT=null,MODEL_ID=Schemaname=pm1,nameInSource=null,uuid=3335,
OUTPUT_COLS=[e1]})
Source(groups=[pm1.g1],props={OUTPUT_COLS=[e1]})
============================================================================
EXECUTINGMergeVirtual
AFTER:
Project(groups=[pm1.g1],props={PROJECT_COLS=[e1],OUTPUT_COLS=[e1]})
Access(groups=[pm1.g1],props={SOURCE_HINT=null,MODEL_ID=Schemaname=pm1,nameInSource=null,uuid=3335,OUTP
UT_COLS=[e1]})
Source(groups=[pm1.g1])
Someimportantplanningdecisionsareshownintheplanastheyoccurasanannotation.Forexamplethesnippetbelowshowsthattheaccessnodecouldnotberaisedastheparentselectnodecontainedanunsupportedsubquery.
Project(groups=[pm1.g1],props={PROJECT_COLS=[e1],OUTPUT_COLS=null})
Select(groups=[pm1.g1],props={SELECT_CRITERIA=e1IN/*+NO_UNNEST*/(SELECTe1FROMpm2.g1),OUTPUT_COLS=nu
ll})
Access(groups=[pm1.g1],props={SOURCE_HINT=null,MODEL_ID=Schemaname=pm1,nameInSource=null,uuid=3341,OU
TPUT_COLS=null})
Source(groups=[pm1.g1],props={OUTPUT_COLS=null})
============================================================================
EXECUTINGRaiseAccess
LOWRelationalPlannerSubqueryInisnotsupportedbysourcepm1-e1IN/*+NO_UNNEST*/(SELECTe1FROMpm2.g
1)wasnotpushed
AFTER:
Project(groups=[pm1.g1])
Select(groups=[pm1.g1],props={SELECT_CRITERIA=e1IN/*+NO_UNNEST*/(SELECTe1FROMpm2.g1),OUTPUT_COLS=nu
ll})
Access(groups=[pm1.g1],props={SOURCE_HINT=null,MODEL_ID=Schemaname=pm1,nameInSource=null,uuid=3341,OU
QueryPlanner
644
TPUT_COLS=null})
Source(groups=[pm1.g1])
ProcedurePlanner
Theprocedureplannerisfairlysimple.Itconvertsthestatementsintheprocedureintoinstructionsinaprogramthatwillberunduringprocessing.Thisismostlya1-to-1mappingandverylittleoptimizationisperformed.
XMLPlanner
TheXMLPlannercreatesanXMLplanthatisrelativelyclosetotheendresultoftheProcedurePlanner–aprogramwithinstructions.Manyoftheinstructionsareevensimilar(whileloop,executeSQL,etc).Additionalinstructionsdealwithproducingtheoutputresultdocument(addingelementsandattributes).
TheXMLplannerdoesseveraltypesofplanning(notnecessarilyinthisorder):
Documentselection-determinewhichtagsofthevirtualdocumentshouldbeexcludedfromtheoutputdocument.Thisisdonebasedonacombinationofthemodel(whichmarkspartsofthedocumentexcluded)andthequery(whichmayspecifyasubsetofcolumnstoincludeintheSELECTclause).
Criteriaevaluation-breaksaparttheuser’scriteria,determinewhichresultsetthecriteriashouldbeappliedto,andaddthatcriteriatothatresultsetquery.
Resultsetordering-thequery’sORDERBYclauseisbrokenupandtheORDERBYisappliedtoeachresultsetasnecessary
Resultsetplanning-ultimately,eachresultsetisplannedusingtherelationalplannerandtakingintoaccountalltheimpactsfromtheuser’squery.Theplannerwillalsolooktoautomaticallycreatestagingtablesanddependentjoinsbaseduponthemappingclasshierarchy.
Programgeneration-asetofinstructionstoproducethedesiredoutputdocumentisproduced,takingintoaccountthefinalresultsetqueriesandtheexcludedpartsofthedocument.Generally,thisinvolveswalkingthroughthevirtualdocumentindocumentorder,executingqueriesasnecessaryandemittingelementsandattributes.
XMLprogramscanalsoberecursive,whichinvolvesusingthesamedocumentfragmentforboththeinitialfragmentandasetofrepeatedfragments(eachanewquery)untilsometerminationcriteriaorlimitismet.
XQuery
XQueryiseligibleforspecificoptimizations.Documentprojectionisthemostcommonoptimization.Itwillbeshowninthedebugplanasanannotation.Forexamplewiththeuserquerycontaining"xmltable('/a/b'passingdoccolumnsxstringpath'@x',valstringpath'/.')",thedebugplanwouldshowatreeofthedocumentthatwilleffectivelybeusedbythecontextandpathXQuerys:
MEDIUMXQueryPlanningProjectionconditionsmetfor/a/b-Documentprojectionwillbeused
childelement(Q{}a)
childelement(Q{}b)
attributeattribute(Q{}x)
childtext()
childtext()
QueryPlanner
645
QueryPlansWhenintegratinginformationusingafederatedqueryplanneritisusefultoviewthequeryplanstobetterunderstandhowinformationisbeingaccessedandprocessed,andtotroubleshootproblems.
Aqueryplan(alsoknownasanexecutionorprocessingplan)isasetofinstructionscreatedbyaqueryengineforexecutingacommandsubmittedbyauserorapplication.Thepurposeofthequeryplanistoexecutetheuser’squeryinasefficientawayaspossible.
GettingaQueryPlan
Youcangetaqueryplananytimeyouexecuteacommand.TheSQLoptionsavailableareasfollows:
SETSHOWPLAN[ON|DEBUG]-ReturnstheprocessingplanortheplanandthefullplannerDebugLog.SeealsotheSETStatement.
Withtheaboveoptions,thequeryplanisavailablefromtheStatementobjectbycastingtotheorg.teiid.jdbc.TeiidStatementinterfaceorbyusingtheSHOWPLANstatement.
RetrievingaQueryPlanUsingTeiidExtensions
statement.execute("setshowplanon");
ResultSetrs=statement.executeQuery("select...");
TeiidStatementtstatement=statement.unwrap(TeiidStatement.class);
PlanNodequeryPlan=tstatement.getPlanDescription();
System.out.println(queryPlan);
RetrievingaQueryPlanUsingStatements
statement.execute("setshowplanon");
ResultSetrs=statement.executeQuery("select...");
...
ResultSetplanRs=statement.executeQuery("showplan");
planRs.next();
System.out.println(planRs.getString("PLAN_XML"));
ThequeryplanismadeavailableautomaticallyinseveralofTeiid’stools.
AnalyzingaQueryPlan
Onceaqueryplanhasbeenobtainedyouwillmostcommonlybelookingfor:
Sourcepushdown–whatpartsofthequerythatgotpushedtoeachsource
Ensurethatanypredicatesespeciallyagainstindexesarepushed
Joins-asfederatedjoinscanbequiteexpensive
Joinordering-typicallyinfluencedbycosting
Joincriteriatypemismatches.
Joinalgorithmused-merge,enhancedmerge,nestedloop,etc.
Presenceoffederatedoptimizations,suchasdependentjoins.
QueryPlans
646
Ensurehintshavethedesiredaffects-seeHintsandOptions,hintsintheFROMClause,SubqueryOptimization,andFederatedOptimizations.
Alloftheaboveinformationcanbedeterminedfromtheprocessingplan.Youwilltypicallybeinterestedinanalyzingthetextualformofthefinalprocessingplan.Tounderstandwhyparticulardecisionsaremadefordebuggingorsupportyouwillwanttoobtainthefulldebuglogwhichwillcontaintheintermediateplanningstepsaswellasannotationsastowhyspecificpushdowndecisionsaremade.
Aqueryplanconsistsofasetofnodesorganizedinatreestructure.IfyouareexecutingaprocedureorgeneratinganXMLdocumentfromanXMLDocumentModel,theoverallqueryplanwillcontainadditionalinformationrelatedthesurroundingproceduralexecution.
Inaproceduralcontexttheorderingofchildnodesimpliestheorderofexecution.Inmostothersituation,childnodesmaybeexecutedinanyordereveninparallel.Onlyinspecificoptimizations,suchasdependentjoin,willthechildrenofajoinexecuteserially.
RelationalQueryPlans
Relationalplansrepresenttheprocessingplanthatiscomposedofnodesrepresentingbuildingblocksoflogicalrelationaloperations.Relationalprocessingplansdifferfromlogicaldebugrelationalplansinthattheywillcontainadditionaloperationsandexecutionspecificsthatwerechosenbytheoptimizer.
Thenodesforarelationalqueryplanare:
Access-Accessasource.Asourcequeryissenttotheconnectionfactoryassociatedwiththesource.(Foradependentjoin,thisnodeiscalledDependentAccess.)
DependentProcedureAccess-Accessastoredprocedureonasourceusingmultiplesetsofinputvalues.
BatchedUpdate-Processesasetofupdatesasabatch.
Project-Definesthecolumnsreturnedfromthenode.Thisdoesnotalterthenumberofrecordsreturned.
ProjectInto-Likeanormalproject,butoutputsrowsintoatargettable.
InsertPlanExecution-Similartoaprojectinto,butexecutesaplanratherthanasourcequery.Typicallycreatedwhenexecutinganinsertintoviewwithaqueryexpression.
WindowFunctionProject-Likeanormalproject,butincludeswindowfunctions.
Select-Selectisacriteriaevaluationfilternode(WHERE/HAVING).
Join-Definesthejointype,joincriteria,andjoinstrategy(mergeornestedloop).
UnionAll-Therearenopropertiesforthisnode,itjustpassesrowsthroughfromit’schildren.Dependinguponotherfactors,suchasifthereisatransactionorthesourcequeryconcurrencyallowed,notalloftheunionchildrenwillexecuteinparallel.
Sort-Definesthecolumnstosorton,thesortdirectionforeachcolumn,andwhethertoremoveduplicatesornot.
DupRemove-Removesduplicaterows.TheprocessingusesatreestructuretodetectduplicatessothatresultswilleffectivelystreamatthecostofIOoperations.
Grouping-Groupssetsofrowsintogroupsandevaluatesaggregatefunctions.
Null-Anodethatproducesnorows.UsuallyreplacesaSelectnodewherethecriteriaisalwaysfalse(andwhatevertreeisunderneath).Therearenopropertiesforthisnode.
PlanExecution-Executesanothersubplan.Typicallythesubplanwillbeanon-relationalplan.
QueryPlans
647
DependentProcedureExecution-Executesasubplanusingmultiplesetsofinputvalues.
Limit-Returnsaspecifiednumberofrows,thenstopsprocessing.Alsoprocessesanoffsetifpresent.
XMLTable-EvaluatesXMLTABLE.ThedebugplanwillcontainmoreinformationabouttheXQuery/XPathwithregardstotheiroptimization-seetheXQuerysectionbeloworXQueryOptimization.
TextTable-EvaluatesTEXTTABLE
ArrayTable-EvaluatesARRAYTABLE
ObjectTable-EvaluatesOBJECTTABLE
NodeStatistics
Everynodehasasetofstatisticsthatareoutput.Thesecanbeusedtodeterminetheamountofdataflowingthroughthenode.Beforeexecutionaprocessorplanwillnotcontainnodestatistics.Alsothestatisticsareupdatedastheplanisprocessed,sotypicallyyou’llwantthefinalstatisticsafterallrowshavebeenprocessedbytheclient.
Statistic Description Units
NodeOutputRows Numberofrecordsoutputfromthenode count
NodeNextBatchProcessTime Timeprocessinginthisnodeonly millisec
NodeCumulativeNextBatchProcessTime
Timeprocessinginthisnode+childnodes millisec
NodeCumulativeProcessTime Elapsedtimefrombeginningofprocessingtoend millisec
NodeNextBatchCalls Numberoftimesanodewascalledforprocessing count
NodeBlocks Numberoftimesablockedexceptionwasthrownbythisnodeorachild count
Inadditiontonodestatistics,somenodesdisplaycostestimatescomputedatthenode.
CostEstimates Description Units
EstimatedNodeCardinalityEstimatednumberofrecordsthatwillbeoutputfromthenode;-1ifunknown
count
Therootnodewilldisplayadditionalinformation.
ToplevelStatistics Description Units
DataBytesSent Thesizeoftheserializeddataresult(rowandlobvalues)senttotheclient bytes
ReadingaProcessorPlan
QueryPlans
648
Thequeryprocessorplancanbeobtainedinaplaintextorxmlformat.Theplantextformatistypicallyeasiertoread,whilethexmlformatiseasiertoprocessbytooling.Whenpossibletoolingshouldbeusedtoexaminetheplansasthetreestructurescanbedeeplynested.
Dataflowsfromtheleafsofthetreetotheroot.Subplansforprocedureexecutioncanbeshowninline,andaredifferentiatedbydifferentindentation.GivenauserqueryofSELECTpm1.g1.e1,pm1.g2.e2,pm1.g3.e3frompm1.g1innerjoin(pm1.g2leftouterjoinpm1.g3onpm1.g2.e1=pm1.g3.e1)onpm1.g1.e1=pm1.g3.e1,thetextforaprocessorplanthatdoesnotpushdownthejoinswouldlooklike:
ProjectNode
+OutputColumns:
0:e1(string)
1:e2(integer)
2:e3(boolean)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Child0:
JoinNode
+OutputColumns:
0:e1(string)
1:e2(integer)
2:e3(boolean)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Child0:
JoinNode
+OutputColumns:
0:e1(string)
1:e1(string)
2:e3(boolean)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Child0:
AccessNode
+OutputColumns:e1(string)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Query:SELECTg_0.e1ASc_0FROMpm1.g1ASg_0ORDERBYc_0
+ModelName:pm1
+Child1:
AccessNode
+OutputColumns:
0:e1(string)
1:e3(boolean)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Query:SELECTg_0.e1ASc_0,g_0.e3ASc_1FROMpm1.g3ASg_0ORDERBYc_0
+ModelName:pm1
+JoinStrategy:MERGEJOIN(ALREADY_SORTED/ALREADY_SORTED)
+JoinType:INNERJOIN
+JoinCriteria:pm1.g1.e1=pm1.g3.e1
+Child1:
AccessNode
+OutputColumns:
0:e1(string)
1:e2(integer)
+CostEstimates:EstimatedNodeCardinality:-1.0
+Query:SELECTg_0.e1ASc_0,g_0.e2ASc_1FROMpm1.g2ASg_0ORDERBYc_0
+ModelName:pm1
+JoinStrategy:ENHANCEDSORTJOIN(SORT/ALREADY_SORTED)
+JoinType:INNERJOIN
+JoinCriteria:pm1.g3.e1=pm1.g2.e1
+SelectColumns:
0:pm1.g1.e1
1:pm1.g2.e2
2:pm1.g3.e3
QueryPlans
649
Notethatthenestedjoinnodeisusingamergejoinandexpectsthesourcequeriesfromeachsidetoproducetheexpectedorderingforthejoin.Theparentjoinisanenhancedsortjoinwhichcandelaythedecisiontoperformsortingbasedupontheincomingrows.Notethattheouterjoinfromtheuserqueryhasbeenmodifiedtoaninnerjoinsincenoneofthenullinnervaluescanbepresentinthequeryresult.
Thesameplaninxmlformlookslike:
<?xmlversion="1.0"encoding="UTF-8"?>
<nodename="ProjectNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
<value>e2(integer)</value>
<value>e3(boolean)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Child0">
<nodename="JoinNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
<value>e2(integer)</value>
<value>e3(boolean)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Child0">
<nodename="JoinNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
<value>e1(string)</value>
<value>e3(boolean)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Child0">
<nodename="AccessNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Query">
<value>SELECTg_0.e1ASc_0FROMpm1.g1ASg_0ORDERBYc_0</value>
</property>
<propertyname="ModelName">
<value>pm1</value>
</property>
</node>
</property>
<propertyname="Child1">
<nodename="AccessNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
<value>e3(boolean)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Query">
<value>SELECTg_0.e1ASc_0,g_0.e3ASc_1FROMpm1.g3ASg_0
ORDERBYc_0</value>
</property>
<propertyname="ModelName">
QueryPlans
650
<value>pm1</value>
</property>
</node>
</property>
<propertyname="JoinStrategy">
<value>MERGEJOIN(ALREADY_SORTED/ALREADY_SORTED)</value>
</property>
<propertyname="JoinType">
<value>INNERJOIN</value>
</property>
<propertyname="JoinCriteria">
<value>pm1.g1.e1=pm1.g3.e1</value>
</property>
</node>
</property>
<propertyname="Child1">
<nodename="AccessNode">
<propertyname="OutputColumns">
<value>e1(string)</value>
<value>e2(integer)</value>
</property>
<propertyname="CostEstimates">
<value>EstimatedNodeCardinality:-1.0</value>
</property>
<propertyname="Query">
<value>SELECTg_0.e1ASc_0,g_0.e2ASc_1FROMpm1.g2ASg_0
ORDERBYc_0</value>
</property>
<propertyname="ModelName">
<value>pm1</value>
</property>
</node>
</property>
<propertyname="JoinStrategy">
<value>ENHANCEDSORTJOIN(SORT/ALREADY_SORTED)</value>
</property>
<propertyname="JoinType">
<value>INNERJOIN</value>
</property>
<propertyname="JoinCriteria">
<value>pm1.g3.e1=pm1.g2.e1</value>
</property>
</node>
</property>
<propertyname="SelectColumns">
<value>pm1.g1.e1</value>
<value>pm1.g2.e2</value>
<value>pm1.g3.e3</value>
</property>
</node>
Notethatthesameinformationappearsineachoftheplanforms.Insomecasesitcanactuallybeeasiertofollowthesimplifiedformatofthedebugplanfinalprocessorplan.FromtheDebugLogthesameplanasabovewouldappearas:
OPTIMIZATIONCOMPLETE:
PROCESSORPLAN:
ProjectNode(0)output=[pm1.g1.e1,pm1.g2.e2,pm1.g3.e3][pm1.g1.e1,pm1.g2.e2,pm1.g3.e3]
JoinNode(1)[ENHANCEDSORTJOIN(SORT/ALREADY_SORTED)][INNERJOIN]criteria=[pm1.g3.e1=pm1.g2.e1]output=[pm
1.g1.e1,pm1.g2.e2,pm1.g3.e3]
JoinNode(2)[MERGEJOIN(ALREADY_SORTED/ALREADY_SORTED)][INNERJOIN]criteria=[pm1.g1.e1=pm1.g3.e1]output
=[pm1.g3.e1,pm1.g1.e1,pm1.g3.e3]
AccessNode(3)output=[pm1.g1.e1]SELECTg_0.e1ASc_0FROMpm1.g1ASg_0ORDERBYc_0
AccessNode(4)output=[pm1.g3.e1,pm1.g3.e3]SELECTg_0.e1ASc_0,g_0.e3ASc_1FROMpm1.g3ASg_0ORDER
BYc_0
AccessNode(5)output=[pm1.g2.e1,pm1.g2.e2]SELECTg_0.e1ASc_0,g_0.e2ASc_1FROMpm1.g2ASg_0ORDERBY
c_0
QueryPlans
651
NodeProperties
Common
OutputColumns-whatcolumnsmakeupthetuplesreturnedbythisnode
DataBytesSent-howmanydatabyte,notincludingmessagingoverhead,weresentbythisquery
PlanningTime-theamountoftimeinmillisecondsspentplanningthequery
Relational
RelationalNodeID-matchesthenodeidsseeninthedebuglogNode(id)
Criteria-thebooleanexpressionusedforfiltering
SelectColumns-thecolumnsthatdefinetheprojection
GroupingColumns-thecolumnsusedforgrouping
GroupingMapping-showsthemappingofaggregateandgroupingcolumninternalnamestotheirexpressionform
Query-thesourcequery
ModelName-themodelname
SharingID-nodessharingthesamesourceresultswillhavethesamesharingid
DependentJoin-ifadependentjoinisbeingused
JoinStrategy-thejoinstrategy(NestedLoop,SortMerge,EnhancedSort,etc.)
JoinType-thejointype(LeftOuterJoin,InnerJoin,CrossJoin)
JoinCriteria-thejoinpredicates
ExecutionPlan-thenestedexecutionplan
IntoTarget-theinsertiontarget
Upsert-iftheinsertisanupsert
SortColumns-thecolumnsforsorting
SortMode-ifthesortperformsanotherfunctionaswell,suchasdistinctremoval
Rollup-ifthegroupbyhastherollupoption
Statistics-theprocessingstatistics
CostEstimates-thecost/cardinalityestimatesincludingdependentjoincostestimates
RowOffset-therowoffsetexpression
RowLimit-therowlimitexpression
With-thewithclause
WindowFunctions-thewindowfunctionsbeingcomputed
TableFunction-thetablefunction(XMLTABLE,OBJECTTABLE,TEXTTABLE,etc.)
Streaming-iftheXMLTABLEisusingstreamprocessing
QueryPlans
652
XMLPlan
Message
Tag
Namespace
DataColumn
NamespaceDeclarations
OptionalFlag
DefaultValue
RecursionDirection
Bindings
IsStagingFlag
SourceInMemoryFlag
Condition
DefaultProgram
Encoding
FormattedFlag
Procedure
Expression
ResultSet
Program
Variable
Then
Else
OtherPlans
XMLdocumentmodelqueriesandprocedureexecution(includinginsteadoftriggers)useintermediateandfinalplanformsthatincluderelationalplans.Generallythestructureofthexml/procedureplanswillcloselymatchtheirlogicalforms.It’sthenestedrelationalplansthatwillbeofinterestwhenanalyzingperformanceissues.
QueryPlans
653
FederatedOptimizations
AccessPatterns
Accesspatternsareusedonbothphysicaltablesandviewstospecifytheneedforcriteriaagainstasetofcolumns.Failuretosupplythecriteriawillresultinaplanningerror,ratherthanarun-awaysourcequery.Accesspatternscanbeappliedinasetsuchthatonlyoneoftheaccesspatternsisrequiredtobesatisfied.
Currentlyanyformofcriteriareferencinganaffectedcolumnmaysatisfyanaccesspattern.
PushdownInfederateddatabasesystemspushdownreferstodecomposingtheuserlevelqueryintosourcequeriesthatperformasmuchworkaspossibleontheirrespectivesourcesystem.Pushdownanalysisrequiresknowledgeofsourcesystemcapabilities,whichisprovidedtoTeiidthoughtheConnectorAPI.AnyworknotperformedatthesourceisthenprocessedinFederate’srelationalengine.
Baseduponcapabilities,Teiidwillmanipulatethequeryplantoensurethateachsourceperformsasmuchjoining,filtering,grouping,etc.aspossible.Inmaycases,suchaswithjoinordering,planningisacombinationofStandardRelationalTechniquesand,costbasedandheuristicsforpushdownoptimization.
Criteriaandjoinpushdownaretypicallythemostimportantaspectsofthequerytopushdownwhenperformanceisaconcern.SeeQueryPlansonhowtoreadaplantoensurethatsourcequeriesareasefficientaspossible.
DependentJoins
Aspecialoptimizationcalledadependentjoinisusedtoreducetherowsreturnedfromoneofthetworelationsinvolvedinamulti-sourcejoin.Inadependentjoin,queriesareissuedtoeachsourcesequentiallyratherthaninparallel,withtheresultsobtainedfromthefirstsourceusedtorestricttherecordsreturnedfromthesecond.Dependentjoinscanperformsomejoinsmuchfasterbydrasticallyreducingtheamountofdataretrievedfromthesecondsourceandthenumberofjoincomparisonsthatmustbeperformed.
TheconditionswhenadependentjoinisusedaredeterminedbythequeryplannerbasedonAccessPatterns,hints,andcostinginformation.TherearethreedifferentkindsofdependentjoinsthatTeiidsupports:
Joinbasedonin/equalitysupport-wheretheenginewilldeterminehowtobreakuplargequeriesbasedupontranslatorcapabilities
KeyPushdown-wherethetranslatorhasaccesstothefullsetofkeyvaluesanddetermineswhatqueriestosend
FullPushdown-wheretranslatorshipsthealldatafromtheindependentsidetothetranslator.Canbeusedautomaticallybycostingorcanbespecifiedasanoptioninthehint.
Teiidsupportshintstocontroldependentjoinbehavior:
MAKEIND-indicatesthattheclauseshouldbetheindependentsideofadependentjoin.
MAKEDEP-indicatesthattheclauseshouldbethedependentsideofajoin.MAKEDEPasanon-commenthintsupportsoptionalmaxandjoinarguments-MAKEDEP(JOIN)meaningthattheentirejoinshouldbepushed,andMAKEDEP(MAX:5000)meaningthatthedependentjoinshouldonlybeperformediftherearelessthanthemaxnumberofvaluesfromtheindependentside.
FederatedOptimizations
654
MAKENOTDEP-preventstheclausefrombeingthedependentsideofajoin.
ThesescanbeplacedineithertheOPTIONClauseordirectlyintheFROMClause.AslongasallAccessPatternscanbemet,theMAKEIND,MAKEDEP,andMAKENOTDEPhintsoverrideanyuseofcostinginformation.MAKENOTDEPsupersedestheotherhints.
Tip TheMAKEDEP/MAKEINDhintshouldonlybeusediftheproperqueryplanisnotchosenbydefault.Youshouldensurethatyourcostinginformationisrepresentativeoftheactualsourcecardinality.
Note AninappropriateMAKEDEP/MAKEINDhintcanforceaninefficientjoinstructureandmayresultinmanysourcequeries.
TipWhilethesehintscanbeappliedtoviews,theoptimizerwillbydefaultremoveviewswhenpossible.Thiscanresultinthehintplacementbeingsignificantlydifferentthantheoriginalintention.YoushouldconsiderusingtheNO_UNNESThinttopreventtheoptimizerfromremovingtheviewinthesecases.
InthesimplestscenariotheenginewilluseINclauses(orjustequalitypredicates)tofilterthevaluescomingfromthedependentside.IfthenumberofvaluesfromtheindependentsideexceedsthetranslatorsMaxInCriteriaSize,thevalueswillbesplitintomultipleINpredicatesuptoMaxDependentPredicates.WhenthenumberofindependentvaluesexceedsMaxInCriteriaSize*MaxDependentPredicates,thenmultipledependentquerieswillbeissuedinparallel.
IfthetranslatorreturnstrueforsupportsDependentJoins,thentheenginemayprovidetheentiresetofindependentkeyvalues.ThiswilloccurwhenthenumberofindependentvaluesexceedsMaxInCriteriaSize*MaxDependentPredicatessothatthetranslatormayusespecificlogictoavoidissuingmultiplequeriesaswouldhappeninthesimplescenario.
IfthetranslatorreturnstrueforbothsupportsDependentJoinsandsupportsFullDependentJoinsthenafullpushdownmaybechosenbytheoptimizerAfullpushdown,sometimesalsocalledasdata-shippushdown,iswhereallthedatafromindependentsideofthejoinissenttodependentside.Thishasanaddedbenefitofallowingtheplanabovethejointobeeligibleforpushdownaswell.ThisiswhytheoptimizermaychoosetoperformafullpushdownevenwhenthenumberofindependentvaluesdoesnotexceedMaxInCriteriaSize*MaxDependentPredicates.YoumayalsoforcefullpushdownusingtheMAKEDEP(JOIN)hint.Thetranslatoristypicallyresponsibleforcreating,populating,andremovingatemporarytablethatrepresentstheindependentside.IfyouareworkingwithcustomtranslatorsseeDependentJoinPushdownastohowtosupportitkeyandfullpushdown.
Note
Key/FullPushdowniscurrentlyonlysupportedout-of-theboxbyasubsetofJDBCtranslators.Toenablesupport,setthetranslatoroverrideproperty"enableDependentJoins"to"true".TheJDBCsourcemustsupportthecreationoftemporarytables,whichtypicallyrequiresaHibernatedialect.Translatorsthatshouldsupportthisfeatureinclude:DB2,Derby,H2,Hana,HSQL,MySQL,Oracle,PostgreSQL,SQLServer,Sybase,Teiid,andTeradata.
CopyCriteria
Copycriteriaisanoptimizationthatcreatesadditionalpredicatesbaseduponcombiningjoinandwhereclausecriteria.Forexample,equi-joinpredicates(source1.table.column=source2.table.column)areusedtocreatenewpredicatesbysubstitutingsource1.table.columnforsource2.table.columnandviceversa.Inacrosssourcescenario,thisallowsforwherecriteriaappliedtoasinglesideofthejointobeappliedtobothsourcequeries
ProjectionMinimization
Teiidensuresthateachpushdownqueryonlyprojectsthesymbolsrequiredforprocessingtheuserquery.Thisisespeciallyhelpfulwhenqueryingthroughlargeintermediateviewlayers.
PartialAggregatePushdown
FederatedOptimizations
655
Partialaggregatepushdownallowsforgroupingoperationsabovemulti-sourcejoinsandunionstobedecomposedsothatsomeofthegroupingandaggregatefunctionsmaybepusheddowntothesources.
OptionalJoin
Theoptionaljoinhintindicatestotheoptimizerthatajoinedtableshouldbeomittedifnoneofitscolumnsareusedbytheoutputoftheuserqueryorinameaningfulwaytoconstructtheresultsoftheuserquery.Thishintistypicallyonlyusedinviewlayerscontainingmulti-sourcejoins.
Theoptionaljoinhintisappliedasacommentonajoinclause.ItcanbeappliedinbothANSIandnon-ANSIjoins.Withnon-ANSIjoinsanentirejoinedtablemaybemarkedasoptional.
ExampleOptionalJoinHint
selecta.column1,b.column2froma,/*+optional*/bWHEREa.key=b.key
SupposethisexampledefinesaviewlayerX.IfXisqueriedinsuchawayastonotneedb.column2,thentheoptionaljoinhintwillcausebtobeomittedfromthequeryplan.TheresultwouldbethesameasifXweredefinedas:
ExampleOptionalJoinHint
selecta.column1froma
ExampleANSIOptionalJoinHint
selecta.column1,b.column2,c.column3from/*+optional*/(ainnerjoinbONa.key=b.key)INNERJOINcONa.
key=c.key
InthisexampletheANSIjoinsyntaxallowsforthejoinofaandbtobemarkedasoptional.SupposethisexampledefinesaviewlayerX.Onlyifbothcolumna.column1andb.column2arenotneeded,e.g."SELECTcolumn3FROMX"willthejoinberemoved.
Theoptionaljoinhintwillnotremoveabridgingtablethatisstillrequired.
ExampleBridgingTable
selecta.column1,b.column2,c.column3from/*+optional*/a,b,cWHEREONa.key=b.keyANDa.key=c.key
SupposethisexampledefinesaviewlayerX.Ifb.column2orc.column3aresolelyrequiredbyaquerytoX,thenthejoinonaberemoved.Howeverifa.column1orbothb.column2andc.column3areneeded,thentheoptionaljoinhintwillnottakeeffect.
Whenajoinclauseisomittedviatheoptionaljoinhint,therelevantcriteriaisnotapplied.Thusitispossiblethatthequeryresultsmaynothavethesamecardinalityoreventhesamerowvaluesaswhenthejoinisfullyapplied.
Left/rightouterjoinswheretheinnersidevaluesarenotusedandwhoserowsundergoadistinctoperationwillautomaticallybetreatedasanoptionaljoinanddonotrequireahint.
ExampleUnnecessaryOptionalJoinHint
selectdistincta.column1fromaLEFTOUTERJOIN/*+optional*/bONa.key=b.key
Note Asimple"SELECTCOUNT(*)FROMVIEW"againstaviewwherealljointablesaremarkedasoptionalwillnotreturnameaningfulresult.
SourceHints
FederatedOptimizations
656
Teiiduserandtransformationqueriescancontainametasourcehintthatcanprovideadditionalinformationtosourcequeries.Thesourcehinthastheform:
/*+sh[[KEEPALIASES]:'arg']source-name[KEEPALIASES]:'arg1'...*/
Thesourcehintisexpectedtoappearafterthequery(SELECT,INSERT,UPDATE,DELETE)keyword.
Sourcehintsmayappearinanysubqueryorinviews.Allhintsapplicabletoagivensourcequerywillbecollectedandpusheddowntogetherasalist.Theorderofthehintsisnotguaranteed.
TheshargisoptionalandispassedtoallsourcequeriesviatheExecutionContext.getGeneralHintsmethod.Theadditionalargsshouldhaveasource-namethatmatchesthesourcenameassignedtothetranslatorintheVDB.Ifthesource-namematches,thehintvalueswillbesuppliedviatheExecutionContext.getSourceHintsmethod.SeetheDeveloper’sGuideformoreonusinganExecutionContext.
Eachoftheargvalueshastheformofastringliteral-itmustbesurroundedinsinglequotesandasinglequotecanbeescapedwithanothersinglequote.OnlytheOracletranslatordoesanythingwithsourcehintsbydefault.TheOracletranslatorwilluseboththesourcehintandthegeneralhint(inthatorder)ifavailabletoformanOraclehintenclosedin/*+…*/.
IftheKEEPALIASESoptionisusedeitherforthegeneralhintorontheapplicablesourcespecifichint,thenthetable/viewaliasesfromtheuserqueryandanynestedviewswillbepreservedinthepush-downquery.Thisisusefulinsituationswherethesourcehintmayneedtoreferencealiasesandtheuserdoesnotwishtorelyonthegeneratedaliases(whichcanbeseeninthequeryplanintherelevantsourcequeries-seeabove).Howeverinsomesituationsthismayresultinaninvalidsourcequeryifthepreservedaliasnamesarenotvalidforthesourceorresultinanamecollision.IftheusageofKEEPALIASESresultsinanerror,thequerycouldbemodifiedbypreventingviewremovalwiththeNO_UNNESThint,thealiasesmodified,ortheKEEPALIASESoptioncouldberemovedandthequeryplanusedtodeterminethegeneratedaliasnames.
SampleSourceHints
SELECT/*+sh:'generalhint'*/...
SELECT/*+shKEEPALIASES:'generalhint'my-oracle:'oraclehint'*/...
PartitionedUnion
Unionpartitioningisinferredfromthetransformation/inlineview.Ifone(ormore)oftheUNIONcolumnsisdefinedbyconstantsand/orhasWHEREclauseINpredicatescontainingonlyconstantsthatmakeeachbranchmutuallyexclusive,thentheUNIONisconsideredpartitioned.UNIONALLmustbeusedandtheUNIONcannothaveaLIMIT,WITH,orORDERBYclause(althoughindividualbranchesmayuseLIMIT,WITH,orORDERBY).Partitioningvaluesshouldnotbenull.Forexampletheviewdefinition"select1asx,yfromfoounionallselectz,afromfoo1wherezin(2,3)"wouldbeconsideredpartitionedoncolumnx,sincethefirstbranchcanonlybethevalue1andthesecondbranchcanonlybethevalues2or3.Notethatmoreadvancedorexplicitpartitioncouldbeconsideredinthefuture.Theconceptofapartitionedunionisusedforperformingpartition-wisejoins,inUpdatableViews,andPartialAggregatePushdown.
StandardRelationalTechniques
Teiidalsoincorporatesmanystandardrelationaltechniquestoensureefficientqueryplans.
Rewriteanalysisforfunctionsimplificationandevaluation.
Booleanoptimizationsforbasiccriteriasimplification.
Removalofunnecessaryviewlayers.
FederatedOptimizations
657
Removalofunnecessarysortoperations.
Advancedsearchtechniquesthroughtheleft-linearspaceofjointrees.
Parallelizingofsourceaccessduringexecution.
SubqueryOptimization
JoinCompensation
Somesourcesystemsonlyallow"relationship"querieslogicallyproducingleftouterjoinresultsevenwhenqueriedwithaninnerjoinTeiidwillattempttoformanappropriateleftouterjoin.Thesesourcesarerestrictedtoonlysupportingkeyjoins.Insomecircumstancesforeignkeyrelationshipsonthesamesourceshouldnotbetraversedatallorwiththereferencedtableontheoutersideofjoin.Theextensionpropertyteiid_rel:allow-joincanbeusedontheforeignkeytofurtherrestrictthepushdownbehavior.Withavalueof"false"nojoinpushdownwillbeallowed,andwithavalueof"inner"thereferencedtablemustbeontheinnersideofthejoin.Ifthejoinpushdownisprevented,thejoinwillbeprocessedasafederatedjoin.
FederatedOptimizations
658
SubqueryOptimizationEXISTSsubqueriesaretypicallyrewriteto"SELECT1FROM…"topreventunnecessaryevaluationofSELECTexpressions.
QuantifiedcompareSOMEsubqueriesarealwaysturnedintoanequivalentINpredicateorcomparisonagainstanaggregatevalue.e.g.col>SOME(selectcol1fromtable)wouldbecomecol>(selectmin(col1)fromtable)
UncorrelatedEXISTsandscalarsubquerythatarenotpushedtothesourcecanbepre-evaluatedpriortosourcecommandformation.
CorrelatedsubqueriesusedinDETELEsorUPDATEsthatarenotpushedaspartofthecorrespondingDELETE/UPDATEwillcauseTeiidtoperformrow-by-rowcompensatingprocessing.Thiswillonlyhappeniftheaffectedtablehasaprimarykey.Ifitdoesnot,thenanexceptionwillbethrown.
WHEREorHAVINGclauseIN,QuantifiedComparison,ScalarSubqueryCompare,andEXISTspredicatescantaketheMJ(mergejoin),DJ(dependentjoin),orNO_UNNEST(nounnest)hintsappearingjustbeforethesubquery.TheMJhintdirectstheoptimizertouseatraditional,semijoin,orantisemijoinmergejoinifpossible.TheDJisthesameastheMJhint,butadditionallydirectstheoptimizertousethesubqueryastheindependentsideofadependentjoinifpossible.TheNO_UNNESThint,whichsupersedestheotherhints,willdirecttheoptimizertoleavethesubqueryinplace.
SELECTscalarsubqueriescantaketheMJ(mergejoin)orNO_UNNEST(nounnest)hintsappearingjustbeforethesubquery.TheMJhintdirectstheoptimizertouseatraditionalorsemijoinmergejoinifpossible.TheNO_UNNESThint,whichsupercedestheotherhints,willdirecttheoptimizertoleavethesubqueryinplace.
MergeJoinHintUsage
SELECTcol1fromtblwherecol2IN/*+MJ*/(SELECTcol1FROMtbl2)
DependentJoinHintUsage
SELECTcol1fromtblwherecol2IN/*+DJ*/(SELECTcol1FROMtbl2)
NoUnnestHintUsage
SELECTcol1fromtblwherecol2IN/*+NO_UNNEST*/(SELECTcol1FROMtbl2)
Thesystempropertyorg.teiid.subqueryUnnestDefaultcontrolswhethertheoptimizerwillbydefaultunnestsubqueriesduringrewrite.Iftrue,thenmostnon-negatedWHEREorHAVINGclauseEXISTSorINsubquerypredicatescanbeconvertedtoatraditionaljoin.
Theplannerwillalwaysconverttoantijoinorsemijoinvariantsifcostingisfavorable.Useahinttooverridethisbehaviorneeded.
EXISTsandscalarsubqueriesthatarenotpusheddown,andnotconvertedtomergejoins,areimplicitlylimitedto1and2resultrowsrespectively.
Conversionofsubquerypredicatestonestedloopjoinsisnotyetavailable.
SubqueryOptimization
659
XQueryOptimizationAtechniqueknownasdocumentprojectionisusedtoreducethememoryfootprintofthecontextitemdocument.DocumentprojectionloadsonlythepartsofthedocumentneededbytherelevantXQueryandpathexpressions.Sincedocumentprojectionanalysisusesallrelevantpathexpressions,even1expressionthatcouldpotentiallyusemanynodes,e.g.//xratherthan/a/b/xwillcausealargermemoryfootprint.Withtherelevantcontentremovedtheentiredocumentwillstillbeloadedintomemoryforprocessing.Documentprojectionwillonlybeusedwhenthereisacontextitem(unnamedPASSINGclauseitem)passedtoXMLTABLE/XMLQUERY.Anamedvariablewillnothavedocumentprojectionperformed.Insomecasestheexpressionsusedmaybetoocomplexfortheoptimizertousedocumentprojection.YoushouldchecktheSHOWPLANDEBUGfullplanoutputtoseeiftheappropriateoptimizationhasbeenperformed.
Withadditionalrestrictions,simplecontextpathexpressionsallowtheprocessortoevaluatedocumentsubtreesindependently-withoutloadingthefulldocumentinmemory.Asimplecontextpathexpressioncanbeoftheform"[/][ns:]root/[ns1:]elem/…",whereanamespaceprefixorelementnamecanalsobethe*wildcard.AswithnormalXQueryprocessingifnamespaceprefixesareusedintheXQueryexpression,theyshouldbedeclaredusingtheXMLNAMESPACESclause.
StreamingEligibleXMLQUERY
XMLQUERY('/*:root/*:child'PASSINGdoc)
Ratherthanloadingtheentiredocin-memoryasaDOMtree,eachchildelementwillbeindependentlyaddedtotheresult.
StreamingIneligibleXMLQUERY
XMLQUERY('//child'PASSINGdoc)
Theuseofthedescendantaxispreventsthestreamingoptimization,butdocumentprojectioncanstillbeperformed.
WhenusingXMLTABLE,theCOLUMNPATHexpressionshaveadditionalrestrictions.Theyareallowedtoreferenceanypartoftheelementsubtreeformedbythecontextexpressionandtheymayuseanyattributevaluefromtheirdirectparentage.Anypathexpressionwhereitispossibletoreferenceanon-directancestororsiblingofthecurrentcontextitempreventstreamingfrombeingused.
StreamingEligibleXMLTABLE
XMLTABLE('/*:root/*:child'PASSINGdocCOLUMNSfullchildXMLPATH'.',parent_attrstringPATH'../@attr',chil
d_valinteger)
ThecontextXQueryandthecolumnpathexpressionallowthestreamingoptimization,ratherthanloadingtheentiredocin-memoryasaDOMtree,eachchildelementwillbeindependentlyaddedtotheresult.
StreamingIneligibleXMLTABLE
XMLTABLE('/*:root/*:child'PASSINGdocCOLUMNSsibling_attrstringPATH'../other_child/@attr')
Thereferenceofanelementoutsideofthechildsubtreeinthesibling_attrpathpreventsthestreamingoptimizationfrombeingused,butdocumentprojectioncanstillbeperformed.
Note Columnpathsshouldbeastargetedaspossibletoavoidperformanceissues.Ageneralpathsuchas`..//child'willcausetheentiresubtreeofthecontextitemtobesearchedoneachoutputrow.
XQueryOptimization
660
XQueryOptimization
661
PartialResultsTeiidprovidesthecapabilitytoobtain"partialresults"intheeventofdatasourceunavailabilityorfailure.Thisisespeciallyusefulwhenunioninginformationfrommultiplesources,orwhendoingaleftouterjoin,whereyouare`appending'columnstoamasterrecordbutstillwanttherecordiftheextrainformationisnotavailable.
Asourceisconsideredtobe`unavailable'iftheconnectionfactoryassociatedwiththesourceissuesanexceptioninresponsetoaquery.Theexceptionwillbepropagatedtothequeryprocessor,whereitwillbecomeawarningonthestatement.SeetheClientGuideformoreonPartialResultsModeandSQLWarnings.
FederatedFailureModes
662
ConformedTablesAconformedtableisasourcetablethatisthesameinseveralphysicalsources.UnlikeMultisourceModelswhichassumeapartitioningparadigm,theplannerassumesanyconformedtablemaybesubstitutedforanothertoimproveperformance.Typicallythiswouldbeusedwhenreferencedataexistsinmultiplesources,butonlyasinglemetadataentryisdesiredtorepresentthetable.
Conformedtablesaredefinedbyaddingthe
{http://www.teiid.org/ext/relational/2012}conformed-sources
extensionmetadatapropertytotheappropriatesourcetables.ExtensionpropertiescanbesetatdesigntimeinDesigner,inthevdb.xmlwhenusingfullDDLMetadataoralterstatements,oratruntimeusingthesetPropertysystemprocedure.Thepropertyisexpectedtobeacommaseparatedlistofphysicalmodel/schemanames.
DDLAlterExample
ALTERFOREIGNTABLE"reference_data"OPTIONS(ADD"teiid_rel:conformed-sources"'source2,source3');
Thereisnoexpectationthatametadataentryexistsontheotherschemas.Justaswiththemulti-sourcefeature,thereisthennosourcespecificmetadataentrytotheconformedsources.Alsojustaswithmulti-sourceplanning,thecapabilitiesareassumedtobethesameacrossconformedsources.
Theenginewilltakethelistofconformedsourcesandassociateasetofmodelmetadataidstothecorrespondingaccessnode.Thelogicconsideringjoinsandsubquerieswillalsoconsidertheconformedsetswhenmakingpushdowndecisions.Thesubqueryhandlingwillonlycheckforconformedsourcesforthesubquery-notintheparent.Sohavingaconformedtableinthesubquerywillpushdownasexpected,butnotviceversa.
ConformedTables
663
TeiidComponents
DesignerPlugin-EclipsePluginbasedTeiiddesignenvironment,usedtoconnect/federate/transformdatasourcestoproducea.vdbfile.
JVM-TeiidisapureJavaDataVirtualizationPlatform.
WildFly-TeiiduseaplugableinstallationwhichneedaWildFlyServerinstalled,alternatively,afullinstalledWildFlykitbedistributed.
Subsystem-DuetoWildFly’sModularandPluggableArchitecture(aseriesofManagementcommandscomposeofasubsystem,aseriesofsubsystemscomposeofthewholeserver),TeiidimplementWildFly’sController/ManagementAPIdevelopedateiidsubsystemandreuselotsofothersubsystemslikeresource-adapter,infinispan,security,logging,datasource.
odata.war-TeiidsupportODataviaodata.war,moredetailsrefertoODataSupport
dashboard.war-Awebbaseddashboardgenerator.
teiid-console-AwebbasedadministrativeandmonitoringtoolforTeiid,moredetailsrefertoTeiidConsole
JDBCDriver-JDBCDrivertoconnecttoTeiidServer.
adminshell-AscriptingbasedMonitor/ManagementTool,moredetailsrefertoAdminShell
quickstarts-AmavenquickstartshowinghowtoutilizeTeiid.
TeiidArchitecture
Architecture
664
Client-ClientDevelopGuide
Transport-ATransportmanagementsclientconnections-securityauthentication,encryption,etc.
QueryEngine-TheQueryEnginehasseverallayers/components.Requestprocessingatahighlevel:
1. SQLisconvertedtoaProcessorPlan.TheenginereceivesanincomingSQLquery.Itisparsedtoainternalcommand.Thenthecommandisconvertedalogicalplanviaresolving,validating,andrewriting.Lastlyruleandcost-basedoptimizationconvertthelogicalplantoafinalProcessorPlan.MoredetailsrefertoFederatedPlanning.
2. BatchProcessing.Thesourceandotheraspectsofqueryprocessingmayreturnresultsasynchronouslytotheprocessingthread.Assoonaspossiblebatchesofresultsaremadeavailabletotheclient.
3. BufferManagementControlsthebulkoftheonandoffheapmemorythatTeiidisusing.Itpreventsconsumingtoomuchmemorythatotherwisemightexceedthevmsize.
4. TransactionManagementdetermineswhentransactionsareneededandinteractswiththeTransactionManagersubsystemtocoordinateXAtransactions.
SourcequeriesarehandledbytheDataTierlayerwhichinterfaceswiththeQueryEngineandtheConnectorLayerwhichutilizesaTranslator/ResourceAdapterpairtointeractdirectlywithasource.Connectivityisprovidedforheterogeneousdatastores,likeDatabases/Datawarehouse,NoSQL,Hadoop,DataGrid/Cache,File,SaaS,etc.-seeDataSources.
Translator-TeiidhasdevelopedaseriesofTranslators,formoredetailsrefertoTranslators.
ResourceAdapter-Providescontainermanagedaccesstoasource,formoredetailsrefertoDevelopingJEEConnectors.
Architecture
665
TerminologyVMorProcess–aJBossASinstancerunningTeiid.
Host–amachinethatis"hosting"oneormoreVMs.
Service–asubsystemrunninginaVM(ofteninmanyVMs)andprovidingarelatedsetoffunctionality.Inadditiontothesemaincomponents,theserviceplatformprovidesacoresetofservicesavailabletoapplicationsbuiltontopoftheserviceplatform.Theseservicesare:
Session–theSessionservicemanagesactivesessioninformation.
BufferManager–theBufferManagerserviceprovidesaccesstodatamanagementforintermediateresults.
Transaction–theTransactionservicemanagesglobal,local,andrequestscopedtransactions.SeealsothedocumentationonTransactionSupport.
Terminology
666
DataManagement
CursoringandBatching
Teiidcursorsallresults,regardlessofwhethertheyarefromonesourceormanysources,andregardlessofwhattypeofprocessing(joins,unions,etc.)havebeenperformedontheresults.
Teiidprocessesresultsinbatches.Abatchissimplyasetofrecords.Thenumberofrowsinabatchisdeterminedbythebuffersystempropertyprocessor-batch-sizeandisscaledupontheestimatedmemoryfootprintofthebatch.
Clientapplicationshavenodirectknowledgeofbatchesorbatchsizes,butratherspecifyfetchsize.Howeverthefirstbatch,regardlessoffetchsizeisalwaysproactivelyreturnedtosynchronousclients.Subsequentbatchesarereturnedbasedonclientdemandforthedata.Pre-fetchingisutilizedatboththeclientandconnectorlevels.
BufferManagement
Thebuffermanagermanagesmemoryforallresultsetsusedinthequeryengine.Thatincludesresultsetsreadfromaconnectionfactory,resultsetsusedtemporarilyduringprocessing,andresultsetspreparedforauser.Eachresultsetisreferredtointhebuffermanagerasatuplesource.
Whenretrievingbatchesfromthebuffermanager,thesizeofabatchinbytesisestimatedandthenallocatedagainstthemaxlimit.
MemoryManagement
Thebuffermanagerhastwostoragemanagers-amemorymanagerandadiskmanager.Thebuffermanagermaintainsthestateofallthebatches,anddetermineswhenbatchesmustbemovedfrommemorytodisk.
DiskManagement
Eachtuplesourcehasadedicatedfile(namedbytheID)ondisk.Thisfilewillbecreatedonlyifatleastonebatchforthetuplesourcehadtobeswappedtodisk.Thefileisrandomaccess.Theprocessorbatchsizepropertydefineshowmanyrowsshouldnominallyexistinabatchassuming2048bitsworthofdatainarow.Iftherowislargerorsmallerthanthattarget,theenginewilladjustthebatchsizeforthosetuplesaccordingly.Batchesarealwaysreadandwrittenfromthestoragemanagerwhole.
Thediskstoragemanagerhasacaponthemaximumnumberofopenfilestopreventrunningoutoffilehandles.Incaseswithheavybuffering,thiscancausewaittimeswhilewaitingforafilehandletobecomeavailable(thedefaultmaxopenfilesis64).
CleanupWhenatuplesourceisnolongerneeded,itisremovedfromthebuffermanager.Thebuffermanagerwillremoveitfromboththememorystoragemanagerandthediskstoragemanager.Thediskstoragemanagerwilldeletethefile.Inaddition,everytuplesourceistaggedwitha"groupname"whichistypicallythesessionIDoftheclient.Whentheclient’ssessionisterminated(byclosingtheconnection,serverdetectingclientshutdown,oradministrativetermination),acallissenttothebuffermanagertoremovealltuplesourcesforthesession.
Inaddition,whenthequeryengineisshutdown,thebuffermanagerisshutdown,whichwillremoveallstatefromthediskstoragemanagerandcauseallfilestobeclosed.Whenthequeryengineisstopped,itissafetodeleteanyfilesinthebufferdirectoryastheyarenotusedacrossqueryenginerestartsandmustbeduetoasystemcrashwherebufferfileswerenotcleaned
DataManagement
667
up.
DataManagement
668
QueryTermination
CancelingQueries
Whenaqueryiscanceled,processingwillbestoppedinthequeryengineandinallconnectorsinvolvedinthequery.Thesemanticsofwhataconnectordoesinresponsetoacancellationcommandisdependentontheconnectorimplementation.Forexample,JDBCconnectorswillasynchronouslycallcancelontheunderlyingJDBCdriver,whichmayormaynotactuallysupportthismethod.
UserQueryTimeouts
UserquerytimeoutsinTeiidcanbemanagedontheclient-sideortheserver-side.Timeoutsareonlyrelevantforthefirstrecordreturned.Ifthefirstrecordhasnotbeenreceivedbytheclientwithinthespecifiedtimeoutperiod,a`cancel'commandisissuedtotheserverfortherequestandnoresultsarereturnedtotheclient.Thecancelcommandisissuedasynchronouslywithouttheclient’sintervention.
TheJDBCAPIusesthequerytimeoutsetbythejava.sql.Statement.setQueryTimeoutmethod.Youmayalsosetadefaultstatementtimeoutviatheconnectionproperty"QUERYTIMEOUT".ODBCclientsmayalsoutilizeQUERYTIMEOUTasanexecutionpropertyviaasetstatementtocontrolthedefaulttimeoutsetting.SeetheClientDevelopersGuideformoreonconnection/executionpropertiesandsetstatements.
Server-sidetimeoutsstartwhenthequeryisreceivedbytheengine.TheremaybeaskewfromthewhentheclientissuedthequeryduetonetworklatencyorserverloadthatmayslowtheprocessingofIOwork.Thetimeoutwillbecancelledifthefirstresultissentbackbeforethetimeouthasended.SeetheVDBssectionformoreonsettingthequery-timeoutVDBproperty.SeetheAdminGuideformoreonmodifyingthefiletosetdefaultquerytimeoutforallqueries.
QueryTermination
669
Processing
JoinAlgorithms
Nestedloopdoesthemostobviousprocessing–foreveryrowintheoutersource,itcompareswitheveryrowintheinnersource.Nestedloopisonlyusedwhenthejoincriteriahasnoequi-joinpredicates.
Mergejoinfirstsortstheinputsourcesonthejoinedcolumns.Youcanthenwalkthrougheachsideinparallel(effectivelyonepassthrougheachsortedsource)andwhenyouhaveamatch,emitarow.Ingeneral,mergejoinisontheorderofn+mratherthann*minnestedloop.Mergejoinisthedefaultalgorithm.
Usingcostinginformationtheenginemayalsodelaythedecisiontoperformafullsortmergejoin.Basedupontheactualrowcountsinvolved,theenginecanchoosetobuildanindexofthesmallerside(whichwillperformsimilarlytoahashjoin)ortoonlypartiallysortthelargersideoftherelation.
Joinsinvolvingequi-joinpredicatesarealsoeligibletobemadeintodependentjoins.
SortBasedAlgorithms
SortingisusedasthebasisoftheSort(ORDERBY),Grouping(GROUPBY),andDupRemoval(SELECTDISTINCT)operations.Thesortalgorithmisamulti-passmerge-sortthatdoesnotrequirealloftheresultsettoeverbeinmemoryyetusesthemaximalamountofmemoryallowedbythebuffermanager.
Itconsistsoftwophases.Thefirstphase("sort")willtakeanunsortedinputstreamandproduceoneormoresortedinputstreams.Eachpassreadsasmuchoftheunsortedstreamaspossible,sortsit,andwritesitbackoutasanewstream.Sincethestreammaybemorethancanfitinmemory,thismayresultinmanysortedstreams.
Thesecondphase("merge")consistsofasetofphasesthatgrabthenextbatchfromasmanysortedinputstreamsaswillfitinmemory.Itthenrepeatedlygrabsthenexttupleinsortedorderfromeachstreamandoutputsmergedsortedbatchestoanewsortedstream.Atcompletionofthepass,allinputstreamsaredropped.Inthisway,eachpassreducesthenumberofsortedstreams.Whenonlyonestreamremains,itisthefinaloutput.
Processing
670
BNFforSQLGrammarMainEntryPoints
callablestatement
ddlstatement
procedurebodydefinition
directlyexecutablestatement
ReservedKeywords
Non-ReservedKeywords
ReservedKeywordsForFutureUse
Tokens
ProductionCross-Reference
Productions
ReservedKeywords
Keyword Usage
ADD addsetchildoption,addsetoption,ADDcolumn
ALLstandardaggregatefunction,function,queryexpressionbody,queryterm,granttype,selectclause,quantifiedcomparisonpredicate
ALTER alter,altercolumnoptions,alterStatement,granttype
AND betweenpredicate,booleanterm
ANY standardaggregatefunction,withrole,quantifiedcomparisonpredicate
ARRAY ARRAYexpressionconstructor
ARRAY_AGG orderedaggregatefunction
AS
alter,ALTERPROCEDURE,ALTERTABLE,ALTERTRIGGER,arraytable,createprocedure,optionnamespace,createtable,createtrigger,derivedcolumn,dynamicdatastatement,function,loopstatement,xmlnamespaceelement,objecttable,selectderivedcolumn,tablesubquery,texttable,tablename,unescapedFunction,withlistelement,xmlserialize,xmltable
ASC sortspecification
ATOMIC compoundstatement,foreachrowtriggeraction
BNFforSQLGrammar
671
AUTHENTICATED withrole
BEGIN compoundstatement,foreachrowtriggeraction
BETWEEN betweenpredicate
BIGDECIMAL simpledatatype
BIGINT simpledatatype
BIGINTEGER simpledatatype
BLOB simpledatatype,xmlserialize
BOOLEAN simpledatatype
BOTH function
BREAK branchingstatement
BY groupbyclause,orderbyclause,windowspecification
BYTE simpledatatype
CALL callablestatement,callstatement
CASE caseexpression,searchedcaseexpression
CAST function
CHAR function,simpledatatype
CLOB simpledatatype,xmlserialize
COLUMN ADDcolumn,altercolumnoptions,DROPcolumn,renamecolumnoptions,CreateGRANT,RevokeGRANT
COMMIT createtemporarytable
CONSTRAINT createtablebody,CreateGRANT,RevokeGRANT
CONTINUE branchingstatement
CONVERT function
CREATE
createprocedure,createdatawrapperakatranslator,createdatabase,createforeigntemptable,createrole,createschema,createserver,akadatasource,createtable,createtemporarytable,createtrigger,procedurebodydefinition
CROSS crossjoin
DATE nonnumericliteral,simpledatatype
BNFforSQLGrammar
672
DAY function
DECIMAL simpledatatype
DECLARE declarestatement
DEFAULT tableelement,xmlnamespaceelement,objecttablecolumn,procedureparameter,xmltablecolumn
DELETE alter,ALTERTRIGGER,createtrigger,deletestatement,granttype
DESC sortspecification
DISTINCT standardaggregatefunction,function,ifstatement,queryexpressionbody,queryterm,selectclause
DOUBLE simpledatatype
DROP
DROPcolumn,dropoption,Dropdatawrapperakatranslator,dropdatabase,dropoption,dropprocedure,droprole,dropschema,dropserver,akadatasource,droptable,droptable,granttype
EACH foreachrowtriggeraction
ELSE caseexpression,ifstatement,searchedcaseexpression
END caseexpression,compoundstatement,foreachrowtriggeraction,searchedcaseexpression
ERROR raiseerrorstatement
ESCAPE matchpredicate,texttable
EXCEPT Importforeignschema,queryexpressionbody
EXEC dynamicdatastatement,callstatement
EXECUTE dynamicdatastatement,granttype,callstatement
EXISTS existspredicate
FALSE nonnumericliteral
FETCH fetchclause
FILTER filterclause
FLOAT simpledatatype
FOR foreachrowtriggeraction,function,textaggreatefunction,texttablecolumn,xmltablecolumn
BNFforSQLGrammar
673
FOREIGN
alterStatement,createprocedure,createdatawrapperakatranslator,createforeigntemptable,createserver,akadatasource,createtable,Dropdatawrapperakatranslator,dropprocedure,droptable,foreignkey,Importforeignschema
FROM deletestatement,fromclause,function,ifstatement,Importforeignschema,RevokeGRANT
FULL qualifiedtable
FUNCTION createprocedure,dropprocedure
GEOMETRY simpledatatype
GLOBAL createtable,droptable
GRANT CreateGRANT,RevokeGRANT
GROUP groupbyclause
HAVING havingclause
HOUR function
IF ifstatement
IMMEDIATE dynamicdatastatement
IMPORT ImportanotherDatabase,Importforeignschema
IN procedureparameter,inpredicate
INNER qualifiedtable
INOUT procedureparameter
INSERT alter,ALTERTRIGGER,createtrigger,function,insertstatement,granttype
INTEGER simpledatatype
INTERSECT queryterm
INTO dynamicdatastatement,Importforeignschema,insertstatement,intoclause
IS ifstatement,isnullpredicate
JOIN crossjoin,makedepoptions,qualifiedtable
LANGUAGE objecttable,granttype
LATERAL tablesubquery
BNFforSQLGrammar
674
LEADING function
LEAVE branchingstatement
LEFT function,qualifiedtable
LIKE matchpredicate
LIKE_REGEX likeregexpredicate
LIMIT Importforeignschema,limitclause
LOCAL createforeigntemptable,createtemporarytable
LONG simpledatatype
LOOP loopstatement
MAKEDEP optionclause,tableprimary
MAKEIND optionclause,tableprimary
MAKENOTDEP optionclause,tableprimary
MERGE insertstatement
MINUTE function
MONTH function
NO makedepoptions,xmlnamespaceelement,textaggreatefunction,texttablecolumn,texttable
NOCACHE optionclause
NOT
altercolumnoptions,betweenpredicate,compoundstatement,tableelement,ifstatement,isnullpredicate,matchpredicate,booleanfactor,procedureparameter,procedureresultcolumn,likeregexpredicate,inpredicate,temporarytableelement
NULLaltercolumnoptions,tableelement,isnullpredicate,nonnumericliteral,procedureparameter,procedureresultcolumn,temporarytableelement,xmlquery
OBJECT simpledatatype
OF alter,ALTERTRIGGER,createtrigger
OFFSET limitclause
ONalter,ALTERTRIGGER,createforeigntemptable,createtemporarytable,createtrigger,CreateGRANT,loopstatement,qualifiedtable,RevokeGRANT,xmlquery
BNFforSQLGrammar
675
ONLY fetchclause
OPTION optionclause
OPTIONS alterchildoptionslist,alteroptionslist,optionsclause
OR booleanvalueexpression
ORDER CreateGRANT,orderbyclause,RevokeGRANT
OUT procedureparameter
OUTER qualifiedtable
OVER windowspecification
PARAMETER altercolumnoptions,renamecolumnoptions
PARTITION windowspecification
PRIMARY tableelement,createtemporarytable,primarykey
PROCEDUREalter,ALTERPROCEDURE,createprocedure,dropprocedure,CreateGRANT,procedurebodydefinition,RevokeGRANT
REAL simpledatatype
REFERENCES foreignkey
RETURN assignmentstatement,returnstatement,datastatement
RETURNS createprocedure
REVOKE RevokeGRANT
RIGHT function,qualifiedtable
ROLLUP groupbyclause
ROW fetchclause,foreachrowtriggeraction,limitclause,texttable
ROWS createtemporarytable,fetchclause,limitclause
SECOND function
SELECT granttype,selectclause
SERVERALTERSERVER,createschema,createserver,akadatasource,dropserver,akadatasource,Importforeignschema
BNFforSQLGrammar
676
SET addsetchildoption,addsetoption,optionnamespace,updatestatement,useschema
SHORT simpledatatype
SIMILAR matchpredicate
SMALLINT simpledatatype
SOME standardaggregatefunction,quantifiedcomparisonpredicate
SQLEXCEPTION sqlexception
SQLSTATE sqlexception
SQLWARNING raisestatement
STRING dynamicdatastatement,simpledatatype,xmlserialize
TABLE
ALTERTABLE,createprocedure,createforeigntemptable,createtable,createtemporarytable,droptable,droptable,CreateGRANT,queryprimary,granttype,RevokeGRANT,tablesubquery
TEMPORARY createforeigntemptable,createtable,createtemporarytable,droptable,granttype
THEN caseexpression,searchedcaseexpression
TIME nonnumericliteral,simpledatatype
TIMESTAMP nonnumericliteral,simpledatatype
TINYINT simpledatatype
TO renamecolumnoptions,CreateGRANT,Importforeignschema,matchpredicate
TRAILING function
TRANSLATE function
TRIGGER alter,ALTERTRIGGER,createtrigger
TRUE nonnumericliteral
UNION crossjoin,queryexpressionbody
UNIQUE otherconstraints,tableelement
UNKNOWN nonnumericliteral
BNFforSQLGrammar
677
UPDATE alter,ALTERTRIGGER,createtrigger,dynamicdatastatement,granttype,updatestatement
USER function
USING dynamicdatastatement
VALUES queryprimary
VARBINARY simpledatatype,xmlserialize
VARCHAR simpledatatype,xmlserialize
VIRTUALalterStatement,createprocedure,createschema,createtable,dropprocedure,dropschema,droptable,procedurebodydefinition
WHEN caseexpression,searchedcaseexpression
WHERE filterclause,whereclause
WHILE whilestatement
WITH assignmentstatement,createrole,ImportanotherDatabase,queryexpression,withrole,datastatement
WITHOUT assignmentstatement,datastatement
WRAPPERALTERDATAWRAPPER,createdatawrapperakatranslator,createserver,akadatasource,Dropdatawrapperakatranslator
XML simpledatatype
XMLAGG orderedaggregatefunction
XMLATTRIBUTES xmlattributes
XMLCAST unescapedFunction
XMLCOMMENT function
XMLCONCAT function
XMLELEMENT xmlelement
XMLEXISTS xmlquery
XMLFOREST xmlforest
XMLNAMESPACES xmlnamespaces
XMLPARSE xmlparse
BNFforSQLGrammar
678
XMLPI function
XMLQUERY xmlquery
XMLSERIALIZE xmlserialize
XMLTABLE xmltable
XMLTEXT function
YEAR function
Non-ReservedKeywords
Name Usage
ACCESS ImportanotherDatabase,non-reservedidentifier
ACCESSPATTERN otherconstraints,non-reservedidentifier
AFTER alter,createtrigger,non-reservedidentifier
ARRAYTABLE arraytable,non-reservedidentifier
AUTO_INCREMENT altercolumnoptions,tableelement,non-reservedidentifier
AVG standardaggregatefunction,non-reservedidentifier
CHAIN sqlexception,non-reservedidentifier
COLUMNS arraytable,non-reservedidentifier,objecttable,texttable,xmltable
CONDITION CreateGRANT,non-reservedidentifier,RevokeGRANT
CONTENT non-reservedidentifier,xmlparse,xmlserialize
CONTROL ImportanotherDatabase,non-reservedidentifier
COUNT standardaggregatefunction,non-reservedidentifier
DATAALTERDATAWRAPPER,createdatawrapperakatranslator,createserver,akadatasource,Dropdatawrapperakatranslator,non-reservedidentifier
DATABASEALTERDATABASE,createdatabase,dropdatabase,CreateGRANT,ImportanotherDatabase,non-reservedidentifier,RevokeGRANT,usedatabase
DELIMITER non-reservedidentifier,textaggreatefunction,texttable
DENSE_RANK analyticaggregatefunction,non-reservedidentifier
BNFforSQLGrammar
679
DISABLED alter,ALTERTRIGGER,non-reservedidentifier
DOCUMENT non-reservedidentifier,xmlparse,xmlserialize
EMPTY non-reservedidentifier,xmlquery
ENABLED alter,ALTERTRIGGER,non-reservedidentifier
ENCODING non-reservedidentifier,textaggreatefunction,xmlserialize
EVERY standardaggregatefunction,non-reservedidentifier
EXCEPTION compoundstatement,declarestatement,non-reservedidentifier
EXCLUDING non-reservedidentifier,xmlserialize
EXTRACT function,non-reservedidentifier
FIRST fetchclause,non-reservedidentifier,sortspecification
HEADER non-reservedidentifier,textaggreatefunction,texttablecolumn,texttable
INCLUDING non-reservedidentifier,xmlserialize
INDEX otherconstraints,tableelement,non-reservedidentifier
INSTEAD alter,ALTERTRIGGER,createtrigger,non-reservedidentifier
JAAS non-reservedidentifier,withrole
JSONARRAY_AGG non-reservedidentifier,orderedaggregatefunction
JSONOBJECT jsonobject,non-reservedidentifier
KEY tableelement,createtemporarytable,foreignkey,non-reservedidentifier,primarykey
LAST non-reservedidentifier,sortspecification
MASK CreateGRANT,non-reservedidentifier,RevokeGRANT
MAX standardaggregatefunction,makedepoptions,non-reservedidentifier
MIN standardaggregatefunction,non-reservedidentifier
NAME function,non-reservedidentifier,xmlelement
NAMESPACE optionnamespace,non-reservedidentifier
BNFforSQLGrammar
680
NEXT fetchclause,non-reservedidentifier
NONE non-reservedidentifier
NULLS non-reservedidentifier,sortspecification
OBJECTTABLE non-reservedidentifier,objecttable
ORDINALITY non-reservedidentifier,texttablecolumn,xmltablecolumn
PASSING non-reservedidentifier,objecttable,xmlquery,xmlquery,xmltable
PATH non-reservedidentifier,xmltablecolumn
PRESERVE createtemporarytable,non-reservedidentifier
PRIVILEGES non-reservedidentifier,granttype
QUERYSTRING non-reservedidentifier,querystringfunction
QUOTE non-reservedidentifier,textaggreatefunction,texttable
RAISE non-reservedidentifier,raisestatement
RANK analyticaggregatefunction,non-reservedidentifier
RENAME renamecolumnoptions,non-reservedidentifier
REPOSITORY Importforeignschema,non-reservedidentifier
RESULT non-reservedidentifier,procedureparameter
ROLE createrole,droprole,non-reservedidentifier,withrole
ROW_NUMBER analyticaggregatefunction,non-reservedidentifier
SCHEMAcreateschema,dropschema,CreateGRANT,Importforeignschema,non-reservedidentifier,RevokeGRANT,useschema
SELECTOR non-reservedidentifier,texttablecolumn,texttable
SERIAL altercolumnoptions,tableelement,non-reservedidentifier,temporarytableelement
SKIP non-reservedidentifier,texttable
SQL_TSI_DAY timeinterval,non-reservedidentifier
SQL_TSI_FRAC_SECOND timeinterval,non-reservedidentifier
BNFforSQLGrammar
681
SQL_TSI_HOUR timeinterval,non-reservedidentifier
SQL_TSI_MINUTE timeinterval,non-reservedidentifier
SQL_TSI_MONTH timeinterval,non-reservedidentifier
SQL_TSI_QUARTER timeinterval,non-reservedidentifier
SQL_TSI_SECOND timeinterval,non-reservedidentifier
SQL_TSI_WEEK timeinterval,non-reservedidentifier
SQL_TSI_YEAR timeinterval,non-reservedidentifier
STDDEV_POP standardaggregatefunction,non-reservedidentifier
STDDEV_SAMP standardaggregatefunction,non-reservedidentifier
SUBSTRING function,non-reservedidentifier
SUM standardaggregatefunction,non-reservedidentifier
TEXTAGG non-reservedidentifier,textaggreatefunction
TEXTTABLE non-reservedidentifier,texttable
TIMESTAMPADD function,non-reservedidentifier
TIMESTAMPDIFF function,non-reservedidentifier
TO_BYTES function,non-reservedidentifier
TO_CHARS function,non-reservedidentifier
TRANSLATORALTERDATAWRAPPER,createdatawrapperakatranslator,createserver,akadatasource,Dropdatawrapperakatranslator,non-reservedidentifier
TRIM function,non-reservedidentifier,texttablecolumn,texttable
TYPE altercolumnoptions,createdatawrapperakatranslator,createserver,akadatasource,non-reservedidentifier
UPSERT insertstatement,non-reservedidentifier
USE non-reservedidentifier,usedatabase
VARIADIC non-reservedidentifier,procedureparameter
VAR_POP standardaggregatefunction,non-reservedidentifier
VAR_SAMP standardaggregatefunction,non-reservedidentifier
BNFforSQLGrammar
682
VERSIONcreatedatabase,createserver,akadatasource,dropdatabase,ImportanotherDatabase,non-reservedidentifier,usedatabase,xmlserialize
VIEW alter,ALTERTABLE,createtable,droptable,non-reservedidentifier
WELLFORMED non-reservedidentifier,xmlparse
WIDTH non-reservedidentifier,texttablecolumn
XMLDECLARATION non-reservedidentifier,xmlserialize
ReservedKeywordsForFutureUse
ALLOCATE ARE ASENSITIVE
ASYMETRIC AUTHORIZATION BINARY
CALLED CASCADED CHARACTER
CHECK CLOSE COLLATE
CONNECT CORRESPONDING CRITERIA
CURRENT_DATE CURRENT_TIME CURRENT_TIMESTAMP
CURRENT_USER CURSOR CYCLE
DATALINK DEALLOCATE DEC
DEREF DESCRIBE DETERMINISTIC
DISCONNECT DLNEWCOPY DLPREVIOUSCOPY
DLURLCOMPLETE DLURLCOMPLETEONLY DLURLCOMPLETEWRITE
DLURLPATH DLURLPATHONLY DLURLPATHWRITE
DLURLSCHEME DLURLSERVER DLVALUE
DYNAMIC ELEMENT EXTERNAL
FREE GET HAS
HOLD IDENTITY INDICATOR
INPUT INSENSITIVE INT
INTERVAL ISOLATION LARGE
BNFforSQLGrammar
683
LOCALTIME LOCALTIMESTAMP MATCH
MEMBER METHOD MODIFIES
MODULE MULTISET NATIONAL
NATURAL NCHAR NCLOB
NEW NUMERIC OLD
OPEN OUTPUT OVERLAPS
PRECISION PREPARE RANGE
READS RECURSIVE REFERENCING
RELEASE ROLLBACK SAVEPOINT
SCROLL SEARCH SENSITIVE
SESSION_USER SPECIFIC SPECIFICTYPE
SQL START STATIC
SUBMULTILIST SYMETRIC SYSTEM
SYSTEM_USER TIMEZONE_HOUR TIMEZONE_MINUTE
TRANSLATION TREAT VALUE
VARYING WHENEVER WINDOW
WITHIN XMLBINARY XMLDOCUMENT
Tokens
Name Definition Usage
allingroupidentifier <identifier><period><star> allingroup
binarystringliteral "X"|"x""\'"(<hexit><hexit>)+"\'" nonnumericliteral
colon ":" makedepoptions,statement
alterchildoptionslist,alteroptionslist,ARRAYexpressionconstructor,columnlist,createprocedure,typedelementlist,createtablebody,createtemporarytable,derivedcolumnlist,sqlexception,namedparameterlist,expressionlist,fromclause,function,CreateGRANT,limitclause,nestedexpression,objecttable,option
BNFforSQLGrammar
684
comma "," clause,optionsclause,orderbyclause,simpledatatype,queryexpression,queryprimary,querystringfunction,identifierlist,RevokeGRANT,selectclause,setclauselist,inpredicate,textaggreatefunction,texttable,xmlattributes,xmlelement,xmlquery,xmlforest,xmlnamespaces,xmlquery,xmltable
concat_op "||" commonvalueexpression
decimalnumericliteral (<digit>)*<period><unsignedintegerliteral> unsignednumericliteral
digit \["0"\-"9"\]
dollar "$" parameterreference
double_amp_op "&&" commonvalueexpression
eq "="
assignmentstatement,callablestatement,declarestatement,namedparameterlist,comparisonoperator,setclauselist
escapedfunction "{""fn" unsignedvalueexpressionprimary
escapedjoin "{""oj" tablereference
escapedtype "{"("d"|"t"|"ts"|"b") nonnumericliteral
approximatenumericliteral<digit><period><unsignedintegerliteral>\["e","E"\](<plus>|<minus>)?<unsignedintegerliteral>
unsignednumericliteral
ge ">=" comparisonoperator
gt ">" namedparameterlist,comparisonoperator
hexit \["a"\-"f","A"\-"F"\]|<digit>
identifier <quoted_id>(<period><quoted_id>)*
identifier,unsignedvalueexpressionprimary
id_part ("@"|"#"|<letter>)(<letter>|"_"|<digit>)*
lbrace "{" callablestatement,matchpredicate
le "⇐" comparisonoperator
letter \["a"\-"z","A"\-"Z"\]|\["\u0153"\-"\ufffd"\]
BNFforSQLGrammar
685
lparen "("
standardaggregatefunction,alterchildoptionslist,alteroptionslist,analyticaggregatefunction,arraytable,callablestatement,columnlist,otherconstraints,createprocedure,createtablebody,createtemporarytable,filterclause,function,groupbyclause,ifstatement,Importforeignschema,jsonobject,loopstatement,makedepoptions,nestedexpression,objecttable,optionsclause,orderedaggregatefunction,simpledatatype,queryprimary,querystringfunction,inpredicate,callstatement,subquery,quantifiedcomparisonpredicate,tablesubquery,tableprimary,textaggreatefunction,texttable,unescapedFunction,whilestatement,windowspecification,withlistelement,xmlattributes,xmlelement,xmlquery,xmlforest,xmlnamespaces,xmlparse,xmlquery,xmlserialize,xmltable
lsbrace "[" ARRAYexpressionconstructor,datatype,valueexpressionprimary
lt "<" comparisonoperator
minus "-" plusorminus
ne "<>" comparisonoperator
ne2 "!=" comparisonoperator
period "."
plus "+" plusorminus
qmark "?" callablestatement,parameterreference
quoted_id <id_part>|"\""("\"\""|~\["\""\])+"\""
rbrace "}"callablestatement,matchpredicate,nonnumericliteral,tablereference,unsignedvalueexpressionprimary
rparen ")"
standardaggregatefunction,alterchildoptionslist,alteroptionslist,analyticaggregatefunction,arraytable,callablestatement,columnlist,otherconstraints,createprocedure,createtablebody,createtemporarytable,filterclause,function,groupbyclause,ifstatement,Importforeignschema,jsonobject,loopstatement,makedepoptions,nestedexpression,objecttable,optionsclause,orderedaggregatefunction,simpledatatype,queryprimary,querystringfunction,
BNFforSQLGrammar
686
inpredicate,callstatement,subquery,quantifiedcomparisonpredicate,tablesubquery,tableprimary,textaggreatefunction,texttable,unescapedFunction,whilestatement,windowspecification,withlistelement,xmlattributes,xmlelement,xmlquery,xmlforest,xmlnamespaces,xmlparse,xmlquery,xmlserialize,xmltable
rsbrace "]" ARRAYexpressionconstructor,datatype,valueexpressionprimary
semicolon ";" delimitedstatement
slash "/" starorslash
star "*"standardaggregatefunction,dynamicdatastatement,selectclause,starorslash
stringliteral ("N"|"E")?"\'"("\'\'"|~\["\'"\])*"\'" string
unsignedintegerliteral (<digit>)+ unsignedinteger,unsignednumericliteral
ProductionCross-Reference
Name Usage
addsetchildoption alterchildoptionslist
addsetoption alteroptionslist
standardaggregatefunction unescapedFunction
allingroup selectsublist
alter directlyexecutablestatement
ADDcolumn ALTERTABLE
alterchildoptionpair addsetchildoption
alterchildoptionslist altercolumnoptions
altercolumnoptions ALTERPROCEDURE,ALTERTABLE
ALTERDATABASE alterStatement
DROPcolumn ALTERTABLE
alteroptionpair addsetoption
BNFforSQLGrammar
687
alteroptionslist ALTERDATABASE,ALTERPROCEDURE,ALTERSERVER,ALTERTABLE,ALTERDATAWRAPPER
ALTERPROCEDURE alterStatement
renamecolumnoptions ALTERPROCEDURE,ALTERTABLE
ALTERSERVER alterStatement
alterStatement ddlstatement
ALTERTABLE alterStatement
ALTERDATAWRAPPER alterStatement
ALTERTRIGGER alterStatement
analyticaggregatefunction unescapedFunction
ARRAYexpressionconstructor unsignedvalueexpressionprimary
arraytable tableprimary
assignmentstatement delimitedstatement
assignmentstatementoperand assignmentstatement,declarestatement
betweenpredicate booleanprimary
booleanprimary filterclause,booleanfactor
branchingstatement delimitedstatement
caseexpression unsignedvalueexpressionprimary
character matchpredicate,textaggreatefunction,texttable
columnlist otherconstraints,createtemporarytable,foreignkey,insertstatement,primarykey,withlistelement
commonvalueexpressionbetweenpredicate,booleanprimary,comparisonpredicate,sqlexception,matchpredicate,likeregexpredicate,inpredicate,texttable
comparisonpredicate booleanprimary
booleanterm booleanvalueexpression
booleanvalueexpression condition
compoundstatement statement,directlyexecutablestatement
otherconstraints createtablebody
BNFforSQLGrammar
688
tableelement ADDcolumn,createtablebody
createprocedure ddlstatement
createdatawrapperakatranslator ddlstatement
createdatabase ddlstatement
typedelementlist arraytable,dynamicdatastatement
createforeigntemptable directlyexecutablestatement
optionnamespace ddlstatement
createrole ddlstatement
createschema ddlstatement
createserver,akadatasource ddlstatement
createtable ddlstatement
createtablebody createforeigntemptable,createtable
createtemporarytable directlyexecutablestatement
createtrigger ddlstatement,directlyexecutablestatement
condition expression,havingclause,ifstatement,qualifiedtable,searchedcaseexpression,whereclause,whilestatement
crossjoin joinedtable
ddlstatement ddlstatement
declarestatement delimitedstatement
deletestatement assignmentstatementoperand,directlyexecutablestatement
delimitedstatement statement
derivedcolumnderivedcolumnlist,objecttable,querystringfunction,textaggreatefunction,xmlattributes,xmlquery,xmlquery,xmltable
derivedcolumnlist jsonobject,xmlforest
dropoption alterchildoptionslist
Dropdatawrapperakatranslator ddlstatement
BNFforSQLGrammar
689
dropdatabase ddlstatement
dropoption alteroptionslist
dropprocedure ddlstatement
droprole ddlstatement
dropschema ddlstatement
dropserver,akadatasource ddlstatement
droptable directlyexecutablestatement
droptable ddlstatement
dynamicdatastatement datastatement
raiseerrorstatement delimitedstatement
sqlexception assignmentstatementoperand,exceptionreference
exceptionreference sqlexception,raisestatement
namedparameterlist callablestatement,callstatement
existspredicate booleanprimary
expression
standardaggregatefunction,ARRAYexpressionconstructor,assignmentstatementoperand,caseexpression,tableelement,derivedcolumn,dynamicdatastatement,raiseerrorstatement,namedparameterlist,expressionlist,function,nestedexpression,objecttablecolumn,orderedaggregatefunction,procedureparameter,querystringfunction,returnstatement,searchedcaseexpression,selectderivedcolumn,setclauselist,sortkey,quantifiedcomparisonpredicate,unescapedFunction,xmltablecolumn,xmlelement,xmlparse,xmlserialize
expressionlist callablestatement,otherconstraints,function,groupbyclause,queryprimary,callstatement,windowspecification
fetchclause limitclause
filterclause function,unescapedFunction
foreachrowtriggeraction alter,ALTERTRIGGER,createtrigger
foreignkey createtablebody
fromclause query
function unescapedFunction,unsignedvalueexpressionprimary
CreateGRANT ddlstatement
BNFforSQLGrammar
690
groupbyclause query
havingclause query
identifier
alter,alterchildoptionpair,altercolumnoptions,ALTERDATABASE,DROPcolumn,alteroptionpair,ALTERPROCEDURE,renamecolumnoptions,ALTERSERVER,ALTERTABLE,ALTERDATAWRAPPER,ALTERTRIGGER,arraytable,assignmentstatement,branchingstatement,callablestatement,columnlist,compoundstatement,tableelement,createprocedure,createdatawrapperakatranslator,createdatabase,typedelementlist,createforeigntemptable,optionnamespace,createrole,createschema,createserver,akadatasource,createtable,createtablebody,createtemporarytable,createtrigger,declarestatement,deletestatement,derivedcolumn,dropoption,Dropdatawrapperakatranslator,dropdatabase,dropoption,dropprocedure,droprole,dropschema,dropserver,akadatasource,droptable,droptable,dynamicdatastatement,exceptionreference,namedparameterlist,foreignkey,function,CreateGRANT,ifstatement,ImportanotherDatabase,Importforeignschema,insertstatement,intoclause,loopstatement,xmlnamespaceelement,objecttablecolumn,objecttable,optionclause,optionpair,procedureparameter,procedureresultcolumn,queryprimary,identifierlist,RevokeGRANT,selectderivedcolumn,setclauselist,statement,callstatement,tablesubquery,temporarytableelement,textaggreatefunction,texttablecolumn,texttable,tablename,updatestatement,usedatabase,useschema,withlistelement,xmltablecolumn,xmlelement,xmlserialize,xmltable
ifstatement statement
ImportanotherDatabase ddlstatement
Importforeignschema ddlstatement
insertstatement assignmentstatementoperand,directlyexecutablestatement
integerparameter fetchclause,limitclause
unsignedintegerdynamicdatastatement,CreateGRANT,integerparameter,makedepoptions,parameterreference,simpledatatype,RevokeGRANT,texttablecolumn,texttable
timeinterval function
intoclause query
isnullpredicate booleanprimary
joinedtable tableprimary,tablereference
jsonobject function
limitclause queryexpressionbody
BNFforSQLGrammar
691
loopstatement statement
makedepoptions optionclause,tableprimary
matchpredicate booleanprimary
xmlnamespaceelement xmlnamespaces
nestedexpression unsignedvalueexpressionprimary
nonnumericliteral alterchildoptionpair,alteroptionpair,optionpair,valueexpressionprimary
non-reservedidentifier identifier,unsignedvalueexpressionprimary
booleanfactor booleanterm
objecttablecolumn objecttable
objecttable tableprimary
comparisonoperator comparisonpredicate,quantifiedcomparisonpredicate
optionclause callablestatement,deletestatement,insertstatement,queryexpressionbody,callstatement,updatestatement
optionpair optionsclause
optionsclause
tableelement,createprocedure,createdatawrapperakatranslator,createdatabase,createschema,createserver,akadatasource,createtable,createtablebody,Importforeignschema,procedureparameter,procedureresultcolumn
orderbyclause function,orderedaggregatefunction,queryexpressionbody,textaggreatefunction,windowspecification
orderedaggregatefunction unescapedFunction
parameterreference unsignedvalueexpressionprimary
datatype
altercolumnoptions,tableelement,createprocedure,typedelementlist,declarestatement,function,objecttablecolumn,procedureparameter,procedureresultcolumn,temporarytableelement,texttablecolumn,unescapedFunction,xmltablecolumn
simpledatatype datatype
numericvalueexpression commonvalueexpression,valueexpressionprimary
plusorminus alterchildoptionpair,alteroptionpair,optionpair,numericvalueexpression,valueexpressionprimary
primarykey createtablebody
BNFforSQLGrammar
692
procedureparameter createprocedure
procedureresultcolumn createprocedure
qualifiedtable joinedtable
query queryprimary
queryexpression
alter,ALTERTABLE,assignmentstatementoperand,createtable,insertstatement,loopstatement,subquery,tablesubquery,directlyexecutablestatement,withlistelement
queryexpressionbody queryexpression,queryprimary
queryprimary queryterm
querystringfunction function
queryterm queryexpressionbody
raisestatement delimitedstatement
identifierlist createschema,Importforeignschema,withrole
granttype CreateGRANT,RevokeGRANT
withrole createrole
likeregexpredicate booleanprimary
returnstatement delimitedstatement
RevokeGRANT ddlstatement
searchedcaseexpression unsignedvalueexpressionprimary
selectclause query
selectderivedcolumn selectsublist
selectsublist selectclause
setclauselist dynamicdatastatement,updatestatement
inpredicate booleanprimary
sortkey sortspecification
sortspecification orderbyclause
datastatement delimitedstatement
BNFforSQLGrammar
693
statementalter,ALTERPROCEDURE,compoundstatement,createprocedure,foreachrowtriggeraction,ifstatement,loopstatement,procedurebodydefinition,whilestatement
callstatement assignmentstatement,subquery,tablesubquery,directlyexecutablestatement
string
character,createdatabase,optionnamespace,createserver,akadatasource,dropdatabase,function,CreateGRANT,ImportanotherDatabase,xmlnamespaceelement,nonnumericliteral,objecttablecolumn,objecttable,RevokeGRANT,texttablecolumn,texttable,usedatabase,xmltablecolumn,xmlquery,xmlquery,xmlserialize,xmltable
subquery existspredicate,inpredicate,quantifiedcomparisonpredicate,unsignedvalueexpressionprimary
quantifiedcomparisonpredicate booleanprimary
tablesubquery tableprimary
temporarytableelement createtemporarytable
tableprimary crossjoin,joinedtable
tablereference fromclause,qualifiedtable
textaggreatefunction unescapedFunction
texttablecolumn texttable
texttable tableprimary
term numericvalueexpression
starorslash term
tablename tableprimary
unescapedFunction unsignedvalueexpressionprimary
unsignednumericliteral alterchildoptionpair,alteroptionpair,optionpair,valueexpressionprimary
unsignedvalueexpressionprimary integerparameter,valueexpressionprimary
updatestatement assignmentstatementoperand,directlyexecutablestatement
usedatabase ddlstatement
useschema ddlstatement
BNFforSQLGrammar
694
directlyexecutablestatement datastatement
valueexpressionprimary arraytable,term
whereclause deletestatement,query,updatestatement
whilestatement statement
windowspecification unescapedFunction
withlistelement queryexpression
xmlattributes xmlelement
xmltablecolumn xmltable
xmlelement function
xmlquery booleanprimary
xmlforest function
xmlnamespaces xmlelement,xmlquery,xmlforest,xmlquery,xmltable
xmlparse function
xmlquery function
xmlserialize function
xmltable tableprimary
Productions
string::=
<stringliteral>
Astringliteralvalue.Use''toescape'inthestring.
Example:
'astring'
'it''sastring'
non-reservedidentifier::=
INSTEAD
BNFforSQLGrammar
695
VIEW
ENABLED
DISABLED
KEY
SERIAL
TEXTAGG
COUNT
ROW_NUMBER
RANK
DENSE_RANK
SUM
AVG
MIN
MAX
EVERY
STDDEV_POP
STDDEV_SAMP
VAR_SAMP
VAR_POP
DOCUMENT
CONTENT
TRIM
EMPTY
ORDINALITY
PATH
FIRST
LAST
NEXT
SUBSTRING
EXTRACT
TO_CHARS
TO_BYTES
TIMESTAMPADD
TIMESTAMPDIFF
BNFforSQLGrammar
696
QUERYSTRING
NAMESPACE
RESULT
INDEX
ACCESSPATTERN
AUTO_INCREMENT
WELLFORMED
SQL_TSI_FRAC_SECOND
SQL_TSI_SECOND
SQL_TSI_MINUTE
SQL_TSI_HOUR
SQL_TSI_DAY
SQL_TSI_WEEK
SQL_TSI_MONTH
SQL_TSI_QUARTER
SQL_TSI_YEAR
TEXTTABLE
ARRAYTABLE
SELECTOR
SKIP
WIDTH
PASSING
NAME
ENCODING
COLUMNS
DELIMITER
QUOTE
HEADER
NULLS
OBJECTTABLE
VERSION
INCLUDING
EXCLUDING
XMLDECLARATION
BNFforSQLGrammar
697
VARIADIC
RAISE
EXCEPTION
CHAIN
JSONARRAY_AGG
JSONOBJECT
PRESERVE
UPSERT
AFTER
TYPE
TRANSLATOR
JAAS
CONDITION
MASK
ACCESS
CONTROL
NONE
DATA
DATABASE
PRIVILEGES
ROLE
SCHEMA
USE
REPOSITORY
RENAME
Allowsnon-reservedkeywordstobeparsedasidentifiers
Example:SELECTCOUNTFROM…
identifier::=
<identifier>
<non-reservedidentifier>
Partialorfullnameofasingleentity.
Example:
BNFforSQLGrammar
698
tbl.col
"tbl"."col"
createtrigger::=
CREATETRIGGER(<identifier>)?ON<identifier>((INSTEADOF)|AFTER)(INSERT|UPDATE|DELETE)AS<foreachrowtriggeraction>
Createsatriggeractiononthegiventarget.
Example:
CREATETRIGGERONvwINSTEADOFINSERTASFOREACHROWBEGINATOMIC...END
alter::=
ALTER((VIEW<identifier>AS<queryexpression>)|(PROCEDURE<identifier>AS<statement>)|(TRIGGER(<identifier>)?ON<identifier>((INSTEADOF)|AFTER)(INSERT|UPDATE|DELETE)((AS<foreachrowtriggeraction>)|ENABLED|DISABLED)))
Alterthegiventarget.
Example:
ALTERVIEWvwASSELECTcolFROMtbl
foreachrowtriggeraction::=
FOREACHROW((BEGIN(ATOMIC)?(<statement>)*END)|<statement>)
Definesanactiontoperformoneachrow.
Example:
FOREACHROWBEGINATOMIC...END
directlyexecutablestatement::=
<queryexpression>
<callstatement>
<insertstatement>
<updatestatement>
<deletestatement>
BNFforSQLGrammar
699
<droptable>
<createtemporarytable>
<createforeigntemptable>
<alter>
<createtrigger>
<compoundstatement>
Astatementthatcanbeexecutedatruntime.
Example:
SELECT*FROMtbl
droptable::=
DROPTABLE<identifier>
Dropthegiventable.
Example:
DROPTABLE#temp
createtemporarytable::=
CREATE(LOCAL)?TEMPORARYTABLE<identifier><lparen><temporarytableelement>(<comma><temporarytableelement>)*(<comma>PRIMARYKEY<columnlist>)?<rparen>(ONCOMMITPRESERVEROWS)?
Createsatemporarytable.
Example:
CREATELOCALTEMPORARYTABLEtmp(colinteger)
temporarytableelement::=
<identifier>(<datatype>|SERIAL)(NOTNULL)?
Definesatemporarytablecolumn.
Example:
colstringNOTNULL
raiseerrorstatement::=
ERROR<expression>
BNFforSQLGrammar
700
Raisesanerrorwiththegivenmessage.
Example:
ERROR'somethingwentwrong'
raisestatement::=
RAISE(SQLWARNING)?<exceptionreference>
Raisesanerrororwarningwiththegivenmessage.
Example:
RAISESQLEXCEPTION'somethingwentwrong'
exceptionreference::=
<identifier>
<sqlexception>
areferencetoanexception
Example:
SQLEXCEPTION'somethingwentwrong'SQLSTATE'00X',2
sqlexception::=
SQLEXCEPTION<commonvalueexpression>(SQLSTATE<commonvalueexpression>(<comma><commonvalueexpression>)?)?(CHAIN<exceptionreference>)?
createsasqlexceptionorwarningwiththespecifiedmessage,state,andcode
Example:
SQLEXCEPTION'somethingwentwrong'SQLSTATE'00X',2
statement::=
((<identifier><colon>)?(<loopstatement>|<whilestatement>|<compoundstatement>))
<ifstatement>|<delimitedstatement>
Aprocedurestatement.
Example:
IF(x=5)BEGIN...END
BNFforSQLGrammar
701
delimitedstatement::=
(<assignmentstatement>|<datastatement>|<raiseerrorstatement>|<raisestatement>|<declarestatement>|<branchingstatement>|<returnstatement>)<semicolon>
Aprocedurestatementterminatedby;.
Example:
SELECT*FROMtbl;
compoundstatement::=
BEGIN((NOT)?ATOMIC)?(<statement>)*(EXCEPTION<identifier>(<statement>)*)?END
AprocedurestatementblockcontainedinBEGINEND.
Example:
BEGINNOTATOMIC...END
branchingstatement::=
((BREAK|CONTINUE)(<identifier>)?)
(LEAVE<identifier>)
Aprocedurebranchingcontrolstatement,whichtypicallyspecifiesalabeltoreturncontrolto.
Example:
BREAKx
returnstatement::=
RETURN(<expression>)?
Areturnstatement.
Example:
RETURN1
whilestatement::=
WHILE<lparen><condition><rparen><statement>
Aprocedurewhilestatementthatexecutesuntilitsconditionisfalse.
BNFforSQLGrammar
702
Example:
WHILE(var)BEGIN...END
loopstatement::=
LOOPON<lparen><queryexpression><rparen>AS<identifier><statement>
Aprocedureloopstatementthatexecutesoverthegivencursor.
Example:
LOOPON(SELECT*FROMtbl)ASxBEGIN...END
ifstatement::=
IF<lparen>((<identifier>IS(NOT)?DISTINCTFROM<identifier>)|<condition>)<rparen><statement>(ELSE<statement>)?
Aprocedureloopstatementthatexecutesoverthegivencursor.
Example:
IF(boolVal)BEGINvariables.x=1ENDELSEBEGINvariables.x=2END
declarestatement::=
DECLARE(<datatype>|EXCEPTION)<identifier>(<eq><assignmentstatementoperand>)?
Aproceduredeclarationstatementthatcreatesavariableandoptionallyassignsavalue.
Example:
DECLARESTRINGx='a'
assignmentstatement::=
<identifier><eq>(<assignmentstatementoperand>|(<callstatement>((WITH|WITHOUT)RETURN)?))
Assignsavariableavalueinaprocedure.
Example:
x='b'
assignmentstatementoperand::=
<insertstatement>
BNFforSQLGrammar
703
<updatestatement>
<deletestatement>
<expression>
<queryexpression>
<sqlexception>
Avalueorcommandthatcanbeusedinanassignment.{note}Allassigmentsexceptforexpressionaredeprecated.{note}
datastatement::=
(<directlyexecutablestatement>|<dynamicdatastatement>)((WITH|WITHOUT)RETURN)?
AprocedurestatementthatexecutesaSQLstatement.AnupdatestatementcanhaveitsupdatecountaccessedviatheROWCOUNTvariable.
procedurebodydefinition::=
(CREATE(VIRTUAL)?PROCEDURE)?<statement>
DefinesaprocedurebodyonaProceduremetadataobject.
Example:
BEGIN...END
dynamicdatastatement::=
(EXECUTE|EXEC)(STRING|IMMEDIATE)?<expression>(AS<typedelementlist>(INTO<identifier>)?)?(USING<setclauselist>)?(UPDATE(<unsignedinteger>|<star>))?
Aprocedurestatementthatcanexecutearbitrarysql.
Example:
EXECUTEIMMEDIATE'SELECT*FROMtbl'ASxSTRINGINTO#temp
setclauselist::=
<identifier><eq><expression>(<comma><identifier><eq><expression>)*
Alistofvalueassignments.
Example:
col1='x',col2='y'...
BNFforSQLGrammar
704
typedelementlist::=
<identifier><datatype>(<comma><identifier><datatype>)*
Alistoftypedelements.
Example:
col1string,col2integer...
callablestatement::=
<lbrace>(<qmark><eq>)?CALL<identifier>(<lparen>(<namedparameterlist>|(<expressionlist>)?)<rparen>)?<rbrace>(<optionclause>)?
AcallablestatementdefinedusingJDBCescapesyntax.
Example:
{?=CALLproc}
callstatement::=
((EXEC|EXECUTE|CALL)<identifier><lparen>(<namedparameterlist>|(<expressionlist>)?)<rparen>)(<optionclause>)?
Executestheprocedurewiththegivenparameters.
Example:
CALLproc('a',1)
namedparameterlist::=
(<identifier><eq>(<gt>)?<expression>(<comma><identifier><eq>(<gt>)?<expression>)*)
Alistofnamedparameters.
Example:
param1=>'x',param2=>1
insertstatement::=
(INSERT|MERGE|UPSERT)INTO<identifier>(<columnlist>)?<queryexpression>(<optionclause>)?
Insertsvaluesintothegiventarget.
Example:
INSERTINTOtbl(col1,col2)VALUES('a',1)
BNFforSQLGrammar
705
expressionlist::=
<expression>(<comma><expression>)*
Alistofexpressions.
Example:
col1,'a',...
updatestatement::=
UPDATE<identifier>SET<setclauselist>(<whereclause>)?(<optionclause>)?
Updatevaluesinthegiventarget.
Example:
UPDATEtblSET(col1='a')WHEREcol2=1
deletestatement::=
DELETEFROM<identifier>(<whereclause>)?(<optionclause>)?
Deleterowsfromthegiventarget.
Example:
DELETEFROMtblWHEREcol2=1
queryexpression::=
(WITH<withlistelement>(<comma><withlistelement>)*)?<queryexpressionbody>
Adeclarativequeryfordata.
Example:
SELECT*FROMtblWHEREcol2=1
withlistelement::=
<identifier>(<columnlist>)?AS<lparen><queryexpression><rparen>
Aqueryexpressionforuseintheenclosingquery.
Example:
BNFforSQLGrammar
706
X(Y,Z)AS(SELECT1,2)
queryexpressionbody::=
<queryterm>((UNION|EXCEPT)(ALL|DISTINCT)?<queryterm>)*(<orderbyclause>)?(<limitclause>)?(<optionclause>)?
Thebodyofaqueryexpression,whichcanoptionallybeorderedandlimited.
Example:
SELECT*FROMtblORDERBYcol1LIMIT1
queryterm::=
<queryprimary>(INTERSECT(ALL|DISTINCT)?<queryprimary>)*
UsedtoestablishINTERSECTprecedence.
Example:
SELECT*FROMtbl
SELECT*FROMtbl1INTERSECTSELECT*FROMtbl2
queryprimary::=
<query>
(VALUES<lparen><expressionlist><rparen>(<comma><lparen><expressionlist><rparen>)*)
(TABLE<identifier>)
(<lparen><queryexpressionbody><rparen>)
Adeclarativesourceofrows.
Example:
TABLEtbl
SELECT*FROMtbl1
query::=
<selectclause>(<intoclause>)?(<fromclause>(<whereclause>)?(<groupbyclause>)?(<havingclause>)?)?
ASELECTquery.
BNFforSQLGrammar
707
Example:
SELECTcol1,max(col2)FROMtblGROUPBYcol1
intoclause::=
INTO<identifier>
Usedtodirectthequeryintoatable.{note}Thisisdeprecated.UseINSERTINTOwithaqueryexpressioninstead.{note}
Example:
INTOtbl
selectclause::=
SELECT(ALL|DISTINCT)?(<star>|(<selectsublist>(<comma><selectsublist>)*))
Thecolumnsreturnedbyaquery.Canoptionallybedistinct.
Example:
SELECT*
SELECTDISTINCTa,b,c
selectsublist::=
<selectderivedcolumn>
<allingroup>
Anelementintheselectclause
Example:
tbl.*
tbl.colASx
selectderivedcolumn::=
(<expression>((AS)?<identifier>)?)
Aselectclauseitemthatselectsasinglecolumn.{note}ThisisslightlydifferentthanaderivedcolumninthattheASkeywordisoptional.{note}
Example:
BNFforSQLGrammar
708
tbl.colASx
derivedcolumn::=
(<expression>(AS<identifier>)?)
Anoptionallynamedexpression.
Example:
tbl.colASx
allingroup::=
<allingroupidentifier>
Aselectsublistthatcanselectallcolumnsfromthegivengroup.
Example:
tbl.*
orderedaggregatefunction::=
(XMLAGG|ARRAY_AGG|JSONARRAY_AGG)<lparen><expression>(<orderbyclause>)?<rparen>
Anaggregatefunctionthatcanoptionallybeordered.
Example:
XMLAGG(col1)ORDERBYcol2
ARRAY_AGG(col1)
textaggreatefunction::=
TEXTAGG<lparen>(FOR)?<derivedcolumn>(<comma><derivedcolumn>)*(DELIMITER<character>)?((QUOTE<character>)|(NOQUOTE))?(HEADER)?(ENCODING<identifier>)?(<orderbyclause>)?<rparen>
Anaggregatefunctionforcreatingseparatedvalueclobs.
Example:
TEXTAGG(col1ast1,col2ast2DELIMITER','HEADER)
standardaggregatefunction::=
BNFforSQLGrammar
709
(COUNT<lparen><star><rparen>)
((COUNT|SUM|AVG|MIN|MAX|EVERY|STDDEV_POP|STDDEV_SAMP|VAR_SAMP|VAR_POP|SOME|ANY)<lparen>(DISTINCT|ALL)?<expression><rparen>)
Astandardaggregatefunction.
Example:
COUNT(*)
analyticaggregatefunction::=
(ROW_NUMBER|RANK|DENSE_RANK)<lparen><rparen>
Ananalyticaggregatefunction.
Example:
ROW_NUMBER()
filterclause::=
FILTER<lparen>WHERE<booleanprimary><rparen>
Anaggregatefilterclauseappliedpriortoaccumulatingthevalue.
Example:
FILTER(WHEREcol1='a')
fromclause::=
FROM(<tablereference>(<comma><tablereference>)*)
Aqueryfromclausecontainingalistoftablereferences.
Example:
FROMa,b
FROMarightouterjoinb,c,djoine".</p>
tablereference::=
(<escapedjoin><joinedtable><rbrace>)
<joinedtable>
Anoptionallyescapedjoinedtable.
BNFforSQLGrammar
710
Example:
a
ainnerjoinb
joinedtable::=
<tableprimary>(<crossjoin>|<qualifiedtable>)*
Atableorjoin.
Example:
a
ainnerjoinb
crossjoin::=
((CROSS|UNION)JOIN<tableprimary>)
Acrossjoin.
Example:
aCROSSJOINb
qualifiedtable::=
(((RIGHT(OUTER)?)|(LEFT(OUTER)?)|(FULL(OUTER)?)|INNER)?JOIN<tablereference>ON<condition>)
AnINNERorOUTERjoin.
Example:
ainnerjoinb
tableprimary::=
(<texttable>|<arraytable>|<xmltable>|<objecttable>|<tablename>|<tablesubquery>|(<lparen><joinedtable><rparen>))((MAKEDEP<makedepoptions>)|MAKENOTDEP)?((MAKEIND<makedepoptions>))?
Asinglesourceofrows.
Example:
BNFforSQLGrammar
711
a
makedepoptions::=
(<lparen>(MAX<colon><unsignedinteger>)?((NO)?JOIN)?<rparen>)?
optionsforthemakedephint
Example:
(min:10000)
xmlserialize::=
XMLSERIALIZE<lparen>(DOCUMENT|CONTENT)?<expression>(AS(STRING|VARCHAR|CLOB|VARBINARY|BLOB))?(ENCODING<identifier>)?(VERSION<string>)?((INCLUDING|EXCLUDING)XMLDECLARATION)?<rparen>
SerializesanXMLvalue.
Example:
XMLSERIALIZE(col1ASCLOB)
arraytable::=
ARRAYTABLE<lparen><valueexpressionprimary>COLUMNS<typedelementlist><rparen>(AS)?<identifier>
TheARRAYTABLEtablefunctioncreatestabularresultsfromarrays.Itcanbeusedasanestedtablereference.
Example:
ARRAYTABLE(col1COLUMNSxSTRING)ASy
texttable::=
TEXTTABLE<lparen><commonvalueexpression>(SELECTOR<string>)?COLUMNS<texttablecolumn>(<comma><texttablecolumn>)*((NOROWDELIMITER)|(ROWDELIMITER<character>))?(DELIMITER<character>)?((ESCAPE<character>)|(QUOTE<character>))?(HEADER(<unsignedinteger>)?)?(SKIP<unsignedinteger>)?(NOTRIM)?<rparen>(AS)?<identifier>
TheTEXTTABLEtablefunctioncreatestabularresultsfromtext.Itcanbeusedasanestedtablereference.
Example:
TEXTTABLE(fileCOLUMNSxSTRING)ASy
BNFforSQLGrammar
712
texttablecolumn::=
<identifier>((FORORDINALITY)|((HEADER<string>)?<datatype>(WIDTH<unsignedinteger>(NOTRIM)?)?(SELECTOR<string><unsignedinteger>)?))
Atexttablecolumn.
Example:
xINTEGERWIDTH6
xmlquery::=
XMLEXISTS<lparen>(<xmlnamespaces><comma>)?<string>(PASSING<derivedcolumn>(<comma><derivedcolumn>)*)?<rparen>
ExecutesanXQuerytoreturnanXMLresult.
Example:
XMLQUERY('<a>...</a>'PASSINGdoc)
xmlquery::=
XMLQUERY<lparen>(<xmlnamespaces><comma>)?<string>(PASSING<derivedcolumn>(<comma><derivedcolumn>)*)?((NULL|EMPTY)ONEMPTY)?<rparen>
ExecutesanXQuerytoreturnanXMLresult.
Example:
XMLQUERY('<a>...</a>'PASSINGdoc)
objecttable::=
OBJECTTABLE<lparen>(LANGUAGE<string>)?<string>(PASSING<derivedcolumn>(<comma><derivedcolumn>)*)?COLUMNS<objecttablecolumn>(<comma><objecttablecolumn>)*<rparen>(AS)?<identifier>
Returnstableresultsbyprocessingascript.
Example:
OBJECTTABLE('z'PASSINGvalASzCOLUMNScolOBJECT'teiid_row')ASX
objecttablecolumn::=
<identifier><datatype><string>(DEFAULT<expression>)?
objecttablecolumn.
BNFforSQLGrammar
713
Example:
yinteger'teiid_row_number'
xmltable::=
XMLTABLE<lparen>(<xmlnamespaces><comma>)?<string>(PASSING<derivedcolumn>(<comma><derivedcolumn>)*)?(COLUMNS<xmltablecolumn>(<comma><xmltablecolumn>)*)?<rparen>(AS)?<identifier>
ReturnstableresultsbyprocessinganXQuery.
Example:
XMLTABLE('/a/b'PASSINGdocCOLUMNScolXMLPATH'.')ASX
xmltablecolumn::=
<identifier>((FORORDINALITY)|(<datatype>(DEFAULT<expression>)?(PATH<string>)?))
XMLtablecolumn.
Example:
yFORORDINALITY
unsignedinteger::=
<unsignedintegerliteral>
Anunsignedintergervalue.
Example:
12345
tablesubquery::=
(TABLE|LATERAL)?<lparen>(<queryexpression>|<callstatement>)<rparen>(AS)?<identifier>
Atabledefinedbyasubquery.
Example:
(SELECT*FROMtbl)ASx
tablename::=
(<identifier>((AS)?<identifier>)?)
BNFforSQLGrammar
714
AtablenamedintheFROMclause.
Example:
tblASx
whereclause::=
WHERE<condition>
Specifiesasearchcondition
Example:
WHEREx='a'
condition::=
<booleanvalueexpression>
Abooleanexpression.
booleanvalueexpression::=
<booleanterm>(OR<booleanterm>)*
AnoptionallyORedbooleanexpression.
booleanterm::=
<booleanfactor>(AND<booleanfactor>)*
AnoptionalANDedbooleanfactor.
booleanfactor::=
(NOT)?<booleanprimary>
Abooleanfactor.
Example:
NOTx='a'
booleanprimary::=
BNFforSQLGrammar
715
(<commonvalueexpression>(<betweenpredicate>|<matchpredicate>|<likeregexpredicate>|<inpredicate>|<isnullpredicate>|<quantifiedcomparisonpredicate>|<comparisonpredicate>)?)
<existspredicate>
<xmlquery>
Abooleanpredicateorsimpleexpression.
Example:
colLIKE'a%'
comparisonoperator::=
<eq>
<ne>
<ne2>
<lt>
<le>
<gt>
<ge>
Acomparisonoperator.
Example:
=
comparisonpredicate::=
<comparisonoperator><commonvalueexpression>
Avaluecomparison.
Example:
='a'
subquery::=
<lparen>(<queryexpression>|<callstatement>)<rparen>
Asubquery.
Example:
(SELECT*FROMtbl)
BNFforSQLGrammar
716
quantifiedcomparisonpredicate::=
<comparisonoperator>(ANY|SOME|ALL)(<subquery>|(<lparen><expression><rparen>))
Asubquerycomparison.
Example:
=ANY(SELECTcolFROMtbl)
matchpredicate::=
(NOT)?(LIKE|(SIMILARTO))<commonvalueexpression>(ESCAPE<character>|(<lbrace>ESCAPE<character><rbrace>))?
Matchesbaseduponapattern.
Example:
LIKE'a_'
likeregexpredicate::=
(NOT)?LIKE_REGEX<commonvalueexpression>
Aregularexpressionmatch.
Example:
LIKE_REGEX'a.*b'
character::=
<string>
Asinglecharacter.
Example:
'a'
betweenpredicate::=
(NOT)?BETWEEN<commonvalueexpression>AND<commonvalueexpression>
Acomparisonbetweentwovalues.
Example:
BNFforSQLGrammar
717
BETWEEN1AND5
isnullpredicate::=
IS(NOT)?NULL
Anulltest.
Example:
ISNOTNULL
inpredicate::=
(NOT)?IN(<subquery>|(<lparen><commonvalueexpression>(<comma><commonvalueexpression>)*<rparen>))
Acomparisonwithmultiplevalues.
Example:
IN(1,5)
existspredicate::=
EXISTS<subquery>
Atestifrowsexist.
Example:
EXISTS(SELECTcolFROMtbl)
groupbyclause::=
GROUPBY(ROLLUP<lparen><expressionlist><rparen>|<expressionlist>)
Definesthegroupingcolumns
Example:
GROUPBYcol1,col2
havingclause::=
HAVING<condition>
Searchconditionappliedaftergrouping.
BNFforSQLGrammar
718
Example:
HAVINGmax(col1)=5
orderbyclause::=
ORDERBY<sortspecification>(<comma><sortspecification>)*
Specificesrowordering.
Example:
ORDERBYx,yDESC
sortspecification::=
<sortkey>(ASC|DESC)?(NULLS(FIRST|LAST))?
Defineshowtosortonaparticularexpression
Example:
col1NULLSFIRST
sortkey::=
<expression>
Asortexpression.
Example:
col1
integerparameter::=
<unsignedinteger>
<unsignedvalueexpressionprimary>
Aliteralintegerorparameterreferencetoaninteger.
Example:
?
limitclause::=
BNFforSQLGrammar
719
(LIMIT<integerparameter>(<comma><integerparameter>)?)
(OFFSET<integerparameter>(ROW|ROWS)(<fetchclause>)?)
<fetchclause>
Limitsand/oroffsetstheresultantrows.
Example:
LIMIT2
fetchclause::=
FETCH(FIRST|NEXT)(<integerparameter>)?(ROW|ROWS)ONLY
ANSIlimit.
Example:
FETCHFIRST1ROWSONLY
optionclause::=
OPTION(MAKEDEP<identifier><makedepoptions>(<comma><identifier><makedepoptions>)*|MAKEIND<identifier><makedepoptions>(<comma><identifier><makedepoptions>)*|MAKENOTDEP<identifier>(<comma><identifier>)*|NOCACHE(<identifier>(<comma><identifier>)*)?)*
Specifiesqueryoptions.
Example:
OPTIONMAKEDEPtbl
expression::=
<condition>
Avalue.
Example:
col1
commonvalueexpression::=
(<numericvalueexpression>((<double_amp_op>|<concat_op>)<numericvalueexpression>)*)
Establishestheprecedenceofconcat.
Example:
BNFforSQLGrammar
720
'a'||'b'
numericvalueexpression::=
(<term>(<plusorminus><term>)*)
Example:
1+2
plusorminus::=
<plus>
<minus>
The+or-operator.
Example:
+
term::=
(<valueexpressionprimary>(<starorslash><valueexpressionprimary>)*)
Anumericterm
Example:
1*2
starorslash::=
<star>
<slash>
The*or/operator.
Example:
/
valueexpressionprimary::=
<nonnumericliteral>
BNFforSQLGrammar
721
(<plusorminus>)?(<unsignednumericliteral>|(<unsignedvalueexpressionprimary>(<lsbrace><numericvalueexpression><rsbrace>)*))
Asimplevalueexpression.
Example:
+col1
parameterreference::=
<qmark>
(<dollar><unsignedinteger>)
Aparameterreferencetobeboundlater.
Example:
?
unescapedFunction::=
((<textaggreatefunction>|<standardaggregatefunction>|<orderedaggregatefunction>)(<filterclause>)?(<windowspecification>)?)|(<analyticaggregatefunction>(<filterclause>)?<windowspecification>)|(<function>(<windowspecification>)?)
(XMLCAST<lparen><expression>AS<datatype><rparen>)
nestedexpression::=
(<lparen>(<expression>(<comma><expression>)*)?(<comma>)?<rparen>)
Anexpressionnestedinparens
Example:
(1)
unsignedvalueexpressionprimary::=
<parameterreference>
(<escapedfunction><function><rbrace>)
<unescapedFunction>
<identifier>|<non-reservedidentifier>
<subquery>
<nestedexpression>
BNFforSQLGrammar
722
<ARRAYexpressionconstructor>
<searchedcaseexpression>
<caseexpression>
Anunsignedsimplevalueexpression.
Example:
col1
ARRAYexpressionconstructor::=
(ARRAY<lsbrace>(<expression>(<comma><expression>)*)?<rsbrace>)
Createsandarrayofthegivenexpressions.
Example:
----ARRAY[1,2]
----
windowspecification::=
OVER<lparen>(PARTITIONBY<expressionlist>)?(<orderbyclause>)?<rparen>
Thewindowspecificationforananalyticalorwindowedaggregatefunction.
Example:
OVER(PARTIONBYcol1)
caseexpression::=
CASE<expression>(WHEN<expression>THEN<expression>)+(ELSE<expression>)?END
If/then/elsechainusingacommonsearchpredicand.
Example:
CASEcol1WHEN'a'THEN1ELSE2
searchedcaseexpression::=
CASE(WHEN<condition>THEN<expression>)+(ELSE<expression>)?END
If/then/elsechainusingmultiplesearchconditions.
Example:
CASEWHENx='a'THEN1WHENy='b'THEN2
BNFforSQLGrammar
723
function::=
(CONVERT<lparen><expression><comma><datatype><rparen>)
(CAST<lparen><expression>AS<datatype><rparen>)
(SUBSTRING<lparen><expression>((FROM<expression>(FOR<expression>)?)|(<comma><expressionlist>))<rparen>)
(EXTRACT<lparen>(YEAR|MONTH|DAY|HOUR|MINUTE|SECOND)FROM<expression><rparen>)
(TRIM<lparen>((((LEADING|TRAILING|BOTH)(<expression>)?)|<expression>)FROM)?<expression><rparen>)
((TO_CHARS|TO_BYTES)<lparen><expression><comma><string>(<comma><expression>)?<rparen>)
((TIMESTAMPADD|TIMESTAMPDIFF)<lparen><timeinterval><comma><expression><comma><expression><rparen>)
<querystringfunction>
((LEFT|RIGHT|CHAR|USER|YEAR|MONTH|HOUR|MINUTE|SECOND|XMLCONCAT|XMLCOMMENT|XMLTEXT)<lparen>(<expressionlist>)?<rparen>)
((TRANSLATE|INSERT)<lparen>(<expressionlist>)?<rparen>)
<xmlparse>
<xmlelement>
(XMLPI<lparen>((NAME)?<identifier>)(<comma><expression>)?<rparen>)
<xmlforest>
<jsonobject>
<xmlserialize>
<xmlquery>
(<identifier><lparen>(ALL|DISTINCT)?(<expressionlist>)?(<orderbyclause>)?<rparen>(<filterclause>)?)
Callsascalarfunction.
Example:
func('1',col1)
xmlparse::=
XMLPARSE<lparen>(DOCUMENT|CONTENT)<expression>(WELLFORMED)?<rparen>
ParsesthegivenvalueasXML.
Example:
XMLPARSE(DOCUMENTdocWELLFORMED)
BNFforSQLGrammar
724
querystringfunction::=
QUERYSTRING<lparen><expression>(<comma><derivedcolumn>)*<rparen>
ProducesaURLquerystringfromthegivenarguments.
Example:
QUERYSTRING('path',col1ASopt,col2ASval)
xmlelement::=
XMLELEMENT<lparen>((NAME)?<identifier>)(<comma><xmlnamespaces>)?(<comma><xmlattributes>)?(<comma><expression>)*<rparen>
CreatesanXMLelement.
Example:
XMLELEMENT(NAME"root",child)
xmlattributes::=
XMLATTRIBUTES<lparen><derivedcolumn>(<comma><derivedcolumn>)*<rparen>
Createsattributesforthecontainingelement.
Example:
XMLATTRIBUTES(col1ASattr1,col2ASattr2)
jsonobject::=
JSONOBJECT<lparen><derivedcolumnlist><rparen>
ProducesaJSONobjectcontainingnamevaluepairs.
Example:
JSONOBJECT(col1ASval1,col2ASval2)
derivedcolumnlist::=
<derivedcolumn>(<comma><derivedcolumn>)*
alistofnamevaluepairs
Example:
BNFforSQLGrammar
725
col1ASval1,col2ASval2
xmlforest::=
XMLFOREST<lparen>(<xmlnamespaces><comma>)?<derivedcolumnlist><rparen>
Producesanelementforeachderivedcolumn.
Example:
XMLFOREST(col1ASELEM1,col2ASELEM2)
xmlnamespaces::=
XMLNAMESPACES<lparen><xmlnamespaceelement>(<comma><xmlnamespaceelement>)*<rparen>
DefinesXMLnamespaceURI/prefixcombinations
Example:
XMLNAMESPACES('http://foo'ASfoo)
xmlnamespaceelement::=
(<string>AS<identifier>)
(NODEFAULT)
(DEFAULT<string>)
Anxmlnamespace
Example:
NODEFAULT
simpledatatype::=
(STRING(<lparen><unsignedinteger><rparen>)?)
(VARCHAR(<lparen><unsignedinteger><rparen>)?)
BOOLEAN
BYTE
TINYINT
SHORT
SMALLINT
BNFforSQLGrammar
726
(CHAR(<lparen><unsignedinteger><rparen>)?)
INTEGER
LONG
BIGINT
(BIGINTEGER(<lparen><unsignedinteger><rparen>)?)
FLOAT
REAL
DOUBLE
(BIGDECIMAL(<lparen><unsignedinteger>(<comma><unsignedinteger>)?<rparen>)?)
(DECIMAL(<lparen><unsignedinteger>(<comma><unsignedinteger>)?<rparen>)?)
DATE
TIME
TIMESTAMP
(OBJECT(<lparen><unsignedinteger><rparen>)?)
(BLOB(<lparen><unsignedinteger><rparen>)?)
(CLOB(<lparen><unsignedinteger><rparen>)?)
(VARBINARY(<lparen><unsignedinteger><rparen>)?)
GEOMETRY
XML
Anon-collectiondatatype.
Example:
STRING
datatype::=
<simpledatatype>(<lsbrace><rsbrace>)*
Adatatype.
Example:
STRING[]
timeinterval::=
SQL_TSI_FRAC_SECOND
SQL_TSI_SECOND
BNFforSQLGrammar
727
SQL_TSI_MINUTE
SQL_TSI_HOUR
SQL_TSI_DAY
SQL_TSI_WEEK
SQL_TSI_MONTH
SQL_TSI_QUARTER
SQL_TSI_YEAR
Atimeintervalkeyword.
Example:
SQL_TSI_HOUR
nonnumericliteral::=
<string>
<binarystringliteral>
FALSE
TRUE
UNKNOWN
NULL
(<escapedtype><string><rbrace>)
((DATE|TIME|TIMESTAMP)<string>)
Anescapedorsimplenonnumericliteral.
Example:
'a'
unsignednumericliteral::=
<unsignedintegerliteral>
<approximatenumericliteral>
<decimalnumericliteral>
Anunsignednumericliteralvalue.
Example:
1.234
BNFforSQLGrammar
728
ddlstatement::=
<createtable>(<createtable>|<createprocedure>)?
<optionnamespace>
<alterStatement>
<createtrigger>
<createserver,akadatasource>
<createrole>
<droprole>
<CreateGRANT>
<RevokeGRANT>
<dropserver,akadatasource>
<droptable>
<Importforeignschema>
<ImportanotherDatabase>
<createdatabase>
<dropdatabase>
<usedatabase>
<dropschema>
<useschema>
<createschema>
<createprocedure>(<ddlstatement>)?
<createdatawrapperakatranslator>
<Dropdatawrapperakatranslator>
<dropprocedure>
Adatadefinitionstatement.
Example:
CREATEFOREIGNTABLEX(YSTRING)
optionnamespace::=
SETNAMESPACE<string>AS<identifier>
Anamespaceusedtoshortenthefullnameofanoptionkey.
Example:
SETNAMESPACE'http://foo'ASfoo
BNFforSQLGrammar
729
createdatabase::=
CREATEDATABASE<identifier>(VERSION<string>)?(<optionsclause>)?
createanewdatabase
Example:
CREATEDATABASEfoo[VERSION'version']OPTIONS(...)
usedatabase::=
USEDATABASE<identifier>(VERSION<string>)?
databaseintoworkingcontext
Example:
USEDATABASEfoo[VERSION'version']
dropdatabase::=
DROPDATABASE<identifier>(VERSION<string>)?
dropdatabase
Example:
DROPDATABASEfoo[VERSION'version']
createschema::=
CREATE(VIRTUAL)?SCHEMA<identifier>(SERVER<identifierlist>)?(<optionsclause>)?
createaschemaindatabase
Example:
CREATE[VIRTUAL]SCHEMAfooSERVER(s1,s2,s3)OPTIONS(...)
dropschema::=
DROP(VIRTUAL)?SCHEMA<identifier>
dropaschemaindatabase
Example:
BNFforSQLGrammar
730
----DROPSCHEMAfoo
----
useschema::=
SETSCHEMA<identifier>
useschemaforfollowingdatabaseresources
Example:
USESCHEMAfoo
createdatawrapperakatranslator::=
CREATEFOREIGN(DATAWRAPPER|TRANSLATOR)<identifier>(TYPE<identifier>)?(<optionsclause>)?
Definesatranslator;usetheoptionstooverridethetranslatorproperties.
Example:
CREATEFOREIGN(DATAWRAPPER|TRANSLATOR)wrapperOPTIONS(properties)
Dropdatawrapperakatranslator::=
DROPFOREIGN(DATAWRAPPER|TRANSLATOR)<identifier>
Deletesatranslator
Example:
DROPFOREIGN(DATAWRAPPER|TRANSLATOR)wrapper
createrole::=
CREATEROLE<identifier>(WITH<withrole>)?
Definesdataroleforthedatabase
Example:
CREATEDATAROLE<data-role>[WITHJAASROLE<string>(,<string>)*]
withrole::=
(JAASROLE<identifierlist>|ANYAUTHENTICATED)(WITH(JAASROLE<identifierlist>|ANYAUTHENTICATED))*
BNFforSQLGrammar
731
droprole::=
DROPROLE<identifier>
Removesdataroleforthedatabase
Example:
DROPROLE<data-role>
CreateGRANT::=
GRANT<granttype>(<comma><granttype>)*ON(DATABASE(<identifier>)?|TABLE<identifier>(CONDITION(CONSTRAINT)?<string>)?|PROCEDURE<identifier>(CONDITION(CONSTRAINT)?<string>)?|SCHEMA<identifier>|COLUMN<identifier>(MASK(ORDER<unsignedinteger>)?<string>)?(CONDITION(CONSTRAINT)?<string>)?)TO<identifier>
DefinesGRANTforarole
Example:
GRANT(SELECT|INSERT|UPDATE|DELETE|EXECUTE|LANGUAGE|ALTER|DROP|ALLPRIVILEGES|TEMPORARYTAB
LES)
'''
===[[revokeGrantOption]]_<<usage_revokeGrantOption,RevokeGRANT>>_::=
*<<token_REVOKE,REVOKE>><<token_GRANT,GRANT>><<<readGrantTypes,granttype>>>(<<<token_COMMA,comma>>><<<re
adGrantTypes,granttype>>>)*<<token_ON,ON>>(<<token_DATABASE,DATABASE>>(<<<id,identifier>>>)?|<<token_
TABLE,TABLE>><<<id,identifier>>>(<<token_CONDITION,CONDITION>>(<<token_CONSTRAINT,CONSTRAINT>>)?<<<strin
gVal,string>>>)?|<<token_PROCEDURE,PROCEDURE>><<<id,identifier>>>(<<token_CONDITION,CONDITION>>(<<token
_CONSTRAINT,CONSTRAINT>>)?<<<stringVal,string>>>)?|<<token_SCHEMA,SCHEMA>><<<id,identifier>>>|<<token_C
OLUMN,COLUMN>><<<id,identifier>>>(<<token_MASK,MASK>>(<<token_ORDER,ORDER>><<<intVal,unsignedinteger>>>
)?<<<stringVal,string>>>)?(<<token_CONDITION,CONDITION>>(<<token_CONSTRAINT,CONSTRAINT>>)?<<<stringVal,
string>>>)?)<<token_FROM,FROM>><<<id,identifier>>>
RevokesGRANTforarole
Example:
[source,sql]
REVOKEGRANT(SELECT|INSERT|UPDATE|DELETE|EXECUTE|LANGUAGE|ALTER|DROP|ALLPRIVILEGES|TEMPORARYTABLES)
createserver,akadatasource::=
CREATESERVER<identifier>(TYPE<string>)?(VERSION<string>)?FOREIGN(DATAWRAPPER|TRANSLATOR)<identifier>(<optionsclause>)?
Definesconnectiontoforeignsource
Example:
BNFforSQLGrammar
732
CREATESERVERserver_name[TYPE'server_type'][VERSION'server_version']FOREIGN(<DATA><WRAPPER>|<TRANSL
ATOR>)fdw_name[OPTIONS(option'value'[,...])]
dropserver,akadatasource::=
DROPSERVER<identifier>
Definesdroppingconnectiontoforeignsource
Example:
----DROPSERVERserver_name
----
createprocedure::=
CREATE(VIRTUAL|FOREIGN)?(PROCEDURE|FUNCTION)<identifier>(<lparen>(<procedureparameter>(<comma><procedureparameter>)*)?<rparen>(RETURNS(<optionsclause>)?(((TABLE)?<lparen><procedureresultcolumn>(<comma><procedureresultcolumn>)*<rparen>)|<datatype>))?(<optionsclause>)?(AS<statement>)?)
Definesaprocedureorfunctioninvocation.
Example:
CREATEFOREIGNPROCEDUREproc(paramSTRING)RETURNSSTRING
dropprocedure::=
DROP(VIRTUAL|FOREIGN)?(PROCEDURE|FUNCTION)<identifier>
Dropsatableorview.
Example:
DROP[FOREIGN(TABLE|VIEW)table-name
procedureparameter::=
(IN|OUT|INOUT|VARIADIC)?<identifier><datatype>(NOTNULL)?(RESULT)?(DEFAULT<expression>)?(<optionsclause>)?
Aprocedureorfunctionparameter
Example:
OUTxINTEGER
BNFforSQLGrammar
733
procedureresultcolumn::=
<identifier><datatype>(NOTNULL)?(<optionsclause>)?
Aprocedureresultcolumn.
Example:
xINTEGER
createtable::=
CREATE((FOREIGNTABLE)|((VIRTUAL)?VIEW)|(GLOBALTEMPORARYTABLE))<identifier>(<createtablebody>|(<optionsclause>)?)(AS<queryexpression>)?
Definesatableorview.
Example:
CREATEVIEWvwASSELECT1
droptable::=
DROP((FOREIGNTABLE)|((VIRTUAL)?VIEW)|(GLOBALTEMPORARYTABLE))<identifier>
Dropsatableorview.
Example:
DROP(FOREIGNTABLE|[VIRTUAL]VIEW)table-name
createforeigntemptable::=
CREATE(LOCAL)?FOREIGNTEMPORARYTABLE<identifier><createtablebody>ON<identifier>
Definesaforeigntemptable
Example:
CREATEFOREIGNTEMPORARYTABLEt(xstring)ONz
createtablebody::=
<lparen><tableelement>(<comma><tableelement>)*(<comma>(CONSTRAINT<identifier>)?(<primarykey>|<otherconstraints>|<foreignkey>)(<optionsclause>)?)*<rparen>(<optionsclause>)?
Definesatable.
Example:
(xstring)OPTIONS(CARDINALITY100)
BNFforSQLGrammar
734
foreignkey::=
FOREIGNKEY<columnlist>REFERENCES<identifier>(<columnlist>)?
Definestheforeignkeyreferentialconstraint.
Example:
FOREIGNKEY(a,b)REFERENCEStbl(x,y)
primarykey::=
PRIMARYKEY<columnlist>
Definestheprimarykey.
Example:
PRIMARYKEY(a,b)
otherconstraints::=
((UNIQUE|ACCESSPATTERN)<columnlist>)
(INDEX<lparen><expressionlist><rparen>)
DefinesACCESSPATTERNandUNIQUEconstraintsandINDEXes.
Example:
UNIQUE(a)
columnlist::=
<lparen><identifier>(<comma><identifier>)*<rparen>
Alistofcolumnnames.
Example:
(a,b)
tableelement::=
<identifier>(SERIAL|(<datatype>(NOTNULL)?(AUTO_INCREMENT)?))((PRIMARYKEY)|((UNIQUE)?(INDEX)?))(DEFAULT<expression>)?(<optionsclause>)?
BNFforSQLGrammar
735
Definesatablecolumn.
Example:
xINTEGERNOTNULL
optionsclause::=
OPTIONS<lparen><optionpair>(<comma><optionpair>)*<rparen>
Alistofstatementoptions.
Example:
OPTIONS('x''y','a''b')
optionpair::=
<identifier>(<nonnumericliteral>|(<plusorminus>)?<unsignednumericliteral>)
Anoptionkey/valuepair.
Example:
'key''value'
alteroptionpair::=
<identifier>(<nonnumericliteral>|(<plusorminus>)?<unsignednumericliteral>)
AlterAnoptionkey/valuepair.
Example:
'key''value'
alterStatement::=
ALTER(VIRTUAL|FOREIGN)?(<ALTERTABLE>|<ALTERPROCEDURE>|<ALTERTRIGGER>|<ALTERSERVER>|<ALTERDATAWRAPPER>|<ALTERDATABASE>)
ALTERTABLE::=
(VIEW<identifier>AS<queryexpression>(<alteroptionslist>|<altercolumnoptions>)?)
(TABLE<identifier>(<alteroptionslist>|<ADDcolumn>|<DROPcolumn>|<altercolumnoptions>|<renamecolumnoptions>)?)
BNFforSQLGrammar
736
altersoptionsofdatabase
Example:
ALTERTABLEfoo(ADD|DROP|ALTER)COLUMN<name><type>OPTIONS((ADD|SET|DROP)xy)
ADDcolumn::=
ADDCOLUMN<tableelement>
alterstableandaddsacolumn
Example:
ADDCOLUMNbartypeOPTIONS(ADDupdatabletrue)
DROPcolumn::=
DROPCOLUMN<identifier>
alterstableandaddsacolumn
Example:
----DROPCOLUMNbar
----
altercolumnoptions::=
ALTER(COLUMN|PARAMETER)<identifier>(TYPE(SERIAL|(<datatype>(NOTNULL)?(AUTO_INCREMENT)?))|<alterchildoptionslist>)
altersasetofcolumnoptions
Example:
ALTERCOLUMNbarOPTIONS(ADDupdatabletrue)
renamecolumnoptions::=
RENAME(COLUMN|PARAMETER)<identifier>TO<identifier>
renameseitheratablecolumnorprocedure’sparametername
Example:
RENAMECOLUMNbarTOfoo
BNFforSQLGrammar
737
ALTERPROCEDURE::=
PROCEDURE<identifier>(AS<statement>)?(<alteroptionslist>|<altercolumnoptions>|<renamecolumnoptions>)?
altersoptionsofdatabase
Example:
ALTERPROCEDUREfoo[AS<stmt>]OPTIONS(ADDxy)
ALTERTRIGGER::=
TRIGGERON<identifier>INSTEADOF(INSERT|UPDATE|DELETE)(AS<foreachrowtriggeraction>|ENABLED|DISABLED)
altersoptionsoftabletriggers
Example:
ALTERTRIGGERON<id>INSTEADOF(INSERT|UPDATE|DELETE)AS[ENABLED|DISABLED]
ALTERSERVER::=
SERVER<identifier><alteroptionslist>
altersoptionsofdatabase
Example:
ALTERSERVERfooOPTIONS(ADDxy)
ALTERDATAWRAPPER::=
(DATAWRAPPER|TRANSLATOR)<identifier><alteroptionslist>
altersoptionsofdatawrapper
Example:
ALTER[DATAWRAPPER|TRANSLATOR]fooOPTIONS(ADDxy)
ALTERDATABASE::=
DATABASE<identifier><alteroptionslist>
altersoptionsofdatabase
Example:
ALTERDATABASEfooOPTIONS(ADDxy)
BNFforSQLGrammar
738
alteroptionslist::=
OPTIONS<lparen>(<addsetoption>|<dropoption>)(<comma>(<addsetoption>|<dropoption>))*<rparen>
alistofalterationstooptions
Example:
OPTIONS(ADDupdatabletrue)
dropoption::=
DROP<identifier>
dropoption
Example:
DROPupdatable
addsetoption::=
(ADD|SET)<alteroptionpair>
addorsetanoptionpair
Example:
ADDupdatabletrue
alterchildoptionslist::=
OPTIONS<lparen>(<addsetchildoption>|<dropoption>)(<comma>(<addsetchildoption>|<dropoption>))*<rparen>
alistofalterationstooptions
Example:
OPTIONS(ADDupdatabletrue)
dropoption::=
DROP<identifier>
dropoption
Example:
BNFforSQLGrammar
739
DROPupdatable
addsetchildoption::=
(ADD|SET)<alterchildoptionpair>
addorsetanoptionpair
Example:
ADDupdatabletrue
alterchildoptionpair::=
<identifier>(<nonnumericliteral>|(<plusorminus>)?<unsignednumericliteral>)
AlterAnoptionkey/valuepair.
Example:
'key''value'
Importforeignschema::=
IMPORTFOREIGNSCHEMA<identifier>(LIMITTO<lparen><identifierlist><rparen>|<lparen>EXCEPT<identifierlist><rparen>)?FROM(SERVER|REPOSITORY)<identifier>INTO<identifier>(<optionsclause>)?
importsschemametadatafromserver
Example:
IMPORTFOREIGNSCHEMAfoo[LIMITTO(x,y,z)|EXCEPT(x,y,z)]FROMSERVERbar
ImportanotherDatabase::=
IMPORTDATABASE<identifier>VERSION<string>(WITHACCESSCONTROL)?
importsanotherdatabaseintocurrentdatabase
Example:
IMPORTDATABASE<id>VERSION<string-val>[WITHACCESSCONTROL]
identifierlist::=
<identifier>(<comma><identifier>)*
BNFforSQLGrammar
740
granttype::=
SELECT
INSERT
UPDATE
DELETE
EXECUTE
LANGUAGE
ALTER
DROP
ALLPRIVILEGES
TEMPORARYTABLE
BNFforSQLGrammar
741
SecurityGuideTheTeiidsystemprovidesarangeofbuilt-inandextensiblesecurityfeaturestoenablesecuredataaccess.Thisintroductionprovidesahigh-levelguidetosecurityconcerns.Therestoftheguideprovidesspecificsonconfiguringclients,theTeiidserver,andtheapplicationserver.
Authentication
ClientAuthentication
JDBC/ODBC/WebServiceclientsmayusesimplepasswordstoauthenticateauser.
Typicallyausernameisrequired,howeverusernamesmaybeconsideredoptionaliftheidentityoftheusercanbediscernedbythepasswordcredentialalone.Inanycaseitisuptotheconfiguredsecuritydomaintodeterminewhetherausercanbeauthenticated.Ifyouneedauthentication,theadministratormustconfigureLoginModulesforTeiid.
CautionBydefault,accesstoTeiidisNOTsecure.ThedefaultLoginModulesareonlybackedbyfilebasedauthentication,whichhasawellknownusernameandpassword.WeDONOTrecommendleavingthedefaultsecurityprofileasdefinedwhenyouareexposingsensitivedata.
TeiidJDBC/ODBCalsosupportsKerberosauthenticationwithadditionalconfiguration.
Auto-generatedwebservices,suchasOData,forconsumingTeiidtypicallysupportHTTPBasicauthentication,whichinturnshouldutilizePass-throughAuthentication.
SourceAuthentication
SourceauthenticationisgenerallydeterminedbythecapabilitiesofJCAresourceadaptersusedtoconnecttoexternalresources.ConsulttheASJCAdocumentationforthecapabilitiesofsourcepoolingandsuppliedresourceadaptersformoreinformation.Typicallyasingleusername/passwordcredentialissupported,suchaswhencreatingJDBCDataSources.Inmoreadvancedusagescenariosthesourceand/ortranslatormaybeconfiguredorcustomizedtouseanexecutionpayload,theTeiidsubject,oreventhecallingapplicationsubjectviaPass-throughAuthentication.SeealsoDevelopingJEEConnectorsandTranslatorDevelopment
Pass-throughAuthentication
Ifyourclientapplication(webapplicationorWebservice)residesinthesameWildFlyinstanceasTeiidandtheclientapplicationusesasecuritydomain,thenyoucanconfigureTeiidtousethesamesecuritydomainandnotforcetheusertore-authenticate.Inpass-throughmodeTeiidlooksforanauthenticatedsubjectinthecallingthreadcontextandusesitforsessioningandauthorization.ToconfigureTeiidforpass-throughauthentication,changetheTeiidsecurity-domainnametothesamenameasyourapplication’ssecuritydomainname.ThischangecanbemadeviatheCLIorinthestandalone-teiid.xmlfileifrunninginstandalonemode.ThesecuritydomainmustbeaJAASbasedLoginModuleandyourclientapplicationMUSTobtainitsTeiidconnectionusingaLocalConnectionwiththe_PassthroughAuthentication=trueconnectionflagset.Youmayalsosetthesecurity-domainontheVDB.
Authorization
Authorizationcoversbothadministrativeactivitiesanddataroles.Adataroleisacollectionofpermissions(alsoreferredtoasentitlements)andacollectionofentitledprincipalsorgroups.WiththedeploymentofaVDBthedeployercanchoosewhichprincipalsandgroupshavewhichdataroles.CheckoutReferenceGuideDataRoleschapterformoreinformation.Anysourcelevelauthorizationdecisionsareuptothesourcesystemsbeingintegrated.
SecurityGuide
742
VDBswithoutdatarolesdefinedareaccessiblebyanyauthenticateduser.Ifyouwanttoensuresomeattempthasbeenmadeatsecuringaccess,thensetthedata-roles-requiredconfigurationelementtotrueviatheCLIorinthestandalone.xmlontheteiidsubsystem.
Encryption
TeiidTransports
Teiidprovidesbuilt-insupportforJDBC/ODBCoverSSL.JDBCdefaultstojustsensitivemessageencryption(loginmode),whileODBC(thepgtransport)defaultstojustcleartextpasswordsifusingsimpleusername/passwordauthentication.
TheASinstancemustbeconfiguredforSSLaswellsothatAnywebservicesconsumingTeiidmayuseSSL.
Configuration
Passwordsinconfigurationfilesarebydefaultstoredinplaintext.Ifyouneedthesevaluestobeencrypted,pleaseseeencryptingpasswordsforinstructionsonencryptionfacilitiesprovidedbythecontainer.
SourceAccess
Encryptingremotesourceaccessistheresponsibilityfortheresourceadapterandlibrary/driverusedtoaccessthesourcesystem.
TemporaryData
TeiidtemporarydatawhichcanbestoredonthefilesystemasconfiguredbytheBufferManagermayoptionallybeencrypted.Setthebuffer-service-encrypt-filespropertytotrueontheTeiidsubsystemtouse128-bitAEStoencryptanyfileswrittenbytheBufferManager.AnewsymmetrickeywillbegeneratedforeachstartoftheTeiidsystemoneachserver.Aperformancehitwillbeseenforprocessingthatismemoryintensivesuchthatdatatypicallyspillstodisk.ThissettingdoesnotaffecthowVDBs(eithertheartifactoranexplodedform)orlogfilesarewrittentodisk.
SecurityGuide
743
LoginModules
LoginModulesareanessentialpartoftheJAASsecurityframeworkandprovideTeiidcustomizableuserauthenticationandtheabilitytoreuseexistingLoginModulesdefinedforWildFly.RefertotheWildFlysecuritydocumentationforinformationaboutconfiguringsecurityinWildFly,http://docs.jboss.org/jbossas/admindevel326/html/ch8.chapter.html.
TeiidcanbeconfiguredwithmultiplenamedapplicationpoliciesthatgrouptogetherrelevantLoginModules.Thesesecurity-domainnamescanbereferencedonapervdb.
Thesecurity-domainattributeundertheauthenticationelementinteiidsubsysteminthe<jboss-install>/standalone/configuration/standalone-teiid.xmlfileisusedsetthesecurity-domainname.Forexample,indefaultconfigurationunderteiidsubsystemyouwillfind
<authenticationsecurity-domain="teiid-security"/>
<transportname="jdbc"protocol="teiid"socket-binding="teiid-jdbc">
<sslmode="login"/>
</transport>
Ifnodomaincanauthenticatetheuser,theloginattemptwillfail.Detailsofthefailedattemptincludinginvalidusers,whichdomainswereconsulted,etc.willbeintheserverlogwithappropriatelevelsofseverity.
security-domaininVDB
AVDBcanbeconfiguredtouseasecurity-domainotherthantheTeiiddefaultsecurity-domain.Thisconfigurationisdefinedinthevdb.xmlfile,seeVDBPropertiesformoreinformation.Thesecurity-domaindefinedontransportconfigurationwillbeusedasdefaultsecurity-domain,ifasecurity-domainisnotconfiguredforaspecificVDB.
<vdbname="vdb"version="1">
<propertyname="security-domain"value="custom-security"/>
...
</vdb>
TipInexistinginstallationsanappropriatesecuritydomainmayalreadybeconfiguredforusebyadministrativeclients(typicallyforadmin-console).Iftheadminconnections(CLIandadminshell)arenotsecured,itisrecommendedthatyousecurethatinterfacebyexecutingadd-user.shscriptinthebin/scriptsdirectory.
Built-inLoginModules
JBossASprovidesseveralLoginModulesforcommonauthenticationneeds,suchasauthenticatingfromaTextBasedLoginModuleoraLDAPBasedLoginModule.
Youcaninstallmultipleloginmodulesaspartofsinglesecuritydomainconfigurationandconfigurethemtobepartoftheloginprocess.Forexample,forteiid-securitydomain,youcanconfigureafilebasedandalsoLDAPbasedloginmodules,andhaveyouruserauthenticatedwitheitherorbothloginmodules.Ifyouwanttowriteyourowncustomloginmodule,refertotheDeveloper’sGuideforinstructions.
Foralltheavailableloginmodulesrefertohttp://community.jboss.org/docs/DOC-11287.
RealmBasedLoginModule
LoginModules
744
TheRealmDirectLoginModuleutilizesacommonsecurityrealmacrossinstalledWildFly/EAPinstancedefinedbydefaultApplicationRealmtoperformauthenticationandauthorization.TousethissecurityrelamaddthefollowingXMLunder"security"subsysteminstandalone-teiid.xmlordomain.xml
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="teiid-security"cache-type="default">
<authentication>
<login-modulecode="RealmDirect"flag="required">
<module-optionname="password-stacking"value="useFirstPass"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
</subsystem>
Whenusingthissecuritydomain,use<wildfly>/bin/add-user.shor<wildfly>/bin/add-user.batscriptstoadd/updateauserin"ApplicationRelam".Whenusingthisrelam,thepasswordasstoredinencryptedform.Thisisthedefaultsecuritymodulethatisused.
TextBasedLoginModuleTheUsersRolesLoginModuleutilizessimpletextfilestoauthenticateusersandtodefinetheirgroups.TousethisaddthefollowingXMLunder"security"subsysteminstandalone-teiid.xmlordomain.xml
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="teiid-security"cache-type="default">
<authentication>
<login-modulecode="UsersRoles"flag="required">
<module-optionname="usersProperties"value="$(jboss.server.config.dir)/users.properties"/>
<module-optionname="rolesProperties"value="$(jboss.server.config.dir)/roles.properties"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
</subsystem>
Warning TheUsersRolesLoginModuleisnotrecommendedforproductionuseandisstronglyrecommendedthatyoureplacethisloginmodule.
Peraboveconfiguration,Usernamesandpasswordsarestoredinthe<wildfly>/standalone/configuration/users.propertiesfile,anexampleuser.propertiesfilelookslikebelow
users.properties
#Ausers.propertiesfileforusewiththeUsersRolesLoginModule
#username=password
fred=password
george=password
...
LoginModules
745
Theroleassignmentsarestoredinthe<wildfly>/standalone/configuration/roles.propertiesfile,anexampleroles.propertiesfilelookslikebelow
roles.properties
#Aroles.propertiesfileforusewiththeUsersRolesLoginModule
#username=role1,role2,...
data_role_1=fred,sally
data_role_2=george
Userandrolenamesareentirelyuptotheneedsofthegivendeployment.ForexampleeachapplicationteamcansettheirownsecurityconstraintsfortheirVDBs,bymappingtheirVDBdatarolestoapplicationspecificJAASroles,e.g.app_role_1=user1,user2,user3.
Note Whenyouconfigurethissecuritydomain,youmustprovidetheemptyuser.propertiesandroles.propertiesfilesatthecorrectpathdefinedintheconfiguration,otherwisetheinitializationofsecuritydomainwillendupinfailure.
Note TeiiddatarolesnamesareindependentofJAASroles.VDBcreatorscanchoosewhatevernametheywantfortheirdataroles,whicharethenmappedatdeploymenttimetoJAASroles.
LDAPBasedLoginModuleFormorecompleteinformationtoconfigureaLDAPbasedloginmoduleconsultEAPdocumentation
ConfigureLDAPauthenticationbyeditingstandalone-teiid.xmlunder'security'subsystem.Oncethesecurity-domainisdefined,theneditthe'security-domain'attributeforTeiid’s'transport'forwhichyouwantusethisLDAPlogin.
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="ldap_security_domain">
<authentication>
<login-modulecode="LdapExtended"flag="required">
<module-optionname="java.naming.factory.initial"value="com.sun.jndi.ldap.LdapCtxFactory"
/>
<module-optionname="java.naming.provider.url"value="ldap://mydomain.org:389"/>
<module-optionname="java.naming.security.authentication"value="simple"/>
<module-optionname="bindDN"value="myuser"/>
<module-optionname="bindCredential"value="mypasswd"/>
<module-optionname="baseCtxDN"value="ou=People,dc=XXXX,dc=ca"/>
<module-optionname="baseFilter"value="(cn={0})"/>
<module-optionname="rolesCtxDN"value="ou=Webapp-Roles,ou=Groups,dc=XXXX,dc=ca"/>
<module-optionname="roleFilter"value="(member={1})"/>
<module-optionname="uidAttributeID"value="member"/>
<module-optionname="roleAttributeID"value="cn"/>
<module-optionname="roleAttributeIsDN"value="true"/>
<module-optionname="roleNameAttributeID"value="cn"/>
<module-optionname="roleRecursion"value="-1"/>
<module-optionname="searchScope"value="ONELEVEL_SCOPE"/>
<module-optionname="allowEmptyPasswords"value="false"/>
<module-optionname="throwValidateError"value="true"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
</subsystem>
LoginModules
746
Note IfusingSSLtotheLDAPserver,ensurethattheCorporateCACertificateisaddedtotheJREtruststore.
Note SometimesroleinformationisDN,thenyouwillrequiretheproperty"parseRoleNameFromDN=true".
DatabaseLoginModule
ForinformationtoconfigureaDatabasebasedloginmoduleconsultEAPdocumentation
CertLoginModule
FormorecompleteinformationtoconfigureaCertificatebasedloginmoduleconsultEAPdocumentation
RoleMappingLoginModule
IftheLoginModuleyouareusingexposesrolenamesthatyouwishtomaptomoreapplicationspecificnames,thenyoucanusetheRoleMappingLoginModule.Thisusesapropertiesfiletoinjectadditionalrolenames,andoptionallyreplacetheexistingrole,onauthenticatedsubjects.
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="ldap_security_domain">
<authentication>
...
<login-modulecode="org.jboss.security.auth.spi.RoleMappingLoginModule"flag="optional">
<module-optionname="rolesProperties"value="${jboss-install}/standalone/configuration/role
s.properties"/>
<module-optionname="replaceRole"value="false"/>
</login-module>
...
</authentication>
</security-domain>
</security-domains>
</subsystem>
CustomLoginModules
IfyourauthenticationneedsgobeyondtheprovidedLoginModules,pleaserefertotheJAASdevelopmentguideathttp://java.sun.com/j2se/1.5.0/docs/guide/security/jaas/JAASLMDevGuide.html.Therearealsonumerousguidesavailable.
Ifyouareextendingoneofthebuilt-inLoginModules,refertohttp://community.jboss.org/docs/DOC-9466.
LoginModules
747
TeiidServerTransportSecurityTherearetwotypesofremotetransports,eachwithit’sownencryptionconfiguration:
"teiid"-Defaultstoonlyencryptlogintraffic,inwhichnoneoftheotherconfigurationpropertiesareused.
"pg"-DefaultstonoSSL
Warning ThepgtransportforODBCaccessdefaultstocleartextusernamepasswordauthentication.Youshouldconsiderusingasecuritydomainthatutilizesnon-plaintextpasswords,kerberos,orSSL.
SSLconfigurationispartofthetransportconfigurationintheTeiidsubsystem.
EncryptionModesTeiidsupportsacoupledifferentencryptionmodesbasedonthemodeattributeonsslelement.
logIn-Thisisthedefaultsettingforthetransports.
JDBC(non-databydefault)messagesbetweenclientandserverareencryptedusing128bitAESwithaDiffie-Hellmankeythatisnegotiatedperconnection.Whenpossiblea2048bitkeyexchangewillbeusedotherwise1024bitwillbeused.Oracle/Sun1.7JREsareknownnottosupportkeylengthsover1024bits.TheconnectionpropertyencryptRequestcanbeusedtoencryptrequestsandresultsusingthesame128AESscheme.
Forthepgtransportauthenticationisexpectedtobesecure-whichcurrentlyisonlyGSSlogins.Pre9.xandunpatchedclient/servercombinationswillusealesssecureECBblockmode,whichisnotrecommendedforlargeauthenticationpayloadsandtheencryptRequestoption.
enabled-ModetoenableSSL.ClientsarerequiredtoconnectusingSSL.
disabled-turnsoffanykindofencryption.Thisisthedefaultforthepgtransport.
SSLAuthenticationModesanonymous–Nocertificatesarerequired,butallcommunicationsarestillencryptedusingtheTLS_DH_anon_WITH_AES_128_CBC_SHASSLciphersuite.Inmostsecureintranetenvironments,anonymousissuitabletojustbulkencrypttrafficwithouttheneedtosetupSSLcertificates.Nocertificatesareexchanged,andsettingsarenotneededforthekeystoreandtruststoreproperties.JDBCClientsmusthave'org.teiid.ssl.allowAnon'settotrue(thedefault)toconnecttoananonymousserver.
Note
ODBCclientsandsomeVMs,suchasIBM,maynothavetheTLS_DH_anon_WITH_AES_128_CBC_SHAciphersuiteavailable.Whentheclientorserverlacktheanonymousciphersuite,considerusing1-waywithaself-signedcertificate.ODBCclientstypicallydonotrequireservercertificatevalidation.TeiidJDBCclientsbydefaultvalidatetheservercertificate,butcanusetheorg.teiid.ssl.trustAllpropertytoacceptanyservercertificate.
1-way–Thedefault.Onlyauthenticatestheservertotheclient.Requiresaprivatekeykeystoretobecreatedfortheserver.Iftheclientisconfiguredtovalidatetheservercertificate,theclientwillneedanappropriatetruststoreconfigured.
2-way–Mutualclientandserverauthentication.Theserverandclientapplicationseachhaveakeystorefortheirprivatekeysandeachhasatruststorethatauthenticatestheother.Theserverwillpresentacertificate,whichisobtainedfromthekeystorerelatedproperties.Theclientshouldhaveatruststoreconfiguredtoaccepttheservercertificate.Theclientisalsoexpectedtopresentacertificate,whichisobtainedfromitskeystore.Theclientcertificateshouldbeacceptedbythetruststoreconfiguredbythetruststorerelatedproperties.
TeiidServerTransportSecurity
748
Fornon-anonymousSSL,thesuiteisnegotiated-seeenabled-cipher-suitesbelowbelow.
DependingupontheSSLmode,followtheguidelinesofyourorganizationaroundcreating/obtainingprivatekeys.Ifyouhavenoorganizationalrequirements,thenfollowthisguidetocreateself-signedcertificateswiththeirrespectivekeystoresandtruststores.ThefollowingkeystoreandtruststorecombinationsarerequiredfordifferentSSLmodes.Thenamesofthefilescanbechosenbytheuser.Thefollowingfilesareshownforexamplepurposesonly.
1-way
1. server.keystore-hasserver’sprivatekey
2. server.truststore-hasserver’spublickey
2-way
1. server.keystore-hasserver’sprivatekey
2. server.truststore-hasserver’spublickey
3. client.keystore-client’sprivatekey
4. client.truststore-hasclient’spublickey
FullConfigurationOptions
ExampleXMLConfiguration
<sslmode="enabled"authentication-mode="1-way"ssl-protocol="TSLv1"keymanagement-algorithm="algo"
enabled-cipher-suites="SSL_RSA_WITH_RC4_128_MD5,SSL_RSA_WITH_RC4_128_SHA">
<keystorename="cert.keystore"password="passwd"type="JKS"key-alias="alias"key-password="passwd1"
/>
<truststorename="cert.truststore"password="passwd"/>
</ssl>
Properties
mode-diabled|login|enableddisabled=notransportormessagelevelsecuritywillbeused.login=onlythelogintrafficwillbeencryptedatamessagelevelusing128bitAESwithanephemeralDHkeyexchange.Onlyappliestotheteiidtransportandnootherconfigvaluesareneededinthismode.enabled=trafficwillbesecuredwithSSLusingtheotherconfigurationproperties.teiidtransportclientsmustconnectusingSSLwiththemmsprotocol.ODBC"pg"transportclientsmayoptionallyuseSSL.
ssl-protocol-TypeofSSLprotocoltobeused.Optional-bydefaultTLSv1.
Caution SSLv3isnotrecommendedduetothePOODLEsecurityvulnerability.
keystore/type-Keystoretypecreatedbythekeytool.Optional-bydefault"JKS"isused.
authentication-mode-anonymous|1-way|2-way,TypeofSSLAuthenticationMode.
keymanagement-algorithm-Typeofkeyalgorithmused.Optional-bydefaultisbasedupontheVM,e.g."SunX509"
keystore/name-Thefilenameofthekeystore,whichcontainstheprivatekeyoftheServer.ThefilenamecanberelativeresourcepathavailabletotheTeiiddeployerclassloaderoranabsolutefilesystempath.AtypicalinstallationwouldplacethekeystorefileintheconfdirectoryoftheprofilewhereTeiidisdeployedwithafilenamerelativetotheconfpath.Typicallyrequiredif1-wayor2-wayauthenticationisused.
keystore/password-passwordforthekeystore.Requiredifthekeystorehasapassword.
keystore/key-alias-Aliasnamefortheprivatekeytouse.Optional-onlyneedediftherearemultipleprivatekeysinthekeystoreandyouneedtochoosewhichonetouse.
TeiidServerTransportSecurity
749
keystore/key-password-Aliasnamefortheprivatekeytouse.Optional-onlyneededifthekeypasswordisdifferentthanthekeystorepassword.
truststore/name-Thisisthetruststorecontainingthepubliccertificate(s)forclientkeys.Dependinguponhowyoucreatedthekeystoreandtruststores,thismaybesamefileasdefinedunder"keystore/name"property.Requiredif"authenticationMode"is"2-way".
truststore/password-passwordforthetruststore.Requiredifthetruststorehasapassword.
truststore/check-expired-Whethertocheckforexpiredclientcertificates.Defaultfalse.
enabled-cipher-suites-Acommaseparatedlistofciphersuitesallowedforencryptionbetweenserverandclient.ThevaluesmustbevalidsupportedciphersuitesotherwiseSSLconnectionswillfail.Optional-defaultstoallsupportedciphersuitesforthevm.
Alternatively,youcanusetheCLItoaddormodifythetransportconfiguration
/subsystem=teiid/transport=jdbc:write-attribute(name=ssl-mode,value=enabled)
/subsystem=teiid/transport=jdbc:write-attribute(name=ssl-authentication-
mode,value=1-way)
/subsystem=teiid/transport=jdbc:write-attribute(name=ssl-ssl-protocol,value=TLSv1)
/subsystem=teiid/transport=jdbc:write-attribute(name=ssl-keymanagement-
algorithm,value=SunX509)
/subsystem=teiid/transport=jdbc:write-attribute(name=ssl-enabled-cipher-
suites,value="SSL_RSA_WITH_RC4_128_MD5,SSL_RSA_WITH_RC4_128_SHA")
/subsystem=teiid/transport=jdbc:write-attribute(name=keystore-name,value=ssl-
example.keystore)
/subsystem=teiid/transport=jdbc:write-attribute(name=keystore-
password,value=redhat)
/subsystem=teiid/transport=jdbc:write-attribute(name=keystore-type,value=JKS)
/subsystem=teiid/transport=jdbc:write-attribute(name=keystore-key-
alias,value=teiid)
/subsystem=teiid/transport=jdbc:write-attribute(name=keystore-key-
password,value=redhat)
/subsystem=teiid/transport=jdbc:write-attribute(name=truststore-name,value=ssl-
example.truststore)
/subsystem=teiid/transport=jdbc:write-attribute(name=truststore-
password,value=redhat)
NoteIfyoudonotliketoleavecleartextpasswordsintheconfigurationfile,thenyoucanuseWildFlyvaultmechanismforstoringthekeystoreandtruststorepasswords.Usethedirectionsdefinedherehttps://community.jboss.org/docs/DOC-17248
EncryptionStrengthBothanonymousSSLandloginonly(JDBCspecific)encryptionareconfiguredtouse128bitAESencryptionbydefault.Bydefault1-wayand2-waySSLallowforciphersuitenegotiationbaseduponthedefaultciphersuitessupportedbytherespectiveJavaplatformsoftheclientandserver.Userscanrestricttheciphersuitesusedbyspecifyingtheenabled-cipher-suitespropertyaboveintheSSLconfiguration.
Examples
TeiidServerTransportSecurity
750
1-waysslauthenticationmode
TeiidServerTransportSecurity
751
JDBC/ODBCSSLconnectionusingself-signedSSLcertificatesWhenyouareoperatinginasecureenvironment,youneedtothinkaboutmutualauthenticationwiththeserveryouconnectingtoandalsoencryptallthemessagesgoingbackandforthbetweentheclientandserver.InTeiid,bothJDBCandODBCprotocolssupportSSLbasedconnections.TypicallyfordevelopmentpurposesyouwillnothaveCAsignedcertificates,andyouneedtovalidatewithself-signedcertificates.Inarticle,Iwillshowthestepstogenerateaself-signedcertificateandthenconfiguringtheminTeiid.ThenconfiguringtheJDBCandODBCclientswiththedefinedSSLcertificatestocommunicatewiththeTeiidserver.
Creatingself-signedcertificates
Ifyoudonotalreadyhaveit,downloadthe"openssl"librariesforyourenvironment.Followthebelowscriptforcreatingthecertificate(s).
CreaterootCACertificate
Tobeginwith,youneedtogeneratetherootCAkey(thisiswhatsignsallissuedcerts),makesureyougiveastrongpassphrase.
opensslgenrsa-des3-passoutpass:changeme-outrootCA.key2048
opensslrsa-passinpass:changeme-inrootCA.key-outrootCA.key
Generatetheself-signed(withthekeypreviouslygenerated)rootCAcertificate:
opensslreq-new-keyrootCA.key-outrootCA.csr
opensslreq-x509-inrootCA.csr-keyrootCA.key-days365-outrootCA.crt
YoucaninstallthisonTeiidServermachinethatwillbecommunicatingwithservicesusingSSLcertificatesgeneratedbythisrootcertificate.Typically,you’llwanttoinstallthisonalloftheserversonyourinternalnetwork.
ToworkwithTeiidserver,youneedtoimportthiscertificateintokeystore.Followthebelowsteps
opensslpkcs12-export-inrootCA.crt-inkeyrootCA.key-outrootCA.p12-noiter-
nomaciter-nameroot
keytool-importkeystore-destkeystorerootCA.keystore-srckeystorerootCA.p12-
srcstoretypepkcs12-aliasroot
GeneratingclientsidecertificatesOnceyouhavetherootCAcertificategenerated,youcanusethattogenerateadditionalSSLcertificatesforotherJDBCorODBCandforotherservices.
1-WAYSSL
For1-WAYSSL,wewouldneedtoextractrootCA’strustcertificate(publickey)andcreateakeystorewithit.
JDBC/ODBCSSLconnectionusingself-signedSSLcertificates
752
opensslx509-trustout-inrootCA.crt>rootCA_trust.crt
keytool-importcert-v-trustcacerts-aliasrootCA-filerootCA_trust.crt-keystore
teiid.keystore
opensslx509-inrootCA_trust.crt-outrootCA_trust.cer-outformder
Herewecreatedkeystore(teiid.keystore)thatcanbeusedwithjavabasedapplicationslikeJDBCdriver,andalsocreatedcertificate(rootCA_trust.cer)thatcanbeusedinWindowsplatform.
2-WAYSSL
for2-WAYSSL,youwouldneedananothercertificateonclientside.TocreateanSSLcertificateyoucanuseforoneofyourservices,thefirststepistocreateacertificatesigningrequest(CSR).Todothat,youneedakey(separatefromtherootCAkeyyougeneratedearlier).ThengenerateaCSR
opensslgenrsa-outteiid.key2048
opensslrsa-passinpass:changeme-inteiid.key-outteiid.key
Generatetheself-signedcertificate,andgeneratesignedcertificateusingtherootCAcertificateandkeyyougeneratedpreviously.MakesuretheCommonName(CN)issettotheFQDN,hostnameorIPaddressofthemachineyou’regoingtoputthison.
opensslreq-new-keyteiid.key-outteiid.csr
opensslx509-req-inteiid.csr-CArootCA.crt-CAkeyrootCA.key-CAcreateserial-
outteiid.crt-days365
NowyouhaveanSSLcertificate(inPEMformat)calledteiid.crtThisisthecertificateyouwantyourJDBCorODBCtouse.Importthiscertificateintoaexistingkeystoreorcreateanewoneusing
opensslpkcs12-export-inteiid.crt-inkeyteiid.key-outteiid.p12-noiter-
nomaciter-nameteiid
keytool-importkeystore-destkeystoreteiid.keystore-srckeystoreteiid.p12-
srcstoretypepkcs12-aliasteiid
keytool-importcert-filerootCA_trust.crt-keystoreteiid.keystore
Also,importtheclientcertificate’spublickeyintorootCAkeystore
opensslx509-trustout-inteiid.crt>teiid_trust.crt
keytool-importcert-fileteiid_trust.crt-keystorerootCA.keystore
Ialsofoundagreatreferencehere[1]&[2]forcertificategeneration.Noteinabovethat,IhadissueswithrecognizingthePKCS12formattedkeystoreinJavaVM,IhadtoconvertintoaJKSformat.
ConfiguringtheTeiidServerwithCertificates
InstallTeiidserverifyoudonotalreadyhaveone.
Editthestandalone-teiid.xmlfile,andfind"teiid"subsystemandinsidefindJDBCandODBCtransportsandaddasfollowing.
<transportname="jdbc"socket-binding="teiid-jdbc"protocol="teiid">
JDBC/ODBCSSLconnectionusingself-signedSSLcertificates
753
<sslmode="enabled"authentication-mode="1-way">
<keystorename="/path/to/rootCA.keystore"password="changeme"type="JKS"/>
<!--uncommentandconfigurefor2-wayauthentication
<truststorename="/path/to/rootCA.keystore"password="changeme"/>
-->
</ssl>
</transport>
<transportname="odbc"socket-binding="teiid-odbc"protocol="pg">
<sslmode="enabled"authentication-mode="1-way">
<keystorename="/path/to/rootCA.keystore"password="changeme"type="JKS"/>
<!--uncommentandconfigurefor2-wayauthentication
<truststorename="/path/to/rootCA.keystore"password="changeme"/>
-->
</ssl>
</transport>
ThenrestarttheservertostartacceptingtheconnectionsusingSSL.Nowserversetupiscomplete.
ConfiguringJDBCclienttouseSSL
WhenusingaJDBCclienttousetheSSL,copytheserver.truststorefiletothetargetmachine.OneofthemainchangeisdifferenceinJDBCconnectionURLyouneedtouse.ForexampleifyourJDBCconnectionstringis
jdbc:teiid:<vdb>:mm://<host>:31000
thenchangeitto
jdbc:teiid:<vdb>:mms://<host>:31000
note"mm[s]"torepresent[s]forsecure.Youalsoneedtoaddthefollowingsystempropertiestoyourclientfor
1-WAYSSL
-Djavax.net.ssl.trustStore=/path/to/teiid.keystore
-Djavax.net.ssl.trustStorePassword=changeme
-Djavax.net.ssl.keyStoreType=JKS
2-WAYSSL
-Djavax.net.ssl.keyStore=/path/to/teiid.keystore
-Djavax.net.ssl.keyStorePassword=changeme
-Djavax.net.ssl.trustStore=/path/to/teiid.keystore
-Djavax.net.ssl.trustStorePassword=changeme
-Djavax.net.ssl.keyStoreType=JKS
Thestartyourclientapplicationnormally,thatshouldmakesuretheSSLcertificatesusedforencryption.
JDBC/ODBCSSLconnectionusingself-signedSSLcertificates
754
ConfiguringODBCclienttouseSSL(Windows)
InstallthePostgresqlODBCdriverinyourWindowsmachine.Youcandownloadthedriverfromhttp://www.postgresql.org/ftp/odbc/versions/
1-WAYSSL
Copythe"rootCA.crt"and"rootCA_trust.cer"filesintoyourWindowsmachineintodirectoryc:\Users\<yourname>\AppData\Roaming\postgresql.Notethisdirectorymaybehiddenornonexistent,ifnon-existentcreateanewfolder.NotethatifyouaredealingwithCAsignedcertificate,youdonothavetoshareyourprivatecertificate"rootCA.crt".Howeversinceweareusingselfsignedthiswillbecometherootcertificate.
Rename"rootCA.crt"to"root.crt"
Rename"rootCA_trust.cer"to"postgresql.cer"
Nowopenthe"ODBCDataManager"application,createDSNfortheconnectionyouarereadytomakeusingpreviouslyinstalledPostgresODBCdriver.Providethecorrecthostnameandport(35432),anduseVDBnameasDatabasename,andselectthe"ssl-model"propertyto"verify-ca"or"verify-full"andsavetheconfiguration.
2-WAYSSL
Copythe"rootCA.crt","teiid.crt","teiid.key"filesintoyourWindowsmachineintodirectoryc:\Users\<yourname>\AppData\Roaming\postgresql.Notethisdirectorymaybehiddenornonexistent,ifnon-existentcreateanewfolder.NotethatifyouaredealingwithCAsignedcertificate,youdonothavetoshareyourprivatecertificate"rootCA.crt".Howeversinceweareusingselfsignedthiswillbecometherootcertificate.
Rename"rootCA.crt"to"root.crt"
Rename"teiid.crt"to"postgresql.crt"
Rename"teiid.key"to"postgresql.key"
Nowopenthe"ODBCDataManager"application,createDSNfortheconnectionyouarereadytomakeusingpreviouslyinstalledPostgresODBCdriver.Providethecorrecthostnameandport(35432),anduseVDBnameasDatabasename,andselectthe"ssl-model"propertyto"verify-ca"or"verify-full"andsavetheconfiguration.
NowuseanyODBCclientapplication/toollike(QTODBC)andmakeODBCconnectionusingtheDSNcreatedandstartissuingtheSQLqueries.
JDBC/ODBCSSLconnectionusingself-signedSSLcertificates
755
SecurityattheDataSourceLevel
Insomeusecases,theusermightneedtopass-indifferentcredentialstotheirdatasourcesbasedontheloggedinuserratherthanusingthesharedcredentialsforalltheloggedusers.Tosupportthisfeature,WildFlyandTeiidprovidemultipleloginmodulestobeusedinconjunctionwithTeiid’smainsecuritydomain.Seethisdocumentfordetailsonconfiguration.Notethatthesedirectionsneedtobeusedinconjunctionwiththecontainerdocument.
CallerIdentity
Ifclientwantstopassinsimpletextpasswordoracertificateoracustomserializedobjectastokencredentialtothedatasource,theadmincanconfigurethe"CallerIdentity"loginmodule.Usingthisloginmoduleausercanpass-intheirTeiidsecuritydomainlogincredentialtothedatasource.Hereisasampleconfiguration:
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="my-security-domain">
<authentication>
<login-modulecode="RealmDirect"flag="required">
<module-optionname="password-stacking"value="useFirstPass"/>
</login-module>
<login-modulecode="org.picketbox.datasource.security.CallerIdentityLoginModule"flag="required"
>
<module-optionname="password-stacking"value="useFirstPass"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
</subsystem>
Note Thissecuritydomainshouldonlybeusedtosecuredatasources,andnotasgenericpurposesecuritydomain.
Note
"applicability"-CallerIdentityLoginmoduleisonlyapplicablewhentheloggedinsubjectcontainsthetextbasedcredentials.Theloginmoduleretrievesandusestheusernameandpasswordforthedatasourceauthenticationpurposes.Whenworkingwithnon-characterbasedpasswordsusePassthoughIdentitydefinedbelow.
Inthedatasourceconfiguration,insteadofsupplyingtheusername/passwordyouneedtoaddthefollowingelement:
InJDBCDatasource
<datasourcejndi-name="java:/mysql-ds"pool-name="mysql-ds"enabled="true">
<connection-url>jdbc:mysql://localhost:3306/txns</connection-url>
<driver>mysql</driver>
<pool>
<allow-multiple-users>true</allow-multiple-users>
</pool>
<security>
<security-domain>my-security-domain</security-domain>
</security>
</datasource>
Inaconnectionfactoryex:ldap
<resource-adapter>
DataSourceSecurity
756
<archive>teiid-connector-ldap.rar</archive>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.ldap.LDAPManagedConnectionFactory"
jndi-name="java:/ldapDS"
enabled="true"
use-java-context="true"
pool-name="ldap-ds">
<config-propertyname="LdapUrl">ldap://ldapServer:389</config-property>
<config-propertyname="LdapAdminUserDN">cn=???,ou=???,dc=???</config-property>
<config-propertyname="LdapAdminUserPassword">pass</config-property>
<config-propertyname="LdapTxnTimeoutInMillis">-1</config-property>
<security>
<security-domain>my-security-domain</security-domain>
</security>
</connection-definition>
</connection-definitions>
</resource-adapter>
Whenuserlogsinwithapassword,thesameusernameandpasswordwillbealsosetontheloggedinSubjectafterauthentication.ThesecredentialscanbeextractedbythedatasourcebyaskingforSubject’sprivatecredentials.
PleasenotethatencodinganddecodingofthiscredentialisstrictlyuptotheuserasWildFlyandTeiidwillonlyactasacarrieroftheinformationfromloginmoduletoconnectionfactory.UsingthisCallerIdentitymodule,theconnectionpoolfordatasourceissegmentedbySubject.
PassThroughIdentityThisissimilartotheCallerIdentityloginmodule,wherethecallinguser’scredentialsandrolesarepassedasis.Thisisespeciallyusefulwhendealingwithnon-textbasedcredentialswhereyouwanttopassdownthepayloadasis.
standalone-teiid.xml
<subsystemxmlns="urn:jboss:domain:security:1.1">
<security-domains>
<security-domainname="passthrough-security">
<authentication>
<login-modulecode="org.teiid.jboss.PassthroughIdentityLoginModule"flag="required"module="org
.jboss.teiid">
<module-optionname="username"value="guest"/>
<module-optionname="password"value="guest"/>
</login-module>
</authentication>
</security-domain>
</security-domains>
</subsystem>
Note Thissecuritydomainshouldonlybeusedtosecuredatasources,andnotasgenericpurposesecuritydomain.
Inthedatasourceconfiguration,insteadofsupplyingtheusername/passwordyouneedtoaddthefollowingelement
InJDBCDatasource
<datasourcejndi-name="java:/mysql-ds"pool-name="mysql-ds"enabled="true">
<connection-url>jdbc:mysql://localhost:3306/txns</connection-url>
<driver>mysql</driver>
<pool>
<allow-multiple-users>true</allow-multiple-users>
DataSourceSecurity
757
</pool>
<security>
<security-domain>passthrough-security</security-domain>
</security>
</datasource>
OAuthAuthenticationSecuredRestserviceswithOAuthauthenticationcanbeusedinTeiid,howeverthedatasourcesneedtobeconfiguredwithOAuthRefreshTokenorJsonWebToken(JWT)basedsecuritydomains.
RefreshToken
AconnectedapplicationisdifferentamongvendorslikeGoogle,LinkedIn,SalesForceetc.Fordetailsaboutcreatingaconnectedapplicationconsultthevendor’sdocumentation.Onceyouhavecreatedaconnectedapplication,thenrunteiid-oauth-util.shin"<eap>/bin"directory,useclient_id,client_pass,andcallbackfromsourcespecificconnectedapplication.Thisscriptwillprovidethenecessaryvaluestoplug-inbelowCLIscript.
createasecurity-domainbyexecutingCLI
/subsystem=security/security-domain=oauth2-security:add(cache-type=default)
/subsystem=security/security-domain=oauth2-security/authentication=classic:add
/subsystem=security/security-domain=oauth2-security/authentication=classic/login-module=oauth:add(code=org.teii
d.jboss.oauth.OAuth20LoginModule,flag=required,module=org.jboss.teiid.security,
module-options=[client-id=xxxx,client-secret=xxxx,refresh-token=xxxx,
access-token-uri=https://login.salesforce.com/services/oauth2/token])
reload
thiswillgeneratethefollowingXMLinthestandalone.xmlordomain.xml(thiscanalsobedirectlyaddedtothestandalone.xmlordomain.xmlfilesinsteadofexecutingtheCLI)
standalone.xml
<security-domainname="oauth2-security">
<authentication>
<login-modulecode="org.teiid.jboss.oauth.OAuth20LoginModule"flag="required"module="org.jboss.teiid.s
ecurity">
<module-optionname="client-id"value="xxxx"/>
<module-optionname="client-secret"value="xxxx"/>
<module-optionname="refresh-token"value="xxxx"/>
<module-optionname="access-token-uri"value="https://login.salesforce.com/services/oauth2/token"/>
</login-module>
</authentication>
</security-domain>
JSONWebToken(JWT)
AconnectedapplicationisdifferentamongvendorslikeGoogle,LinkedIn,SalesForceetc.Fordetailsaboutcreatingaconnectedapplicationconsultthevendor’sdocumentation.OnceyouhavecreatedconnectedapplicationthatusestheJWT,gatherthebelowinformationclient-id,client-secret,access-token-uri,jwt-audience,jwt-subject,keystore-type,keystore-password,keystore-url,certificate-alias,signature-algorithm-nameandprovideinthebelowCLI.(onlytestedwithSalesForce)
/subsystem=security/security-domain=oauth2-jwt-security:add(cache-type=default)
/subsystem=security/security-domain=oauth2-jwt-security/authentication=classic:add
/subsystem=security/security-domain=oauth2-jwt-security/authentication=classic/login-module=oauth:add(code=org.
teiid.jboss.oauth.OAuth20LoginModule,flag=required,module=org.jboss.teiid.security,
module-options=[client-id=xxxx,client-secret=xxxx,access-token-uri=https://login.salesforce.com/services/o
auth2/token,jwt-audience=https://login.salesforce.com,jwt-subject=your@sf-login.com,
DataSourceSecurity
758
keystore-type=JKS,keystore-password=changeme,keystore-url=${jboss.server.config.dir}/salesforce.jks,cert
ificate-alias=teiidtest,signature-algorithm-name=SHA256withRSA])
reload
thiswillgeneratefollowingXMLinthestandalone.xmlordomain.xml(thiscanalsobedirectlyaddedtothestandalone.xmlordomain.xmlfilesinsteadofexecutingtheCLI)
standalone.xml
<security-domainname="oauth2-jwt-security">
<authentication>
<login-modulecode="org.teiid.jboss.oauth.JWTBearerTokenLoginModule"flag="required"module="org.jboss.
teiid.security">
<module-optionname="client-id"value="xxxxx"/>
<module-optionname="client-secret"value="xxxx"/>
<module-optionname="access-token-uri"value="https://login.salesforce.com/services/oauth2/token"/>
<module-optionname="jwt-audience"value="https://login.salesforce.com"/>
<module-optionname="jwt-subject"value="your@sf-login.com"/>
<module-optionname="keystore-type"value="JKS"/>
<module-optionname="keystore-password"value="changeme"/>
<module-optionname="keystore-url"value="${jboss.server.config.dir}/salesforce.jks"/>
<module-optionname="certificate-alias"value="teiidtest"/>
<module-optionname="signature-algorithm-name"value="SHA256withRSA"/>
</login-module>
</authentication>
</security-domain>
Kerberos
Kerberoscanalsousedasdatasourcesecurity.ThebelowconfigurationistoconfigureastaticKerberosticketatdatasource.PleasenotethatKerberoscanbeusedwithRDBMS,RESTwebservices.
/subsystem=security/security-domain=host:add(cache-type=default)
/subsystem=security/security-domain=host/authentication=classic:add
/subsystem=security/security-domain=host/authentication=classic/login-module=Kerberos:add(code=Kerberos,flag=r
equired,
module-options=[storeKey=true,refreshKrb5Config=true,useKeyTab=true,
principal=host/testserver@MY_REALM,keyTab=/path/to/service.keytab,doNotPrompt=true,debug=false])
reload
TheabovecommandwillgenerateresultingXMLinthestandalone.xmlfileordomain.xmlfile.
standalone.xml
<security-domainname="host">
<authentication>
<login-modulecode="Kerberos"flag="required">
<module-optionname="storeKey"value="true"/>
<module-optionname="useKeyTab"value="true"/>
<module-optionname="principal"value="host/testserver@MY_REALM"/>
<module-optionname="keyTab"value="/path/to/service.keytab"/>
<module-optionname="doNotPrompt"value="true"/>
<module-optionname="debug"value="false"/>
<module-optionname="refreshKrb5Config"value="true"/>
<module-optionname="addGSSCredential"value="true"/>
</login-module>
</authentication>
</security-domain>
KerberosDelegation
DataSourceSecurity
759
ForusingthesamekerberostokenatTeiidandaswellasatthedatasourcelevel,thetokennegotiatedattheTeiidenginecanbepassedintodatasource.Thedatasourcemustbeconfiguredtosupportthis.MajordatabasevendorslikeOracle,MS-SQLServer,DB2,HIVE,Impalasupportkerberos.Somealsosupportpassthroughmode.Tomakedelegationwork,followthedirectionsheretosetuptheKerberosatTeiidenginelevel[KerberossupportthroughGSSAPI]andusethemoduleoptiondelegationCredential:
<module-optionname="delegationCredential"value="USE"/>
Tip
WhenworkingwithKerberos/GSSsecuritytoken(GssCredential),someJDBCdrivers(MS-SQLServer)uponcloseoftheconnectiontheyinvalidatetheGssCredentialsecuritytoken,toavoidaccidentalinvalidation,addanoptiontoabovesecurity-domain’slogin-moduleconfigurationtowrapthepassedinsecuritytokenbyaddingbelowconfiguration
<module-optionname="wrapGSSCredential"value="true"/>
TranslatorCustomizationTeiid’sextensibleTranslatorframeworkalsoprovideshooksforsecuringaccessattheDataSourcelevel.TheExecutionFactory.getConnectionmaybeoverriddentoinitializethesourceconnectioninanynumberofways,suchasre-authentication,basedupontheTeiidSubject,executionpayload,sessionvariables,andanyoftheotherrelevantinformationaccessibleviatheExecutionContextandtheCommandContext.YoumayevenalsomodifythegeneratedsourceSQLinanywaythatisseenfitintherelevantExecution.
DataSourceSecurity
760
KerberossupportthroughGSSAPI
TeiidsupportskerberosauthenticationusingGSSAPIforsinglesign-onapplications.ThisserviceticketnegotiationbasedauthenticationissupportedthroughremoteJDBC/ODBCdriversandLocalConnections.Clientconfigurationisdifferentforeachclienttype.
LocalConnection
SettheJDBCURLpropertyPassthroughAuthenticationastrueanduseJBossNegotiationforauthenticationofyourweb-applicationwithkerberos.Whenthewebapplicationauthenticateswiththeprovidedkerberostoken,thesamesubjectauthenticatedwillbeusedinTeiid.Fordetailsaboutconfiguration,checktheconfiguringtheSSOwithKerberosinEAP
ServerconfigurationforRemoteJDBC/ODBCConnections
TosupportkerberosSSOonremoteJDBCandODBCconnections,bothclientsideandserversideconfigurationsneedtobemodified.Ontheserverside,EAPneedstobeconfiguredwithtwodifferentloginmodules.ThebelowCLIscriptshowsexamplesofit.Makenecessarychangesrelatedtoyourconfigurationintermsofkeytablocations,serviceprincipaletc.
Configuresecuritydomaintorepresenttheidentityoftheserver.
Thefirstsecuritydomainauthenticatesthecontaineritselftothedirectoryservice.Itneedstousealoginmodulewhichacceptssometypeofstaticloginmechanism,becausearealuserisnotinvolved.Thisexampleusesastaticprincipalandreferencesakeytabfilewhichcontainsthecredential.
/subsystem=security/security-domain=host:add(cache-type=default)
/subsystem=security/security-domain=host/authentication=classic:add
/subsystem=security/security-domain=host/authentication=classic/login-module=Kerberos:add(code=Kerberos,flag=r
equired,
module-options=[storeKey=true,refreshKrb5Config=true,useKeyTab=true,
principal=host/testserver@MY_REALM,keyTab=/path/to/service.keytab,doNotPrompt=true,debug=false])
reload
TheabovecommandwillgenerateresultingXMLinthestandalone.xmlfileordomain.xmlfile.
standalone-teiid.xml
<security-domainname="host">
<authentication>
<login-modulecode="Kerberos"flag="required">
<module-optionname="storeKey"value="true"/>
<module-optionname="useKeyTab"value="true"/>
<module-optionname="principal"value="host/testserver@MY_REALM"/><!--serviceprincipal-->
<module-optionname="keyTab"value="/path/to/service.keytab"/>
<module-optionname="doNotPrompt"value="true"/>
<module-optionname="debug"value="false"/>
<module-optionname="refreshKrb5Config"value="true"/>
</login-module>
</authentication>
</security-domain>
ConfiguresecuritydomaintosecuretheTeiidapplication.
KerberossupportthroughGSSAPI
761
ThesecondsecuritydomainisusedtoauthenticatetheindividualusertotheKerberosserver.Youneedatleastoneloginmoduletoauthenticatetheuser,andanothertosearchfortherolestoapplytotheuser.ThefollowingXMLcodeshowsanexampleSPNEGOsecuritydomain.Itincludesanauthorizationmoduletomaprolestoindividualusers.Youcanalsouseamodulewhichsearchesfortherolesontheauthenticationserveritself.Notethenameofsecurity-domainMUSTmatchrealm.ThefollowingCLIscriptshowsexampleofcreatingtheloginmodule
/subsystem=security/security-domain=MY_REALM:add(cache-type=default)
/subsystem=security/security-domain=MY_REALM/authentication=classic:add
/subsystem=security/security-domain=MY_REALM/authentication=classic/login-module=SPNEGO:add(code=SPNEGO,flag=r
equisite,
module-options=[serverSecurityDomain=host,password-stacking=useFirstPass])
/subsystem=security/security-domain=MY_REALM/authentication=classic/login-module=UserRoles:add(code=SPNEGO,fla
g=requisite,
module-options=[usersProperties=spnego-users.properties,rolesProperties=spnego-roles.properties])
reload
TheaboveCLIwillresultinfollowingresultXMLinstandalone.xmlordomain.xmldependinguponconfiguration
standalone-teiid.xml
<security-domainname="MY_REALM">
<authentication>
<!--Checktheusernameandpassword-->
<login-modulecode="SPNEGO"flag="requisite">
<module-optionname="password-stacking"value="useFirstPass"/>
<module-optionname="serverSecurityDomain"value="host"/>
</login-module>
<!--Searchforroles-->
<login-modulecode="UserRoles"flag="requisite">
<module-optionname="password-stacking"value="useFirstPass"/>
<module-optionname="usersProperties"value="spnego-users.properties"/>
<module-optionname="rolesProperties"value="spnego-roles.properties"/>
</login-module>
</authentication>
</security-domain>
Note
"UserRoles/Groupsassociations"Kerberosdoesnotassignanyuserrolestotheauthenticatedsubject,thatisreasonyouneedtoconfigureaseparaterolemappingmoduletoassignroles.Asanexampleintheabove,"UserRoles"login-moduleisadded.Userneedtoedit"spnego-roles.properties"fileandaddgroupsintheformatof`user@MY_REALM=my-group.CheckJBossEAPdocumentation,astoalltheavailablemappingmodulesthatareavailable.
SPENGOsecurity-domaindelegatesthecallsrelatingtoKerberostoKerberosserverbasedon"serverSecurityDomain"property.IfyouwouldlikeconfigurethechoiceofauthenticatingusingKerberosorsomeotheradditionalsecuritydomainonthesameJDBC/ODBCtransport,thenyouneedtosupplyanadditionalmoduleoption(thiscanalsobeviewedasfallbackauthenticationmodel)
<module-optionname="usernamePasswordDomain"value="{user-name-based-auth}"/>
theresultingxmlwilllooklikebelowwhere{user-name-based-auth}replacedwithaJAASbasedsimpleusername/passwordloginmodule"app-fallback"
standalone-teiid.xml
<security-domainname="MY_REALM">
<authentication>
<!--Checktheusernameandpassword-->
<login-modulecode="SPNEGO"flag="requisite">
<module-optionname="password-stacking"value="useFirstPass"/>
<module-optionname="serverSecurityDomain"value="host"/>
<module-optionname="usernamePasswordDomain"value="app-fallback"/>
KerberossupportthroughGSSAPI
762
</login-module>
<!--Searchforroles-->
<login-modulecode="UserRoles"flag="requisite">
<module-optionname="password-stacking"value="useFirstPass"/>
<module-optionname="usersProperties"value="spnego-users.properties"/>
<module-optionname="rolesProperties"value="spnego-roles.properties"/>
</login-module>
</authentication>
</security-domain>
<security-domainname="app-fallback"cache-type="default">
<authentication>
<login-modulecode="UsersRoles"flag="required">
<module-optionname="usersProperties"value="file:${jboss.server.config.dir}/fallback-u
sers.properties"/>
<module-optionname="rolesProperties"value="file:${jboss.server.config.dir}/fallback-r
oles.properties"/>
</login-module>
</authentication>
</security-domain>
ServerTransportConfiguration
Theaboveconfigurationdefinedsecurity-domains,beforeyoucanusethesedomainsforloginintoTeiid,theyneedtobeassociatedwithTeiid’stransportconfigurationorVDBconfiguration.Paragraphsbelowofferbothsolutions.
Defininga"default"authenticationbasedonTeiidTransport
Usercandefinea"default"authenticationpertransportasbelowthatcanbeusedforalltheVDBssystemwide.
ForJDBC:
UsebelowCLIcommandstoedittheconfiguration
----
/subsystem=teiid/transport=jdbc:write-attribute(name=authentication-security-domain,value=MY_REALM)
/subsystem=teiid/transport=jdbc:write-attribute(name=authentication-type,value=GSS)
----
Willresultinfollowingchanges(oryoucaneditthestandalone-teiid.xmlfiledirectly)
<transportname="jdbc"protocol="teiid"socket-binding="teiid-jdbc"/>
<authenticationsecurity-domain="MY_REALM"type="GSS"/>
</transport>
ForODBC:
UsebelowCLIcommandstoedittheconfiguration
----
/subsystem=teiid/transport=odbc:write-attribute(name=authentication-security-domain,value=MY_REALM)
/subsystem=teiid/transport=odbc:write-attribute(name=authentication-type,value=GSS)
----
<transportname="odbc"protocol="pg"socket-binding="teiid-odbc"/>
<authenticationsecurity-domain="MY_REALM"type="GSS"/>
</transport>
"WhatisthevalueofType"
KerberossupportthroughGSSAPI
763
The"type"attributeabovedefinesthetypeofauthenticationthatneedstobeenforcedonthetransport/vdb.Theallowedvaluesfortypeare
USERPASSWORD-onlyallowusername/passwordbasedauthentications
GSS-onlyallowGSSAPIbasedauthentication(Kerberos5).
DefiningVDBbasedauthentication
YoucanaddfollowingcombinationVDBpropertiesinthevdb.xmlfiletoselectorforcethesecurity-domainandauthenticationtype.
<propertyname="security-domain"value="MY_REALM"/>
<propertyname="gss-pattern"value="{regex}"/>
<propertyname="password-pattern"value="{regex}"/>
<propertyname="authentication-type"value="GSSorUSERPASSWORD"/>
AllthepropertiesaboveareoptionalonaVDB.IfyouwanttodefineVDBbasedsecurityconfiguration"security-domain"propertyisrequired.Ifyouwanttoenforcesingleauthenticationtypeuse"authentication-type"propertyisrequired.IfyoursecuritydomaincansupportbothGSSandUSERPASSWORD,thenyoucandefine"gss-pattern"and"password-pattern"properties,anddefinearegularexpressionasthevalue.Duringtheconnection,theseregularexpressionsarematchedagainsttheconnectinguser’snameprovidedtoselectwhichauthenticationmethoduserprefers.Forexample,iftheconfigurationisdefinedasbelow
<propertyname="security-domain"value="MY_REALM"/>
<propertyname="gss-pattern"value="logasgss"/>
andifyoupassedthe"user=logasgss"intheconnectionstring,thenGSSauthenticationisselectedasloginauthenticationmechanism.Iftheusernamedoesnotmatch,thendefaulttransport’sauthenticationmethodisselected.Alternatively,ifyouwantchooseUSERPASSWORD
<propertyname="security-domain"value="MY_REALM"/>
<propertyname="password-pattern"value="*-simple"/>
andiftheusernameislike"mike-simple",thenthatuserwillbesubjectedtoauthenticateagainstUSERPASSWORDbasedauthenticationdomain.Youcanconfiguredifferentsecurity-domainsfordifferentVDBS.VDBauthenticationwillnolongerbedependentuponunderlyingtransport.Ifyoulikeforce"GSS"allthetimethenuseconfigurationlikebelow
<propertyname="security-domain"value="MY_REALM"/>
<propertyname="authentication-type"value="GSS"/>
RequiredSystemPropertiesonServer
JBossEAPofferstheabilitytoconfiguresystempropertiesrelatedtoconnectingtoKerberosservers.DependingontheKDC,KerberosDomain,andnetworkconfiguration,thebelowsystempropertiesmayormaynotberequired.
Editthe"standalone.conf"ordomain.conffileinthe"${jboss-as}/bin"directoryandaddthefollowingJVMoptions\(changingtherealmandKDCsettingsaccordingtoyourenvironment)
JAVA_OPTS="$JAVA_OPTS-Djava.security.krb5.realm=EXAMPLE.COM-
Djava.security.krb5.kdc=kerberos.example.com-
Djavax.security.auth.useSubjectCredsOnly=false"
KerberossupportthroughGSSAPI
764
or
JAVA_OPTS="$JAVA_OPTS-Djava.security.krb5.conf=/path/to/krb5.conf-
Djava.security.krb5.debug=false-Djavax.security.auth.useSubjectCredsOnly=false"
oryoucanalsoaddthesepropertiesinsidestandalone-teiid.xmlfile,rightafter\{<extensions>}segmentas
<system-properties>
<propertyname="java.security.krb5.conf"value="/pth/to/krb5.conf"/>
<propertyname="java.security.krb5.debug"value="false"/>
<propertyname="javax.security.auth.useSubjectCredsOnly"value="false"/>
</system-properties>
Thisfinishestheconfigurationontheserverside,restarttheserverandmakesuretherearenoerrorsduringstartup.
JDBCClientConfiguration
YourworkstationwheretheJDBCClientexistsmusthavebeenauthenticatedusingGSSAPIagainstActiveDirectoryorEnterprisedirectoryserver.Seethiswebsitehttp://spnego.sourceforge.netoninstructionsastohowtoverifyyoursystemisauthenticatedintoenterprisedirectoryserver.Contactyourcompany’soperationsteamifyouhaveanyquestions.
InyourclientVMtheJAASconfigurationforKerberosauthenticationneedstobewritten.Asampleconfigurationfile(client.conf)isshowbelow
"client.conf"
Teiid{
com.sun.security.auth.module.Krb5LoginModulerequired
useTicketCache=true
storeKey=true
useKeyTab=true
keyTab="/path/to/krb5.keytab"
doNotPrompt=true
debug=false
principal="user@EXAMPLE.COM";
};
Makesureyouhaveconfiguredthe"keytab"properly,youcancheckthiswebsiteforutilitiesandinstructionstocheckyouraccesstoKDCserverandtocreatekeytabespeciallyonwindowsenvironmentshttp://spnego.sourceforge.net.ForRedhatLinuxseehttps://access.redhat.com/site/solutions/208173
AddthefollowingJVMoptionstoyourclient’sstartupscript-changeRealmandKDCsettingsaccordingtoyourenvironment
"Basedonkrb5.conffile"
-Djava.security.krb5.conf=/path/to/krb5.conf(defaultonLinux/etc/krb5.conf)
-Djava.security.auth.login.config=/path/to/client.conf
-Djavax.security.auth.useSubjectCredsOnly=false
-Dsun.security.krb5.debug=false
or
"BasedonKDCandRealmfile"
-Djava.security.krb5.realm=EXAMPLE.COM
KerberossupportthroughGSSAPI
765
-Djava.security.krb5.kdc=kerberos.example.com
-Djavax.security.auth.useSubjectCredsOnly=false
-Dsun.security.krb5.debug=false
-Djava.security.auth.login.config=/path/to/client.conf
AddthefollowingadditionalURLconnectionpropertiestoTeiidJDBCconnectionstringalongwithURLproperty.NotethatwhenconfiguredwithKerberos,inordertoparticipateinKerberosbasedauthenticationyouneedtoconfigure"user"propertyasrequiredby"gss-pattern"ordefinethe"authentication-type"propertyontheVDBortransport.However,aftersuccessfulloginintosecurity-domain,theusernamefromGSSlogincontextwillbeusedforrepresentingthesessionintheTeiid.
jaasName=Teiid;user={pattern};kerberosServicePrincipleName=host/testserver@MY_REALM
jassNamedefinestheJAASconfigurationnameinlogin.configfile.Thispropertyisoptional,ifomittedthe"Teiid"isusedasthedefaultconfigurationname.
kerberosServicePrincipleNamedefinesserviceprinciplethatneedstoberequestedonbehalfoftheservicethatisbeingconnectedtousingtheKerberosprincipleconfigured.Ifthispropertyisomittedthedefaultserviceprinciplewouldbe"TEIID/hostname"andhostnameisderivedfromtheJDBCconnectionURL.
Note
InordertoavoidaddingtheserviceprinciplenametoallyourJDBCandODBCclients,Teiidcanusethedefaultserviceprinciplenameas"TEIID/hostname".CreatethisserviceticketinKDC.ThisalsohelpsifyoumoveyourTeiidserveronehosttoanotherbysimplycreatinganewprincipleinKDCwithnewhostname.ThenyouwouldonlyrequiredtoupdatehostnameintheURL.
ODBCClientConfiguration
CreateaDSNfortheVDBontheclientmachinetotheVDBthatyouwouldliketoconnectusingPostgreSQLODBCdriver.InordertoparticipateinKerberosbasedauthenticationyouneedtoconfigure"user"propertyasrequiredby"gss-pattern"ordefinethe"authentication-type"propertyontheVDBortransport.
Noadditionalconfigurationisneededaspartofthis,exceptthatyourworkstationwheretheODBCDSNexistsmusthavebeenauthenticatedusingGSSAPIagainstActiveDirectoryorotherEnterprisedirectoryserver.Seethiswebsitehttp://spnego.sourceforge.netoninstructionsastohowtoverifyyoursystemisauthenticatedintoenterprisedirectoryserver.Contactyourcompany’soperationsteamifyouhaveanyquestions.
ODataClient
ThedefaultODataclientisconfiguredwithHTTPBasicauthentication,toconvertthisauthenticationmethodintokerberos,cloneorcopythemavenprojectfromhttps://github.com/teiid/teiid-web-securityandthenedittheweb.xmlandjboss-web.xmlfilesandthenreplaceMY_RELAMpropertywiththepropertyofsecuritydomaincreatedabove.Oncethepropertiesareupdated,createaWARfilebyrunning
mvncleaninstall
ThiswillgenerateanewWARfilein"odata-kerberos/target"directory.Followthebelowdeploymentdirectionbasedonyourserver.
Note TouseKerberosoranyweblayerauthentication,theODatawarmustusePassthroughAuthentication=true(whichisthedefault).
CommunityTeiidServerbasedonWildFly
KerberossupportthroughGSSAPI
766
Replacethe<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-odata4.war"filewithnewWARfile,byexecutingacommandsimilarto
{code}cpteiid-web-security/odata-kerberos/target/teiid-odata-kerberos-{version}.war<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-odata4.war{code}
JDVServer
IfyouareworkingwithJDV6.3serverorgreater,thenrunthefollowingCLIscript,youmayhavechangethebelowscripttoadopttothecorrectversionoftheWARanddirectorynameswherethecontentislocated.
undeployteiid-olingo-odata4.war
deployteiid-web-security/odata-kerberos/target/teiid-odata-kerberos-{version}.war
oroverlaythenewoneusingCLIscriptlike
deployment-overlayadd--name=myOverlay--content=/WEB-INF/web.xml=teiid-web-
security/odata-kerberos/src/main/webapp/WEB-INF/web.xml,/WEB-INF/jboss-
web.xml=teiid-web-security/odata-kerberos/src/main/webapp/WEB-INF/jboss-
web.xml,/META-INF/MANIFEST.MF=teiid-web-security/odata-
kerberos/src/main/webapp/META-INF/MANIFEST.MF--deployments=teiid-olingo-odata4.war
--redeploy-affected
KerberossupportthroughGSSAPI
767
CustomAuthorizationValidatorInsituationswhereTeiid’sbuilt-inDataRolesmechanismisnotsufficient,acustomorg.teiid.PolicyDecidercanbeinstalledviaaJBossmodule.NotethataPolicyDecideronlymakeshigh-levelauthorizationdecisionsbasedupontheaccesscontext(INSERT,UPDATE,DELETE,etc.),thecaller,andtheresource(column,table/view,procedure,function,etc.).Data-levelcolumnmaskingandrowbasedsecuritypolicyinformationduetoitsinteractionwiththeTeiidplannercannotbeinjectedviaacustomorg.teiid.PolicyDecider.Youmayaddcolumnmaskingandrowbasedsecuritypermissionsviatheorg.teiid.MetadataFactoryincustomaorg.teiid.MetadataRepositoryorcustomtranslator.
Toprovideacustomauthorizationvalidator,youmustextendtheorg.teiid.PolicyDeciderinterfaceandbuildacustomjavaclass.Ifyouareusingmavenasyourbuildprocess,youcanusefollowingdependencies:
<dependencies>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-api</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.jboss.teiid</groupId>
<artifactId>teiid-common-core</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
ThePoilcyDeciderinterfaceisloadedbytheTeiidusingtheJava’sstandardserviceloadermechanism.Forthistowork,addthefollowingnamedfileMETA-INF/services/org.teiid.PolicyDeciderwithfullnameofyourPolicyDeciderimplementationclassasitscontents.forexample:
META-INF/services/org.teiid.PolicyDecider
org.jboss.teiid.auth.MyCustomPolicyDecider
NowpackageallthesefilesintoaJARarchivefileandbuildJBossmoduleinjboss-as/modulesdirectory.IfyourPolicyDeciderhasanythirdpartydependenciesthosejarfilescanalsobeaddedasdependenciestothesamemodule.Makesureyoulistallthefilesinthemodule.xmlfile.Belowissamplemodule.xmlfilealongwithTeiidspecificdependencies
module.xml
<?xmlversion="1.0"encoding="UTF-8"?>
<modulexmlns="urn:jboss:module:1.0"name="org.jboss.teiid.auth">
<resources>
<resource-rootpath="my_custom_policy.jar"/>
<!--addanyotherdependentjarshere,iftheyarenotdefinedasmodules-->
</resources>
<dependencies>
<modulename="org.jboss.teiid.common-core"/>
<modulename="org.jboss.teiid.api"/>
<modulename="javax.api"/>
</dependencies>
</module>
createfolderinthe"<jboss-as>/modules/org/jboss/teiid/auth/main",copytheabovemodule.xmlfilealongwithallthejarfiles.Thisdirectorycanbedifferentifyouchoose,justmakesurethenameofthemoduleandthedirectorynamematch.
CustomAuthorizationValidator
768
Afterthemodulehasbeenadded,changetheconfiguration.Editeitherthestandalone-teiid.xmlortedomain-teiid.xmlfile,andinthe"teiid"subsystemxmlfragmentaddthefollowingxmlwiththemodulenamecreated.
<policy-decider-module>name</policy-decider-module>
thenrestartthesystem.APolicyDecidermaybeconsultedmanytimesforasingleusercommand,butitisonlycalledtomakedecisionsbaseduponresourcesthatappearinuserqueries.Anyfurtheraccessofresourcesthroughviewsorstoredprocedures,justaswithdataroles,isnotcheckedagainstaPolicyDecider.
CustomAuthorizationValidator
769
SAMLBasedSecurityForODataBydefaulttheODataaccesstoaVirtualDatabase(VDB)inWildFlyisrestrictedtoauthenticationusingtheHTTPBasic.However,itpossiblewithbelowinstructionsonecanconfigureODataaccesstoparticipateinaSingle-Sign-On(SSO)basedsecurityusingSAML2.ThebelowinstructionsarebasedonJBossEAPplatformusingPicketlinksecurityframework.
InSAMLbasedauthenticationthereareIdentityProviders(IDP)whoprovideauthenticationservicesandServiceProviders(SP),aenduserservicelikeodataanduser(you).ItisexpectedthatyoualreadyhaveIDP,configuredandworkingwithsecuritydomainofyourchoicelikeLDAPorKerberoesetc.TheSPinthiscaseistheODataWARfilethatissuppliedwithTeiiddistributionalongwithPicketlinkbasedframework.PicketlinkframeworkdoesnotexplicitlymentiontheinteroperabilitywithotherthirdpartyexternalvendorssuppliedIDP,butTeiidteamhastestedsuccessfullywith
Shibboleth
PicketlinkIDP
SalesforceIDP(thisisdocumentedonPicketlink,notverified)
SocialLoginswithPicketlinkIDP(like,google,facebooketc.ThishasbeenmentionedinPicketlinkdocumentationbutnotverified)
Note SinceSAML2isstandard,webelieveanystandardscomplaintIDPvendorwillworkwithPicketlinkSP.
requisites
CollectthecertificateforauthenticationthatisusedbyIDPtosigntheSAMLmessages.
GathertheSSOPOSTbasedURLforyourIDP,thatyourSPcanusetoredirectforauthenticationcall.
Note "DNSNames"-DonottrytouseIPaddressorlocalhostexceptforthetestingscenarios.ConfigureproperDNSnamesforbothIDPandSPserversandmakesurebothcanaccesseachotherusingtheURLsconfigured.
ConfigureforSAMLbasedauthenticationtheOData
Insecurity-domainsaddfollowingloginmoduleusingthefollowingCLI
/subsystem=security/security-domain=teiid-security/authentication=classic/login-
module=RealmDirect:write-attribute(name=flag,value=sufficient)
/subsystem=security/security-domain=teiid-security/authentication=classic/login-
module=saml2:add(code=org.picketlink.identity.federation.bindings.jboss.auth.SAML2L
oginModule,flag=sufficient)
reload
theabovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilesimilarto:
"Security-DomainforSAMLAuthentication"
<security-domainname="teiid-security">
<authentication>
<login-modulecode="org.picketlink.identity.federation.bindings.jboss.auth.SAML2LoginModule"flag="
sufficient"/>
<login-modulecode="RealmDirect"flag="sufficient">
<module-optionname="password-stacking"value="useFirstPass"/>
SAMLBasedSecurityForOData
770
</login-module>
</authentication>
</security-domain>
ModifytheODataWARFiletouseSAMLbasedauthenticationExtractthe"teiid-olingo-odata4.war"filefrom"modules/system/base/dv/org/jboss/teiid/main/deployments"toanotherlocation.TheWARfileissimpleZIPfilesoyoucan"jar-xteiid-olingo-odata4.war/modified"
Edit"WEB-INF/jboss-web.xml"file,anditshouldlooklike
"jboss-web.xml"
<?xmlversion="1.0"encoding="UTF-8"?>
<jboss-web>
<context-root>odata4</context-root>
<security-domain>teiid-security</security-domain>
<valve>
<class-name>org.picketlink.identity.federation.bindings.tomcat.sp.ServiceProviderAuthenticator</class-na
me>
<param>
<param-name>configProvider</param-name>
<param-value>org.picketlink.identity.federation.web.config.SPPostMetadataConfigurationProvider</param-val
ue>
</param>
</valve>
</jboss-web>
Edit"web.xml"fileandremovethesectionbelow
"web.xml"
<login-config>
<auth-method>BASIC</auth-method>
<realm-name>yourdomain.com</realm-name>
</login-config>
AddthecertificatekeystorefromyourIDPtotheclassesdirectory.Thisis{KEYSTORE-FILE}inbelowconfiguration.oryoucanaddtoaexistingkeystoreusingfollowingcommand
keytool-import-fileidp_cert.cer-keystore\{KEYSTORE-FILE\}-alias\
{CERTIFICATE-ALIAS\}
Add"picketlink.xml"filetoWEB-INFdirectorywithfollowingcontent
"picketlink.xml"
<PicketLinkxmlns="urn:picketlink:identity-federation:config:2.1">
<PicketLinkSPxmlns="urn:picketlink:identity-federation:config:2.1"
ServerEnvironment="tomcat"BindingType="POST"SupportsSignatures="true">
<KeyProvider
ClassName="org.picketlink.identity.federation.core.impl.KeyStoreKeyManager">
<AuthKey="KeyStoreURL"Value="\{KEYSTORE-FILE\}"/>
<AuthKey="KeyStorePass"Value="\{KEYSTORE-PASSWORD\}"/>
<AuthKey="SigningKeyAlias"Value="\{CERTIFICATE-ALIAS\}"/>
<AuthKey="SigningKeyPass"Value="\{CERTIFICATE-PASSWORD\}"/>
<ValidatingAliasKey="localhost"Value="\{CERTIFICATE-ALIAS\}"/>
<ValidatingAliasKey="127.0.0.1"Value="\{CERTIFICATE-ALIAS\}"/>
</KeyProvider>
</PicketLinkSP>
<Handlersxmlns="urn:picketlink:identity-federation:handler:config:2.1">
SAMLBasedSecurityForOData
771
<Handlerclass="org.picketlink.identity.federation.web.handlers.saml2.SAML2LogOutHandler"/>
<Handlerclass="org.picketlink.identity.federation.web.handlers.saml2.SAML2AuthenticationHandler"/>
<Handlerclass="org.picketlink.identity.federation.web.handlers.saml2.RolesGenerationHandler"/>
<Handlerclass="org.picketlink.identity.federation.web.handlers.saml2.SAML2SignatureGenerationHandler"
/>
<Handlerclass="org.picketlink.identity.federation.web.handlers.saml2.SAML2SignatureValidationHandler"
/>
</Handlers>
</PicketLink>
Note {CERTIFICATE-ALIAS}istypicallysomethinglike"idp.example.com"forwhichthecertificateiscreatedfor
AddthecertificatereceivedfromIDPvendorto"WEB-INF/classes"directory.Notethismustbesamenameas{CERTIFICATE-FILE-NAME}usedin"ConfiguringthePicketlinkSubsystem"
Add"sp-metadata.xml"totheclassesdirectory.Notethatyour"sp-metadata.xml"contentswillentirelydependentuponyourIdentityProvidersettings.ThebelowsampleONLYprovidedasanexample
"sp-metadata.xml"
<?xmlversion="1.0"encoding="UTF-8"?>
<EntitiesDescriptorName="urn:mace:shibboleth:testshib:two"
xmlns:shibmd="urn:mace:shibboleth:metadata:1.0"xmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xmlns:ds="http://www.w3.org/2000/09/xmldsig#"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<EntityDescriptorentityID="http://localhost:8080/idp-metadata/">
<IDPSSODescriptor
protocolSupportEnumeration="urn:oasis:names:tc:SAML:1.1:protocolurn:oasis:names:tc:SAML:2.0:protoc
ol">
<NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:transient
</NameIDFormat>
<SingleSignOnServiceBinding="urn:mace:shibboleth:1.0:profiles:AuthnRequest"
Location="http://localhost:8080/idp-metadata/"/>
<SingleSignOnServiceBinding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
Location="http://localhost:8080/idp-metadata/"/>
<SingleSignOnService
Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect"
Location="http://localhost:8080/idp-metadata/"/>
<SingleLogoutService
Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"
Location="http://localhost:8080/idp-metadata/?GLO=true"/>
<SingleLogoutService
Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect"
Location="http://localhost:8080/idp-metadata/SLO"/>
</IDPSSODescriptor>
<Organization>
<OrganizationNamexmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">JBoss</OrganizationName>
<OrganizationDisplayNamexmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">JBossbyRedHat</OrganizationDisplayName>
<OrganizationURLxmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">http://www.jboss.org</OrganizationURL>
</Organization>
<ContactPersoncontactType="technical">
<GivenName>The</GivenName>
<SurName>Admin</SurName>
<EmailAddress>admin@mycompany.com</EmailAddress>
</ContactPerson>
</EntityDescriptor>
<EntityDescriptorentityID="http://localhost:8080/odata4/">
<SPSSODescriptor
protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocolurn:oasis:names:tc:SAML:1.1:protoc
olhttp://schemas.xmlsoap.org/ws/2003/07/secext">
<NameIDFormat>urn:oasis:names:tc:SAML:2.0:nameid-format:transient
</NameIDFormat>
<AssertionConsumerService
Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST"Location="http://localhost:8080/odata4
/"
SAMLBasedSecurityForOData
772
index="1"isDefault="true"/>
</SPSSODescriptor>
<Organization>
<OrganizationNamexmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">JBoss</OrganizationName>
<OrganizationDisplayNamexmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">JBossbyRedHat</OrganizationDisplayName>
<OrganizationURLxmlns="urn:oasis:names:tc:SAML:2.0:metadata"
xml:lang="en">http://localhost:8080/odata4/</OrganizationURL>
</Organization>
<ContactPersoncontactType="technical">
<GivenName>The</GivenName>
<SurName>Admin</SurName>
<EmailAddress>admin@mycompany.com</EmailAddress>
</ContactPerson>
</EntityDescriptor>
</EntitiesDescriptor>
Createadeployment-overlayusingthecliwiththemodifiedcontents:
deployment-overlayadd--name=myOverlay--content=/WEB-INF/web.xml=/modified/web.xml,/WEB-INF/jboss-web.xml=/mo
dified/jboss-web.xml--deployments=teiid-odata-odata4.war--redeploy-affected
SAMLBasedSecurityForOData
773
ThisdocumentwillprovidedetailedinstructionstoenableOAuthV2authenticationonTeiid’sODatainterfaceusingtheKeycloakasauthenticationserver(IDP).PleasenotethatuseadifferentIDPserverwillnotworkwiththisimplementaionasOAuthimplementionsarenotinteroperable.ToworkwithseparateIDPthanKeycloakconsulttheirdocumentation,replacetheweblayersemantics,likethe"login-config"inweb.xmlfileetc.ProvidingthedetailsofotherIDPisbeyondthescopeofthisdocument.
Thisexampleswillshowcaseanexample,whereTeiid’sODatarestinterfaceissecuredusingOAuthusingKeycloakasIDP.TheVDBaccessedbytheODatainterfacealsodependsonanotherwebservicewhichisusedasadatasource,thatisalsosecuredwithOAuthusingthesameKeycloakIDP.Thecentralideabehindthisexampleistopassthesame"access-token"usedatODatainterfacelayertopassthroughtheTeiidlayertobottomdatasourcelayerandgainaccesstothesource.
DownloadandinstallKeycloakasaseparatewebserver.
Loginusingthedefault"admin/admin"credentialsintotheKeycloak"master"realm.
Addanewrealmcalled"oauth-demo"
Addanewusercalled"user"andaddcredentials.
Addtworoles"odata"and"user".Theseareenterpriseroles,thatwillbeusedbythewebservicestogranttheaccesstouser.Alsotheserolesareusedas"scopes"intheOAuthprotocol.
OAuth2BasedSecurityForODataUsingKeyCloak
774
Addanewclientcalled"odata4-oauth",thisclientrepresentstheTeiid’sODataclientthatwearegoingtocreate
andchoosescopes"odata"and"user"forthisclient.NotethattheredirectURIneedstobewheretheactualserviceisgoingtobeavailable.
Note
Theclientweb-servicetypicallydefineswhatrolesthatloggedinusermusthaveinorderfortogranttheaccess.IntheKeycloakOAuthimplementation,theserolesareusedas"scopes".Notethatthe"odata4-oauth"clientMUSThaveALLthescopesthatitisgoingtodelegatetheaccess-tokenforgainingaccesstobottomdataservices.InthisexampleTeiid’sODatawebservicesrequires"odata"role,thebottomweb-servicerequiresthe"user"role.SincetheODataaccessesthebottomweb-serviceitrequiresboththeroles.
OAuth2BasedSecurityForODataUsingKeyCloak
775
Addanotherclientcalled"database-service"andchoosescope"user".Choosetypeas"Bearer".
InstallandconfigureTeiidserver
DownloadandinstallTeiidserver
DownloadKeycloakadapterfortheEAP,andunzipovertheTeiidserverinstallation
DownloadKeycloakSAMLadapterforEAP,andunzipovertheTeiidserverinstallation.(optionalbutneedforanotherexercise)
Editthestandalone-teiid.xml,addthefollowingsections
RunthefollowingCLItoaddKeycloakspecificmodulestotheserver
OAuth2BasedSecurityForODataUsingKeyCloak
776
/extension=org.keycloak.keycloak-saml-adapter-subsystem:add(module=org.keycloak.keycloak-saml-adapter-subsystem
)
/extension=org.keycloak.keycloak-adapter-subsystem:add(module=org.keycloak.keycloak-adapter-subsystem)
abovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelike
<extensionmodule="org.keycloak.keycloak-saml-adapter-subsystem"/>
<extensionmodule="org.keycloak.keycloak-adapter-subsystem"/>
Addthesetwosubsystemsanywhereinthefile,usethefollowingtheCLIscript
/subsystem=keycloak:add
/subsystem=keycloak-saml:add
abovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelike
<subsystemxmlns="urn:jboss:domain:keycloak-saml:1.1"/>
<subsystemxmlns="urn:jboss:domain:keycloak:1.1"/>
Insecurity-domainsaddfollowingsecuritydomainsusingthefollowingCLI
/subsystem=security/security-domain=oauth:add(cache-type=default)
/subsystem=security/security-domain=oauth/authentication=classic:add
/subsystem=security/security-domain=oauth/authentication=classic/login-
module=oauth:add(code=org.teiid.jboss.PassthroughIdentityLoginModule,
flag=required,module=org.jboss.teiid)
/subsystem=security/security-domain=keycloak:add(cache-type=default)
/subsystem=security/security-domain=keycloak/authentication=classic:add
/subsystem=security/security-domain=keycloak/authentication=classic/login-
module=keycloak:add(code=org.keycloak.adapters.jboss.KeycloakLoginModule,
flag=required)
reload
abovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelike(youcanalsoeditstandalone.xmldirectly)
<security-domainname="oauth">
<authentication>
<login-modulecode="org.teiid.jboss.PassthroughIdentityLoginModule"flag="required"module="org.jbo
ss.teiid"/>
</authentication>
</security-domain>
<security-domainname="keycloak">
<authentication>
<login-modulecode="org.keycloak.adapters.jboss.KeycloakLoginModule"flag="required"/>
</authentication>
</security-domain>
UnderTeiidsubsystem,changethe"security-domain"ofthe"odata"transport,to
/subsystem=teiid/transport=odata:write-attribute(name=authentication-security-domain,value=oauth)
OAuth2BasedSecurityForODataUsingKeyCloak
777
resultsinXML
<transportname="odata">
<authenticationsecurity-domain="oauth"/>
</transport>
ThisfinishesalltheserversidechangesthatarerequiredtomakeOAuthauthenticationusingKeycloak.
ODataApplicationWARInordertouseOAuthauthentication,theODataWARneedstobeupdatedtomakeuseoftheOAuthbasedsecuritydomain.BydefaultTeiidinstallationcomeswithODatawebserviceWARfileconfiguredwith"HTTPBasic"authentication.ThisWARneedstoeitherreplacedorupdated.
BuildthenewODataWARfilethatsupportsOAuth.
TobuildOAuthbasedODataWARfile,Teiidprovidesatemplatemavenproject,eitherdownloadorclonetheprojectfromhttps://github.com/teiid/teiid-web-security
TheabovelinkprovidestemplatesforcreatingtwoWARfiles,oneWARfileistocreateTeiid’sODataservicewithOAuth,thenextisasample"database-service"forthisdemo.Pleasenotethat"database-service"istomimicthedatabaseservice,thatwillbedifferentinarealuse-case,howeverthestepsdefinedfortheaccesswillbesame.
Replacethe"teiid-web-security/teiid-odata-oauth-keycloak/src/main/webapp/WEB-INF/keyclock.json"filecontentswith"installation"scriptin"keycloak.json"formatfromKeycloakadminconsole’s"odata4-client"clientapplication.
Similarlyreplacethe"teiid-web-security/examples/database-service/src/main/webapp/WEB-INF/keyclock.json"filecontentswith"installation"scriptin"keycloak.json"formatfromKeycloakadminconsole’s"database-client"clientapplication.
tobuildtheWARfilesrunningthemavencommand
mvncleanpackage
TheabovecommandwillgenerateanewWARfilefordeployment.FollowthebelowdirectionstodeploythisnewWARfile.
TeiidServeronWildFly
Replacethe<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-odata4.war"filewithnewWARfile,byexecutingacommandsimilarto
{code}cpteiid-web-security/odata-oauth-keycloak/target/teiid-odata-oauth-keycloak-{version}.war<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-odata4.war{code}
JDVServer
IfyouareworkingwithJDV6.3serverorgreater,thenrunthefollowingCLIscript,youmayhavechangethebelowscripttoadopttothecorrectversionoftheWARanddirectorynameswherethecontentislocated.
undeployteiid-olingo-odata4.war
deployteiid-web-security/odata-oauth-keycloak/target/teiid-odata-oauth-keycloak-
{version}.war
oroverlaythenewoneusingCLIscriptlike
OAuth2BasedSecurityForODataUsingKeyCloak
778
deployment-overlayadd--name=myOverlay--content=/WEB-INF/web.xml=teiid-web-
security/odata-oauth-keycloak/src/main/webapp/WEB-INF/web.xml,/WEB-INF/jboss-
web.xml=teiid-web-security/odata-oauth-keycloak/src/main/webapp/WEB-INF/jboss-
web.xml,/META-INF/MANIFEST.MF=teiid-web-security/odata-oauth-
keycloak/src/main/webapp/META-INF/MANIFEST.MF,/WEB-INF/keycloak.json=teiid-web-
security/odata-oauth-keycloak/src/main/webapp/WEB-INF/keycloak.json/WEB-
INF/lib/teiid-odata-oauth-keycloak-{version}.jar=teiid-web-security/odata-oauth-
keycloak/src/main/webapp/WEB-INF/lib/teiid-odata-oauth-keycloak-{version}.jar--
deployments=teiid-olingo-odata4.war--redeploy-affected
WorkingwithexampleVDB
Editthestandalone-teiid.xmlandunderresource-adapterssubsystem,addthefollowingtoaddaccesstoadatabase-servicefromtheTeiidqueryengine.
<resource-adapterid="database">
<moduleslot="main"id="org.jboss.teiid.resource-adapter.webservice"/>
<transaction-support>NoTransaction</transaction-support>
<connection-definitions>
<connection-definitionclass-name="org.teiid.resource.adapter.ws.WSManagedConnectionFactory"
jndi-name="java:/database"enabled="true"use-java-context="true"
pool-name="teiid-database-ds">
<config-propertyname="SecurityType">
OAuth
</config-property>
<config-propertyname="EndPoint">
http://localhost:8180/database/
</config-property>
<security>
<security-domain>oauth</security-domain>
</security>
</connection-definition>
</connection-definitions>
</resource-adapter>
AddaVDBwithfollowingcontents(oauthdemo-vdb.xml)
<vdbname="oauthdemo"version="1">
<modelvisible="true"name="PM1">
<sourcename="array"translator-name="loopback"/>
<metadatatype="DDL"><![CDATA[
CREATEFOREIGNTABLEG1(e1integerPRIMARYKEY,e2varchar(25),e3double);
]]>
</metadata>
</model>
<modelname="view"type="VIRTUAL">
<metadatatype="DDL"><![CDATA[
createviewmessage(msgtostringprimarykey,msgfromstring,headingstring,bodystring)
as
SELECTA.msgto,A.msgfrom,A.heading,A.body
FROM
(EXECrestsvc.invokeHttp(action=>'GET',endpoint=>'sample',stream=>'TRUE'))ASf,
XMLTABLE('/note'PASSINGXMLPARSE(DOCUMENTf.result)COLUMNS
msgtostringPATH'to',
msgfromstringPATH'from',
headingstringPATH'heading',
bodystringPATH'body')ASA;
]]>
</metadata>
</model>
OAuth2BasedSecurityForODataUsingKeyCloak
779
<modelname="restsvc"type="PHYSICAL"visible="true">
<propertyname="importer.importWSDL"value="false"/>
<sourcename="restsvc"translator-name="ws"connection-jndi-name="java:/database"/>
</model>
</vdb>
StartbothKeycloakandTeiidServers.Ifbothoftheseserversareinthesamemachine,thenweneedtooffsettheportsofTeiidserversuchthattheywillnotconflictwiththatoftheKeycloakserver.Forthisexample,IstartedtheTeiidserveras
./standalone.sh-cstandalone-teiid.xml-Djboss.socket.binding.port-offset=100
whereallportsareoffsetby100.Sothemanagementportis10090anddefaultJDBCportwillbe31100.TheKeycloakserverisstartedondefaultports.
Testingtheexample
Therearetwodifferentmechanismsfortestingthisexample.Oneispurelyfortestingtheusingthebrowser,thenotherisprogramatically.TypicallyusingthebrowserisNOTcorrectforaccessingtheTeiid’sODataservice,butitisshownbelowfortestingpurposes.
UsingtheWebBrowser
Usingthebrowserissueaquery(theuseofbrowserisneededbecause,thisprocessdoesfewredirectsonlybrowserscanautomaticallyfollow)
http://localhost:8180/odata4/kerberos/auth
thenyoushouldseeamessagelike"Congratulations!!!Loginsuccessful..".Whatthisprocessisdoingisnegotiatinga"access-token"fromtheKeycloakauthenticationserverandplacesthisintheclient’sweb-session,suchthatsubsequentcallstotheserviceusethistokenforaccess.
Nowtofetchthedatafromthe"database-service"usingthenegotiated"access-token"issueaquery
http://localhost:8180/odata4/oauthdemo/view/message
Ifalltheconfigurationissetupcorrectly,thenyouwillseetheresponselikebelow.
<?xmlversion='1.0'encoding='UTF-8'?>
<a:feedxmlns:a="http://www.w3.org/2005/Atom"xmlns:m="http://docs.oasis-open.org/odata/ns/metadata"
xmlns:d="http://docs.oasis-open.org/odata/ns/data"m:context="$metadata#mesage">
<a:id>http://localhost:8180/odata4/saml.1/RestViewModel/mesage</a:id>
<a:entry>
<a:id>mesage('Tove')</a:id>
<a:title/>
<a:summary/>
<a:updated>2016-01-18T20:10:48Z</a:updated>
<a:author>
<a:name/>
</a:author>
<a:linkrel="edit"href="mesage('Tove')"/>
<a:categoryscheme="http://docs.oasis-open.org/odata/ns/scheme"
term="#saml.1.RestViewModel.mesage"/>
<a:contenttype="application/xml">
<m:properties>
<d:msgto>Tove</d:msgto>
<d:msgfrom>Jani</d:msgfrom>
OAuth2BasedSecurityForODataUsingKeyCloak
780
<d:heading>Reminder</d:heading>
<d:body>Don'tforgetmethisweekend!</d:body>
</m:properties>
</a:content>
</a:entry>
</a:feed>
Warning Whenabovemethodisusedtocaptureaccesstoken,itispossiblethattheaccesstokengetsexpiredafteritslifespan,inthatsituationanewaccesstokenneedstobenegotiated.
Callingprogramatically
Thisprocessofcallingdoesnotneedtoinvolveaweb-browser,thisistypicalofscenariowhereanotherweb-applicationormobileapplicationiscallingtheTeiid’sODataweb-servicetoretrievethedata.Howeverinthisprocess,theprocessofnegotiatingthe"access-token"isexternalizedandisdefinedbytheIDP,whichinthiscaseisKeycloak.
FordemonstrationpurposeswecanuseCURLtonegotiatethistokenasshownbelow(client_secretcanfoundtheKeycloakadminconsoleunderclientcredentialstab)
curl-vPOSThttp://localhost:8080/auth/realms/oauth-demo/protocol/openid-connect/token-H"Content-Type:appl
ication/x-www-form-urlencoded"-d'username=user'-d'password=user'-d'grant_type=password'-d'client_id=oda
ta4-oauth'-d'client_secret=36fdc2b9-d2d3-48df-8eea-99c0e729f525'
thisshouldreturnaJSONpayloadsimilarto
{"access_token":"eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiI0YjI4NDMzYS1..",
"expires_in":300,
"refresh_expires_in":1800,
"refresh_token":"eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJmY2JmNjY2ZC0xNzIwLTQwODQtOTBiMi0wMjg4ODdhNDkyZWYiLCJl..",
"token_type":"bearer",
"id_token":"eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiIwZjYyNDQ1MS1iNTE0LTQ5YjUtODZlNy1jNTI5MDU2OTI3ZDIiLCJleH..",
"not-before-policy":0,
"session-state":"6c8884e8-c5aa-4f7a-a3fe-9a7f6c32658c"
}
fromtheaboveyoucantakethe"access_token"andissuethequerytofetchresultslike
curl-k-H"Authorization:BearereyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiI0YjI4NDMzYS1.."http://localhost:8180/odata4/
oauthdemo/view/message
YoushouldseesameXMLresponseasabove.Pleasenotethattoprogramaticallyachievetheaccess_tokeninyourownprogram(notusingcurl)youcanseesomesuggestionsinthisdocument[http://keycloak.github.io/docs/userguide/keycloak-server/html/direct-access-grants.html]
OAuth2BasedSecurityForODataUsingKeyCloak
781
ThisdocumentwillprovidedetailedinstructionstoenableSAMLauthenticationonTeiid’sODatainterfaceusingtheKeycloakasauthenticationserver(IDP).SAMLisstandard,sothemodifiedODataWARshouldworkfinewithanyothercompatibleSAMLAuthorizationserver,howevertheconfigurationmaybelittledifferent.PleaseconsulttheirdocumentationforanysuchspecificsofdifferentauthorizationserverotherthenKeyCloak.
Thisexampleswillshowcaseanexample,whereTeiid’sODatarestinterfaceissecuredusingSAMLusingKeycloakasIDP.TheVDBaccessedbytheODatainterface,thepass-throughofSAMLAssertionforOAuthtoken(SAMLBearer)isnotyetavailableinKeyCloak,whenthefeatureisavailablethenTeiidwillsupportit.However,ifyouareworkingwithaIDPthatsupportstheSAMLBearer,Teiiddoessupportthemechanismwhereonecanpassthe"access-token"fromweblayertothedatasourcelayer.SeetheOAuthexampleastemplateandpossibleconfigurationneeded.(noteitisnotexactlysame,butverysimilar)
DownloadandinstallKeycloakasaseparatewebserver.
Loginusingthedefault"admin/admin"credentialsintotheKeycloak"master"realm.
Addanewrealmcalled"oauth-demo"
Addanewusercalled"user"andaddcredentials.
Addtworoles"odata"and"user".Theseareenterpriseroles,thatwillbeusedbythewebservicestogranttheaccesstouser.Alsotheserolesareusedas"scopes"intheOAuthprotocol.
SAMLBasedSecurityForODataUsingKeyCloak
782
Addanewclientcalled"odata4-saml",thisclientrepresentstheTeiid’sSAMLclientthatwearegoingtocreate
ClickonSAMLKeys,eitherimportyourcertificateorgenerateanewone.Thenclickexport,andkeeptheexportedcertificateforlateruse.
SAMLBasedSecurityForODataUsingKeyCloak
783
InstallandconfigureTeiidserver
DownloadandinstallTeiidserver
DownloadKeycloakadapterfortheEAP,andunzipovertheTeiidserverinstallation
DownloadKeycloakSAMLadapterforEAP,andunzipovertheTeiidserverinstallation.(optionalbutneedforanotherexercise)
Editthestandalone-teiid.xml,addthefollowingsections
RunthefollowingCLItoaddKeycloakspecificmodulestotheserver
/extension=org.keycloak.keycloak-saml-adapter-
subsystem:add(module=org.keycloak.keycloak-saml-adapter-subsystem)
/extension=org.keycloak.keycloak-adapter-
subsystem:add(module=org.keycloak.keycloak-adapter-subsystem)
abovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelike
<extensionmodule="org.keycloak.keycloak-saml-adapter-subsystem"/>
<extensionmodule="org.keycloak.keycloak-adapter-subsystem"/>
Addthesetwosubsystemsanywhereinthefile,usethefollowingtheCLIscript
/subsystem=keycloak:add
/subsystem=keycloak-saml:add
abovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelike
<subsystemxmlns="urn:jboss:domain:keycloak-saml:1.1"/>
<subsystemxmlns="urn:jboss:domain:keycloak:1.1"/>
SAMLBasedSecurityForODataUsingKeyCloak
784
Insecurity-domainsaddfollowingloginmoduleusingthefollowingCLI
subsystem=security/security-domain=teiid-security/authentication=classic/login-
module=RealmDirect:write-attribute(name=flag,value=sufficient)
/subsystem=security/security-domain=teiid-security/authentication=classic/login-
module=keycloak:add(code=org.keycloak.adapters.jboss.KeycloakLoginModule,
flag=sufficient)
reload
theabovecommandswillresultinXMLinstandalone.xmlordomain.xmlfilelikesimilarto:
<security-domainname="teiid-security">
<authentication>
<login-modulecode="org.keycloak.adapters.jboss.KeycloakLoginModule"flag="sufficient"/>
<login-modulecode="RealmDirect"flag="sufficient">
<module-optionname="password-stacking"value="useFirstPass"/>
</login-module>
</authentication>
</security-domain>
ThisfinishesalltheserversidechangesthatarerequiredtomakeOAuthauthenticationusingKeycloak.
ODataApplicationWAR
InordertouseOAuthauthentication,theODataWARneedstobeupdatedtomakeuseoftheOAuthbasedsecuritydomain.BydefaultTeiidinstallationcomeswithODatawebserviceWARfileconfiguredwith"HTTPBasic"authentication.ThisWARneedstoeitherreplacedorupdated.
BuildthenewODataWARfilethatsupportsSAML.
TobuildSAMLbasedODataWARfile,Teiidprovidesatemplatemavenproject,eitherdownloadorclonetheprojectfromhttps://github.com/teiid/teiid-web-security
TheabovelinkprovidestemplatesforcreatingtwoWARfiles,oneWARfileistocreateTeiid’sODataservicewithOAuth,thenextisforSAML.ChoosetheSAMLone.
Replacethe"teiid-web-security/teiid-odata-saml-keycloak/src/main/webapp/WEB-INF/keyclock.json"filecontentswith"installation"scriptin"keycloak.json"formatfromKeycloakadminconsole’s"odata4-saml"clientapplication.
Similarlyreplacethe"teiid-web-security/teiid-odata-saml-keycloak/src/main/webapp/WEB-INF/keystore.jks"filewiththeexportedkeystorefromearliersteps.
buildthe"keycloak-saml.xml"file,andaddallthesectionsof"metadata"specifictoyourservice.ThisiswhereserviceknowswhereIDPlocatedandwhichservicethisrepresentsetc.
ThebuildtheWARfilesrunningthemavencommand
mvncleanpackage
TheabovecommandwillgenerateanewWARfilefordeployment.FollowthebelowdirectionstodeploythisnewWARfiletotheserver
SAMLBasedSecurityForODataUsingKeyCloak
785
CommunityTeiidServeronWildfly
Replacethe<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-odata4.war"filewithnewWARfile,byexecutingacommandsimilarto
cpteiid-web-security/teiid-odata-saml-keycloak/target/teiid-odata-saml-keycloak-
{version}.war
<wildfly>/modules/system/layers/dv/org/jboss/teiid/main/deployments/teiid-olingo-
odata4.war
JDVServer
IfyouareworkingwithJDV6.3serverorgreater,thenrunthefollowingCLIscript,youmayhavechangethebelowscripttoadopttothecorrectversionoftheWARanddirectorynameswherethecontentislocated.
undeployteiid-olingo-odata4.war
deployteiid-web-security/teiid-odata-saml-keycloak/target/teiid-odata-saml-
keycloak-{version}.war
oroverlaythenewoneusingCLIscriptlike
deployment-overlayadd--name=myOverlay--content=/WEB-INF/web.xml=teiid-web-
security/teiid-odata-saml-keycloak/src/main/webapp/WEB-INF/web.xml,/WEB-INF/jboss-
web.xml=teiid-web-security/teiid-odata-saml-keycloak/src/main/webapp/WEB-INF/jboss-
web.xml,/META-INF/MANIFEST.MF=teiid-web-security/teiid-odata-saml-
keycloak/src/main/webapp/META-INF/MANIFEST.MF,/WEB-INF/keycloak-saml.xml=teiid-web-
security/teiid-odata-saml-keycloak/src/main/webapp/WEB-INF/keycloak-saml.xml,/WEB-
INF/keycloak.jks=teiid-web-security/teiid-odata-saml-keycloak/src/main/webapp/WEB-
INF/keycloak.jks--deployments=teiid-olingo-odata4.war--redeploy-affected
TestingtheexampleusingWebBrowser
TotestanySAMLbasedapplicationyoumustuseaWebbrowser.UsingabrowserissueanyODataspecificquery,andyouwillberedirectedtodoSAMLauthentication.
http://localhost:8180/odata4/<vdb>.<version>/<model>/<view>
SAMLBasedSecurityForODataUsingKeyCloak
786
top related