best practices for high availability and performance ... · performance tuning for oracle...

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|

BestPracticesforHighAvailabilityandPerformanceTuningforOracleGoldenGateMicroservices

NickWagnerDirectorofProductManagement- Replication

StephanHaisleyMaximumAvailabilityArchitecture(MAA)

October3rd,2017

Confidential– OracleInternal/Restricted/HighlyRestricted


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Confidential– OracleInternal/Restricted/HighlyRestricted 2


ProgramAgenda

MicroservicesArchitecture

OverviewofOracleActiveDataguard/FSFO

OverviewofnewGridInfrastructureBundledAgent(XAG)

ConfigurationforGoldenGatewithRACandDataGuard

Summary

1

2

3

4

5



ProgramAgenda





Summary

1

3

4

5


2

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| 5

OracleGoldenGate12.3NewFeaturesNewGoldenGate ServicesArchitectureRESTinterfacesforConfiguration,AdministrationandMonitoringwithincludedHTML5WebapplicationsDeployatcloudscalewithfullysecureHTTPSinterfacesandSecureWebSocketsforstreamingdata

OracleDatabaseSharding SupportEmbedded&fullyautomatedactive-activereplicationwithinandacrossDatabaseShards

AutomaticConflictDetectionandResolution(CDR)ConflictdetectionandresolutionbuiltdirectlyintoOracleDatabaseKernel

HeterogeneousEnhancementsGenericJDBCSupport.MySQL DDLreplication.Remotecaptureandapplyforz/OS

CloudIntegratedUIforDesign,Development,ManagementandMonitoringforGoldenGate CloudService

ParallelApplyHighlyscalableclientsideapplyenginewithgainsofover5xthroughputonOracle

EnhancedBigDataandNoSQL SupportSampleadaptersforOracleNoSQL,MongoDB.SupportforHiveMetadata

ExpandedOracleDatabase12.2SupportLongidentifiers,ExpandedSCN,LocalUndoforPDBs,Top-levelVARRAYs,REFs.ProceduralreplicationofAdvancedQueues,VirtualPrivateDatabase,OnlineRedefinition,….


NewServicesArchitectureforCloudandLarge-scaleDeployments

Source Target

TrailFiles

Extract

ManagerGGSCIAdminServer

TrailFiles

Replicat

PumpsDistributionService

CollectorsReceiverService

SourceDatabase

TargetDatabase

HTTPSRESTFulServiceInterfacesManagerGGSCI

AdminServer

MetricsServer

ServiceManager Service

Manager

MetricsServer

Confidential– OracleInternal/Restricted/HighlyRestricted


HowOracleGoldenGateWorks– MicroservicesArchitecture

SourceOracle&Non-OracleDatabase(s)

TargetOracle&Non-Oracle

Database(s)

Capture:committedtransactionsarecaptured(andcanbefiltered)astheyoccurbyreadingthetransactionlogs.Asof V.11.2.1,GoldenGateofferstwooptionsforcaptureforOracle;Classic&IntegratedCapture

Trail:stagesandqueuesdataforrouting.

DistributionServer/Receiver:distributesdataforroutingtotarget(s).

Route:dataiscompressed,encryptedforroutingtotarget(s).

Capture

Delivery

TrailFiles Dist.

Service

TrailFiles

Delivery

Capture

Bi-directional

LAN/WAN/InternetOverTCP/IP Trail

Files

TrailFiles

Delivery:appliesdatawithtransactionintegrity.NewwithGoldenGate12c,IntegratedDelivery.

Dist.Service

ReceiverService

ReceiverService

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| 8

CommandLine,Browsers,ProgrammaticRESTful InterfacesAdministerGoldenGate withVarietyofClients

ServiceInterface

Anything viaService IF

Scripts & Interpreters

PLSQL

Browsers

AdminClient

AdminServer

TrailFiles

ReplicatReceiverServer

TrailFiles

ExtractDistributionServer

HTTPS

ServiceManager

MetricsServer


SolutionsForEveryone


Administrators Traditionalists DevOps

• SimpletouseWebinterface

• Getupandrunningquickly

• Muchlowerlearningcurve

• CommandLineaccessviaadminclient – runanywhere

• SimilarGoldenGatecommandsasinGGSCI

• Interactionthesameaspreviousreleases

• UsesRESTfulAPIs

• Runanywhere

• Rollyourownmonitoring

• Automatecommonactivities


MicroservicesArchitecture:BrowserBasedClient



ProgramAgendawithHighlight


OverviewofOracleActiveDataGuard/FSFO



Summary

1

2

3

4

5



OracleActiveDataGuardDisasterRecoveryandRead-OnlyOffloadtoanActiveStandby

StandbySitePrimarySite

DataGuard

SYNC/ASYNC

PrimaryDatabase

ActiveStandbyDatabase

Read-writeWorkload

Real-timeReporting

FastIncrementalBackupsObserver


OracleDataGuardConcepts• Switchover:Plannedroletransitionfromaprimarydatabasetooneofitsstandbydatabases.– DGMGRL>SWITCHOVERTOCHICAGO• Requiresconnectivitytobothprimaryandstandbydatabases• Ensuresallredohasshipped• Orderlytransition(bothdatabasesagreeoneverything)

• Failover:Unplannedactivity– DGMRL>FAILOVERTOCHICAGO• Noconnectivitywithprimary• Newprimarybecomesthesourceoftruth

OracleConfidential– Internal 13


ProgramAgendawithHighlight


OverviewofOracleActiveDataGuard/FSFO



Summary

1

3

2

4

5



OracleGridInfrastructureBundledAgent(XAG)• ClusterwarespecifictomanagingGoldenGateresources– XAGallowsyoutoregisteraGoldenGateinstancewithCRStoprovideHA

• ItsolvesthekeyprocessrelatedissueofensuringavailabilityoftheGoldenGateinstanceinthefaceoffailures.

• Lossofinstance(RACnodefailover)

• Lossofprimarydatabase(DataGuardFailoverintegration)

• UseAGCTLforregistering andstarting/stoppingresources

15


OracleGridInfrastructureBundledAgent(XAG)• SupportcominginGoldenGate 12.3.0.1.2 (notyetavailable)• NewversionofXAGforGoldenGateMicroserviceswillbemadeavailableat*

http://www.oracle.com/technetwork/database/database-technologies/clusterware/downloads/index.html

• InstalloutsideofGridInfrastructureORACLE_HOMEandmakesureOSuserPATHfindsthisXAGbeforetheGIinstalledversion

16

*ThecurrentversionofXAG(v8)doesnotsupportGoldenGate Microservices Architecture


OracleGridInfrastructureBundledAgent(XAG)HighLevelConfigurationSteps

• CreateOGGdeploymentusingoggca.sh(similartoDBCA)• Chooseto‘IntegratedwithXAG’(seenextslide)• RegisterServiceManageranddeploymentwithXAG– CreatestheCRSresourcesandalldependencieswithfilesystem,rolebasedservice

• XAGqueriestheGoldenGate ServiceManagerforhealthofthedeployment• XAGinteractsdirectlywithServiceManager,nolongerrunningGGSCIandparsingtheoutputfromcommands–Moreintegrated,simplerandrobust



OracleGridInfrastructureBundledAgent(XAG)ServiceManagerCreation



ProgramAgenda




ConfigurationforGoldenGatewithDataGuard

Summary

1

2

3

4

5



SampleDeployment

20

Observer

PrimaryDatabase StandbyDatabase

RedoTransport(SYNCorASYNC)

IntegratedExtract Logmining

Server

TrailandotherOGGFilesInDBFS

RedoTransport

OCIConnection

FileI/OWarehouse

BidirectionalGoldenGateReplication


SampleDeployment– PostRoleTransition

21

Observer

(OLD)PrimaryDatabase (NEW)PrimaryDatabase

IntegratedExtractLogMining

Server

Trail/Checkpoint/BRFilesInDBFS

LogMiningServer

RedoTransport

OCIConnection

FileI/O

Warehouse

BidirectionalGoldenGateReplication


DatabaseConnectivity• MakeallOGGcomponentsconnecttodatabaseusingRole-BasedServices– Declarativewaytospecifyaserviceshouldbepublishedonlywhenthedatabasehasaspecificrole– PublishaserviceonlywhendatabasehasthePRIMARYrole

• Createrole-basedservice(12csyntax):srvctl add service -db GGS2PRMY -service oggserv -role PRIMARY

-preferred GGS21 -available GGS22

srvctl add service -db GGS2STBY -service oggserv -role PRIMARY -available GGS21

22


GoldenGate ConfigurationFileSystem&HostnameRequirements

• ConsistencyofGoldenGatedataandmetadataensuredwithDBFS• StoretrailandcheckpointdataonDBFStoguaranteeconsistencyonroletransitions

• DBFSoffersintegratedhandsoffautomaticfilesystemmounting,replicatedthroughDataGuard

• Providesflexibleroutingoftraildatapostroletransition– UseaVirtualHostname toidentifytheTARGEThostmachine• Transferredbetweenprimaryandstandbyonroletransition

– UsethevirtualhostnamewhencreatingaDistributionServerpath

23


CreateDBFSCRSResource• DBFSismountedonRACnodethatwillrunGoldenGate• CreateCRSactionscript(MOSnote1054431.1)–MustcontainSTART,STOP,CHECKfunctions

• CreateCRSresource:crsctl addresourcedbfs_mount -typecluster_resource \

-attr "ACTION_SCRIPT=$ACTION_SCRIPT,CHECK_INTERVAL=30,RESTART_ATTEMPTS=10,\

START_DEPENDENCIES='hard(ora.$DBNAME.db)pullup(ora.$DBNAME.db)',\

STOP_DEPENDENCIES='hard(ora.$DBNAME.db)‘,SCRIPT_TIMEOUT=300"

• DBFSservicewillbemanagedby XAG

24


GoldenGate ConfigurationServiceManagerCreation



GoldenGateConfigurationDeploymentAutoStart

• EachdeploymentisautomaticallyenabledtoautostartwhenServiceManagerstarts• StateofdeploymentispersistentacrossServiceManagerrestarts– IfdeploymentwasrunningwhenServiceManagerwasstopped,thedeploymentwillbestartedautomaticallywhenServiceManagerstarts.

• MustenableGoldenGate Extract/Replicat autostart– Extract,DistributionPathandReplicat autostartisenabledwiththeAdministrationServeruserinterface



GoldenGate ConfigurationGoldenGate ProcessAutoStartandAutoRestart



• Autorestarttaskscanalsobecreatedusingadminclient:OGG (not connected) 1> HELP SHOWSYNTAX *AUTORESTART*

Admin Client Command Summary:ADD AUTORESTART - Create an autorestart rule.Syntax:| ADD AUTORESTART <rule-name> ( EXTRACT | REPLICAT | ER ) <group-name-wildcard>| [ RETRIES <retries-number> ]| [ WAITSECONDS <wait-number> ]| [ RESETSECONDS <reset-number> ]DELETE AUTORESTART - Remove an autorestart rule.Syntax:| DELETE AUTORESTART <rule-name>INFO AUTORESTART - Returns information about autorestart rules.Syntax:| INFO AUTORESTART <rule-name-wildcard>


GoldenGate ConfigurationGoldenGate ProcessAutoRestart


RegisterGoldenGate withBundledAgent(XAG)• RegisterwithXAGattheprimary(asroot)

agctl addgoldengate GGPRMY--gg_home /u01/oracle/goldengate/gg123_MA--service_manager--config_home /mnt/dbfs/goldengate/deployments/ggsm01/etc/conf--port9100--adminuser admin--useroracle--groupoinstall --vipname xag.GG_MA1-vip.vip--filesystems dbfs_mount --db_services ora.ggdg.oggserv.svc --dataguard_autostart yes

• RegisterwithXAGatthestandby(asroot)agctl addgoldengate GGSTBY--gg_home /u01/oracle/goldengate/gg123_MA--service_manager--config_home /mnt/dbfs/goldengate/deployments/ggsm01/etc/conf--port9100--adminuser admin--useroracle--groupoinstall --vipname xag.GG_MA1-vip.vip--filesystems dbfs_mount --db_services ora.ggdg.oggserv.svc --dataguard_autostart yes--nodesstdby1

• StartExtractusingAgentControl– agctl startgoldengate ggprmy


GoldenGate withBundledAgent(XAG)ExampleCommands

• CheckGoldenGate statusagctl statusgoldengate ggprmy

• StartGoldenGateagctl startgoldengate ggprmy --nodeGGsourceNode1

• StopGoldenGateagctl stopgoldengate ggprmy

• Replocate GoldenGate toanotherRACnodeagctl relocategoldengate ggprmy --nodeGGsourceNode2



DataGuard– BoundedDataLoss(ASYNC)• DataGuardinMaxPerformance(ASYNC)modepermitsdatalossonthestandby• AmountofdatalosswithFSFOcontrolledbyFastStartFailoverLagLimit(defaultis30seconds)• ExtractmustonlyminefromredoalreadyappliedtoStandby– Preventsdatabeingreplicatedtoatargetdatabaseandthenmissingfromthesourceafteradatalossfailover

• AddIntegratedExtractparameterTRANLOGOPTIONS HANDLEDLFAILOVER

• IfFSFOisnotenabled,specifyTRANLOGOPTIONS FAILOVERTARGETDESTID n• IdentifieswhichlogarchivedestinationofstandbyExtractmustremainbehind

31


DataGuard– BoundedDataLoss(ASYNC)EffectsofTRANLOGOPTIONSHANDLEDLFAILOVER

• AfterDGfailoverifnewstandby(oldprimary)cannotbequeriedforredoapplystatus,Extractabends with:ERROR OGG-02077 Extract encountered a read error in the asynchronous reader thread and is abending: Query to retrieve applied SCN of the target standby database failed.

• Canconfiguretimeouttoallowstandbyreinstatementafterroletransition– TRANLOGOPTIONSDLFAILOVER_TIMEOUT300– Ifstandbynotavailablebytheendofthetimeout,Extractwillabend.

• NOTE:Duringtimeoutperioddataisbeingreplicatedthathasnotyetbeenappliedtostandby,sothereistemporarydatadivergence.



GoldenGate withRACNodeFailurePerformanceMetricsServer



GoldenGate withDataGuardRoleTransitionTargetReceiverServer


AfteraDataGuardroletransitionthetargetdatabaseReceiverServerautomaticallyrefresheswithnewsourcehost.


ConfigurationRecommendationSummary• BundledAgentinstalledoutsideofGridInfrastructureinstalldirectory• GoldenGate 12.3.0.1.2andabove• GoldenGate IntegratedExtractandIntegratedReplicat• Role-basedservices forGoldenGate databaseconnectivity• DBFSfor GoldenGate sharedfilesystem• IntegratedExtractparameters• TRANLOGOPTIONSHANDLEDLFAILOVER• TRANLOGOPTIONSFAILOVERTARGETDESTIDn• TRANLOGOPTIONSDLFAILOVER_TIMEOUT300

35

best practices for high availability and performance ... · performance tuning for oracle...

Documents