zero data lossrecoveryappliance - oracle.com · windows(2-6 hours) versus waiting ... ra replica:...

ZeroDataLoss Recovery Appliance:Maintenance&OperationalBestPractices

MAABestPracticesTeamServerTechnologiesJuly2018

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.|

SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

3


Agenda

4

UnderstandRequirementsFirst

RADeploymentDecisionsandKeyPractices

RARolesandResponsibilities

RAStayHealthyPlan

BestPracticesBackupandRestore

OracleSupport&ServiceRequest

1

2

3

4

5

6

Copyright©2017, Oracleand/oritsaffiliates.Allrightsreserved.| 5

MAARecommendations

• OneRecoveryAppliance(RA)perdatacenter

• BackupprimaryandstandbydatabasestotheirrespectivelocalRA

• NoRAreplicationforanydatabasewitharemotestandby

• RestoreoperationcanuseanyRAinanylocation


DoyourbusinessrequirementscallformultipleRecoveryAppliances?Questionstoask:

1. Doyouneedtheabilitytobackup,restoreandrecoverduringRAplannedmaintenancewindows(2-6hoursformajorupgrades)versuswaitinguntiltheplannedmaintenancecompletes?

2. Doyourequiredisasterrecoveryprotection?(e.g.datacenterfailure,powerfailure)

3. DoyourequireminimaldatalossforallprotecteddatabasesbackinguptoRAduringitsplannedmaintenancewindows?

4. DoyourequirearchiveandbackupsoperationstocontinueduringRAplannedmaintenancewindows(2-6hours)versuswaitinguntiltheplannedmaintenancecompletes?

If youanswered“Yes” ,toanyofthesequestionsmultipleRecoveryAppliancesarerequired.

Yes/No

Yes/No

Yes/No

Yes/No


MultipleRecoverAppliancesRequired?

q Didyouanswer“yes”?

ü ToANYofthequestions:YouwillneedanotherRecoveryAppliance

ü Yestoquestion#1or#2:UseBackupandRecoveryFailovertoReplicaApplianceSolution

ü Yestoquestion#3or#4Only:UseBackupFailovertoAlternateApplianceSolution

Note:ifyouhaveDataGuardorGoldenGate?Backuptheproduction,standbyorGoldenGatereplicatotheirrespectivelocalRA.


TapeLibrary

OneWay

Bi-Directional

Hub&Spoke

RemoteDataCenterLocalDataCenter

RAReplica:DataLossProtectionfromSiteDisasters

BENEFITS• ReplicationtoRemote

Applianceprotectsdatafromdisastersorsitefailures

• AutomatedrestorefromLocalApplianceordirectlyfromRemoteAppliance

8


RAOsakanormallyreplicatestoRATokyo

Whenupstreamappliance(RAOsaka)isnotavailable,backupsandredoareredirectedtoRemoteappliance(RATokyo)

• Virtualfullsarecreatedasnormal

• SizingDRapplianceIngeneral,sizeperRecoveryWindowGoal(RWG)businessrequirement:1xfullbackup+NRWGdaysofincrementalandredo/archlogbackupsBareminimum:1xfullbackup+1dayredo/archlogsbackups.

Whenupstreamisbackonline,DRappliancebackupsaretransferred

• Backupsareingestedandprocessedintovirtualfulls

• Normalbackupstoupstreamcanberestartedimmediately

• Virtualfullsfornewbackupsarecreatedafteralltransferredbackupshavecompletedprocessing

Benefits

• BestpracticetopreserveHAduringplannedorunplanneddowntime

• Databasebackup&restore/recoverabilityavailablefromeitherupstreamanddownstream

9

X

Backup& RecoveryFailovertoReplica

RAOsaka

Replication

RAOsaka RATokyo

RATokyo

Replication

RAOsaka RATokyo

BackupstoDRAppliance

DRAppliancebackupstransferredtoUpstream


BackupFailovertoAlternateRA• IncrementalsandRedonormallysenttoPrimaryRA

• AlternateRAservesasbackupstagingareawhenprimaryRAisunavailable,thensyncswithprimaryRAafterwards– Novirtualfullscreatedonalternate,hencerecoverabilitynotsupported

– Spacesizedfor‘n’incrementalsandarchivedlogbackupsduringprimarydowntimeperiod

• Benefits:– Preservesbackupandredoshipmentcontinuityduringplannedmaintenance/upgrades

– PreventslocalFastRecoveryAreasfromfillingupwitharchivedlogs

– BlockChangeTrackingcontinues

10

RAOsakaPrimaryAppliance

XRATokyoAlternateAppliance

Alternateappliancetakesoverbackupsandredotransport

Incrementals&

Redo/Arch

“BF_FORWARD”PolicyonAlternateRA:STORE_AND_FORWARD=‘YES’


BackupFailovertoAlternateRA- Continue• Backupandredofailoverdestinationwhenprimaryapplianceisunavailable– Alternateappliancetakesoverbackupsandredotransport

– Whenprimaryapplianceisbackonline,allinterimbackupsarereplicatedfromalternateandvirtualfullbackupsarecreatedonprimary

– Onceallvirtualfullsarecompleted,backupsandredotransportcanrestarttotheprimaryappliance

11

“BF_FORWARD”PolicyonAlternateRA:STORE_AND_FORWARD=‘YES’

RAOsakaPrimaryAppliance

BFOSAKA

“BF_BACKUP”PolicyonPrimaryRA

RATokyoAlternateAppliance


MultipleRecoveryApplianceSolutionSummary

– BackupandRecoveryFailovertoReplicaAppliance• BackupandRedotoRAReplicaduringRAmaintenanceandunplanneddowntime• RestorefromRAReplica• MorecapacityrequiredduetoRAreplication

– BackupFailovertoAlternateAppliance• BackupsandRedotoAlternateRAduringRAmaintenanceandunplanneddowntime• RestorefromAlternateNOTavailable• LesscapacityrequiredbecauseRAreplicationwasnotsetup

12


RecoveryAppliance&DataGuard

PrimaryDatabase

RemoteStandbyDatabase

DataGuardTransportsync/async

FastIncremental FastIncremental

Redo(ASYNC)

Redo(ASYNC)

MAARecommendations:• OneRecoveryAppliance(RA)perdatacenter• BackupprimaryandstandbydatabasestothelocalRA• NoRAreplicationforanydatabaseswitharemotestandbyneeded• RestoreoperationcanuseanyRAinanylocation

StandbyLocationPrimaryLocation

Fastincremental(anincrementalusingBlockChangeTracking)onastandbydatabaserequiresanActiveDataGuardLicense.

RAReal-TimeRedodoesNOTrequireanActiveDataGuardLicense.


RecoveryApplianceandDataGuard

• PostDataGuardroletransition– Nochangeinbackupoperations.ContinuetobackupboththeprimaryandstandbydatabasestothelocalRA

• DeployingtheZeroDataLossRecoveryApplianceinaDataGuardConfiguration– Refertohttp://www.oracle.com/technetwork/database/availability/recovery-appliance-data-guard-2767512.pdf / DeployingZeroDataLossRecoveryApplianceinaDataGuardConfiguration (UpdatedMarch2018)

MaximumAvailabilityArchitecture(MAA)WhitePaper

14


Agenda

15




RAStayHealthyPlan



1

2

3

4

5

6


NetworkConfigurationOptions

• ZDLRASupports10GigE/25GigEandInfiniBand– 10GigEor25GigEistherecommendedforprotecteddatabasetoRAconnectivity• EthernetbasedconnectivityprovidesoptimalbackupandrestoreratesduetoRA’sresourcemanagement.

• Note:WhileIBconnectivityissupported,beawareofthefollowing:– IBwillnotrealizethefullIBnetworkbandwidthduetoRAresourcemanagement(managedthrottling).– IBsetupwillintroducesoftwareupdatecomplexity.– Owner’sGuide,Chapter9hasdetailsonhowtoconfigureBackupIngestoverIB

– Note:Real-TimeRedoTransportuses10GigEor25GigEnetworkONLY

16


NetworkConfigurationOptions- continue

• VLANfornetworkisolation– BackupandrestoretrafficfromdifferentVLANsisnotrouted.– RecoveryAppliancesupportsVLANtaggingontheingestnetwork• ProtectedDBhostsondifferentandisolatedVLANscanbeconnecteddirectlytotheRA• Enabling8021.QVLANTagginginZeroDataLossRecoveryApplianceOverIngestNetworks(DocID2047411.1)

• Replication:TypicallyconfiguredviaOEDAbeforeRASWinstallation– Replicationconfiguration– PostInstall• PostInstall- ReplicationNetworkConfigurationforZDLRA(DocID2126047.1)

17


RecoveryApplianceSecurity

• ClienttoRecoveryAppliance,orRecoveryAppliancetoClient– Upcoming:SecurityinFlight(TCPS,HTTPS)è Provideduponrequest– AddingaprotecteddatabaseRecoveryApplianceVPCusercredentialstoanexistingOIDwallet(DocID2211759.1)

• SecurityintheRecoveryAppliance– RecoveryApplianceadministratorsresponsibilities• CreateVirtualPrivateCatalog(VPC)User• AssignprotecteddatabasestoaspecificVPCUser• TheprotecteddatabaseadministratorcanseealldatabasesthatshareacommonVPCuser

Customersrequiringendtoendsecurity

18


Agenda

19




RAStayHealthyPlan



1

2

3

4

5

6


TypicalquestionsfromnewZDLRAcustomers

?

NewZDLRA

Customer

1.Who willmanagethis?

2.How willmyteamsupportit?

Thispresentationprovidesinformationtohelpyouanswerthesequestions.


TheStartingPoint:ASetofSpecialistTeams

Database AdministratorsDB DB DB

Storage AdministratorsDisk DiskDiskDisk

System AdministratorsServerServer

Network AdministratorsNetwork

Security AdministratorsResponsesEventsPolicy


Option1:MultipleSpecialistTeams(statusquo1)

Database AdministratorsStorage Administrators

System Administrators

Network Administrators

Security Administrators

1- EachteamisassignedaspecificZDLRAsetofresponsibilities.


Option2:ExtendedRATeam





Primarystakeholders1

Secondarystakeholders2

1-PerfrommainZDLRAmonitoringandadministrationactivities.

2-Perfromadditionaltasksrequiredbythemainstakeholders.


Option3:RAMachineAdministrator





Centralizedstakeholders1

1-ResponsibleforallZDLRAactivitiesincludingmonitoring,administrationandmaintenance.


OperationalModel1:MultipleSpecialistTeams

25

RACILegend:R =ResponsibleA =AccountableS =SupportiveC =ConsultedI =Informed

RASCIModel- MultipleSpecialistTeams

Task DBAOS

SYSADMIN NETWORK STORAGE CIO SECURITY

Day-to-dayOperation R R R A C

Testing R R R A C/IMonitoring R R R A C/IConfigurationManagement R R R A C/IPatching

StorageCells R A C/IInfiniBand C R A C/ICiscoPublicNetwork C R A C/IDbComputeNodesOS C R A C/IDatabase R A C/I

BackingUpRA R A C/IUpgradingSW R R A C/IReplacingHW R R A C/IMeteringandCharging






OperationalModel2:ExtendedRATeam(ERMA)

26


RASCIModel– ExtendedRATeam

Task DBAOS


Day-to-dayOperation R S S A C

Testing R S S A C/IMonitoring R A C/IConfigurationManagement R A C/IPatching

StorageCells R A C/IInfiniBand R S A C/ICiscoPublicNetwork R S A C/IDbComputeNodesOS R S A C/IDatabase R A C/I

BackingUpRA R A C/IUpgradingSW R S S A C/IReplacingHW R S S A C/IMeteringandCharging






OperationalModel3:RAMachineAdministrator(RAMA)

27

RASCIModel- RAMACHINE ADMINISTRATOR

Task DBAOS


Day-to-dayOperation R A C

Testing R A C/IMonitoring R A C/IConfigurationManagement R C A C/IPatching

StorageCells R A C/IInfiniBand R C/I A C/ICiscoPublicNetwork R C/I A C/IDbComputeNodesOS R A C/IDatabase R A C/I

BackingUpRA R A C/IUpgradingSW R A C/IReplacingHW R A C/IMeteringandCharging







SummaryofStaffingModels

Allthreemodelswillwork.

Youdecidewhatisbestinyourorganization.


Agenda

29




RAStayHealthyPlan



1

2

3

4

5

6


AdjustingDefaultSettings• DonotmakeanychangestoRecoveryAppliance– IT’SANAPPLIANCEandit’salreadyoptimized– RestrictionsandsupportedconfigurationexceptionsinMOSnote2172842.1

• IfrequireduseMAX_RETENTION_WINDOW toenforcehardlimitsondataretentionforalldatabaseswithinaprotectionpolicy.–Usecaseswheretherearestringentcompliance/regulatoryrequirements• Bestpracticesistosetthevalueto5daysorgreater(50%abovetheRecoveryWindowforlargeranges)

– Backupsareforciblyremovedafterexceedingwindow– RecoveryWindowGoalsshouldbeusedtomanagebackupspaceconsumption– UsinganaggressiveMAX_RETENTION_WINDOW(onenotadheringtobestpractice)maydelaybackupprocessingandindexingduetoincreasedandlessefficientPURGEtaskjobs.

30


UseRecommendedRASoftware• SubscribetoMOSalertsandreferperiodicallytothefollowingnotes:

• RecoveryApplianceCriticalIssuesMOSnoteforcriticalissuesalerts• RecoveryApplianceSupportedVersionsMOSnoteforlatestsoftwareupdate

• UseRecommendedRASoftwaretoavoidknowncriticalissues– Number1methodtoavoidproblems:Upgradetorecommendedsoftwarerelease– ZeroDataLossRecoveryApplianceSupportedVersions(DocID1927416.1)– ZeroDataLossRecoveryApplianceUpgradeandPatching(DocID2028931.1)– Patchesarecumulativeandinclude

- Bugfixes(Avoidabugbeforeithappens)- Enhancements(ModificationtoRAprocessflow)

• CoordinatewithPlatinumPatching– Scheduleearly&OpenProactiveSR

31

VeryImportant


Don’ts

– Don’tignoreincidents:• Resolveandunderstand

– Don’tneglecttheRA• Monitorthesystem

– Don’tSubmitmultiplerequesttoDeletedatabases• Multiple– concurrentdatabasedeletionscanhaveanegativeimpactontheRA• AvoidDeleteDatabasecommandsasmuchaspossibleorlimitdeletionsto1or2concurrentlyforlargedatabasedeletions• Startwiththesmallestandworkup,monitorforasuccessfulcompletion.

32


Don’ts

– Don’trepeatbackuporrecoveryscriptsafterafailure• Firsttroubleshootthereason.MostcasesithasnothingtodowithRecoveryAppliance.• Rapidandrepeatedexecutionsofascriptcancausevariousissues

– Don’tmakemodificationstotheRAconfiguration• ZeroDataLossRecoveryAppliance- InstallingThird-PartySoftwareandModifyingInternalApplianceSoftware(DocID2014361.1

– Don’ttakeperiodicLevel0• VirtualLevel0requiresonlyonelevel0followedbylevel1s• Note:Alevel0mayberequested– BySupport– TopreserverecoverabilityineventofbackupcorruptiondetectedontheRA

33


Don’ts– Don’tbackupaDataGuardprimary& standbytothesameRA• SeeDeployingtheZeroDataLossRecoveryApplianceinaDataGuardConfiguration

– Don’tregister databaseswithidenticalDBIDstothesameRA• SeeCloningaProtectedDatabase”withintheZDLRAProtectedDatabaseConfigurationGuide

– Don’tbackuptoanothermedia• SwitchingtoanothermediacanimpactpastandfuturebackupsmadetotheRA

– Dualbackupstrategiesarecomplexandshouldbeavoidedifpossible• Dualbackupstrategiesaredesignedformigration• DualBackupStrategyMOSNotes:– ImplementingaDualBackupStrategywithBackupstoDiskandRecoveryAppliance(DocID2154461.1)– ImplementingaDualBackupStrategywithBackupstoTapeandRecoveryAppliance(DocID2154471.1)

34


Dos

– Validatethebackup/restorenetworkfirst:• UseZeroDataLossRecoveryApplianceNetworkTestThroughputscript(DocID2022086.1)– requiresqperf,NOTOSagnostic

• HowtomeasurenetworkperformancefromRMANforZDLRAorCloudBackups(DocID2371860.1)– usesRMAN’s“NETTEST”option,OSagnostic

– FileaNEWSRforanynewissue• RefertoServiceRequestsandEscalationProcesssection

– SetNLS_DATE_FORMATbeforecallingrmanscripts• e.g.inLinux:exportNLS_DATE_FORMAT‘yyyy-mm-ddhh24:mi:ss’

35


Dos

– UseMultiSection:setSectionSizeto64GB• Numberofsectionsperdatafileislimitedto256.• Whenasectionsizeof64GBisused:– Largedatafiles>16TBwillautomaticallyseeanincreaseinsectionsize• if(sizeof(datafile)>16TB),sectionsize=sizeof(datafile)/256

– Smalldatafiles<64GBwillnothavesections• A64GBsectionsize:allowsforefficientprocessinginZDLRA’sflashcache– Forcesfilespersetto1

– Usethelatestlibra(TheRMANclientsbtlibrarythatsupportsZDLRA)• ZDLRA:Wheretodownloadnewsbtlibrary(libra.somodule)(DocID2219812.1)

36


MonitoringRA’sHealth

• MonitortheApplianceonadailybasis– EMUnifiedManagementDashboard• Reviewtwicedaily

– SystemActivityScript(DocID2275176.1)• Rundailyandmonitortrends

– EMnotifications• Reviewandactonnotifications

– RunexachkMonthlyandreviewfindings• HowToupdateexachkoutsideZDLRAInstall,PatchingandUpgrade(DocID2399688.1)• Usedifftocomparemonthtomonth• Runpreandpostpatching

– ReviewCapacityPlanningReportMonthlyorBi-Monthly

37


IstheZDLRAHealthy?OracleEnterpriseManager

38


UnifiedEnd-to-EndControl

• RecoveryApplianceAdmincentrallymonitorsandmanagesalldatabaseprotectionactivityacrossalltiers

• DatabaseAdminmonitorstheprotectionstatusoftheirdatabasefromdisk,totape,toreplica–OffloadedreplicasandtapebackupsappearinRecoveryCatalog

Tape

RemoteAppliance

Enterprise Manager

Tape

39


BestPractice– UnifiedManagementDashboard– 1)Summary

• Showsthenumberofprotecteddatabases,andsummarizestheirhealthstatus,currentactivity,andactivitywithinthelast24hours.Formoreinformation,clickthelinksintheOperationcolumn:Backup,Copy-to-Tape,Replication,andRestore.

– 2)ProtectedDatabaseIssues• Highlightsissuesrelatingtobackupandrecoverystatusforprotecteddatabases.TheViewmenufiltersthedataonkeycategories.

– 3)DataSent/Received(Daily)• Displaysdailythroughputoverthepastweek.

– 4)Performance• ChartsperformancestatisticsforDataRateandQueuedData.Thestatisticsarefilterablebyday,week,ormonth.

– 5)MediaManagers• Displaystheconfiguredmediamanagerforcopy-to-tapeoperations.

– 6)StorageLocations• Summarizestotalavailablespaceandusagebyindicatinghowmuchhasbeenconsumedtomeetthediskrecoverywindowgoalforall

databases,andwhatpercentageoftotalspaceisreservedspacefordatabasesbackinguptothespecifiedstoragelocation.

– 7)Replication• ListsthedownstreamRecoveryAppliancestowhichthisRecoveryApplianceisreplicating,andalsotheupstreamRecoveryAppliances from

whichthisRecoveryApplianceisreceiving

– 8)IncidentsandEvents• SummarizesallwarningsoralertsthathavebeengeneratedbyCloudControlmonitoringofalltargetsassociatedwiththeRecoveryAppliance.

Fromthissection,drilldownforfurtherdetailontheissues.

40


BestPractices– UnifiedManagementDashboard- Continue

1

2

3

4

6

5

7

8


BestPractices– LeverageOEMnotifications

RecoveryAppliancealertnotificationsetupusingOracleEnterpriseManager(DocID2262003.1)

Exampleofalertsandnotificationforoldbackups:


ZeroDataLossRecoveryApplianceSystemActivityScript• ZeroDataLossRecoveryApplianceSystemActivityScript(DocID2275176.1)

• Containsmultiplequeries,providing:– Catalogversion– Generalstateofthesystem– Examinationofrunningtasksonthesystem– Taskhistoryforthelastday– Spaceusage– Lockinginformation– Checkstatusofreplicationserverifitexists– Incidentsforthelastfivedays– Displayeachdatabase'scurrentprogressprocessingtheirdatafiles– APIcommandsoverthelast2weeks– Notableconfigchanges

43


IstheZDLRAHealthy?SystemActivityReport

44

• Itemstowatchfor:BadreportWatchfor:

IftherearetasksofWORKTYPEinRUNNINGstateandcreatedadayearliertheninvestigate.

IftherearetasksofMAINTENANCEorSBTTYPEpresentandtheircreationtimeisolderthanoneweekthenitshouldbeinvestigated.

Alargenumberoftaskinexecutablestateforthesametask_type.

Thesametasktypeisincreasinginthenumberofjobs(current_count)overtime.Asystemwithorderingwaitsthatareolderthan1day.

Asystemwithtasksinstall_when_wait.ThisshouldonlybeseenifOracleSupportistroubleshootingtheRA.


IstheZDLRAHealthy?SystemActivityReport

45

TheGood:

Minimumcreationtimeforactivetasksiswithinthelast24hoursforworktasks.

Taskhistorystateshouldindicateworkisbeingcompleted.

Reviewothersections


IstheZDLRAHealthy?exachk

46

UpdateexachkforRA:• HowtoupdateexachkoutsideZDLRAInstall,Patching

andUpgrade(DocID2399688.1)


exachk&RAInformationCenter

InformationCenter:OverviewZeroDataLossRecoveryAppliance(DocID1683791.2)


HowtoDetermineIfZDLRAIsKeepingupWithLoad• Determiningwhetherunprocessedtasklistisgrowing(over7dayperiod):

SELECT count(*) FROM ra_task

WHERE archived=‘N’;

• Determinewhetherincidentsareactivefordelaysinperformingbusywork:SELECT error_text

FROM ra_incident_log WHERE status=‘ACTIVE’

AND error_text like '%has not run%';

• Taskswithhighestnumbers(RA_TASK.PRIORITY) arebusyworktasks– DB_STATS_REFRESH,RESTORE_RANGE_REFRESH,OPT_DF,OPTIMIZE,REBUILD_INDEX,VALIDATE,CHECK_FILES,CROSSCHECK_DB

OracleConfidential– Internal 48

Problem:taskqueuegrowingovertimeAction:reviewsystemactivityreport/EM

openSR

Problem:ActiveincidentslogindicatesbusyworknotrunningAction:Reviewsystemactivityreport/EM

openSR


ORDERING_WAITtasks

49

• Occurswhenabackuppiecewon’ttilewithexistingDeltaPools• Causes:• Unprocessedincrementalsfrompollingdirectory• Missingincrementallevel1backup• Incremental level1backuptakenagainst alevel0backupthatisnotinDeltaPool(maybeonlocalstorageinProtectedDBorreplicaorontape• Missingdatafileincarnationinformation• Unsupportedfeaturesusedinbackuppiece:

maxpiecesize,rmanencryption,datafilecopy


ORDERING_WAITtasks-- investigating

• CurrentscriptforinvestigatingORDERING_WAITissuesisavailablefromsupportinaMOSnote:– DiagnosticSQLscriptfortasksinORDERING_WAITstatusonRecoveryAppliance(DocID2095949.1)

50


LostRESTORE_RANGE

• Usesofrestoreranges:– RECOVERYRANGEoutputfromEM– RA_<disk|sbt>_RESTORE_RANGEviews– UNPROTECTED_WINDOWinformationandalerts

• Forarestorerangetobevalidweneed:– Archivelogstocovertherange– ZDLRADiskfilebackupsforalldatafilesthatweretakenduringtherange– Backupofcontrolfile

51


DiagnosingRESTORERANGEproblems• ComparerestorerangeofZDLRAstorageagainstallstorage

select*fromra_disk_restore_rangewheredb_key=<db_key>orderbylow_time;------------------------------select*fromrc_restore_rangewheredb_key=<db_key>orderbylow_time;

• Makesurethattherestorerangehasbeenrecomputed

selectcount(*),max(completion_time)fromra_taskwheretask_type=‘RESTORE_RANGE_REFRESH’andarchived=‘Y’anddb_key=<db_key>;

• SeewhatRMANthinksaboutrecoveryofdatabase

RMAN>restorepreviewdatabase;

52


DiagnosingRESTORERANGEproblems(cont.)• Getlistoflogstomakesurethattheytile

selectthread#,first_change#,first_time,next_change#,next_timefromrc_backup_redologjoinrc_backup_pieceusing(bs_key,db_key)wheredb_key=<db_key>andba_access='Local'andfirst_time>sysdate-<RWG>orderbythread#,first_change#;

• Getlistofdatafilestomakesurethattheyoverlapwitharchivelogsselectfile#,checkpoint_change#,checkpoint_time,absolute_fuzzy_change#fromrc_backup_datafilejoinrc_backup_pieceusing(bs_key,db_key)wheredb_key=<db_key>andba_access='Local'orderbyfile#,checkpoint_change#;

• Getlistofcontrolfilestomakesurethatoneisusableselectc.completion_timefromrc_backup_controlfilecjoinrc_backup_pieceusing(bs_key,db_key)wheredb_key=<db_key>andba_access='Local'orderbyc.completion_time;

53


RecoveryApplianceBIReports• ReportsareavailabletohelptheRecoveryApplianceadministratorunderstandresourceutilization,alerts,andhistoricaloperations• Reportsexamplesinclude:– ActiveIncidents– APIHistory– CapacityPlanning– RecoveryWindowSummary– Top10DatabasesbyDataTransfer– ProtectedDatabasesDetails– ChargebackReports

54


ProtectedDatabaseReport

Confidential– OracleInternal

Recoverabilityinformationperdatabase!

Topsectioncontainsthefollowinginformation:

- ProtectedDatabase- SpaceUsed- RPO/RecoveryWindow- UnprotectedWindow- LastBackuptime- LastTapeCopy- LastReplication


ChargebackReport– PayasRAStorageisUtilized(Least)

• Scenario:Thinkofthisassimilartoanmeteredmodelwherethecustomeronlypaysforutilization.Thedatabasebeingprotectedischargedonlyforthespaceutilized.

• Example:A6monthretentionwouldrampupincostsinceutilizationonmonth1islowerthanmonth6.

56

ChargeforspaceasitisusedontheRecoveryAppliance


ChargebackReport- PayasRAStorageisUtilized-cont’d

• MonthlyRAStoragespaceconsumptionforthisdatabase.

• DisplayofRAspaceusedtocalculatethechargeback.

57



• MonthlycostforthisdatabaseonRAStorage.

58



• Monthlytapestoragespaceconsumptionforthisdatabase.

59



• Monthlycostforthisdatabaseontape.

60


Agenda

61




RAStayHealthyPlan



1

2

3

4

5

6


Backup&ConfigurationOverview

• StepstoBackup&ConfigureaDatabaseusingEnterpriseManagerCloudControl– CreateProtectionPolicyonRecoveryAppliance(RA)– AddProtectedDatabasetoRA– ConfigureBackupSettingsforProtectedDatabase– ScheduleONE-TIME Level0(Full)Backupwith“CustomBackup”,thenLevel1swith“Oracle-SuggestedRecoveryApplianceBackup”


WhatToDoWithExistingBackups?

• RMANbackupstodiskorNFSshare(includingDataDomainshare)– CanbeimportedintotheRecoveryAppliancevia“polling”– BecomepartoftheIncrementalForeverstrategyontheRA

• Backupstakenusing3rd partybackupsoftware– LeavetheagentinplaceontheprotectedDBhostsuntilretentionexpires– Leveragedualbackup(disk &tape)strategyifneededininterim– Removingagentssavessystemresources

63


BackupBestPractices• UseTransparentDataEncryption(TDE)insteadofRMANencryption– RMANencryptionwillpreventZDLRAfromcreatingVirtualFullBackups(VB$).

• UsenativedatabasecompressioninsteadofRMANcompression– RMANcompressionwillresultinadditionalCPUutilizationontheProtectedDatabaseclient– RMANcompressedbackupsrequiredecompressionandrecompressionontheZDLRA

• Useblockchangetrackingforallprotecteddatabases

64

– $rmantarget<targetstring>catalog<catalogstring>backupdevicetypesbtcumulativeincrementallevel1filesperset1sectionsize64gdatabaseplusarchivelognotbackedupfilesperset32;


BackupBestPractices– Cont.• CreateIncrementalL0(full)asfirstbackup• CreatesubsequentCumulativeL1backups“IncrementalForever”• UseSectionSizeof64GB• Limitchannelallocationto4pernode• VirtualFullBackupCreationMonitoring– AfteraL1incrementalbackuptheRAindexesitandbuildsthecorrespondingVirtualFull.CheckEMforerrormessageslike“ORA-64760:DatabaseXYZhashadtasksinorderingwaitstateforoverXdays.”• RefertoMOSnote: DiagnosticSQLscriptfortasksinORDERING_WAITstatusonRecoveryAppliance(DocID2095949.1)

65


Restore&RecoveryBestPractices

• WhenthereisnovalidateddisasterrecoveryplanandnoRecoveryAppliance,badthingshappen.• TrueStoryofaRestoreOperationgoneterriblywrong.– DatabaseFailureOccurs– mustberestoredfrombackup• Backupwasnotavailableondisk• Backuprestoredfromtape• Foundsometapeshadbeenexpiredbymistake– tookdaystore-scanandre-catalogthepieces• Tapelibraryhadissues– movedtapestoanotherlibrarythatwasonly1GigEconnectivity• Taperestoreswerefailingaftermanyhours– init.oraparameterwaswrong

– RTO=8daysandRPO>8hours!

WhatareyourRTOandRPOrequirements?

66


Restore&RecoveryBestPractices– Cont.• UseRMANRestoreDatabase/RecoverDatabaseasyouwouldtoday– NonewRMANcommandstolearn.Intelligentbuilt-inrecoverycataloginRA– RMANisawareofthevalidatedbackupsondisk,tapeorreplica.Restoreistransparentandsimple– RestoredirectlyfromtapeorRAReplicawithoutstagingonlocalRAiflocaldiskbackupsarenotpresent

• Performanceconsiderations– Maximize#ofRMANchannelsforRestoreunlesstheyareotheractivedatabasesonthetarget– RestoreoperationsarealwaysautoprioritizedwithinRAwithoutpreventingotherbackupoperations– BCT:Continuebenefituntilallslotsareused.Defaultis7days.

• BigfileTablespacePracticesandConsiderations(recallbackupbestpracticeusingSECTIONSIZE)– Oracle11gdatabasescanrestoreinitialL0withSECTIONSIZEtoparallelizesectionsacrosschannels• Restoringvirtualfulls(createdfromL1s)does parallelizesections(AsofRASoftware12.1.1.1.8.201710)

– Oracle12cdatabasescanrestoreL0andvirtualfullswithSECTIONSIZEparallelismacrosschannels

67


Agenda

68




RAStayHealthyPlan



1

2

3

4

5

6


OpeningaServiceRequest• ProactivelyupdatetheSRwithasMUCHdetailsasyoucan

***criticaltime-sensitiveinformationmightbelost!***

1. ProblemstatementwithUseCase,Timeline(beforeproblem,anychanges,whenproblemoccurs)2. ImpactAnalysis3. RefertoMOSnotes:

Ø SRDC- ZeroDataLossRecoveryAppliance(ZDLRA)DataCollection(DocID2154189.1)Ø ZeroDataLossRecoveryApplianceSystemActivityScript(DocID2275176.1)Ø ZDLRADetailedTroubleshootingMethodology(DocID2408256.1)Ø HowtocreateaTechnicalServiceRequest(SR)inMyOracleSupport(DocID1321379.1)Ø HowtoCreateServiceRequestsforSoftwareIssuesUsingaHardwareSupportIdentifier(DocID1439980.1)Ø Whotocontacttodiscussdetails,correctionsorchangestoaSupportContract(DocID1250376.1)

4. RemembertoescalateanSRwhichisnotprogressingtoyoursatisfaction.Escalationbringsmanagementattentiontotheissue.YoucanescalateanSRofanyseverity

69


Escalations:BringingManagementAttentiontoyourServiceRequest• AskingtoincreasetheseverityofyourservicerequestisNOT anescalation,eventhoughcustomers

followthesameprocess

• AnescalationisbringingOracleSupportManagement’sattentiontoyourservicerequest– Proactivereasonstoescalateanissue:

• CommunicatebusinessissuestomanagerswithinOracleSupport– Reactivereasonstoescalateanissue

• Encounteringcriticalroadblocks• Dissatisfiedwithresolutionorresponse

• DescribeImpactofProblem:Projectdeadlines?,LostRevenue?,Governmentreporting?IncreaseinRecoverabilityandDataLossPotential

70


HowtoEscalateorChangeSeverity1. Callthe24x7SupportHotline

TollFree:0800.891.5899http://www.oracle.com/support/contact.html

2. Chooseoption#1foranexistingservicerequest

3. EnteralldigitsoftheSRnumberfollowedby#

4. ChoosetheoptiontoESCALATE theSRDoNOT choosetheoptiontospeaktotheowningengineer,youwanttospeaktoanEscalationManager (formerlycalledDutyManager)inordertoescalateorchangetheseverityoftheSR.AlwaysrequestacallbackfromtheEscalationManagerwhenescalatinganSR!

• EngineeredSystemsHardwareReplacements:ChoosetheoptionforFieldDeliveryDispatchtoschedule,changeorcheckthestatusofaFieldEngineervisitforhardwarereplacement

71


WhenCallingforEngineeredSys.Support…

Thefollowingoptionswilldirectyourcalltotherelevantteam:• Press"1" forExistingServiceRequests– EnterServiceRequest#.Iflookupissuccessful:

– Press“1”tospeaktotheengineerworkingyourservicerequest• IfEngineerisavailable,callwillconnect• Ifengineerisnotavailable– Press“1”toleaveVMor,– Press“2“tospeaktothenextavailableEESTengineer

– IfthereisnoFSTask,press“2”toEscalate theServiceRequest– IfthereisaFStask,• Press“2”forFieldDeliveryDispatch• Press“3”forEscalation Request

• Press“2”forNewServiceRequests– Press“1”EngineeredSystems• ForTechnicalProductIssuesPress“1”• ForNon-TechnicalIssuessuchasMyOracleSupportorSupportIdentifierIssuesPress“2”

72


SeverityLevel Business&TechnicalImpact 1st Response UpdateFrequency Resolution

Time

1 MissionCriticalBusinessImpact <1Hour(telephonepreferred)

Continual Updates24x7 Co-Owned

2 SeriousBusinessImpact CommunicationPreference

MultipleUpdates24-48hours Co-Owned

3 MinorBusinessImpact CommunicationPreference

Updates2-3BusinessDays Co-Owned

4 NoBusinessImpact CommunicationPreference

Updates3-5BusinessDays Co-Owned

Partnership

ServiceRequestSeverityLevels


PlatinumSupportforRecoveryAppliance


OpenWorld2017PresentationsforRecoveryAppliance

• ZeroDataLossRecoveryAppliance:TheWorld'sBestDatabaseProtection• ZeroDataLossRecoveryAppliance:DeepDiveandBestPracticesfromDevelopment• OracleRecoveryManagerTipsandTricksforOn-PremisesandCloudDatabases

ZDLRADocSet:“OracleZeroDataLossRecoveryAppliance”withinhttps://docs.oracle.com/en/engineered-systems/

zero data lossrecoveryappliance - oracle.com · windows(2-6 hours) versus waiting ... ra replica:...

Documents