linear scalability, geo-distribution and extreme data gsm –gsm sharddirector1 -listener 1571...

Download Linear Scalability, Geo-distribution and Extreme Data   gsm –gsm sharddirector1 -listener 1571 -pwd passwd_gsmcatuser -catalog shardcat host:1521: shardcat ... Used 2-node RAC at shard-level ShardCat shard a1 Shard A ShardDirs shard a2 shard c1 Shard C shard c2

Post on 10-Mar-2018

219 views

Category:

Documents

3 download

Embed Size (px)

TRANSCRIPT

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    LinearScalability,Geo-distributionandExtremeDataAvailabilityOracleSharding

    NageshBattula,Sr.PrincipalProductManagerOracleHAProductManagement

    @nageshbattula

    MarkDilman,DirectorOracleProductDevelopment

    CharlieBaker,Sr.DirectorOracleDyn GBU

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOraclesproductsremainsatthesolediscretionofOracle.

    2

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    ProgramAgenda

    OverviewofOracleSharding

    WhatsnewinOracleDatabase18c Sharding

    ChinaTelecom ShardingforWeChatIoTApplication

    OracleDynsexperiencewithOracleSharding

    Summary

    1

    2

    3

    4

    3

    5

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    ProgramAgenda

    OverviewofOracleSharding

    WhatsnewinOracleDatabase18c Sharding

    ChinaTelecom ShardingforWeChatIoTApplication

    OracleDynsexperiencewithOracleSharding

    Summary

    1

    2

    3

    4

    4

    5

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.| 5

    OracleSharding AnElasticRelationalDatabase

    BA

    DCFEHG

    JILK

    BADC

    FEHG

    JILK

    OnegiantDBshardedinto

    manysmallDBs

    TablePartitions

    Shard #1 Shard #3Shard #2

    Horizontalpartitioningofdataacrossupto1000independentOracleDatabases(shards)

    Shared-nothinghardwarearchitecture Eachshardrunsonaseparateserver Nosharedstorage Noclusterware

    Dataispartitionedusingashardingkey(i.e.account_id)

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.| 6

    OracleDatabaseSharding Benefits

    LinearScalability

    Addshardsonlinetoincreasedatabasesizeandthroughput.Onlinesplitandrebalance.

    ExtremeAvailability

    Shared-nothinghardwarearchitecture.Faultofoneshard

    hasnoimpactonothers.

    GeographicDistribution

    Userdefineddataplacementforperformance,availability,DRortomeetregulatoryrequirements.

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.| 7

    ShardCatalog/Coordi

    nator

    ConnectionPool

    ShardDirectors

    ShardingKey

    Architecture&KeyFeaturesofOracleSharding

    Shards

    Autodeploymentofupto1000shardswithreplication Declarativespecificationoftopology SupportsActiveDataGuardandOracleGoldenGate

    Multipleshardingmethods ConsistentHash,List,RangeorComposite

    Centralizedschemamaintenance NativeSQLforshardedandduplicatedtables

    DirectroutingandProxyrouting Onlinescale-outw/autoreshardingorscale-in

    OracleDatabase18c ShardingFeatures: MidtierSharding Affinityformidtierandshards PDBSharding SupportsMultitenant RACSharding Virtuallysharddataacrossinstances AutomaticShardingWorkbench

    ShardingAdvisor&DataMigration

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    AutomatedDataAvailabilityConfiguration

    8

    ActiveDataGuardwithFast-StartFailover

    GoldenGatechunk-levelactive-activereplicationwithautomaticconflictdetection/resolution(OGG12.3)

    Partition_F Partition_F

    Partition_A

    Partition_C

    Optionally complementreplicationwithOracleRACforserverHA

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    DeclarativeSpecificationofSDBConfiguration

    9

    createshardcatalog-database:1521:shardcat-usersdb_admin/passwd_sdb_admin

    addgsmgsmsharddirector1-listener1571-pwdpasswd_gsmcatuser-catalog:1521:shardcat..addcredential-credentialoracle_cred-osaccountoracle-ospassword

    addshardgroup-shardgroupshgrp1-deploy_asprimaryregionavail_domain1

    addinvitednode..createshard-shardgroupshgrp1-destination -credentialoracle_cred

    deploy

    addservice -serviceoltp_rw_srvc-roleprimary

    ShardCatalog/Coordi

    nator

    ConnectionPool

    ShardDirectors

    ShardingKey

    Shards

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    DataModelingConsiderations Toreapshardingbenefits,schemamustbedesignedtomaximizenumberofsingle-shardrequests

    Sharding-amenableschemaconsistsof: ShardedTableFamily:

    Setoftablesequi-partitionedbythesharding key Relateddataisalwaysstoredandmovedtogether Joins&integrityconstraintchecksaredonewithinashard

    ShardingmethodandkeyarebasedonApprequirements Sharding keymustbetheleadingcolumnofaprimarykey

    DuplicatedTables: Non-shardedtablesarereplicatedtoallshards Usuallycontaincommonreferencedata Canbereadandandupdated(18.1)oneachshard

    10

    Customers

    Orders

    LineItems

    Products

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    ShardedTables

    Shard1 Shard2 ShardN

    11

    SchemaCreation ShardedandDuplicatedTables

    Customers Orders LineItems

    Products

    DatabaseTables

    DuplicatedTable

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.| 12

    CreatingaShardedTableFamilywithReferentialIntegrity

    CREATE DUPLICATED TABLE Products ( ProductId INTEGER PRIMARY KEY, Name VARCHAR2(128), LastPrice NUMBER(19,4), )TABLESPACE products_tsp ;

    ExecuteDDLsontheShardCatalog

    CREATE TABLESPACE SET tbs1 ;

    CREATE SHARDED TABLE Customers( CustId VARCHAR2(60) NOT NULL,

    FirstName VARCHAR2(60), LastName VARCHAR2(60),CONSTRAINT pk_customers

    PRIMARY KEY(CustId))PARTITION BY CONSISTENT HASH (CustId) PARTITIONS AUTO TABLESPACE SET tbs1 ;

    CREATE SHARDED TABLE Orders ( OrderId INTEGER,CustId VARCHAR2(60),OrderDate TIMESTAMP,CONSTRAINT pk_orders

    PRIMARY KEY (CustId, OrderId),CONSTRAINT fk_orders_parent

    FOREIGN KEY (CustId) REFERENCES Customers(CustId) )PARTITION BY REFERENCE (fk_orders_parent) ;

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    FastPathforkey-basedaccessAppTier

    Routing Tier

    DataTier

    DirectRoutingviaShardingKey

    Connectionpoolmaintainstheshardtopologycache Uponfirstconnectiontoashard

    Connectionpoolretrievesallshardingkeyrangesintheshard

    Connectionpoolcachesthekeyrangemappings

    DBrequestforakeythatisinanyofthecachedkeyrangesgoesdirectlytotheshard(i.e.,bypassessharddirector)

    13

    ApplicationServer

    ShardDirectors

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    Coordinator(shardcatalog)

    ProxyRoutingforMulti-shardQueries

    ApplicationsconnecttoQuerycoordinator/ShardCatalog E.g. SELECT GEO, CLASS, COUNT(*) FROM CUSTOMERS GROUP BY GEO, ROLLUP(CLASS);

    Coordinatorrewritesthequerytodomostprocessingontheshards

    Supportsshardpruningandscatter-gather Finalaggregationperformedonthecoordinator

    14

    ForWorkloadsthatcannotpassshardingkeyduringconnectioncheck-out

    ApplicationServer

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    LifecycleManagementofSDB DBAcanmanuallymoveorsplitachunkfromoneshardtoanother Whenanewshardisadded,chunksareautomaticallyrebalanced

    UsesRMANIncrementalBackup&TransportableTablespace

    Connectionpoolsgetnotified(viaONS)aboutsplit,move,add/removeshards,auto-resharding Applicationcaneitherreconnectoraccessread-only

    AllMAApracticesapplyforbackupandrecoveryofshardsandshardcatalog Canpatchallshardswithonecommandviaopatchauto EM supportsmonitoring&managementofSDB OracleSQLDeveloperintegration

    15

  • Copyright2017, Oracleand/oritsaffiliates.Allrightsreserved.|

    ProgramAgenda

    OverviewofOracleSharding

    WhatsnewinOracleDatabase18c Sharding

    ChinaTelecom ShardingforWeChatIoTApplication

    OracleDynsexperiencewithOracleSharding

    Summary

    1

    2

    3

    4

    16

    5

  • PPTwww.1ppt.com/moban/ PPTwww.1ppt.com/hangye/ PPTwww.1ppt.com/jieri/ PPTwww.1ppt.com/sucai/PPTwww.1ppt.com/beijing/ PPTwww.1ppt.com/tubiao/ PPTwww.1ppt.com/xiazai/ PPT www.1ppt.com/powerpoint/ Word www.1ppt.com/word/ Excelwww.1ppt.com/excel/ www.1ppt.com/ziliao/ PPTwww.1ppt.com/kejian/ www.1ppt.com/fanwen/ www.1ppt.com/shiti/ www.1ppt.com/jiaoan/ PPTwww.1ppt.cn

    Oracle Sharding for China Telecoms

    WeChat IoT Application

  • About China Telecom

    18

    n Large-scale and leading integrated information service operator in the world n Provides

    Landline & mobile telecommunications services Internet access services Information services and other value-added telecommunication services

    n As of 2016, mobile subscribers of about 215 millionn China Telecom is building Business Support System(CRM/Billing etc.) for Narrow-

    Band IoT (NB-IoT) network Provides packaged offers, self service on account balance, call billing, number status, etc.,

    process of problem repairs, repair progress inquiry, etc.n WeChat Service (part of BSS ecosystem) provides customer service by WeChat

    System will also manage 2/3/4G network usersn NB-IoT Network user base is projected to grow exponentially

  • Motivation for Oracle Sharding

    19

    n We took sharding into consideration for the projected large number of NB-IoTNetwork users

    n Main benefit is scalability We do not need to worry about what to do when we need to scale to larger number of users

    n Evaluated Oracle Sharding, MySQL, MongoDB and Maria DB Determined that migration co