aws storage gateway tips

Download AWS Storage Gateway Tips

Post on 14-Feb-2017

224 views

Category:

Documents

7 download

Embed Size (px)

TRANSCRIPT

  • AWS Black Belt Online Seminar

    AWS Storage Gateway

    2017.01.25

  • AWS Black Belt Online Seminar AWSJTechAWS

    12:00~13:00AWS(IoT etc.)

    18:00~19:00AWS(EC2RDSLambda etc.)

    & https://aws.amazon.com/jp/about-aws/events/webinars/

    2

    https://aws.amazon.com/jp/about-aws/events/webinars/

  • 2017125AWS(http://aws.amazon.com)

    AWSAWS

    AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to change in accordance with the AWS Customer Agreement available at http://aws.amazon.com/agreement/. Any pricing information included in this document is provided only as an estimate of usage charges for AWS services based on certain information that you have provided. Monthly charges will be based on your actual use of AWS services, and may vary from the estimates provided.

    3

  • Agenda

    1. AWS Storage Gateway

    2. Gateway

    3. AWS Storage Gateway

    4. AWS Storage Gateway Tips

    5.

    6.

    4

  • AWS Storage Gateway

    5

  • Amazon EFS

    File

    Amazon EBSAmazon EC2

    Instance Store

    Block

    Amazon

    S3/SIAAmazon Glacier

    Object

    Data Transfer

    AWS Direct

    ConnectAWS

    SnowballISV Connectors

    Amazon

    Kinesis

    Firehose

    S3 Transfer

    Acceleration

    Storage

    Gateway

    AWS

    6

  • AWS Storage GatewayAWS

    AWS Storage

    Gateway

    Amazon EBS

    snapshots

    Amazon S3

    Amazon Glacier

    AWS Identity and Access

    Management (IAM)

    AWS Key Management

    Service (KMS)

    AWS

    CloudTrail

    Amazon

    Cloudwatch

    7

  • AWS Storage Gateway

    NFS (v3 and v4.1)

    Amazon S3

    iSCSI

    Amazon S3 EBS snapshots

    iSCSI (VTL)

    Amazon S3 GlacierVTL

    **NEW!**

    8

  • AWS Storage Gateway

    Amazon S3Amazon Glacier(1)

    ()

    (99.999999999% 3DC)

    ( 1GB / 0.025$) 2

    (DC/)

    1 Gateway-VTL2 20171S39

  • AWS Storage Gateway

    AWS

    AWS

    10

  • AWS Storage Gateway

    AWS

    AWS

    AWS

    11

  • Gateway

    12

  • AWS Storage Gateway

    Gateway-Stored VolumesGateway-Cached Volumes

    13

  • AWS Storage Gateway

    Gateway-Stored VolumesGateway-Cached Volumes

    14

  • AWS Storage GatewayAmazon S3

    NFS v3 / V4.1

    AWS

    11

    S3

    15

  • AWS Storage Gateway(File Gateway)

    Amazon S3

    Internet

    Architecture

    DAS/SAN

    NFS

    v3 / v4.1

    Cache Storage()

    NFS

    GlacierS3

    Standard - IA

    HTTPS

    16

  • S3- NFS- S3

    S3-

    S3

    -

    IAM

    1()10 16TB

    - Write/ReadAWS

    VMware17

  • AWS Storage Gateway

    Gateway-Stored VolumesGateway-Cached Volumes

    18

  • Gateway-Stored Volumes

    AWS Storage GatewaySnapshotAmazon S3AWSDisaster Recovery

    iSCSI

    (Volume Storage)AWS

    AWS

    Storage GatewayAWSEBS

    19

  • Gateway-Stored Volumes

    AWS Storage Gateway(Gateway-Stored Volumes)

    Amazon S3

    Internet

    AWS

    Snapshot

    Snapshot

    EC2

    EBS

    Volume Storage()

    Upload Buffer() Snapshot

    1

    2

    Architecture

    DAS/SAN

    Disk

    Volume

    20

  • Gateway-Stored Volumes Gateway-Stored Volumes

    Volume 1Volume 1GiB16TiB

    1 Gateway32Volume Storage (0.5PiB)

    Volume StorageS3EBS Snapshot- ESB Snapshot

    - Snapshot

    Upload Buffer 1GB2TB- Upload BufferGatewayAWS

    Upload Buffer

    Upload Rate120MB/s()Download Rate20MB/s

    256KBIOI/O21

  • Gateway-Cached Volumes

    Storage GatewayAmazon S3 iSCSI

    - Storage GatewayS3

    AWSEC2

    Storage GatewayAWSEC2 Gateway

    22

  • Gateway-Cached Volumes

    AWS Storage Gateway(Gateway-Cached Volumes)

    Amazon S3

    Internet

    Snapshot

    EC2

    EBS

    Upload Buffer()

    DAS/SAN

    Architecture

    Cache Storage()

    Volume Storage

    AWS Storage Gateway(Gateway-Cached Volumes)

    /

    Amazon S3

    Disk

    Volume

    23

    AWS

  • Gateway-Cached Volumes

    Gateway-Cached Volumes 1 Volume 1GiB32TiB

    - Cache StorageGatewayGateway 1GB16TB

    1 Gateway321PiBVolume Storage

    Volume StorageS3Snapshot- Snapshot- Snapshot

    Upload Buffer 1GB2TB- Upload BufferGatewayAWS

    Upload Buffer

    Upload Rate120MB/s()Download Rate20MB/s

    256KBIOI/O24

  • AWS Storage Gateway

    Gateway-Stored VolumesGateway-Cached Volumes

    25

  • Storage Gateway

    VTL(Virtual Tape Library)Storage GatewayS3Glacier

    AWSEC2()

    (S3 Glacier)

    26

  • Gateway-VTL

    AWS Storage Gateway(Gateway-VTL)

    Amazon S3

    Internet

    EC2

    EBS

    Upload Buffer()

    DAS/SAN

    Cache Storage()

    AWS Storage Gateway(Gateway-VTL)

    Media

    Changer

    Tape

    Drive

    Amazon GlacierVTS)

    Architecture

    Amazon S3

    Media

    Changer

    Tape

    Drive

    /

    Backup

    27

  • GatewayiSCSI

    - 10, 1

    1 Gateway1,5001PiB- 100GiB, 200GiB, 400GiB, 800GiB, 1.5TiB, 2.5TiB

    S3Glacier- S3

    Glacier(Virtual Tape Shelf)- GlacierGateway-VTL-

    Cache Storage1GB16TBUpload Buffer 1GB2TB- Cache StorageGateway- Upload BufferGatewayAWS

    Upload Buffer

    Upload Rate120MB/s()Download Rate20MB/s

    256KBIOI/O28

  • Gateway Type

    Gateway Type Gateway Gateway

    NFSAWSAWS

    Gateway-stored Volume

    AWSDRAWS

    Gateway-CachedVolume

    132TB

    AWS

    SW

    AWS

    29

  • Ingestion

    DB

    Amazon S3

    AWS Lambda

    Amazon EMR

    AmazonCloudFront

    Amazon EC2

    Storage

    Gateway

    30

  • Gateway-Stored VolumeDR

    EC2

    EBS

    Snapshot Snapshot Snapshot

    EC2

    EBS

    EC2

    EBS

    AWS Storage Gateway(Gateway-Stored Volume)

    SystemA SystemB SystemC

    VolumeA VolumeB VolumeC

    VolumeA VolumeB VolumeC

    VolumeA VolumeB VolumeC

    SystemA SystemB SystemC

    S3

    EBS

    AWS

    AMI

    31

  • ()

    EC2

    EBS

    AWS Storage Gateway(Gateway-VTL)

    S3

    Glacier

    3DC

    on AWS

    AWS Storage Gateway(Gateway-VTL)

    S3

    Glacier

    3DC

    AWS

    32

  • CRR

    Amazon

    Glacier

    Amazon S3

    Standard

    Amazon S3

    Standard -

    Infrequent

    Access

    Amazon

    EBS

    snapshots

    Gateway-Cached Volume

    Gateway-Cached Volume

    Amazon

    EBS

    snapshots

    33

  • AWS Storage Gateway

    34

  • Gateway

    2

    VMWare ESXi 4.1, 5.0,5.1,5.5,6.0 Microsoft Hyper-V 2008R2,2012,2012R2

    CPU 8Virtual Core

    Memory 7.5GB

    HDD 75GB(SGW)2

    1 GatewayInternetNW

    EC2

    m3,i2,c3,c4,r3,d2,m4,x1xlarge

    EC2

    2017013 Gateway

    Gateway 1 EC2 Gateway

    2 VMware ESXi

    35

  • ()

    NFS V3V4.1

    SOCKSProxy)

    SOCKS5

    Source Destination

    S3 TCP 443 Storage Gateway Internet1

    Gateway Activation2 TCP 80 NW Storage Gateway

    UDP 53 Storage Gateway DNS Server

    Support Channel3 TCP 22 Storage Gateway Internet

    NFS v4.14 TCP 2049 NW Storage Gateway

    1. AWS Service Endpoint

    client-cp.storagegateway.region.amazonaws.com:443

    dp-1.storagegateway.region.amazonaws.com:443

    anon-cp.storagegateway.region.amazonaws.com:443

    proxy-app.storagegateway.region.amazonaws.com:443

    storagegateway.region.amazonaws.com:443

    3 AWS

    2 Activation

    4 NFSv3

    36

  • ()

    iSCSI Initiator1Windows Server 2008, 2008R22012, 2012R2Windows 7Red Hat Enterprise Linux 5,,VMWare ESX Initiator

    VTL

    Arcserve Backup r17.0Bakup Exec 2012,2014,15,16Dell NetVault Backup 10.0EMC NetWorker 8.xHPE Data Protector 9.xMicrosoft System Center Data Protection Manager 2012R2Symantec Netbackup 7.xVeeam Backup & Replication v7,v8,v9

    SOCKS SOCKS5

    Source Destination

    S3 TCP 443 Storage Gateway Internet2

    Gateway Activation TCP 80 NW Storage Gateway

    UDP 53 Storage Gateway DNS Server

    Support Channel3 TCP 22 Storage Gateway Internet

    iSCSI TCP 3260 NW Storage Gateway

    1 MPIO23 ()

    37

  • Network Architecture

    Local network

    ManagementConsole

    InternetSOCKS5

    Proxy()

    Proxy()

    Internet VPN

    Direct Connect()

    1

    2

    3

    VPCEC2

    Storage Gateway

    S3 Glacier

    Storage Gateway 1

    2VPN

    3

    Storage GatewayActivationManagement ConsoleInternetStorage Gateway

    38

  • Network Architecture

    EC2 Gateway

    Local network

    ManagementConsole

    Internet

    VPCInternet VPN

    EIP orNAT

    PrivateIP

    Storage

    Gateway

    S3 Glacier

    Storage

    Gateway

    EC2

    Direct Connect()

    Storage GatewayActivationManagement ConsoleInternetStorage Gateway39

  • Gateway

    Gateway Type

    VMGateway

    AWS MarketplaceEC2

    GatewayActivation

    GatewayStorage(Upload Buffer/Cache Storage)

    iSCSI

    Volume

    AWS

    Cache Storage

    NFS

    40

  • Gateway ~

    () GW

    -

    EC2AWS Market Place- VPCSecurity Group

    VM- -

    VMCacheUpload Buffer Thick Provisioning

    Gateway Timezone

    Cache,Upload Buffer

    41