chesa-aws preservation presentation may 2019...• implementing data archival best practices to...

12

Upload: others

Post on 04-Sep-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;
Page 2: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

CHESAPEAKE SYSTEMSMEDIA TECHNOLOGY ARCHITECTS AND WORKFLOW SOLUTIONS PROVIDER

Page 3: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

• Scaling Technologies and Media Workflows to support growth.

• Creative content is not being produced at the pace and quality that the business needs.

• Implementing Disaster Recovery and Business Continuity strategies.

• Enhancing internal support offerings for technologies and workflows.

• Implementing systems and workflows to support monetization of existing content.

• Implementing data archival best practices to preserve and protect existing content.

• Confidence is lost in current support resources; the current provider lacks the appropriate resources, coverage, and experience to handle a crisis.

Media Content Creators turn to Chesapeake Systems when:

Page 4: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

CASE STUDY: STUDIO ARCHIVEScaling ingest to support growing content library

Page 5: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

ARCHIVE MISSION• Provide Archive as a service

to the organization

• Store & Protect Studio history

• Provide library search & retrieval services

• Enhance search with technical and qualitative metadata

• Provide processing services

Page 6: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

Sidecar and Metadata

Captions, Metadata, Subtitles, Manifests, LUT Tables, etc

Packages

Image Sequences, Games, iTunes Packages, etc

Application Project Files

Documents Media Files

PRODUCTS IN THE ARCHIVE

Page 7: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

ON-PREMISES WORKFLOW

Vendors submit data packages into the archive over WAN

Data packages arrive on cache storage, triggering

MAM workflow to validate and process

Proxies are created for customers to browse in

the MAM and trigger restores

Checksums are processed for

validation of ingest and integrity checks

Upon successful tape archival an API call

updates Front End MAM with success and

checksum validation

Upon successful tape archival an API call

updates Front End MAM with success and

checksum validation

Site #2 can serve as an “air gapped”

site, duplicate tapes vaulted

Restored data flows back directly from archive

site(s) to the Studio, with automation triggered from

the Front End MAM

Page 8: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

WORKFLOW CHALLENGES

Vendor submissions increase in quantity and

package size consistently

New and changing video codecs require frequent

package updates to transcoders for

compatibility

Hardware requires 3-9 months budgeting and

procurement process to expand and support

ingest growth

Network bandwidth across all paths needs to increase

with ingest growth

A disruption to tape production will halt

ingests

All Systems require ~5 year

refresh

Tape technology refresh may require full restore

and re-archive to be automated and

monitoredDevOps and Admins have

roadblocks to managing and developing against the sites

Page 9: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

HYBRID APPROACH

Vendors submit data packages into the

archive over WAN or Direct Connect

Expansion of Ingest capabilities can be

handled on demand with Spot Instances

First Archive data set stays in the cloud,

limiting total egress

Restores triggered in MAM are sourced from on-premises archive to

limit total egress

Transcode as a service eliminates managing

updates for format compatibility

AI/ML as a service can be leveraged on demand from cloud archive to

enhance metadata

MAM Architecture is scalable and

replicated through containerization and

serverless architecture

On-premises tape archive keeps offline “air gap” by vaulting

duplicate tapes

Two separate archive storage technologies

protect the archive from external threats

caused by markets

DevOps and Admins now have minimal roadblocks to developing against the

environment

Page 10: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

MIGRATION TO HYBRID

Periodic data integrity checks require data retrieval

Piggyback off of integrity checks and

send data to S3Migrate database

records from retained archive site

Lambda can monitor S3 bucket for migration

data. Triggers MAM to run workflow to validate

files and link data objects to placeholder

records from DB migration.

MAM workflow can perform API calls to on-

premises system to trigger purge of local

copy from disk cache, no need to egress data

back to physical archive

Migration is limited by both network speed

and disk/tape speed at source.

Page 11: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;

QUALITIES OF THE MAM• Taxonomy System (or ability to integrate with one)

• Customizable Objects - no organization is equal

• Hierarchical object and asset structures to support “complex assets” and packages

• APIs and Metadata Exchange capabilities

• Workflow Automation/Business Process Management - with clear visibility into processes

• Agile Workflow Development capabilities

• UIs catering to multiple sets of required User Experiences

• Modernized (cloud ready) software architecture - easily scale and manage costs

• Dashboards/Analytics - track processes, library statistics, and costs

Page 12: Chesa-AWS Preservation Presentation May 2019...• Implementing data archival best practices to preserve and protect existing content. • Confidence is lost in current support resources;