how netflix works? · netflix space/power core router-2 oca-1 oca-2 netflix core router-2 ix sw1 ix...

of 50 /50
How Netflix Works? ThaiNOG 8 th May 2019 NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION Jocelyn Ooi

Upload: others

Post on 16-Mar-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

How Netflix Works?

ThaiNOG

8th May 2019

NETFLIX CONFIDENTIAL

NOT FOR DISTRIBUTION

Jocelyn Ooi

Page 2: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

• > 148 million members

• > 190 countries

Page 3: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

How

Works

Page 4: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Inventing Internet TV:

• The architecture of internet streaming

• Quality of Experience

• Cost

Page 5: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

DATA PLANE Open Connect

Content

Delivery

Network

CONTROL PLANE

STREAM NETFLIX DEVICE

Page 6: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 7: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

before streaming starts = control plane =

Page 8: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

streaming = data plane = Open Connect

Page 9: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Data Plane (streaming playback)

Page 10: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Open Connect

A NETFLIX ORIGINAL

Page 11: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Q: What is a Content Delivery Network?

A: Geographically distributed content servers attached to networks

+ a way of routing requests to the closest (and/or best performing)

server / network path

Page 12: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

History

Streaming Launched (“EHub”)

2007

Third Party CDN 2009

Open Connect 2011

Page 13: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Hardware Open Connect Appliances

Page 14: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Designed for bulk storage of the entire large regional content catalogs

“Storage” appliance

Page 15: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

High-throughput servers, used to serve very popular content

“Flash” or “Offload” appliance

Page 16: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 17: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 18: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Open Connect connects directly to residential ISP networks. 95% of our traffic is directly connected.

Page 19: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

We provide free Open Connect Appliances for Internet Service Provider (ISP) data centers

Page 20: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

We also provide interconnection to ISPs at Internet Exchange locations throughout the world

Page 21: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

wait, what’s an IXP?

An Internet exchange point

(IX or IXP) is

a physical infrastructure

through which

Internet Service

Providers (ISPs)

and

Content Delivery

Networks (CDNs)

exchange Internet traffic

between their networks

-wikipedia

Singapore

Equinix

SG

3

Page 22: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 23: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 24: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 25: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Netflix Site

© 2015 Netflix Inc

Single CORE Sample Configuration o 1 Site, 2 OCAs in a redundant configuration

o OCAs can be clustered for cache fill o Connections to 2 Core Routers

NETFLIX

Core

Router-1

Bangkok

Core

Site

Core

Router-1

PARTNER

PROVIDED

SPACE/Power

Core

Router-2

OCA-1 OCA-2

NETFLIX

Core

Router-

2

IX

SW1

IX

SW2

NETFLIX

PROVIDED

EQUIPMENT

IX Connection IX Connection

NETFLIX CONFIDENTIAL

NOT FOR DISTRIBUTION

Page 26: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 27: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Control Plane (streaming setup)

Page 28: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 29: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Determine the preferred experience DEVICE TITLE

CONNECTIONS

LOCATION

NETWORK

Broadband - wired or

wifi

Cellular - Edge, 3G,

LTE, ...

LANGUAGE

HDCP support,

etc.

Page 30: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

That’s exactly what I want

Page 31: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

...now where can I get it?

Page 32: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

GENERATE PLAYBACK MANIFEST

PLAYBACK MANIFEST

PLAYBACK

This is how we steer the request to the correct server/network/location

LICENSE

Page 33: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 34: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 35: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Content Filling

pre-

positioning

Page 36: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Caches fill while you sleep

Page 37: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Files are copied off-peak (in the middle of the night) when networks are not loaded, using pre-positioning algorithms

Page 38: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

CONTENT RANK

PREDICTABLE VIEWING PATTERNS

Page 39: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Using the purpose-built Open Connect network, we can copy each new Netflix show into a region once, then show it thousands or millions of times.

Page 40: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 41: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION
Page 42: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Networks are dimensioned for peak capacity

Off-peak cache fill is “free”

Page 43: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

peak

required

upstream

capacity

1 Tbps

peak required

Netflix

capacity

350 Gbps

ISP

IS

P cu

stom

er

hom

es

Page 44: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

peak

required

upstream

capacity

650 Gbps

peak required Netflix capacity 0

ISP

IS

P cu

stom

er

hom

es

Page 45: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Insight into performance Quality of Experience

Page 46: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

input rate (from network)

output rate (to screen)

buffer level

video buffer (leaky bucket)

Page 47: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

The Tradeoff

quality interruption risk

playdelay

Page 48: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

https://openconnect.netflix.com

Page 49: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Questions?

Page 50: How Netflix Works? · NETFLIX SPACE/Power Core Router-2 OCA-1 OCA-2 NETFLIX Core Router-2 IX SW1 IX SW2 PROVIDED EQUIPMENT IX Connection NETFLIX CONFIDENTIAL NOT FOR DISTRIBUTION

Thank you!