ARCHITECTS OF VIRTUALIZED MEDIA PRODUCTION

Andy Rayner, Chief Technologistarayner@nevion.com +44 7711 196609

ST2110 – the emerging standard and its practical application

Are you in the right place?

Networking the moving picture II2nd November 2017 10am-4pm

London transport museum

nevion.com/news/events/network-moving-picture-2/

VirtuosoSoftware-defined media node

VideoIPathManagement & Orchestration

The digital video journey

SDI 2022-6 TR-04 2110

In the beginning……

Betty Bolton

‘Composite’ video has value!

Composite → Essence

HANCVANC

1400km = ~7ms

Cost ofBandwidth

&Latency

‘Composite’ transport

Different timing domains - adaptive

Comprehensive Protection

Mature – 15+ years

WAN

‘Essence’ transport

Common timing domain – SMPTE2059

Protection?

In its infancy – still maturing

CAMPUS

ConnectivitySecurity

ContentSecurity

RTP clock

SMPTE 2022-6

Packetpacing

VSF TR-04

SMPTE2022-6 6 7

SMPTE2110 – based on existing standards

SYSTEM AUDIOVIDEO

ANCDATA

AES3-32 bit

TIMING! 2022-8

The audio guys got there first – mostly!

6 7

SMPTE2110

SYSTEMAUDIO24 bitVIDEO

ANCDATA

AES3-32 bit

TIMING! 2022-8

ST2110-10

• 1460 UDP octets with 8060 octets option

• Multicast with IGMP and Unicast

• IPv4 & IPv6

• PTP to SMPTE ST 2059-2 with SMPTE ST 2059-1 Epoch

• RTP clocks & timestamps locked to media clock

• Zero offset on timestamp mandated

• One SDP object (IETF RFC 4566) per RTP Stream

• 90kHz RTP clock

SYSTEM

ST2110-20

• RTP based

• Subset of RFC4175

• Watch out for line numbering!

• Sampling YCbCr-4:4:4, YCbCr-4:2:2, YCbCr-4:2:0, ICtCp-4:4:4, ICtCp-4:2:2, ICtCp-4:2:0

• Bit depth 8, 10, 12, 16, 16f

• Colorimetry BT601, BT709, BT2020, BT2100, ST2065-1, ST2065-3, DCI-D65, DCI-D60

• TCS (Transfer Characteristic System): “SDR” “PQ” “HLG” “LINEAR” “DENSITY”

VIDEO

ST2110-30

• Based on AES67

• 24 bit linear PCM, 48kHz or 96 kHz

• 1ms packet period with option of 125us

• Up to 8 audio channels per stream – with option of up to 80

AUDIO

ST2110-21TIMING!

• When I need to send the IP packet data

• How I need to send IP packet data

ST2110-40

ANCDATA

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|V=2|P|X| CC |M| PT | sequence number |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| timestamp |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| synchronization source (SSRC) identifier |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Extended Sequence Number | Length |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| ANC_Count | reserved |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|C| Line_Number | Horizontal_Offset | reserved |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| DID | SDID | Data_Count |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| User_Data_Words...

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Checksum_Word | word_align |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

|C| Line_Number | Horizontal_Offset | reserved |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| DID | SDID | Data_Count |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| User_Data_Words...

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Checksum_Word |word_align |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

• Based on (IETF) RFC RTP Payload for SMPTE ST 291 Ancillary Data

ST2110-31

• Full AES3 transport

• Based on “AM824” Ravenna

AES3-32 bit

ST2110-50 → ST2022-8

• Integration of ‘legacy’ composite ST2022-6

2022-8

SMPTE2022-6

ST2110-?? Compressed video transport

• VC-2

• J2K

Other

28

Media payload RTP UDP IP VLAN MAC/PoS/GFP/MPLS

IETF RFC3550

Media origination timing

29

Media payload RTP UDP IP VLANMAC/PoS/GFP/MPLS

Media presentation timing

30

Media payload RTP UDP IP VLANMAC/PoS/GFP/MPLS

Media Synchronisation history – frequency & phase alignment

GENLOCKBlack burst/tri-level syncFREQUENCY

& PHASE ALIGNMENT

SMPTE 2059using PTPIEEE 1588

Sub uS accuracyFREQUENCY

& PHASE ALIGNMENT

32 32

SMPTE 2059 revision now in progress

SMPTE 2059using PTPIEEE 1588

PTP

→ scalability→ security

PTPMaster

PTPMaster

TransparentSwitch

BoundarySwitch

BoundarySwitch

BoundarySwitch

TransparentSwitch

TransparentSwitch

TransparentSwitch

TransparentSwitch

TransparentSwitch

Device Device Device Device Device Device DeviceDevice Device

NOTE: this timing approach is different to Transport Streams with PCR

TS

Data transit time

System processing time

System processing time – with multiple sources

Graceful performance with timing reference issues essential!

Scaling the number of media flows

1 x SMPTE2022-6

1 x ST2110-2016 x ST2110-303 x ST2110-40

= 20 flows!

AUDIOVIDEOANCDATA

System timing- November 2, 1936

42 42

Linear stream flows – our raster & hardware heritage

Sample nSample

n+1Sample

n+2Sample

n+3Sample

n+4Sample

n+5Sample

n+6Sample

n+7Sample

n+8Sample

n+9Sample n+10

Sample n+11

Sample n+12

Sample n+13

Sample n+14

Sample n+15

Packet xPacket x+1

Packet x+2

Pacing of arrivals is important

44

Linear video 2022-6 from SDI

45 45

Inherently less-linear flows…….

• High bit rate video necessitates careful control & shaping.

• SMPTE 2022-6 -> SMPTE 2110 already creates non linearity

VBI ACTIVE FRAME

LINEARISED FLOW

HBI ACTIVE LINE

LINEARISED FLOW

Packet egress from senders

47 47

Narrow & Wide senders

NarrowTypically hardware based

Linked to linear raster-based videoSmall buffering requirement

Capable of low latency chaining

WideTypically software based using NIC

Not linear raster related – frame basedLarger buffering required

Only capable of frame-delay-based chaining

N

W

48 30 October 2017Nevion Confidential 48

Why keep N?

not

W frame delivered in 95% of time

Ideal & observed W & N sender behaviour

30 October 2017

Frame period

N frame delivered in Active Picture time

Frame period

VBI

Cumulative delay in system – options N&W

51 51

The future is software based video sources…

Both native & virtualised

52 52

How many HD-SDI flows can you fit in 10GE?

SMPTE 2022-6/TR-04

1.485Gbps + RTP, UDP, IP, VLAN, MAC = ~ 1.65Gbps

10/1.65 = ~6?

SMPTE2110 Essence based 1080i50 = 1.0368 Gb/s

10/1.04 = ~ 9?

53 53

Post-raster timing & non-linear sources

0

2

4

6

8

10

12

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

bit rate

54 54

Stream aggregation in switch fabric

buffe

rbuffe

rbuffe

r

55 55

Data rates – HD-SDI 1080i50

1.485 Gbps

1.037 Gbps

1.141 Gbps

HD-SDI rate

Video essence rate * β

Video essence rate

β is the multiplier used on the video essence rate to create a ‘Committed Information Rate’

56 56

IP packets per video line *

HD-SDI 3 or 4 UHD 7 or 8

HS-SDI 3600 packets per video frame = 90,000 pkts/s

* Ethernet frames size and line-continuation dependant

57 57

More queuing = more latency

NN-1N-2N-3N-4N-5N-5N-6N-7N-8N-9

58 58

Packet formation time

NN-2N-3 N-1

NN-2 N-1N-3

NN-2 N-1N-3

NN-2 N-1N-3

59 59

Source bursting

NN-2N-3 N-1 NN-2 N-1N-3 NN-2 N-1N-3 NN-2 N-1

NN-1N-2N-3N-4N-5N-5N-6N-7N-8N-9NNN

60 60

Sender emission rate behaviour model

SENDER

Buffer modelwith max value

Buffer drains at β * essence rate

Ideal sender

Gapped (narrow) senderTransmit at 3 Gbps during active line

Importance of betaBeta is required for long-term bitrate fluctuations (e.g. VBI)

While beta cannot absorb short-term fluctuations, it allows for long-term bitrate excursions above the ideal rate.A gapped sender will start to overflow Cmax after ~60 video lines if beta is reduced to 1.0. This is due to the increased bitrate during the active frame.

Increasing sender burst sizeAn example of bursty sender behaviour: accumulate 32 packets, then burst

Once a burst from a sender exceeds Cmax, the Cfull buffer will drop packets.With a 10 Gbps line rate, a 24 packet buffer would be required to avoid loss following a 32 packet burst.

For Cmax = 9 and beta = 1.1, we get 47% packet loss as Cfull cannot drain packets in time.

65 65

Packet pacing on sender egress NIC

• NICs with hardware flow control

• NICs with no hardware flow control

• NICs shared by VMs

66 66

Policing/shaping in switches

• SHOULDN’T NEED TO DO THIS

• Switch policing has limited integration period flexibility

• Impossible to fully cross protect from rogue sender

• Harder at UHD rates

67 67

De-rating switch provisioning

• Necessary with W senders

• Switch-specific numbers

• Initial maths is pro-rata to the switch real buffering

• Restrict port loading in orchestrator to achieve required

control

68

VideoIPath – orchestration

Studio A

Studio B

IP Studio C

Processing Farm

Orchestration layer needs to also plan for sub-optimal senders!

IP media edge

PTP TIMING

DEVICEDISCOVERY & CTL

MEDIA FLOW IP ADDRESSING

ALTERNATIVE TIMING DOMAINS

RESTRICTED/PROXYDISCOVERY & CTL

DIFFERENT IP ADDRESSING (NAT)

ESSENCEFLOWS

ESSENCE ORCOMPOSITE FLOWS

PROTECTIONTERMINATION

PROTECTIONTERMINATION

Protection

n n+1 n+2

n+4 n+5 n+6

n+8 n+9 n+10

n+3

n+7

n+11

n+12 n+13 n+14

n+16 n+17 n+18

n+20 n+21 n+22

n+15

n+19

n+23

n+24 n+25 n+26 n+27

n+28 n+29 n+30 n+31

n+32 n+33 n+34 n+35

n+36 n+37 n+38 n+39

n+40 n+41 n+42 n+43

(1,1) (L,1)

(1,2)

(1,D) (L,D)

(2,1)

(2,2)

1 L2

flow A

flow B

SOURCE DESTINATION

flow A

flow B

DESTINATIONSOURCE

delay

Spatial

Spatial + temporal

FEC

Leaf spine

71

LEAF

SPINE SPINE

LEAF LEAF LEAF LEAF

72

Switch buffers handling flow non-linearity

buffe

rbuffe

rbuffe

r

aggregation

typically 10,000 packets buffering in total across switch ASIC?

1 video frame HD-SDI = 20ms (50P) = 3.7 MB = 2600 pkts

1ms @ HD 50P burst = 185KB = 130 pkts

1ms @ 4K100P burst = 30MB = 1040 pkts

73

Video packet sending profile

1.039Gbps

1.485Gbps

Startof‘frame’

Startof‘frame’

1080i50

Software sender – phase aligned?

74

Explicit routing & control essential!

Full network simulation – core & edges – essential to prove scalability & performance

75

Virtual Device n

Virtual Device n+1

Virtual Device n+2

Virtual Device n+3

Virtual Device n+4

Virtual Device n+5

Virtual Device n+6

Virtual Device n+7

Switch fabric requirement

Wire-speedNon-

blockingControl

Flow density

PTP support

Port density

Buffer memory

Scalability - abstract user from the underlying technology

Technology-agnostic user experience

Scalable orchestration

VideoIPath

Orchestration hierarchy for very large scale

VideoIPathVideoIPath

VideoIPath

Scalability: Address Translation

Media payload RTP UDP IP VLANMAC/MPLS/SR

Media payload RTP

Control integration

• Media node registration, discovery and control

• Network topology discovery and control

NMOS is work in progress

IS04/5/6

Registration, Discovery & Control

Scalability

Timing domains

MasterClock

MasterClock

Phase-Frequency offsets

Datacentre Datacentre

studio

studio

Contribution / Servers

Studio Production

#4

Studio+Control MCB, Live Points, etc.

Studio 3, Control 1&2,

Studio A

Studio B, Operations Test Lab

MCR, Studio 1&2, Control

3&4, Post

Studio Production

#1

Studio Production

#2

Studio Production

#3

Contribution / Processing

Ingest / Video Servers

Playout / Distribution

IP WAN

VideoIPathmanagement and orchestration

88 88

Complete end to end linear IP workflow

Non real-time processing (with time stamping)

UDP flows on interconnect?!

Think about the scalability of your network

89

Think about speed and capacity

Nevion Confidential90

And how to manage traffic effectively!

91

Networking the moving picture II2nd November 2017 10am-4pm

London transport museum

nevion.com/news/events/network-moving-picture-2/

Andy Rayner, Chief Technologistarayner@nevion.com +44 7711 196609