#howtovideo

Warning: lots of acronyms ahead

(I didn’t come up with them)

What is a video?• A bit more than a few pictures stacked together (GIF)

• Array of I-frames and P/B-frames

• I-frames are “reference” frames

• P/B-frames are “predicted” frames - expressed in motion vectors and transform coefficients

• GIFs take up a lot of memory because they are basically all I-frames

• Which is why converting our GIFs to video would be awesome

Video Formats• MP4, MKV, Quicktime, etc are all container formats

• MPEG2, H.264/MPEG4, etc are video formats

• AAC, MP3, etc are audio formats

• Container formats generally contain both audio and video tracks

• Responsible for keeping audio/video in sync

Streaming - TCP vs UDP• Each TCP packet is buffered by the OS, adding latency

• Each TCP packet requires receipt confirmation, adding latency

• When a packet gets dropped, other packets begin to pile up behind while the OS waits for retransmission of dropped packet

• Eventually, video gets paused

• VBR video - lots of movement leads to higher bitrate

• TCP buffer is a constant size - another potential for packet backup

• Losing a few packets isn’t a huge deal

• Video players/decoders are written with this in mind

• Quality may degrade, but video will keep playing

• (most of the time)

HTTP Streaming Options• MP4 FastStart

• MOOV atom at beginning of file

• HLS

• index file + chunks

• supported by Apple, and few others

• Relatively high latency due to client needing to buffer several chunks at a time

• Great for VOD, or when latency is not a huge concern

Other Streaming Options• (S)RTP/RTCP

• Roll your own!

• WebRTC

• Data sent via SRTP

• P2P, built-in NAT traversal

• BYOS (Bring Your Own Signaling)

• Exchange ICE candidates - network information

• Native Chrome/FF support. Soon to be IE. Safari MIA

• Flash (RTMP/RTMFP)

• yuck

• RTMP is TCP

• RTMFP is UDP, could be P2P

• Works well when latency must be as low as possible - conversational video and real-real-time broadcasting

Large Groups Difficulties• Conversations are difficult because they require extremely low latency to feel real-time

• P2P

• Each user uploads their own video stream

• Each user receives N-1 video streams

• Lots of bandwidth, CPU - especially difficult on mobile

• Starts to fail with more than ~4 people in group

• Hub and Spoke

• Each user uploads their own video stream

• Each user receives single video stream

• Requires real-time video-multiplexing server to pose as another “user”

• NOT EASY

• Twilio, TokBox, others working on this

–Donovan

“Video is hard”