realtime video streaming the open source way
TRANSCRIPT
MADRID · NOV 27-28 · 2015
Realtime video streaming
MADRID · NOV 27-28 · 2015the opensource way
Iván Belmonte
MADRID · NOV 27-28 · 2015
Github: ibelmonte
SysdivisionTwitter: @sysdivisionWebsite: http://sysdivision.com
Twitter: @ibelmonte
Iván Belmonte
MADRID · NOV 27-28 · 2015
Real time weather reporting system for outdoor sports.2 main elements:- HD cameras in surfing spots (beaches)- Weather stations in surfing spots
GOAL:
Weather forecast typically fails. Spotfav gives you a window to your spot, to check the weather conditions before moving.
MADRID · NOV 27-28 · 2015
What forecast says the day before
MADRID · NOV 27-28 · 2015
What you expect when you take your car to go to the spot
MADRID · NOV 27-28 · 2015
What’s actually going on
MADRID · NOV 27-28 · 2015
IMPORTANT NOTE:
I’m a backend developer/sysadmin
I know nothing about video and audio processing
In this project I am A VICTIM
MADRID · NOV 27-28 · 2015
Video manipulation and transmission is complex
- Plenty of difficult concepts: encoding, muxing, transcoding, encapsulation, bitrates, fps…
- Different media servers, different players, some codecs may not be supported
- Several transmission protocols depending on your needs: RTP, RTSP, RTMP, HLS, HDS, Shoutcast, HTML5…
Added complexity: LIVE streaming
MADRID · NOV 27-28 · 2015
Looking for a standard setup
Most common case: H264 over RTSP
Affiliates: get this if you want to get connected to our network
MADRID · NOV 27-28 · 2015
There are several options
Commercial: Adobe media server, Wowza, Flussonic, Flumotion…
SaaS:Ustream, Justin.tv (defunct)…
Open source: ffmpeg, ffserver, libav, Live555, Red5, crtmpd, vlc, Erlyvideo…
MADRID · NOV 27-28 · 2015
OUR SCENARIO
1 Camera, H264 over RTSP
1. The goal
Embed the camera in a website and also in a smartphone app
2. Nice to have
Not permanently connected (connect on demand)Tolerant to network cuts
3. Science fiction
Be able to filter by referral (avoid other websites to embed our cameras)
MADRID · NOV 27-28 · 2015
STEP 1: WEB player (RTMP)
MADRID · NOV 27-28 · 2015
ROUND 1
A javascript serving a Directx (wtf?) script.
MADRID · NOV 27-28 · 2015
ROUND 1
Cons:
- The camera has a limited pool of connections available. Visitors exceeding this poll get rejected.
Works?
- In some browsers
MADRID · NOV 27-28 · 2015
ROUND 2
A VLC cli process reading the RTSP, transcoding and publishing an RTMP stream in a socket
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES…
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
MADRID · NOV 27-28 · 2015
ROUND 2
Works?
- YES… for 1 camera.
What if more cameras?
- 1 VLC process for each camera.
MADRID · NOV 27-28 · 2015
ROUND 2
Round 2 version with multiple cameras
MADRID · NOV 27-28 · 2015
ROUND 2
Cons:
- A lot of processes in the server eating memory as hungry animals.
- VLC is unable to recover from network cuts.
- Each VLC process opens a socket in a different port, which in my case is a mess as I’m hosting the VLC processes inside a container and then routing the outside in.
- VLC is permanently connected to the camera
MADRID · NOV 27-28 · 2015
AND THEN ONE DAY…
Adobe updates flash player, breaks my streamings
MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
MADRID · NOV 27-28 · 2015
BROKEN STREAMING
Two possible reasons:
1. Something is wrong with my VLC commandvlc-wrapper-vvvh,p://camera.ip.address/cgi-bin/faststream.jpg?stream=full&fps=16&size=640x480&preview"-sout'#transcode{vcodec=FLV1,vb=800,ab=32,fps=16,width=640,height=480,acodec=none}:duplicate{dst=std{access=h,p{mime=video/x-flv},mux=ffmpeg{mux=flv},dst=LINUX.BOX.IP.ADDRESS:8080/stream.flv},dst=display}'
2. Something is wrong with the new version of Flash
Of course option 2. I contacted Adobe: - ME: your upgrade has a bug and it broke my streamings- ADOBE: …(silence)
MADRID · NOV 27-28 · 2015
ROUND 3
A super cool media server capable to encapsulate H264 RSTP into RTMP for flash players: Erlyvideo.
MADRID · NOV 27-28 · 2015
ROUND 3
Pros:
- Written in Erlang: amazingly light and fast- Connects to the source on demand
- One process, several cameras- Tolerant to network cuts (due to the previous point)
- Referrer filtering
Cons:
- Documented in russian (partially translated into english). Support forums also 80% russian.
- HLS support (for smartphone apps) only supported in its new commercial version: Flussonic.
MADRID · NOV 27-28 · 2015
STEP 2: Smartphone app (HLS)
MADRID · NOV 27-28 · 2015
Nginx RTMP module
An Nginx module capable to read streams in several formats as input, store them and then serve them encapsulated in other protocols (eg: HLS)
MADRID · NOV 27-28 · 2015
Nginx RTMP module
Works?
- YES, perfectly
Cons:
- Not very well documented
- Not tolerant to network cuts
- Unable to regain lost connectivity
MADRID · NOV 27-28 · 2015
Overall vision
MADRID · NOV 27-28 · 2015
Service schema
A video source. A separate server for each streaming format. A website visitor and an smartphone user.
MADRID · NOV 27-28 · 2015
An automated process
Using linux containers and some glue code everything can play together in an automated way
MADRID · NOV 27-28 · 2015
CONSLUSIONS
MADRID · NOV 27-28 · 2015
What I would do now if starting from scratch
1. Do it the opensource way: there are powerful tools
2. Look for a single solution for both web and mobile (probably libav, ffmpeg or Red5)
3. Spend time looking for a solution capable to serve also HTML5 formats (Mp4, Webm, Ogg)
MADRID · NOV 27-28 · 2015
THANK YOU!