webrtc xamu [autosaved] · the basics of webrtc web real-time-communication new(ish) open-sourced...
TRANSCRIPT
Cross-PlatformWebRTCALEXDUNN
TWITTER:@SUAVE_PIRATE
BLOG:HTTPS://ALEXDUNN.ORG
AgendaBriefIntroductiontoWebRTC
HowWebRTC Works
AdaptingtoNativePlatforms
ToolsandResources
LookatSomeCode
2
ResourcesFinishedsource:https://github.com/SuavePirate/Xamarin.WebRTC
WebRTC docs:https://webrtc.org
IceLink docs:http://docs.frozenmountain.com/icelink2/
Nativelibraries:https://webrtc.org/native-code/native-apis/
3
Whatis
4
TheBasicsofWebRTCWebReal-Time-Communication
New(ish)open-sourcedstandardfortransmittingdatafromonepeertoanother
Asetofframeworksandtoolsforcreatingconnectionsandsendingdata
Datatransmittedoverpeer-to-peerconnections
Agnostictotypesofdatasent◦ Audiobuffers◦ Videoframes◦ Rawdata◦ Files
5
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
6
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
STUN- SessionTraversalofUserDatagramProtocol[UDP]ThroughNetworkAddressTranslators[NATs]
7
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
STUN- SessionTraversalofUserDatagramProtocol[UDP]ThroughNetworkAddressTranslators[NATs]
TURN– TraversalUsingRelaysaroundNAT
8
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
STUN- SessionTraversalofUserDatagramProtocol[UDP]ThroughNetworkAddressTranslators[NATs]
TURN– TraversalUsingRelaysaroundNAT
ICE- InteractiveConnectivityEstablishment
9
VanillaIce
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
STUN- SessionTraversalofUserDatagramProtocol[UDP]ThroughNetworkAddressTranslators[NATs]
TURN– TraversalUsingRelaysaroundNAT
ICE- InteractiveConnectivityEstablishment
Signaling– ameansofcommunicatingconnectioninfobetweenclients
10
ImportantComponentsandTermsPeerConnections– theactualconnectionbetweentwoclients
STUN- SessionTraversalofUserDatagramProtocol[UDP]ThroughNetworkAddressTranslators[NATs]
TURN– TraversalUsingRelaysaroundNAT
ICE- InteractiveConnectivityEstablishment
Signaling– ameansofcommunicatingconnectioninfobetweenclients
Codecs◦ Video
◦ VP8
◦ Audio◦ Opus
11
MakingPeerConnections
CreatingaP2PConnectionviaSTUN
13
CreatingaP2PConnectionviaSTUNCoversmostscenarios
Failswhenpeershave2incompatibleNATtypesorpeersrunthroughheavyfirewalls
14
CreatingaP2PConnectionviaSTUN+TURN
15
CreatingaP2PConnectionviaSTUN+TURNTURNserverrelaysdatabetweentwopeers
Meansnodirectpeerconnectionismade
Canhaveperformanceimplications
Canbeexpensive
Covers99.999%ofcases
16
Multi-connectionMeshModelEachclientcreatesaconnectiontoeveryotherclient
Cancreateunreliableconnections
Individualconnectionscanbedroppedwithoutdroppingthemall
Requiresextendedsignalingandaccountability
17
Multi-connectionMixerModelEachclientconnectstoasingleserver
Theservermixesaudioandvideostreamsperpeer
Theserverrelaysthemixedstreamstoeachpeer
Heavyliftingputonserverratherthandistributed
Expensive
Hardtoscale
18
UsingaConnection
19
SendingAudioandVideoDataGetUserMedia◦ Getsaccesstodevicehardware(micandcamera)
20
SendingAudioandVideoDataGetUserMedia◦ Getsaccesstodevicehardware(micandcamera)
Encodethemedia◦ Pickacodec◦ Encodethedatawiththatcodec
21
SendingAudioandVideoDataGetUserMedia◦ Getsaccesstodevicehardware(micandcamera)
Encodethemedia◦ Pickacodec◦ Encodethedatawiththatcodec
Sendencodedmediaoverconnection
22
SendingAudioandVideoDataGetUserMedia◦ Getsaccesstodevicehardware(micandcamera)
Encodethemedia◦ Pickacodec◦ Encodethedatawiththatcodec
Sendencodedmediaoverconnection
Switchmediasources(switchcamera,switchmic)
23
AdvancedFunctionalityScreenstreaming
Multi-sourcestreaming◦ Streamingmultiplevideostreamsfromoneclient◦ Streamingmultipleaudiostreamsfromoneclient
AudioMixing
Non-TURNrelaying
Recordingstreams
Distributedstreaming
Streamsignatures
24
GoingCross-Platform
25
PlatformsSupportedWeb*Windows◦ WPF◦ WindowsForms◦ Win8◦ UWP*
WindowsPhone◦ 8.0Silverlight+◦ 8.1RT*
AndroidiOSMac
26
WebSupport
27
28
WebWorkaroundsNpapi plugins
Chrome+Firefoxextensions
ActiveXplugins
JavaAppletplugins
Stilldoesn’tcatchall◦ EdgedoesnotsupportWebRTC (theyarepushingORTC)◦ EdgedoesnotsupportActiveXplugins,JavaApplets,ornpapi
29
NativePlatformSupportandIssuesWebRTC isbuiltonanopensourcedC++platform-agnosticframework◦ Youtheoreticallycanrunitanywhere!
Noteveryplatformsupportsgivencodecs
VP8isnotnativetoanyWindowsSDK!
WeneedtogetacustombuildofthenativeVP8(C++)codec
CompiletheVP8codecagainstVisualC++perplatform
SamewithOpus
UnlikewebbrowsersthatshipwithVP8andOpuscodecsaspartoftheapplication,weneedtoincludethelibrariesseparately
30
BuildingWindows◦ BuildVP8codec◦ BuildOpuscodec◦ WrapcallstoC++library
Android◦ IncludereferencetonativecodecsbuiltintoOS
iOS+Mac◦ Addofficiallibrariesfornativecodecs
Xamarin◦ Addbindingstonativelibrariesforcodecsandwrapcalls
31
IceLinkAlibraryandframeworkwrappingalotofnativeWebRTC
Languagesandplatformssupported◦ Android◦ iOS◦ Windows(WinForms,WPF,Win8,UWP)◦ WindowsPhone(8,8.1)◦ Web
◦ ActiveXcontrol◦ JavaApplet◦ Npapi intheworks
◦ Mac◦ Xamarin (Android,iOS,Mac)◦ Java
32
Demo