Copyright © ���� -�� • TechJini Solutions Pvt Ltd • All rights reserved �
Video Streaming on iOS
Shravya Shridhar,JinilabsProduct division,TechJini Solutions Pvt Ltd
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 2
Agenda
● What is streaming● Http live streaming● What do you need to set up HLS● How HLS work● Timed metadata in stream● Media encryption
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 3
● What is streaming● Multimedia is constantly received by and presented to an end-
user while being delivered by a streaming provider● Listening or watching content in ‘real time'● Content can be audio, video, plain text
● Types of streaming● Live streaming ex: Listening to radio● Video on demand ex: Watching a movie
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 4
Codec, bitstream, Transport,bit rate
● Video codec ex: H.264 or VP8
● Audio codec ex: MP3 or AAC
● Transport ex: MPEG-2
● quality of an audio or video file
● Protocols :
● HTTP/HTTP/RTSP/RTMP/RTP
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 5
HTTP Live Streaming
● Adaptive streaming communications protocol
● Supported devices: iOS, Apple TV devices , Macs running OSX in Snow Leopard or late, android 4.0 and above
● Any app store app delivering video greater than 5Mb should use HLS
● Supported in most of the streaming servers adobe,wowza,realnetworks,akmai etc
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 6
What do you need to implement HLS
● Server Components ● Web server (apache may be)● Content storage ● CDN● Apple provided tools (segmenter ,encoder ,validator)
● Protocol● HTTP
● Client Components● Apple default player or AVFoundation framework or html 5 player embedded
in mobile web page
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 7
HTTP Live streaming
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 8
Preparing videos for delivery● Current Apple recommendation:
– Video: H.264 Baseline Level 3.0,3.1and Main Level 3.1– Audio:
● HE-AAC or AAC-LC up to 48 kHz, stereo audio● MP3 (MPEG-1 Audio Layer 3) 8 kHz to 48 kHz, stereo
audio– Bit rate recommended :
● 64,150,240,440,640kbps for cellular● 1240,1840,2540,4540Kbps for wifi
– Frame rate● 10 fps < 200 kbps. ● 12 to 15 fps < 300 kbps. ● 29.97 > 300Kbps
● Each segment is a MPEG-2 transport stream with a .ts extension.
Copyright © ���� -�� • TechJini Solutions Pvt Ltd • All rights reserved �
Video Segments and playlist structure:● mediastreamsegmenter -s 3 -D -f /Library/WebServer/Documents/stream 239.4.1.5:20103
● mediafilesegmenter -f VDO_DIR -l log/log.q1 -i avatar-q3.m3u8 -B q1- -t 10 -I src/avatar.ts
Output .m3u8 file structure
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 10
Creating variant playlist● Command:
variantplaylistcreator -resolution-tags -no-codecs-tags -o Avathar/avathar.m3u8 avathar-q1.m3u8 avathar/low.plist -iframe-url avathar-q2.m3u8 avathar/medium.plist avathar-q3.m3u8 avathar/high.plist
● Output:
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 11
Final m3u8 file for player
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 12
Validating streams● MediaStreamValidator -validate
http://devimages.apple.com/iphone/samples/bipbop/gear3/prog_index.m3u8
● Output:
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 13
Client Player set up:
● Html tag
<video src="http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8"/>
● MPMediaViewController can be used as shown below:
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 14
Using AVFoundation framework
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 15
Using AVFoundation framework contd:
NSURL *videoUrl = [NSURL URLWithString:@"http://"];
/* Creating asset from the url */AVURLAsset *asset = [AVURLAsset URLAssetWithURL:newMovieURL options:nil]; /* Requesting tracks and playable key*/ NSArray *requestedKeys = [NSArray arrayWithObjects:kTracksKey, kPlayableKey, nil];
[asset loadValuesAsynchronouslyForKeys:requestedKeys completionHandler: ^{
dispatch_async( dispatch_get_main_queue(), ^{
/* If requested key present , then play else error*/[self playAsset:asset withKeys:requestedKeys];
}); }];
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 16
- (void) playAsset:(AVUrlAsset *)asset withKeys:(NSArray *)requestedKeys {
NSError *error = nil;AVKeyValueStatus keyStatus = [asset statusOfValueForKey:thisKey error:&error];
/* If there is error in fetching tracks */if (keyStatus == AVKeyValueStatusFailed){
[self assetFailedToPrepareForPlayback:error];return;
}
/* Creating AVPlayer Item*/AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];
/* Creating AVPlayer Item from AVPlayerItem*/AVPlayer *player = [AVPlayer playerWithPlayerItem: playerItem];
player.allowsAirPlayVideo = YES; /* BY default YES*/
/* Creating view layer for AVPlayer*/AVPlayerLayer *playerLayer = [AVPlayerLayer playerLayerWithPlayer:player];
playerLayer.frame = [self getInitialFrameForPlayer];
/* Adding to the view layer */[self.view.layer addSublayer:playerLayer];
Copyright © ���� -�� • TechJini Solutions Pvt Ltd • All rights reserved ��
/* Adding status key to observe the status change of player */ [[player currentItem] addObserver:self
forKeyPath:@”s t a t us ” options:NSKeyValueObservingOptionInitial | NSKeyValueObservingOptionNew context:MyStreamingMovieViewControllerPlayerItemStatusObserverContext];
/* Update the scrubber during normal playback. */timeObserver = [[player
addPeriodicTimeObserverForInterval:CMTimeMakeWithSeconds(1, NSEC_PER_SEC) queue:NULL usingBlock: ^(CMTime time) { [self updateSeekbar]; /* Here you can update seekbar , or fi nd out the buffer
level */
}] retain];
/* Notifi cation when player reaches end */[NSNotifi cationCenter defaultCenter] addObserver:self
selector:@selector(realPlayerItemDidReachEnd:) name:AVPlayerItemDidPlayToEndTimeNotifi cation object:[self.player currentItem]]; playerLayer.videoGravity = AVLayerVideoGravityResize;
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 18
Timed meta data in streams
● Adding song's artist name to audio streams, cricket score to live cricket match.
● ID3 format or an image file (JPEG or PNG)
● Apple client software automatically displays the still image with audio only file
● Adding observer to AVPlayer
/* Adding observer to read meta data */[self.player addObserver:self
forKeyPath:@”currentItem.timedMetadata” options:0
context:MyStreamingMovieViewControllerTimedMetadataObserverContext];
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 19
Reading the meta data from AVPlayer
- (void)observeValueForKeyPath:(NSString*) path ofObject:(id)object change:(NSDictionary*)change context:(void*)context {
if (context == MyStreamingMovieViewControllerTimedMetadataObserverContext) {
NSArray* array = [[player currentItem] timedMetadata]; /* get meta data object from player*/for (AVMetadataItem *metadataItem in array) {
if ([(NSString *)[timedMetadata key] isEqualToString: AVMetadataID3MetadataKeyGeneralEncapsulatedObject])
{
NSDictionary *propertyList = (NSDictionary *)[timedMetadata value];
/* Metadata payload could be the list of ads. */NSArray *newAdList = [propertyList objectForKey:@"ad-list"];
/* Or it might be an image url or cricket score*/NSString *imageURL = [propertyList objectForKey:@"cricket_scorel"];
}}}}
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 20
Media Encryption● Change of encryption key periodically through https● Supply the credentials in the didReceiveAuthenticationChallenge callback of NSURLConnection
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 21
Links for further help
● https://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/Introduction/Introduction.html
● http://tools.ietf.org/html/draft-pantos-http-live-streaming-08
● http://www.streamingmedia.com/Articles/Editorial/What-Is-.../What-is-HLS-%28HTTP-Live-Streaming%29-78221.aspx
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 22
Questions ??
Copyright © 2012-13 • TechJini Solutions Pvt Ltd • All rights reserved 23
For further queries