1

Watching Video over the Web, Part I Streaming Protocols  

AliC.Begen,TankutAkgulandMarkBaugher,Cisco

AU.S. consumerwatches TV for about five hours a day on average.While themajority of viewedcontent is still thebroadcastTVprogramming, the shareof the time‐shifted contenthasbeen everincreasing.OnethirdoftheU.S.consumerscurrentlyuseadigitalvideorecorder(DVR)‐likedevicefortime‐shifting,however,thetrendsareshowingthatmoreandmoreconsumersaregoingtotheWebtowatch their favorite shows andmovies on a computer ormobile device. Increasingly, theWeb iscomingtothedigitalTV,whichincorporatesmoviedownloadsandstreamingusingWebprotocols.Inthis firstpartofatwo‐partarticle,theauthorsdescribebothconventionalandemergingstreamingsolutionsusingWebandnon‐Webprotocolsandprovideadetailedcomparison.

Summer2010,timeforthe19thFIFAWorldCup.Itwasanexhilaratingwholemonthwithtotal64matchesplayedby32nationalteams.ItisnosurprisethattheWorldCuphasbeenoneofthemost

watchedeventsworldwide throughouthistory.Thisyear, therewasevenmore interestas itwas

thefirsttimeinthehistoryofthetournamentthatsome25gameswerebroadcastin3D.Butmoreimportantly, many more viewers than ever watched the games over the Web. With the recent

developments in video streaming technologies and increase in the broadband Internet access

penetration, fans enjoyed watching the games in HD or near‐HD quality on their computers,smartphonesandotherconnecteddevicesincludingTVs.IntheU.S.,45%ofthedailyWorldCupTV

audiencewatchedthematchesinanon‐homeenvironmentoronanon‐TVplatform1.ESPN3.com

reachedoversevenmillionuniqueviewersanddeliveredover15millionhoursofcontent.

Justafewmonthsago,wewitnessedthesamephenomenonintheVancouver2010WinterGames.

InCanadawherethepopulationisabout34million,therewerealmostfourmillionuniqueviewers

whowatched thegameson the Internet.CTV, thenationalTVchannel thatbroadcast thegames,made 300 events available on theWeb. Canadian Internet users consumed a total of 6.3million

hours of live and 0.9 million hours of on‐demand content, resulting in a total 6.2 petabytes

delivered in about two weeks2. Two years ago, the Beijing Summer Games also attracted manyonlineviewers‐bothfor liveandon‐demandviewing.IntheU.S.,NBCdeliveredmorethan1100

yearsofvideocontenttoabout52millionuniqueonlineviewersduringthegames.

1http://www.espnmediazone3.com/us/2010/07/espn‐xp‐world‐cup‐dispatch‐4‐through‐742010/2http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?casestudyid=4000007347

2

Thesporteventsarenotthesoletypeofcontentattractivetoonlineviewers, though. Inthepast

fewyears,manycontentprovidershavestartedmakingtheirregularandpremiumcontentsuchasnews, series, shows and movies available on their Web sites. Despite the certain geographical

restrictionsonmanyoftheseWebsites,theconsumerssawagreatproliferationintheamountof

content they had access to. Some content providers or TV channels actually did not impose anyrestrictionsontheviewerlocation,andthusweretransformedfrombeingaregionalTVchannelto

becomeaglobalone,extendingtheirreachforadrevenuesinanunforeseenmanner.

Theconsumers’desireforbeingabletoaccesspracticallylimitlessamountofcontentanytimetheywanted to and the drop in the delivery costs further hastened the deployment of streaming

services.NewWebsites thatdocontentaggregationsuchasHulucamealive. InMay2010,Hulu

hadmore than 40million unique viewers in the U.S., streamingmore than one billion videos amonth3. Remarkably, these numbers have been steadily increasing. Netflix, who is the largest

subscriptionserviceforDVDrentalandstreamingvideo,currentlyhas13millionmembersinthe

U.S.,55%ofwhomuseitsstreamingservicesonavarietyofdevices4.

Internetvideo,alsoknownasover‐the‐top(OTT)services,canbedividedintodistinctcategoriesof

(i)user‐generatedcontentfrommostlyamateurs(e.g.,majorityofthecontentservedbyYouTube),

(ii) professionally generated content from studios and networks to promote their commercialofferingsandprogramming(e.g.,thecontentservedbyABC.com,hulu.com),and(iii)directmovie

salestoconsumersovertheInternet(alsoreferredtoasElectronicSell‐Through–EST).Inthelast

category, Netflix, Apple TV and new undertakings such as Ultraviolet will greatly increase theamountofvideoofferingsontheInternet.

Cisco’s Visual Networking Index (VNI)5 suggests that traffic volumes in the order of tens and

hundredsofexabytes(onebilliongigabytes)andzettabytes(onethousandexabytes)arenotthatremote.Overthenextfewyears,about90%ofthebitscarriedintheInternetwillbevideorelated

andconsumedbyoverabillionusers.Whilesomeportionofthesevideobitswillbeformanaged

servicessuchascableTVandIPTV,wecannotignoretheamountofbitsfortheunmanaged(OTT)services. Actually, by the end of 2010, the Internet video traffic will surpass peer‐to‐peer file

sharingforthefirsttimeeverandemergeasthetopInternettrafficcontributor.

Cable TV and IPTV services offered by a service provider run over managed networks fordistribution since these services usemulticast transport and require certainQoS features (1). In

contrast, conventional streaming technologies such as Microsoft’s Windows Media, Apple’s

3http://www.comscore.com/Press_Events/Press_Releases/2010/6/comScore_Releases_May_2010_U.S._Online_Video_Rankings4http://www.netflix.com/MediaCenter?id=53795http://www.cisco.com/web/go/vni

3

QuickTime and Adobe’s Flash as well as the emerging adaptive streaming technologies such as

Microsoft’sSmoothStreaming,Apple’sHTTPLiveStreamingandAdobe’sHTTPDynamicStreamingrun over mostly unmanaged, i.e., best‐effort, networks. These streaming technologies send the

contenttothevieweroveraunicastconnection(fromaserverorcontentdeliverynetwork(CDN)

facilitatingcontentdistribution)througheitheraproprietarystreamingprotocolrunningontopofan existing transportprotocol,mostlyTCPandoccasionallyUDP, or the standardHTTPprotocol

thatisoverTCP.

Historically,progressivedownload,whichusesHTTPoverTCP,hasbeenquitepopularforonlinecontent viewing due to its simplicity. In progressive download, the playout can start as soon as

enough and necessary data is retrieved and buffered. Today, YouTube delivers over two billion

videosadaywiththisapproach.However,progressivedownloaddoesnotoffertheflexibilityandrich features of streaming. Before the download starts, the viewer needs to choose the most

appropriate version if there aremultiple offeringswith different resolutions and qualities of the

samecontent.Ifthereisnotenoughbandwidthfortheselectedversion,theviewermayexperiencefrequent freezesandre‐buffering.The trickmodessuchas fast‐forwardseek/playorrewind,are

often unavailable or limited. These limitations are likely to inhibit the growth of large volume

(includingHD)moviedistributionontheInternet.Anewapproach,whichwerefertoasAdaptiveStreaming, is emerging to address these shortcomings while preserving the simplicity of

progressivedownload.

Adaptive streaming is a hybrid of progressive download and streaming. On one hand, it is pull‐basedasisprogressivedownload:theadaptivestreamingclientsendsHTTPrequestmessagesto

retrieveparticularsegmentsofthecontentfromanHTTPserver,andthenrendersthemediawhile

the file is being transferred.On the other hand, these segments are short duration, enabling theclient to download only whatever is necessary and employ trick modes much more efficiently,

which gives the impression that the client is streaming. However, more importantly, the short‐

durationsegments(e.g.,MPEG4filefragments)areavailableatmultiplebitrates,correspondingtodifferentresolutionsandqualitylevels,andtheclientmayswitchbetweendifferentbitratesateach

request. The client strives to always retrieve the next best segment after examining a variety of

parametersrelatedtotheavailablenetworkresourcessuchasavailablebandwidthandthestateoftheTCPconnection(s),devicecapabilitiessuchasdisplayresolutionandavailableCPU,andcurrent

streamingconditionssuchassizeoftheplaybackbuffer.Thegoalistoprovidethebestqualityof

experiencebyenablingdisplayof thehighest achievablequality, faster start‐up,quicker seeking,andreducingskips,freezesandstutters.

As adaptive streaming usesHTTP, it benefits from the ubiquitous connectivity thatHTTPhas to

offer.Today,anyconnecteddevicesupportsHTTP insome form.Since it isapull‐basedprotocolthateasily traverses firewalls andNATdevices, it keepsminimal state informationon the server

side,whichmakesHTTPserverspotentiallymorescalablethanconventionalpush‐basedstreaming

4

servers.To theexistingHTTPcaching infrastructure, adaptive streaming isnodifferent thanany

other HTTP application. Individual segments of any content are separately cacheable as regularWeb objects using HTTP or any RESTful (conforming to the Representational State Transfer

constraints) Web protocol. This allows distributed CDNs to greatly enhance the scalability of

contentdistribution.

In this first installment of a two‐part series,we describe several streaming, both push and pull‐

based,solutionsthatexisttoday,motivatetheneedforadaptivestreaming,andprovideadetailed

comparisonbetweendifferentmethods.

Media Streaming 

Transmissionof contentbetweendifferentnodeson anetwork canbeperformed in a variety of

ways. The type of the content being transferred and the underlying network conditions usually

determine the methods used for communication. In case of a simple file transfer over a lossynetwork, the emphasis is on reliable delivery. The packets can be protected against losseswith

added redundancy or the lost packets can be recovered by retransmissions. When it comes to

audio/videomediadeliverywith real‐timeviewingrequirements, theemphasis ison low latencyand efficient transmission in order to enable the best possible viewing experience. Occasional

lossesmaybetoleratedinthiscase.Thestructureofthepacketsandthealgorithmsusedforreal‐

time media transmission on a given network collectively define the media streaming protocol.Althoughvariousmediastreamingprotocolsavailabletodaydifferinimplementationdetails,they

canbeclassifiedintotwomaincategories:push‐basedprotocolsandpull‐basedprotocols.

Push‐Based Media Streaming Protocols 

Inpush‐basedstreamingprotocols,onceaconnectionisestablishedbetweenaserverandaclient,

the server is always on and streams packets to the client until the session is torn down or

interruptedbytheclient.Consequently,inpush‐basedstreaming,theservermaintainsaconnectionstatewiththeclientand listens forcommandssentbytheclientregardingsessionstatechanges.

Real‐timeStreamingProtocol(RTSP),specified inRFC23266, isoneof themostcommonsession

controlprotocolsusedinpush‐basedstreaming.

Push‐basedstreamingprotocolsgenerallyutilizeReal‐timeTransportProtocol(RTP),specifiedin

RFC3550,orequivalentpacketformatsfordatatransmission.RTPusuallyrunsonUserDatagram

Protocol(UDP),which isastatelessprotocolwithoutany inherentrate‐controlmechanisms.Thisallows theserver topushpackets to theclientatabitrate that isdependenton theclient/server

6AllRFCdocumentsarefreelyaccessiblefromhttp://www.ietf.org/rfc.html.

5

implementationattheapplicationlevelratherthantheunderlyingtransportprotocol.Thismakes

RTPanicefitforlow‐latencyandbest‐effortmediatransmission.

In conventional push‐based streaming, the content is usually transmitted at themedia encoding

bitratetomatchtheclient’smediaconsumptionrate.Innormalcircumstances,thisensuresthatthe

clientbufferlevelsstaystableovertime.Italsooptimizestheuseofnetworkresourcesbecausetheclientusuallycannotconsumeat anyrateabove theencodingbitrate; consequently, transmitting

abovetheencodingbitratewouldunnecessarilyloadthenetwork.Moreover,thismaynotbeeven

possibleforlivestreamswherethestreamisbeingencodedon‐the‐fly.However,ifpacketlossortransmissiondelaysoccurover thenetwork, theclient’spacketretrievalratemaydropbelowits

consumptionrateandtheclient’sbuffermaystartdraining,whichmayeventuallyresultinabuffer

underflowandinterruptthemediaplayback.Thisiswherebitrateadaptationcomesintoplay.

Topreventabufferunderflow,themediaservermaydynamicallyswitchtoalowerbitratestream.

This,inturn,reducesthemediaconsumptionrateattheclient‐sideandthereforecounteractsthe

effectofnetworkbandwidthcapacity loss.Sincea suddendrop inencodingbitratemayresult innoticeablevisualqualitydegradation,theencodingbitratereductionmaybeperformedinmultiple

intermediatestepsuntiltheclient’sconsumptionratematchesordropsbelowitsavailablereceive

bandwidth.Whenthenetworkconditionsimprove,theserverdoestheoppositeandswitchestoahigherbitratestreaminstead.Again,thisprocessmaybeperformedinmultipleintermediatesteps

nottocauseasuddenoverloadonthenetwork.Providedthatthestreamisnotalivestreamand

thenetworkcapacityallowsforahigherbitratetransmission,theservermayalsochoosetosendover packets at a higher rate than the media encoding rate to fill up the client’s buffer. By

dynamicallymonitoringtheavailablebandwidthandbuffer levelsandadjustingthetransmission

rateviastreamswitching,push‐basedadaptivestreamingmayachievesmoothplaybackexperienceatthebestpossiblequalitylevelwithoutpausesorstuttering.

Thebandwidthmonitoringisusuallyperformedontheclient.Theclientcomputesnetworkmetrics

suchasround‐triptime(RTT),packetlossandnetworkjitterperiodically.Theclientmayusethisinformationdirectly tomakedecisions forwhen to switch to ahigheror a lowerbitrate stream.

Alternatively,theclientmaycommunicatethisinformationalongwithitsbufferlevelstotheserver

viareceiverreportsandlettheservermakethosedecisions.SuchreportsareusuallytransmittedviaReal‐timeControlProtocol(RTCP).

In the following section,we study themechanismsdefined in3rdGenerationPartnershipProject

(3GPP) that constituteanexample forhowbitrateadaptationcanbe implemented inpush‐basedstreamingservers.

Bitrate Ad

3GPP istechnical

(GSM) ne

popularQuickTim

In its Re

dynamicastatistics

areimple

Figure1.

Figure1:E

During sbitrate an

response

withinth

Then,the

daptation in 3

collaboratiospecification

etworks. Th

push‐basedmeStreaming

lease 6, 3GP

ally switchinabout availa

ementations

Example3GPP

ession setupnd codec to

toclient’sR

heavailablea

eclientsend

3GPP 

on betweennsfor3Gmo

he bitrate ad

streaminggServerbyA

PP definedm

ng the transmablebandwi

specific.How

streamingses

p, the serverthe client v

RTSPdescri

audioandvid

sRTSPsetu

telecommuobilesystem

daptation m

servers suchApple.

mechanisms

mission bitraidth and clie

wever,inaty

ssionsetupwit

r sends a lisvia SessionD

ibemessag

deostreamst

upmessages

6

unications assbasedonG

echanisms d

h as Helix

for themed

ate (2) (3).ent buffer lev

ypicalimplem

thbitrateadap

st of availabDescription P

e.Receiving

thatbestfitt

s(oneforea

ssociationsGlobalSystem

defined in 3

streaming s

dia client to

The algorithvels and ada

mentation,th

ptation.

ble streamsProtocol (SD

sessiondesc

toitslinkspe

chuniqueau

whose scopmforMobile

3GPP standa

server by R

inform the

hms used toapting the bi

heprocessgo

and their prDP), specified

cription,the

eedanddeco

udioandvid

pe is to proeCommunica

ards are use

Real Network

media serve

collect realitrate accord

oesasdepict

roperties sud in RFC 456

clientpicks

odingcapabi

deostream)t

oduceations

ed by

ks or

er for

l‐timedingly

tedin

ch as66, in

from

ilities.

tothe

7

server to prepare the server for sending out the selected streams. The RTSP message header

contains a3GPP-Adaptation parameter that informs the server about the client’s buffer size

(size attribute) andminimum required buffering (target‐time attribute) to ensure interrupt‐free

playback.Italsocontainsa3GPP-Link-Charparameterthatprovidesinformationaboutclient’s

guaranteedreceivebandwidth(GBWattribute).

After the serverobtains informationabout the client’sbuffer and receivebandwidth, it setsupa

buffermodeltosimulatethechangesinclient’sreceivebufferlevels.Inthisbuffermodel,theservertries to satisfy the minimum required buffer level while at the same time it tries to prevent

overflows. Since the network conditions and buffering requirementsmay change over time, the

clientkeepsnotifying the serverviaperiodicRTCP receiver reports.TheseRTCPmessages carryinformationsuchasavailablefreespaceinclient’sreceivebuffer,playoutdelay(thetimedifference

between the presentation time of the next frame in the decoder buffer and the time the RTCP

message isbeing sent)andestimatednetworkbandwidth.Using this information, the server canmaintainanaccuratemodelofclient’sreceivebufferandcanmakeinformeddecisionsaboutwhen

toswitchtoahigheroralowerbitratestreamdynamically.

Theupshiftanddownshiftbuffer level thresholds for thestreamareusuallypre‐programmedonthe server. For instance, if the server detects that the client’s buffer level is below the next

downshift threshold, it switches to the next lower encoding rate available to prevent the buffer

fromdraininganyfurther.Ontheotherhand,ifthebufferlevelexceedsthenextupshiftthreshold,the server switches to the next higher encoding rate provided that the network bandwidth can

support that rate. Ifnot, theservermaychoose toslowdown the transmissionrate topreventa

bufferoverflow.

Pull‐Based Media Streaming Protocols 

In pull‐based streaming protocols, themedia client is the active entity that requests the content

fromthemediaserver.Therefore,theserverresponsedependsontheclient’srequestswheretheserver is otherwise idle or blocked for that client. Consequently, the bitrate at which the client

receives the content is dependent upon the client and the available network bandwidth. As the

primarydownloadprotocoloftheInternet,HTTPisacommoncommunicationprotocolthatpull‐basedmediadeliveryisbasedon.

Progressive download is one of the most widely used pull‐based media streaming methods

availableonIPnetworkstoday.Inprogressivedownload,themediaclientissuesanHTTPrequestto the server and startspulling the content from the server as fast aspossible.Onceaminimum

required buffer level is obtained, the client starts playing the media while at the same time it

continuestodownloadthecontentfromtheserverinthebackground.Aslongasthedownloadrateisnotsmaller than theplaybackrate, theclientbuffer iskeptatasufficient level tocontinue the

playback

mayfallb

Similarto

bufferun

bitrate adfragment

independ

reconstruthe right

fragment

according

Figure2:E

Although

forfragm

audiofraframeisu

stuffed in

fragmentindepend

video, th

withoutany

behindthepl

omethodsus

nderflowinp

daptation.Thts) each of

dently. When

uctedseamlet encoding b

t from the m

gtotheavail

Examplebitrat

hthestructur

mentconstruc

meusuallycusuallydeco

nto a media

t duration. Fdently due to

he partitionin

yinterruptio

laybackrate

sedinpush‐b

pull‐basedstr

hemedia cowhich is e

n the fragme

essly.Duringbitrate that m

media serve

lablereceive

teadaptationi

reoftheme

ctionisthes

consistsofcodableonitso

a fragment b

For video, oo common t

ng for fragm

n.However,

andaneven

basedstream

reamingprot

ntent isdiviencoded at

ents are play

gdownload,tmatches or

er. This way

bandwidth.

npull‐baseda

diafragmen

same.Inthe

onstantduratownforcom

by combinin

on the othetemporal pre

ment constru

8

ifthenetwo

ntualbufferu

ming,bitrate

tocols.Figur

ided into shovarious bit

yed back to

themediaclis below the

y the client

adaptivestream

tsdiffersam

casewhere

tionaudiosammonaudioc

ng sufficient

er hand, theediction app

uction is per

orkcondition

underflowma

adaptationi

e2showsan

ort‐durationtrates. Each

back, the o

lientdynamie available b

can adapt

ming.

mongimplem

audioandvi

amplesinthecodecs.There

t number of

e frames arplied betwee

rformed at

nsdegrade,t

ayresult.

isameasure

nexampleim

nmedia segmh fragment

original med

icallypickstbandwidth a

its media c

mentations,th

ideoarenot

emillisecondefore,audio

f audio fram

e not necesen the frame

the Group o

hedownload

etakentopr

mplementatio

ments (also ccan be dec

ia stream ca

thefragmentand requests

consumption

hebasicprin

interleaved,

dsrangeanddatacaneas

mes to match

ssarily decoes. Therefor

of Pictures (

drate

event

onfor

calledcoded

an be

twiths that

n rate

nciple

,each

deachilybe

h the

dablee, for

(GoP)

boundary

frame (I‐thatdepe

the same

closedGofragment

beadjust

duration.preferabl

In the fo

streaminMicrosoft

Microsoft

Microsoft(PIFF) (4

implemen

there istypically

Figure3:F

Figure 3

hierarchi

QuickTimEachbox

y instead. In

‐frame) thatendonother

e GoP, the Go

oPisself‐contfromaclose

tedsuchthat

. However, iletopackmo

ollowing sec

g protocolstandHTTPL

t Smooth Stre

t’s Smooth S4), which is

ntation, all f

a separate fcontainasin

FragmentedMP

shows the s

icaldatastru

me).Thereartypeisdesig

n video codin

can be decoframes.Ifth

oP is called

ntained(i.e.,iedGoPstraig

tthetotaldu

if the fragmeorethanone

ctions, we lo

that are basLiveStreamin

eaming 

Streaming ims an extens

fragmentsha

file for eachngleGoP.

P4fileformat

structure of

ucture.Theb

reboxesthatgnatedbyaf

ng, a GoP is

oded indepehepredictedf

a closedGoP

itcanbedecghtforwardly

urationofth

ent durationGoPintoafr

ook into tw

sed on multngbyApple.

mplementatioion to MPE

aving the sam

available b

(Reproducedf

a fragmente

uildingblock

tcontainaudfour‐letterid

9

a sequence

ndently folloframeswithi

P. Otherwise

codedindepey.Duringenc

heframesin

n is large coragment.

o different

i‐bitrate frag

on is basedEG4 (MP4)

mebitrate a

itrate. Fragm

from(4)).

edMP4 file.

kofthishiera

dio/videodatdentifier.The

of frames th

owed by preinaGoPdepe

e, the GoP is

endentofothcoding,then

theGoPise

ompared to

implementa

gments. The

on Protectespecification

are stored in

ments are u

A fragmente

archyiscalle

taaswellasefilestartsw

hat startwit

edicted framendonlyont

s called ano

herGoPs),onnumberoffra

equaltothe

a typical Go

ations of pul

ese are Smoo

ed Interopern (5). In Sm

na singleMP

sually two s

edMP4 file

edabox(kno

sboxesthatcwithanftypb

th an intra‐c

mes (P‐/B‐fratheframesw

openGoP. Sin

necanconstramesinaGo

desiredfrag

oP size, then

ll‐based ada

oth Streamin

rable File Fomooth Strea

P4 file.There

seconds long

is composed

ownasanat

containmetaboxthatdesc

coded

ames)within

nce, a

ructaPcan

gment

n it is

aptive

ng by

ormataming

efore,

g and

d of a

tomin

adata.cribes

theversio

that descfragment

mdatbox

Themoofragment

Inorder

knowwhclientatt

sideman

theavaila

Figure4:

Oncethe

for the inpair.The

and the t

Streamin

Whenthe

fragment

informatifilemaps

informati

correspo

Afterthe

achieved

contain tinformati

theserve

box,andt

GET/s

oninformati

cribes the mtiscontained

xisimmedia

fboxmaycot,thenumber

fortheclien

hether that fthebeginnin

nifestfiledes

ablestreams

AnexampleH

clienthasdo

ndividual fraHTTPreque

timeoffseto

ngTransport

emediaserv

t, it first ne

ioniscontainsMP4 files t

ion it receiv

ndingMP4fi

correctfile

via indexing

the locationionitreceive

er locates the

thesemakeu

sample/v_72

ionforthesp

media tracksdwithinabo

telyprecede

ontainmetadrofsamples

nttorequest

fragment is agofthesessi

scribescodec

.

HTTPrequestfo

ownloadedt

agments. Eacestmessage

of thereques

clientreque

vergetsanH

eds to dete

nedinanotho thebitrate

ves from the

iletosearch.

isfound,the

gdata.Asse

and presentesfromthec

e fragment i

upthefragm

0p.ism/Quali

pecification(

available inoxoftypem

edbyaboxo

dataandsignwithinthefr

afragment

availableonionviaaclie

cs,videores

oradownload

themanifest

ch fragmentheadercont

sted fragmen

st.

HTTP‐encaps

rmine which

ermanifestfesof the frag

e client (as s

enextstepis

een inFigure

tation time oclienttofind

in the file, it

mentonthew

ityLevels(150

10

s)thefileco

n the file. Tmdat.Inafra

oftypemoof

nalinginformragmentand

withaspeci

the server.Tnt‐sidemani

olutions,cap

dingafragmen

file,itstarts

is downloadtainstwopie

ntas shown

sulatedrequ

h MP4 file

file,whichisgments they

seen in Figu

stolocateth

e3,anMP4

of a fragmendamatchwi

sends thec

wire.

00000)/Frag

omplieswith

The audio/viagmentedMP

thatcontain

mationsuchadurationof

ificbitratean

This informaifestfile.Ina

ptionsandot

nt.

makingHTT

ded via a unecesof infor

inFigure4,

est fromthe

to search to

calledserveycontain.Th

ure 4) in the

herequested

filealsocon

nt. Themediithafragmen

ontainedmo

gments(video

h.Itisfollowe

ideo mediaP4container

nsmetadataf

asasequenceachsample

ndstarttime

ation is comadditiontob

therauxiliar

TPGETrequ

niqueHTTPrmation.The

which isaM

eclientfora

o locate tha

er‐sidemanifhe server loo

e manifest f

dfragmentin

ntainsboxes

ia server usentindexint

oofbox follo

o=160577243

edbyamoo

data for a srstructure(5

forthatfragm

cecounterfoe.

e, itfirstnee

mmunicated titrates,thec

ryinformatio

ueststothes

request‐respsearetheb

MicrosoftSm

aparticularm

at fragment.

fest.Thismanoksup theb

file and find

nthatfile.T

of typemfra

es the time otheMP4file.

owedby the

3)HTTP/1.1

vbox

single5),an

ment.

orthe

edsto

to theclient‐

onfor

erver

ponseitrate

mooth

media

This

nifestitrate

ds the

Thisis

a that

offsetOnce

mdat

Apple HTT

Apple’s H(referred

ISO/IEC1

mediasewhich is

intoequa

comparedbetterco

number

decisionstodynam

Figure5:E

Similarto

oftheme

playlist f

organized

#EXTM3#EXT‐X

http://e

#EXT‐X

http://e

#EXT‐X

TP Live Strea

HTTP Lived to asmedia

13818‐1MPE

gment is stotypically a s

aldurationse

d to twosecmpressions

of fragment

scanbemadmicbandwidt

Exampleshowi

oMicrosoft’s

ediasegment

file standard

d ina two‐le

3U‐STREAM‐INF:P

example.com/lq

‐STREAM‐INF:P

example.com/h

‐ENDLIST

ming 

Streaming ia segment in

EG2Transpo

ored ina sepsoftware com

egments.The

condsused iinceitprovid

ts. However

de.Thereforethchanges.

ingHTTPLive

sSmoothStr

tsavailablef

d and is def

evelhierarch

PROGRAM‐ID=1

q/prog1.m3u8

PROGRAM‐ID=1

q/prog1.m3u8

mplementatn the implem

ortStreamfil

parate transpmponent, rea

edefaultseg

nMicrosoftdesbetterte

r, it also red

e,longfragm

Streamingpla

reamingimpl

fortheclient

fined in (6)

hy.Each file

1,BANDWIDTH

1,BANDWIDTH

11

ion (6) follmentation) st

leformat.As

port streamads a contin

gmentduratio

SmoothStreemporalredu

duces the g

mentduration

aylists.

lementation,

t.Themanif

. Figure 5 s

startswitha

H=100000

H=2000000

ows a diffetorage.HTTP

sopposedto

file.Aproceuous transp

onis10seco

eaming.Longundancyand

granularity a

nmaynotbe

,aclient‐sid

festfileform

shows an ex

anEXTM3U

#EXTM3

#EXT‐X‐M

#EXT‐X‐T

#EXTINF

http://ex#EXTINF

http://ex

…

#EXT‐X‐E

#EXTM3#EXT‐X‐M

#EXT‐X‐T

#EXTINF

http://ex

#EXTINF

http://ex…

#EXT‐X‐E

erent approaPLiveStrea

SmoothStre

essnamedSort stream f

onds.Thisis

ger fragmendatthesame

at which fra

easgoodint

emanifestfi

matisanexte

xample of t

tag thatdist

U

MEDIA‐SEQUEN

TARGETDURAT

F:10,

xample.com/seF:10,

xample.com/se

ENDLIST

UMEDIA‐SEQUEN

TARGETDURAT

F:10,

xample.com/se

F:10,

xample.com/se

ENDLIST

ach for fragaming isbase

eaming,here

StreamSegmefile anddivid

alongerdur

ntdurationaetimereduce

agment‐swit

termsofada

ilekeepsare

ensiontothe

the manifest

tinguishes th

NCE:0

TION:10

gment‐HQ1.ts

gment‐HQ2.ts

NCE:0

TION:10

gment‐LQ1.ts

gment‐LQ2.ts

gmentedon

eeach

enter,des it

ration

allowsesthe

tching

apting

ecord

eMP3

t files

he file

fromano

level file

Identifier

has an a

manifest

alternate

quality(2

Thelowe

download

number o

sequencesequence

EXTINF

correspo

Example

Aclient‐s

theserve

identifiesfrom frag

othercon

Figure6:A

Thebasic

theclient

ordinaryMP3

s on the rig

r(URI)thati

ttribute nam

for an alter

encodingsf

2Mbps)enco

er‐levelfiles

d.Following

of the first s

e number, ae number. F

tag.Thistag

ndingmedia

e Functiona

sidepull‐bas

erprovidess

s files that cogments of a

nditions.

Aclient‐sidepu

cclient/serv

t‐sideofFigu

3playlist.Th

ght. The lin

salwayspre

med BANDWI

rnate encodi

foreachsegm

oding.

ontheright

theEXTM3U

segment in t

nd sequenceFinally, the U

ghasanattr

asegment.In

al Diagram 

edadaptive

standardres

ontainmediafile over on

ull‐basedadap

veradaptive

ure6.

hefileonthe

nk for each

ecededbyan

IDTH that in

ing of the se

ment, that is

providethe

Utag,isanE

the playlist.

e numbers iURIs for the

ibutesepara

theexample

for Client‐S

streamingim

ponsestoHT

apresentatioeormore co

ptivestreaming

streamingco

12

leftisahigh

lower‐level

nEXT-X-STR

ndicates that

egments it li

s,onelow‐qu

linksforthe

EXT-X-MEDI

Each segme

increment be individual

atedbyasem

e,eachsegme

Side Pull‐Ba

mplementati

TTPGETreq

onsat alternonnections a

gimplementat

onfiguration

her‐levelfile

file is spec

REAM-INFt

t the corres

ists. In the e

uality(100K

eindividual

IA-SEQUENC

ent in the p

by one startsegments f

micolonthat

entis10seco

ased Adapt

ionisillustra

quests.Thec

nativebitrataccording to

tion.

requiresthe

thatlinksto

cified in a U

tag.EXT-X-S

sponding low

example, the

Kbps)encodi

mediasegm

CEtagthatl

playlist typica

ting from thfollow each

t indicatesth

ondslong.

tive Stream

atedinFigur

clientgetsa

es.Theclieno the playout

egeneralfun

otwootherlo

Uniform Reso

STREAM-IN

wer‐level file

e stream has

ingandone

mentsavailab

liststhesequ

ally has a un

he first segmmarked wit

hedurationo

ming 

re6.Atminim

manifestfile

nt acquiresmt buffer state

nctionsshow

ower‐

ource

NFtag

e is a

s two

high‐

lefor

uence

nique

ment’sth an

ofthe

mum,

ethat

mediae and

wnon

13

Theclientneedsafile,calledaclient‐sidemanifestoraplaylisttomapfragmentrequeststo

specific files or byte ranges into files. In some adaptive streaming schemes, there is asimilarfileontheservertotranslateclientrequests.

Managing a playout buffer is also a basic function that is always needed on an adaptive

streamingclient,whichusesadaptationlogictoselectfragmentsfromafileataparticular

bitrate in response to buffer state and potentially other variables. Typically, an adaptivestreamingclientkeepsaplayoutbufferofseveralseconds,e.g.,5–30seconds.

Atransportisneededtocommunicaterequestsfromtheclienttotheserver;thepull‐based

adaptivestreamingschemesthatwesurveyinthisarticleuseHTTPGETrequests,eithertoa standardHTTPWeb serveror to a specializedWeb servicesAPI that is supportedby a

specialserviceontheserver(suchasaSmoothStreamingTransportapplicationrunningon

Microsoft’sInternetInformationServicesserver).

The GET requests use a single TCP connection by default, but some adaptive streaming

implementations support using multiple concurrent TCP connections for requesting

multiplefragmentsatthesametimeorforpullingaudioandvideoinparallel.

Theclientispre‐configuredtorequestamovieatacertainbitrate,orprofile,basedontheresultof

networktestsorsimpleconfigurationscript.WhenaprofileisselectedandtheURIassociatedwith

theprofile is found in themanifest, the clientestablishesoneormore connections to the server.Wehaveobservedthatdifferentproductsemploydifferentstrategies.Forexample,someproducts

useonlyoneTCPconnectionforGETrequeststoafilewhereasothersopenandclosemultipleTCP

connectionsduringanadaptivestreamingsession.Astheclientmonitorsitsbuffer,itmaychoose

toupshifttoahigher‐bitrateprofileordownshifttoaloweronedependingonhowmuchtherateofvideo transport does or does not exceed the rate of media playout. Some adaptive streaming

productsopenanewconnectionwhenupshiftingordownshiftingtoanewprofile.

Alternative Methods for Bitrate Adaptation 

We havementioned adaptive streamingmethods thatwork based on the principle of switching

betweenalternateencodingsofastreamasawholeorswitchingbetweenindividualfragmentsofitencoded at several bitrates. There is also an alternative technology called Scalable Video Coding

(SVC). In SVC‐based systems, a client can choosemedia streams appropriate for the underlying

network conditions and its decoding capabilities. SVC has been standardized as an extension toH.264/MPEG4 AVC video compression specification. SVC in its current form applies to video

streamsonly.

InSVC,avideobitstreamismadeupofahierarchicalstructureoflayers.Thebaselayerprovidesthelowest levelofquality intermsof framerate,resolutionandsignal‐to‐noiseratio(SNR).Each

enhancement layer on top of the base layer provides an improvement for one ormore of these

scalable quality parameters. Enhancement layers can be independently stored or sent over the

14

network.Therefore,theoverallstreambitratecanbemodifiedbyselectivelyaddingorsubtracting

enhancementlayersto/fromastream.

The quality knobs in SVC are referred to as temporal scalability, spatial scalability and SNR

scalability.Temporalscalabilityprovidesawaytoaddordropcompletepicturesto/fromastream.

Temporal scalability is not something newly introduced by SVC. Temporal scalability in itsmostbasicformhasbeenusedintraditionalpush‐basedstreamingserversintermsofamethodknown

as stream thinning. In spatial scalability, on the other hand, video signal is encoded at multiple

resolutions. The reconstructed lower‐resolution frames can be used to predict and reconstructhigher‐resolution frameswithadded informationsent in theenhancement layers.Finally, inSNR

scalability,videosignalisencodedatasingleresolution,butatdifferentqualitylevelsbymodifying

theprecisionofencoding.Eachenhancementlayerincreasestheprecisionofthelowerlayers.

A key advantage of SVC lies in its ability to distribute information among various layers with

minimaladdedredundancy.Inotherwords,whileastreamthatistraditionallyencodedatdifferent

quality levels has significant redundancy between the encodings; each layer in an SVC‐encodedstreamhasminimalcommoninformationbetweenthelayers.ThismakesSVCefficientforstorage

ofmediaatvariousqualitylevels.AnotheradvantageofSVCisthegracefuldegradationofstream

quality without client or server intervention when packets from enhancement layers cannot bedeliveredduetoabruptchangesinnetworkconditions.Thisisincontrasttomulti‐bitrateencoding

whereitrequiresswitchingfromoneencodingtoanotherviaadecisionbytheclientortheserver

toadjusttothechangesinnetworkconditions.SVCstreams,ontheotherhand,aretypicallymorecomplex tobegeneratedand imposecodecrestrictionscompared tomulti‐bitratestreams.Thus,

theadoptionrateforSVChasbeenratherlow.

Comparison of Push‐Based vs. Pull‐Based Streaming Protocols 

Above,wedescribedpushandpull‐basedstreamingprotocols,mentionedtheirbitrateadaptation

mechanisms and provided case studies. In this section, we present a qualitative comparison

betweenthesetwoclassesofmediastreamingprotocols.

Table 1 summarizes the differences between push and pull‐based streaming7. One of the main

differencesbetweenpushandpull‐basedstreamingisthecomplexityoftheserverarchitecture.As

mentionedbefore, inpull‐basedstreaming, thebitratemanagement isusuallyataskof theclient,which significantly simplifies the server implementation. Furthermore, pull‐based streaming can

runon topof simpleHTTP.Therefore,withminorprovisions, anordinaryWeb server can serve

mediacontentinpull‐basedstreaming,althoughcomplexitiesmayariseinstreaminglivecontentto

7Inthiscomparison,pull‐basedstreamingexclusivelyreferstostreamingoverHTTP.Albeitunlikely,apull‐basedstreamingmethodrunningoveradifferenttransportcanbelaterdevelopedandadopted.

15

a largenumberofclients.Push‐basedstreaming,ontheotherhand,requiresaspecializedserver

thatimplementsRTSPorasimilarpurpose‐builtprotocolandhasbuilt‐inalgorithmsfortaskssuchasbitratemanagement,retransmissionandcontentcaching.Thismakespull‐basedstreamingcost

effectivecomparedtopush‐basedstreaming.

Despite the lower server costs, pull‐based streaming is usually less efficient overhead‐wise thanpush‐based streaming due to the underlying transport protocol being used. Compared to HTTP

overTCP,RTPimposesalowertransmissionoverhead.Moreover,sinceRTPusuallyrunsontopof

UDP, the retransmission dynamics and congestion controlmechanisms of TCP do not inherentlyexistinRTP.

Both protocols allow client buffering both at the beginning of a session and also after trick‐play

transitionssuchasfastforwardtoplay.Thisisperformedtopreventbufferunderflowsandachievesmoothplayback experience. In adaptive streamingmethods, the initial client bufferingduration

canbesubstantiallylessthannon‐adaptivestreamingmethodsduetothefacttheclientcanpicka

lower‐bitratestreamtobeginwith.Thisallowsfaststartuptimesandincreasesresponsiveness.

Oneofthekeybenefitsofpush‐basedstreamingismulticastsupport.Multicastallowsaserverto

sendapacketonlyoncetoagroupofclientswaitingtoreceivethatpacket.Thepacketisduplicated

alongthenetworkpathintheoptimalway.Aclientcanjoinorleaveamulticastgroupondemand.Thisway,aclientcanreceiveonly thepackets itwants.Pull‐basedstreaming,ontheotherhand,

worksbasedontheunicastdeliveryscheme.Inunicastdelivery,thereisaone‐to‐onepathbetween

theserverandeachclient.Therefore,intheworstcase,aserverhastosendapacketasmanytimesasthenumberofclientsrequestingthatpacket,and,similarly,anetworknodeinthemiddlehasto

passalongthesamepacketmultipletimes.Thisreducestheefficiencyofthenetwork.

The routing and packet delivery over the network can bemademore efficient by using contentcaching.Thecontentiscachedalongthenetworkondedicatedcacheservers.Thiswayaclientmay

obtaincontentfromanearbycacheserverinnetworktopologyinsteadofgoingallthewayupto

theoriginserver.Probably, themostefficientuseofcaching is in thecaseofpull‐basedadaptivestreaming.Inthiscase,eachfragmentcanbecachedinthenetworkindependently.

Table1:Comparisonbetweenpush‐basedandpull‐basedstreamingprotocols.

Push‐BasedStreaming Pull‐BasedStreaming

ServerType Streamingserver

WindowsMedia,AdobeFlash,AppleQuickTime,CiscoCDS

Webserver

LAMP,MicrosoftIIS,CiscoCDS

URLFormat rtsp://…,rmtp://… http://…

Protocols RTSP,RTMP,RTP,UDP HTTP

16

BandwidthUsage (Likely)Moreefficient (Likely)Lessefficient

ClientBuffering Yes Yes

VideoMonitoringand

UserTracking

Standardtools (Currently)Proprietary

MulticastSupport Yes (Currently)No

Contentcaching Requiresspecialservers Yes

Concluding Remarks 

PopularityofwatchingtraditionalbroadcastTVprogrammingisweakeningeverydayagainstthe

rapidlyincreasinginterestinwatchingthecontentontheWeb.ConsumersareaccessingtheWebcontentnotjustfromaTVinthelivingroombutfromavarietyofdevicesfromavarietyofplaces

connected throughdifferent typesofaccessnetworks.Adaptivestreamingmethods that candeal

withthechallengespresentedbythevarietyofsuchdevicesandnetworksaswellasthescalabilityofcontentdistributiontolargeaudiencesarefurtheracceleratingthistrendtransition,whichwill

clearly impact theexistingbusinessmodelsandcreatenewrevenueopportunities. In thesecond

partofthisarticle,wewilllookintoapplicationsforstreamingincludingend‐to‐endmobileandin‐home streaming, contrasting adaptive approaches to other videodeliveryparadigms, discuss the

current standardization efforts and highlight the areas that still require further research and

investigation.

References 

1.  IPTV: Reinventing  Television  in  the  Internet Age.  Thompson, Greg  and Chen, Yih‐Farn Robin. May 

2009, IEEE Internet Computing, Vol. 13, pp. 11‐14 . 

2.  Transparent  End‐to‐end  Packet‐switched  Streaming  Service  (PSS);  Protocols  and  Codecs.  3GPP  TS 

26.234. [Online] Oct. 2010. http://ftp.3gpp.org/specs/html‐info/26234.htm. 

3. Adaptive  Streaming within  the 3GPP Packet‐Switched  Streaming  Service.  Fröjdh, Per  , et al., et al. 

Mar. 2006, IEEE Network, Vol. 20, pp. 34‐40. 

4. Protected Interoperable File Format. [Online] May 2010. http://go.microsoft.com/?linkid=9682897. 

5.  ISO/IEC  14496‐12  Base  Media  File  Format.  [Online]  2008. 

http://www.iso.org/iso/catalogue_detail.htm?csnumber=51533. 

17

6. HTTP Live Streaming. IETF Internet Draft. [Online] June 2010. http://tools.ietf.org/html/draft‐pantos‐

http‐live‐streaming. 

 

Biographies 

AliC.Begen(abegen@cisco.com)iswiththeVideoandContentPlatformsResearchandAdvanced

DevelopmentGroupatCisco.Hisinterestsincludenetworkedentertainment,Internetmultimedia,transport protocols, and content distribution. He is currentlyworking on architectures for next‐

generationvideotransportanddistributionoverIPnetworks,andheisanactivecontributorinthe

IETFintheseareas.HeholdsaPh.D.degree inelectricalandcomputerengineeringfromGeorgiaTech. He received the Best Student‐Paper Award at IEEE ICIP 2003, and the Most Cited Paper

AwardfromElsevierSignalProcessing:ImageCommunicationin2008.Heisalsoamemberofthe

ACM.

TankutAkgul(akgult@cisco.com)isasoftwareengineerintheServiceProviderVideoTechnology

Group at Cisco. His research interests are embedded software design, video compression and

multimedia streaming.He currently designs and develops software for the next‐generation IPTVset‐topboxes.PriortoCisco,heworkedasaseniorscientistatSoftNetworks,LLCanddeveloped

several video codecs for Intel embedded SoCs. He received his Ph.D. degree in electrical and

computerengineeringfromGeorgiaTech.HisPh.D.thesiswasondebuggingandreverseexecutionofprogramsforembeddedsystems.Hehasseveralconferenceandjournalpublicationsinthearea

ofembeddedsoftware.

Mark Baugher (mbaugher@cisco.com) is a technical leader in the Research and AdvancedDevelopmentgroupatCisco.Prior toCisco,Markco‐foundedandservedas theCTOofPassedge

Inc.,whichdevelopedamulticastmediasecurityproduct.Overtheyears,Markhasledteamsthat

deliveredinnovativetechnologiessuchasIntel'sPC‐RSVP,thefirstnetworkreservationsystemona PC, andLANServerUltimedia, IBM's firstmedia server product.Markhas co‐authored several

widely used international standards in the IETF and ISMA, and he is currently co‐chairing the

Internet GatewayDeviceWorking Committee in the UPnP Forum.Mark holds anM.A. degree incomputersciencefromtheUniversityofTexasatAustin.

1

Watching Video over the Web, Part II Applications, Standardization and Open Issues  

AliC.Begen,TankutAkgulandMarkBaugher,Cisco

In this secondpartofa two‐partarticle, theauthors look intoapplications for streaming includingend‐to‐endmobileand in‐home streaming, contrastingadaptiveapproaches toothervideodelivery

paradigms, discuss the current standardization efforts and highlight the areas that still require

furtherresearchandinvestigation.

Applications: End‐to‐End Mobile and Home Streaming 

There aremany video applications and services on the Internet, and the first part of this article

mentionedseveralofthem.Themostimportantoftheseapplicationsarguablyfallintothemobile

and home entertainment categories. These two categories span the range of video presentationformatsforportable,standardandhigh‐definitionvideo. Theyalsooperateondiversenetworks

and use both managed and unmanaged end‐to‐end network services. Mobile and home video

applications are among the most popular uses of video and include both walled‐garden andInternetapplications.This section identifiesmobileandhome‐streamingusecasesandconsiders

thesuitabilityofpull‐basedversuspush‐basedadaptivestreamingforeachusecase.

Mobile Streaming 

Peoplestreammovies,YouTubevideosandothermediaon theroad, inhotels,oncampusesand

while inmotion to theirmobile devices. Portable‐definition (PD) encoding is a video format for

mobile devices, which is a lower resolution and lower‐bitrate encoding than standard (SD) andhigh‐definition(HD)encoding,whicharecommoninhomes.PDvideodeliveryscalesdowntothe

lowerspeedsandhigher‐lossfoundonmostmobilenetworks:Variationsinloadonnetworkcells,

changes in radio‐network conditions and fluctuations in available throughput force videoapplications to adapt to the network rather than vice versa in GPRS, EDGE, UMTS and CDMA 1x

networks. Thus, it is no surprise that mobile handset vendors and operators have been in the

forefrontindefininganddevelopingadaptivestreamingstandards.

When the mobile connection is to a walled‐garden server, the network provider can manage

networkvideodelivery toa service‐compliant,walled‐gardenclient. This isoneuse case. If the

end‐to‐end connection is to a server on the Internet, the network provider cannot provide QoS

2

management,and theservice isnotmanaged. This isanotherusecase.There isalsoa thirduse

casethatcombinesmobileandhomestreaming.

Walled Garden 

In this case, the end‐to‐end path is contained in the cellular provider’s network. The network

operator can specialize service for video delivery. The provider can choose to use push‐basedadaptivestreamingservicesfromspecializedstreamerstogetoptimaluseofthenetworkresources

and improved video quality over a range of network conditions. Alternatively, the provider can

choose to reuse their data network services such asHTTP servers and caches, and employ pull‐basedadaptivestreamingmethodssuchasApple’siPhoneHTTPLiveStreaming.

Mobile Internet 

The cellular provider cannotmanage the service quality on an end‐to‐end basiswhen the video

connectiontraversesthepublicInternet.Inthiscase,pull‐basedadaptivestreaminghasadvantagesoverpush‐basedmethods.

Hybrid Mobile‐Home Use Case 

Thecellularprovidercannotguaranteeaservicewhenthefirstnetworkhop(s)orthelastnetworkhop(s)areontheclient’shomenetwork.Femtocellnetworksandhome‐networkservicestomobile

clients are examples. When an unmanaged homenetwork is a hop on the end‐to‐end path, and

particularlywhen it is awireless hop, pull‐based adaptive streaming is desired instead of push‐basedmethods.

Home‐Network Streaming 

The home network use cases are crucial to video streaming since video is so widely used byconsumersathome.Althoughinmanyresidences,thehomenetworkconsistsofasingleEthernet

cable connected to a broadband modem and the home PC, more complex home networks are

becomingcommontodayinmanycountries.Increasingly,theself‐installedhomeLANsarecapableoftransportingvideo.

802.11(WiFi)LANsarewidespreadinmanyregionsoftheworld.Onewell‐knownvideo‐delivery

problem in WiFi networks is the divergence of network speed and reliability due to networktopology,load,andothervariables. Thepresenceofmetalobjects,commonhomeappliancesand

other sources of interferencemean that a television in one room often cannot sustain the same

quality of presentation as a device in another. The reality of diverse network capacity amongreceivers coupledwith thedesignof the802.11mediumaccess control also causeaproblem for

multicaststreamingoverwirelessLANs(WLAN)(1).

Video‐on‐demand, not broadcast TV, is likely to be the dominant video application on the homenetworks.Unfortunately,wirelessandother typesofcommonhomeLANssuchaspowerline,are

3

vulnerable to interferenceandoverload. AlthoughthemigrationofWiFiproducts to5GHz from

2.4GHzreducessomeoftheinterferenceproblems,even802.11nisvulnerabletointerferencethatresultsinwidelydifferinglossandthroughputcharacteristicsontheWLAN(2).AnotherhomeLAN

technology that isvulnerable to interference ispowerline communications (PLC) in thehome. In

fact,withtheexceptionsofwell‐laidEthernetoverCat5orovercable(asdefinedbytheMultimediaover Coax Alliance ‐ MoCA), it is hard to imagine a home networking technology that can offer

uniform,high‐speedservicesthroughoutthehome.

Whereas HTTP adaptive streaming is productively used on the public Internet and unmanagednetworks in general, adaptive streaming is arguably crucial for unicast, point‐to‐point video

deliveryonhomeWLANs.Homenetworksaremostlyunmanagedandexhibitgreatdifferencesin

networkspeedsandfeedsavailabletoadiversesetofdevices.Itisunreasonabletoexpectthatatypical home userwillmanage and select particular resolutions of amovie title based upon the

qualityofnetworkservicethataparticulardevicecanobtain fromaparticularspotonthehome

network.

Itisthereforeunlikelythatmanagedvideowilleverbecommonforin‐homestreaming.Managed

videowill likely continue to provide a superior, trouble‐free and gold‐level video service to the

home, however, which unmanaged services try to replicate. In the use cases that follow, bothmanaged andunmanagedmethods are considered, andpush‐based streaming is compared to its

pull‐basedcounterpart.

Streaming to a Home Client from a Home Server 

Thisusecaseisarguablythemostinterestingbuttheleastcommoncase:Foryears,hobbyistsand

digitalmediaenthusiastshaveconnected theirhomeTVs to theirhomenetworks toplaymovies

fromaPC,gamingconsoleorotherservers,butthisisnotyetamainstreampractice.IntheU.S.,theWiFinetworkcanautomaticallyconnectanoff‐the‐shelfhomenetworkattachedservertoaSony

PlayStation and a host of other devices implementing Digital Living Network Alliance (DLNA)

specifications1.However,thiscapabilityisnotwidelyusedbynon‐technicalconsumers.Settingupa home media server requires sufficient knowledge of how to load a server with commercial

movies, which are usually encrypted, match formats with players, manage Digital Rights

Management(DRM)whenitisexposedornotcircumvented,andperformothertasksthatpeopleoutside of the computer industry do not routinely do. Nonetheless, home servers can be quite

usefultopeoplewhowanttoplaytheirdigitalmovieswithoutexperiencingthebottlenecksofthe

Internetorserviceprovider’snetwork.Thehomeusershouldbeabletoplayafileevenwhentheirbroadbandnetworkconnectionisdown,andplaytheirmoviesonalltheTVsorotherdisplaysin

1http://www.dlna.org

4

thehomebesides justone. iTuneshasachieved in‐homemusicsharing;Appleandothervendors

willundoubtedlydevelopvideosharingamonghomedevicesaswell.Ontheotherhand,standardsolutions such as DLNA can open the market to a greater diversity of products and are likely

importanttofuturein‐homevideousecase.Streamingtoahomeclientfromahomeserverisause

casethatisarguablymoreimportanttothefutureofhomevideothanitistoday.

Whether using proprietary or standard players, home wireless networks are limited by the

available bandwidth of the radio spectrumand various types of noise and interference;wireless

networks,ingeneral,arearguablytheweakestlinkinthevideotransmissionchain.Thequalityofthe video presentation on a wireless home network is subject to all of the problems of an

unmanagednetworkplusproblemswithsignalloss,interference,andcontentionwithothertraffic

ontheWLAN(3).

Whenpush‐basedadaptivestreamingisused,thehome‐networkownermustinstall,configureand

manageaspecializedpush‐basedstreamingserveronthehomenetworkthatisnotmanagedbya

businesssuchastheInternetserviceprovider. Moreover,themostpopularprotocolforvideoonthe home network is HTTP followed by RTP/RTSP. HTTP is used on the Web and in home

networking protocols such as DLNA. It will be used bymany new services such as Ultraviolet.

TherearemanyturnkeyHTTPvideoapplicationsandcommodityserversavailable inthemarket,andHTTPworks throughhome firewalls. Thus, pull‐based streaming overHTTPhas advantages

overpush‐basedstreamingoverRTP.

Streaming to a Home Client from an Internet Server 

This use case includes,Webcasts, YouTube, Hulu, iTunes and other Internet video services to a

homePCordigitalTV.Asexplainedabove,apotentiallytroublesomehomenetworkhop,suchasa

homeWLAN,willoftenbepartoftheend‐to‐endvideopathwheneverthenetworksupportsmorethanasingledeviceconnected toamodem;aconfiguration that isbecoming increasinglyrare in

developed countries. Managed videowithin the home is unlikely to become common since the

home network is usually unmanaged. The public Internet,moreover, only supports unmanagedvideodelivery.Thus,pull‐basedadaptivestreamingistheonlyfeasibleapproachforthisusecase.

Streaming to a Home Client from a Managed Server 

Most digital television services today, including video‐on‐demand, originates from video service

providers.Aprovider’svideoservicemightbeanATSC,IPTVorHTTPservice,whichinallcasescanbeoptimizedformanagingvideoovertheprovider’snetworktoahomedevice.Managedvideo

generally achieves a level of quality that unmanaged video attempts to meet – often without

success.Thus,amanagedservercanuseapush‐basedadaptivestreamingmethod.

5

HTTP progressive download or adaptive streaming can of course run over a managed video

network andevenuseamanaged server.Thus, thisuse case includesbothpush‐basedandpull‐basedstreaming.

Streaming to a Home Client via Peer‐to‐Peer (P2P) Delivery 

In certain parts of theworld, particularly Asia and some parts of Europe, a popularmethod forstreamingvideoistouseP2Pnetworks.ResearchhasshownthatP2Ptransportcouldreducethe

load on the source servers and provide a better scalability for large streaming populations.

However, with the advances in server and caching infrastructure designs, large streamingcapacities are now easily achievable without needing P2P transport. On the business side,

uncontrolledandunmanagedP2Psystemsoftenfailtoprovideafairrevenuesharingbetweenthe

content and Internet serviceproviders.While thereare industry‐wideefforts toaddress someofthespecificissuesrelatedtonetworkandprovider‐friendlyP2Ptransport,thebusinessmodelsthat

willbeprofitableforboththecontentowners/providersandInternetserviceprovidersarestillnot

inplace.Thus,inthisarticle,wewillnotgointofurtherdetails;interestedreadersmayrefertotheDecember 2007 issue of the IEEE Journal on Selected Areas in Communications for a detailed

coverageofvarioustopicsinP2Pstreaming.

Summary 

ThetwomajortrendsinInternetvideo,therefore,arestreamingtomobiledevicesandtoaTVor

other in‐home screen. Of the six use cases of mobile and home‐network streaming, most have

unmanagednetworksalongtheirend‐to‐endpaths.

Two of the cases shown in Table 1 can benefit from push‐based adaptive streaming using

specializedserversandclients,i.e.,the(walled‐garden)managedserverusecasesinwhichtheend‐

to‐endpathisoveraprovider‐managednetwork.ManagednetworksareincreasinglyoptimizedforHD video delivery; managed network devices comply with the technical protection measures

required by major video producers. And video service providers have the strategic business

relationships to allow them an early spot in the windowed movie release schedules as well asspecialpay‐per‐viewvideoevents.Thisgivesserviceprovidersaroleforpremiumcontentdelivery

ofmajortitles,particularlyexpensivemovies.Asstatedabove,managedvideoisthestandardthat

Internetvideomustmeet.

6

Table1:End‐to‐endpathsandpreferredstreamingmethods.

InternetServer ManagedServer HomeServer

MobileClient Pull‐basedstreaming Pushandpull‐based

streaming

Pull‐basedstreaming

HomeClient Pull‐basedstreaming Pushandpull‐basedstreaming

Pull‐basedstreaming

Howclosecanunmanagedvideocometomanaged‐videoquality?Andhowwellcansuchservices

scale? These questions are crucial to the future of Internet video services that use pull‐basedadaptivestreaming.Arguably,theInternetserverpathinTable1offersusersthegreatestaccessto

themostcontent–includingbroadcasters’sites,iTunes,Netflix,Hulu,andevensocialnetworking

sites. Ifpull‐basedadaptivestreamingcanmatchthequalityofmanagedvideowellenough,thenthisusewillfavorthefuturesuccessofthisvideostreamingtechnology.

Ofthesixusecases,streamingfromahomeservertoahomeclientismuchlesspopularcompared

to Netflix, Hulu, iTunes, and managed video from service providers/retailers. This use casebecomesmoreimportant,however,asdigitalfilesreplacedigitaldisksinthehome.Andwhenthe

broadband connection becomes unavailable, as broadband connections sometimes do, then the

usershouldat leasthaveaccess to their titleswithin theboundsofa functioninghomenetwork.Streamingtoahomeclientisalsointerestingbecausethisusecaseoperatesonthehomenetwork,

andthevideo‐deliveryperformancecharacteristicsofhomenetworkssuchas802.11areimportant

totheInternetandmanagedservercasesaswell.Wethereforeconcludethatpull‐basedadaptivestreaming methods have considerable advantages over push‐based adaptive methods for most

Internetvideoapplications.

Standardization Efforts 

MicrosoftPIFF (Protected InteroperableFileFormat) (4) andSST (SmoothStreamingTransport)(5)arecompletespecifications foranadaptive‐streamingcontainerandstreamingprotocol.PIFF

and SST use the standard MP4 fragmented file structure. Microsoft has also opened up these

specifications forwidespread use by offering theMicrosoft Community Promise,which providesintellectual property rights guarantees for organizations that choose to use their adaptive

streamingprotocols.ApplehassimilarlyoffereditsHTTPLiveStreamingontheWebandinanIETF

draft (6). Both the Microsoft and Apple specifications reflect the current adaptive streamingproductsthatareinthemarketandofferedbythetwocompanies.

Theseandotherspecificationeffortsareappearing invariousvenues.These include3GPP,MPEG

and industry consortia such as the Digital Entertainment Content Ecosystem (DECE) LLC,which

7

announced a new service calledUltraviolet in July 2010.DECEhas notmade public its technical

specification at the time of thewriting of this article. It has been announced, however, that theDECE Ultraviolet service would support a common container format, movie downloads and

streamingwithsupportformultipleDRMs2.Itseemslikelythatthefinalspecificationswillsupport

at leastsomeof thestate‐of‐the‐artmediadistributiontechnologiesthathavebeenconsidered inthisarticle.

Although DECE LLC’s membership includes dozens of companies who are involved in media

distribution, it isaprivateeffortbyaconsortiumof companies.Adaptivestreaming technologieshavemadetheirwayintopublicstandardsdevelopmentorganizationsincluding3GPPandMPEG.

3GPPisthefirstorganizationthatreleasedaspecificationrelatedtoadaptivestreamingoverHTTP.

3GPPRelease9(7)waspublishedinMarch2010,andtheSA4WorkingGroupiscurrently intheprocess of bug‐fixing this specification taking into account experience from the first

implementations. In addition, 3GPP is preparing an extended version for Release 10, which is

scheduledtobepublishedlater in2011.Thisreleasewill includeanumberofclarifications,offerimprovementsandaddnewfeatures.OntheMPEGside,theStreamingofMPEGMediaoverHTTP

AdHocGrouphasreceived15submissionstotheircallforproposalspublishedinMay2010.After

an initialevaluationperiod in July2010, thisadhocgroup,whichhasbeenrenamedasDynamicAdaptiveStreamingoverHTTP(DASH)AdHocGroup,hasadoptedthe3GPPRelease9asabaseline

specification and started running several evaluation experiments. The DASH Ad Hoc Group will

workonstandardizingthemanifest file,delivery format,conversion from/toexisting file formatsand the use ofMPEG2 Transport Streams as amedia format. TheDASHAdHoc Groupwill also

coordinatecloselywith3GPPSA4WorkingGrouptobetteraligntherespectivespecifications.

Open Issues and Future Directions 

Atthewritingofthisarticle,therearestillmanyquestionsaboutadaptivestreamingthathavenotbeenadequatelyansweredyet.Whilethereareearlyinvestigationssuchas(8)andthereferences

therein,most of the questions still remain openmostly due to the lack of field data required to

conductarigidanalysis.Todate,TCPandHTTPhavebeenindividuallystudiedingreatdetail forconventionalapplications.Proposalshavebeenmadeforbothprotocolstomakethemabetterfit

for streaming‐like applications supported by simulations and experiments. However, making

generalizationsanddrawingconclusionsforlarge‐scalestreamingdeploymentsbasedonnarrowlyscopedstudiesisnottrivialandmaybeeasilymisleading.

2http://www.homemediamagazine.com/digital‐copy/dece‐becomes‐ultraviolet‐20060

8

Historically,TCPhasbeendesignedandoptimizedforFTP‐likeapplications,andimplicationsofits

congestion control algorithm and currently adopted best practices such as Nagle’s algorithm,delayedacknowledgments,slow‐startrestart(SeeRFC5634)havetobeexaminedinthecontextof

adaptivestreaming.EffectsofrunningmultipleTCPconnectionssimultaneouslyforovercomingthe

head‐of‐line blocking problem or another transport protocol that inherently provides a bettersupport for multiple streams such as SCTP have also to be studied carefully. Most importantly,

thesestudiesmustrepresentlarge‐scaledeploymentswithmanyoriginservers,cachesandclient

streamingavarietyofcontentoverdiverselycharacterizednetworkpaths.

A niche area is to develop instrumentation tools to assess the effectiveness and performance of

adaptivemediatransport.Thesetoolsmustbeuser‐friendly,abletoprovideadequateinformation

fordiagnosticsandfault isolation.Suchinformationbecomesquitehandyforserviceprovidersinfixingproblemsandmakingthenecessaryprovisionsandenhancementsintheirnetworks.

An interesting future research direction is to understand how network elements can help a

providerachieveabetterperformance,whichcouldbesupportingmoreclientsordeliveringtheirclientsabetterandmoreconsistentstreamingquality.Inotherwords,ratherthanconsideringthe

underlyingnetworkasablackboxorjustbunchofpipesthatcarrythebits,havingtheendpoints

(e.g.,originserversandclients)andmiddleboxes(e.g.,cacheservers)talktothenetwork,exchangeinformationandactaccordinglymaypotentiallyprovidemanybenefits.Forexample,thenetwork

itselfisfirsttoknowaboutacongestionorfailure.Suchinformationcertainlyhelpstheserversand

clientsadaptfasterandmoreaccurately.

Lastbutnotleastimportantopenissueiswhetherthenetworkcapacityinamulti‐accessnetwork

can sufficiently supportmany clients concurrently. In sucha scenario, each client competeswith

othersformorebandwidthandislikelyexperienceacontinualbitratefluctuation,whichadverselyimpacts quality of experience. Prepositioning the content or using multicast (especially for live

streaming)mayprovetobeuseful.

Inafuturearticle,wehopetoprovidetheresultsfromourongoingstudiesinvestigatingtheseopenissues.

Acknowledgments

WethankourcolleaguesatCiscofortheirreviewsandfeedbackforthisarticle.

9

References 

1.  Multicast  in  802.11  WLANs:  an  experimental  study.  Dujovne,  Diego  and  Turletti,  Thierry  . 

Torremolinos, Malaga, Spain : ACM, 2006. ACM MSWiM. 

2.  Understanding  and  Mitigating  the  Impact  of  RF  Interference  on  802.11  Networks.  Gummadi, 

Ramakrishna , et al., et al. Kyoto, Japan : ACM, 2007. ACM SIGCOMM. 

3.  Performance  Evaluation  of  Video  Streaming  with  Background  Traffic  over  IEEE  802.11  WLAN 

Networks. Cranley, Nicola and Davis, Mark . Montreal, QC, Canada : ACM, 2005. ACM WMuNeP. 

4. Protected Interoperable File Format. [Online] May 2010. http://go.microsoft.com/?linkid=9682897. 

5. Smooth Streaming Transport Protocol. [Online] Sept. 2009. http://go.microsoft.com/?linkid=9682896. 

6. HTTP Live Streaming. IETF Internet Draft. [Online] June 2010. http://tools.ietf.org/html/draft‐pantos‐

http‐live‐streaming. 

7.  Transparent  End‐to‐end  Packet‐switched  Streaming  Service  (PSS);  Protocols  and  Codecs.  3GPP  TS 

26.234. [Online] Oct. 2010. http://ftp.3gpp.org/specs/html‐info/26234.htm. 

8. An experimental evaluation of rate‐adaptation algorithms in adaptive streaming over HTTP. Akhshabi, 

Saamer, Begen, Ali C. and Dovrolis, Constantine. San Jose, CA : ACM, 2011. ACM MMSys. 

Biographies 

AliC.Begen(abegen@cisco.com)iswiththeVideoandContentPlatformsResearchandAdvanced

DevelopmentGroupatCisco.Hisinterestsincludenetworkedentertainment,Internetmultimedia,transport protocols, and content distribution. He is currentlyworking on architectures for next‐

generationvideotransportanddistributionoverIPnetworks,andheisanactivecontributorinthe

IETFintheseareas.HeholdsaPh.D.degree inelectricalandcomputerengineeringfromGeorgiaTech. He received the Best Student‐Paper Award at IEEE ICIP 2003, and the Most Cited Paper

AwardfromElsevierSignalProcessing:ImageCommunicationin2008.Heisalsoamemberofthe

ACM.

TankutAkgul(akgult@cisco.com)isasoftwareengineerintheServiceProviderVideoTechnology

Group at Cisco. His research interests are embedded software design, video compression and

multimedia streaming.He currently designs and develops software for the next‐generation IPTV

10

set‐topboxes.PriortoCisco,heworkedasaseniorscientistatSoftNetworks,LLCanddeveloped

several video codecs for Intel embedded SoCs. He received his Ph.D. degree in electrical andcomputerengineeringfromGeorgiaTech.HisPh.D.thesiswasondebuggingandreverseexecution

ofprogramsforembeddedsystems.Hehasseveralconferenceandjournalpublicationsinthearea

ofembeddedsoftware.

Mark Baugher (mbaugher@cisco.com) is a technical leader in the Research and Advanced

DevelopmentgroupatCisco.Prior toCisco,Markco‐foundedandservedas theCTOofPassedge

Inc.,whichdevelopedamulticastmediasecurityproduct.Overtheyears,MarkhasledteamsthatdeliveredinnovativetechnologiessuchasIntel'sPC‐RSVP,thefirstnetworkreservationsystemon

a PC, andLANServerUltimedia, IBM's firstmedia server product.Markhas co‐authored several

widely used international standards in the IETF and ISMA, and he is currently co‐chairing theInternet GatewayDeviceWorking Committee in the UPnP Forum.Mark holds anM.A. degree in

computersciencefromtheUniversityofTexasatAustin.