cloud streaming

Download Cloud Streaming

Post on 25-Feb-2016




0 download

Embed Size (px)


Cloud Streaming. Jingwen Wang. Video content distribution. Nearly 90% of all the consumer IP traffic is expected to consist of video content distribution Web video like YouTube, P2P video like BitTorrent Content distribution requirements: - PowerPoint PPT Presentation



Cloud StreamingJingwen WangVideo content distributionNearly 90% of all the consumer IP traffic is expected to consist of video content distributionWeb video like YouTube, P2P video like BitTorrent

Content distribution requirements:Scalable and secure media storage, processing and distributionAnytime, anywhere, any device consumptionLow latency, global distributionCloud Provides a Better wayMassive ScaleRapid File TransferLow IT CostsHigh ReliabilityAccredited SecurityCloudStreamMotivation:Current solution for deliver videos: progressive download via CDNNon-adaptive codecVideo freeezes

WANT: a SVC based video proxy that delivers high-quality Internet streaming adapting to variable conditionsVideo transcoding from original formats to SVCVideo streaming to different users under Internet dynamics

CloudStreamImplement on one processor:Video transcoding to SVC is highly complex and transcoding speed is relatively slow a long duration before a user can access the transcoded videovideo freezes because of unavailability of transcoded video data

To enable real-time transcoding and allow scalable support for multiple concurrent videos:Use Cloud: CloudStreamPartition a video into clips and maps them to different compute nodes in order to achieve encoding parallelization5 ConcernsEncoding parallelization:Multiple video clips can be mapped to compute nodes at different timeFirst-task first-server scheme can introduce unbalanced computation load transcoding jitterThe transcoding component should not speed up video encoding at the expense of degrading the encoded video quality

Streaming jitter:Video clips arrive at the streaming component in batchesDemand surge of network resources leads to some data not arrive at the user at the expected arrival time


Metrics affecting Streaming QualityStreaming Quality:Access timeTranscoding and streaming latenciesVideo freezesTranscoding and streaming jitters

Video Content:The temporal motion metric TMThe spatial detail metric SDEncoding ParallelizationSVC coding structure:A video non-overlapping coding-independent GOPsA picture layersA layer coding-independent slicesA slice macro-blocks

ParallelismAcross different compute nodes: inter-node parallelismShared-memory address parallelism inside on compute node: intra-node parallelismMulti-level parallelization SchemeMulti-level encoding parallelization:GOPs: have the largest work granularityInter-node parallelism !

Slices: independence, relative larger amount of workIntra-node parallelism!Each slice on a different CPU

Intra-node ParallelismIntra-node ParallelismLimit the average computation time spend over the GOP to an upper bound Tth Shorten the access time !The minimum number of slices encoded in parallel: Mmin

NotationsDefinitionsMNumber of encoded parallel slices in a pictureNMB, i, Nslice, i Number of MBs or slices in the i-th layer of a pictureTMB, i(M), Tslice, i(M)Average encoding time of one NM or slice in the i-th layer with M parallel slicesTpic, i(M)Average encoding time of the i-th layer of a pictureTpic(M)Average encoding time of a pictureTGOP(M)Average encoding time of a GOPInter-node ParallelismInter-node ParallelismAchieve real-time transcodingTranscoding jitters introduced by variation of GOP encoding timeGoal:Minimize transcoding jittersMinimize the number of compute nodesEstimation of GOPs Encoding TimeA multi-variable regression modelAt a given encoding configurationTrain videos with different video content characteristics TM and SD to build the regression model90% of predicted values of the testing data are fallen within the 10% of errorProblem FormulationProblem FormulationBased on the approximation of each GOPs encoding timeGiven Q jobsEach job i has a deadline di and a processing time piMultiple nodes in parallel, each job is processed with out preemption on each machine until its completionLateness li can be computed as ci (actual completion time) diUpper bound of lateness: WANT: bound the lateness of these jobs find the minimal number of machines N and minimize

Complexity: NP-hardSolution:Hallsh-based MappingLateness-first MappingHallsh-based MappingHallsh-based Mapping(HM): Set an upper bound of and find the minimal number of N satisfies itUse Hallsh machine scheduling algorithm as a blackboxminMS2approx algorithm1. Pick = mini{(di - pi)/}2. Run HallSh by increasing the number of machines until the maximum lateness among all jobs satisfies 87%System Architecture

Video request

Data downloadData store/cacheData transfer(high data rate)Check cacheComponent FunctionISP Proxy: receive & restrict requests in each ISPTask Manager: check cacheTask Dispatcher: load balanceDownloaders: download dataCloud Cache: store and upload dataHardware Composition Building Block# of serversMemoryStorageBandwidthISP Proxy68 GB250 GB1 Gbps (Intranet), 0.3 Gbps (Internet)Task Manager48 GB250 GB1 Gbps (Intranet)Task Dispatcher38 GB460 GB1 Gbps (Intranet)Downloaders1408 GB460 GB1 Gbps (Intranet), 0.325 Gbps (Internet)Cloud Cache400 chunk servers93 upload servers3 index servers8 GB4 TB (chunk server), 250GB (upload server)1 Gbps (Intranet), 0.3 Gbps (Internet)Cache Capacity Planning & Replacement StrategyHandel 0.22M daily requestsAverage video size: 379MBVideo cache duration: