can we benefit from solid state drives in rich multimedia content processing, storage and streaming?

.

The Big Picture: YALMS

M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 2/17...

2/17

.

YALMS: Yet Another LMS• feature: input, processing, and delivery in equal shares 17

Step 1: Record and Stream

V A

S Creator

Viewers

Content Server

HTML5

HTML5 Muxer

Record Screen Streaming

Step 2: Annotate

Lecturer

S

Auto- annotate

C

17 M.Zhanikeev+1 "YALMS: Yet Another Lecture Management System..." IEICE・ET研 (July 2013)M.Zhanikeev -- [email protected] Can We Benefit from SSDs in Rich Multimedia...? -- http://tinyurl.com/kyutech131024 3/17...

3/17

.

A Business Day at YALMS

• many screen and video streams uploaded by users 15

• many concurrent video annotation tasks (PDF + VIDEO) 16

• constantly changing popularity distribution of content, needs

dynamic relocations

15 "YALMS Project Home" https://github.com/maratishe/yalms (June 2013)

16 "YALMS Annotation Robot (software behind this paper)" https://github.com/maratishe/yalms.annotation (September 2013)


4/17

.

YALMS Topics


5/17

.

YALMS: The Big SSD Question

.The Question is..

.... can such a rich multimedia architecture benefit fromSSDs?• WHY1: SSDs have properties different from those of HDDs

◦ read/write, random/sequential

• WHY2: SSDs are becoming cheaper (MLC) so why not to use them?

• WHY3: parallel/multicore processing is not kind to HDDs but SSDscan cope

• WHY4: 50Mbytes/s throughput is not enough any more


6/17

.

Where Are We Now? ExistingResearch


7/17

.

Existing Research (1)• roughly, two groups:

1. SSD versus HDD research• filesystem abstraction and scheduling -- load balancing between SSD and HDD 01• SSD as cache memory 0203

2. parallel processing and themulticore• job graph partitioning 18• local Big Data processing 19• multicore streaming 11-- 14

01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)

02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)

03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)

18 X.Sui+3 "Parallel Graph Partitioning on Multicore Architectures" 23rd LCPC (2010)

19 R.Chen+2 "Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications on Multicore..." 19th PACT (2010)

11 Y.Wei+3 "Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-Core Processors" ACM SAC (2010)

14 V.Galiano+3 "SSD architecture such as multi-channel and multi-way" EURASIP Journal ... Signal Processing (2013)


8/17

.

Existing Research: What is Missing?

.In a Nutshell, Existing Research is.....

.

• extremely specialized and is built for one application only

• ... or, in business, the use is very primitive -- a temporary memory buffer

• ... at the same time.. YALMS needs generic load balancing◦ one design fits all

• changes along the timeline should cause changes -- dynamic optimization• ...


9/17

.

The YALMS.hiperf project


10/17

.

Meet the YALMLS.hiperf Project

Multimedia System

Creators

Viewers

Manage

HDDs HDDs HDDs HDDs HDDs SSDs

Cores Cores Cores

• a pool of SSDs,HDDs, etc.

• a separate pool ofCPU cores

• steady arrival anddeparture rates oftasks/processes


11/17

.

YALMS.hiperf : The Overall Design

YALMS.hiperf

YALMS.hiperfRobot

A Multimedia System

ProcessProcessProcessProcessProcessProcessProcessReq(details)

Rep(agent)

AgentsAgentsAgentsAgentsAgentsHDDHDDHDDHDDHDDs

SSDsSSDsSSDsSSDsSSDsSSDs ….

• a firmware approach

• tasks/processes sendrequests

• replies point to agents


12/17

.

YALMS.hiperf : The Basic Unit

…

Proc

essin

g

logic

SSD

HDD

ENGINE

Network storage

Input Next step

Read/write? Seq./rand.? (Chunk) size

HiPerf/Normal? Longer plans?

Multicore

Fork Join

• traditional parallelprocessingformulation

• engine performsruntimeoptimization

• tasks are assignedclasses

01 J.Kim+4 "Disk Schedulers for Solid State Drives" 7th ACM EMSOFT (2009)

02 S.Rizvi+1 "Flash SSD vs HDD: High Performance .... Multimedia Storage Systems" 2nd ICCET (2010)


13/17

.

YALMS.hiperf : Runtime Optimization.Basic Formulation..

.

start from zero, collect history, thenoptimize accordingly • tasks should naturally have the few

heavy hitters distribution◦ if not, tasks should be partitioned

• then simply

minimize∑

i∈tasks

∑j∈time

lagi,j (1)

• ... details in future publications

03 Y.Lee+1 "Mapping granularity and performance tradeoffs for solid state drive" The Journal of Supercomputing (2013)


14/17

.

Wrapup


15/17

.

How Far is The Project?• software for multicore video annotation on top of SSD

◦ mostly reads, very few writes

• currently working on the generic version of software, will release soon

screen video

SlidesPDF

LaTeX/BeamerKeynote

PowerPoint…

VLCffmpeg

I-Frames I-Frames

pages PNGs

ANNOTATION

slices

PNGstimestamps

End productscenes cleanup

splitpdf2png


16/17

.

That’s all, thank you ...


17/17

.

[01] J.Kim+4 (2009)Disk Schedulers for Solid State Drives7th ACM EMSOFT

[02] S.Rizvi+1 (2010)Flash SSD vs HDD: High Performance .... Multimedia Storage Systems2nd ICCET

[03] Y.Lee+1 (2013)Mapping granularity and performance tradeoffs for solid state driveThe Journal of Supercomputing

[04] (2012)Data Compression in the Intel Solid-State Drive 520 SeriesIntel Technology Brief

[05] (2013)Cisco Content Delivery Engine 250Cisco Data Sheet


17/17

.

[06] B.Sajadi+3 (2011)...SSDs for Large-Scale Interactive Graphics ApplicationsSymp. on 3D Graphics and Games (I3D)

[07] D.Zhao+1 (2013)HyCache: a User-Level Caching Middleware for Distributed File SystemsACM HPDC

[08] M.Ryu+2 (2011)Impact of Flash Memory on Video-on-Demand Storage: Analysis of Tradeoffs2nd ACM MMSys

[09] (2013)Performance ... On DemandConcurrent Computer Corporation (STEC), Case Study

[10] M.Ryu+2 (2012)Why ...flash-based multi-tiered storage systems performing poorly for HTTP videostreaming?22nd NOSSDAV


17/17

.

[11] Y.Wei+3 (2010)Energy-Efficient Real-Time Scheduling of Multimedia Tasks on Multi-CoreProcessorsACM SAC

[12] S.Sankaraiah+3 (2011)GOP Level Parallelism on H.264 Video Encoder for Multicore ArchitectureIPCSIT

[13] T.Dias+2 (2010)H.264/AVC framework for multi-core embedded video encodersSoC

[14] V.Galiano+3 (2013)SSD architecture such as multi-channel and multi-wayEURASIP Journal ... Signal Processing

[15] (June 2013)YALMS Project Homehttps://github.com/maratishe/yalms


17/17

.

[16] (September 2013)YALMS Annotation Robot (software behind this paper)https://github.com/maratishe/yalms.annotation

[17] M.Zhanikeev+1 (July 2013)YALMS: Yet Another Lecture Management System...IEICE・ET研

[18] X.Sui+3 (2010)Parallel Graph Partitioning on Multicore Architectures23rd LCPC

[19] R.Chen+2 (2010)Tiled-MapReduce: Optimizing Resource Usages of Data-parallel Applications onMulticore...19th PACT


17/17

can we benefit from solid state drives in rich multimedia content processing, storage and streaming?

Technology

rich multimedia

multimedia systemyalms

hiperf project pool

multimedia storage systems

yalms screen

yalms topicsm

flash ssd vs hdd

hiperf projectm