BEing on Time Saves energY

The BETSY Framework

ΕΚΕΦΕ Δημόκριτος, 23/5/2007Δημήτριος Βογιατζής

2

Overview

1.Project overview

2.Summary

3.Project objectives

4.Methodology

5.Framework

6.Conclusions

3

BETSY strep project  Sep 2004 – Mar 2007

Participants

• NXP – Netherlands• CSEM - Switzerland• IMEC - Belgium• ISI - Greece• TUK - Germany• Siemens C-Lab - Germany• TU/e - Netherlands• University of Cyprus - Cyprus

4

Example of home network

5

Example of a hotspot

6

BETSY (BEing on Time Saves energY)

• BETSY aimed to deliver the theory, models & design methods to make

• trade-offs between – network & terminal resource consumption– power consumption of the terminal– timeliness of the streaming data

7

Summary

• BETSY manipulates of video streams on wireless hand-held devices such that – hand-held devices seamlessly adapt to fluctuating network

conditions and available terminal resources– energy consumption for processing the video is reduced

• True multi-media experience the device is required to– handle trade-offs between the use and consumption of

network and terminal resources such as bandwidth, CPU time, Buffer space, and power, and

– to guarantee to end-to-end timeliness for the streaming data.

• This leads to the following (next …)

8

Summary (cont.)

• Provide an integrated approach to real-time requirements for dynamic networked streaming systems

• Define a common resource model that can be used as an abstraction layer to hide lower level system parameters from higher level temporal descriptions and QoS strategies

• Understand the trade-off in energy consumption at the overall system level & balance energy consumption over different sources of energy

9

Trade off triangle

• Trade-offs between the use and consumption of network & terminal resources such as:– Bandwidth– CPU time– Buffer space– Power

• Guarantee end-to-end timeliness for streaming data

10

Methodology

• Design & reference implementation of an end-to-end quality-of-service framework

• Timing model for a top-down approach • Resource model to calculate the proper distribution of

computing resources: bandwidth & energy consumption

• Verify timing & resource model framework populated by selected components or modules– Use the framework’s mechanisms to adapt the processing

chain to changes in the resources• Framework & its components are implemented in a

streaming server and mobile clients– evaluation scenarios

11

BETSY functions

• Functions used by streaming applications

• Breeze::= is a piece of content, processed by a sequence of functions for processing, storing and communicating data items in an end-to-end delivery chain, on which only one entity is in control

12

BETSY functions II

• Capturing• Retrieving• Recording• Encoding• Decoding• Delay buffering• Rendering• Multiplexing• Demultiplexing• Transcoding• Transporting

13

Relations between BETSY functions and data types

•data types are represented as colored rectangles

•functions are represented by the rounded rectangles

•input and output data types with arrow from and to the data types

14

Energy Consumption Modelling

• Desired energy models– Desired breeze parametersenergy– for

• Separate functional components• Complete sub-breeze on one device

– Alternative• Parameters = f(lower level interm.) • Intermediate params = f(lower level parm.)

15

Battery model

Battery status

Battery

power

Remaing life time

16

Network Model

# streamsmax.packe

t size

Network

bandwidth

Bit rate per stream

17

FR FS IP QP

MPEG-4 Bit Rate

Bit rate

MPEG-4 Quality

PSNR

MPEG-4 stream Model

18

Scenario I (Evaluation)

19

Composed model I

FR FS IP QP

MPEG-4Bit Rate

MPEG-4QualityModel

Quality(PSNR)

Encode, Mux, Transport

Network

available bandwidth

requiredbandwidth

1 streammax.packe

t size

availablebandwidth

sufficient

20

Scenario II Evaluation

Breeze Media Center (PC)

DisplayWLAN-Camera(battery driven)

Battery Status,FR, FS

Change FR, FS

Access Point

Breeze’

Battery High/LowSwitch

CurrentMeter

PDA (GUI)

Current Param(Camera)

Requested Remaining Time(Camera-Param)

21

Composed Model II

# streams(2)

max.packet size

Network

raw bandwidth

bandwidth per stream

FRFSIPQP

MPEG-4Bit Rate

MPEG-4QualityModel

Quality

Encode, Mux, Transport

(Media Center)

bandwidthsufficient

Transport, Decode, Rendering

(PDA)

Power

Battery status

Battery

Remaing life time

22

Composition Rules

• To make trade-offs, – latency, energy, and quality models, have to be

combined into a single all-encompassing model – The parameters are key to combining the models– three independent models,

• Latency, Energy, & Quality, • each functional component, could be combined to a set

of independent models for the entire breeze

23

Compositions Rules

• Depending on the intended use of the model the required models of the parts can be composed into an end-to-end model

• Models represents a set of configurations or tuples of attributes • The essential ingredients of model composition are

– Cartesian product of tuples when two models are combined freely (without any constraints).

– combinations matching on selected attributed (like the join of a relational database). For instance all components have to work with the same stream and hence share the same values for the FR, FS, IP and QP parameters.

• IA so-called ’producer-consumer constraint’ expressing the matching connections between parameters of different models such as available bandwidth and required bandwidth (Pareto) optimization after combination and abstracting internal parameters can be used to reduce the set of candidate configurations

24

Composition Rules (cont.)

FR IPFS QP %I

LatencyModel

EnergyModel

QualityModel

Latency Energy Quality

FR IPFS QP %IFR IPFS QP %I

LatencyModel

EnergyModel

QualityModel

LatencyModel

EnergyModel

QualityModel

Latency Energy QualityLatency Energy Quality

25

Software framework

System Level

Subsystem / Device /

Application Level

Resource Level

Application Adaptor

Global Coordinator

Scheduler/AdaptorMonitor& Predictor

Application Coordinator

Monitor& Predictor

Resource Manager

OrderManager

Local Scheduler

Local Monitor

System Resource Manager

Local Resource Manager

Controller Predictor

Resource Control

Application Manager

Mode Manager

Quality Manager

Resource Manager

RCE Controller

GRACE MATRIX PCES OZONE

26

Software frameworks common characteristics

• Recognize the benefits – application level adaptation & resource level management – provide a sustained user-level quality of the multimedia flows

• Define a hierarchy of control elements based – structural and temporal scope differences of the controlled

elements• A root element with a global knowledge of the system status• Resource managers / brokers at the base, which

– receive resource allocation commands & enforcing them• Address QoS concepts at all domains,

– Resource-level QoS (for the network and the CPU resources) – Application / video QoS (with the exception of PCES for an

explicitly defined view of the latter)

27

Software frameworks differences

• Design level differences at, – structure, number, scope & specific behavior of

the intermediate control elements – between the root system-level manager &

resource brokers at the bottom of the hierarchy

• Engineering level differences at, – definition of the details of the interfaces of their

software components– realization of the inter-component

communication mechanisms, local or distributed,

28

Ozone Architecture

Application Manager

Mode Manager

Quality Manager

Resource Manager

RCE

RCE Control

RCE Operation

SF3SF2SF1

RCE OperationOne-to-One mapping

ApplicationManager : ResourceConsumerController

ModeManager : ResourceConsumerController

QualityManager : ResourceConsumerController

RCEControl : ResourceConsumerController

RCEOperation : StreamingFunction

ResourceManager : ResourceServiceController

: ResourceService

29

Distributed ozone architecture

Application Manager

Mode Manager

Terminal Quality Manager

Terminal Resource Manager

RCE

Subnet Resource Manager

NCE

Subnet Quality Manager Terminal Quality Manager

Terminal Resource Manager

RCE

TERMINAL1 TERMINAL2

30

Breeze Creation and Control Signaling (I)

• User instructs the system to create a new breeze from a source to a number of sink devices, streaming a selected content with a possible set of preferences (quality, duration).

• User interface translates the user’s input to a createBreeze() API signal, which actually transfers the request to a previously discovered BreezeManager in the distributed system.

UI BMcreateBreeze(src, dests[], content, pref)

31

Breeze Creation and Control Signaling (II)

• BreezeManager according to its global view of the system and its configuration decides on the acceptance of the breeze starts the creation of the configured elements in the appropriate devices (createElement() signal), connects them (connectElements() signal) according to the system configuration and returns to the user interface a global handle of the breeze for breeze handling (play(), pause(), stop() and destroy() signals).

BM BMCreate(), Connect(), Set()/Get()/Subscribe()

BMX

32

Breeze Creation and Control Signaling (III)

• Each element in the control / stream chain of the breeze, on reception of a connect() signal from the BreezeManager, invokes its own startup signals which are mainly a number of get()/set() and subscribe() API calls.

• The control policy which the whole chain implements is then continuously running through the exchange of variable change events and set() / get() signals.

Controller Function

Set() / Get()

Event()

33

Breeze Control

Controller

Function Resource mappings

Stream Data

Function

Resource

RService

Control Data

34

Core Component Libraries

• Components built in the framework:

– BreezeChain and StreamingFunction interfaces for :• the VLC streaming framework (www.videolan.org) • the AXIS camera

– Resource and ResourceService interfaces for:• the CPU and the OS scheduler• the network interface and the protocol stack• the memory and the stream buffers• the battery and the power consumption

– Access and protocol interfaces for:• UPnP discovery, signaling and control• Raw TCP/UDP signaling and control

35

System-level Component Libraries

• Components built with the framework for demonstration, validation and testing reasons:– A typical breeze manager– A set of breeze controllers implementing various control

policies– A pareto modeling element– A set of low level resource modeling elements– A main user interface controller for device discovery,

enumeration and breeze management– A resource monitoring infrastructure and display– A resource knob monitoring and control panel– A breeze knob monitoring and control panel

36

Main user interface

• Enumerate existing devices, breezes and elements• Create and destroy breezes

37

Breeze knob panel

• Monitor controller decisions and breeze adaptations• Freely control any of the available knobs

38

Breeze knob panel

• Monitor controller decisions and breeze adaptations• Freely control any of the available knobs

39

Resource knob panel

• Monitor resource status, controller decisions and resource adaptations

• Freely control any of the available knobs

40

Resource consumption panel

• Monitor resource consumption status as a result of controller decisions and various knob adaptations

41

Controller Example

• Controller (CTLS1) connected to the wireless LAN concrete resource interface (input), to a Pareto modeling element (ParetoM1) and to the encoding function of the source breeze chain (output)

• Based on the existence of an automatic rate fallback WLAN driver

ControllerParetoM1

Encode

WLAN

42

Controller Example

• Signaling of CTLS1:– Receives an event for each raw bandwidth change, which actually captures

a very fast change in the link quality.– Consults the pareto model to get the best configuration set for the encoder

(FS, FR, IP, QP)– Adapts the encoding function by applying the new configuration set

ControllerParetoM1

Encode

WLAN

1

23

43

Framework overview

• software framework for QoS and resource control in and end-to-end stream delivery chain

• Provides the infrastructure for implementations of functional entities/devices with the necessary interfaces to control the streaming and device resource parameters

• Provides the means for end to end performance and resource consumption assessments for different trade-off handling and control policies

• Abstracts the main building components needed by the control policies making extensive reuse possible and guaranteeing interoperability between devices of different vendors

• Raises the domain of the problem hiding all technical details of distribution and inter-component communication, allowing the engineer to concentrate on the control policy and the trade-offs under study

44

Thank you

45

Wireless Network Savings

• Resource consumption cost for each resource parameter over each resource

• Important: study resource consumption behaviour of the transport function

• Time, Energy: Primary resources – <= abstract resources: processing, storage,

bandwidth

• We have energy costs of a stream transition= f(costs of protocol processing, data packetisation, transmission)

46

Configurations, 100 kbit/s

([IP], [QP], [FS], [FR], PSNR, bit rate)

([IP8], [QP10], [QCIF], [12.5], 26.1, 24.9), ([IP16], [QP5], [QCIF], [12.5], 26.8, 47.7), ([IP16], [QP10], [CIF], [25], 28.6, 95.6), highest PSNR([IP16], [QP10], [QCIF], [12.5], 26.1, 18.4), ([IP16], [QP15], [CIF], [25], 27.8, 67.8), ([IP16], [QP15], [CIF], [12.5], 26.2, 41.3), ([IP16], [QP15], [QCIF], [12.5], 25.6, 12.3), ([IP32], [QP5], [QCIF], [12.5], 26.8, 41.3), ([IP32], [QP10], [CIF], [25], 28.5, 77.6), ([IP32], [QP10], [QCIF], [12.5], 26.1, 15.2), ([IP32], [QP10], [QCIF], [5], 24.3, 9.3), ([IP32], [QP15], [CIF], [25], 27.8, 54.8), ([IP32], [QP15], [CIF], [12.5], 26.2, 35.0), ([IP32], [QP15], [QCIF], [12.5], 25.6, 10.1), ([IP32], [QP15], [QCIF], [5], 24.1, 6.08)}

47

Overview of Video Coding

I-frames (Intra): exploit spatial correlation within the frameP-frames (Predictive): temporal correlation (prediction from previous frames) Higher compression efficiency

48

Impact of Intra/Predictive on Error Propagation

Concealment of lost data: for example copy from previous frame

Error propagates as later frames predict from wrong data

Stop error propagation by inserting Intra information (no reference to previous frames)Lower compression efficiency of Intra -> Bit rate overhead

Tradeoff error robustness and coding efficiency

49

Intra Frame insertion vs Gradual Intra Refreshment

Using Periodical Intra frames

Intra Period T = 6

Updating in every frame an Intra portion %Intra = 16%

%Intra information has a big impact on the Quality-Rate modeling under network errors