1 tm8106 optical networking energy consumption in optical networks by ameen chilwan syllabus: [1]...

1

TM8106Optical Networking

Energy Consumption in Optical Networks

By

Ameen Chilwan

Syllabus:

[1] Energy Efficiency For Network Equipment: Two Steps Beyond Greenwashing, Juniper Networks, Whitepaper, 2009.

[2] Network and Telecom Equipment – Energy and Performance Assessment, Test Procedure and Measurement Methodology, ECR Initiative, Draft 2.1.1, October 04, 2009.

2

Paper # 1

Energy Efficiency For Network Equipment: Two Steps Beyond Greenwashing.

13.09.2012TM8106 Optical Networking - Green Networking

3

Introduction• Attention drawn to energy footprint of data networking

Due to rising energy costs and environmental standards

• Missing verifiable data to support “green” claim In this paper

• Discuss practical aspects of energy efficiency measurement• Define efficiency metric related to equipment energy consumption

• Reducing energy-related office & transport expenses Fairly straight-forward Reduce the operation cost but has a high initial expenditure

• Determines the right point to invest in energy-saving technologies

• Current state-of-the-art in network industry No firm numbers or commitments, but an assorted collection of

power management technologies Not so simple, depends upon many parameters


4

Basics of Moving Data

• “An arbitrary message can be represented via units of informational entropy” Modern computing and communication use “bits” for this

• Energy consumption in networking equipment Due to loss during transfer of electric charges

• Caused by imperfect conductors and isolators

Depends upon technology, frequency of transitions, no. of gates• These details are not very interesting to end-users

Instead, a list of features required is maintained, containing:• Minimum packet processing requirements• Scaling parameters (bandwidth, queues, other revenue-generating)• Packaging format• Relative cost


5

Step 1: Efficiency Criteria

• In theory, to compare energy consumption: Sufficient to put two or more devices under the same load and

measure their respective power draw In reality, this is rarely possible

• Requires non-trivial investment in test gear• Different devices have different capabilities => not comparable

To define efficiency metric, we normalize energy consumption (E) by effective full-duplex throughput (T)

• , ECR is Energy Consumption Rate [watts/Gbps]• E & T come from tests or vendor data, has to be verifiable• ECR is amount of energy (in Joules) required to move an array of

data (in bits) across the device.• ECR is a peak metric

– Highest performance capacity of the device


6

ECR Metric• Valid differentiator within product class

Equipment with lower ECR use less energy to transfer same amount of data

• Unlike light-bulbs, system capacity must be a factor into efficiency estimates Growing capacity requirement may or may not fit energy

budget

• Provides unified way of testing Can be also be used for defining long-term R&D goals Also mean, specific setting for each customer may not be met

• Customers needs 600 Gbps, ECR reported for 1.6 Tbps

Relative ECR standings remain same across wide config range• Assumption that provides compromise between custom and

standard settings• Expectation: standard ECR rating will be adequate energy

performance estimate 13.09.2012TM8106 Optical Networking - Green Networking

7

Decoding Vendor Datasheets – “T”• Parameters E & T to be collected by same methodology

So, even if ECR not reported by vendor should be possible to be approximated from publicly available data (datasheets, tests)

• Platform Throughput Amount of data platform can process per second Vendors often report platform capacity, not throughput

• Capacity is half-duplex bandwidth in Gbps• Has to be divided by two to get “T”

– Not always the case

• Some vendors report capacity as theoretical peak utilization based on single isolated metric

– This neglects limitations like: line cards that can’t use their whole BW, inter- slot restrictions and other limits

• For ECR, “T” should include only effective capabilities Calculated as sum of capabilities of all the line cards in mesh


8

Decoding Vendor Datasheets – “E”• Several energy metrics in datasheets

Power System Rating [watts or amps] (aka Agent Label)• Site preparation requirements recommended by vendor• Actual consumption maybe fraction of what power supplies deliver

Maximum Energy Consumption [watts]• Vendor’s estimate for highest configuration footprint• Can serve as an upper boundary estimate for power draw• Max energy consumption estimate changes with replacing modules

Component-based Consumption Estimate [watts]• Adding power draws for configured components to make estimate• Arithmetic sum of components ratings never represents real system

Typical (Average) Power Draw [watts]• Vendors are free to report this metric with underpowered config,

load profiles that yield best results and so on• Without disclosure of conditions, can’t be used for comparison


9

Advanced Topics• ECR represents performance of fully-configured system

running at maximum load• Platform Modularity (Partial configuration)

Energy consumption of modular router/switch is sum of:• Fixed parts “F” (chassis, host system, fabric, clocking)• Variable parts “V” (line cards, ports, physical line drivers)• E = F + V• More efficient in full config are more efficient in partial config too

– If not, cross-over point can be found

• Absolute Energy Footprint Relative standings not changed in partial config, but interesting

to know how much energy a platform will consume• Component-based energy calculators provided by vendors• ECR method can be adapted to collect and report realistic

component footprints in addition to “ratings”13.09.2012TM8106 Optical Networking - Green Networking

10

Advanced Topics (continued…)

• Energy Efficiency and Time Scales When network not fully loaded, it has on- and off- peak periods Dynamic power management require measuring energy

consumption under variable load Thus ECR is complemented with Energy Efficiency Rate (EER)

• , Ef=full load, Eh=half load, Ei=idle• EER is synthetic and stimulates in area of Barroso’s principle of

load proportionality• Any elasticity in energy consumption should not involve pkt drop• Any state change should happen automatically and in real-time• Extended Idle conditions may happen (weekends, day/night cycle)

– Maybe useful a device state transition to standby with reduced capacity, i.e. small energy footprint

– Transition between active and standby required not to be instant


11

Advanced Topics (continued…)

• Metric Accuracy and Practical Impact Accuracy of ECR and EER reflects averaging and fluctuations

• Averaging relates to sampling during test runs• Fluctuations relates to technological deviations• suggested accuracy within range ± 2.5%• Practical ECR/EER advantage can be within 10% or higher• Two ways to express ECR diversity

– Monetary difference: related to cost of ownership• A datacenter consumes 100 kW for 5 years, ECR difference of 10% is

438,300 kWh (24*365.25*5) excluding cooling & power conversion

• Consider fully burdened consumption, 1.5-3 times of input, 2x overhead, a cost of $ 0.10 and 10% yearly rate increase will give $107,000 for 10% ECR improvement, 200k for 20% and 500k for 50%

– Environmental Impact: following Kyoto Protocol (CO2 reduction)• ECR forms basis for equipment selection criteria

• Choosing equipment with 10% ECR advantage can mean 1 year advantage


12

Step 2: Design Goals• Energy related improvements in equipment design are:

Organic (passive)• Inline with Dennard’s scaling law: every new generation of network

silicon packs more performance in smaller energy budget

Engineered: active energy management, including:• Idle state logic, gate count optimization, memory access algorithm

Some enhancements of both are in step with building networks• E.g. better density, integration and heat management

On the other hand, dynamic power management proportionate to an instant load is a technology with no effect on e.g. capacity

• But such technologies form a pool that may improve energy efficiency at pace even faster than Moore’s Law

• Return on Investment is not always material Being a pioneer is also expensive and challenging


13

Step 2: Design Goals (continued…)• Network Design

Guidelines for improved results during network design• Select equipment with best ECR/EER rating• Plan for good system & network utilization• Include rack space, cooling and power conversion in assessment• Do not multiply entities beyond necessity• Consider local energy costs and trends

• Conclusion Modern networks growing faster than Moore’s Law

• Offset by reduction in areas like commuting, offline shopping, offline banking and goods manufacturing

• Net-centric world: better life quality with less material footprint• So, need more environmental friendly equipment to build networks• Two much-needed steps are: standardize network efficiency, and

rise of sustainable network technologies13.09.2012TM8106 Optical Networking - Green Networking

14

Paper # 2

Network and Telecom Equipment –

Energy and Performance Assessment,

Test Procedure and Measurement Methodology.


15

Beginning• Purpose

Framework for first-order energy efficiency approximation for• Packet-based network• Telecom equipment

Covers• Peak efficiency, Variable-load efficiency and Idle (statically

configurable) energy efficiency

• Theoretical Basis Energy Efficiency = Energy Consumption / Effective Throughput Basic energy efficiency test: measurement alongside EER

• Amount of actual data equal or less than theoretical load

Additional tests for properties like:• Idle (static) energy management, energy management for

connected (cascaded) devices and embedded energy monitoring


16

Scope• Efficiency definition suited

For medium to large scale network systems Less for small office, home, consumer grade devices

• TP less relevant• Efficiency metrics can be based on allowances per unit functionality

• Tests in this paper applicable to Core & edge routers, L2/L3 switches, optical packet shelves etc Anything with performance numbers less than face value of

connected ports Can also be adopted for

• equipment with face value of ports same as effective bandwidth (TDM systems, optical converters)

• Bandwidth is not a measure of load (control planes, AUTH servers)– Other metric to measure load, e.g. CPU Utilization


17

Test Procedure 1

Energy consumption in relation to dynamically changing load


18

Test Procedure 1

• Packet-based systems based on statistical MUXing May or may not correspond to theoretical BW as on port face

• Simultaneous performance and energy consumption measurements Under load profile and conditions typical to environment where

system under test (SUT) is intended operate

• Class-specific requirements in Appendix B• SUT Preparation

Configured according to class requirements Exposed to conditions as in App. A to settle potential

temperature and humidity differences prior to test Router testing equipment: to simulate load and collect data AC or DC inline meters to calculate energy consumption


19

Test 1 – Steps

• Step 1 (Qualification) Determine maximum Load (Lmax) sustain SUT at zero pkt loss

Any methodology can be used: binary search, heuristics etc No time limit for this run

• The following runs separated with idle time ≤ 300sec• Step 2 (full load)

Offer Lmax to SUT for 1200 sec, avg consumption E100 found

• Step 3 (half load) (can be automated, reset without pkt loss) Reduce load to Lhalf (=0.5 x Lmax) and run for 1200 sec

Calculate E50 which is avg energy consumption in this period

Load reduced by reducing pkt rate on all configured ports• Not by idling or disconnecting ports

• Packet loss resets the test (step1), reset to Lmax should be possible13.09.2012TM8106 Optical Networking - Green Networking

20

Test 1 – Steps (continued…)

• Step 4 (30% load) (can be automated, reset without pkt loss) Reduce load to L30 (=0.3 x Lmax) and run for 1200 sec


Load reduced by reducing pkt rate on all configured ports• Not by idling or disconnecting ports• Packet loss resets the test, reset to any load should be possible

• Step 5 (10% load) (can be automated, reset without pkt loss) Reduce load to L10 (=0.1 x Lmax) and run for 1200 sec


Load reduced by reducing pkt rate on all configured ports• Not by idling or disconnecting ports• Packet loss resets the test, reset to any load should be possible


21

Test 1 – Steps (continued…)

• Step 6 (idle run) Remove all the load and run for 1200 sec

Calculate Ei which is avg energy consumption in this period

Load removal by idling pkt rate on all configured ports• Not by disconnecting or shutting down ports• Packet loss resets the test, reset to any load should be possible


22

Test Procedure 2 (Optional)

Energy consumption in relation to statically changing load


23

Test Procedure 2

• Due to extended periods of low utilization• Demonstrate energy saving potential• Pre-requisite is to have Lmax from test 1

• Configuration, preparation, requirement same as test 1• The following runs separated with idle time ≤ 300sec• Steps

Step 1 (half ports in use)• Reduce to Lhalf (= 0.5 x Lmax) and run for 1200 sec

• Measure energy for entire period and find avg P50

• Load reduction by sending traffic at full rate to every second port– Active ports should be evenly mixed with inactive

– Block designation (1 line card with all active, other none) not allowed

• Packet loss resets to test 1, reset to any higher level not required13.09.2012TM8106 Optical Networking - Green Networking

24

Test 2 (continued…)• Steps

Step 2 (one quarter of all ports in use)• Reduce to L25 (= 0.25 x Lmax) and run for 1200 sec


• Load reduction by sending traffic at full rate to every fourth port– Active ports should be evenly mixed with inactive


• Packet loss resets to test 1, reset to any higher level not required

Step 3 (one-tenth ports in use)• Reduce to L10 (= 0.1 x Lmax) and run for 1200 sec


• Load reduction by sending traffic at full rate to every tenth port– Active ports should be evenly mixed with inactive


• Packet loss resets to test 1, reset to any higher level not required13.09.2012TM8106 Optical Networking - Green Networking

25

Test Procedure 3 (Optional)

Component level energy footprint (Fc)


26

Test Procedure 3• A number of configurations for networks are possible

Impractical to test all of them Perform detailed energy and performance analysis for some Approximate for the rest of configurations by sum of

component-level energy budgets• Less precise compared to actual measurements• Trade-off between operational cost and precise metric values

• Vendor builds and publishes Library of component-level energy consumption

• For power budget estimations• Suitable for arbitrary configurations

• Detailed measurements not applicable Concentrates on peak energy performance only


27

Test 3 – Steps• SUT Preparation

Configured according to class requirements including CUT SUT neither required to be fully loaded nor to bear all modules CUT that energy and traffic impact of CUT is minimum

• Step 1 (Qualification) Determine maximum Load (Lmax+) sustain SUT at zero pkt loss

Design Lmax+ to exercise all components to highest possible TP

Any methodology can be used: binary search, heuristics etc No time limit for this run until max load is determined

• The following runs separated with idle time ≤ 300sec• Step 2 (full load)

Offer Lmax+ to SUT for 1200 sec, measure energy for the period

average footprint F+ is calculated

Packet loss resets the test to step 1 13.09.2012TM8106 Optical Networking - Green Networking

28

Test 3 – Steps (continued…)• Step 3 (CUT Removal)

Remove CUT and readjust SUT to compensate

Determine new max load (Lmax-) sustain SUT at zero pkt loss

Design Lmax- to exercise all components to highest possible TP

• Step 4 (final measurement) Offer Lmax- to SUT for 1200 sec, measure energy for the period

average footprint F- is calculated

Packet loss resets the test to step 1 Energy footprint of CUT is approximated

• Fc = F+ - F-

• Steps 1-4 repeated as many times as needed• Not applicable for non-redundant components

Only for some components as system chassis and backplane13.09.2012TM8106 Optical Networking - Green Networking

29

Test Procedure 4 (Optional Functional Test)

Embedded energy monitoring capabilities


30

Test Procedure 4

• Real-time energy consumption estimates pivotal to design and monitor energy conversation policies

• Device-level granularity energy utilization reports not provided by power facilities

• Networks to monitor their own energy consumption by using sensors and embedded probes

• Useful to report in parallel with system utilization Allows for traffic affinity and energy pattern analysis

• This test can be done in parallel with Test 1 and test 2


31

Test 4 – Steps• Step 1 (Qualification) (can be combined with test1 step1)

Same as for test1 step1, Lmax is found

• Step 2 (full load) (can be combined with test1 step2)

Offer Lmax to SUT for 1200 sec, avg consumption E100 found

Average recorded energy consumption for the period R100 is read from the SUT

Optional: Avg system utilization is recorded U100

• Step 3 (half load) (can be combined with test2 step1)

Reduce to Lhalf (= 0.5 x Lmax) and run for 1200 sec

Measure energy for entire period and find avg P50, R50 and U50

Load reduction by sending traffic at full rate to every 2nd port• Active ports should be evenly mixed with inactive• Block designation not allowed• Packet loss resets to test 1, reset to any higher level not required13.09.2012TM8106 Optical Networking - Green Networking

32

Test 4 – Steps (continued...)• Step 4 (25% load) (can be combined with test2 step2)

Reduce to L25 (= 0.25 x Lmax) and run for 1200 sec


Load reduction by sending traffic at full rate to every 4th port• Active ports should be evenly mixed with inactive• Block designation not allowed• Packet loss resets to test 1, reset to any higher level not required

• Step 5 (1/10th ports in use) (can be combined with test2 step3)

Reduce to L10 (= 0.25 x Lmax) and run for 1200 sec


Load reduction by sending traffic at full rate to every 10th port• Active ports should be evenly mixed with inactive• Block designation not allowed• Packet loss resets to test 1, reset to any higher level not required13.09.2012TM8106 Optical Networking - Green Networking

33

Test Procedure 5 (Optional Functional Test)

Collateral energy management


34

Test Procedure 5

• Devices can have impact on energy consumption in connected (cascaded) devices

• Examples include: Ability to control power states in Power over Ethernet (PoE) Ability to assist power states in LAN-connected devices Ability to control IP and non-IP devices via remote control

mechanisms


35

Effective TP Calculation

• Lmax converted into full-duplex TP Tf (in Gbps) To normalize SUT energy consumption to performance

• First method: Lmax reported by traffic generator as a combination of egress

packet-per-second rate and packet size compared to the load.• For variable pkt size: average proportion is computed

All applicable minimum L2 & L1 overhead added to compute effective wire rate

Example:• SUT is eth switch, 10 x 10 GbE ports @ 7,291,702 fps, 64 B/frame• Implies 7,291,702 pps

• Tf = 10 x 7,291,702 x 8 x (64 + 1 + 7 + 12) = 49.000237440 Gbps

– (1 + 7 + 12): accounts for eth start of frame, preamble, min interpkt gap


36

Effective TP Calculation (continued…)• Second method:

Lmax reported tester equipment itself as highest achieved utlization on per-port basis (in percentage)

Line rates multiplied by port utilization to get final data rate Example:

• SUT is MPLS edge platform, 10x10 GbE ports each on core & access• Frames forwarded towards core with 100% utilization• Frames forwarded towards access with 99.22% utilization• Data rate for 10 GbE is 10,000 Mbps

• Tf = 10 x 10,000 x 1.0 + 10 x 10,000 x 0.9922 = 199.22 Gbps

Example:• SUT is eth switch with 8 GbE ports opearting at

– 100% line utilization when configured for VLAN

– 90% line utilization when not configured for VLAN

– Result from 2nd case is used as eth switches don’t need VLAN headers13.09.2012TM8106 Optical Networking - Green Networking

37

Results Representation• Array of results obtained from test 1-4 form energy “passport” of the SUT

and can be used for evaluation and energy planning purposes

• Base Metrics and Device Comparisons Most straight-forward metric is to normalize energy

consumption to the highest sustained throughput recorded [W/Gbps]

• Unlikely to obtain efficiency numbers over sustained intervals– Due to off- (low utilization) and on- (burst) peak periods that cause

provider to size on the higher end of traffic profile and lose energy efficiency during off-peak times

• Instead, providers can optimize efficiency using middle of their load band

[W/Gbps]• ECR-VL is the measure of dynamic energy management• ECR-VL rating should be close to ECR following Barroso’s principle


38

Energy Bill Estimates• Cost of operation over projected lifetime

• N is projected no. of years, CkWhj is cost of kWh in year of operation

This method gives estimate based on automatic (variable-load) energy management capabilities of reference system

Modest, but provide lossless operation under all conditions

• Alternative cost estimate Build upon static (idle-load) energy saving capabilities of device Example: a switch downgraded to 25% at night and 10% on

Sun.

• Both systems are complementary & work on different time intervals

Relative slow configuration change, not applicable to bursty env. Requires personnel to take care of policy-driven degradation


39

Site Planning

• Providers use Agency Labels provided by vendors Ill-suited because call for energy reservation at highest end Makes it confusing whether device will work in case on-site

power rating is less that this value

• Useful resources for site planning Value E100

• describes average SUT energy consumption under highest possible load

• Provides upper boundary for energy requirements

If E100 not provided

• can be approximated from sum of required components• Provided that component footprint Fci were published


40

Functional Compliance

• Tests 4&5 are intended for functional compliance testing Only pass/fail criteria

In test 4, externally measured consumption values (E100, P50, P25, P10) should be compared to (R100, R50, R25, R10) reported by SUT

Adjusted result should be in acceptable range of effective consumption (±10% or less)

Upon vendor’s approval an offset R can be added to compensate electronics not covered by embedded power monitoring circuitry

Externally recorded load L should correlate to utilization U• Where Lmax = 100% SUT Utilization

• Static offset can also be added in this case, upon vendor’s approval

Knowledge of system w.r.t time crucial for planning static energy management routines

Test 5 describes value-added functionality that might be in SUT13.09.2012TM8106 Optical Networking - Green Networking

41

Reporting Format

• Results shall be fully reproducible by any laboratory Thus has to include at least the following documentation

• All SUT software versions, hardware board revisions and device configurations used during the test.

• All commands applied for static config to SUT or run-time queries• Traffic generator tool passports, actual voltage in power feeds and

ambient (environmental) conditions at test side• The test setup should fully described, including topology, the

choice of offered load structure and test actions within a range of possible choices.


42 13.09.2012TM8106 Optical Networking - Green Networking

1 tm8106 optical networking energy consumption in optical networks by ameen chilwan syllabus: [1]...

Documents

networking equipment

equipment energy consumption

telecom equipment energy

rising energy costs

fit energy budget

network equipment

optical networks

data platform