energy profiles of java collections...

12
Energy Profiles of Java Collections Classes Samir Hasan Computer Science and Software Engineering Auburn University [email protected] Zachary King Computer Science and Software Engineering Auburn University [email protected] Munawar Hafiz Computer Science and Software Engineering Auburn University [email protected] Mohammed Sayagh MCIS Polytechnique Montreal [email protected] Bram Adams MCIS Polytechnique Montreal [email protected] Abram Hindle Dept. Of Computing Science University of Alberta [email protected] ABSTRACT We created detailed profiles of the energy consumed by com- mon operations done on Java List, Map, and Set abstrac- tions. The results show that the alternative data types for these abstractions dier significantly in terms of energy con- sumption depending on the operations. For example, an Ar- rayList consumes less energy than a LinkedList if items are inserted at the middle or at the end, but consumes more energy than a LinkedList if items are inserted at the start of the list. To explain the results, we explored memory usage and bytecode executed during an operation. Expensive com- putation tasks in bytecode traces appeared to have energy impact, but memory usage did not contribute. We evaluated our profiles by using them to selectively replace Collections types used in six applications and libraries. We found that choosing a wrong Collections type, as indicated by our pro- files, can cost as much as 300% more energy than the more ecient choice. Our work shows that the usage context of a data structure and our measured energy profiles can be used to decide between alternative Collections implementations. Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics—complexity mea- sures, performance measures General Terms Experimentation, Measurement Keywords Energy Profile, Collections, API, Java 1. INTRODUCTION Energy consumption is rapidly becoming an emerging topic for software engineering and research [15, 16, 27, 29, 31, 35, Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WOODSTOCK ’97 El Paso, Texas USA Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$15.00. 39, 41, 42]. In particular, the scale of data centers and lim- ited battery lifetime of ubiquitous mobile devices have forced the owners of these systems to monitor and budget for en- ergy at all fronts—software included. Although there is a growing need for developers to optimize the energy-eciency of their software, they are typically unaware of how to do this [34, 47]. Researchers have recently started focusing on autotuning the energy consumption inside software to optimize energy- eciency [15,16,20,21,31]. G¨ otz and colleagues [20,21] con- tributed the initial work following the autotuning optimiza- tion approaches in performance improvement (e.g., [37]). Bunse and colleagues [15, 16] focused on adapting systems at runtime to use the most energy-ecient sorting algo- rithm. In recent work, Manotas and colleagues [31] designed a tool for autotuning Java applications by selecting the most energy-ecient implementations for Collections APIs. Another approach to optimize energy-eciency is to in- form developers about the energy consequences of their high- level coding decisions, specifically to find alternative coding idioms. Researchers have explored the energy impact of de- sign patterns [17, 30, 40] and refactoring [35, 41]. However, they were not able to provide specific guidelines, perhaps because the energy footprints of these coding decisions were too small. Manotas and colleagues [31] achieved significant energy saving only by replacing Java Collections classes, but they were not able to explain what is contributing to the im- provement. Instead, they followed a search-based software engineering approach to find the alternative that produces the most energy-ecient result. Our work focuses on creating energy profiles of Java Col- lections classes in order to guide developers. We created energy consumption profiles of commonly used API meth- ods for variants of three Collections datatypes: List, Map, and Set. Then, we investigated the reasons behind the dif- ference in energy profiles by considering memory usage and bytecode executed during an operation. Finally, we explored how the Collections instances are used in real applications and how the usage impacts the overall energy consumption. Using the per-method energy profiles as building blocks, a developer can estimate the energy impact of each Collections instance and choose a more ecient alternative, if available. An essential property is that the profiles respect the con- straints developers are tied to, since programmers choose a Collections class on purpose, e.g., a List instead of a Set or a Map. Hence, proposing a Set or a Map to swap a List

Upload: others

Post on 04-Sep-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

Energy Profiles of Java Collections Classes

Samir HasanComputer Science andSoftware Engineering

Auburn [email protected]

Zachary KingComputer Science andSoftware Engineering

Auburn [email protected]

Munawar HafizComputer Science andSoftware Engineering

Auburn [email protected]

Mohammed SayaghMCIS

Polytechnique [email protected]

Bram AdamsMCIS

Polytechnique [email protected]

Abram HindleDept. Of Computing Science

University of [email protected]

ABSTRACTWe created detailed profiles of the energy consumed by com-mon operations done on Java List, Map, and Set abstrac-tions. The results show that the alternative data types forthese abstractions di↵er significantly in terms of energy con-sumption depending on the operations. For example, an Ar-rayList consumes less energy than a LinkedList if items areinserted at the middle or at the end, but consumes moreenergy than a LinkedList if items are inserted at the start ofthe list. To explain the results, we explored memory usageand bytecode executed during an operation. Expensive com-putation tasks in bytecode traces appeared to have energyimpact, but memory usage did not contribute. We evaluatedour profiles by using them to selectively replace Collectionstypes used in six applications and libraries. We found thatchoosing a wrong Collections type, as indicated by our pro-files, can cost as much as 300% more energy than the moree�cient choice. Our work shows that the usage context of adata structure and our measured energy profiles can be usedto decide between alternative Collections implementations.

Categories and Subject DescriptorsD.2.8 [Software Engineering]: Metrics—complexity mea-sures, performance measures

General TermsExperimentation, Measurement

KeywordsEnergy Profile, Collections, API, Java

1. INTRODUCTIONEnergy consumption is rapidly becoming an emerging topic

for software engineering and research [15, 16, 27, 29, 31, 35,

Permission to make digital or hard copies of all or part of this work for

personal or classroom use is granted without fee provided that copies are

not made or distributed for profit or commercial advantage and that copies

bear this notice and the full citation on the first page. To copy otherwise, to

republish, to post on servers or to redistribute to lists, requires prior specific

permission and/or a fee.

WOODSTOCK ’97 El Paso, Texas USA

Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$15.00.

39, 41, 42]. In particular, the scale of data centers and lim-ited battery lifetime of ubiquitous mobile devices have forcedthe owners of these systems to monitor and budget for en-ergy at all fronts—software included. Although there is agrowing need for developers to optimize the energy-e�ciencyof their software, they are typically unaware of how to dothis [34,47].

Researchers have recently started focusing on autotuningthe energy consumption inside software to optimize energy-e�ciency [15,16,20,21,31]. Gotz and colleagues [20,21] con-tributed the initial work following the autotuning optimiza-tion approaches in performance improvement (e.g., [37]).Bunse and colleagues [15, 16] focused on adapting systemsat runtime to use the most energy-e�cient sorting algo-rithm. In recent work, Manotas and colleagues [31] designeda tool for autotuning Java applications by selecting the mostenergy-e�cient implementations for Collections APIs.

Another approach to optimize energy-e�ciency is to in-form developers about the energy consequences of their high-level coding decisions, specifically to find alternative codingidioms. Researchers have explored the energy impact of de-sign patterns [17, 30, 40] and refactoring [35, 41]. However,they were not able to provide specific guidelines, perhapsbecause the energy footprints of these coding decisions weretoo small. Manotas and colleagues [31] achieved significantenergy saving only by replacing Java Collections classes, butthey were not able to explain what is contributing to the im-provement. Instead, they followed a search-based softwareengineering approach to find the alternative that producesthe most energy-e�cient result.

Our work focuses on creating energy profiles of Java Col-lections classes in order to guide developers. We createdenergy consumption profiles of commonly used API meth-ods for variants of three Collections datatypes: List, Map,and Set. Then, we investigated the reasons behind the dif-ference in energy profiles by considering memory usage andbytecode executed during an operation. Finally, we exploredhow the Collections instances are used in real applicationsand how the usage impacts the overall energy consumption.Using the per-method energy profiles as building blocks, adeveloper can estimate the energy impact of each Collectionsinstance and choose a more e�cient alternative, if available.An essential property is that the profiles respect the con-straints developers are tied to, since programmers choose aCollections class on purpose, e.g., a List instead of a Set ora Map. Hence, proposing a Set or a Map to swap a List

Page 2: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

is confusing as a guideline. This is di↵erent from an auto-tuning approach that aggressively swaps Collections classesbased only on API match [31].

This paper presents profiles for Collections types chosenfrom the Java Collections Framework [5], Apache CommonsCollections [1], and Trove [9]. We collected the energy usagedata using GreenMiner, a hardware-based energy consump-tion framework [26].

Our profiles indicate how the energy consumption of theseimplementations varies with input sizes and data types. Wenoticed that for smaller lists (size < 500), the energy con-sumption di↵erences are small, while the di↵erences are moreprominent if there are many elements. We found that opera-tions that contain expensive computation tasks in bytecodetraces appear to consume more energy. Surprisingly, mem-ory usage did not seem to have an impact. In fact, a Listcontaining integer elements is more expensive than a Listcontaining objects; extra operations for auto-boxing con-tribute, rather than the memory requirement of elements.

We evaluated our energy consumption profiles by modify-ing the List classes in four open source Java libraries, oneJava application, and one Android application. The energymeasurement shows that choosing wrong Collections classescan impact energy consumption by over 300%, while choos-ing a “green” option can improve energy consumption by asmuch as 38%.

Our results indicate that the di↵erences in energy con-sumption follow the trends suggested by our profiles. Theimpact is large when large collections are created during ex-ecution. This paper makes the following contributions:

• We describe a method in which the energy consumptionprofiles are measured on coding idioms in isolation, andare then used to provide guidance for choosing alterna-tive coding idioms.

• We measure the energy profiles of various kinds of Col-lections classes obtained from di↵erent sources, and alsoprofile energy consumption for varying input sizes andelement types (Section 4).

• We explored two possible alternatives to explain energyconsumption di↵erences between operations (Section 5).

• We evaluate on real applications that the alternative Col-lections classes can be swapped to predictably improveor worsen energy consumption (Section 6).

Our results are encouraging, since energy consumptionprofiles seem to have the potential to provide a developerguidance about choosing among Java Collections. If energyprofiles of alternative coding idioms are available, develop-ers can use them as a guideline to choose the “green” optionbased on their coding context.

2. COLLECTIONS CLASSES IN OUR STUDYJava Collections classes store group of objects and pro-

vide API to access, modify, or iterate over the elements.Java ships with the Java Collections Framework (JCF) thatprovides reusable and convenient implementations of pop-ular data-structures and algorithms. There are also manythird-party implementations of similar structures.

In this paper, we studied two third-party implementa-tions: Apache Commons Collections (ACC) and Trove. FormACC, we studied implementations that are alternative tothose already in JCF. The Trove collections hold only prim-itive data types; the goal is to reduce memory usage and

Table 1: Profiled Collections Classes

Trove

HashMapTreeMapLinkedHashMap

Map Collections Library

TIntHashSetTIntIntHashMapTIntArrayListTIntLinkedList

HashedMapLinkedMap

TreeList

Java Collections Framework

ListOrderedSetMapBackedSet

ArrayListLinkedList

List

HashSetTreeSetLinkedHashSet

Apache Commons Collections

Set

improve performance (Trove requires three times less heapspace than JCF implementations for larger collections [9]).

The Collections classes we studied are shown in Table 1.We profiled how much energy is consumed by single APImethods common across the implementations. More de-tailed results and profiles of other methods are availableat the project webpage: https://sites.google.com/site/collectionsenergy/.

3. ENERGY MEASUREMENT INFRASTRUC-TURE SETUP

We used GreenMiner’s hardware infrastructure to mea-sure the actual energy consumed by our test programs [26].We measured energy in joules (J) to determine the absoluteenergy consumption di↵erence. Each individual test was al-lowed to run to completion and the energy consumed byeach test was recorded.

3.1 GreenMiner InfrastructureThe GreenMiner [26] is a hardware/software continuous

testing suite. It instruments numerous devices, runs testson these devices, and measures the energy consumption andpower use of the entire device as the tests run. The Green-Miner client, the test-bed, is a Raspberry Pi that controlsan Android test device. The Pi executes tests on the device,and collects the results from an Arduino that monitors theenergy consumption of the test device. These results areuploaded to the GreenMiner webservice.

3.2 Measurement ProcessTo measure the e↵ect of using di↵erent collections on dif-

ferent workloads, a basic Android app was created. Thistest-app displays a blank screen and sits idle. The screen en-ergy consumption is constant throughout the test [18]. Thetest-app is a sca↵old for jUnit tests act as the experiments.Each unit test for the test-app is a separate experiment orrun. In each of these tests, a Collections class was createdand initialized with some data, and a workload (insertion,iteration, etc.,) was run against it. The energy consumed bythe test was measured and recorded with the Green Minerframework.

Given the use case of inserting items at the beginningof a list, we wrote jUnit test-cases. These test-cases wereparameterized by 5 list alternatives, described in Section 2.In each test, N items were added to the beginning of the list.We varied the input size N from 1 to 5000 (13 di↵erent sizes)and prepared test cases for each of them. Thus, to measurethe energy impact of inserting items at the beginning of lists,test cases were run against 65 di↵erent sets of parameters,each representing one of the 13 di↵erent sizes for one of the5 di↵erent lists. The energy consumption of each test-runfor each set of parameters (size and type of list used) wasrecorded as the tests were executed.

Page 3: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

●●●●● ● ● ● ● ●●

●●●●● ● ● ● ● ●●

●●●●● ● ● ● ● ●● ●

0 500 1250 2000 2750 3500 4250 5000

24

68

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mea

n Jo

ules

(J)

# Elements in List

(a) Insertion at beginning

●●●●● ● ● ● ● ● ● ● ●●●●●● ● ● ● ● ● ● ● ●●●●●● ●

●●

0 500 1250 2000 2750 3500 4250 5000

05

1015

2025 ●

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mea

n Jo

ules

(J)

# Elements in List

(b) Insertion in middle

●●●●●●

● ● ● ● ● ● ●●●●●● ● ● ● ● ● ● ● ●●●●●● ● ● ● ● ● ● ● ●

0 500 1250 2000 2750 3500 4250 5000

24

68

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mea

n Jo

ules

(J)

# Elements in List

(c) Insertion at end

●●●●●

●●

●●●●●

●●

0 500 1250 2000 2750 3500 4250 5000

24

68

10 ●

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mea

n Jo

ules

(J)

# Elements in List

(d) Iteration

●●●●● ● ● ●●

●●

●●

●●●●● ● ● ●●

●●

● ●

0 500 1250 2000 2750 3500 4250 50000

510

1520

2530

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mea

n Jo

ules

(J)

# Elements in List

(e) Random access

Figure 1: List: Energy profiles for insertions. Figures (a) – (c) shows list instances that have been initialized

with capacity if the API allows it (eg. ArrayList). Iteration and random access performance is shown in (d)

and (e).

Each GreenMiner run executes unit tests for a specific usecase, such as Insertions At Beginning of Lists mentionedabove. Each test-case, given all parameters, was run 20times on GreenMiner, and the results were collected. Thereports were downloaded and collated, as they report theenergy consumed during each run and also the mean of 20runs. This provided additional consistency to the data wegathered from GreenMiner. Finally, we plotted these meansagainst the list sizes to generate the energy consumptionprofiles.

There were, however, a few issues with this approach.First, we needed to ensure that each unit test encountersthe same overhead. Second, since our code fragments weresmall, their energy consumed could also be too small to beobservable. Third, the Java Virtual Machine (JVM) can in-voke the garbage collector at undefined stages of the test,and this may have polluted the results. Finally, the actualenergy consumed by a test suite varies from device to device.The GreenMiner system is attached to 4 di↵erent devices;the actual energy consumed varies slightly when the sametest runs on di↵erent devices. We took several measures todeal with these issues.

Ensuring a Fixed Overhead.We created a new instance of all the five lists in the

setUp()method, irrespective of the one that is actually usedfor the particular test. For instance, when inserting itemsinto a LinkedList, all the five list instances were first cre-ated through the setUp() method, followed by the actualinsertions.

Producing Observable Changes.Inside a test method, we repeated the API invocation mul-

tiple times. For example, when inserting 50 items, therewere 20 runs of: (1) invoking setUp(); (2) making the ac-tual insertions; (3) invoking tearDown(). All the unit testswere designed similarly. Thus, our numbers on the graphare an aggregate instead of the performance of a single run.This produces observable e↵ect on the energy consumptionof the test suite.

4. ENERGY PROFILE RESULTSWe profiled the energy consumption of some of the com-

mon API methods provided by List, Map, and Set imple-mentations, and record how it varies with input sizes. Specif-ically, we asked six research questions.

RQ1. Can we identify a List implementation that is energyoptimal for insertions, iteration, and random access?

RQ2. Can we identify a Map implementation that is energyoptimal for insertions, iteration, and random access?

RQ3. Can we identify a Set implementation that is energyoptimal for insertions, iteration, and random access?

RQ4. From what input size the energy di↵erences becomesignificant?

RQ5. Is there a noticeable energy consumption di↵erencewhen collections store di↵erent elements?

RQ6. Can we use the profiles to choose the most energy ef-ficient implementation of List, Map, and Set?

Page 4: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

●●●●● ● ● ●●

●●●●● ● ● ●

● ●

●●

0 500 1250 2000 2750 3500 4250 5000

1.0

1.5

2.0

2.5

3.0

3.5 ●

HashMap (JCF)LinkedHashMap (JCF)TreeMap (JCF)TIntIntHashMap (Trove)HashedMap (ACC)LinkedMap (ACC)

Mea

n Jo

ules

(J)

# Elements in Map

(a) Insertion

●●●●●

●●

●●●●●

●●

0 500 1250 2000 2750 3500 4250 5000

24

68 ●

HashMap (JCF)LinkedHashMap (JCF)TreeMap (JCF)TIntIntHashMap (Trove)HashedMap (ACC)LinkedMap (ACC)

Mea

n Jo

ules

(J)

# Elements in Map

(b) Iteration

●●●●●

●●

●●

●●●●●

●●

0 500 1250 2000 2750 3500 4250 5000

24

68

HashMap (JCF)LinkedHashMap (JCF)TreeMap (JCF)TIntIntHashMap (Trove)HashedMap (ACC)LinkedMap (ACC)

Mea

n Jo

ules

(J)

# Elements in Map

(c) Random Query

Figure 2: Energy profiles for insertions, iteration, and queries on random keys on Map

RQ1, RQ2, and RQ3 compare the energy profiles createdfor List, Map, and Set implementations; RQ4 and RQ5 areabout measuring the impact of input sizes and data types;RQ6 is about using the results as a guideline for developers.

RQ1. Can we identify a List implementationthat is energy optimal for insertions, iteration,and random access?Key Result: For insertions at the beginning, JCF’sLinkedList consumes the least energy, followed by Trove’sLinkedList. For insertions at the middle and end, Trove’sArrayList is the most energy e�cient, followed by JCF’sArrayList.

Figure 1(a), 1(b), and 1(c) demonstrate the energy con-sumption trends for insertion tests for the five List imple-mentations.

InsertionFor small sizes (1–500), the di↵erence in energy consump-tion for insertions at the beginning of the list is not evi-dent for all but two implementations—TreeList and TIntAr-rayList. Even at size 250, TreeList consumes ⇡ 31% moreenergy than TIntArrayList, while others consume about thesame energy. However, for larger sizes, LinkedList beginsto perform much better than others. At input size of 5,000,LinkedList consumes⇡ 13% less energy than TIntLinkedList,the next best performer. Compared to the worst performingTreeList, LinkedList consumes ⇡ 79% less energy.

When inserting items at the middle, an interesting pat-tern emerges between the di↵erent list implementations. Ar-rayList and TIntArrayList have very similar, and quite ef-ficient, energy performance. Next, TreeList and LinkedListboth have similar, yet not quite as e�cient, performance.And finally, TIntLinkedList has the worst performance byfar. At input size of 500, ArrayList and TIntArrayList per-form ⇡ 48% better than TIntLinkedList, a large di↵erencewhich increases to ⇡ 93% at size 5,000. There is a substan-tial amount of extra energy required by TreeList, LinkedList,and TIntLinkedList to perform insertion at the middle asopposed to at the beginning.

For insertions at the end of the list, the energy di↵erencesare not obvious for input sizes below 1000 for all lists, withthe exception of TreeList, which has a noticeable degrada-tion of ⇡ 32% at the size 250 (Figure 1(c)). For largersizes, however, the di↵erences become more evident. TIn-tArrayList saves ⇡ 25% energy compared to ArrayList and

⇡ 87% when compared to TreeList, the next best and worstenergy rated lists, respectively.

We also gathered similar profiles for the case when Ar-rayList and TIntArrayList are not set to a predefined ca-pacity during creation. Uninitialized array lists need toreallocate memory when current capacity is not su�cient.Due to this dynamic resizing, we expected a di↵erence inenergy consumption trends compared to the previous initial-ized version. However, there was no di↵erence in the energyconsumption for the uninitialized version, especially whenadding items at the middle and at the end of the list. Evenwith dynamic expansion, ArrayList and TIntArrayList arestill more energy e�cient than others. Therefore, initializ-ing an ArrayList variant with a capacity is not necessary—itwill perform well anyway.

IterationFigure 1(d) shows the energy consumption profile for itera-tion with an iterator. For small sizes, iterating an ArrayListis slightly more energy e�cient, while for 5000 items, it hasa maximum energy savings of ⇡ 4%. The results show thatthere is not much di↵erence when comparing energy con-sumption of iteration over the lists.

Random AccessWhen accessed through randomly generated indices, we didnot observe any major di↵erences in energy consumption forlist sizes smaller than 500 as shown in Figure 1(e). For largerinput, ArrayList, TIntArrayList and TreeList were the mostenergy e�cient, producing a savings of ⇡ 40% compared toLinkedList and ⇡ 77% when compared to TIntLinkedList.

RQ2. Can we identify a Map implementationthat is energy optimal for insertions, iteration,and random query?

Key Result: HashMap is the most energy e�cient alter-native for insertions and random query. If insertion orderis required to be preserved, ACC’s LinkedMap is slightlybetter on insertions than JCF’s LinkedHashMap. TreeMapis energy hungry and should be avoided unless explicitlyneeded.

InsertionFigure 2(a) shows the energy consumed by inserting key-value pairs in Map implementations. Unlike List implemen-

Page 5: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

●●●●● ● ● ●●

●●●●● ● ●●

●●

● ●

0 500 1250 2000 2750 3500 4250 5000

1.0

1.5

2.0

2.5

3.0 ●

HashSet (JCF)LinkedHashSet (JCF)TreeSet (JCF)TIntHashSet (Trove)MapBackedSet (ACC)ListOrderedSet (ACC)

Mea

n Jo

ules

(J)

# Elements in Set

(a) Insertion

●●●●●

●●

●●●●●

●●

0 500 1250 2000 2750 3500 4250 5000

24

68

10 ●

HashSet (JCF)LinkedHashSet (JCF)TreeSet (JCF)TIntHashSet (Trove)MapBackedSet (ACC)ListOrderedSet (ACC)

Mea

n Jo

ules

(J)

# Elements in Set

(b) Iteration

●●●●●

●●

●●●●●

●●

0 500 1250 2000 2750 3500 4250 5000

24

68

10

HashSet (JCF)LinkedHashSet (JCF)TreeSet (JCF)TIntHashSet (Trove)MapBackedSet (ACC)ListOrderedSet (ACC)

Mea

n Jo

ules

(J)

# Elements in Set

(c) Random Query

Figure 3: Energy profiles for insertions, iteration, and queries on random keys on Set.

tations, there are some variations in energy consumptioneven for smaller maps. For sizes up to 250 items, all exceptTreeMap perform equally well. TreeMap energy consump-tion increases drastically with larger input size. For 5000insertions, it is ⇡ 73% more expensive than HashMap and⇡ 88% more expensive than LinkedHashMap.

All other maps perform equally well for sizes up to 1000.Interestingly, HashMap performs consistently better thanall other maps until size 5000 where LinkedHashMap has adrop in energy consumption and saves ⇡ 8% energy overHashMap. For most of the cases, Trove’s TIntIntHashMapuses slightly higher energies than HashMap. This was sur-prising, since Trove implementation with primitive data typesdid not improve upon JCF HashMap, and for some inputsizes it was noticeably worse.

IterationSimilar to our findings for lists, the iteration performanceis almost the same for all implementations (Figure 2(b)).For larger lists, JCF’s HashMap requires a little less energy,while ACC’s HashedMap ended up being the most expen-sive. However, the di↵erences are very small for large lists,and even more so than for smaller ones.

QueryThe random query performance has an interesting trend asshown in Figure 2(c). For sizes up to 500, TreeMap is consis-tently one of the two most energy e�cient maps. However,for larger lists, TreeMap queries become the most expensive,while HashMap consumes the least energy—a minimum sav-ings of ⇡ 2% compared to LinkedHashMap, and a savingsof ⇡ 12% when compared to TreeMap.

RQ3. Can we identify a Set implementationthat is energy optimal for insertions, iteration,and random access?Key Result: HashSet is the most energy e�cient alterna-tive for insertions and random query. ACC’s ListOrdered-Set is the most energy e�cient Set for iterations, thoughnot by a large margin. TreeSet is energy hungry and shouldbe avoided unless explicitly needed.

InsertionFigure 3(a) shows the energy consumed by inserting valuesinto Set implementations. For input sizes less than 750, all

implementations are quite close but for a larger size, notice-able di↵erences arise.

Trove’s TIntHashSet is consistently the most e�cient, sav-ing ⇡ 13% energy over HashSet and ⇡ 49% over TreeSet.

IterationThe iteration performance, as shown in Figure 3(b), is verysimilar for all implementations—there are no apparent dif-ferences for sizes up to 1000. At the larger sizes, ACC’s Lis-tOrderedSet is the most energy e�cient, with a maximumenergy saving of ⇡ 4%. Again, there are often larger energysavings between smaller input sizes than there are betweenlarger; for example, at size 1500 ACC’s ListOrderedSet hasan ⇡ 27% savings over JCF’s LinkedHashSet where the sav-ings between the same implementations at size 5000 is onlya mere ⇡ 2% savings.

QueryThe random query performance is shown in Figure 3(c).There are many energy spikes throughout the various inputsizes. Interestingly, the largest di↵erences between imple-mentations are in the medium size inputs, between sizes 1000and 3000. Another notable trend is that TreeMap starts outin the smaller inputs to be one of the most e�cient imple-mentations for smaller sizes and then for larger sizes hasan ⇡ 13% degradation from the optimal performing Hash-Set. Whereas at size 50, TreeSet actually saves ⇡ 4% overHashSet.

RQ4. From what input size on the energy dif-ferences become significant?

Key Result: For input sizes 1–500, all alternative imple-mentations of List, Map, and Set perform equally well.

To determine the statistical significance of energy di↵er-ences, we determined for each alternative collection and eachsize the 95% confidence intervals of the energy consumptionmeasurements. For each size, we then compared the con-fidence intervals of each alternative collection in terms ofoverlap. Non-overlapping intervals indicate a significant dif-ference between the corresponding collection types.

We found that smaller Collections with less than 500 el-ements do not show a significant di↵erence in energy con-sumption between alternative implementations. The di↵er-ences get larger and become significant as we deal with moreelements. For example, for list insertions at the beginning,

Page 6: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

Random Access

InsertionIteration

TreeList

LinkedList

TIntArrayList

TIntLinkedList

At Beginning At Middle

ArrayList

At End

(a) List Matrix

QueryInsertion Iteration

TreeMap

LinkedMap

HashedMap

TIntIntHashMap

LinkedHashMap

HashMap

(b) Map Matrix

QueryInsertion Iteration

TreeSet

ListOrderedSet

MapBackedSet

TIntHashSet

LinkedHashSet

HashSet

(c) Set Matrix

1

2

3

4

6

5

Rank (best - worst):

Figure 4: Color map showing the rank of each List, Map and Set implementations based on the use case.

Green identifies an implementation as energy e�cient, while red denotes it as energy hungry. Therefore, the

greener the color, the better.

we compared ArrayList and LinkedList (among others) andfound that for a size of 250, the confidence intervals wereoverlapping. However, for 500 elements, the intervals be-came disjoint. After comparing all other lists in a similarway, we found 500 to be an appropriate threshold across allCollections types.

RQ5. Is there a noticeable energy consumptiondifference when collections store different ele-ments?Key Result: When running List insertion and iterationtests on small objects, we expected the energy consumptionto be higher than the integer based results. However, theresults show that integers are actually as energy expensive,and in many cases more expensive, than small objects aselements.

We report here the tests on inserting into List of smallobjects. We expected the energy consumption to be higherthan when running the tests on List of integers due to stor-ing larger data inside the lists. But, the energy profiles ofinteger runs were consistently more than those of the smallobjects. Figure 5 shows a scatterplot comparing the en-ergy consequences of List of integers vs List of small objects;the results are shown for ArrayList and LinkedList. Someinputs of the ArrayList are not shown since it skews thegraph. The majority of the points lie below the line y=x,whereas we were expecting for all points to lie above it. Forsize 4000, inserting an integer in the beginning of a JCFLinkedList is ⇡ 13% more expensive than inserting a smallobject. Though only insertion at beginning of LinkedListand ArrayList are shown, these results are consistent withother scenarios. More details and graphs are on the projectwebpage.

Since Java Generics do not support primitive types, theintegers are auto-boxed as Integer objects in order to beheld in the Collections. The lists are constantly adding andshedding this Integer wrapper at will. The results are likelydue to this.

RQ6. Can we use the profiles to choose the mostenergy efficient implementation of List, Map,and Set?Key Result: In general, TIntArrayList is the most energye�cient list implementation, followed by ArrayList. Formaps, HashMap has the best. For Sets, HashSet is themost energy e�cient with TIntHashSet as a close second.

Above the minimum threshold on the input size of 500

●●●●●●

0.9 1.2 1.5

0.9

1.2

1.5

● ArrayList (JCF)LinkedList (JCF)

Mea

n Jo

ules

(J) S

mal

l Obj

ects

Mean Joules (J) Integers

Figure 5: Comparing insertion of small objects to

insertion of integers

items (as mentioned in RQ4), we can use our profiles tochoose the most energy e�cient implementation based onthe way the Collections classes are used. Figure 4 summa-rizes our findings as choice matrices that can help in makingthese decisions. Each color denotes a rank: green identifiesthe most e�cient implementation, while red indicates theworst among the six. A row in the table with more green init is likely to be energy conservative on average.

In general, TIntArrayList, HashMap and HashSet (notshown) are the standout Collections implementations, fol-lowed closely by ArrayList and TIntHashSet. Lists storedas arrays are preferred in general. Linked list variants onlywork better if they are required to behave like a stack, i.e.,a datatype with items added and removed from the front.

5. WHY THESE ENERGY DIFFERENCES?We carried out further investigations to discover the key

factors that may explain the di↵erent energy consumptionprofiles. We explored two possible factors—(1) memory us-age during API operations and (2) time-consuming byte-code instructions executed during API operations. Here,we explain the di↵erences in energy consumed during dif-ferent kinds of add() operations to List implementations.Other Collections types and API methods are covered onour project webpage.

5.1 Memory UsageWe recorded memory consumption for List instances be-

fore and after invoking the add() operation, while adding500 items to the list. We chose 500, since RQ4 showed thatCollections with 500 or more items show significant di↵er-

Page 7: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

●● ● ●

●● ● ●

0 50 125 200 275 350 425 500

05

1015

2025

30

ArrayList (JCF)LinkedList (JCF)TIntArrayList (Trove)TIntLinkedList (Trove)TreeList (ACC)

Mem

ory

Usa

ge (K

B)

# Elements in List

Figure 6: Memory usage of List instances during

insertion

ences in energy consumption.Figure 6 shows the resulting memory consumption when

inserting at the beginning of the list. The graphs for inser-tions at the middle and at the end of the list were almost thesame, which indicates that no matter how the items are in-serted into the list, the memory footprints are similar. Sincewe did find di↵erent energy profiles (Figures 1(a)–1(c)) forthe di↵erent insertion approaches, this suggests that mem-ory consumption is not a (significant) driving factor behindthe energy consumption in this context.

5.2 Executed Dalvik BytecodesWe generated bytecode traces during the lifecycle of an

add() operation of two List instances (ArrayList and LinkedList)and compared them. First, we used dexdump to extract theapplication’s bytecode. Then, we instrumented each line ofthe bytecode using the AndBug debugger tool, which imple-ments the Java Debug Wire Protocol (JDWP). Upon exe-cution, the tool prints out the executed bytecode and thecorresponding source line.

Comparing the traces, we identified two bytecodes thatmay have an impact on the runtime (and therefore energy)performance: iget-object and invoke-static. When ele-ments are inserted in the middle, iget-object is executedmany more times than the other instructions. This is be-cause LinkedList traverses half of the list to reach to themiddle and locate the position for the new item. The largerthe List becomes, the more traversals are needed. For exam-ple, when the 500th element is inserted to a list, iget-objectis executed 63 times more than the next frequently occur-ring instruction. This may explain why LinkedList consumesmore energy than ArrayList, as shown in the energy profile(Figure 1(b)).

The impact of bytecodes are less obvious when elementsare inserted at the beginning or at the end. When in-serting at the beginning, invoke-static dominates the ex-ecution for an ArrayList (used to execute the expensiveSystem.arraycopy() method). This di↵erence in workloadis probably why an ArrayList instance consumes more en-ergy than a LinkedList instance for insertions at the begin-ning, as shown in our profiles. However, invoke-static

is also executed while inserting elements at the end. Inthis case, its impact likely is o↵set by many other byte-code instructions only found in LinkedList execution traces(new-instance and invoke-direct). Hence, this analysis isnot enough to explain why ArrayList performs better in this

context. This is left as future work.

6. EVALUATIONThe energy profiles compare the Collections classes for

each API method and suggest better alternatives (cf. RQ6).However, when Collections instances are used in applica-tions, multiple API methods are invoked on each object, de-pending on the role of the object in the system and the loadof the system. Hence, we expect that the energy footprint ofeach Collections object in an application is determined by acombination of the energy impact of all invoked API meth-ods. To analyze this, we ask two more research questions:

RQ7. Do the di↵erent Collections classes have energy im-pact in real applications compared to what we foundfor similar collections in the profiles? How big is theimpact?

RQ8. Can we use the energy profiles to switch to an alter-native collection and improve (or degrade) the energyconsumption of an application?

To answer RQ7, we modified real applications to use al-ternative Collections classes and measured the energy con-sumed by the modified applications. Previous work hasdemonstrated that Collections classes do have an impact [31].We extend this state of the art by selectively (based on theusage profile of each instance) modifying the Collections in-stances using the energy profiles (RQ8): we create “good”and “bad” versions of the original program when possible,and compare their energy consumption using GreenMiner.

Using the methodology of Figure 7, we studied the en-ergy consumption of four popular Java libraries—GoogleGson [4], Apache Commons Math [3], XStream [11], ApacheCommons Configuration [2]—, an open source email client,K-9 Mail [7], and a Stock Exchange Trading Simulator ap-plication. Each library came with a large test suite (16–137KLOC). We analyzed the list datatypes used in the codeto create usage profiles, i.e., which API methods are beinginvoked and where. We wrote an inter-procedural programanalyzer based on WALA [8] that automatically analyzesprogram bytecode.

Our WALA analyzer detected three kinds of Collectionsinstances: (1) Collections instances declared as fields of aclass and used in multiple methods, (2) Collections instanceslocally created inside methods and used in the same method,and (3) Collections instances locally created inside methodsbut used in multiple methods since it is passed as a returntype. The inter-procedural analysis uses call graphs andcontrol flow graphs created by WALA to collect usage pro-files for these instances. Currently, we do not support theanalysis of Collections instances when they are passed asan argument to a method. Adding this would require an-other inter-procedural analysis, but we did not find enoughinstances of Collections passed as parameters to justify theimplementation. We manually analyzed these remaining in-stances.

For each Collections instance found by WALA or our man-ual analysis, we identified if it is used in an energy-appropriatemanner or a better alternative is available, based on simpleheuristics derived from RQ6. When creating a “good” ver-sion, we looked into the usage of Collections instances to seewhether swapping one with another may save energy. Forexample, our profiles suggest that ArrayList is more energy-e�cient than LinkedList when inserting items at the end

Page 8: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

App

App + Test Suite

Inter-procedural Program Analysis

Manual Analysis

Collections Usage Profile

Good and

Bad Use

Bytecode

App + Test Suite

Modified Application

Original Application Analysis Output

Source CodeScript

for Rewriting Source Code

Source Code

Figure 7: Evaluation Workflow

of the list or when iterating over the list (these two are themost common List methods). So, if a LinkedList is used in aprogram for insertions at the end and iterations, our WALAprogram will detect it and indicate that we can improve en-ergy consumption by replacing LinkedList with ArrayList.

Similarly, we prepared the “bad”versions by going againstour profiles. For example, the profiles suggested that Ar-rayList is more energy e�cient than LinkedList for commonlist operations. Instead of following this recommendation,the ‘bad’ version replaces ArrayList with LinkedList. Weexpected this change to increase the energy consumption.

Next, we used a Python script to perform lexical anal-ysis on the source code and transform the List instancesto alternatives that should improve (or degrade) the energyconsumption. A simple lexical analysis was su�cient, sincewe swapped between alternative Collections instances with(almost) the same API (similar to Manotas et al. [31]). Fur-thermore, we chose to deal only with lists during our eval-uation. There are two reasons behind the choice. First,changing ArrayList to LinkedList (or vice-versa) is safe—the code, if it compiles, behaves the same way irrespectiveof the implementation. This may not be the case if wechange a HashMap to a TreeMap, since if the key objectdoes not have an appropriate compareTo() method defined,the maps may behave di↵erently. It is even more di�cult toconvert a TreeMap to a HashMap, since the sorting behaviorof a TreeMap may be desired in a usage scenario. Second,lists are more widely used than other collections such asmaps or sets (Gson: 60%, K-9 Mail: 57%, Apache Com-mons Math: 56%, XStream: 50%, Apache Commons Con-figuration: 53%, Stock Exchange Trading Simulator: 57%).Therefore, the energy contribution from lists is probablyhigher than that from other collections.

Eventually, we created four“bad”versions and three“good”versions. For the first three libraries, the developers al-most exclusively used ArrayList whenever they needed alist data structure and followed the common usage profileof adding an item at the end of a list and/or iterating thelist. Thus, we found little scope to improve on the energyconsumption for these libraries. Instead, it was more inter-esting for those three systems to demonstrate worse energyperformance by changing most of the ArrayList instancesto LinkedList. In K-9 Mail, we had comparatively moreLinkedLists, and thereby had a scope to show improvements.We therefore had both a “good” and a “bad” version for it.We also made “good” changes in the Apache Commons Con-figuration library and the Stock Exchange Trading Simulatorapplication, since they also use LinkedLists for operationsthat could be optimized for energy consumption.

The next sections describe the results of our analyses forthe six applications. To address RQ7, we report the energyimpact when we used alternative Collections instances, whilefor RQ8, we report that the changes that we made were

deliberately done (guided by our profiles) to get a “good” ora “bad” version.

6.1 Google GsonGoogle Gson is a serialization/deserialization library that

provides mechanisms to convert Java objects to JSON andback [4]. We used version 2.1 for our study, consisting ofabout 13 KLOC and a test suite of 16 KLOC. The GsonAPI refers to most of the collection instances through Col-lections interfaces, e.g., List, Map, and Set. The developerchooses whether to use an ArrayList or LinkedList instance,for example, where a List is required.

Changes Made. We found 53 ArrayList instances in thecodebase (Table 2). Only 4 of these instances are part ofthe library code, while 49 are in the test suite. Our WALAprogram discovered 4 instances on which end-insertions anditerations were performed; modifying these to LinkedList in-stances should be a“bad” change. We studied the rest of theinstances manually, and changed 47 instances to LinkedListthat had the same usage pattern.

Energy Impact. Table 2 shows the percentage of changein energy consumption of the modified application. WithLinkedList, the energy consumption increases by 309%. Thereare two factors contributing to this large increase. First, thelibrary performs 4 times slower when LinkedList instancesare used, which may cause more energy consumption duringthe test run. Second, the Gson test suite has a number ofperformance tests that perform serialization and deserializa-tion on large inputs (⇡ 2-4 MB). Our profiles, as discussedin RQ4, indicate that the energy di↵erences are more sig-nificant with larger collections, which is directly reflectedthrough the results that we achieved for Gson.

6.2 Apache Commons MathThe Apache Commons Math library provides implemen-

tations of mathematical and statistical algorithms that areotherwise unavailable in the standard Java distribution [3].We used version 3.4.1 (209 KLOC app + 137 KLOC test).The library creates 167 instances of ArrayLists, while wehave about 91 ArrayList instances in the test suite.

Changes Made. Since it was not possible to run all thetests on our device due to memory constraints, we selecteda subset of tests for our study (71%).

Out of a total of 258 instances of ArrayList in the codebaseand tests (Table 2), WALA detected 77 instances that wereused mostly for end-insertions (60 occurrences), iterations(18 occurrences) and random access (5 occurrences). Aftermanual inspection, we found 169 more instances that wereused similarly. Our profiles suggest that LinkedList is a badchoice for these instances. We made the ‘bad’ change.

Energy Impact. The modified version consumed 15% moreenergy (Table 2). Again, the changed library runs ⇡ 1.2times slower than the original version, and thereby consumesmore energy during the test run.

6.3 XStreamThe XStream library can be used to serialize Java objects

in XML and deserialize it back [11]. XStream version 1.5has a library of 34 KLOC and a unit test suite of 30 KLOC.There are 33 ArrayList instances in the library code and 128instances in the test code.

Page 9: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

1.47%

77 21148 4 K-9 Mail 294

64 1

7

161

29

0

Collections LinkedList

0

258 Apache Commons Math

XStream

KLOC

0

461

53

324

ArrayList

100

2

Google Gson

Program

346

125

246

153

51

# Changes

Good Bad

0.25%

0

0

0

0.32%

15%

5%

309%

Changes in Energy Consumption

Good Bad

76 1269 12 Apache Commons Configuration 154 0 0

38%11 80 8 Stock Exchange Trading Simulator 14 0 0

Table 2: Libraries and applications for evaluating the energy profiles

Changes Made. We choose a subset of the test suite (80%)as some tests were incompatible with the GreenMiner plat-form. Out of 161 instances of ArrayList, WALA detected23 instances that were used for end-insertions (20 occur-rences), iterations (4 occurrences) and random access (2 oc-currences). We manually found another 130 instances usedin the same way. In total, 153 ArrayList instances were con-verted to LinkedList, expecting higher energy consumption.

Energy Impact. There is a degradation of 5% when swap-ping the ArrayList instances with LinkedList (Table 2). Themodified version runs ⇡ 1.05 times slower than the original,which may explain why it has a higher energy consumption.

6.4 K-9 MailThe K-9 Mail version 5.101 codebase has 34 KLOC, with

a test suite of 2 KLOC. There are a total of 294 instancesof collections used, out of which only 28 were covered bythe test suite. To make sure more of the collections areexercised, we augmented the original test suite by generating256 more test cases for the app. We used JTExpert [43] toautomatically generate tests. As this generates tests for theJava platform, and Android tests should inherit from theclass AndroidTestCase, we modified the generated tests toadapt them to the Android platform by using JavaParser [6].In order to know which collection method is called by theexecuted tests, we did a dynamic analysis using AspectJ.

Changes Made. K-9 Mail application uses 148 ArrayListinstances and 21 LinkedList instances (Table 2). We ana-lyzed the program with WALA, and found that there wasscope to prepare both a “good” and a “bad” version of theprogram.

Our WALA program found that 53 instances of ArrayListsthat were used for ArrayList-friendly operations (40 occur-rences of end-insertions, 9 occurrences of iterations and 2 oc-currences of random accesses). We manually found 72 otherinstances having a similar usage pattern. We changed theseinstances to LinkedList, thereby creating a “bad” version.

With our WALA analysis, we also found 21 LinkedListinstances in the codebase. Our heuristics suggested thatwe should change 4 of these instances to ArrayList, becausethe lists were being used for end-insertions, insertions at arandom index, and queries using the contains() API. Ac-cording to our profiles, ArrayList is the most energy e�cientchoice in this context. We therefore created a“good”versionof the app by changing these 4 instances to ArrayList.

Energy Impact. Table 2 shows the di↵erences in energyconsumption of the two versions. For bad changes, K-9 Mailperformed only slightly worse, with an overall degradationof 0.32%. We did expect an increase in energy consumption,although it is only by a small amount. For the good changes,

we achieved an improvement of 0.25%.For both versions, we noticed that the di↵erences were

very small. The K-9 Mail test suite is significantly di↵erentfrom the rest of the applications that we studied—it does notexercise large collections. In Gson, the tests were feeding ahuge load (⇡ 2-4 MB) to store in lists. On the contrary, K-9 Mail tests were dealing with lists of only a few elements.Therefore, the impact was not very large.

6.5 Apache Commons ConfigurationThe Apache Commons Configuration library facilitates

storage and retrieval of configuration information for Javaapplications [2]. We studied version 1.10 that has 40 KLOCof library code, and a test suite of 36 KLOC. There are 13instances of LinkedList and 166 instances of ArrayList in theoriginal codebase.

Changes Made. We again choose a subset of the program(83%) that was compatible with the testing platform. Inthe reduced version, we had 69 ArrayList instances and 12LinkedList instances. Out of these 12 LinkedList instances,WALA detected 8 that were used for end-insertion and it-eration. We manually found the other 4 of them used ina similar way. Since our profiles indicate that ArrayList isa better choice for these operations, we changed these 12instances to ArrayList, expecting a decrease in the energyconsumption of the test suite.

Energy Impact. Changing the LinkedList instances to Ar-rayList improved the energy consumption by 1.47% (Ta-ble 2). The modified version of the library ran ⇡ 1.02 timesfaster than the original, which is probably why the energyconsumption was lesser.

6.6 Stock Exchange Trading SimulatorThis is a Java based simulation application developed in-

house at Auburn University. The program has 11 KLOClines of application code, with 8 instances of LinkedList usedin the codebase.

Changes Made. Our WALA program detected 7 instancesof LinkedList that were used for end-insertions and itera-tions. We manually found 1 more instance being used in asimilar way. Since ArrayList is better for both these opera-tions, we made “good” changes by swapping the LinkedListinstances with ArrayList.

Energy Impact. Our modified program demonstrated a 38%reduction in energy consumption and ran ⇡ 1.6 times faster.

6.7 DiscussionFor all our test applications, we were able to get di↵er-

ences in energy consumption by changing the Collectionsinstances (RQ7). The magnitude of change, however, de-

Page 10: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

pends on how aggressively the instances are exercised duringprogram execution. With significant usage, we can get largechanges in energy consumption when we make bad choicesfor Collections instances (RQ8).

We also noticed that for each of the applications, thedegradation factor for energy consumption was the sameas the slowdown factor of the bad version of the program.For example, the version of Gson with bad changes ran ⇡4 times slower and consumed ⇡ 4 times more energy thanthe original version. However, the power consumption ofboth versions (i.e., the rate of energy consumption per timeunit) was very similar, which indicates that the bad version(with ArrayList instances changed to LinkedList) just doesmore work during the extra time it takes. This may explainwhy it consumes more energy. We found a similar trend inexecution times while generating our profiles. But is it aconsistent trend that slower applications will consume moreenergy? More investigation is needed to answer this.

In our WALA analyzer, we chose a simple heuristic to helpus decide whether to change a Collections instance. Thisworked well owing to the fact that we dealt only with lists,and that most of the ArrayList instances that we found wereperforming end-insertions and iterations anyways. In thefuture, we want to focus on developing a more sophisticatedheuristic to handle other Collections.

7. THREATS TO VALIDITYThere are a number of issues a↵ecting the validity of our

work. There is noise in measuring physical systems; specif-ically phones are more noisy and nondeterministic devices.Thus we make a calm test-bed and repeat our measurements20 times to control for these di↵erences in measurement.

A more subtle issue may rise due to the range of the num-bers we achieved. As we saw in the profiles, the energymeasurements are quite small, specially for small collections.This is expected since a single API usage is a maximum ofthree lines of code performing an operation. There is a sig-nificant overhead introduced by setup and teardown meth-ods of each test. Can this large noise overshadow the other-wise small energy consumption of a single API invocation?To verify that this did not happen, we ran a separate baselinetest corresponding to each case that ran only the setup andteardown methods—this is the noise. We found that eachof our actual tests consumed more energy than the baseline.Therefore, our measurements reflect the energy contributionfrom the API usage.

8. RELATED WORKThere has been a body of empirical work in measuring

the impact of code change in various domains. Source levelmodifications such as refactorings [35, 41], design patterns[17,30,40], and code obfuscation [42] have been found to af-fect an application’s energy consumption. Hindle proposedGreen Mining to study how changes across software ver-sions a↵ect energy consumption [25]. Others have workedon evaluating energy behavior of sorting algorithms [15,16],web-servers [32], lock-free data structures [27], API usageof Android apps [29], and the e↵ect of advertisements andad-blocking on energy consumption [39].

Li et al. [28] repeatedly profiled java bytecode instruc-tions to produce a model linking source code and bytecodeto energy consumption in order to estimate the energy con-sumption of a line of Java code.

Researchers have measured energy in a number of ways.Hardware systems such as the Atom LEAP platform [45]and WattsUp meters [10] can measure actual power con-sumed by an application. Cycle accurate simulators suchas SoftWatt [23], Sim-Panalyzer [33], and simplepower [48]provides an energy estimate by simulating CPU cycles foreach component used in executing the application. Estima-tion based approaches [12–14,19,24,46] use empirical data topropose a model for estimating energy consumption. Pathaket al. [36] and Aggarwal et al. [12] show that dynamic anal-ysis of running systems, specifically by extracting systemcalls, can produce accurate runtime models of a system andestimate the energy consumption impact of a change. Simi-lar work on execution logs was executed by Gupta et al. [22]whereby they fingerprinted modules for their energy con-sumption profile. Zhang et al. [50] describe an online profilerPowerTutor that models energy consumption by aggregatemodels of individual components such as network and CPU.

Our work is closest to the approach taken by [31]. The pa-per describes an autotuning framework, SEEDS, that aids inchoosing the most energy-e�cient collection from the JavaCollections API automatically. SEEDS achieves this by run-ning an exhaustive trial-and-error on all compatible collec-tion implementations and measuring the impact of each onthe overall energy consumption of a given test suite for theapplication. Our approach is significantly di↵erent in mak-ing the comparison. Instead of an exhaustive search onwhich implementation is best for the particular application(test suite), we use empirical evidence, or the energy profiles,that we discovered in this work and the API usage patternsto predict the best alternative.

Another closely related work describes Chameleon [44], anautotuning approach for optimizing collection usage. How-ever, it is particularly focused on runtime performance (e.g.,clock time) and memory usage. Runtime performance couldbe a proxy for energy consumption. We believe that an au-totuning framework can be equipped with our profiles tomake a more accurate and realistic tool.

All of this work is well motivated as all studies on devel-oper and consumer knowledge about software energy con-sumption indicate that developers and consumers are notvery aware of what in their software consumes energy [34,38,47,49].

9. CONCLUSIONOur results provide a guideline about the scenarios in

which the energy consumptions of alternative Collectionsclasses become an issue. For insertion operations, the en-ergy di↵erences are significant, but not that much for otherlist operations. Also for lists of small size, the energy con-sumption does not vary much between the lists. Further-more, many of the di↵erences in energy consumption can beexplained by expensive bytecode operations.

Overall, our results will be especially useful for developersof large scale software who commonly work with large Col-lections instances. They can guide the developers and makethem aware of the consequences of their programming de-cisions. Our approach can also be used in making smarterautotuning tools. Most importantly, this should motivatefuture work on creating better guidelines for many otheralternative programming choices.

More information is available on the project webpage:https://sites.google.com/site/collectionsenergy/.

Page 11: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

10. REFERENCES[1] Apache commons collections.

http://commons.apache.org/proper/

commons-collections/source-repository.html.[2] Apache commons configuration.

https://commons.apache.org/proper/

commons-configuration/index.html.[3] Commons math: The apache commons mathematics

library. https://commons.apache.org/proper/commons-math/.

[4] Google gson.https://code.google.com/p/google-gson/.

[5] Java collections framework.http://docs.oracle.com/javase/8/docs/

technotes/guides/collections/.[6] Java parser.

https://github.com/javaparser/javaparser.[7] K-9 mail. http://k9mail.org/.[8] T.j. watson libraries for analysis (wala). http://wala.

sourceforge.net/wiki/index.php/Main_Page.[9] Trove. http://trove.starlight-systems.com/.

[10] Watts up. https://www.wattsupmeters.com/secure/products.php?pn=0.

[11] Xstream. http://xstream.codehaus.org/.[12] K. Aggarwal, C. Zhang, J. C. Campbell, A. Hindle,

and E. Stroulia. The power of system call traces:Predicting the software energy consumption impact ofchanges. In Press of the 2014 Conference of theCenter for Advanced Studies on CollaborativeResearch, IBM Corp, 2014.

[13] N. Amsel and B. Tomlinson. Green tracker: a tool forestimating the energy consumption of software. InCHI’10 Extended Abstracts on Human Factors inComputing Systems, pages 3337–3342. ACM, 2010.

[14] D. Brooks, V. Tiwari, and M. Martonosi. Wattch: aframework for architectural-level power analysis andoptimizations, volume 28. ACM, 2000.

[15] C. Bunse, H. Hopfner, E. Mansour, andS. Roychoudhury. Exploring the energy consumptionof data sorting algorithms in embedded and mobileenvironments. In MDM’09. Tenth InternationalConference on, pages 600–607. IEEE, 2009.

[16] C. Bunse, H. Hopfner, S. Roychoudhury, andE. Mansour. Choosing the” best” sorting algorithm foroptimal energy consumption. In ICSOFT (2), pages199–206, 2009.

[17] C. Bunse, Z. Schwedenschanze, and S. Stiemer. On theenergy consumption of design patterns. In EASED@BUIS, pages 7–8. Citeseer, 2013.

[18] M. Dong, Y.-S. K. Choi, and L. Zhong. PowerModeling of Graphical User Interfaces on OLEDDisplays. In DAC 2009, DAC ’09, pages 652–657, NewYork, NY, USA, 2009. ACM.

[19] M. Dong and L. Zhong. Self-constructive high-ratesystem energy modeling for battery-powered mobilesystems. In Proceedings of the 9th MobiSys, pages335–348. ACM, 2011.

[20] S. Gotz, C. Wilke, S. Richly, and U. Aßmann.Approximating quality contracts for energyauto-tuning software. In GREENS 2012, pages 8–14,June 2012.

[21] S. Gotz, C. Wilke, M. Schmidt, S. Cech, and Uwe.

Towards energy auto tuning, Aug. 21 2013.[22] A. Gupta, T. Zimmermann, C. Bird, N. Naggapan,

T. Bhat, and S. Emran. Detecting Energy Patterns inSoftware Development . Technical ReportMSR-TR-2011-106, Microsoft Research, 2011.

[23] S. Gurumurthi, A. Sivasubramaniam, M. J. Irwin,N. Vijaykrishnan, and M. Kandemir. Using completemachine simulation for software power estimation:The softwatt approach. In Proceedings of the HPCA8,pages 141–150. IEEE, 2002.

[24] S. Hao, D. Li, W. G. Halfond, and R. Govindan.Estimating android applications’ cpu energy usage viabytecode profiling. In GREENS, 2012 FirstInternational Workshop on, pages 1–7. IEEE, 2012.

[25] A. Hindle. Green mining: A methodology of relatingsoftware change and configuration to powerconsumption–web edition. 2014.

[26] A. Hindle, A. Wilson, K. Rasmussen, E. J. Barlow,J. C. Campbell, and S. Romansky. Greenminer: ahardware based mining software repositories softwareenergy consumption framework. In Proc. of the 11thMSR, pages 12–21. ACM, 2014.

[27] N. Hunt, P. S. Sandhu, and L. Ceze. Characterizingthe performance and energy e�ciency of lock-free datastructures. In INTERACT, pages 63–70. IEEE, 2011.

[28] D. Li, S. Hao, W. G. Halfond, and R. Govindan.Calculating source line level energy information forandroid applications. In Proceedings of the 2013ISSTA, pages 78–89. ACM, 2013.

[29] M. Linares-Vasquez, G. Bavota, C. Bernal-Cardenas,R. Oliveto, M. Di Penta, and D. Poshyvanyk. Miningenergy-greedy api usage patterns in android apps: anempirical study. In Proceedings of the 11th WorkingConference on MSR, pages 2–11. ACM, 2014.

[30] A. Litke, K. Zotos, A. Chatzigeorgiou, andG. Stephanides. Energy consumption analysis ofdesign patterns. In Proceedings of the InternationalConference on Machine Learning and SoftwareEngineering, pages 86–90. Centre for Telematics andInformation Technology, University of Twente, 2005.

[31] I. Manotas, L. Pollock, and J. Clause. Seeds: Asoftware engineer’s energy-optimization decisionsupport framework. In Proceedings of the 36th ICSE,pages 503–514. ACM, 2014.

[32] I. Manotas, C. Sahin, J. Clause, L. Pollock, andK. Winbladh. Investigating the impacts of web serverson web application energy usage. In GREENS, 20132nd International Workshop on, pages 16–23. IEEE,2013.

[33] T. Mudge, T. Austin, and D. Grunwald. The referencemanual for the sim-panalyzer version 2.0.

[34] C. Pang, A. Hindle, B. Adams, and A. E. Hassan.What do programmers know about the energyconsumption of software? PeerJ PrePrints, 3, 2015.

[35] J.-J. Park, J.-E. Hong, and S.-H. Lee. Investigation forsoftware power consumption of code refactoringtechniques.

[36] A. Pathak, Y. C. Hu, and M. Zhang. Bootstrappingenergy debugging on smartphones: a first look atenergy bugs in mobile devices. In Proceedings of the10th ACM Workshop on Hot Topics in Networks,page 5. ACM, 2011.

Page 12: Energy Profiles of Java Collections Classeswebdocs.cs.ualberta.ca/~hindle1/2016/hindle-icse2016.pdf · 2016. 4. 15. · ArrayList LinkedList List HashSet TreeSet LinkedHashSet Apache

[37] P. M. Phothilimthana, J. Ansel, J. Ragan-Kelley, andS. Amarasinghe. Portable performance onheterogeneous architectures. In ASPLOS 2013,ASPLOS ’13, pages 431–444, New York, NY, USA,2013. ACM.

[38] G. Pinto, F. Castor, and Y. D. Liu. Mining questionsabout software energy consumption. In Proceedings ofthe 11th Working Conference on MSR, pages 22–31.ACM, 2014.

[39] K. Rasmussen, A. Wilson, and A. Hindle. Greenmining: energy consumption of advertisement blockingmethods. In H. A. Muller, P. Lago, M. Morisio,N. Meyer, and G. Scanniello, editors, GREENS 2014,pages 38–45. ACM, 2014.

[40] C. Sahin, F. Cayci, I. L. M. Gutierrez, J. Clause,F. Kiamilev, L. Pollock, and K. Winbladh. Initialexplorations on design pattern energy usage. InGREENS, 2012, pages 55–61. IEEE, 2012.

[41] C. Sahin, L. Pollock, and J. Clause. How do coderefactorings a↵ect energy usage? In ESEM, pages36:1–36:10, New York, NY, USA, 2014. ACM.

[42] C. Sahin, P. Tornquist, R. McKenna, Z. Pearson, andJ. Clause. How does code obfuscation impact energyusage?

[43] A. Sakti, G. Pesant, and Y.-G. Gueheneuc. Instancegenerator and problem representation to improveobject oriented code coverage. IEEE Transactions onSoftware Engineering, pages 1–1, To appear, 2015.

[44] O. Shacham, M. Vechev, and E. Yahav. Chameleon:

adaptive selection of collections. In ACM SigplanNotices, volume 44, pages 408–418. ACM, 2009.

[45] D. Singh and W. J. Kaiser. The atom leap platformfor energy-e�cient embedded computing. Center forEmbedded Network Sensing, 2010.

[46] V. Tiwari, S. Malik, and A. Wolfe. Power analysis ofembedded software: a first step towards softwarepower minimization. Very Large Scale Integration(VLSI) Systems, IEEE Transactions on, 2(4):437–445,1994.

[47] C. Wilke, S. Richly, S. Gotz, C. Piechnick, andU. Aßmann. Energy consumption and e�ciency inmobile applications: A user feedback study. InGreenCom 2013, (iThings/CPSCom) and CPSCom,pages 134–141. IEEE, 2013.

[48] W. Ye, N. Vijaykrishnan, M. Kandemir, and M. J.Irwin. The design and use of simplepower: acycle-accurate energy estimation tool. In Proceedingsof DAC, pages 340–345. ACM, 2000.

[49] C. Zhang, A. Hindle, and D. M. German. The impactof user choice on energy consumption. IEEE Software,31(3):69–75, 2014.

[50] L. Zhang, B. Tiwana, Z. Qian, Z. Wang, R. P. Dick,Z. M. Mao, and L. Yang. Accurate Online PowerEstimation and Automatic Battery Behavior BasedPower Model Generation for Smartphones. InCODES/ISSS ’10, pages 105–114, New York, NY,USA, 2010. ACM.