Race Detection forEvent-driven Mobile Applications

Chun-Hung Hsiao University of MichiganJie Yu University of Michigan / Twitter

Satish Narayanasamy University of MichiganZiyun Kong University of Michigan

Cristiano Pereira IntelGilles Pokam Intel

Peter Chen University of MichiganJason Flinn University of Michigan

2

Rise of Event-Driven Systems

Mobile apps

Web apps

Data-centers

Lack tools for finding concurrency errors in these systems

3

Why Event-Driven Programming Model?

Need to process asynchronous input from a rich set of sources

4

Events and Threads in Android

Event Queue

wait(m)

rd(x)

wr(x)

signal(m)

Looper Thread ThreadsRegular Threads

send( )

5

Conventional Race Detection

Looper Thread Regular Threads

rd(x)

wr(x)

signal(m)

wait(m)

send( )

Conflict: Read-Write or Write-Write data accesses to same location

Causal order: happens-before ( ) defined by synchronization operations

Race ( ): Conflicts that are not causally ordered

e.g., FastTrack [PLDI’09]

6

Looper Thread Regular Threads

NullPointerException!

Conventional race detectors cannot find such errors in Android

Problem: Causality model is too strictShould not assume program order between events

Conventional Race Detection: Problem

7

Model Events as Threads?

Event Regular ThreadsEvent Event

Race

8

Events as Threads: ProblemRegular Threads

Event

Event

False race

send( )

send( )

Missing causal order!

Problem: Causality model is too weakAndroid system guarantees certain causal ordersbetween events

9

Challenge 1: Modeling Causality

Goal: Precisely infer causal order between eventsthat programmers can assume

A → BC || B

A

B

C

Looper Thread

B

10

Challenge 2: Not All Races are Bugs

Races between events(e.g., ~9000 in ConnectBot)

Order violations

Atomicity violations

Not a problem in Android events!

Solution: Commutativity analysis identifies races that cause order violations

One looper thread executes all events non-preemptively

11

Outline

• Causality Model • Commutativity Analysis• Implementation & Results

12

Causality Model

• Android uses both thread-based and event-based models

• Causal order is derived based on following rules:

1. Conventional causal order in thread-based model2. Event atomicity 3. Event queue order

Conventional causal order; Event atomicity; Event queue order

Conventional causal order;Event atomicity;Event queue order

13

fork(thread)

send(B)

Program order

Fork-join

Send

Looper Thread

Regular Thread

begin(thread)

fork(thread) → begin(thread)end(thread) → join(thread)signal(m) → wait(m)

send(event) → begin(event)

begin(A)

end(A)

begin(B)

end(B)

signal(m)

wait(m)Signal-wait

Conventional causal order; Event atomicity; Event queue order

14

One looper thread executes all events non-preemptively => events are atomic

Ordered due to event atomicity

begin(A) → end(B)

end(A) → begin(B)

fork(thread)

send(B)

Looper Thread

Regular Thread

begin(thread)

begin(A)

end(A)

begin(B)

end(B)

Conventional causal order; Event atomicity; Event queue order

15

Ordered due to FIFO queue order

send(A) → send(B)

end(A) → begin(B)

send(B)

Looper Thread Regular Thread

begin(A)

end(A)

begin(B)

end(B)

Conventional causal order; Event atomicity; Event queue order

Event Queue

send(A)A

B

16

It’s Not That Simple…

Special send APIs can overrule the FIFO order – Event with execution delay– Prioritize an event• sendAtFront(event): inserts event to queue’s front

Conventional causal order; Event atomicity; Event queue order

Special event queue rules handle these APIs.

See paper for details.

17

Event Orders due to External Input

A

B

C

Looper Thread Assume all events generated by the external environment are ordered

B

18

What is External Input?

External Environment

IPC

surfaceflinger

App

context_manager

system_server

19

Outline

• Causality Model • Commutativity Analysis• Implementation & Results

20

Problem: Not All Races are BugsRaces between events

Order violations

Atomicity violations

Not a problem in Android events!

21

Order Violations in EventsLooper Thread Looper Thread

Race between non-commutative events => order violation

22

Races in Commutative Events

Hard to determine if events are commutative!

Looper Thread Looper Thread

racy events are commutative=> not a race bug

23

Report races between known non-commutative operations -- uses & frees

Solution: Commutativity Analysis

Free

A

B

C

Looper Thread

UseHeuristics to handle commutative events with uses and frees.

See paper for details.

B

24

Outline

• Causality Model • Commutativity Analysis• Implementation & Results

25

CAFA: Race Detection Tool for Android

Logs synchronization operations for causality inferenceLogs data access operations related to uses and freesAlso logs the system service processes for complete causalityLogger device in the kernel for trace collectionOffline race detector based on graph reachability test

surfaceflingerAppcontext_manager

system_server

Android Kernel

Java Libs

Dalvik VM

Native Libs

IPC BinderCAFA

Analyzer

Java Libs

Dalvik VM

Native Libs

Java Libs

Dalvik VM

Native Libs

CAFAAnalyzer

LoggerLogger

Java Libs

Dalvik VM

Native Libs

IPC Binder

26

Tested Applications

Use-after-Free Races115 races; 69 race bugs (67 unknown bugs)

27

38 (33.0%)

31 (27.0%)

46 (40.0%)

Races in conventional causality model

Races in Android causality model

False positives

32 benign races (27.8%):Imprecise commutative analysis

14 false races (12.2%):Imprecise causal order: -- Imperfect implementation

Between events

Between threads25 (21.7%)

13 (11.3%)

28

Performance Overhead

• Trace collection– 2x to 6x; avg: ~3.2x– Interactive performance is fair

• Offline analysis – Depends on number of events– 30 min. to 16 hrs. for analyzing ~3000 to ~7000 events

29

Summary• Races due to asynchronous events is wide spread

• Contributions– Causality model for Android events– Commutativity analysis identifies races that can cause

order violations– Found 67 unknown race bugs with 60% precision

• Future work– Commutativity analysis for finding a broader set of order

violations– Optimize performance