pyparis2017 / incremental computation in python, by philip schanely
TRANSCRIPT
Incremental Computation in Python
Phillip Schanely
pschanely @medium
pschanely @twitter
The New Messaging App
You have no new messages
Joe IDK, let’s go to Sal’s
Sarah LOLZ
Mike Right? I’ve never …
You have no new messages
Joe IDK, let’s go to Sal’s
Sarah LOLZ
Mike Right? I’ve never …
You have no new messages
Joe IDK, let’s go to Sal’s
Sarah LOLZ
Mike Right? I’ve never …
The Model-View Problem
Incremental MapReduce in Python
Contents:
The Model-View Problem
Model, M
View, V
Model, M State Change Model, M’
View, V
Model, M State Change Model, M’
View, V View, V’Deci
sion T
ime!
Model, M State Change
Reco
mpu
te
Model, M’
View, V View, V’
Model, M State Change
Infer
Model, M’
View, V View, V’
Model, M State Change
Invalidate
Model, M’
View, V View, V’
Recompute
Invalidate
Infer
Recompute
Invalidate
Infer
Incremental MapReduce
It’s Everywhere
Javascript Objects
DOM
Javascript Objects
DOM
Web Apps
Infer Reco
mpu
te
DOM
Pixels
DOM
Pixels
Browsers
Invalidate
Application Pixels
Display
Application Pixels
Display
OS Display
Infer
Database State
API Responses
Database State
API Responses
Backend Logic
Reco
mpu
te
Normalized Data
Computed Data
Normalized Data
Computed Data
Application Databases
Infer Reco
mpu
te
Database Row
Database Index
Database Row
Database Index
Databases
Infer
RAM
Processor Cache
RAM
Processor Cache
Multiprocessor L1/L2 caches
Invalidate
Code
Build
Code
Build
Build Systems
Reco
mpu
te
Code
Test Results
Code
Test Results
Test Runners
Reco
mpu
te
Incremental MapReduce
Model, M State Change Model, M’
View, V View, V’
Incremental MapReduce
4 6
10
1 5
6
16
4 6
10
1 5
6
16
4 6 1 53
Insert “3”
4 6
10
1 5
6
16
4 6 1 53
Insert “3”
10 9
19
4
ScenicOverlook$ pip install ScenicOverlook
4 6 1 5>>> l = viewablelist([4,6,1,5])
4 6 1 5>>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16
4 6 1 5 4 6 1 53
Insert “3”
>>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16 >>> l = l[:2] + [3] + l[2:]
4 6 1 5 4 6 1 53
Insert “3”
>>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16 >>> l = l[:2] + [3] + l[2:] >>> l.reduce(add, initializer=0) 19
>>> import heapq >>> def sort(l): ... l = l.map(lambda x:[x]) ... return l.reduce(heapq.merge, initializer=()) >>> l = viewablelist([9, 3, 5, 7]) >>> sort(l) [3, 5, 7, 9] >>> sort(l + [ 4 ]) [3, 4, 5, 7, 9]
3 5 79
9 5 73
5 7 93
Graphics Demo
$ python demo_graphics.py
Compiler Demo
$ python demo_graphics.py
Normalize
Generate Call Tree
Normalize
Type Check
Generate Call Tree
Normalize
Type Check
Generate Call Tree
Normalize
LLVM Generation
Type Check
Generate Call Tree
Normalize
LLVM Generation
link
Umut Acar (CMU): “self-adjusting computation”
Matthew A. Hammer: “Adapton”
Interesting Related Work
github.com/pschanely/ScenicOverlook
pschanely @medium
pschanely @twitter
Questions?