![Page 1: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/1.jpg)
Nada AminUniversity of Cambridge, UK — [email protected]
joint work with Tiark Rompf Purdue University, USA — [email protected]
Collapsing Towers of Interpreters
![Page 2: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/2.jpg)
The ChallengeCollapse
a tower of interpreters
(languages L0, …, Ln & interpreters for Li+1 written in Li)
into
a one-pass compiler from Ln to L0
removing all interpretive overhead
![Page 3: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/3.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 4: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/4.jpg)
I1
I2
In-1
…
In
Ln
L0
Ln
L0
One-Pass Compiler
![Page 5: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/5.jpg)
I1
I2
In-1
…
In
Ln
L0
I1
I2
In-1
In
Python
ARM CPU
JavaScript VM
x86 runtime
bytecode
![Page 6: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/6.jpg)
I1
I2
In-1
…
In
Ln
L0
I1
I2
In-1
In
Python
ARM CPU
JavaScript VM
x86 runtime
bytecode
Python
ARM CPU
One-Pass Compiler
![Page 7: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/7.jpg)
I1
I2
In-1
…
In
Ln
L0
I1
I2
In-1
In
Python
ARM CPU
JavaScript VM
x86 runtime
bytecode
Python
ARM CPU
One-Pass Compiler
Python
JavaScript VM
One-Pass Compiler
![Page 8: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/8.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 9: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/9.jpg)
VM
Evaluator
Regexp Matcher
Low Level
High Level
Evaluator
…
Regexp Regexp
Low Level
One-Pass Compiler
![Page 10: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/10.jpg)
VM
Evaluator
Modified Evaluator
Low Level
High Level
Evaluator
…
High Level High Level
Low Level
One-Pass Transformer
![Page 11: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/11.jpg)
VM
Evaluator
Modified Evaluator
Low Level
High Level
Evaluator
…
High Level
I1
I2
In-1
In
Python
ARM CPU
JavaScript VM
x86 runtime
bytecode
VM
Evaluator
Regexp Matcher
Low Level
High Level
Evaluator
…
Regexp High Level
Low Level
One-Pass Compiler
(/ Transformer)
![Page 12: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/12.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 13: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/13.jpg)
I1
I2
In-1
…
In
Ln
L0 base L0 = variant of λ-calculus
~ conceptually infinite
~ reflective can be inspected and modified
at runtime
![Page 14: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/14.jpg)
I1
I2
In-1
…
In
Ln
L0
shifting to meta (reify)
shifting to object (reflect)
most user / object / high level
most impl. / meta / low level
![Page 15: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/15.jpg)
I1
I2
In-1
…
In
Ln
L0
most impl. / meta / low level
most user / object / high level
shifting to meta (reify)
shifting to object (reflect)
![Page 16: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/16.jpg)
I1
I2
In-1
…
In
Ln
L0
conceptually infinite…L∞
default semantics finite execution
most user / object / high level
shifting to meta (reify)
shifting to object (reflect)
![Page 17: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/17.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 18: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/18.jpg)
I1
I2
In-1
…
In
Ln
L0 base L0 = variant of λ-calculus
~ conceptually infinite
~ reflective can be inspected and modified
at runtime
![Page 19: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/19.jpg)
I1
I2
In-1
…
In
Ln
L0
Ln
L0
One-Pass Compiler
![Page 20: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/20.jpg)
Example in Purple/Black
(define fib (lambda (n) (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2))))))
> (fib 7) ;; => 13
![Page 21: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/21.jpg)
Reflection in Purple/Black(EM (begin ;; EM = Execute-at-Metalevel (define counter 0) (define old-eval-var eval-var)(set! eval-var (lambda (e r k) (if (eq? e ’n) (set! counter (+ counter 1)) (old-eval-var e r k)))))
> (fib 7) ;; => 13> (EM counter) ;; => 102
![Page 22: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/22.jpg)
Compilation in Purple
(set! fib (clambda (n) ;; c = compiled (if (< n 2) n (+ (fib (- n 1)) (fib (- n 2))))))
> (EM (set! counter 0))> (fib 7) ;; => 13> (EM counter) ;; => 102
![Page 23: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/23.jpg)
Compilation in Purple(EM (set! eval-var old-eval-var)) > (EM (set! counter 0))> (fib 7) ;; => 13> (EM counter) ;; => 102
(set! fib (clambda (n) …) > (EM (set! counter 0))> (fib 7) ;; => 13> (EM counter) ;; => 0
![Page 24: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/24.jpg)
Collapse in Purple{(k, xs) => _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_1 => _cell_set(<cell counter>, c_1) _app(‘<, _cons(_car(xs), ‘(2)), _cont{v_1 => _if(_true(v_1),_app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_2 => _cell_set(<cell counter>, c_2)_app(k, _cons(_car(xs), ‘()), _cont{v_2 => v_2})}), _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_3 => _cell_set(<cell counter>, c_3) _app(‘-, _cons(_car(xs), ‘(1)), _cont{v_3 => _app(_cell_read(<cell fib>), _cons(v_3, ‘()), _cont{v_4 => _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_4 => _cell_set(<cell counter>, c_4) _app(‘-, _cons(_car(xs), ‘(2)), _cont{v_5 => _app(_cell_read(<cell fib>), _cons(v_5, ‘()), _cont{v_6 =>_app(‘+, _cons(v_4, _cons(v_6, ‘())), _cont{v_7 => _app(k, _cons(v_7, ‘()), _cont{v_8 => v_8})})})})})})})}))})})}
![Page 25: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/25.jpg)
Solving the Challenge
![Page 26: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/26.jpg)
YoshihikoFutamura
1971
![Page 27: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/27.jpg)
TheFirstFutamuraProjec<on
interpreter
sourceprogram input
result
(a)
mix
input
result
interpreter
target
(b)
sourceprogram
The 1st Futamura Projection
image credit: Ruby Tahboub (Purdue)
![Page 28: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/28.jpg)
TheFirstFutamuraProjec<on
Specializinganinterpreterwithrespecttoaprogramproducesacompiledversionofthatprogram.
interpreter
sourceprogram input
result
(a)
mix
input
result
interpreter
target
(b)
sourceprogram
The 1st Futamura Projection
image credit: Ruby Tahboub (Purdue)
Specializing an interpreter with respect to a program produces a compiled version of that program.
![Page 29: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/29.jpg)
Prac<calRealiza<on
QueryInterpreter ThefirstFutamuraProjec<on ThefirstFutamuraProjec<onrealiza<onthroughspecializa<on
SQL Engine(interpreter)
SQL Query(source)
input(data)
result
(a)
input
(staged_interpreter)
SQL Query(source)
input(data)
result
target
(c)
input
mix
SQL Query(source)
input(data)
result
SQL Engine(interpreter)
target
(b)
input
result = interpreter(source, input) target = staged_interpreter(source) target = mix(interpreter, source)
result = target(input) result = target(input)
source
source source
source source
sourceinput
input input
• Automa<cpar<alevalua<onisahardproblemduetobinding-<meanalysis(BTA)
• Solu<on:usegenera<veprogrammingandcompilerframeworkLMS.Thetechniqueenablesspecializingenginecodewithoutmix
Automatic partial evaluation is a hard problem due to binding-time analysis (BTA).
Solution: start with a binding-time annotated (staged) program, in a multi-level language.
Practical Realization
image credit: Ruby Tahboub (Purdue)
![Page 30: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/30.jpg)
A staged interpreter yields a compiler.
![Page 31: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/31.jpg)
Staging
• multi-level language n | x | e @b e | λbx.e | …
• MetaMLn | x | e e | λx.e | <e> | ~e | run e
• Lightweight Modular Staging (LMS) in Scala driven by types: T vs Rep[T]
![Page 32: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/32.jpg)
A staged interpreter yields a compiler.
![Page 33: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/33.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 34: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/34.jpg)
staged I1
staged I2
staged In-1
…
staged In
Ln
L0
![Page 35: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/35.jpg)
C1
C2
Cn-1
…
Cn
Ln
L0
![Page 36: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/36.jpg)
C1
C2
Cn-1
…
Cn
Ln
L0
![Page 37: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/37.jpg)
C1
C2
Cn-1
…
Cn
Ln
L0
Ln
L0
One-Pass?Compiler
![Page 38: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/38.jpg)
C1
C2
Cn-1
…
Cn
Ln
L0
Ln
L0
One-Pass?Compiler reflection?
![Page 39: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/39.jpg)
staged I1
staged I2
staged In-1
…
staged In
Ln
L0
Ln
L0
One-Pass?Compiler reflection?
![Page 40: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/40.jpg)
Stage Polymorphism
![Page 41: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/41.jpg)
I1
I2
In-1
…
In
Ln
L0
![Page 42: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/42.jpg)
I1
I2
In-1
…
staged In
Ln
L0 base L0 = λ↑↓ = multi-level λ-calculus
![Page 43: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/43.jpg)
I1
I2
In-1
…
staged In
Ln
L0
Ln
One-Pass Compiler
base L0 = λ↑↓ = multi-level λ-calculus
![Page 44: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/44.jpg)
VM
Evaluator
stagedMod. Evaluator
Low Level
High Level
Evaluator
…
High Level
I1
I2
In-1
staged In
Python
ARM CPU
JavaScript VM
x86 runtime
bytecode
VM
Evaluator
stagedRegexp Matcher
Low Level
High Level
Evaluator
…
Regexp High Level
Low Level
One-Pass Compiler
(/ Transformer)
Multi-Level
![Page 45: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/45.jpg)
I1
I2
In-1
…
staged In
Ln
L0
Ln
One-Pass Compiler
base L0 = λ↑↓ = multi-level λ-calculus
![Page 46: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/46.jpg)
Pink & PurpleTowers of Interpreters
• abstract over interpretation & compilation: stage-polymorphic multi-level lambda-calculus λ↑↓ or LMS & polytypic programming via type classes
• control collapse: explicit lifting or compilation unit
• tower size: finite or conceptually infinite
• reflection, mutation, …
![Page 47: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/47.jpg)
λ↑↓• Multi-level λ-calculus
• Lift operator
• Let-insertion
• Stage polymorphism
• Akin to manual online partial evaluation
![Page 48: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/48.jpg)
Definitional Interpreter in Scala (or Scheme)
![Page 49: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/49.jpg)
// Multi-stage evaluation
def evalms(env: Env, e: Exp): Val = e match { case Lit(n) => Cst(n) case Var(n) => env(n) case Cons(e1,e2) => Tup(evalms(env,e1),evalms(env,e2)) case Lam(e) => Clo(env,e) case Let(e1,e2) => val v1 = evalms(env,e1); evalms(env:+v1,e2) case App(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Clo(env3,e3), v2) => evalms(env3:+Clo(env3,e3):+v2,e3) case (Code(s1), Code(s2)) => reflectc(App(s1,s2)) } case If(c,a,b) => evalms(env,c) match { case Cst(n) => if (n != 0) evalms(env,a) else evalms(env,b) case Code(c1) => reflectc(If(c1, reifyc(evalms(env,a)), reifyc(evalms(env,b)))) } case IsNum(e1) => evalms(env,e1) match { case Code(s1) => reflectc(IsNum(s1)) case Cst(n) => Cst(1) case v => Cst(0) } case Plus(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Cst(n1), Cst(n2)) => Cst(n1+n2) case (Code(s1),Code(s2)) => reflectc(Plus(s1,s2)) } … case Lift(e) => liftc(evalms(env,e)) case Run(b,e) => evalms(env,b) match { case Code(b1) => reflectc(Run(b1, reifyc(evalms(env,e)))) case _ => evalmsg(env, reifyc({ stFresh = env.length; evalms(env, e) })) }
![Page 50: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/50.jpg)
Stage Polymorphism
![Page 51: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/51.jpg)
// Multi-stage evaluation
def evalms(env: Env, e: Exp): Val = e match { case Lit(n) => Cst(n) case Var(n) => env(n) case Cons(e1,e2) => Tup(evalms(env,e1),evalms(env,e2)) case Lam(e) => Clo(env,e) case Let(e1,e2) => val v1 = evalms(env,e1); evalms(env:+v1,e2) case App(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Clo(env3,e3), v2) => evalms(env3:+Clo(env3,e3):+v2,e3) case (Code(s1), Code(s2)) => reflectc(App(s1,s2)) } case If(c,a,b) => evalms(env,c) match { case Cst(n) => if (n != 0) evalms(env,a) else evalms(env,b) case Code(c1) => reflectc(If(c1, reifyc(evalms(env,a)), reifyc(evalms(env,b)))) } case IsNum(e1) => evalms(env,e1) match { case Code(s1) => reflectc(IsNum(s1)) case Cst(n) => Cst(1) case v => Cst(0) } case Plus(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Cst(n1), Cst(n2)) => Cst(n1+n2) case (Code(s1),Code(s2)) => reflectc(Plus(s1,s2)) } … case Lift(e) => liftc(evalms(env,e)) case Run(b,e) => evalms(env,b) match { case Code(b1) => reflectc(Run(b1, reifyc(evalms(env,e)))) case _ => evalmsg(env, reifyc({ stFresh = env.length; evalms(env, e) })) }
![Page 52: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/52.jpg)
// Multi-stage evaluation
def evalms(env: Env, e: Exp): Val = e match { case Lit(n) => Cst(n) case Var(n) => env(n) case Cons(e1,e2) => Tup(evalms(env,e1),evalms(env,e2)) case Lam(e) => Clo(env,e) case Let(e1,e2) => val v1 = evalms(env,e1); evalms(env:+v1,e2) case App(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Clo(env3,e3), v2) => evalms(env3:+Clo(env3,e3):+v2,e3) case (Code(s1), Code(s2)) => reflectc(App(s1,s2)) } case If(c,a,b) => evalms(env,c) match { case Cst(n) => if (n != 0) evalms(env,a) else evalms(env,b) case Code(c1) => reflectc(If(c1, reifyc(evalms(env,a)), reifyc(evalms(env,b)))) } case IsNum(e1) => evalms(env,e1) match { case Code(s1) => reflectc(IsNum(s1)) case Cst(n) => Cst(1) case v => Cst(0) } case Plus(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Cst(n1), Cst(n2)) => Cst(n1+n2) case (Code(s1),Code(s2)) => reflectc(Plus(s1,s2)) } … case Lift(e) => liftc(evalms(env,e)) case Run(b,e) => evalms(env,b) match { case Code(b1) => reflectc(Run(b1, reifyc(evalms(env,e)))) case _ => evalmsg(env, reifyc({ stFresh = env.length; evalms(env, e) })) }
![Page 53: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/53.jpg)
// Multi-stage evaluation
def evalms(env: Env, e: Exp): Val = e match { case Lit(n) => Cst(n) case Var(n) => env(n) case Cons(e1,e2) => Tup(evalms(env,e1),evalms(env,e2)) case Lam(e) => Clo(env,e) case Let(e1,e2) => val v1 = evalms(env,e1); evalms(env:+v1,e2) case App(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Clo(env3,e3), v2) => evalms(env3:+Clo(env3,e3):+v2,e3) case (Code(s1), Code(s2)) => reflectc(App(s1,s2)) } case If(c,a,b) => evalms(env,c) match { case Cst(n) => if (n != 0) evalms(env,a) else evalms(env,b) case Code(c1) => reflectc(If(c1, reifyc(evalms(env,a)), reifyc(evalms(env,b)))) } case IsNum(e1) => evalms(env,e1) match { case Code(s1) => reflectc(IsNum(s1)) case Cst(n) => Cst(1) case v => Cst(0) } case Plus(e1,e2) => (evalms(env,e1), evalms(env,e2)) match { case (Cst(n1), Cst(n2)) => Cst(n1+n2) case (Code(s1),Code(s2)) => reflectc(Plus(s1,s2)) } … case Lift(e) => liftc(evalms(env,e)) case Run(b,e) => evalms(env,b) match { case Code(b1) => reflectc(Run(b1, reifyc(evalms(env,e)))) case _ => evalmsg(env, reifyc({ stFresh = env.length; evalms(env, e) })) }
![Page 54: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/54.jpg)
Liftdef lift(v: Val): Exp = v match { case Cst(n) => Lit(n) case Tup(a,b) => val (Code(u),Code(v))=(a,b); reflect(Cons(u,v)) case Clo(env2,e2) => reflect(Lam(reifyc(evalms( env2:+Code(fresh()):+Code(fresh()),e2)))) case Code(e) => reflect(Lift(e)) }
def liftc(v: Val) = Code(lift(v))
![Page 55: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/55.jpg)
def lift(v: Val): Exp = v match { case Cst(n) => Lit(n) case Tup(a,b) => val (Code(u),Code(v))=(a,b); reflect(Cons(u,v)) case Clo(env2,e2) => reflect(Lam(reifyc(evalms( env2:+Code(fresh()):+Code(fresh()),e2)))) case Code(e) => reflect(Lift(e)) }
def liftc(v: Val) = Code(lift(v))
Lift
![Page 56: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/56.jpg)
Liftdef lift(v: Val): Exp = v match { case Cst(n) => Lit(n) case Tup(a,b) => val (Code(u),Code(v))=(a,b); reflect(Cons(u,v)) case Clo(env2,e2) => reflect(Lam(reifyc(evalms( env2:+Code(fresh()):+Code(fresh()),e2)))) case Code(e) => reflect(Lift(e)) }
def liftc(v: Val) = Code(lift(v))
![Page 57: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/57.jpg)
Liftdef lift(v: Val): Exp = v match { case Cst(n) => Lit(n) case Tup(a,b) => val (Code(u),Code(v))=(a,b); reflect(Cons(u,v)) case Clo(env2,e2) => reflect(Lam(reifyc(evalms( env2:+Code(fresh()):+Code(fresh()),e2)))) case Code(e) => reflect(Lift(e)) }
def liftc(v: Val) = Code(lift(v))
![Page 58: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/58.jpg)
var stFresh: Int = 0
var stBlock: List[Exp] = Nil
def fresh() = {stFresh += 1; Var(stFresh-1)}
def run[A](f: => A): A = {val sF = stFresh; val sB = stBlock; try f finally {stFresh = sF; stBlock = sB}}
def reify(f: => Exp) = run{stBlock = Nil; val last = f; (stBlock foldRight last)(Let)}
def reflect(s:Exp) = {stBlock :+= s; fresh()}
def reifyc(f: => Val) = reify{val Code(e) = f; e}
def reflectc(s: Exp) = Code(reflect(s))
def reifyv(f: => Val) = run{stBlock = Nil; val res = f; if (stBlock == Nil) res else { val Code(last) = res; Code((stBlock foldRight last)(Let))}}
API for Let-Insertion
![Page 59: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/59.jpg)
API for Let-Insertionvar stFresh: Int = 0
var stBlock: List[Exp] = Nil
def fresh() = {stFresh += 1; Var(stFresh-1)}
def run[A](f: => A): A = {val sF = stFresh; val sB = stBlock; try f finally {stFresh = sF; stBlock = sB}}
def reify(f: => Exp) = run{stBlock = Nil; val last = f; (stBlock foldRight last)(Let)}
def reflect(s:Exp) = {stBlock :+= s; fresh()}
def reifyc(f: => Val) = reify{val Code(e) = f; e}
def reflectc(s: Exp) = Code(reflect(s))
def reifyv(f: => Val) = run{stBlock = Nil; val res = f; if (stBlock == Nil) res else { val Code(last) = res; Code((stBlock foldRight last)(Let))}}
![Page 60: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/60.jpg)
var stFresh: Int = 0
var stBlock: List[Exp] = Nil
def fresh() = {stFresh += 1; Var(stFresh-1)}
def run[A](f: => A): A = {val sF = stFresh; val sB = stBlock; try f finally {stFresh = sF; stBlock = sB}}
def reify(f: => Exp) = run{stBlock = Nil; val last = f; (stBlock foldRight last)(Let)}
def reflect(s:Exp) = {stBlock :+= s; fresh()}
def reifyc(f: => Val) = reify{val Code(e) = f; e}
def reflectc(s: Exp) = Code(reflect(s))
def reifyv(f: => Val) = run{stBlock = Nil; val res = f; if (stBlock == Nil) res else { val Code(last) = res; Code((stBlock foldRight last)(Let))}}
API for Let-Insertion
![Page 61: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/61.jpg)
λ↑↓
• Multi-level λ-calculus
• Lift operator
• Let-insertion
• Stage polymorphism
![Page 62: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/62.jpg)
Pink: Stage-Polymorphic
Meta-Circular Evaluator
![Page 63: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/63.jpg)
;; Stage-Polymorphic Meta-Circular Evaluator for Pink
(lambda _ maybe-lift (lambda _ eval (lambda _ exp (lambda _ env (if (num? exp) (maybe-lift exp) (if (sym? exp) (env exp) (if (sym? (car exp)) (if (eq? '+ (car exp)) (+ ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '- (car exp)) (- ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '* (car exp)) (* ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'eq? (car exp)) (eq? ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'if (car exp)) (if ((eval (cadr exp)) env) ((eval (caddr exp)) env) ((eval (cadddr exp)) env)) (if (eq? 'lambda (car exp)) (maybe-lift (lambda f x ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) f (if (eq? y (caddr exp)) x (env y))))))) (if (eq? 'let (car exp)) (let x ((eval (caddr exp)) env) ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) x (env y))))) (if (eq? 'lift (car exp)) (lift ((eval (cadr exp)) env)) (if (eq? 'run (car exp)) (run ((eval (cadr exp)) env) ((eval (caddr exp))env)) (if (eq? 'num? (car exp)) (num? ((eval (cadr exp)) env)) (if (eq? 'sym? (car exp)) (sym? ((eval (cadr exp)) env)) (if (eq? 'car (car exp)) (car ((eval (cadr exp)) env)) (if (eq? 'cdr (car exp)) (cdr ((eval (cadr exp)) env)) (if (eq? 'cons (car exp)) (maybe-lift (cons ((eval (cadr exp)) env) ((eval (caddr exp)) env))) (if (eq? 'quote (car exp)) (maybe-lift (cadr exp)) ((env (car exp)) ((eval (cadr exp)) env))))))))))))))))) (((eval (car exp)) env) ((eval (cadr exp)) env)))))))))
![Page 64: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/64.jpg)
;; Stage-Polymorphic Meta-Circular Evaluator for Pink
(lambda _ maybe-lift (lambda _ eval (lambda _ exp (lambda _ env (if (num? exp) (maybe-lift exp) (if (sym? exp) (env exp) (if (sym? (car exp)) (if (eq? '+ (car exp)) (+ ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '- (car exp)) (- ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '* (car exp)) (* ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'eq? (car exp)) (eq? ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'if (car exp)) (if ((eval (cadr exp)) env) ((eval (caddr exp)) env) ((eval (cadddr exp)) env)) (if (eq? 'lambda (car exp)) (maybe-lift (lambda f x ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) f (if (eq? y (caddr exp)) x (env y))))))) (if (eq? 'let (car exp)) (let x ((eval (caddr exp)) env) ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) x (env y))))) (if (eq? 'lift (car exp)) (lift ((eval (cadr exp)) env)) (if (eq? 'run (car exp)) (run ((eval (cadr exp)) env) ((eval (caddr exp))env)) (if (eq? 'num? (car exp)) (num? ((eval (cadr exp)) env)) (if (eq? 'sym? (car exp)) (sym? ((eval (cadr exp)) env)) (if (eq? 'car (car exp)) (car ((eval (cadr exp)) env)) (if (eq? 'cdr (car exp)) (cdr ((eval (cadr exp)) env)) (if (eq? 'cons (car exp)) (maybe-lift (cons ((eval (cadr exp)) env) ((eval (caddr exp)) env))) (if (eq? 'quote (car exp)) (maybe-lift (cadr exp)) ((env (car exp)) ((eval (cadr exp)) env))))))))))))))))) (((eval (car exp)) env) ((eval (cadr exp)) env)))))))))
![Page 65: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/65.jpg)
;; Stage-Polymorphic Meta-Circular Evaluator for Pink
(lambda _ maybe-lift (lambda _ eval (lambda _ exp (lambda _ env (if (num? exp) (maybe-lift exp) (if (sym? exp) (env exp) (if (sym? (car exp)) (if (eq? '+ (car exp)) (+ ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '- (car exp)) (- ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '* (car exp)) (* ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'eq? (car exp)) (eq? ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'if (car exp)) (if ((eval (cadr exp)) env) ((eval (caddr exp)) env) ((eval (cadddr exp)) env)) (if (eq? 'lambda (car exp)) (maybe-lift (lambda f x ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) f (if (eq? y (caddr exp)) x (env y))))))) (if (eq? 'let (car exp)) (let x ((eval (caddr exp)) env) ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) x (env y))))) (if (eq? 'lift (car exp)) (lift ((eval (cadr exp)) env)) (if (eq? 'run (car exp)) (run ((eval (cadr exp)) env) ((eval (caddr exp))env)) (if (eq? 'num? (car exp)) (num? ((eval (cadr exp)) env)) (if (eq? 'sym? (car exp)) (sym? ((eval (cadr exp)) env)) (if (eq? 'car (car exp)) (car ((eval (cadr exp)) env)) (if (eq? 'cdr (car exp)) (cdr ((eval (cadr exp)) env)) (if (eq? 'cons (car exp)) (maybe-lift (cons ((eval (cadr exp)) env) ((eval (caddr exp)) env))) (if (eq? 'quote (car exp)) (maybe-lift (cadr exp)) ((env (car exp)) ((eval (cadr exp)) env))))))))))))))))) (((eval (car exp)) env) ((eval (cadr exp)) env)))))))))
![Page 66: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/66.jpg)
;; Stage-Polymorphic Meta-Circular Evaluator for Pink
(lambda _ maybe-lift (lambda _ eval (lambda _ exp (lambda _ env (if (num? exp) (maybe-lift exp) (if (sym? exp) (env exp) (if (sym? (car exp)) (if (eq? '+ (car exp)) (+ ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '- (car exp)) (- ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '* (car exp)) (* ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'eq? (car exp)) (eq? ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'if (car exp)) (if ((eval (cadr exp)) env) ((eval (caddr exp)) env) ((eval (cadddr exp)) env)) (if (eq? 'lambda (car exp)) (maybe-lift (lambda f x ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) f (if (eq? y (caddr exp)) x (env y))))))) (if (eq? 'let (car exp)) (let x ((eval (caddr exp)) env) ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) x (env y))))) (if (eq? 'lift (car exp)) (lift ((eval (cadr exp)) env)) (if (eq? 'run (car exp)) (run ((eval (cadr exp)) env) ((eval (caddr exp))env)) (if (eq? 'num? (car exp)) (num? ((eval (cadr exp)) env)) (if (eq? 'sym? (car exp)) (sym? ((eval (cadr exp)) env)) (if (eq? 'car (car exp)) (car ((eval (cadr exp)) env)) (if (eq? 'cdr (car exp)) (cdr ((eval (cadr exp)) env)) (if (eq? 'cons (car exp)) (maybe-lift (cons ((eval (cadr exp)) env) ((eval (caddr exp)) env))) (if (eq? 'quote (car exp)) (maybe-lift (cadr exp)) ((env (car exp)) ((eval (cadr exp)) env))))))))))))))))) (((eval (car exp)) env) ((eval (cadr exp)) env)))))))))
![Page 67: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/67.jpg)
;; Stage-Polymorphic Meta-Circular Evaluator for Pink
(lambda _ maybe-lift (lambda _ eval (lambda _ exp (lambda _ env (if (num? exp) (maybe-lift exp) (if (sym? exp) (env exp) (if (sym? (car exp)) (if (eq? '+ (car exp)) (+ ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '- (car exp)) (- ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? '* (car exp)) (* ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'eq? (car exp)) (eq? ((eval (cadr exp)) env) ((eval (caddr exp)) env)) (if (eq? 'if (car exp)) (if ((eval (cadr exp)) env) ((eval (caddr exp)) env) ((eval (cadddr exp)) env)) (if (eq? 'lambda (car exp)) (maybe-lift (lambda f x ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) f (if (eq? y (caddr exp)) x (env y))))))) (if (eq? 'let (car exp)) (let x ((eval (caddr exp)) env) ((eval (cadddr exp)) (lambda _ y (if (eq? y (cadr exp)) x (env y))))) (if (eq? 'lift (car exp)) (lift ((eval (cadr exp)) env)) (if (eq? 'run (car exp)) (run ((eval (cadr exp)) env) ((eval (caddr exp))env)) (if (eq? 'num? (car exp)) (num? ((eval (cadr exp)) env)) (if (eq? 'sym? (car exp)) (sym? ((eval (cadr exp)) env)) (if (eq? 'car (car exp)) (car ((eval (cadr exp)) env)) (if (eq? 'cdr (car exp)) (cdr ((eval (cadr exp)) env)) (if (eq? 'cons (car exp)) (maybe-lift (cons ((eval (cadr exp)) env) ((eval (caddr exp)) env))) (if (eq? 'quote (car exp)) (maybe-lift (cadr exp)) ((env (car exp)) ((eval (cadr exp)) env))))))))))))))))) (((eval (car exp)) env) ((eval (cadr exp)) env)))))))))
![Page 68: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/68.jpg)
Pink Interpretation
(define eval ((lambda ev e (((eval-poly (lambda _ e e)) ev) e)) #nil)))
(define fac-src (quote (lambda f n (if (eq? n 0) 1 (* n (f (- n 1)))))))
> ((eval fac-src) 4) ;=> 24
![Page 69: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/69.jpg)
Double & Triple Pink Interpretation
> ((eval fac-src) 4)
> (((eval eval-src) fac-src) 4)
> ((((eval eval-src) eval-src) fac-src) 4)
;=>24
![Page 70: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/70.jpg)
Pink Compilation
(define evalc ((lambda ev e (((eval-poly (lambda _ e (lift e))) ev) e)) #nil)))
(define fac-src (quote (lambda f n (if (eq? n 0) 1 (* n (f (- n 1)))))))
> (evalc fac-src) ;=> <code of fac>
![Page 71: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/71.jpg)
Pink Collapsing
> (evalc fac-src)
> ((eval evalc-src) fac-src)
> ((eval eval-src) evalc-src) fac-src)
;=> <code of fac>
![Page 72: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/72.jpg)
I1
I2
In-1
…
staged In
Ln
L0
Ln
One-Pass Compiler
base L0 = λ↑↓ = multi-level λ-calculus
![Page 73: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/73.jpg)
eval
eval-src
eval-src
…
evalc-src
fac-src
λ↑↓
fac-src
evalc
<code of fac>
> (((eval eval-src) … eval-src)
evalc-src) fac-src)
;=> <code of fac>
![Page 74: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/74.jpg)
eval
eval-src
eval-src
…
modifiedevalc-src
fac-src
λ↑↓
fac-src
modified evalc
= transformer
<modified code of fac>
> (((eval eval-src) … eval-src)
modified-evalc-src) fac-src)
;=> <code of fac>
![Page 75: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/75.jpg)
Pink Transformers> (evalc fac-src) ;; => (lambda f0 x1
(let x2 (eq? x1 0)
(if x2 1
(let x3 (- x1 1)
(let x4 (f0 x3)
(* x1 x4))))))
> (trace-n-evalc fac-src) ;; => (lambda f0 x1
(let x2 (log 0 x1)
(let x3 (eq? x2 0)
(if x3 1
(let x4 (log 0 x1)
(let x5 (log 0 x1)
(let x6 (- x5 1)
(let x7 (f0 x6)
(* x4 x7)))))))))
> (cps-evalc fac-src) ;; => (lambda f0 x1 (lambda f2 x3
(let x4 (eq? x1 0)
(if x4 (x3 1)
(let x5 (- x1 1)
(let x6 (f0 x5)
(let x7 (lambda f7 x8
(let x9 (* x1 x8) (x3 x9)))
(x6 x7))))))))
![Page 76: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/76.jpg)
Purple• unit of compilation: lambda becomes clambda
• conceptually infinite reflective tower based on Black
• Lightweight Modular Staging (LMS)
• roughly akin to manual offline partial evaluation
• staged polymorphism through type classes: Rep[T] vs NoRep[T] (= T) abstracted as R[T]
![Page 77: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/77.jpg)
Recall: Collapse in Purple{(k, xs) => _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_1 => _cell_set(<cell counter>, c_1) _app(‘<, _cons(_car(xs), ‘(2)), _cont{v_1 => _if(_true(v_1),_app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_2 => _cell_set(<cell counter>, c_2)_app(k, _cons(_car(xs), ‘()), _cont{v_2 => v_2})}), _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_3 => _cell_set(<cell counter>, c_3) _app(‘-, _cons(_car(xs), ‘(1)), _cont{v_3 => _app(_cell_read(<cell fib>), _cons(v_3, ‘()), _cont{v_4 => _app(‘+, _cons(_cell_read(<cell counter>), ‘(1)), _cont{c_4 => _cell_set(<cell counter>, c_4) _app(‘-, _cons(_car(xs), ‘(2)), _cont{v_5 => _app(_cell_read(<cell fib>), _cons(v_5, ‘()), _cont{v_6 =>_app(‘+, _cons(v_4, _cons(v_6, ‘())), _cont{v_7 => _app(k, _cons(v_7, ‘()), _cont{v_8 => v_8})})})})})})})}))})})}
![Page 78: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/78.jpg)
trait Ops[R[_]] { implicit def _lift(v: Value): R[Value] def _liftb(b: Boolean): R[Boolean]
def _app(fun: R[Value], args: R[Value], cont: Value): R[Value] def _fun(f: Fun[R]): R[Value] def _cont(f: FunC[R]): Value
def _true(v: R[Value]): R[Boolean] def _if(c: R[Boolean], a: =>R[Value], b: =>R[Value]): R[Value]
def _cons(car:R[Value],cdr:R[Value]): R[Value] def _car(p: R[Value]): R[Value] def _cdr(p: R[Value]): R[Value]
def _cell_new(v: R[Value], memo: String): R[Value] def _cell_set(c: R[Value], v: R[Value]): R[Value] def _cell_read(c: R[Value]): R[Value]
def inRep: Boolean }
![Page 79: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/79.jpg)
“Classic” vs Rigid Structures for Reflective Towers
user levelCPU 0
meta levelCPU 1
meta… levelCPU n
user level
Global CPU
meta level
meta… level
env
env
env
env
env
env
Black Purple
![Page 80: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/80.jpg)
Purple: Modified Semantics• (EM (begin (define counter 0) (define old-eval-var eval-var) (set! eval-var (clambda (e r k) (if (eq? e 'n) (set! counter (+ counter 1))) (old-eval-var e r k)))))
• > (fib 7) ;=> 13> (EM counter) ;=> 102
• (set! fib (clambda (n) …))(EM (set! counter 0))> (fib 7) ;=> 13> (EM counter) ;=> 102
![Page 81: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/81.jpg)
Purple: User-Level “Interpreter”
• (define matches (clambda (r) (clambda (s) (if (null? r) #t (if (null? s) #f (if (eq? (car r) (car s)) ((matches (cdr r)) (cdr s)) #f))))))
• > ((matches '(a b)) '(a c)) ;=> #f
• > ((matches '(a b)) '(a b)) ;=> #t
• > ((matches '(a b)) '(a b c)) ;=> #t
![Page 82: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/82.jpg)
Purple: User-Level Collapse
• (define start_ab ((clambda () (matches '(a b)))))
• (define start_ab (clambda (s) (if (null? s) #f (if (eq? 'a (car s)) ((clambda (s) (if (null? s) #f (if (eq? 'b (car s)) ((clambda (s) #t) (cdr s)) #f))) (cdr s))))))
![Page 83: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/83.jpg)
Summary• collapse towers of interpreters
• using a stage-polymorphic multi-level lambda-calculus
• using LMS and polytypic programming via type classes
• design: how to expose compilation/collapse?
• explicit stage lifting
• clambda
• JIT? …
![Page 84: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/84.jpg)
Potential Applications• Towers in the Wild: e.g. Python on top of x86
runtime on top of JavaScript VM
• Modified semantics: e.g. instrumentation/tracing for debugging, sandboxing for security, virtualization, transactions
• Non-standard interpretations: program analysis, verification, synthesis, e.g. Racket interpreter on top of miniKanren, Abstracting abstract machines
![Page 85: Collapsing Towers of Interpreters · The Challenge Collapse a tower of interpreters (languages L0, …, Ln & interpreters for Li+1 written in Li) into a one-pass compiler from Ln](https://reader035.vdocuments.mx/reader035/viewer/2022071100/5fd975d73d282c5b402220c0/html5/thumbnails/85.jpg)
Code for Pink & Purple: popl18.namin.net
Reflective towers: … Brown, Blond, Black, …
Staging & LMS: scala-lms.github.io
Stage polymorphism: github.com/GeorgOfenbeck/SpaceTime
Thank you!