algol and haskellmsagiv/courses/pl17/lambda.pdf · 3 historical context like alan turing, another...

Lambda Calculus

Oded Padon & Mooly Sagiv

(original slides by Kathleen Fisher, John Mitchell, Shachar Itzhaky, S. Tanimoto )

Benjamin Pierce

Types and Programming Languages

http://www.cs.cornell.edu/courses/cs3110/2008fa/recitations/rec26.html

Computation Models

• Turing Machines

• Wang Machines

• Counter Programs

• Lambda Calculus

3

Historical Context

Like Alan Turing, another mathematician, Alonzo Church, was very

interested, during the 1930s, in the question “What is a computable

function?”

He developed a formal system known as the pure lambda calculus, in

order to describe programs in a simple and precise way

Today the Lambda Calculus serves as a mathematical foundation for

the study of functional programming languages, and especially for the

study of “denotational semantics.”

Reference: http://en.wikipedia.org/wiki/Lambda_calculus

What is calculus

• A complete computational model

• An assembly language for functional programming

– Powerful

– Concise

– Counterintuitive

• Can explain many interesting PL features

Basics

• Repetitive expressions can be compactly represented using functional abstraction

• Example:

– (5 * 4* 3 * 2 * 1) + (7 * 6 * 5 * 4 * 3 * 2 *1) =

– factorial(5) + factorial(7)

– factorial(n) = if n = 0 then 1 else n * factorial(n-1)

– factorial= n. if n = 0 then 0 else n * factorial(n-1)

– factorial= n. if n = 0 then 0 else n * apply (factorial(n-1))

Untyped Lambda Calculus

t ::= terms

x variable

x. t abstraction

t t application

Syntactic Conventions

• Applications associates to left

e1 e2 e3 (e1 e2) e3

• The body of abstraction extends as far as possible

• x. y. x y x x. (y. (x y) x)

Terms can be represented as abstract syntax trees

apply

apply

e1 e2 e3

Untyped Lambda Calculus

t ::= terms

x variable

x. t abstraction

t t application

Syntactic Conventions

• Applications associates to left

e1 e2 e3 (e1 e2) e3

• The body of abstraction extends as far

as possible

• x. y. x y x x. (y. (x y) x)

Terms can be represented as abstract syntax trees

x

y

apply

apply

x y x

Example Lambda Expressions

• x.”1”

• x.x

• x.y

• x.s x

• x.s (s x)

• f. g. f g

• b. “if” b “fls” “tru”

• b. b’. “if” b b’ “fls”

Lambda Calculus in Python

( x. x) y (lambda x: x) (y)

Substitution

• Replace a term by a term

– x + ((x + 2) * y)[x ↦ 3, y ↦ 7] = ?

– x + ((x + 2) * y)[x ↦ z + 2] = ?

– x + ((x + 2) * y)[t ↦ z + 2] = ?x + ((x + 2) * y)[x ↦ y]

– More tricky in programming languages

– Why?

Free vs. Bound Variables

• An occurrence of x is bound in t if it occurs in x. t – otherwise it is free– x is a binder

• Examples– Id= x. x – y. x (y z)– z. x. y. x (y z)– (x. x) x

FV: t 2Var is the set free variables of t

FV(x) = {x}

FV( x. t) = FV(t) – {x}

FV (t1 t2) = FV(t1) FV(t2)

Beta-Reduction

[x↦s] x = s

[x↦s] y = y if y x

[x↦s] (y. t1) = y. [x ↦s] t1 if y x and yFV(s)

[x↦s] (t1 t2) = ([x↦s] t1) ([x↦s] t2)

( x. t1) t2 [x ↦t2] t1 (-reduction)

y

t1

[x↦s]y

[x↦s] t1

Beta-Reduction

[x↦s] x = s

[x↦s] y = y if y x

[x↦s] (y. t1) = y. [x ↦s] t1 if y x and yFV(s)

[x↦s] (t1 t2) = ([x↦s] t1) ([x↦s] t2)


apply

t1

[x↦s]

t2 [x↦s] t1

apply

[x↦s] t2

Example Beta-Reduction

( x. t1) t2 [x ↦ t2] t1 (-reduction)

( x. x) y y

redex

apply

x

x y x

[x ↦ y]

y

Example Beta-Reduction (ex 2)


( x. x ( x. x) ) (u r) u r ( x. x)

redex

x

apply

x

x

apply

apply

u r

x

apply

u r

x ↦

apply

x

x

x

apply

x

x

x

apply

u r

x ↦

apply

u r

x ↦

Example Beta-Reduction (ex 3)


( x (w. x w)) (y z) w. y z w

redex

x

apply

w

apply

apply

y z

x

w

apply

y z

x ↦

apply

wx

w

apply

wx

wapply

y z

x ↦

17

Alpha- Conversion

Alpha conversion:

Renaming of a bound variable and its bound occurrences

x.y.y x.z.z

Simple Exercise

• Adding scopes to expressions

• Proposed syntax “let x = t1 in t2”

• Informal meaning:

– all the occurrences of x in t2 are replaced by t1

• Example: let a = x. (w. x w) in a a =

• How can we simulate let?

Divergence


( x.y) (( x.(x x)) ( x.(x x)))

Apply

x

y

Apply

x

Apply

x x

x

Apply

x x

Divergence


Apply

x

Apply

x x

x

Apply

x x

Apply

x

Apply

x x

x

Apply

x x

( x.(x x)) ( x.(x x))

Different Evaluation Orders


( x.y) (( x.(x x)) ( x.(x x)))

Apply

x

y

Apply

x

Apply

x x

x

Apply

x x

Apply

x

y

Apply

x

Apply

x x

x

Apply

x x



( x.y) (( x.(x x)) ( x.(x x)))

Apply

x

y

Apply

x

Apply

x x

x

Apply

x x

y



( x.y) (( x.(x x)) ( x.(x x)))

Apply

x

y

Apply

x

Apply

x x

x

Apply

x x

def f():

while True: pass

def g(x):

return 2

print g(f())



( x. x) ((x. x) (z. (x. x) z)) id (id (z. id z))

Apply

x

x

Apply

z

Apply

z

x

x

x

x

Order of Evaluation

• Full-beta-reduction– All possible orders

• Applicative order call by value (Eager)– Left to right– Fully evaluate arguments before function

• Normal order– The leftmost, outermost redex is always reduced first

• Call by name– Evaluate arguments as needed

• Call by need– Evaluate arguments as needed and store for subsequent usages– Implemented in Haskel

Different Evaluation Orders( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

Call By Value( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

Apply

x w

wy

Apply

z

z

y

y

Apply

z

z

y ⇏

Call By Name (Lazy)( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

y

Apply

z

z

y ⇏

Normal Order( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

y

Apply

z

z

y

y

y⇏

v ::= values

x. t abstraction values

other values

Call-by-value Operational Semantics

t ::= terms

x variable

x. t abstraction

t t application

( x. t1) v2 [x ↦v2] t1 (E-AppAbs)

t1 t’1

t1 t2 t’1 t2(E-APPL1)

t2 t’2v1 t2 v1 t’2

(E-APPL2)

Call By Value( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

Apply

x w

wy

Apply

z

z

y

y

Apply

z

z

y ⇏

Call By Value OS( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

u

u

w

w

w

w

E-AppAbs

t1

v2

Call By Value OS (2)( x. y. ( z. z) y) (( u. u) ( w. w))

Apply

u

u

w

w

w

w

v1 t2

Apply

x Apply

u

u

w

w

y

Apply

z

z

y

Apply

x

y

Apply

z

z

y

w

w

E-Appl2

Call By Value OS (3)( x. y. ( z. z) y) (( u. u) ( w. w))

t1v2

Apply

x

y

Apply

z

z

y

w

w

y

Apply

z

z

y

EAppAbs

Programming in Calculus

• Functions with multiple arguments

• Simulating values

– Tuples

– Booleans

– Numerics

Programming in the Multiple arguments

f= (x, y). s

-> Currying

f= x. y. s

f v w = ((x. y.s) v) w (y.[x ↦v]s) w [x ↦v] [y ↦w] s(f v) w =

((x. y. x*x+y*y) 3) 4 (y.[x ↦3] x*x+y*y) 4 =

3*3+4*4

(y.3*3+y*y) 4

[y ↦4] 3*3+y*y =

Adding Values

• Can be explicitly added

• Can be simulated

t ::= terms

x variable

x. t abstraction

t t application

TT | FF | 1 | 2 | …

Simulating Booleans

• tru = t. f. t

• fls = t. f. f

Simulating Tests

• tru = t. f. t fls = t. f. f

• test = l. m. n. l m n

• test tru then else = (l. m. n. l m n) (t. f. t) then else

t

m

n

Apply

Apply

m

n

Apply Apply

f

t

then else

m

n

Apply

Apply

m

n

l

l

Apply

Apply Apply

f

t

tthen else

value value

Simulating Tests(2)




t

m

n

Apply

Apply

m

n

Apply Apply

f

t

then else

t

n

Apply

Apply

then

n

Apply

f

t

else

Simulating Tests(3)




t

n

Apply

Apply

then

n

Apply

f

t

else

t

Apply

Apply

then

else

f

t

Simulating Tests(4)




t

Apply

Apply

then

else

f

t

Apply

elsethen

f

Simulating Tests(5)




Apply

elsethen

f

then

Simulating Tests

• tru = t. f. t

• fls = t. f. f


• test tru then else = (l. m. n. l m n) (t. f. t)

• test fls then else = (l. m. n. l m n) (t. f. f)

Programming in Booleans

• tru = t. f. t

• fls = t. f. f


• test tru then else = (l. m. n. l m n) (t. f. t)

• test fls then else = (l. m. n. l m n) (t. f. f)

• and = b. b’. test b b’ fls

• or = ?

• not = ?

Programming in Numerals

• c0 = s. z. z• c1 = s. z. s z• c2 = s. z. s (s z)• c3 = s. z. s (s (s z))• succ = n. s. z. s (n s z)• plus = m. n. s. z. m s (n s z)• times = m. n. m (plus n) c0

> Turing Complete

Combinators

• A combinator is a function in the Lambda Calculus having no free variables

• Examples– x. x is a combinator– x. y. (x y) is a combinator– x. y. (x z) is not a combinator

• Combinators can serve nicely as modular building blocks for more complex expressions

• The Church numerals and simulated Booleans are examples of useful combinators

Loops in Lambda Calculus

• omega= (x. x x) (x. x x)

• Recursion can be simulated – Y = (x . (y. x (y y)) (y. x (y y)))

–Y f * f (Y f)

49

Factorial in the Lambda Calculus

Define H as follows, to represent 1 step of recursion.

Note that ISZERO, MULT, and PRED represent particular

combinators that accomplish these functions

H = ( f. n.(ISZERO n) 1 (MULT n (f (PRED n))))

Then we can create

FACTORIAL = Y H

= (x . (y. x (y y)) (y. x (y y))) ( f. n.(ISZERO n) 1 (MULT n (f (PRED n))))

Reference: http://en.wikipedia.org/wiki/Y_combinator

Consistency of Function Application

• Prevent runtime errors during evaluation

• Reject inconsistent terms

• What does ‘x x’ mean?

• Cannot be always enforced

– if <tricky computation> then true else (x. x)

Typed Lambda Calculus

Chapter 9

Benjamin Pierce

Types and Programming Languages

v ::= values

x. t abstraction values

Call-by-value Operational Semantics

( x. t12) v2 [x v2] t12 (E-AppAbs)

t ::= terms

x variable

x. t abstraction

t t application

t1 t’1

t1 t2 t’1 t2

(E-APPL1)

t2 t’2

v1 t2 v1 t’2

(E-APPL2)

Consistency of Function Application

• Prevent runtime errors during evaluation

• Reject inconsistent terms

• What does ‘x x’ mean?

• Cannot be always enforced

– if <tricky computation> then true else (x. x)

A Naïve Attempt

• Add function type

• Type rule x. t :

– x. x :

– If true then (x. x) else (x. y y) :

• Too Coarse

Simple Types

T ::= typesBool type of BooleansT T type of functions

T1 T2 T3 = T1 ( T2 T3)

Explicit vs. Implicit Types

• How to define the type of abstractions?

– Explicit: defined by the programmer

– Implicit: Inferred by analyzing the body

• The type checking problem: Determine if typed term is well typed

• The type inference problem: Determine if there exists a type for (an untyped) term which makes it well typed

t ::= Type terms

x variable

x: T. t abstraction

t t application

Simple Typed Lambda Calculus

t ::= terms

x variable

x: T. t abstraction

t t application

T::= types

T T types of functions

x : T1 t2 : T2

(x : T1. t2 ): T1 T2

Typing Function Declarations

A typing context maps free variables into types

(T-ABS)

, x : T1 t2 : T2

( x : T1 . t2 ): T1 T2

(T-ABS)

Typing Free Variables

x : T

x : T(T-VAR)

Typing Function Applications

t1 : T11 T12 t2 : T11

t1 t2 : T12

(T-APP)

Typing Conditionals

t1 : Bool t2 : T t3 : T

if t1 then t2 else t3 : T(T-IF)

if true then (x: Bool. x) else (y: Bool. not y)

t1 t2 t’1 t2

SOS for Simple Typed Lambda Calculus

t ::= terms

x variable

x: T. t abstraction

t t application

v::= values

x: T. t abstraction values

T::= types


t1 t2

t1 t’1(E-APP1)

t2 t’2

v1 t2 v1 t’2

(E-APP2)

( x: T11. t12) v2 [x v2] t12 (E-APPABS)

t ::= terms

x variable

x: T. t abstraction

T::= types


::= context

empty context

, x : T term variable binding

t : T

x : T

x : T(T-VAR)

, x : T1 t2 : T2

x : T1. t2 : T1 T2

(T-ABS)

t1 : T11 T12

t1 t2 : T12

(T-APP) t2 : T11

Type Rules

t ::= terms

x variable

x: T. t abstraction

t t application

true constant true

false constant false

if t then t else t conditional

T::= types

Bool Boolean type


::= context

empty context

, x : T term variable binding

t : T

x : T

x : T(T-VAR)

, x : T1 t2 : T2

x : T1. t2 : T2 : T1 T2

(T-ABS)

t1 : T11 T12

t1 t2 : T12

(T-APP) t2 : T11

true : Bool (T-TRUE)

false : Bool (T-FALSE)

t1 : Bool t2 : T t3 : T


Examples

• )x:Bool. x ) true

• if true then )x:Bool. x) else ) x:Bool. x)

• if true then )x:Bool. x) else ) x:Bool. y:Bool. x)

Simple Example

)x:Bool. x ) true

t : T

x : T

x : T(T-VAR)

, x : T1 t2 : T2

x : T1. t2 : T2 : T1 T2

(T-ABS)

t1 : T11 T12

t1 t2 : T12

(T-APP) t2 : T11



t1 : Bool t2 : T t3 : T


Another Example

if true then )x:Bool. x) else ) x:Bool. x)

t : T

x : T

x : T(T-VAR)

, x : T1 t2 : T2

x : T1. t2 : T2 : T1 T2

(T-ABS)

t1 : T11 T12

t1 t2 : T12

(T-APP) t2 : T11



t1 : Bool t2 : T t3 : T


Another Example

if true then )x:Bool. x) else

x:Bool. y:Bool. x)

t : T

x : T

x : T(T-VAR)

, x : T1 t2 : T2

x : T1. t2 : T2 : T1 T2

(T-ABS)

t1 : T11 T12

t1 t2 : T12

(T-APP) t2 : T11



t1 : Bool t2 : T t3 : T


The Typing Relation

• Formally the typing relation is the smallest ternary relation on contexts, terms and types

– in terms of inclusion

• A term t is typable in a given context (well typed) if there exists some type T such that t : T

• Interesting on closed terms (empty contexts)

Inversion of the typing relation

• x : R x: R

• x : T1. t2 : R R = T1 R2 for some R2 with t2 : R2

• t1 t2 : R there exists T11 such that t1 : T11 R and t2 : T11

• true : R R = Bool

• false : R R = Bool

• if t1 then t2 else t3 : R t1: Bool, t2 : R, t3: R

Uniqueness of Types

• Each term t has at most one type in any given context

– If t is typable then • its type is unique

• There is a unique type derivation tree for t

Type Safety

• Well typed programs cannot go wrong

• If t is well typed then either t is a value or there exists an evaluation step t t’ [Progress]

• If t is well typed and there exists an evaluation step t t’ then t’ is also well typed [Preservation]

Canonical Forms

• If v is a value of type Bool then v is either true or false

• If v is a value of type T1 T2 then v= x: T1.t2

Progress Theorem

• Does not hold on terms with free variables

• For every closed well typed term t, either t is a value or there exists t’ such that t t’

Preservation Theorem

• If t : T and is a permutation of then t : T [Permutation]

• If t : T and x dom() then ,t t : T with a proof of the same depth [Weakening]

• If , x: S t : T and s: Sthen [x s] t : T [Preservation of types under substitution]

• t : T and t t’ then t’ : T

t1 t2 t’1 t2

SOS for Simple Typed Lambda Calculus

t ::= terms

x variable

x: T. t abstraction

t t application

v::= values

x: T. t abstraction values

T::= types


t1 t2

t1 t’1(E-APP1)

t2 t’2

v1 t2 v1 t’2

(E-APP2)

( x: T11. t12) v2 [x v2] t12 (E-APPABS)

Erasure and Typability

• Types are used for preventing errors and generating more efficient code

• Types are not used at runtime

• If t t’ under typed evaluation relation, then erase(t) erase(t’)

• A term t in the untyped lamba calculus is typable if there exists a typed term t’ such that erase(t’) = t

erase(x) = xerase(x: T1. t2) = x.erase(t2)erase(t1 t2) = erase(t1) erase(t2)

Summary

• Constructive rules for preventing runtime errors in a Turing complete programming language

• Efficient type checking

• Unique types

• Type safety

• But limits programming

Summary: Lambda Calculus

• Powerful• The ultimate assembly language• Useful to illustrate ideas• But can be counterintuitive• Usually extended with useful syntactic sugars• Other calculi exist

– pi-calculus– object calculus– mobile ambients– …

algol and haskellmsagiv/courses/pl17/lambda.pdf · 3 historical context like alan turing, another...

Documents