building “real world” software in academia matthias felleisen plt, rice university

Building “Real World” Software in Academia

Matthias FelleisenPLT, Rice University

July 16, 1999

What is the “Real World”? … the Problem?

The TeachScheme! Project

Problems with Scheme

Building DrScheme

Technical Problems Today: Extensibility or Single Point of Control

July 16, 1999

The TeachScheme! Project

Everyone should learn to program … should do so early … and it should be in Scheme

Scheme programming teaches critical skills: algebraic problem solving

Every student benefits …. … not just the few who continue to program

July 16, 1999

4 + 5 = 4 + 5 = 9 (+ 4 5) (+ 4 5) 9

Program Add Program Add (Output);(Output);

BeginBegin

Writeln Writeln (4 + 5)(4 + 5)

End.End.

<compile> <run><compile> <run>

9

AlgebraAlgebra SchemeScheme PascalPascal

Example #1 Arithmetic

July 16, 1999

f(x) f(x) = 4 + = 4 + xx

( define ( f x ) ( + 4 ( define ( f x ) ( + 4 x )) x ))

Program Program ff (Input, Output) (Input, Output) ;;

VarVar

x x :: Integer Integer ;;

BeginBegin

Readln ( Readln ( xx ) ) ;;

Writeln Writeln ( 4 + x )( 4 + x )

End End ..

AlgebraAlgebra PascalPascal

SchemeScheme

Example #2 A Simple Function

July 16, 1999

The Problems We Are Facing

psychology of teachers, students and parents

sociology of AP, principals, and administrators

a technical problem: Scheme’s syntax isn’t all that simple

July 16, 1999

Scheme’s Syntax is Bad #1

(define (length alist) (cond ((empty? alist) 0) (else 1 + (length (rest alist)))))

July 16, 1999

Emacs #1

July 16, 1999

Scheme’s Syntax is Bad #2

(define (length alist) (cond (null? (alist) 0) (else (+ 1 (length (rest alist))))))

July 16, 1999

Emacs #2

July 16, 1999

Our Solution: DrScheme

a hierarchy of languages each level matches a student’s skill level each level provides matching error messages

… plus a few tools: a syntax checker & alpha renamer a symbolic stepper a type-flow analysis

currently used at around 80 universities/schools by around 8000 to 10000 students

July 16, 1999

d = (define (f v …) e)e = v | (f e …) | (primitive e …) | (cond (e e) …)f = namev = name

d = (define-struct v (v …))e = (local (d …) e)

d = (define v e) | (set! v e) | (cond (e …) …)

The Language Hierarchy (Excerpts)

each level extends the level below

July 16, 1999

DrScheme

July 16, 1999

DrScheme #1

July 16, 1999

DrScheme #2

July 16, 1999

DrScheme Tools: Bindings

July 16, 1999

DrScheme Tools: Alpha Renaming

July 16, 1999

DrScheme Tools: Type Analysis

Building Extensible Software

Matthias FelleisenPLT, Rice University

July 16, 1999

The Technical Problem:

We need extensible software components: extensible parser extensible evaluator …

By developing each language level once and reusing it for all extensions, we save a huge amount of work (and maintenance work)

Principle: Single Point of Control

July 16, 1999

The Technical Solutions:

What does “extensible component” mean?

The programmer’s programmer’s perspective composing patterns composing patterns, again

The programming language perspective

July 16, 1999

Extensible Software Components

Base

Modified Component

cut, paste, edit

Extensible Base

Extension

add / link

Matthias Felleisen:

Matthias Felleisen:

July 16, 1999

Extensible Components are Good

No access to source code needed: critical for a Web-based model of software production & sales

Even if source code is accessible: programming for extensibility forces programmers to create a single point of control for many “functions”

Finally, it increases the reuse potential.

July 16, 1999

Extensibility is Not Guaranteed

Rumor 1: OOPLs guarantee extensibility.

Rumor 2: OO Design patterns do.

Rumor 3: Scheme does it :-)

Krishnamurthi and Felleisen [FSE98]

July 16, 1999

The “Big” Problem for Extensibility

E = var | (lambda (var) E) | (E E)

void f(E some_e) { … }atyp g(E some_e) { … }E h(String s) { … }

btyp m(E some_e) { … }E o(E an_e) { … }

functional extension

| (let (var E) E) | (if E E E)

variant extension

July 16, 1999

Previous Attempts to Solve “It”

Reynolds (76): “It” is a problem …

Guy Steele (93): quasi-monads for interpreters

Cartwright and Felleisen (94): extensible interpreters

Hudak, Jones, and Liang (92-97): extensible “geometry server” for Navy

July 16, 1999

Solution 1: The Magic of Composing Patterns

July 16, 1999

A Closer Look at Objects and Classes

Classes make variant extensions straightforward.

The visitor pattern makes functional extensions straightforward.

Let’s combine these two virtues!

July 16, 1999

e = c | v | (lambda (x ...) e) | (e e) | ...

Datatypes as Class Hierarchies

C on s tan tsc

V ariab lesv

P roced u res(lam b d a (x) e )

A p p lica tion s(e e )

E x p re s s io ns

July 16, 1999

Datatype Visitors

C on s tan tsc

V ariab lesv



E xp re ss io n s

forConstantsforVariables forProceduresforApplications



July 16, 1999

A Concrete Visitor

class Constant extends Expression {

void toVisitor(VisitorIF aVisitor) { aVisitor.forConstants(this) ; }

}

class PrintVisitor implements VisitorIF {

void forConstants(… aConstant) { … }

void forVariables( …aVariable) { … }

void forProcedures(… aProcedure){ … }

void forApplication(… anApplication){ … }

Expression k; … k.toVisitor(new PrintVisitor()); ...

July 16, 1999

A Variant Extension

C on s tan tsc

V ariab lesv



E xp re ss io n s


Expression Conditional (if e e e)

forConditionals

July 16, 1999

So What’s the Problem?

C on s tan tsc

V ariab lesv



E xp re ss io n s


Expression Conditional (if e e e)

forConditionals

new PrintVisitor(…)

The Fix: Use Virtual Constructor Pattern

July 16, 1999

The Solution Imposes

Composing patterns: Interpreter Pattern (natural) Visitor Pattern Virtual Constructor Pattern

Complex interactions between patterns

Numerous run-time type checks

July 16, 1999

Solution 2: More Magic with Patterns

July 16, 1999

Another Closer Look at Objects and Classes

Classes make variant extensions straightforward.

Inheritance can add functionality.

Let’s combine these two virtues!

July 16, 1999

Datatypes and Clients

C on s tan tsc

V ariab lesv



E xp re ss io n s

Parserfor Expression

Expression

July 16, 1999

Adding Functionality to Datatypes

M eth od 1

C on s tan ts

M eth od 1

V ariab les

M eth od 1

P roced u res

M eth od 1

A p p lica tion s

E xp ress ion

July 16, 1999

So What’s the Problem Now?

C on s tan tsc

V ariab lesv



E xp re ss io n s


Expression

Method1 Method1 Method1 Method1

Ouch -- It’s the wrongkind of expression!

July 16, 1999

Linking the Parser via a Factory

C on s tan tsc

V ariab lesv



E xp re ss io n s


Expression

Method1 Method1 Method1 Method1

ExpressionFactory

ExpressionFactory

July 16, 1999

The Second OO Solution Imposes

Composing patterns: interpreter pattern abstract factory virtual constructor (or two AF)

Complex interactions between patterns

Numerous run-time type checks

July 16, 1999

Summary of OO Solutions

Programming for extensibility and for single point of control is feasible in existing OOPLs

But, the code is difficult to produce and maintain.

It suggests a new challenge to language designers.

July 16, 1999

Solution 3: The Language Designer at Work

July 16, 1999

How a Language Designer Can Help:

A programming language should support both

• classes

and

• hierarchical modules

with external connectors.

July 16, 1999

Modules

A

B

C

importsimports

module definitions

exportsexports

July 16, 1999

D

E

A

B

C

Linking and Nesting Modules

July 16, 1999

Classes in Modules

C on s tan tsc

V ariab lesv



E xp re ss io n s

Conditionals(cond (e e) …)

exports: classes, abstract classes, and interfaces

imports: classes, abstract classes, and interfaces

July 16, 1999

Clients of Datatypes

C on s tan tsc

V ariab lesv



E xp re ss io n s

new Constants(…) ; … new Procedure(…)

July 16, 1999

Adding Variants, Again

C on s tan tsc

V ariab lesv



E xp re ss io n s


link to old clients, link to new clients now

July 16, 1999

Adding Functionality, Again

C on s tan tsc

V ariab lesv



E xp re ss io n s


new_method

new_method

new_method

new_method

new_method

link to old clients, link to new clients now

July 16, 1999

Modules are “Natural”

C on s tan tsc

V ariab lesv



E xp re ss io n s


new_method

new_method

new_method

new_method

new_method

… as if God had told us about allpossible extensions ….

July 16, 1999

Lessons for Language Designers

Programmers must be able to connect components through external mechanisms.

Classes must extend interfaces, not fixed classes.

Modules must link to signatures, not fixed modules.

July 16, 1999

Hardware designers have known thisidea for a long time.

Why are language designers behind?

July 16, 1999

“Good” Programming Languages

MzScheme provides modules and classes with external connectors (units and mixins).

Typing for MzScheme-like modules exists Flatt and Felleisen [PLDI98]

OCAML provides modules and classes with external connectors (functors and classes), but types are still in the programmer’s way.

Typing is possible, but OCAML is is to weak

July 16, 1999

The Costs of External Connectors

Layout of objects is unknown due to unknown superclass.

Intermodule optimization of datatypes becomes more difficult.

But, the cost is the nearly same as if we had programmed to an interface (which is what we are supposed to do anyway)

July 16, 1999

Future Work:

Work on more technical details (types, performance)

Work with language designers and “revisers” to affect their technical understanding psychology sociology

(And yes, these are the same problems that we face with TeachScheme)

July 16, 1999

Thank You and Credits

Robby Findler (*) Matthew Flatt (Utah) Cormac Flanagan (DEC SRC) Shriram Krishnamurthi (*)

Dan Friedman (Indiana) Corky Cartwright (Rice)

building “real world” software in academia matthias felleisen plt, rice university

Documents

drscheme slide

program n

simple slide

administrators n

good n

early n

arithmetic slide

project n problems