wildheart.httplistener an open source web server for dyalog apl v11.0

WildHeart.HTTPListeneWildHeart.HTTPListenerr

an open source Web Server an open source Web Server for Dyalog APL v11.0for Dyalog APL v11.0

Stefano“WildHeart”

Lanzavecchia

2006-10-17 Dyalog User Group Conference 2

AgendaAgenda

APL should be a popular language for developing web pages, because:– APL has all you need to generating static and

dynamic web content

It is not used as much as it deserves, because:– Even the best tools need good frameworks to

level the learning curve a bit– No frameworks are readily available

2006-10-17 Dyalog User Group Conference 3

GoalGoal

Explore the problem which needs to be solved and demonstrate a proposed solution... (obviously )

So, what’s the problem?The easy part is generating HTML Hopefully nobody will disagree that APL is

an EXCELLENT language for generating character vectors

2006-10-17 Dyalog User Group Conference 4

A ”web”

Some very simple HTMLSome very simple HTML

2006-10-17 Dyalog User Group Conference 5

The hard partThe hard part

If we have a form like this and the user hits the submit button…

2006-10-17 Dyalog User Group Conference 6

Before we look at the response…Before we look at the response…

<form action="RUN" method="post" name="FrontPage_Form1"><div> <table border="0" cellpadding="3"> <tr> <td>Purchase Amount</td> <td><input type="text" size="9" name="LoanAmt"/></td> </tr> <tr> <td>Percent Down</td> <td><input type="text" size="6" maxlength="6" name="PercentDown"/></td> </tr> <tr> <td>Maximum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMax"/></td> </tr> <tr> <td>Minimum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMin"/></td> </tr>

… continued on the right …

<tr>

<td>Maximum Interest Rate (%)</td>

<td><select name="IntrMax" size="1">

<option>15.0</option>

<option>14.5</option>

..etc down to…

<option>1.0</option></select></td> </tr><tr> <td>Minimum Interest Rate (%)</td> <td><select name="IntrMin" size="1"> <option>15.0</option> <option>14.5</option> <option>14.0</option> <option>13.5</option> <option selected="selected">13.0</option> <option>12.5</option> <option>12.0</option>

…etc… <option>1.5</option>

<option>1.0</option> </select></td> </tr> </table> <p><input type="submit" value="Calculate

Repayments"/></p></div></form>

Some of you will want to knowhow the HTML looks for the form?

2006-10-17 Dyalog User Group Conference 7

When the user hits the button…When the user hits the button…

POST /loan/RUN HTTP/1.1Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel,application/vnd.ms-powerpoint, application/msword, */*Referer: http://localhost:8080/loan/loan.htmAccept-Language: itContent-Type: application/x-www-form-urlencodedAccept-Encoding: gzip, deflateUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; WinFX RunTime 3.0.50727)Host: localhost:8080Content-Length: 74Connection: Keep-AliveCache-Control: no-cache

LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15&IntrMax=15.0&IntrMin=13.0

2006-10-17 Dyalog User Group Conference 8

The hard part…The hard part…

APL is not a bad tool for taking character vectors apart, either

But: Some knowledge about the HTTP protocol is required to decipher it

A rule for calling the code to deal with the request (in this case #.loan.RUN) is needed

Some way to decode the body of the request

This is what Web Server frameworks like Microsoft IIS coupled with ASP.NET do…

2006-10-17 Dyalog User Group Conference 9

What is a web application?What is a web application?

“A collection of functions that take an HTTP Request as input and produce

an HTTP response as output”

2006-10-17 Dyalog User Group Conference 10

SolutionsSolutions

Dyalog APL integrates quite well with Microsoft IIS+ASP.NET, and can be called from other web servers, but:– These solutions require a lot of heavy machinery from

other vendors– The learning curve is still quite high

Dyalog provide a sample SERVER workspace, but:– It leaves almost all the work of deciphering HTTP to you– It’s based on old standards

2006-10-17 Dyalog User Group Conference 11

SolutionsSolutions

The author has:A keen (geek) interest in OO, web

and other software development methodologies

Written numerous tools on top of the Dyalog TCPSocket object

2006-10-17 Dyalog User Group Conference 12

Enter WildHeart.HTTPListenerEnter WildHeart.HTTPListener

The advent of Object Orientation in Dyalog 11.0 makes it attractive to start again

My goal is to create a framework which:– Requires a little knowledge of HTML and hardly any

knowledge of HTTP– Allows ”ordinary” APL developers to easily write Web

applications, yet:– Is built from reusable, replaceable components which

can be extended by APL developers who learn a little more...

– Is portable across all platforms which support TCP– Is run as an open source project, to be shared and

extended by anyone who wishes to contribute

2006-10-17 Dyalog User Group Conference 13

To write or To write or Not to writeNot to write

Why NOT TO WRITE a web server in APL:– This is the third web server I develop:

• I was paid to develop the first (and it was never used)

• I was almost paid to the develop/steal the second (and it’s heavily used)

• I was NOT paid to develop the third and I had to find the time to do it at night and during weekends

A wise man once said:

“only a fool would do that”

2006-10-17 Dyalog User Group Conference 14

Not to write, seriouslyNot to write, seriously

HTTP is easy but trickyReliabilitySecuritySpeedSo many web servers out there

2006-10-17 Dyalog User Group Conference 15

To writeTo write or not to write or not to write

Why write a web server in APL?(I wrote an editorial about it in Vector a few years ago)

– Easily embeddable in an APL application

– Easily understood by APLers– Can stay near the core of the

application

2006-10-17 Dyalog User Group Conference 16

To writeTo write or not to write or not to write

"One of the reasons to use Lisp in writing Web-based application is that

you can use Lisp. When you're writing software that is only going to

run on your own servers, you can use whatever language you want.“

Paul Graham

2006-10-17 Dyalog User Group Conference 17

To write!To write!

“Morten needs a demo. I need an HTTP/1.1 web server, written in APL. APL Italiana, the company I work for, also needs one, even if they haven’t

realised this yet.”

Stefano Lanzavecchia

2006-10-17 Dyalog User Group Conference 18

Components of… WildServer’06Components of… WildServer’06

HTTPListener HTTPRequest

– Headers– Cookies– Request Variables

HTTPResponse– Cookies

HTML Utils Event pipeline

– User Validation– Sessions

Static file server Namespace as

filesystem…

File Handlers:– aplx files– apls files– …

2006-10-17 Dyalog User Group Conference 19

Dyalog APL: an OO languageDyalog APL: an OO language

Web server: a good test bed for the OO extensions– Abstractions: divide et impera– Inheritance: this looks a bit like that– Keyed properties: syntax tricks

What is missing in the language?Is the IDE good enough?

2006-10-17 Dyalog User Group Conference 20

HTTPListenerHTTPListener

fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start

Then point your web browser at http://localhost:8080/

“It’s a kind of magic”

2006-10-17 Dyalog User Group Conference 21

The trip of an HTTP requestThe trip of an HTTP request

TCPSocket gets TCPRecv eventsHTTPRequest objects collects them,

and parses the request according to the HTTP specs

When the request is complete, it gets shipped to a dispatcher…

2006-10-17 Dyalog User Group Conference 22

The dispatcher: FileSystemThe dispatcher: FileSystem

To me a FileSystem looks like a namespace:– Its folders are sub-namespaces– Idea: we can inject virtual folders:

• A virtual folder can be a folder on the HD coming from a different sub-folder

• A virtual folder can be… an APL namespace! Its functions would be files…

2006-10-17 Dyalog User Group Conference 23

APLNamespaceAPLNamespace

http://localhost:8080/loan/RUNexecutes #.loan.RUN

buthttp://localhost:8080/loan/bullet.gif

will transmit the file c:\aplserve\loan\bullet.gif

fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\

aplserve\loan')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start

2006-10-17 Dyalog User Group Conference 24

ZipFileSystemZipFileSystem

Now the folders and files come from a zip file. Neat, uh? In fact this presentation…

fs←⎕NEW web.ZipFileSystem('c:\aplserve.zip?aplserve')

fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\aplserve\loan')

listener←⎕NEW web.HTTPListener(8080 fs)listener.Start

2006-10-17 Dyalog User Group Conference 25

FileSystemFileSystemBasically one method: ProcessRequest∇ ProcessRequest ctx;html;ns;raw;aplo :Access Public Instance Override ctx←#.WildHeart.HTTPContext.Current html←_NS⍎MapFileName ctx.URI r←ctx._Response r._IsBodyRaw←raw←82≠⎕DR html r._Body←html :If ''≡⊃r._Headers[⊂'Content-type'] r._Headers[⊂'Content-type']←u.MimeMappings['htm'

'bin'[1+raw]] :EndIf ctx.SendResponse ∇

2006-10-17 Dyalog User Group Conference 26

Tricks: thread local storageTricks: thread local storage

[…] ctx←#.WildHeart.HTTPContext.Current[…]

Thread context: wouldn’t it be nice to have an object local to a thread?

2006-10-17 Dyalog User Group Conference 27

HTTPHandlerHTTPHandler

A FileSystem delegates the handling of the requested resource to the appropriate handler:– StaticFileHandler by default– AplxHandler– AplsHandler– … more to come fs._FileHandlers[⊂'aplx']←#.WildHeart.AplxFileHandler

2006-10-17 Dyalog User Group Conference 28

HTTPHandlerHTTPHandler

One method: ProcessRequest ∇ ProcessRequest ctx;fname;tn;ext;etag :Access Public Instance Overridable fname←ctx._FileSystem.MapFileName ctx.URI tn←fname ⎕NTIE 0 ctx._Response._IsBodyRaw←1 ctx._Response._Body←⎕NREAD tn,83,⎕NSIZE tn ⎕NUNTIE tn ctx.SendResponse ∇

2006-10-17 Dyalog User Group Conference 29

AplxHandlerAplxHandler

“This is only a test”<?@ master="../newindex.maplx" execute="" ?>

<apl:content id="title"><title>Rain Graphics: Climate</title></apl:content><apl:content id="cssstyle"><link rel="stylesheet" type="text/css"

href="../styles/spiffy.css"/></apl:content>

<apl:content id="main" execute="#.rain.Climate"></apl:content>

2006-10-17 Dyalog User Group Conference 30

A master pageA master page<!DOCTYPE html PUBLIC "-//W3C//DTD

XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns=“http://www.w3.org/1999/xhtml” xmlns:apl="http://wildheart.com/2006/10/"

><head>

<apl:placeholder id="cssstyle"><link rel="stylesheet"

type="text/css" href="styles/spiffy.css"/>

</apl:placeholder><apl:placeholder id="title"><title/></apl:placeholder></head>

<body><div id="header"><h1

class="banner">WildHeart.HTTPListener</h1></div>

<div id="subheader"> <apl:placeholder id="bcrumb" execute="#.WildHeart.WebDemos.BreadCrumb" />

</div><div id="toc">

<apl:placeholder id="apltoc"><div class="toc"><h2>APL Web

Server</h2><ul><li><a

href="/intro.aplx">Introduction</a></li>

<li><a href="/cv/Index.aplx">View Code</a></li>

…

2006-10-17 Dyalog User Group Conference 31

The event pipelineThe event pipeline

As a request goes through the FileSystem hierarchy, events are raised:– OnEnter– OnAuthenticate– OnAuthorize– …

2006-10-17 Dyalog User Group Conference 32

Tricks: the observer patternTricks: the observer pattern

:Class Event :field _Observers ∇ make name :Access Public Instance :Implements Constructor _Observers←⍬ ⎕DF'Event: ',name ∇ ∇ Raise(sender arg);obj;fnname :Access Public Instance :For obj fnname :In

_Observers (obj⍎fnname)(sender arg) :EndFor ∇

∇ Add(obj fnname) :Access Public Instance _Observers,←⊂obj fnname ∇ ∇ Del(obj fnname) :Access Public Instance _Observers← _Observers~⊂obj fnname ∇ :EndClass

2006-10-17 Dyalog User Group Conference 33

The observed shows what it’s gotThe observed shows what it’s got

onAuthenticate←⎕NEW #.WildHeart.Event('Authenticate')

auth←⎕NEW web.WebDemos.FormsAuth fs.onAuthenticate.Add auth'OnAuthenticate'

onAuthenticate.Raise ⎕THIS lurl

2006-10-17 Dyalog User Group Conference 34

Two simple authentication modulesTwo simple authentication modules

SimpleAuth: based on HTTP challenge response:– Quick and dirty

FormsAuth: based on cookies– More flexible– More work

2006-10-17 Dyalog User Group Conference 35

SessionsSessions

HTTP is a stateless protocolPeople want to have conversations

with peers who don’t forget instantly what they just said

Have a cookie!

2006-10-17 Dyalog User Group Conference 36

SessionsSessions

As easy as: fs.EnableSession ⎕NULL fs.onSessionStart.Add #’DoSomething' fs.onSessionEnd.Add #’UndoSomething'

In the application: :If ⎕NULL≡ns←ctx._Session[⊂'footer'] ns←⎕NS'' ns.count←0 ctx._Session[⊂'footer']←ns :EndIf ns.count+←1

2006-10-17 Dyalog User Group Conference 37

ExtensibilityExtensibility

Some of the features (ZipFileSystem, session module, authentication, aplx handler) were hacked together to prove that the server could support them

Even before its completion, the WildServer has:– Had a file handler written (by Nic & Morten)– Been used to run a course in writing Web

Applications using Dyalog APL

2006-10-17 Dyalog User Group Conference 38

Open SourceOpen Source

Open Source means: “free as in free beer” (except: is there such a thing as free beer?)

No strings attached:– You get the source code and whatever

documentations exists with it at the time– You get the right to do whatever you want with

it (even sell it! But only a bad boy would do that…)– You get from me all the support I can give you when I

can if I can (alas, I am very busy and maybe one day I’ll get a life)

2006-10-17 Dyalog User Group Conference 39

Why Open Source?Why Open Source?

Have you looked at Ruby on Rails?Tons of contributors do a lot of workTons of clever contributors do a lot

of good work!

OK, maybe that’s a good product.

2006-10-17 Dyalog User Group Conference 40

TODO listTODO list

It’s long, it’s scary, but…APL Prototypes seem to acquire a

life of their ownThe “ducks” have started

contributing

2006-10-17 Dyalog User Group Conference 41

ThanksThanks

Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays

2006-10-17 Dyalog User Group Conference 42

ConclusionConclusion

Initial feedback from users (Nic, Morten and Mondays course participants) is promising

The WildServer is still very much ”work in progress”, and the author wants to keep control of it until about Christmas

However, you are all welcome to try it now and suggest or code enhancements

2006-10-17 Dyalog User Group Conference 43

Questions?Questions?

If not… now I need some sleep.

輝く空と君の声を感じたい

2006-10-17 Dyalog User Group Conference 44

What is missing in the languageWhat is missing in the language

No public operators! C# has them!No protected members! C# has them!No object serialization! C# has it!

C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby”