wildheart.httplistener an open source web server for dyalog apl v11.0 stefano “wildheart”...

WildHeart.HTTPListeneWildHeart.HTTPListenerr

an open source Web Server an open source Web Server for Dyalog APL v11.0for Dyalog APL v11.0

Stefano“WildHeart”

Lanzavecchia

2006-10-17 Dyalog User Group Conference 2

AgendaAgenda

APL should be a popular language for developing web pages, because:– APL has all you need to generating static and

dynamic web content

It is not used as much as it deserves, because:– Even the best tools need good frameworks to

level the learning curve a bit– No frameworks are readily available


GoalGoal

Explore the problem which needs to be solved and demonstrate a proposed solution... (obviously )

So, what’s the problem?The easy part is generating HTML Hopefully nobody will disagree that APL is

an EXCELLENT language for generating character vectors


A ”web”

Some very simple HTMLSome very simple HTML


The hard partThe hard part

If we have a form like this and the user hits the submit button…


Before we look at the response…Before we look at the response…

<form action="RUN" method="post" name="FrontPage_Form1"><div> <table border="0" cellpadding="3"> <tr> <td>Purchase Amount</td> <td><input type="text" size="9" name="LoanAmt"/></td> </tr> <tr> <td>Percent Down</td> <td><input type="text" size="6" maxlength="6" name="PercentDown"/></td> </tr> <tr> <td>Maximum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMax"/></td> </tr> <tr> <td>Minimum Years in Loan</td> <td><input type="text" size="2" maxlength="2" name="LenMin"/></td> </tr>

… continued on the right …

<tr>

<td>Maximum Interest Rate (%)</td>

<td><select name="IntrMax" size="1">

<option>15.0</option>

<option>14.5</option>

..etc down to…

<option>1.0</option></select></td> </tr><tr> <td>Minimum Interest Rate (%)</td> <td><select name="IntrMin" size="1"> <option>15.0</option> <option>14.5</option> <option>14.0</option> <option>13.5</option> <option selected="selected">13.0</option> <option>12.5</option> <option>12.0</option>

…etc… <option>1.5</option>

<option>1.0</option> </select></td> </tr> </table> <p><input type="submit" value="Calculate

Repayments"/></p></div></form>

Some of you will want to knowhow the HTML looks for the form?


When the user hits the button…When the user hits the button…

POST /loan/RUN HTTP/1.1Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel,application/vnd.ms-powerpoint, application/msword, */*Referer: http://localhost:8080/loan/loan.htmAccept-Language: itContent-Type: application/x-www-form-urlencodedAccept-Encoding: gzip, deflateUser-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; WinFX RunTime 3.0.50727)Host: localhost:8080Content-Length: 74Connection: Keep-AliveCache-Control: no-cache

LoanAmt=10000&PercentDown=50&LenMax=10&LenMin=15&IntrMax=15.0&IntrMin=13.0


The hard part…The hard part…

APL is not a bad tool for taking character vectors apart, either

But: Some knowledge about the HTTP protocol is required to decipher it

A rule for calling the code to deal with the request (in this case #.loan.RUN) is needed

Some way to decode the body of the request

This is what Web Server frameworks like Microsoft IIS coupled with ASP.NET do…


What is a web application?What is a web application?

“A collection of functions that take an HTTP Request as input and produce

an HTTP response as output”


SolutionsSolutions

Dyalog APL integrates quite well with Microsoft IIS+ASP.NET, and can be called from other web servers, but:– These solutions require a lot of heavy machinery from

other vendors– The learning curve is still quite high

Dyalog provide a sample SERVER workspace, but:– It leaves almost all the work of deciphering HTTP to you– It’s based on old standards


SolutionsSolutions

The author has:A keen (geek) interest in OO, web

and other software development methodologies

Written numerous tools on top of the Dyalog TCPSocket object


Enter WildHeart.HTTPListenerEnter WildHeart.HTTPListener

The advent of Object Orientation in Dyalog 11.0 makes it attractive to start again

My goal is to create a framework which:– Requires a little knowledge of HTML and hardly any

knowledge of HTTP– Allows ”ordinary” APL developers to easily write Web

applications, yet:– Is built from reusable, replaceable components which

can be extended by APL developers who learn a little more...

– Is portable across all platforms which support TCP– Is run as an open source project, to be shared and

extended by anyone who wishes to contribute


To write or To write or Not to writeNot to write

Why NOT TO WRITE a web server in APL:– This is the third web server I develop:

• I was paid to develop the first (and it was never used)

• I was almost paid to the develop/steal the second (and it’s heavily used)

• I was NOT paid to develop the third and I had to find the time to do it at night and during weekends

A wise man once said:

“only a fool would do that”


Not to write, seriouslyNot to write, seriously

HTTP is easy but trickyReliabilitySecuritySpeedSo many web servers out there


To writeTo write or not to write or not to write

Why write a web server in APL?(I wrote an editorial about it in Vector a few years ago)

– Easily embeddable in an APL application

– Easily understood by APLers– Can stay near the core of the

application


To writeTo write or not to write or not to write

"One of the reasons to use Lisp in writing Web-based application is that

you can use Lisp. When you're writing software that is only going to

run on your own servers, you can use whatever language you want.“

Paul Graham


To write!To write!

“Morten needs a demo. I need an HTTP/1.1 web server, written in APL. APL Italiana, the company I work for, also needs one, even if they haven’t

realised this yet.”

Stefano Lanzavecchia


Components of… WildServer’06Components of… WildServer’06

HTTPListener HTTPRequest

– Headers– Cookies– Request Variables

HTTPResponse– Cookies

HTML Utils Event pipeline

– User Validation– Sessions

Static file server Namespace as

filesystem…

File Handlers:– aplx files– apls files– …


Dyalog APL: an OO languageDyalog APL: an OO language

Web server: a good test bed for the OO extensions– Abstractions: divide et impera– Inheritance: this looks a bit like that– Keyed properties: syntax tricks

What is missing in the language?Is the IDE good enough?


HTTPListenerHTTPListener

fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start

Then point your web browser at http://localhost:8080/

“It’s a kind of magic”


The trip of an HTTP requestThe trip of an HTTP request

TCPSocket gets TCPRecv eventsHTTPRequest objects collects them,

and parses the request according to the HTTP specs

When the request is complete, it gets shipped to a dispatcher…


The dispatcher: FileSystemThe dispatcher: FileSystem

To me a FileSystem looks like a namespace:– Its folders are sub-namespaces– Idea: we can inject virtual folders:

• A virtual folder can be a folder on the HD coming from a different sub-folder

• A virtual folder can be… an APL namespace! Its functions would be files…


APLNamespaceAPLNamespace

http://localhost:8080/loan/RUNexecutes #.loan.RUN

buthttp://localhost:8080/loan/bullet.gif

will transmit the file c:\aplserve\loan\bullet.gif

fs←⎕NEW #.WildHeart.FileSystem('c:\aplserve')fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\

aplserve\loan')listener←⎕NEW web.HTTPListener(8080 fs)listener.Start


ZipFileSystemZipFileSystem

Now the folders and files come from a zip file. Neat, uh? In fact this presentation…

fs←⎕NEW web.ZipFileSystem('c:\aplserve.zip?aplserve')

fs['loan']←⎕NEW web.APLNamespace(#.loan'c:\aplserve\loan')

listener←⎕NEW web.HTTPListener(8080 fs)listener.Start


FileSystemFileSystemBasically one method: ProcessRequest∇ ProcessRequest ctx;html;ns;raw;aplo :Access Public Instance Override ctx←#.WildHeart.HTTPContext.Current html←_NS⍎MapFileName ctx.URI r←ctx._Response r._IsBodyRaw←raw←82≠⎕DR html r._Body←html :If ''≡⊃r._Headers[⊂'Content-type'] r._Headers[⊂'Content-type']←u.MimeMappings['htm'

'bin'[1+raw]] :EndIf ctx.SendResponse ∇


Tricks: thread local storageTricks: thread local storage

[…] ctx←#.WildHeart.HTTPContext.Current[…]

Thread context: wouldn’t it be nice to have an object local to a thread?


HTTPHandlerHTTPHandler

A FileSystem delegates the handling of the requested resource to the appropriate handler:– StaticFileHandler by default– AplxHandler– AplsHandler– … more to come fs._FileHandlers[⊂'aplx']←#.WildHeart.AplxFileHandler


HTTPHandlerHTTPHandler

One method: ProcessRequest ∇ ProcessRequest ctx;fname;tn;ext;etag :Access Public Instance Overridable fname←ctx._FileSystem.MapFileName ctx.URI tn←fname ⎕NTIE 0 ctx._Response._IsBodyRaw←1 ctx._Response._Body←⎕NREAD tn,83,⎕NSIZE tn ⎕NUNTIE tn ctx.SendResponse ∇


AplxHandlerAplxHandler

“This is only a test”<?@ master="../newindex.maplx" execute="" ?>

<apl:content id="title"><title>Rain Graphics: Climate</title></apl:content><apl:content id="cssstyle"><link rel="stylesheet" type="text/css"

href="../styles/spiffy.css"/></apl:content>

<apl:content id="main" execute="#.rain.Climate"></apl:content>


A master pageA master page<!DOCTYPE html PUBLIC "-//W3C//DTD

XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns=“http://www.w3.org/1999/xhtml” xmlns:apl="http://wildheart.com/2006/10/"

><head>

<apl:placeholder id="cssstyle"><link rel="stylesheet"

type="text/css" href="styles/spiffy.css"/>

</apl:placeholder><apl:placeholder id="title"><title/></apl:placeholder></head>

<body><div id="header"><h1

class="banner">WildHeart.HTTPListener</h1></div>

<div id="subheader"> <apl:placeholder id="bcrumb" execute="#.WildHeart.WebDemos.BreadCrumb" />

</div><div id="toc">

<apl:placeholder id="apltoc"><div class="toc"><h2>APL Web

Server</h2><ul><li><a

href="/intro.aplx">Introduction</a></li>

<li><a href="/cv/Index.aplx">View Code</a></li>

…


The event pipelineThe event pipeline

As a request goes through the FileSystem hierarchy, events are raised:– OnEnter– OnAuthenticate– OnAuthorize– …


Tricks: the observer patternTricks: the observer pattern

:Class Event :field _Observers ∇ make name :Access Public Instance :Implements Constructor _Observers←⍬ ⎕DF'Event: ',name ∇ ∇ Raise(sender arg);obj;fnname :Access Public Instance :For obj fnname :In

_Observers (obj⍎fnname)(sender arg) :EndFor ∇

∇ Add(obj fnname) :Access Public Instance _Observers,←⊂obj fnname ∇ ∇ Del(obj fnname) :Access Public Instance _Observers← _Observers~⊂obj fnname ∇ :EndClass


The observed shows what it’s gotThe observed shows what it’s got

onAuthenticate←⎕NEW #.WildHeart.Event('Authenticate')

auth←⎕NEW web.WebDemos.FormsAuth fs.onAuthenticate.Add auth'OnAuthenticate'

onAuthenticate.Raise ⎕THIS lurl


Two simple authentication modulesTwo simple authentication modules

SimpleAuth: based on HTTP challenge response:– Quick and dirty

FormsAuth: based on cookies– More flexible– More work


SessionsSessions

HTTP is a stateless protocolPeople want to have conversations

with peers who don’t forget instantly what they just said

Have a cookie!


SessionsSessions

As easy as: fs.EnableSession ⎕NULL fs.onSessionStart.Add #’DoSomething' fs.onSessionEnd.Add #’UndoSomething'

In the application: :If ⎕NULL≡ns←ctx._Session[⊂'footer'] ns←⎕NS'' ns.count←0 ctx._Session[⊂'footer']←ns :EndIf ns.count+←1


ExtensibilityExtensibility

Some of the features (ZipFileSystem, session module, authentication, aplx handler) were hacked together to prove that the server could support them

Even before its completion, the WildServer has:– Had a file handler written (by Nic & Morten)– Been used to run a course in writing Web

Applications using Dyalog APL


Open SourceOpen Source

Open Source means: “free as in free beer” (except: is there such a thing as free beer?)

No strings attached:– You get the source code and whatever

documentations exists with it at the time– You get the right to do whatever you want with

it (even sell it! But only a bad boy would do that…)– You get from me all the support I can give you when I

can if I can (alas, I am very busy and maybe one day I’ll get a life)


Why Open Source?Why Open Source?

Have you looked at Ruby on Rails?Tons of contributors do a lot of workTons of clever contributors do a lot

of good work!

OK, maybe that’s a good product.


TODO listTODO list

It’s long, it’s scary, but…APL Prototypes seem to acquire a

life of their ownThe “ducks” have started

contributing


ThanksThanks

Peter Michael Haager: I have adapted his XML parser to produce trees of objects instead of nested arrays


ConclusionConclusion

Initial feedback from users (Nic, Morten and Mondays course participants) is promising

The WildServer is still very much ”work in progress”, and the author wants to keep control of it until about Christmas

However, you are all welcome to try it now and suggest or code enhancements


Questions?Questions?

If not… now I need some sleep.

輝く空と君の声を感じたい


What is missing in the languageWhat is missing in the language

No public operators! C# has them!No protected members! C# has them!No object serialization! C# has it!

C# is a statically typed language. OK, substitute in the slide “C#” with “Ruby”

wildheart.httplistener an open source web server for dyalog apl v11.0 stefano “wildheart”...

Documents

web server frameworks

web application

web pages

web servers

http response

http request

dyalog apl v11

dynamic web contentit