The FileSender project:collaborative opportunitiesin NREN land

APAN33, Chiang Mai

Guido Aben, AARNet Pty Ltd

Talk purpose:

Share our experience with grass-roots, collaborative inter-NREN software/service development

Tell you about the availability of a great service that you can run at your NREN for free, without any serious effort

Open the door wide for interested contributors (code, translations, testing, funds…)

FileSender, very high level

FileSender: User Adoption

<1 helpdesk call per week (mostly re: federation!) ~ 99.995% uptime ~4% of users sending >2GB files, without

instruction (seen 67GB in the field) At February 2010, >2000 users & >6 TB uploaded

stats are world-viewable at https://cloudstor.aarnet.edu.au/mrtg/

Statistics from AARNet’s FileSender installation:

FileSender: Existing Installations

ASU Uni Lithuania AARNet Australia ARNES Slovenia CESNET* Czech

Rep. BELNET Belgium FCCN Portugal HEAnet Ireland i2CAT

Catalonia Internet2* USA

Okinawa Institute of Science & Technology*

Japan NIIF* Hungary North-West Uni

South Africa

RESTENA*Luxembourg

SIDN*Netherlands

SRCE Croatia SURFnet*

Netherlands TERENA*

Netherlands UNINETT Norway

* test installation

18 known installations across 16 countries:

FileSender: project factsheet

Started April 2009 1.0 released 31 January 2011 Core team of 5 NREN people in 3 countries Agile, online collaboration Back: php/apache/postgresSQL/simpleSAMLphp Front: Flash + HTML5 (for >2GB) Version 1.5 (beta out yesterday!):http://blog.filesender.org/2012/02/13/filesender-1-5-beta1-released/

plugin-free for >2GB uploads; MySQL support Handles hanzi/hiragana/katakana/quốc ngữ(W3C, i18n, UTF8; auto language selection support)

Splendid isolation: top-down SW dev

“Every good work of software starts by scratching a developer's personal itch” – Eric S. Raymond Match personal itch with internal req’ment Run survey;

look for clues to back up new service idea Propose swiss army knife to funding body Start in-house development straight away, using

toolchain most convenient to developers Run resultant SW on server box of your taste Package what you have on the deadline; declare

success*; move on. Patch bugs with documentation

*at 100 users, at 200 logins, at 95% uptime, take your pick

How we optimised for collaboration

Still started with a personal itch of course!(who would dare contradict Eric Raymond?)

But then consulted with int’l eR / NREN orgs, exchanged ideas, inspected similar services and SW

Reduced scope of initial planned release Turned our dev group outward, not inward Planned for ease of install, thus wide deployment,

thus critical mass and sustainability Resisted urge to use comfortable/hip toolchain; Tested, retested and tested again: refused to fix

with documentation; instead fixed user experience

Old vs. New

The start’s the same – a personal itch.It’s very hard to start from requirements taken form surveys or focus groups. You have to trust your instincts somehow.

“It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them.” – Steve Jobs

Old: Run survey;look for clues to back up new service idea

New: consulted with int’l eR / NREN orgs, exchanged ideas, inspected similar services and SW(even tried to get 3rd party code public-domained; weren’t successful, but would try again)

Our challenges are unlikely to be unique.And our survey reach is limited.Increasing the catchment area for “accidental discoveries” helped. Surveying the same people again doesn’t, usually, for us.If your challenge is really unique, perhaps you’re just scratching your own itch? Should that be funded with infrastructure monies?

Old: propose swiss army knife to funding body

New: Reduced scope of initial planned release

A tradition exists of promising the world on a silver platter to the funding bodies and then having to push lots of features overboard during the project. This makes planning hard and distracts from the real outcome.

Instead we chose to work on a pared-down initial release that we could pay for without funding. Lay the foundations first, don’t get distracted by artificial deadlines.If the resulting product is great, we’ll run a funding application later.

Old: start in-house development straight away, using toolchain most convenient to developersRun resultant SW on server box of your taste(script, install binary dependencies, chroot and recompile at will)

New: turned our dev group outward, not inward- the full “open source” works: license, bug tracker, project wiki, packaging, releasingPlanned for ease of install everywhere, thus wide deployment, thus critical mass and sustainabilityResisted urge to use comfortable/hip toolchain(decided against java; picked php, forbade ourselves to use external dependencies)We made a conscious choice to sacrifice coding speed for long-term viability.The project was designed as an open source project; to have to be open about internal disagreement was… a learning curve. To have to refrain from grabbing a “more elegant” tool or language was hard at times. Having to wait for consensus… frustratingThe result has been a much bigger userbase, more input, more corner case testing and a much reduced per-user cost

Old: Package what you have on the deadline; declare success*; move on.

Corollary: run service as-is, treat SW as abandonware, patch bugs with documentation

*at 100 users, at 200 logins, at 98% uptime, take your pick

New: tested, retested and tested again: refused to fix with documentation; instead fixed user experience

We like it so far

It’s been a rewarding journey:we have the service we wanted

our project looks like it’s sustainable

We’re keen to try more of this:a) More features delivered for filesender in a

modular, collaborative wayb) A new service idea delivered in collaboration

Thoughts? Volunteers?

More Info:

www.filesender.org -- main project site

blog.filesender.org -- news, releases

And finally for those of you who’ve really become enthusiastic about contributing:www.assembla.com/spaces/file_sender/wiki/Workplan_2012