lessons learned from a large scale osgii web app - p bakker & j de vreede
DESCRIPTION
Building a large scale, cloud hosted, multi device product that changes the way students are taught in high schools, turns out to be not entirely trivial. We have been building PulseOn for the last two years. We faced many technical challenges, and have tried many different technologies. Some ideas turned out to be extremely helpful, some other ideas that looked promising turned out to be bad practices. In this talk we will discuss what we have learned about building modular, scalable web applications. We will go into different areas of the technology stack, from storage and (nosql) databases to a modular RESTful backend and a multi device HTML5 based frontend. We will also go into the cloud, and discuss topics such as auto scaling and failover. We will discuss what works and what doesn’t, and hopefully help other to make the right decisions. Some topics that we will touch upon: - Modular architecture with OSGi - Failing at linked data and semantic databases - Using MongoDB from OSGi - Release process and semantic versioning - Continuous deployment - JavaScript frameworks Bios: Paul Bakker Paul is a software architect for Luminis Technologies and the author of “Building Modular Cloud Apps With OSGi”. He believes that modularity and the cloud are the two main challenges we have to deal with to bring technology to the next level, and is working on making this possible for mainstream software development. Today he is working on educational software focussed on personalised learning for high school students in the Netherlands. Paul is an active contributor on open source projects such as Amdatu, Apache ACE and BndTools. He has a background as a trainer on Java related technology and is a regular speaker on conferences such as JavaOne, Devoxx and JFokus. Jago de Vreede Jago is a software engineer at Luminis Technologies, as a software engineer he has seen a broad-spectrum of projects and he has been working on a large OSGi project for the last year. His work is not exclusive to java development but also does front-end development, and the integration between these. Performance tuning and optimizations are also part of his work.TRANSCRIPT
Lessons learned from a large scale OSGi web app
Jago de Vreede Paul Bakker
Paul Bakker@pbakker
Jago de Vreede
Agenda !
High level architecture Modular architecture Modularity with OSGi Cloud deployments with Apache ACE Frontend frameworks
Ask whenever you want!
The case
PulseOn !
Educational system
focussed on personalized
learning !
Used in high schools in
the Netherlands !Expand to other countries
in the near future
Cloud provisioning
http://ace.apache.org/
bndtools Eclipse OSGi plugin http://bndtools.org/
Eclipse http://eclipse.org/ Source control
Tools
Stash, Jira, Bamboo http://atlassian.com/
WebStorm http://jetbrains.com/
webstorm/
HTML 5 + JavaScript
RESTful services
OSGi services
Mongo
Apache Felix
S3
High level architectureRequirements:
!Agile and modular Modern web app UI mostly offloaded to clients or devices Integration via REST API Horizontally scalable
A m d a t u
Auth Blob stores MongoDB Multi-tenancy OpenSocial Search Remote Services REST Template Web ...
Components
Load
Load Balancer
PulseOn PulseOn
PulseOn
School B
MongoMongo
Mongo
Deployment
Load
Load Balancer
PulseOn PulseOn
PulseOn
School A
MongoMongo
Mongo
Load
Load Balancer
DAMSDAMS
DAMS
Content backend
MongoMongo
Mongo
Profiles Rest
Profiles API
Profiles Service
MongoDB
Progress Rest
Progress API
Progress Service
MongoDB
Curriculum API
Curriculum Service
MongoDB
... Rest
... API
... Service
services
all the way down
A service should only
do one single thing
Services are the (reusable) building
blocks of our application
Benefits of a services based architecture
!
Small services are easy to maintain Small services are composable Services are easily testable
Packaging services Services are packaged in small bundles Related bundles may be generated from a single Bndtools project
Data Storage
Semantic data store (triple store)
MySQL (with JPA)
MongoDB
NowBeginning
Relational databasesin a modular system
A part of data should be owned by 1 service
How to set boundaries in a relational world!?
Relational databasesin the modern web
!
ORM is REALLY difficult to get right Relational databases don’t scale well
MongoDB: our database of choice
!
Object Mapping trivial Easy to scale Powerful aggregation/map-reduce framework
Code example: MongoDB with Amdatu
Amdatu Mongo Service
Setup Object Mapper
Execute query
Release & Deployment
Binaries
Release to release repo
Release to
release OBR
JavaScript minification
closure compiler
deterministic? => fixed March 3, 2014
Baseline OBR Release OBR
CI Build Server
Deployment
Software distribution framework
Manages the installation and upgrade
of bundles, configuration, etc. to
heterogenous targets
Load
Load Balancer
PulseOn PulseOn
PulseOn
School B
MongoMongo
Mongo
Deployment
Load
Load Balancer
PulseOn PulseOn
PulseOn
School A
MongoMongo
Mongo
Load
Load Balancer
DAMSDAMS
DAMS
Content backend
MongoMongo
Mongo
Deployment
Load
Load Balancer
PulseOn PulseOn
PulseOn
School
Load
Load Balancer
DAMSDAMS
DAMS
Content backend
Deployment
Load
Load Balancer
TargetTarget
Target
School
Load
Load Balancer
TargetTarget
Target
Content backend
Release OBR
Apache ACE
Baseline OBR
CI Build Server
Continuous Deployment
Load
Load Balancer
Target
School
Load
Load Balancer
Target
Content backend
Snapshot OBR
Apache ACE
CI Build Server
Like to know more? Continuous Automated Deployment with Apache ACE youtube.com/watch?v=4S_zvgG_MLw
Unit testing OSGi codeis trivial
But what if we wantmore?
Integration testing turnsout to be trivial as well!
Automated dependency injection of
service
Setup Config Admin
Junit style asserts
Frontend frameworks
jQuery Pros
Can do a lot with not much code
Lot of helpful
plugins
available
Easy to get started
jQuery
Hard to maintain
Cons
Hard to test
Backbone.js Pros
Designed towards
consuming REST data
Well structured
Backbone.js Cons
Hard to maintain
A LOT OF CODE!!
Testable
AngularJS
AngularJS Pros
Designed towards
consuming REST data
Well structured
No guessing in htmlTestable
TypeScript Pros
Well structured
Code completion
compile time checks
TypeScript Cons
Well structured
Slow-ish
Need to have a
definition to work
!
A modular architecture gives us:!
Maintainability
Extensibility
Freedom to change
Wrap up
But what if I want
Spring
EJB
Hibernate?
Just don’t… You really don’t need to.
Cloud provisioning
http://ace.apache.org/
Eclipse OSGi plugin http://bndtools.org/
That’s us http://luminis-technologies.com
Cloud OSGi services
http://www.amdatu.org/
Amdatu
Paul Bakker [email protected]
@pbakker
Jago de Vreede [email protected]
Learn more? !
Friday 10.20 - 12.50 or 14.45 - 17.15 !
Tutorial “Developing modular cloud applications with OSGi”