skyscanner journey - from code jungle to state of the art
TRANSCRIPT
Skyscanner is a leading global travel search site offering unbiased, comprehensive and free flight, hotel and car hire comparison.
Unique monthly visitors Over 50 million
App downloads Over 40 million
Languages available Over 30
Global employees Over 700
Number of nationalities Over 50
Global offices 9
A bit of history
hotels meta search website built on top of a Natural Language Processing engine
supported Spanish and English developed a graph-based database to represent the world as
an ontology DDB (Deductive Database) & DQL based in Barcelona (obviously )
A bit of history
Today* in the news
* by today I mean mid-2013
First task
Fogg architecture – simplified edition
Business Logic Service (BLS) aka backend
Frontend(Zend Framework)
JSO
N A
PI
Fogg frontend
5 years of “startup code” team was focused on the backend & DQL MVC(-ish) files between 4000 – 6000 lines (not be opened with Eclipse) no tests / documentation no coding / naming standard
…some blood, sweat & tears later Skyscanner Hotels launches in January 2014 (hooray)
Going mobile
Not really doable on the current code base.
Still going mobile
Develop a separate mobile website tight time constrains not all functionality from the desktop site was implemented no real tests some documentation no separation of concerns
Welcome to code jungle
Desktop applicationZF1 – Apache 8090
Mobile / tablet applicationYii – nginx 8092
Deviceatlas
Since maintaining 2 code bases for the same product was not bad enough, other “features” were included no real tests or useful documentation monolithic architecture – “one class to rule them all” – always expect the unexpected some pages – redirect to the partner site – were shared complicated release process – only for the bravest and brightest
Squadification
Squads & tribes
Hotels Tribe Flights TribeCar hire Tribe
HFE(frontend)
Infrastructure Tribe
HBE(backend)
HD(data)
PE(partners)
…
…
…
…
…
…
…
…
…
…
…
…
Technology stacks
multi-discipline team operate like a startup owns a (micro)service can pick its technology stack based on needs own its internal processes (scrum, kanban) must respect “contracts” with other squads
Does it work? How?
Edge Side Includes
Edge Side IncludesEdge Side Includes or ESI is a small markup language for edge level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling. It is an application of edge computing.
<section title=“Textbook definition” warning=“boring”>
</section>
Everything is a component
/hotels/search-box
/hotels/results-list
Anatomy of ESI component
/hotels/search-box/
requirements(configuration json)
content(serve html)
style(serve <style> tags)
script(serve <script> tags)
Endpoints
Component
self contained entity that renders a piece of HTML
the endpoint is added to the component name to form the URL: /hotels/search-box/content
all ESI templates are preprocessed by a component called Scaffolding
the requirements endpoint is requested first
components cannot access the URL / cookies / client headers directly (they need to be explicitly requested via the configuration json)
ESI architecture
Varnish retrieve content / style / script endpoints cache according to HTTP headers
ESI components(various technologies)
Scaffolding (.NET) loads the ESI template for the URL retrieves the configuration json injects the requested parameters returns the template to Varnish
http request
Scaffolding response
Pretty cool, right?
You are here!
You are here
Choosing the framework
Requirements: PHP micro-framework no database layer required strong community actively supported fast
Testing methods: apache benchmark on “hello world” number of installs on packagist time since the last commit community vibe And the winner is…
Mustache / Handlebars
widely used 30+ programming languages templates can be easily shared with other squads started with mustache but was *too* logicless
Routing
routing app
search-box
results-list
hotel-details
ESI request
Components
Clear separation of concerns
Routing app detect which component is requested bootstrap common services (SilexServiceProvider)
monolog handlebars i18n & l10n caching static routing BLS connector
no controllers / logic error management
Component bootstrap component specific services render the component endpoints (content /
style / script) controllers internal webroot static files (css & js) proxied via Silex in dev,
packaged and served via nginx otherwise
BLS Connector
BLS connector library models encapsulating the BLS json API shared by multiple applications (internal & external
use) composer vcs repository unit tested separately treated as an external library uses JsonPath to extract data
Business Logic Service(BLS)
BLS connector(PHP library)
App1 App2 App3
BLS consuming applications
Json path
https://github.com/Skyscanner/JsonPath-PHPhttp://goessner.net/articles/JsonPath/
Javascript
Javascript common javascript libraries (shared by
multiple components) component specific served minified in non-development
environments can be minified in dev for debugging
purposes all code is namespaced and encapsulated Backbonejs and jQuery no inline javascript in content nodes
CSS / SaSS
shared repository of common styles component specific styles managed via assetic
Testing & code coverage
aiming for 100% unit test coverage hard limit 95% strictly enforced
Coding standard
custom coding standard PSR-1 & PSR-2 compatible based on Symfony’s actively enforced with php_codesniffer repository with custom rules jshint for javascript coding standard
Git hooks & pipeline
Local git feature branches pre-commit – check coding standard pre-push – run phpunit
Merge request Code review
Wiggum (nodejs application)
Teamcity run all previous tests user journeys layout tests (Sauce labs)
Gitlabdevelop branch
Release procedure
Releasing is the only time we bring value to our users!
automated with ansible only 1 manual step (wip) takes 1h to deploy to 4 DCs around the globe all static resources are packaged and optimized version numbers are added to break the cache we *did* release on Friday (morning, but still) 2-3 times per week
Logging / monitoring
Going forward
