project phoenix - from php to the play framework in 3 months

@elmanu#playframework

From PHP to the Play Framework in 3 months

Manuel Bernhardt @elmanu http://manuel.bernhardt.io

@elmanu#playframework

Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Summary & Future

@elmanu#playframework

Your speaker

•Freelance software consultant based in Vienna !

•Web, web, web

•Scala, Akka, Play Framework !

•Vienna Scala User Group !

•Open-Source

@elmanu#playframework

Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Summary & Future

@elmanu#playframework

Talenthouse

•www.talenthouse.com !

•based in Los Angeles !

•connecting brands and artists !

•3+ million users

@elmanu#playframework

Background

•Problems with the old site

• Slow

• Developed by an external company

• Hard to evolve

@elmanu#playframework

Background

•Solution: new site!

• Scala

• Play Framework

• AngularJS

• elasticsearch

• …

@elmanu#playframework

Background

•A few interesting aspects

• Source code may not be available

• Data spread out in many places

• Only 3 months time

@elmanu#playframework

Background

•A few interesting aspects

• Source code may not be available

• Data spread out in many places

• Only 3 months time

@elmanu#playframework

Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Methodologies & Tools

• Summary & Future

@elmanu#playframework

Building the new platform

@elmanu#playframework

Building the new platform

@elmanu#playframework

@elmanu#playframework

• MVC framework !

• Compile as much as possible

• “Reactive”

@elmanu#playframework

http://www.reactive-manifesto.org

@elmanu#playframework

Load balancer

@elmanu#playframework

Load balancer

@elmanu#playframework

Building the new platform

Data access?

@elmanu#playframework

Building the new platform

Anorm

Data access?

Squeryl

@elmanu#playframework

Building the new platform

Anorm

Data access?

Squeryl

@elmanu#playframework

Building the new platform

Anorm

Data access?

Squeryl

SORM Tiramisu

@elmanu#playframework

Building the new platform

• Started with Anorm

• Too verbose !

• Switched to Slick

• Plain SQL with neat string interpolation

• DSL for composing queries

@elmanu#playframework

Building the new platform

def fetchItemWithRelevanceSort(userId: Long, skip: Int, limit: Int)! (implicit s: Session): List[Item] = {! val q = for {! i <- items if !i.deleted! ! && i.visible_on_profile! ! && i.user_id === userId! } yield i! ! q.sortBy(item => (item.selected.desc, item.created_on.desc))! .drop(skip)! .take(limit)! .list!! }

val query = sql"""! with invites_for_market as (! select #$inviteSL, #$inviteMarketSL! from invite i! inner join invite_market im! on i.id = im.invite_id! and im.market_id = ${marketId.id}! where i.art_type = ${artType.toString}! and i.visibility_state = ${InviteVisibilityState.Public}! and i.deleted = false! order by i.submission_start desc, i.id desc! limit $limit! offset $skip! )!! select ifm.*, #$localInviteSL! from invites_for_market ifm! inner join local_invite li! on ifm.id = li.invite_id! and li.completed = true! order by ifm.submission_start desc, ifm.id asc! """! val queryResults = query.as[(Invite, InviteMarket, LocalInvite)].list!

@elmanu#playframework

Building the new platform

Front-end & UI

@elmanu#playframework

Building the new platform

Front-end & UI

@elmanu#playframework

Building the new platform

Front-end & UI

@elmanu#playframework

Building the new platform

• Recently open-sourced by Facebook !

• Virtual DOM, fast !

• Very easy to integrate in a page (less invasive than Angular)

ReactJS

@elmanu#playframework

Building the new platform

Front-end & UI

@elmanu#playframework

Building the new platform

• Build pipeline using gulp.js & bower !

• Assets pipeline

• LESS compilation

• JS minification

• Upload assets to Cloudfront

• Integrates translations from Crowdin

• Washes the dishes

@elmanu#playframework

Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Summary & Future

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: source system(s)

@elmanu#playframework

Data migration: migration schedule

@elmanu#playframework

Data migration

• Akka to the rescue! !

• Concurrent user migration !

• Concurrent item migration

• Using Play’s WS library for async calls to Youtube and friends

@elmanu#playframework

Data migration

User migrator

Worker Worker Worker Worker Worker

@elmanu#playframework

Data migration

Item migrator

@elmanu#playframework

Data migration

Item migrator

User item migrator

User item migrator

User item migrator

@elmanu#playframework

Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

@elmanu#playframework

Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher

@elmanu#playframework

Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher

File uploader

File uploader

@elmanu#playframework

Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher

File uploader

Soundcloud worker

File uploader

@elmanu#playframework

Data migration

@elmanu#playframework

Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Summary & Future

@elmanu#playframework

Operations

Operations

@elmanu#playframework

Operations

@YourTwitterHandle#DVXFR14{session hashtag} @elmanu#playframework

Summar

y

& Futu

re

@elmanu#playframework

Summary & Future

• Play Framework does not get in the way

• Easy to deploy and operate

• Many, many integrations !

• Many plans for the future

• Live activity feed

• Similarity search !

• Talenthouse is hiring!

@YourTwitterHandle#DVXFR14{session hashtag} @elmanu#playframework

Q & A