javaone'17 talk: play on gradle and the midtier behind linkedin.com

@LinkedInEng

Play on GradleMidtier Behind linkedin.com

JavaOne - San Francisco - 10/2017

● Yiming Wang, Szczepan Faber (LinkedIn Dev Tools)

● Gary Lin (LinkedIn Play team)

● Vinyas Maddi (Flagship Productivity)

● Zachary Glagola (not here today)

● We are hiring!

Make the impossible possible

Make the possible ellegant

@LinkedInEng

Play @LI● 300+ apps, 100s of engs

● SBT -> Gradle = productivity

● Play on Gradle project

○ How to drive disruptive

change in a huge company?

○ How to build Gradle

plugins at scale?

@LinkedInEng

Why of Play @LI?● Avoiding thread pool hell

● Really, really hot reload

● Unfulfilled hope for Scala

to improve productivity

● https://engineering.linkedin.com/play/play-framework-linkedin● https://www.youtube.com/watch?v=8z3h4Uv9YbE&t=3s

@LinkedInEng

History of Play @LI● Rollout in 2013

● Hitting problems of scale

○ Hot reload?

○ Dep. Mgmt.

○ Workarounds in build logic

(SBT)

@LinkedInEng

History of Play on Gradle● Play on Gradle?

○ Dep. mgmt (ivy)

○ IDE import

○ Dual infra

○ Dominant build system @LI

(c++, python, go, mobile, Java)

○ Gradle partnership

@LinkedInEng

Play on Gradle● 2014 - sponsored

● 2015 - OSS completed

● 2016 - deprioritized @LI

● 2017 - in production!

@LinkedInEng

2016Q4 - restart● Rapid prototyping

● Identify unknowns

● Getting metrics, soliciting help

● Forming cross-org team

○ Foundation + Product teams

● Engaging with OSS

@LinkedInEng

linkedin.com architecture● Architecture of linkedin.com (“Flagship”),

probably the biggest Play app in the

world

○ Frontend: Play (serves index.html) + emberJS

○ Frontend API: Play (serves JSON)

○ Mid tiers: Play (serves JSON)

● Linkedin.com Frontend API is the biggest

to migrate - 3-quarter effort

@LinkedInEng

● linkedin.com (flagship) is modern single-page app

● LinkedIn has more products than linkedin.com

● Traditional multi-page apps from LinkedIn: Play + Dust

● The hardest to migrate to Gradle

Beyond flagship

@LinkedInEng

2017 - Play on Gradle in Production● 310 total play apps

○ ~40 Frontend apps (modern)

○ ~160 Frontend API + Mid tier

○ ~110 Frontend apps (Play+Dust)

● 110 already migrated to Gradle

○ ~40 Frontend apps (modern)

○ ~60 Frontend API + Mid tier

○ ~10 Frontend apps (Play+Dust)

○ Examples: LinkedIn, LinkedIn Learning, LinkedIn Lite

○ Linkedin.com Frontend API - coming soon!

@LinkedInEng

Gradle vs SBT

● Stats (all measured by

application owners)

● Very useful Gradle parallel

build (100% faster)

● Gradle “-x” is the king (highly

flexible)

Speed up Large scale app

example

Build 2-5 x 36 min -> 9 min

IDE refresh 5-8 x 30 min -> 4 min

Hot reload 2-5 x 7 min -> 1.5 min

@LinkedInEng

User’s feedback

@LinkedInEng

SBT -> Gradle Migration Effort● Most migrations take 2-4 hours, some take months

● Migrations are carried out by application owners (decentralized efforts)

● Provide good migration guides (self-service)

● Set a few good example projects to cover all SBT use cases

● Provide great tools so that teams are motivated to adopt and experiment

@LinkedInEng

Using and learning Gradle● Learning curve

● Debugging &

forking OSS

● Pull Requests

● Sponsoring more

features

@LinkedInEng

Gradle best practices● Java 8 for code

● Groovy for tests (spock!) and

Gradle DSL special cases

● It’s not enough that the task works,

it needs to be incremental

● Modular tasks, well architected,

rewrite when required

● Push with high confidence -

automatically testing with all apps

● Equally important: build code, test

code, production code

@LinkedInEng

Our tactics● Porting our custom SBT plugins

● Fallback strategy: side-by-side

SBT & Gradle

● Incremental adoption: first

developers, then deployment

● Rewriting existing Gradle

plugins @LI for

quality/incremental

● Avoiding new Gradle

configuration model

@LinkedInEng

PonG future at LinkedIn

● Play Frontends

● Gradle Distributed

Cache

● Co-ownership of OSS

● Ambition: steer Play

towards Gradle

@LinkedInEng

Top takeaways● Careful with frameworks

with custom build tool

● Top-down VS bottom-up

○ It’s what you don’t know that

you don’t know that gets you

into trouble

● Gradle: make impossible

possible, make possible

ellegant.

● Cross-org teams rock!

● Pragmatism

@LinkedInEng

You deserve great toolsLinkedIn talks at JavaOne:

● Play on Gradle - Midtier behind

linkedin.com

(this talk - hope you liked it!)

● Commit-to-production pipeline

at LinkedIn

(slides+video on slideshare.net)

Stay tuned for videos and summaries

at LinkedIn Engineering Blog

(@LinkedInEng)

@LinkedInEng

Thank you!!! Questions?Play on Gradle: Midtier Behind linkedin.com

JavaOne - San Francisco - 10/2017

● Yiming Wang, Szczepan Faber (LinkedIn Dev Tools)

● Gary Lin (LinkedIn Play team)

● Vinyas Maddi (Flagship Productivity)

● Zachary Glagola (not here today)

● We are hiring!

Make the impossible possible

Make the possible ellegant