javaone'17 talk: play on gradle and the midtier behind linkedin.com
TRANSCRIPT
@LinkedInEng
Play on GradleMidtier Behind linkedin.com
JavaOne - San Francisco - 10/2017
● Yiming Wang, Szczepan Faber (LinkedIn Dev Tools)
● Gary Lin (LinkedIn Play team)
● Vinyas Maddi (Flagship Productivity)
● Zachary Glagola (not here today)
● We are hiring!
Make the impossible possible
Make the possible ellegant
@LinkedInEng
Play @LI● 300+ apps, 100s of engs
● SBT -> Gradle = productivity
● Play on Gradle project
○ How to drive disruptive
change in a huge company?
○ How to build Gradle
plugins at scale?
@LinkedInEng
Why of Play @LI?● Avoiding thread pool hell
● Really, really hot reload
● Unfulfilled hope for Scala
to improve productivity
● https://engineering.linkedin.com/play/play-framework-linkedin● https://www.youtube.com/watch?v=8z3h4Uv9YbE&t=3s
@LinkedInEng
History of Play @LI● Rollout in 2013
● Hitting problems of scale
○ Hot reload?
○ Dep. Mgmt.
○ Workarounds in build logic
(SBT)
@LinkedInEng
History of Play on Gradle● Play on Gradle?
○ Dep. mgmt (ivy)
○ IDE import
○ Dual infra
○ Dominant build system @LI
(c++, python, go, mobile, Java)
○ Gradle partnership
@LinkedInEng
Play on Gradle● 2014 - sponsored
● 2015 - OSS completed
● 2016 - deprioritized @LI
● 2017 - in production!
@LinkedInEng
2016Q4 - restart● Rapid prototyping
● Identify unknowns
● Getting metrics, soliciting help
● Forming cross-org team
○ Foundation + Product teams
● Engaging with OSS
@LinkedInEng
linkedin.com architecture● Architecture of linkedin.com (“Flagship”),
probably the biggest Play app in the
world
○ Frontend: Play (serves index.html) + emberJS
○ Frontend API: Play (serves JSON)
○ Mid tiers: Play (serves JSON)
● Linkedin.com Frontend API is the biggest
to migrate - 3-quarter effort
@LinkedInEng
● linkedin.com (flagship) is modern single-page app
● LinkedIn has more products than linkedin.com
● Traditional multi-page apps from LinkedIn: Play + Dust
● The hardest to migrate to Gradle
Beyond flagship
@LinkedInEng
2017 - Play on Gradle in Production● 310 total play apps
○ ~40 Frontend apps (modern)
○ ~160 Frontend API + Mid tier
○ ~110 Frontend apps (Play+Dust)
● 110 already migrated to Gradle
○ ~40 Frontend apps (modern)
○ ~60 Frontend API + Mid tier
○ ~10 Frontend apps (Play+Dust)
○ Examples: LinkedIn, LinkedIn Learning, LinkedIn Lite
○ Linkedin.com Frontend API - coming soon!
@LinkedInEng
Gradle vs SBT
● Stats (all measured by
application owners)
● Very useful Gradle parallel
build (100% faster)
● Gradle “-x” is the king (highly
flexible)
Speed up Large scale app
example
Build 2-5 x 36 min -> 9 min
IDE refresh 5-8 x 30 min -> 4 min
Hot reload 2-5 x 7 min -> 1.5 min
@LinkedInEng
SBT -> Gradle Migration Effort● Most migrations take 2-4 hours, some take months
● Migrations are carried out by application owners (decentralized efforts)
● Provide good migration guides (self-service)
● Set a few good example projects to cover all SBT use cases
● Provide great tools so that teams are motivated to adopt and experiment
@LinkedInEng
Using and learning Gradle● Learning curve
● Debugging &
forking OSS
● Pull Requests
● Sponsoring more
features
@LinkedInEng
Gradle best practices● Java 8 for code
● Groovy for tests (spock!) and
Gradle DSL special cases
● It’s not enough that the task works,
it needs to be incremental
● Modular tasks, well architected,
rewrite when required
● Push with high confidence -
automatically testing with all apps
● Equally important: build code, test
code, production code
@LinkedInEng
Our tactics● Porting our custom SBT plugins
● Fallback strategy: side-by-side
SBT & Gradle
● Incremental adoption: first
developers, then deployment
● Rewriting existing Gradle
plugins @LI for
quality/incremental
● Avoiding new Gradle
configuration model
@LinkedInEng
PonG future at LinkedIn
● Play Frontends
● Gradle Distributed
Cache
● Co-ownership of OSS
● Ambition: steer Play
towards Gradle
@LinkedInEng
Top takeaways● Careful with frameworks
with custom build tool
● Top-down VS bottom-up
○ It’s what you don’t know that
you don’t know that gets you
into trouble
● Gradle: make impossible
possible, make possible
ellegant.
● Cross-org teams rock!
● Pragmatism
@LinkedInEng
You deserve great toolsLinkedIn talks at JavaOne:
● Play on Gradle - Midtier behind
linkedin.com
(this talk - hope you liked it!)
● Commit-to-production pipeline
at LinkedIn
(slides+video on slideshare.net)
Stay tuned for videos and summaries
at LinkedIn Engineering Blog
(@LinkedInEng)
@LinkedInEng
Thank you!!! Questions?Play on Gradle: Midtier Behind linkedin.com
JavaOne - San Francisco - 10/2017
● Yiming Wang, Szczepan Faber (LinkedIn Dev Tools)
● Gary Lin (LinkedIn Play team)
● Vinyas Maddi (Flagship Productivity)
● Zachary Glagola (not here today)
● We are hiring!
Make the impossible possible
Make the possible ellegant