tech foundations-slides
Post on 13-Apr-2017
335 Views
Preview:
TRANSCRIPT
Learning Outcomes
How we architect apps and services
How we build insanely great products
How code gets from development to production
What systems power LinkedIn and keep them stable
Where to get the information and data you need
LinkedIn’s Core Values
Our members come first�
Relationships matter�
Be open, honest, and constructive�
Demand excellence�
Take intelligent risks�
Act like an owner�
go/values
The leadership team
Reid Hoffman Executive Chairman & Co-Founder
Jeff Weiner CEO
Nick Besbeas VP
Marketing
Mike Gamson SVP
Global Solutions
Jeff Weiner Interim Leader Product & UX
Kevin Scott SVP
Engineering
Steve Sordello SVP & CFO
Shannon Stubo VP
Corporate Comm
Pat Wadors VP
Talent Org
Mike Callahan General Counsel
& Secretary
Product Development
ProductOperations
Product Development
Engineering
Operations
Product Management
User Experience
Web Dev
Data Science
go/product-teams
LinkedIn’s new experience methodology designed to be simple, unified, delightful, and actionable.
Product Vision
“Build insanely brilliant and simple products that change people's lives” - Deep Nishar
Product Vision
“Build insanely brilliant and simple products that change people's lives” - Deep Nishar
LinkedIn’s experience methodology is designed to be simple, unified, delightful, and actionable.
go/katy
Increase engagement through content translation, localization and geo-customization.
go/i18n
US
94MMNON-US
186MM
Internationalization and Localizationi18n and L10n
Feature scoped
1
Features designed
and reviewed
2
Sprint planning and resources
allocated
3
Specification received from
product manager /or tech lead
Initiate contact with SRE, and stay in touch!
Feature Development Flow
Feature Development Flow
Sprint planning and resources
allocated
Specification received from
Product Manager /or Tech Lead
4
Test Engineer begins test
planning and strategy
5
Developers (back-end/ front-end) write code
6
Code committed and pushed to
integration environment
Initiate contact with SRE, and stay in touch!
Remember to coordinate ramp plan with SRE
Remember to coordinate ramp plan with SRE
Developers (back-end/ front-end) write code
Code committed and pushed to
integration environment
7
Test Engineer begins manual and automated testing
8
Sign-off from Service Owner, then
devise release and ramp plan
9
Ramp until 100% to all members in production
Remember to coordinate ramp plan with SRE
Feature Development Flow
Code committed and pushed to
integration environment
Test Engineer begins manual and automated testing
8
Sign-off from Service Owner, then
devise release and ramp plan
9
Ramp until 100% to all members in production
10
Remember to coordinate ramp plan with SRE
Feature Development Flow
Product Design
Product design sign-offOnly User Experience
Design (UED) leads/managers
Product sign-off
Executives give final approval
BAM office hours
Brand and Marketing (BAM) team and Steve Johnson
go/productreview
Using the vision from Product, the visuals from Design and the infrastructure from Engineering, Web Developers bring the
User Experience to life.
Web Development
go/webdev
Everyone can help! Use our products daily to find bugs and suggest enhancements!
Test Engineers
Test Engineers live by the motto “proud to ship with high quality”.
!Features only make it into production if our
Testing Teams are proud to release it.
How we work: Agile/Scrum
10-20 days
Daily Scrum meeting
IterationIteration backlog
Product backlog as prioritized by Product owner
Backlog tasks expanded by team
Potentially shippable product
increment
go/agile
Aspirational Architecture
“Provide abstractions that make it easier to reason about the LinkedIn architecture.”- Swee Lim, Distinguished Software Engineer
go/superblock
Front end
Business logic layer
Super BlockData access layer
Infrastructure
Mid Tier
Framework and language agnostic way to handle requests for data from internal services.
Rest.li
Profile
go/restli
Fizzy
Combine UI components from different applications into a single page. !Server side (i18n) and client-side (embeds) rendering logic.
go/fizzy
Template Dust
LinkedIn’s template engine for server and client side rendering
Dust
=
Output HTMLData
go/howtodust
QuickDeploy against EI (QEI)
Selectively deploy and run services during development. !Requests for other services handled by our Early Integration environment.
Profile
go/qei
From request to front-end
*PoP may be in a data center that is not running
the full app stack
Point of Presence*
NetScaler External Load Balancers (SSL)
L0 Proxy Apache Traffic Server
L1 Proxy Apache Traffic Server
HAProxy
Context path resolution E.g. /api/
Move all traffic to HTTPS !Enhance security and protection !Redirect logic (mobile devices) !Map url to applications !Multi-colo routing logic
LinkedIn Data Center
Application Containers
go/multicolo-routing
How code gets to production
ReviewBoard approved
!Commit/Push
Run unit tests on CI server
!(1-5 mins)
PCS (integration tests)
PCL Tests the health
of trunk
If unit tests pass, code committed pushed
!Artifact built
PRE PCLPCS
SRE’s notified prod deploy
is about to occur
How code gets to production
Publish artifact (certified good)
Service Owner certifies build is ready to go to
production
Automated service certification.
Sends the status back to CRT
Ramp plan should already have been identified
Artifact is pushed to EI/EI2
EI EI2
How code gets to production
Service Owner certifies build is ready to go to
production
Automated service certification.
Sends the status back to CRT
SRE’s notified prod deploy
is about to occur
Ramp plan should already have been identified
Publish artifact (certified good)
Artifact is pushed to EI/EI2
EI EI2
Artifact deployed to production canary
machines.
If canary looks good, deploy to all
production nodes
LIX experiment used to control ramp to members
LID - LinkedIn Deployment Tool
Deployment
Next generation deployment tool for fine grained control of distributed systems at LinkedIn.
Artifactory
go/lid
CRT
Check the status of your checkin from commit to deployment.
go/crt
How do you know where your code is?
Trunk Development
Trunk development - no branches
If unit test fails in PCS. Auto-revert. If failed in PCL, lock trunk and investigate.
Once issue confirmed resolved, unlock trunk.
“Ship it” from code owners and reviewers. Pass local unit tests and commit to trunk.
Issue identified through CRT. Fix committed.
go/mp
PCS
PCL
Network and Multiproduct
Network Trunk Multiproducts
Network trunk: original svn repository !Multiproduct: Code management containers for testing, building, QA and deployment outside of network trunk. !Mint: Tool for managing multiproducts.
FlexibleModular Dependency ManagementStandardized
go/mp
Code reviews
Small atomic commits. !How well are you solving the problem? !Accompanying unit tests and documentation a must.
��
go/codereview
Ship it!
Why documentation matters
"Instead of imagining that our main task is to instruct a computer what to do, let us
concentrate rather on explaining to human beings what we want a computer to do."
- Donald Knuth
Operations group
Ensure LinkedIn has the infrastructure and resources
needed to be stable at a huge scale.
go/siteops
Autometrics
“Self-service” model for collectively adding metrics into our graphing system.
go/autometrics
Ingraphs
Dynamic visualization for the data collected by Autometrics and other sources.
Week-over-week comparisons
Real-time browsing
Personal dashboards
go/ingraphs
LiX (LinkedIn Experimentation Platform)
Allows for A/B test framework to allow new, modified, or experimental features to be deployed to a small segment of the LI user base. !Also used to carefully control the ramp of new features in production.
20%80%
go/lix
The NOC
Centralized Team for site monitoring and status reporting of GCNs. ! #noc
Network Operations Center
go/noc
Akamai, Limelight and Level 3
Content Delivery Networks (CDNs)
Delivers content to our members with high availability, performance and reduced latency.
Critical site issue escalation
If you see something that doesn’t look right,
report it!
go/issuecritical
1-855-832-LINK
site-issue-critical@linkedin.com
go/issuecritical
Non Critical Email: site-issue@linkedin.com
#noc
Security
Security is everyone’s concern. !
Members must feel that there data is safe with us.
go/learnsecurity
Using data to drive User Centered Design
Product Data Bootcamp
Short course geared toward new Product Managers on the basics of using our massive amounts of data as a PM.
Taught by data scientists and other data experts once every quarter.
go/productdatabootcamp
Data Science
The Data Science Team delivers valuable data to help us understand the
member and their site usage.
A/B testing
Exposing demographic trends
Product analytics and data and business insights
XLNT
XLNT is an end-to-end A/B testing solution, providing data for decisions on product changes.
go/XLNT
Informed
go/informed
A stream of micro-updates about deployments in production and staging.
Web UI, IRC bot, CLI, REST API for reading and writing updates
Tech Life at LinkedIn
Hackdays�
Weekly Tech Talks�
Horizontal Initiatives�
Incubator Program�
Functional Leadership Groups�
Knowledge Sharing�
go/values
Hack, Collaborate, Share and Learn
Learning More
Peoplego/cinco
InfoWiki
go/searchin
Code
go/codesearch-preview go/codingconventions
ResourcesLearnIn
Tech Talks
Data storage and distribution
Data Systems
Espresso
go/espresso
Also known as Pegasus, Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery
and simple asynchronous APIs.
Voldemort
go/voldemort
Voldemort is an open source distributed key-value storage system developed at LinkedIn.
Databus
go/databus
Source-agnostic distributed change data capture system, which is an integral part of
LinkedIn's data processing pipeline. !
Also used for offline data replication.
Tech
System Foundation
Fizzy
go/fizzy
Fizzy is a piece of LinkedIn infrastructure that is responsible for aggregating UI components served off of different web applications into a
single page.
Rest.li (Pegasus)
go/restli
Rest.li is a REST+JSON framework for building robust, scalable service
architectures using dynamic discovery and simple asynchronous APIs
Apache Traffic Server
go/ats
Fast, scalable and extensible reverse proxy and forward proxy HTTP server.
Tech
System Foundation
EKG
go/whatisekg
With canary validation, exception monitoring, GC monitoring and auto-tuning, EKG is our
tool for monitoring the health of our systems in production.
Hadoop
go/hadoop
Enables the distributed processing of large data sets across clusters of commodity
servers.
Avro
go/espresso
Apache Avro™ is a data serialization system, providing a serialization format for persistent
data and a compact binary format for communication between services.
Frameworks
System Foundation
LISpring
go/lispring
LI Spring is an extended version of Spring, the enterprise Java application development
framework with support dependency injection (wiring of components) and much more.
Play
go/play
Play is an open source, lightweight, fully reactive, stateless framework that
supports hot reload for Java and Scala.
Craig Sebenik Staff Site Reliability Engineer
!
Derek Brown Lead, Web Development
Subject Matter Experts
top related