jenkins scalability summit

©2013 CloudBees, Inc. All Rights Reserved 1©2013 CloudBees, Inc. All Rights Reserved

Jenkins Scalability SummitOct 2013

©2013 CloudBees, Inc. All Rights Reserved 2

Logistics• Where is …?• Network


Agenda Part 1: Story time• Learn from our collective experience• Identify high-priority problems• What is hurting serious users?• Concrete details are good


Agenda Part 2: Discuss & Design• Collectively pick a few topics• Split into 2 tracks

• See if we can start shaping up solutions

• Please keep shared notes– Details!


Possible topics includes …• CPU/memory/disk/network/… consumption• Stability/diagnosability of slaves• Access control of builds/UI• Organizing jobs and build records• Master to master communication• Workflow / choreography• User interface• Stability of Jenkins releases• Plugin compatibilities


Goals• No one goes listen-only mode• Produce notes to show to the broader

community• Figure out how to do this better next

year


What We’ve Done&

What We Can


Thread Reduction• Per-slave consumption– SSH (down to 1)– Channel (down to 1)

• Executor thread on demand


Thread Reduction: TODO• NIO• Asynchronous job execution–More about this in workflow


Memory reduction• Lazy loading of build records• Database plugin


Design Choices That Constrain Us• Compatibility– Adding is doable, changing is often hard

• Heterogeneousness in data model• Thread-driven execution model


CloudBees Scalability Efforts


Multi-master / Meta Jenkins• Master-to-master channel• Extensions on top of it– “Cloud” impl to lease slaves– Push security realm– Push update center–…


Multi-master• Where we think we can go with this–Move jobs around– Sharding with UI mashup

• Where this wouldn’t take us– Loss of inflight builds– True horizontal scaling


Workflow• Brand new job type• Kill multiple birds in one stone• Inspired by buildflow & jenkow


Ingredients• No single thread to execute the whole

thing– Analogous to NIO– Check pointing

• BPMN-like workflow execution model– Surface syntax independent

• Groovy DSL–With continuation-passing style execution

model• Post-execution visualization– But no pre-execution visualization


Ingredients• Unified properties, build variables, and

environments• Open up direct access to file store• Hands-free process forking• Interop with existing job types

jenkins scalability summit

Documents

rights reserved2013

good2013 cloudbees

can2013 cloudbees

workflow2013 cloudbees

year2013 cloudbees

buildflow jenkow2013

executor thread

master channelextensions