from concept to clustered jac (jira.atlassian.com) - graham carrick
DESCRIPTION
The development of clustered JIRA was a complex project spanning more than a year, and resulting in significant changes to core components of JIRA. We will discuss some of the changes made to Lucene index architecture, caching and scheduling, and the migration of jira.atlassian.com as the very first production clustered JIRA.TRANSCRIPT
#atlassian
GRAHAM CARRICK • DEV MANAGER • ATLASSIAN • @GREM000
Clustering JIRAFrom concept to clustered JAC
Concept - Massive teams
Concept - Massive teams
Concept - Massive teams
“JIRA will never be clustered” - JIRA Architect
Our Principles
!
•Keep it simple!
•Boy scout - leave the code better than you found it!
Architecture
• Complete index on each node
• Synced via DB queue• Poll every 5s
Lucene index
• Complete index on each node
• Synced via DB queue• Poll every 5s
Lucene index
• Complete index on each node
• Synced via DB queue• Poll every 5s
Lucene index
• In memory caches• Use Ehcache• Distributed invalidation
pattern
Caches
• In memory caches• Use Ehcache• Distributed invalidation
pattern
Caches
• In memory caches• Use Ehcache• Distributed invalidation
pattern
Caches
• Schedule on all nodes or once per cluster
Scheduler
• Schedule on all nodes or once per cluster
Scheduler
• Schedule on all nodes or once per cluster
Scheduler
• Schedule on all nodes or once per cluster
Scheduler
• Database backed lock• Used sparingly but
needed for cases like upgrade tasks
Locking
• Database backed lock• Used sparingly but
needed for cases like upgrade tasks
Locking
APIs
• Makes the code more future proof• Consistency of API cross product• Collaboration cross product• Plugins
- Make it possible and try to make it easy for them- Restricted architecture choices
The hard work
141 Caches
More hard work
15 Locks
Even more hard work
200+ Synchronized blocks
Then the plugins
80+ Plugins each with caches, scheduled jobs, locks and synchronized block
But wait, there’s more
Complete rewrite of ranking algorithm in JIRA Agile
1
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance Testing
Performance
“Better than theoretically possible”
- Me
Performance
~200,000 users 270,000 issues
167 custom fields
~75,000 attachments68 workflows
~500,000 comments
139 projects
Dogfooding: jira.atlassian.com
Monitoring
Hardening: from Coal to Diamond
• User cache race• Invalidation vs. Eviction• Excessive locking
“We can do hard things” - JIRA Team
Questions?
Graham Carrick • Dev Manager • Atlassian • @Grem000
Ask live: bit.ly/ask-atlassian !
Log in with your email you used to register
Thank you!
Graham Carrick • Dev Manager • Atlassian • @Grem000