extending perforce scalability using job content synchronization
DESCRIPTION
Splitting a large depot can be a daunting task, and a hidden difficulty is the impact to the bug tracking system. This session will explain how Job Content Synchronization allows multiple depots to share a single link with the bug tracking system, eliminating needed changes and significantly reducing split preparation time. Adding new depots and supporting further splits is dramatically simplified.TRANSCRIPT
![Page 1: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/1.jpg)
Extending Perforce Scalability Using Job Content Synchronization Shannon Mann Software Developer Research In Motion
![Page 2: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/2.jpg)
Outline
• Problem Definition • Brief History • Investigation
• Solution: Job Content Synchronization • JCS Concept • JCS Details
• Launch • Implementation • Benefits • Concerns
• Wrap-up • Conclusions • Questions
![Page 3: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/3.jpg)
Problem Definition
![Page 4: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/4.jpg)
Brief History
• Early 2000 – first Perforce use • Mid 2005 – large scale automation introduced
• Automated parallel builds • Dramatic growth of continuous integration • Performance issues began
• Late 2006 – performance severe • Decision to split
• Mid 2007 – split not possible at this time • Code inter-dependencies too high
![Page 5: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/5.jpg)
Brief History
• Mid 2007 - Alternatives sought • Monitoring introduced • Hardware replaced - Sparc to Opteron • OS replaced – Solaris to Linux • Performance improved 10X
• Continued accumulated improvements by 100X • Initial 2007 samples at 22 seconds, now .2 seconds
• Userbase activities increased by 6X • Early 2010 - Decision to split revisited
![Page 6: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/6.jpg)
Investigation
• Decision to split by duplicating db and depot data • separate each depot’s trees with protections • best done with explicit protections
• Reduce cost with post-split cleanup • clientspec cleanup reduces db.have size • Offline obliterates completes split
• leaves depot files as-is – expensive • Depot cleanup needed – future research topic
![Page 7: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/7.jpg)
Investigation
• Split Challenges • large number of tool integrations • multiple groups with conflicting needs • moving product delivery dates
• Bug-tracking integration complex – show-stopper • Bug-tracker itself integrated with many tools • Integration complexity dramatically increased • Alternative desperately needed
• Goal - avoid changing bug-tracker integration
• Job Content Synchronization concept saves project
![Page 8: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/8.jpg)
Solution: Job Content Synchronization
![Page 9: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/9.jpg)
JCS Concept
• Middleman depot allows existing bug-tracking integration unchanged
• JCS copies job data from middleman to production
• Perforce Support gave enthusiastic support • Produced reference solution in two weeks
![Page 10: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/10.jpg)
JCS Concept
Bug Tracker
Original Depot
P4DTG
Bug Tracker
P4Jobs Depot
P4DTG New Depot
JCS
New Depot
![Page 11: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/11.jpg)
JCS Details
• Stand-alone depot has P4DTG integration with bug-tracker • Needs P4Broker to redirect ‘p4 jobs’ • Needs P4Auth to make redirect transparent • Needs P4Change to make connections unique • Needs triggers to request initial job data • Uses P4DTG in a depot-to-depot mirror of job data
![Page 12: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/12.jpg)
JCS Details
Bug Tracker
P4Jobs Depot
New Depot
P4Auth P4Change
P4Broker
Users
P4DTG
JCS
![Page 13: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/13.jpg)
Launch
![Page 14: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/14.jpg)
Implementation
• Tested using full test implementation • All parts must be present • Users must test their tool integrations
• Staged production release • P4Broker, P4Auth and P4Change implemented first • Split done at later date – reduces risk • Additional depots added as needed
![Page 15: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/15.jpg)
Benefits
• Greatly increased scalability
• Depot-side and bug-tracker side connections increase linearly and independently
• Near-trivial low-impact scalability
• Non-split depots benefit – bug-tracking connect for free
![Page 16: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/16.jpg)
Concerns
• Significant increase in deployed complexity
• Expert knowledge of subsystems now required
• Complexity extends investigations and corrections
![Page 17: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/17.jpg)
Concerns
Bug Tracker
Original Depot
Users Original Setup
![Page 18: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/18.jpg)
Concerns
Bug Tracker
P4Jobs Depot
Users
P4Auth P4Change
New Depot
One Depot w/JCS
![Page 19: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/19.jpg)
Concerns
Bug Tracker
P4Jobs Depot
Users
P4Auth P4Change
New Depot
New Depot
Two Depots w/JCS
![Page 20: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/20.jpg)
Wrap-up
![Page 21: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/21.jpg)
Conclusions
• JCS allows near-linear scaling of bug-tracking interconnects to multiple depots
• JCS allows a reduction in performance impact
• JCS eliminates split impact to bug-tracking system integrations
![Page 22: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/22.jpg)
Conclusions (cont.)
• JCS also allows multiple non-split depots to share bug-tracking integration
• JCS can be used just for bug-tracking integration sharing • Splits aren’t required to consider using
• P4Auth, P4Change and P4Broker can be used alone
• Perforce Support are your friends. Seek their advice
![Page 23: Extending Perforce Scalability Using Job Content Synchronization](https://reader034.vdocuments.mx/reader034/viewer/2022051313/5484012cb4af9f9b0d8b4a96/html5/thumbnails/23.jpg)
Questions?