what industry and research can learn from open source
DESCRIPTION
At the MT-ITS 2013 conference Dresden , Dec 2013 Based on work from Erich Gamma, John Wiegand, Mike MilinkovichTRANSCRIPT
![Page 1: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/1.jpg)
Open Source Processes: Lessons for Research and Industry
Ralph MüllerDirectorEclipse Founda>on@ralph_mueller
December 2013
Dienstag, 10. Dezember 13
![Page 2: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/2.jpg)
Dienstag, 10. Dezember 13
![Page 3: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/3.jpg)
Dienstag, 10. Dezember 13
![Page 4: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/4.jpg)
Dienstag, 10. Dezember 13
![Page 5: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/5.jpg)
Dienstag, 10. Dezember 13
![Page 6: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/6.jpg)
Dienstag, 10. Dezember 13
![Page 7: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/7.jpg)
72 Projects, 58 MLOC
Dienstag, 10. Dezember 13
![Page 8: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/8.jpg)
Members of Eclipse
Dienstag, 10. Dezember 13
![Page 9: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/9.jpg)
Relying on an open and extensible platform
• Innovate using an open platform
• Play poker, not chess
Build this in and with open source, even if that means working with your direct
competitors.
Platform
Customer Value
Compete on products
Dienstag, 10. Dezember 13
![Page 10: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/10.jpg)
Space
PlaGorms and Ecosystems
Platform
Niches
Complementors
We are here
Dienstag, 10. Dezember 13
![Page 11: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/11.jpg)
Why an Open Source PlaGorm?• Open Source development model encourages open innova>on
– Openness, Transparency, Meritocracy– Vendor neutrality
• Open Source licensing allows compe>tors to collaborate on shared plaGorms– No requirement for royal>es.– No single control point of intellectual property
• Open Source business model encourages rapid adop>on of technology– It is free and easy to access
• Open Source can allow companies to disrupt the business models of their compe>tors
• Open Source can allow companies to disrupt supply chain issues
Dienstag, 10. Dezember 13
![Page 12: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/12.jpg)
Open Source Ques>ons
• Is Open Source chao>c?• How does development really work?• What about the Open Source community?• How do you manage community contribu>ons?
• How do you plan in Open Source?
Dienstag, 10. Dezember 13
![Page 13: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/13.jpg)
Meritocracy
Dienstag, 10. Dezember 13
![Page 14: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/14.jpg)
Transparency
Andrew Magill – flickr.com
Dienstag, 10. Dezember 13
![Page 15: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/15.jpg)
Openness
Chris J. Fry – flickr.com
Dienstag, 10. Dezember 13
![Page 16: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/16.jpg)
Key Success Factors
• Architecture
• Governance
• Process
Dienstag, 10. Dezember 13
![Page 17: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/17.jpg)
PlaGorm Modularity: The Eclipse Experience
Run-time
Plug-insPl
atfo
rm
New Plug-ins are First Class Citizens – same footing for everyone
Open API and commercially friendly licensing – Low barriers to Entry
Ease of Integration and Extensibility Spurs Innovation
Competition can take place on implementations – users decide winners
Successful Ecosystems are built on this model!
Dienstag, 10. Dezember 13
![Page 18: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/18.jpg)
Governance ≠ Management
Dienstag, 10. Dezember 13
![Page 19: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/19.jpg)
Eclipse Governance StructureBoard of Directors
Approves Strategy, Plans, Policies
Membership at LargeApproves Vision, Bylaws,
Builds the Ecosystem
Eclipse Management OrganizationEstablishes the Roadmap, Builds the Platform, Delivers the Vision
PMC 1
Architecture CouncilDefines & Maintains
Architecture
IWG A IWG B
Planning CouncilEstablishes Platform,
Release Plan
PMC 2 PMC 3 PMC 4 PMC 4 PMC 5 PMC 6 PMC 7
Dienstag, 10. Dezember 13
![Page 20: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/20.jpg)
Governance: The Project Lifecycle
Dienstag, 10. Dezember 13
![Page 21: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/21.jpg)
How is the Development Done?
milestonesfirst
APIfirst
endgame
retrospectives
always havea client
built tolast
continuousintegration
community involvement
new & noteworthy
early incremental planning
continuous testing
consume yourown output
componentcentric
drive with open eyes
validate
reduce stress
learn
enable
attract to latest
transparency
validateupdate
dynamic teams
show progress
enable
explore
validate
“The Eclipse Way”Erich Gamma and John Wiegand
Dienstag, 10. Dezember 13
![Page 22: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/22.jpg)
Open Source Rules• OS projects are highly structured
– explicit rules (more than in most closed source projects)– Who may change the source code?– Who is responsible for delivering?– Who decides about the architecture?– …
• Commit rights: public "meritocracy"– only a small number of developers can modify the source code:
commiaers– key architecture defined by a small team of lead developers– peer pressure among commiaers – con>nuous reviewing– con>nuous review and feedback by the community– contribu>ons from outside have to be reviewed by commiaers
Dienstag, 10. Dezember 13
![Page 23: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/23.jpg)
Planning
milestonesfirst
APIfirst
endgame
retrospectives
always havea client
built tolast
continuousintegration
community involvement
new & noteworthy
early incremental planning
continuous testing
consume yourown output
componentcentric
drive with open eyes
validate
reduce stress
learn
enable
attract to latest
transparency
validateupdate
dynamic teams
show progress
enable
explore
validate
Dienstag, 10. Dezember 13
![Page 24: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/24.jpg)
Forces of Influence
Eclipse Councils
Community & Members
product requirements
enhancementsfeature requestsbug votes
suggest improvementscommit to plan
Plan- public -
Planning Councilposts draft plan
Øplans start in embryonic form and are revised throughout the release cycle
Ømilestones/time boxes are fixed early on
Committers
Dienstag, 10. Dezember 13
![Page 25: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/25.jpg)
Planning• Release themes establish big picture
– Community input– Planning council new source of input
• Component teams define component plans• PMC collates ini>al project plan drae
– Tradeoff: requirements vs. available resources– commiaed, proposed, deferred
• Plan ini>ally spells out– themes– milestones– compa>bility (contract, binary, source, workspace)
• Plan is alive
Dienstag, 10. Dezember 13
![Page 26: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/26.jpg)
Ongoing Risk Assessment
• Address high risk items and items with many dependencies early
• Maintain schedule by dropping items (if necessary)– we will drop proposed items – we hate to drop commiaed items– prefer fewer completed items than more items in progress
• High risk items are sandboxed to reduce risk to other items– prefer to serialize highest risk items (to minimize integra>on pain)
Dienstag, 10. Dezember 13
![Page 27: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/27.jpg)
Collec>ve Ownership• Planning team meets at least once a week
– status– planning– iden>fica>on of cross-‐component issues– mee>ng notes posted to the developer mailing lists
• Dynamic teams are established for solving cross-‐component issues– one cross-‐component issue per dynamic team– members are key developers from all effected components– find, implement, and roll-‐out solu>on of the assigned cross component
issue– represented in the weekly planning calls
Dienstag, 10. Dezember 13
![Page 28: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/28.jpg)
Project Rhythm
milestonesfirst
APIfirst
endgame
retrospectives
always havea client
built tolast
continuousintegration
community involvement
new & noteworthy
early incremental planning
continuous testing
consume yourown output
componentcentric
drive with open eyes
validate
reduce stress
learn
enable
attract to latest
transparency
validateupdate
dynamic teams
show progress
enable
explore
validate
Dienstag, 10. Dezember 13
![Page 29: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/29.jpg)
Milestones• break down release cycle into milestones
– We use 6 weeks
• milestones are a miniature development cycle
– Plan, execute, test, retrospec4ve
• milestone builds are good enough to be used by the community
Ø milestones reduce stress, keep quality high
• before/aeer
t
quality
ready to ship
3.5 3.6“hanging rope”
Dienstag, 10. Dezember 13
![Page 30: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/30.jpg)
Con>nuous Integra>on
• Fully automated build process• Build quality verified by automa>c unit tests • Staged builds
–nightly builds (some projects even more frequently)• discover integra>on problems between components
–weekly integra>on builds• all automa>c unit tests must be successful• good enough for our own use
–milestone builds• good enough for the community to use
Dienstag, 10. Dezember 13
![Page 31: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/31.jpg)
Prac>ce Makes Perfect
• 7 milestones, 4 release candidates– 11 chances to prac>ce releasing
• Projects denoted N0, N1, N2, N3 – Build in order of dependencies– Early builds takes days, later builds take hours
• Build to shared repository, make everything available to the community for feedback and tes>ng
Dienstag, 10. Dezember 13
![Page 32: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/32.jpg)
Gemng on the Train
M1# N0#
M2# N0# N1#
M3# N0# N1# N2#
N1#M4# N0# N1# N2# N3#
Dienstag, 10. Dezember 13
![Page 33: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/33.jpg)
Constant Public Status Repor>ng
Dienstag, 10. Dezember 13
![Page 34: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/34.jpg)
Community Involvement• An ac>ve community is the major asset of an OSS project• OSS project gives and takes:
– OSS developer gives: • listen to feedback and react• demonstrate con>nuous progress • transparent development
– OSS developer takes:• answer user ques>ons so that developers do not have to do it• report defects and feature requests• validate technology by wri>ng plug-‐ins• submit patches and enhancements
• Give and take isn’t always balanced– community isn’t shy and is demanding
Dienstag, 10. Dezember 13
![Page 35: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/35.jpg)
Tes>ng
milestonesfirst
APIfirst
endgame
retrospectives
always havea client
built tolast
continuousintegration
community involvement
new & noteworthy
early incremental planning
continuous testing
consume yourown output
componentcentric
drive with open eyes
validate
reduce stress
learn
enable
attract to latest
transparency
validateupdate
dynamic teams
show progress
enable
explore
validate
Dienstag, 10. Dezember 13
![Page 36: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/36.jpg)
Tes>ng • Innovate with confidence
• Tests run aeer each build
• Test kinds– correctness tests
• assert correct behavior– performance tests
• assert no performance regressions– based on a database of previous test run measurements
– resource tests, leak tests• assert no resource consump4on regressions
Defects Testing
Kent Beck – JUnit handbook
Dienstag, 10. Dezember 13
![Page 37: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/37.jpg)
Unit Test Report
Dienstag, 10. Dezember 13
![Page 38: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/38.jpg)
Performance Test Report
Dienstag, 10. Dezember 13
![Page 39: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/39.jpg)
Before (M5) – Aeer (M7)
Dienstag, 10. Dezember 13
![Page 40: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/40.jpg)
Code Coverage
Dienstag, 10. Dezember 13
![Page 41: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/41.jpg)
API Conformance Tes>ng
Dienstag, 10. Dezember 13
![Page 42: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/42.jpg)
End Game
milestonesfirst
APIfirst
endgame
retrospectives
always havea client
built tolast
continuousintegration
community involvement
new & noteworthy
early incremental planning
continuous testing
consume yourown output
componentcentric
drive with open eyes
validate
reduce stress
learn
enable
attract to latest
transparency
validateupdate
dynamic teams
show progress
enable
explore
validate
Dienstag, 10. Dezember 13
![Page 43: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/43.jpg)
The Annual Schedule
Q1Q42012
Q2Q3
3.83.7 M6M5M4M3M2M1 M7 RC2
• convergence process applied before release
• sequence of test-‐fix passes– it is a community event!
End Game
Dienstag, 10. Dezember 13
![Page 44: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/44.jpg)
End Game Convergence
• with each pass the costs for fixing are increased– higher burden to work on fix for a problem– higher burden to release a fix for a problem– focus on higher priority problems
# fixed bugs608 301
85 fix passtest pass
time
447
May 21 May 28 June 11 June 20 June 25
eclipse 3.0 Release
Dienstag, 10. Dezember 13
![Page 45: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/45.jpg)
Decompression
• recover from release • retrospec>ve of the last cycle
– learn from the last cycle• achievements• failures
– “stay aware, adapt, change”– define retrospec>ve ac>ons
• start to plan the next release and cycle
Dienstag, 10. Dezember 13
![Page 46: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/46.jpg)
Where the Time Goes
• release cycle 12 months– milestones – 9 months– endgame – 2 months– decompression – 1 month
milestones
end game
decompression
Dienstag, 10. Dezember 13
![Page 47: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/47.jpg)
Conclusions
• Open source uses highly rigorous and disciplined processes
• Chose your plaGorm carefully• Adopt these principles:
– Meritocracy– Openness– Transparency
Dienstag, 10. Dezember 13
![Page 48: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/48.jpg)
48
Dienstag, 10. Dezember 13
![Page 49: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/49.jpg)
49
Dienstag, 10. Dezember 13
![Page 50: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/50.jpg)
50
Dienstag, 10. Dezember 13
![Page 51: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/51.jpg)
51
Dienstag, 10. Dezember 13
![Page 52: What Industry and Research can learn from Open Source](https://reader034.vdocuments.mx/reader034/viewer/2022051816/54559313b1af9fcf338b4932/html5/thumbnails/52.jpg)
Thank You!
Ques>ons?
the talk is based on materials by:Erich Gamma, John Wiegand
Mike Milinkovich
Dienstag, 10. Dezember 13