enterprise ci as-a-service using jenkins

1 Copyright ©2014 CollabNet, Inc. All Rights Reserved.

Enterprise CI-As-A-Service A Case Study in Jenkins and OSS Tools to Support CI at Scale

Enterprise Transformation Manager [email protected]

Brian Dawson


Presenter

Brian Dawson, Manager Enterprise Transformation, CollabNet

As a Consultant CollabNet, for 10 years Brian Dawson has helped software development organizations transform development processes through the use of TeamForge and development best practices, including Continuous Integration, Continuous Delivery, and Agile.

Prior to CollabNet, Brian spent 12 years in software development most recently at Sony Computer Entertainment where he worked as the Director of Tools and Technology and oversaw the development and deployment of TeamForge as a worldwide collaboration and reuse portal.


• Review of Continuous Integration

• CI In the Enterprise

• Case Study: CI-As-A-Service

• Conclusion

• Q & A

Agenda

4 Copyright ©2014 CollabNet, Inc. All Rights Reserved. 4 Copyright ©2014 CollabNet, Inc. All Rights Reserved.

Review of CI


– Fail fast so problems can be found and corrected early

– CI feedback

• Coding and architectural issues

• Unit tests

• Security issues

• etc

What is CI?

“Continuous integration (CI) is a software engineering practice in which isolated changes are frequently

integrated, immediately tested and reported on. The goal of CI is to provide rapid feedback so that if a

defect is introduced into the code base, it can be identified and corrected as soon as possible.”


Why CI?

• Automation of repetitive manual processes

• More time focusing on adding value

• No need to wait hours, days, months to know how check-in affects the

build

• Constant availability for the build for testing, demo or release

• Shared visibility

• Increased team productivity

• Higher quality software, faster!


CI In The Enterprise


CI In The Enterprise: Current State

– Rapid adoption

– Spanning languages and technologies

– “Shadow” Implementations (Internal and Cloud)

– Implemented at team level

– Implemented and maintained by developers


CI In The Enterprise: Issues and Challenges

– Server, application sprawl and/or external exposure

– Developers spending time as admins

– Lack of shared knowledge

– Slow project start-up and ramp-up

– CI disconnected from enterprise development processes

– Lack of security

– Lack of governance


Centralize Source Code • Reduce ramp-up administration overhead • Manage IP • Supports traceability, visibility and governance

Manage binaries • Reduce risk • Promote reuse

Support Security • Project, Dept. and/or organizationally aligned access controls

Naturally connect the build process to the common platform • Supports traceability, visibility and governance

CI in The Enterprise: Path to Success


Support Dynamically Scaling • Server provisioning

• Virtual Machines

Establish central build platform and/or eco-system • Flexible support for different teams needs

• Support easy on-boarding

Document process and shared knowledge • Develop community to support best practices

• Reduce ramp-up and onboarding time

CI in The Enterprise: Path to Success


Case Study: CI-As-A-Service


Case Study: Customer Overiew

Large Financial Services Company • 4000+ users

• 100’s of applications

• >100 teams

Globally Distributed • US, UK, Eastern Europe, Asia, etc

Multiple Technologies

• Java, .Net, Batch, etc

• Web, Desktop, Mobile,etc


Case Study: Goals and Challenges Goals

– Change the culture of development Create open-source development culture • Deliver productivity to the developer … for the developer • Interested, passionate developer can be 10x more productive than a developer “just doing his(her)

job”

– Provide standard build tool implementations

– Provide build to release management platform

– Leverage private cloud to enable scalable solution

– Provide a support structure

Challenges – Multiple technologies

– Multiple tools and processes

– Regulatory compliance mandates

– Scale


Case Study: Solution

Common Platform = TeamForge SCM = Managed Subversion CI Server = Jenkins w/Integration Binary Artifact Management = Sonatype Nexus w/Integration

Quality = SonarQube Sonar w/Integration Server/VM Provisioning = Lab Management

• VM Ware and EC2 • LM Profiles (IAAS) • Puppet for Config Management

Community Support • Dedicated Build Engineer • TeamForge Trackers • Wikis

Established common CI tooling, with rich integration to a common platform managed by a published process for CI server provisioning and support


Case Study: Process • All application development hosted in TeamForge Projects providing role-based access

controlled workspaces with common tools

• Projects use integrated Subversion for centralized administration and management of source code

• Process and shared knowledge documented in community support project wikis

• Build resources requested via TeamForge Tracker ticket with immediate notificaion to dedicated build engineer

• Jenkins server dynamically provisioned from internal Lab Management cloud using common pofiles and Puppet for configuration management

• Access provided to Jenkins through TeamForge project-level role-based access controls, providing project and role appropriate permissions

• Access provided to Nexus and Sonar via to TeamForge role-based access controls, providing access to common, department and project-specific binary assets.

• Build Environment handed off to team within hours with all necessary info provided via ticket and shared documentation

• Team has ability to configure environment and create jobs on-going support and server configuration is facilitated via Trackers by build engineer with High SLA

17 Copyright ©2014 CollabNet, Inc. All Rights Reserved. Copyright © 2008 CollabNet, Inc. All Rights Reserved.

~29 virtual machines

~20+ Jenkins Servers

1 Nexus Artifact Repository

1 Sonar Server

1 Puppet Server

1 OpenNMS Server

~100 Development Teams >500 Developers ~150 Applications

…Supported by 1 build engineer

Case Study: Current State


Case Study: Benefits Benefits

• Centralized build and test on a cloud platform

• Centralized support

• Increased productivity

• Decreased cost per developer

• Central instance SaaS and CIaaS

• Better economies of scale and higher availability

• Standard tools

• Lower support costs, fewer integration points, better resource mobility

• Central integration and “certification”

• Reduced effort per team, higher quality


Case Study: Reference Architecture

Automated Build/Test

Artifact Publication

Source Code Management

Product / Sprint Planning

Agile Project Development

Deployment Services

Build/Tag - Jenkins Unit Tests - XUnit Functional Tests - XUnit Static Code Analysis - Sonar Code Coverage – Sonar

Essential Services

Dependency Management Governance Process Artifact Reuse

Runtime Dependencies Governance Process Change Management Test Manager

Artifact Creation / Management Monitoring and Analysis Data Consolidation / Reporting Role Based Access Control

Full Subversion integration as well as future integration and support for Git

Focus on automation especially in the test area

Planning: TeamForge, SWP Development: Eclipse,Visual Studio Code Review: ReviewBoard

SCM Management: Subversion, Git

Prod UAT Test

Development Data Warehouse


Conclusion


CollabNet Solutions: TeamForge

TeamForge provides the essential components needed to drive collaborative software development and deployment: Integrated ALM development and collaboration tools, flexible

and repeatable process templates, and a cross-project collaboration architecture.

Reporting & Governance search

traceability documents

wikis

discussions

Release

Lab Management

Build & Test

Track

Code

Plan

My page

Reports

Projects

Monitoring

My settings

Open ALM

Collaboration Architecture

Communities

My workspace Deploy

GitEye

Orchestrate

Process Templates

Dashboards & Report Catalog

Desktops


Agile Delivery & DevOps

Agile Development

Enterprise SCM

Training Training

Implementation Implementation

Consulting Consulting Consulting

Solutions

Product

Services

TeamForge

Enterprise Agility Assessment Service

Solution Pack Solution Pack

Blueprint for Enterprise Agility

DevOps Applied Workshop

Agile Dev Applied Workshop

Custom Continuous Integration Implementation

Private/Public Agile, CSM and ALM Training

CollabNet Solutions: Products and Services


• Solution Pack solutions: • Enterprise SCM leverages TeamForge SCM

(Git and SVN) Integration

• Agile Development (CI/CD) includes Enterprise SCM

• Agile Delivery & DevOps Solution includes Agile Development

CollabNet Solutions: TeamForge Solution Packs

Agile Delivery and DevOps • Build + Release Workflow (Jenkins and ARA) • Auto-build of build and release trackers • Nexus integration – binary repos

Agile Development • Build Workflow (Jenkins) • Auto-build of build trackers • TeamForge File Release System integration

Enterprise SCM • TeamForge template – enterprise agility

template • Dashboard template component • Alerts component

• Solution Packs provide Enterprise Visibility and Traceability promote repeatable, solutions for Agile Development and Delivery (Process and Tools)

• Solution Packs provide integrated solutions (Jenkins, Team City, ARA +)


› Across user stories, sprints, releases, products, teams

› Within workgroups—and across your organization

› Enterprise-grade security with RBAC, permissions management, authentication, encryption, auditing

› For creation and sharing of common, repeatable processes across your organization

› Throughout the development lifecycle and across projects, tools, and teams

With CollabNet Solutions, You Gain

› Organization-wide discovery, collaboration, re-use, and reporting


Questions?

blogs.collab.net twitter.com/collabnet www.facebook.com/collabnet www.linkedin.com/company/collabnet-inc

http://www.facebook.com/collabnet

http://www.linkedin.com/company/collabnet-inc

http://www.facebook.com/collabnet

http://www.linkedin.com/company/collabnet-inc


Thank You

enterprise ci as-a-service using jenkins

Technology

enterprise ci

consultant collabnet

review of ci

continuous integration

goal of ci

development best practices

continuous delivery

years brian dawson