maven application lifecycle management for alfresco

Get your Enterprise Open Source ready

Application Lifecycle Management

best practices and success stories

from the ECM world

Monday, April 6, 2009

A bit of us

๏ Eng. Gabriele Columbro - ECM Product Strategy Engineer for Sourcesense (http://www.sourcesense.com)

๏ Open source integration and large content centric applications

๏ ICTU, NXP, GovUnited, Ministry of Finance


http://www.sourcesense.com

http://www.sourcesense.com

A bit of summary

1. Who needs ALM after all?

2. ECM: or A fool with a tool is still a fool

3. The Long Way to get open source under control

4. NXP demo

5. Calm Framework and Q/A


Before we start...

๏ What’s Application Lifecycle Management?

๏ Wikipedia says:

“Application lifecycle management (ALM) is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management”

๏ But the focus should be on processes rather than technologies


Before we start...

๏ What’s Enterprise Content Management?

๏ Wikipedia says:

“Enterprise content management (ECM) is the strategies, methods and tools used to capture, manage, store, preserve, and deliver content and documents related to organizational processes. ECM tools and strategies allow the management of an organization's unstructured information, wherever that information exists”

๏ Open Source technologies are growing fast in the J2EE world as opposed to proprietary solutions


ReALM of discussion

Application Lifecycle

Management

Qualiy

AssuranceTesting

Automation

Release

management

Reduced Time

To Market

Fast

Prototyping

Flattening the

learning curve

Standard

transition to

operations

Flattening

the learning

curve

Get 3rd

parties

under

control


Who wants ALM after all?


Enterprises need maturity

๏ Business requirements have to cope with applications and architectures modernization

๏ Large process immaturity facing complex architectures and requirements changes

๏ Symptoms: slow deployments to operations, manual intervention on servers, project failures, no or manual testing


Enterprise application lifecycle


Enterprise IT myths

๏ Open Source is BAAAAD! (a bit retro’ nowadays)

๏ Almost everything can be solved with that new technology

๏ We must definitely get on board another consultant of that specific open source product!

๏ INSTEAD: A proper process definition (requirements and IT processes) over open source is the real enabler


Open source and communities

๏ Open source is not just a matter of code

๏ Adopting the Bazaar model is not an easy job for monolithic enterprises

๏ By overlooking inbound open source software flows we enable outbound contributions

๏ Fully mastering open source dynamics enable enterprises to reduce TCO and raise ROI for their contributions/investments


The system integrators’ quest

๏ Main objective: “Produce arbitrarily complex integration solutions to produce added value for the final customer”

๏ Business perspective: Lower TTM (Time to Market) for integration solutions to be operational

๏ Repetitive activities: tech-specific POCs, hype Open Source integrations, inter customers copy-paste reuse


Common objectives

๏ A reproducible and scalable way to manage application build, test, release, maintenance and integration policies

๏ Prototyping of projects to fasten TTM by easy learning curve

๏ Maximize controlled reuse of your code assets

๏ Make tough things simple!


ECM or “a fool with a tool is still a fool”


ECM is a tough world

๏ “46% of large organizations report have more than 25 separate information repositories” “AIIM – ECM Implementation Trends”

๏ Majority of ECM implementations fails due to poor processes and decision making

๏ If ECM is your focus, content centric processes must be reviewed first, IT architecture and development processes can follow

๏ ALM and ECM maturity levels are bound to each other when creating a common Enterprise content repository


A (not always) simple recipe to approach ECM

1. Identify your content centric processes

2. Identify main business drivers to ECM application and customization

3. Define your enterprise content model with a meet in the middle approach

4. Support growing information consolidation and delivery architectures with strong decision making

5. Cope with complexity (requirements, people, technology) with Application Lifecycle Management


ALM and ECM tight coupling


But what target technologies are we talking about?



Target technologies



Target technologies

๏ Alfresco 3.0 EE as “the ECM Framework”



Target technologies


๏ Jboss AS 4.2.0 EE starring “the AppServer”



Target technologies


๏ Jboss AS 4.2.0 EE starring “the AppServer”

๏ Mysql 4 as the “DBMS”


Managing Alfresco applications๏ Alfresco is a fairly complex J2EE application (~70Mb)

and multiple WARs

๏ Our daily job is configure, customize and integrate it with additional functionalities and J2EE artifacts

๏ It’s distributed as a WAR and it’s build with Ant, but the open source provided SDK is not really enabling

๏ It’s modules (plug-ins) are called AMPs (Alfresco Module Packages) which get overlaid on the WAR

๏ Early 2007: first Maven based build for Alfresco extensions for Banca Popolare di Milano

๏ Alfresco community artifacts hosted by Sourcesense Maven2 private repository


The long way to mature ALM


NXP Case Study

๏ 40k employees global semiconductors enterprise

๏ Alfresco ECM chosen as replacement for Lotus Notes as ECM building block in a fully new open source architecture

๏ Several parties involved in the huge migration and (up to) 800 customization projects

๏ Content and code reuse as main rationale but no standard design and processes for ECM and ALM


Some field examples

๏ No Software Configuration Management in place and manual changes in production environments

๏ Manual error prone testing procedures and stressful UAT phases

๏ No way to integrate multiple applications and content silo effect high risk

๏ Multiple development standards and unmanaged versioning policies


A fun place to work ;)


Maturity step 1: A bottom up approach

๏ First Alfresco application built with Maven on SVN in early 2008, featuring:

๏ Embedded run in Jetty (maven-jetty-plugin)

๏ Documentation website (maven-site-plugin) and manuals (doxia-maven-plugin)

๏ Clear versioning and dependency management (easy upgrade of Alfresco version)

๏ Local deploy to Jboss/remote to deploy to Tomcat

๏ To be able to reuse this achievement we exploited the power of Maven Archetypes


Maven Alfresco Archetypeshttp://repository.sourcesense.com/maven2-sites/maven-alfresco-archetypes

๏ Public Release 1.0.0: June 2008

๏ Maven Alfresco Extension Archetype: Support for Alfresco WAR custom build with platform specific profiles

๏ Maven Alfresco Amp Archetype: Definition of a custom Maven packaging type to manage Alfresco modules

๏ Main features:

๏ Reproducible environment aware build

๏ Documentation and release support

๏ Support for WAR depending on AMPs

๏ Restore/LDAP integration/Run embedded in Jetty


http://repository.sourcesense.com/maven2-sites/maven-alfresco-archetypes

http://repository.sourcesense.com/maven2-sites/maven-alfresco-archetypes

Archetypes benefits

๏ Defining a pattern-ish vocabulary to refer to application prototypes assumes an architectural value

๏ Enables complex inter-dependency networks (WARs can depend on AMPs)

๏ They can be improved and updated centrally with changing needs from the enterprise

๏ Enable the system integrator to produce a working demo solution in a matter of minutes


Demo

๏ Maven Alfresco Extension Archetype

๏ Maven Alfresco AMP archetype

๏ Site generation

๏ Q/A


Limitations of 1.0.0 version

๏ Verbose POMs

๏ No support for centralized dependency management and application lifecycle management

๏ No focus on integration testing (very difficult with Alfresco) and continuous integration

๏ Covering only the development part of the application lifecycle

๏ E.g not yet mature for a complex organization and high target as NXP


Maturity step 2: Design and implement your ALM framework๏ Objectives:

1. Maximize reuse of code and content

2. Support multiple business processes in the same target application (Alfresco)

3. Enable clear communication of IT and business

๏ Strategy:

1. Share ALM values and outline processes

2. Set up a reference application development platform

3. Model your archetypes to match the target architecture

4. Exploit a controlled lifecycle management as enabler


ALM Core Values

๏ Application Lifecycle should be reproducible and environment independent

๏ Identify and respect your release

๏ If it’s not tested (automatically) then it’s not working

๏ It it’s not documented then it doesn’t exist

๏ Controlled integration is possible and should not limit business improvement

๏ Centralize common configuration while leave projects enough flexibility for special cases


Support processes outline

Project developmentProject initiation Project release


But what support technologies are we talking about?

Target technologies



Support technologies

Target technologies




Target technologies

๏ Apache Maven starring “the Application Lifecycle glue”




Target technologies


๏ Subversion playing the “Software Configuration Management”




Target technologies



๏ Atlassian Jira as the “Issue tracker and planning tool”




Target technologies


๏ Sonatype Nexus starring “the Enterprise Repository”






Target technologies



๏ Hudson in the role of “the Continuous Integration server”






Target technologies



๏ Hudson in the role of “the Continuous Integration server”

๏ Special guest, ThoughtWorks Selenium as the “functional testing framework”




Apache Maven as glue for enterprise open sourcehttp://maven.apache.org/

๏ Designed for scalability, reduces TTM on both simple POCs and complex architectures

๏ It allows extending the notion of application lifecycle to maintenance and back

๏ Despite not perfect, is the standard de facto in Open Source J2EE projects and has a strong community

๏ Integrates with almost any IDE now

๏ Because we can contribute and improve it !


http://maven.apache.org

http://maven.apache.org

Software configuration managementhttp://subversion.tigris.org/

๏ Best practices:

1. Standard trunk/branches/tags per project structure

2. Maintenance branches

3. Tagging per release

4. Build number based on SVN revision

๏ Target: allow separate business projects (parties) to run independently on separate SVN roots, whilst allowing the final binary product to be integrated in the main Enterprise Alfresco instance


http://subversion.tigris.org

http://subversion.tigris.org

Target Alfresco Architecture


Task and requirements management

๏ Multiple projects often lose track of relation between requirements and tasks

๏ Jira as a planning tool (or the importance of “Fix Version”)

๏ Maven integration with maven-changes-plugin allows automation of release notes

๏ Remote API allow growing integration degrees

๏ Tip: Map 1-1 your development projects to Jira projects


Enterprise Repository Managerhttp://nexus.sonatype.org/

๏ Nexus enforces (together with Maven) the notion of Artifact as uniquely identifiable product of a specific development process (GAV search)

๏ Unique mirror/proxy for all produced enterprise artifacts and single point of contact with external world

๏ Boosts reuse as generates visibility on project artifacts

๏ Can scale to the enterprise with Professional Edition features (LDAP integration, Procurement, Enterprise class documentation)

๏ Separate repositories for enterprise Releases and Snapshots to ensure additional stability


http://nexus.sonatype.org

http://nexus.sonatype.org

Continuous Integration Serverhttps://hudson.dev.java.net/

๏ Advanced Build server tightly integrated with Maven

๏ Fine tune your build schedule

๏ Periodically on early development stages

๏ On commit close to release and in operations

๏ Build server deploys remotely to Test Jboss and run remote Selenium integration tests, whose reports are published on the site

๏ Alternatively, simple modules are tested directly in Jetty embedded in Hudson Maven build

๏ Tip: Define a CI specific profile in your maven build (-Pci) to override ports/locations and avoid clashes


https://hudson.dev.java.net

https://hudson.dev.java.net

The hot issue with testing Alfrescohttp://seleniumhq.org/๏ Alfresco does not provide a POM. So no transitive

dependencies, no easy test classpath

๏ You don’t want to test Alfresco, only your custom modules

๏ Such a complex architecture needs a bullet proof testing harness to survive

๏ Iterative solution:

1. Provide functional test first: Selenium bridges the gap with end users, empowered to produce automated UAT tests

2. Enable unit/integration testing: Contributed improvement to surefire (http://jira.codehaus.org/browse/SUREFIRE-545)

3. Run both in CI: Remote deployment on Jboss as enabler (http://

jira.codehaus.org/browse/CARGO-416) and Xvfb


http://seleniumhq.org

http://seleniumhq.org

http://jira.codehaus.org/browse/CARGO-416




Project development lifecycle


Top Ten ALM implementation best practices

1. Define an Enterprise SuperPOM to centralize dependencies and environments

2. Introduce the Role of the Librarian

3. Version your Maven projects accordingly to your Jira project versions, using snapshots and release candidates

4. Your tests should be self contained and runnable in CI and the must produce intelligible reports

5. Define your Enterprise Repository as single source of truth for QA and Prod installations

6. Document your project inline and use the Jira/Changes Report to automate release notes

7. maven-release-plugin is as cool as complex so handle it with care. Once that it works though, it way better than any manual release process

8. Use Selenium recorder to have your end users record UAT sessions and use the test to ensure no regressions on new releases

9. When dealing with open source software, prefer contribution over complaint

10.Create a maintenance branch at every major/minor release


ALM in Action


Too much talking...

๏ Let’s see it action!

๏ Maven Eclipse Integration (Dependency Graph)

๏ Maven site (changes:jira-report and surefire-report:report-only)

๏ Technology specific and environmental profiles, typical cmd line: mvn iQ/nstall -Pcas,ldap,test1

๏ Selenium Local and CI

๏ Q/A


In the pipeline

๏ ALM in place in 2009Q1 and since then already a fully fledged Alfresco release with 3 integrated business applications (and growing)

๏ High degrees of integration:

๏ SVN/JIRA integration/enforcement

๏ Development workflows and push notifications

๏ Nexus professional and procurement

๏ Separation of concerns and abstraction of best practices in a ALM-only superPOM


Calm Application Lifecycle Management


How can you abstract best practices?

๏ Maven archetypes and inheritance are powerful to delegate responsibility-specific POMs

๏ Still POMs are verbose and error prone, we want to expose as less as possible to junior developers

๏ Best practices are not technology specific and can be reused independently

๏ ALM is about processes, we need to aggregate Maven plugin configurations in more structured flows


Maturity step 3:CALM๏ The idea:

“Create an abstraction layer to provide standard flows and reduce configuration needed per project”

๏ The objective: Reach a complete lifecycle abstraction which is

๏ Cross-customer

๏ Cross-technology

๏ Extensible and flexible

๏ The solution: A technology independent superPOM gathering all common plugin configurations in use-case oriented Maven profiles


CALM - State of the Art

๏ My first project with a recursive acronym! Yeah!

๏ References:

๏ Source Code/ Community: http://code.google.com/p/maven-calm

๏ First commit: December 2007, 2 active committers, growing in interest

๏ `Base for the Maven Alfresco Archetypes 3.0 release

๏ Still under heavy work in progress but getting there ;)


http://code.google.com/p/maven-calm

http://code.google.com/p/maven-calm

How to use it

๏ CALM should be the SuperPOM of your enterprise POM or of your single project POM

๏ Create your enterprise POM and inherit all standard best practices from CALM superPOM

๏ Your project would be invoked always with a standard command line: Text

mvn -Pflow1[,flow2...flowN[,feature1...featureN][env=local]


CALM Profiles decomposition๏ Environmental profiles

๏ Build time properties from POM and profiles.xml๏ Runtime properties by setting the -Denv variable

๏ Flow/UseCase profiles

๏ Actually instruct maven on which plugins and sequence of them should be executed and which is the default phase

๏ Currently supported: integration-test, selenium-test, deploy, run

๏ Feature profiles

๏ Add features to the application under packaging, technology specific or general purpose

๏ Currently supported for Alfresco:cas, ldap, liferay, mysql


CALM - Perspectives

๏ CALM Target audience:

1. Medium/Large Enterprise targeting to standardize IT processes and procedure, independently from the landscape of technologies in use

2. (Open Source) System Integrator which, cross-customer and cross-technology, aims to provide a standard service in terms of quality of deliverables, reducing TTM and TCO

3. Community enthusiast willing to start off an open source project but is scared by Maven learning curve

WE MAKE IT EASY FOR YOU!


Wrap it all up

๏ Enterprise business has to fight to start embracing the bazaar model to cope with IT modernizations

๏ Give priority to processes rather than technologies...

๏ ...but don’t underestimate how automation can improve your release schedule and your business agility

๏ You don’t need hype extremely expensive software or consultants to manage your IT assets, it’s a matter of values!

๏ Maven is tough but you can work it to make it simple for the vast majority of users (80% - 20% rules)


Thank you!


maven application lifecycle management for alfresco

Technology

open source technologies

ecm world

open source dynamics

open source integrations

myths open source

communities open source

specic open source product

requirements management