legacy renewal of central framework in the enterprise

Anatole Tresch

Renewal of Legacy Frameworks

An Experience Report

GeeCon 2014

May 2014

2Renewal of Central Frameworks – An Experience Report May 2014

Bio

Anatole Tresch Consultant, Coach Credit Suisse Core Framework Engineer,

Technical Architect Specification Lead JSR 354 Driving Java EE Config

Twitter/Google+: @atsticks atsticks@java.net anatole.tresch@credit-suisse.com Java Config Discussion https://groups.google.com/forum/#!forum/java-config Java Config Blog: http://javaeeconfig.blogspot.ch Zurich Java Community (

https://plus.google.com/u/0/communities/106363890913137321158) Zurich Hackergarten (

https://plus.google.com/u/0/103936743237262644768/posts)

3Renewal of Central Frameworks – An Experience Report May 2014

Credit Suisse

Global Bank with Headquarter in Zurich, Clients in Private Banking, Investment Banking and Asset Management.

Shares as Credit Suisse Group AG (CSGN) in Switzerland(SIX) and as American Depositary Shares (CS) in New York (NYSE).

47,400 Employees. Ratings: Moody’s A2, Standard & Poor’s A, Fitch

Ratings A.

4Renewal of Central Frameworks – An Experience Report May 2014

Renewal of Central FrameworksAgenda

Where we started

Objectives and Constraints

Challenges

CS Java Application Platform

Implementation

Summary

5Renewal of Central Frameworks – An Experience Report May 2014

Where we started

6Renewal of Central Frameworks – An Experience Report May 2014

The Credit Suisse ITFrom a Bird’s View

On-, Offshore ca. 20’000+ Employees in IT

Multiple global distributed Operation Centers

Protocol-Mix: Corba, WS, Restful, RMI, …

Technology-Mix: Java, DotNet, PL/1, Oracle, Sybase,

MS SQL Server, DB2, IMS, Smalltalk, Python,

MS Access

In-House and COTS Solutions

Total > 1000 applications

7Renewal of Central Frameworks – An Experience Report May 2014

Java Core FrameworksDeployment

Global Deployment

Part of the Java Application Platform

Infrastructure (JAP)

2-3 Major Releases active in parallel

Delivery: 3 Jars + Scripts

Developer-Desktop: Windows

Production: Solaris/Linux

8Renewal of Central Frameworks – An Experience Report May 2014

Java Core Frameworks

Java Core FrameworksPositioning within the Software and Hardware Stack

Infrastructure-Code!

9Renewal of Central Frameworks – An Experience Report May 2014

1.5 Mio LOC

2 Mio LOC

1 Mio LOC

500k LOC

Java in Credit SuisseA typical Application-Stack

Clients

Entry

Host

Web

Business

CSIB (synchron)MQ Series

(asynchron)

WebWeb

Business

Employees

Web

BusinessBusiness

External Interfaces

COTS

BatchBatch

TA Monitoring

Web

EntryEntry

Web WebWeb

BusinessBusiness

Batch

Web

BusinessBusiness

Business

Core FrameworksColors = versions

10Renewal of Central Frameworks – An Experience Report May 2014

Java Core FrameworksThe Java Core Frameworks’s Functionality

Go for the money - JSR 354 - http://java.net/projects/javamoney 23th October 2013

Context Management (Tracing, Accounting)

Exception Propagation (incl. Trace-Context)

High Performance Logging, Audit Logging

Platform-Abstraction Layer

Configuration Management (Deployment, Application)

JMX Registration und Naming Services

i18n

Common Utilities

11Renewal of Central Frameworks – An Experience Report May 2014

Java Core FrameworksFunctionality of the Java Core Frameworks – a Simulation

Entry

Web

Business Business2 HostExternal

Service Bus

ContextContext

ExceptionContext

Error

Audit

Audit

Audit

Audit

Audit

Log

Log

Log

Log

LogLog

Log

MessageBus

Log

Audit

12Renewal of Central Frameworks – An Experience Report May 2014

Java Core FrameworksFrom Daily Business

Increasing Support Efforts

Non solvable Problems

Complicated, invasive APIs

Missing or complicated SPIs

Implementation- and Runtime Dependencies

Lack of Flexibility and Extendability

Slow, uncomprehensive Code

Missing, outdated or uncomprehensive documentation

…

13Renewal of Central Frameworks – An Experience Report May 2014

Quality IssuesAPI?

public interface Manageable {

/** * Invokes different methods in all classes that implement this * interface. What method actually is performed is determined by the * argument <B>cmd</B>. The Stringarray options contains the * arguments for invoking the method. * @param command * The method that should be invoked. * @param options * The command line arguments for the command. * @throws Throwable * If an error occurs. * @return The return value depends on the method that is invoked. */ public Serializable invoke(String command, String[] options) throws Throwable;}

14Renewal of Central Frameworks – An Experience Report May 2014

Quality IssuesSPI?

/** * Manages the storage of a {@link CoreContextHolder} object. Extending this * class, allows to support different storage behaviours. A CoreContextHolder * Store is used in {@link CoreContext}. Hint: Abstract class * is used to be visible package local only. */abstract class CoreContextHolderStore {

/** Returns the previous stored CoreContextHolder. */ abstract CoreContextHolder getCoreContextHolder(); /** Stores a CoreContextHolder. */ abstract void setCoreContextHolder(CoreContextHolder holder);

/** Factory to get CoreContextHolderStore instance. */ final static class Factory { enum Type { DEFAULT, SINGLETON, THREADLOCAL, WORKCONTEXT }; private final static Factory SINGLETON = new Factory();

/** type of factory. */ private Type instanceType = Type.DEFAULT; /** store created last time */ private CoreContextHolderStore storeCreated = null; static Factory getSingleton() { return SINGLETON; } …

15Renewal of Central Frameworks – An Experience Report May 2014

Quality IssuesComprehensivness/Complexity

public void loadDefaultFromRescource(String[] resources) throws ConfigException{ String environment = System.getProperty(PROPERTY_ENVIRONMENT); environment = ENVIRONMENT_MAP.get(environment); String rescourcePath = null; for (int i = 0; i < resources.length; i++) { rescourcePath = CONFIG_IN_JAR_FOLDERS_START + environment + CONFIG_IN_JAR_FOLDERS_END; try { loadDefaultFromRescource(rescourcePath + resources[i]); } catch (ConfigException e) { if("BASE0130".equals(e.getExceptionContext().getMessageID())) { rescourcePath = CONFIG_IN_JAR_FOLDERS_START + FOLDER_ENV_GLOBAL + CONFIG_IN_JAR_FOLDERS_END; loadDefaultFromRescource(rescourcePath + resources[i]); } } } }

16Renewal of Central Frameworks – An Experience Report May 2014

Quality IssuesOther Metrics

17’000 LOC / 8721 Statements

204 Typen, 1400 Methoden

23 Packet Cycles !

Strong Coupling

Big Artifacts (> 1500 LOC)

Only a Few Interfaces

No Annotations

Missing Information Hiding

…

17Renewal of Central Frameworks – An Experience Report May 2014

Where we startedConclusion

Signific. Issues

Widely Used

RISK!

18Renewal of Central Frameworks – An Experience Report May 2014

Objectives and Constraints

19Renewal of Central Frameworks – An Experience Report May 2014

Objectives and ConstraintsTechnical Objectives

Improved Stability, Maintainability and Extendibility

Improved Performance and Scalability

Improved Usability

Convention over Configuration

Easy Testing (independent tests!)

Reduction of API Footprint (decoupling)

Technical Innovation (IoC)

Maximal Backward Compatibility, smooth Migration Path

20Renewal of Central Frameworks – An Experience Report May 2014

Objectives and ConstraintsCompatibility

Backward Compatibility:

Context Propagation

Serialization (Context, Exceptions)

Build and behavioral Compatibility (API)

Reduced Compatibility Requirements on

Implementation Artifacts

Forward Compatibility:

Context Propagation

Serialization (Context, Exceptions)

21Renewal of Central Frameworks – An Experience Report May 2014

Objectives and ConstraintsOrganizational Objectives

Reduce Support Efforts

Simplify Framework Development

Continuous Build

(Semi-) Automatic Deployment

Building up Know How

22Renewal of Central Frameworks – An Experience Report May 2014

Objectives and ConstraintsChallenges

Quality Culture

Non-Standard Tool-Chain

Stakeholder Management

Time & Budget

Skills

23Renewal of Central Frameworks – An Experience Report May 2014

CS Java Application PlatformThe Environment

24Renewal of Central Frameworks – An Experience Report May 2014

Java Application Platform JAPIdea and Motivation

Applikation LogicGUI, Business Logic, DB Schemas, Configuration, etc.Infrastructure Design/ConfigurationHW, OS, Middleware, NetworkSystems ManagementOperating ManualsDevelopment Tools and ProcessesSecurity Services and ProcessesIntegration Services and Processes

Applikation Platform (AP)Optimized for similar applications,Based on Hosting PlatformsHosting PlatformsProvide Common Services• Computation CHP (Hardware, OS)• Persistence DHP (Databases)

25Renewal of Central Frameworks – An Experience Report May 2014

JAP Lifecycle ManagementWhy Lifecycle Management?

• Applications and Components without managed Lifecycle

• Uncoordinated technical Upgrades

• Incompatibilities

• Managed Lifecycle• Coordinated and tested

technical Upgrades (HW+SW!)

• Constant Decommissioning of old Platform Releases

• New Releases wont impact existing applications

26Renewal of Central Frameworks – An Experience Report May 2014

JAP Lifecycle ManagementRelease Planning

27Renewal of Central Frameworks – An Experience Report May 2014

Implementation

28Renewal of Central Frameworks – An Experience Report May 2014

ImplementationRelease Planning

Major Releases synchronous with JAP Platform

(periodically all 2-3 Years)

Patch Releases 2-3/Year

Emergency Releases as necessary

1 Major Framework Release =

Proposals, Requirements, Feature Selection beforehand

6 months Implementation and Component Tests

6 months Integration Testing and Documentation

29Renewal of Central Frameworks – An Experience Report May 2014

ImplementationSpecification/Design

High Level Design (Make or “Buy”)

Existing Functionality in Java EE 5/6, Weblogic?

Third Party / OSS Libraries? Functional Gap?

Low Level Design Minimize APIs, go for Effective Java Immutable Value Objects Thread-Save with Low Latency Multi-Tenancy, Multi-Language, Multi-Timezone etc. Decide on Adaption of existing APIs vs. Recreation and

Coexistence Consider Operational Aspects

30Renewal of Central Frameworks – An Experience Report May 2014

ImplementationTeam and Organization

Agile Process (no Offshoring)

2 Full-Time Programmers

Pair Programming as Needed

Hire the Best you get!

31Renewal of Central Frameworks – An Experience Report May 2014

ImplementationFurther Details

Extend and Improve Existing Tests

Improve/write Component Tests instead of Smoke Tests

Create Compatibility and Interop Tests

Ensure Independent Test Execution (Test Isolation)

Ensure Integration Tests (across multiple releases!)

Integrate early!

API

Deprecate old APIs (for at least(!) one Release)

Check Backward Compatibility with Sonar/clirr

continously

32Renewal of Central Frameworks – An Experience Report May 2014

ImplementationQuality Management

Static Code Analysis: Continuous and integrated, on each code change,

that compiled Based on Sonar, with customized Profile and Custom

Rules Coverage Analysis of Component Tests Basic Integrationstests Regular Check of Metrics, Alarms!

Peer-Reviews of all Code Changes Integration Tests with big Pilots

33Renewal of Central Frameworks – An Experience Report May 2014

ImplementationFramework Release Cycles

Similar to Platform Releases Playground 1 (first drop) Playground 2 (functional) Alpha (API stable) Beta (only patches) Release Candidate Final Release Patch Releases (regular updated) Emergency Releases (as needed)

34Renewal of Central Frameworks – An Experience Report May 2014

ImplementationFist Run 2010 - 5.0.0 -> 6.0.0

Partial API Renewal and API Reduction

Defined SPIs

Implementation Modularization and Encapsulation

(Spring/Lenz)

Focus on

Critical Functionalities

Stabilize Existing Code Base

Focus on Stability, reduce Complexity

3 Jars + Scripts-> Only Partial Renewal

-> significant Compatibility Breaks

-> Massive Performance Gains

35Renewal of Central Frameworks – An Experience Report May 2014

ImplementationSecond Run 2012 - 6.0.0 -> 7.0.0

HW/OS Change from Solaris to Linux Several small Improvements Fixing of Known Issues (sometimes with behavioral

changes!) Low Latency Requirements Introduction of a Runtime Platform Abstraction Layer Replacement of Spring with a leaner Container Introduction of Continuous Build based on Maven

7 Jars + Scripts QC with Sonar -> Renewal still ongoing

-> only a few compatibility breaks

-> New, efficient tool chain

36Renewal of Central Frameworks – An Experience Report May 2014

ImplementationThird Run 2013/2014 - v7.0.0 -> 8.0.0 (today)

Introduction of Java EE 6 (formerly EE 5)

Complex Integration of Low Level Features / CDI

New Annotation based API

Long Lasting Deprecations (New IOC Paradigma)

Renewal of Further Areas Untouched so far

Full CDI-Support -> Renewal Finished

-> only few Compatibility Issues

-> reduced API Footprint

(Annotations)

-> modern API (CDI)

37Renewal of Central Frameworks – An Experience Report May 2014

Conclusion

5.0 6.0 7.0 8.0

……..?

38Renewal of Central Frameworks – An Experience Report May 2014

ConclusionKey Factors

Functional and Effective Tooling

Continuous Build with Integrated Quality Measurement

Agil and Effective Collaboration Culture

Focus on task: minimize Maintenance for former Releases

Ensure enough Budget and Time

Define a realistic Scope

Managed Environment (Lifecycle and Platform

Management)

Use complex Test-Pilots early!

39Renewal of Central Frameworks – An Experience Report May 2014

ConclusionDid we met our Objectives?

Significant Reduction of Support Efforts

Establishment of Maintainability and

Extendibility

Improved Runtime Behavior (Performance,

Memory)

Reduction (Optimization) of API Footprint

Simplify Testing

Enable Technical Progress

Improved Quality

Backward Compatibility, if possible

Yes, but moreartifacts & LOC!

-50%

Yes.

100 x faster

Yes.

Yes EE6/7supported

-50%

mostly

SUCCESS!

40Renewal of Central Frameworks – An Experience Report May 2014

ConclusionActual Metrics

41Renewal of Central Frameworks – An Experience Report May 2014

ConclusionAPI Evolution

Configuration appCfg = ConfigHelper.getConfigManager() .getApplicationConfig();Map<String,String> config = null;if(appCfg.containsArea("a.b.c")){ config = appCfg.getArea("a.b.c", false);}

@Configured@ConfigRoot(value="a.b.c", recursive=false)@Optionalprivate Map<String,String> config;

OLD:

NEW:

42Renewal of Central Frameworks – An Experience Report May 2014

ConclusionAPI Evolution 2

private static final TraceLogger tl = TraceLoggerHelper.getTracLogger( getClass(), TraceType.SERVICE);

public void bookSettlement(Settlement s){ tl.entering(getClass(), "bookSettlement", s); // perform… tl.exiting(getClass(), "bookSettlement", s);}

OLD:

NEW:@TraceLogged(TraceType.SERVICE)public void bookSettlement(Settlement s){ // perform…}

43Renewal of Central Frameworks – An Experience Report May 2014

Q & A

???

44Renewal of Central Frameworks – An Experience Report May 2014

The End

THANK YOU!