Modular Java (Component Oriented Programming with

OSGi)

December 2009

Ilya Rybak

2

Agenda

Modularization and Object-Oriented Modularization in Java Component-Oriented buzz Introduction to OSGi Java 7 ‘wanted’ list

3

Modularization

Recycling old ideas that were raised in the early 70so High Cohesiono Low Coupling

Manage growing complexityo Modularization minimizes complexity by

creating proper boundaries between the parts that compose a system

Support dynamic extensibilityo Properly modularized systems are easier to

maintain and extendo Changes are more localized and affect less of

the overall system

4

Complexity vs. Productivity

5

Object-Oriented

One of the primary goals of object oriented programming (OOP) was, and still is,

Re-Use

It has mostly failed in that goal!

6

What went wrong?

Focus in OO was on encapsulation of instance variables, which is some form of modularization, but granularity is too small

Couplingo Classes can almost never be used in isolation, they depend on other

classeso Those classes depend on other packages, which depend on other

JARs/DLLs…o OO systems become tangled webs quickly

7

A Solution?

Patterns like SOA, Factories, Dependency Injection,Inversion of Control are trying to minimize theconsequences of the OOP’s lack of modularization

Re-use of classes outside their original context is hard, so. . .

Give up and duplicate

Leave them where they are, hide by a façade and call it "SOA"

8

Modularization in Java

9

What’s a Module?

Beyond the simple means of packaging, a true module

should define:

Identity and scope Unit of deployment Dependencies What it provides to other modules

It is up to a module system to see if a module’s requirements can be satisfied and make sure consumers are ‘wired’ to providers

10

JARs

Packagingo Classes encapsulate datao Packages contain classeso Jars contain packages

Visibility Accesso privateo package-private o protectedo public

Jars have no module characteristics

11

Java application

12

“Jar Hell”

13

Class Loading

14

Modularization in Java - Issues

Classes are too fine grained, packages are too simplistic, class loaders are too low level

Jars provide packaging, but…o Have no runtime representation (no identity and scope)o Have no explicit dependencies o Can not be used to restrict access - public class is globally

visible o Are not true deployment units

Severe problems like split packageso Multiple jars have classes with the same name in the same

package No versioning support

o Order on CLASSPATH define chosen versiono Single version of a class in the VM

Has no proper extension/collaboration model

In fact, Java never planned introducing module concept, yet…

15

Assembly

.NET has the built-in module systemo Early attempts were made already in Windows 2000

It defineso security boundaryo type boundaryo reference scope boundaryo version boundaryo deployment unit

Assemblies are the building blocks of .NET applications. An assembly is a collection of types and resources that are built to work together and form a logical unit of functionality

16

What’s a component?

The most common analogy -

Lego?

Lego is a poor analogyo Dead lump of plastic – no life cycleo Not really replaceable - try replacing a block in already

built model o Not really reusable - try using a ‘Duplo’ block in a ‘Lego’

model

Has characteristics closer to that of an object in OO

17

What’s a component?

A better analogy –

Bees!

Active participants in the system Aware of and adapt to their environment May provide services to other components and use services

from other components Have a life cycle

18

Environment

Ecosystem in which a component lives, e.g.:

beehive, Java VM

When the environment is good, the component flourishes.

When the environment is harsh, the component survives.

When very harsh, the component dies.

19

Introduction to OSGi

OSGi™ - The Dynamic Module System for Javao Formerly Open Services Gateway Initiative

Driven by OSGi Alliance - consortium of vendors

o Non-profit corporation founded in 1999

Mission: Build ecosystem to support component based development in Java

Alcatel-Lucent Aplix Corporation Deutsche Telekom Ericsson Mobile Platforms AB Hitachi, Ltd. IBM Corporation LinkedIn Makewave Mitsubishi Electric Corporation NEC Corporation NTT Oracle Corporation Paremus, Ltd. Progress Software

ProSyst Software GmbH Qualcomm Red Hat SAP AG Siemens AG Siemens Enterprise Communications Software AG Sonatype Inc. SpringSource (a Division of VMware) Sun Microsystems, Inc. Telcordia Technologies, Inc. TIBCO Software Inc. Westell Inc.

20

OSGi is about…

Module system for the Java platformo Building systems from smaller componentso Includes visibility, security rules, dependency management and

versioning control

Dynamico Installing, starting, stopping, updating modules, all dynamically at

runtime No downtime or 24 7 are the terms customers like to hear

Service-oriented collaborative modelo POJOs can be registered as services, consumed and managed

inside a VM, again, all dynamically at runtime

21

Work Groups

Multiple groups define the specifications

o Core Platform Expert Group (CPEG) – Core framework

o Mobile Expert Group (MEG) – Mobileo Vehicle Expert Group (VEG) – Automotiveo Enterprise Expert Group (EEG) – Enterpriseo Residential Expert Group (REG) – Home

Automation

No more ‘All in One’ solution – each group has its own purpose and objectives

22

Evolution path

Original use case (Home Automation) faltered around 2000

OSGi used in niche areas (automotive, mobile) for a decade now

Eclipse 3.0 released June 2004 - First Eclipse platform built on OSGi

Gradual adoption and recognition for OSGi after Eclipse

Currently undergoing widespread adoption in the enterprise market

23

OSGi Architecture and Layers

The OSGi Platform specifies a modular architecture for dynamic component based systemso Execution Environmento Module Layero Life Cycle Layero Service Layero Security

Introduces Bundles as modules

L0 - JRE Level•OSGi Minimum Execution Environment•CDC/Foundation•J2SE-1.3 …•J2SE-1.5•JavaSE-6

L1 - Creates the concept of modules(bundles) that use classes fromeach other in a controlled wayaccording to system and bundleconstraints. Deals with packaging, versioning and dependencies

L2 - Manages the life cycle of a bundle• Fine grained events for application and

component lifecycle, e.g.: installing, starting, stopping, updating

• Standard API for registering, listening, and acting on events

• No VM restarts

L3 – Provides a publish/find/bind in-VM service model• Standard methods for registering service instances• Discover and bind to services from other components

Security• Based on Java security model• Example: Vendor A only wants to use the XYZ service provided by Vendor B (could be multiple other implementations of XYZ)• Constraints can be expressed using certificates from signed jars or attributes for bundle, service, etc.

24

Bundle

Bundle is a module in OSGi terminologyo Simply a JAR file plus module metadata stored in jar

manifest• Bundle Identity• Dependencies• Versioning

What does a bundle JAR contain?o Java classes (i.e., standard JAR file content)o Resources (e.g., configuration files, images, etc.)o Native codeo Embedded JAR files

• Bundles can have their own class path

A bundle JAR file can be both a standard JAR file and a bundle at the same time

25

Bundle Manifest

META-INF/manifest.mf

Bundle-ManifestVersion: 2Bundle-SymbolicName: org.apache.servicemix.kernel.filemonitor

Bundle-Version: 2.0.0.0

Bundle-Name: Apache File Monitor

Bundle-Vendor: Apache Foundation

Bundle-RequiredExecutionEnvironment: J2SE-1.5

Bundle-Classpath: ., lib\monitor.jar, lib\test.jar, lib\report.jar

Bundle-Activator: org.apache.servicemix.filemonitor.FileMonitorActivator

Bundle-Description: This bundle monitors the deploy directory for new deployments

Import-Package: org.apache.commons.logging, org.apache.commons.io;version="1.4“, javax.servlet;version=“[2.4,2.5]" , org.osgi.service.packageadminExport-Package: org.apache.jmx.facade,

org.apache.servicemix.kernel.filemonitor; version="1.0.0.0“

26

Bundle Life Cycle

27

Lazy Bundles

Bundles can be activated on demando Defer the creation of the bundle class loader

and activation of the bundle until the bundle is first used

o Can save resources and initialization time on startup

o Triggered by classloading event META-INF/manifest.mf

Bundle-ManifestVersion: 2Bundle-SymbolicName: org.apache.servicemix.kernel.filemonitor

Bundle-Version: 2.0.0.0

Bundle-Activator: org.apache.servicemix.filemonitor.FileMonitorActivator

Bundle-ActivationPolicy: lazy

28

Bundle Activator

Bundle-Activator: Like main() for your bundle Bundles can hook into their life cycle by implementing

a bundle activator interfaceo Notifies them when they are started/stoppedo Gives them access to their bundle context

• Which is how they access framework capabilities, e.g.: Bundle life cycle management Publishing or looking up services

Simple Java interfacepublic interface BundleActivator{ public void start(BundleContext context) throws Exception; public void stop(BundleContext context) throws Exception;}

29

Resolving a Bundle

install bundle

resolve bundle

existing bundle

exported package

package dependency resolution

Resolution of dependencies at deployment If resolution fails, bundle is not available for use No runtime CNFE unless you’re still relying on Class.forName(…) Services and lifecycle methods not called until AFTER resolution

OSGi Framework

30

Multi-version support

Export-Package: foo;version="1.3.0“

Export-Package: foo;version="1.0.0"

Import-Package: foo;version=“[1.3.0, 2.0.0)“

Same package can be installed with different versions Bundles explicitly expose internal packages (i.e., export)o Exporters export precise package versions

Export-Package: bar; version="1.0.0“ Bundles explicitly declare dependencies on external packages (i.e., import)o Importers may specify an open or closed version range

Import-Package:; foo version="[1.0.0,1.5.0)" Allows staged migration of multiple components

foofoo foo

31

Controlling Dependencies

Export-Package: foo;version="1.0.0“;provider=“Pooh”

Export-Package: foo;version="1.0.0"

Import-Package: foo;version=“1.0.0” ;provider=“Pooh”

foofoo foo

Arbitrary export/import attributes for more controlo Exporters may attach arbitrary attributes to their exportso Importers can match against arbitrary attributeso Exporters may declare attributes as mandatory

Mandatory attributes provide simple means to limit package visibilityo Importers influence package selection using arbitrary attribute matching

32

Class space consistency

Exporters may declare package “uses” dependencies Exported packages express dependencies on imported or other exported packages, which constrain the resolve process The framework must ensure that importers do not violate constraints implied by “uses” dependencies

Export-Package: fooExport-Package: foo Import-Package: foo, barImport-Package: foo Export-Package: bar; uses:="foo"

33

Class Filtering

Exporters may declare that certain classes are included/excluded from the exported packageo Ideal solution for building facades around existing jars

Export-Package: foo; exclude:="*Impl", foo; friend="yes“; mandatory:="friend"

foo exclude:=”*Impl”

foo friend=”yes” include:=”*”

Import-Package: fooImport-Package: foo;friend=“yes”

34

Bundle-to-Bundle dependency

Bundle-SymbolicName: AExport-Package: bar, foo

Require-Bundle: AExport-Package: bar

Allows for tight coupling of bundles when required Import everything that another, specific bundle exports

35

Bundle Fragments

Allows bundle content to be extended A special bundle that attaches to a host bundle and uses

the same class loadero Conceptually becomes part of the host bundle, allowing

a logical bundle to be delivered in multiple physical bundles

Fragment-Host: BExport-Package: foo Import-Package: baz

Bundle-SymbolicName: BExport-Package: bar Import-Package: woz

Bundle-SymbolicName: BExport-Package: bar, foo Import-Package: woz, baz

36

Bundles - Summary

META-INF/manifest.mf

…Import-Package: org.apache.commons.logging; resolution:=optional, org.osgi.service.cm; version=“[1.3,1.5.0]" ,

org.osgi.service.packageadminExport-Package: org.apache.jmx.facade; role=“admin“; mandatory:=“role"

org.apache.servicemix.kernel.filemonitor; uses:=“org.apache.commons.logging”; exclude:= “*Impl,*Helper*” Require-Bundle: org.apache.felix.http; bundle-version="[1.0.0.0,2.0.0.0)“

Package level dependencies i.e.: Export/Import-Package:• Can be mandatory(default) or optional

Support for versioning Support arbitrary attributes and filters Control dependency graph with uses attribute Bundle-to-Bundle relationship with Require-Bundle: Fragments as bundles extender model

37

Classloading

The standard Java classloading hierarchy

38

Classloading – Java EE

39

Classloading - OSGi

Tree is simply the wrong shape! What we really need is a graph

40

Class Search Order

Classloading in OSGi is fast compared to the conventional CL tree modelo Classloaders wiring happens at the bundle’s resolution timeo No walking up and down the parent tree when looking up classes

Classloading event is directly delegated to the owning classloader

41

OSGi Module System - Summary

OSGi technology bridges the gap in the area that Java has not addressed

Promotes a better structuring of your application code by defining clear boundaries

Robust classloading model that overcomes the deficiencies of the conventional Java

Control over versioning

Lifecycle

Modularization is rewarded when creating new systems and modifying existing ones

No silver bullet, bad code and bad ideas are still bad

42

What else is missing?

43

Service-Oriented-Architecture for the JVM

OSGi introduces service layer as primary means of collaboration between bundles

Lightweight implementationo No protocols, no stubs - direct Java method callso Cross-VM communication is out of scope (addressed by EE

group) Service Registry is where decoupling between provider

and consumer happens

Registryoffer query

44

Services

Services are POJOso Services are identified with componentso Registered and looked up using their interface name

Follow publish/find/bind modelo Used as bundle’s input/output portso Allow loose coupling between modules – modules can be substituted

Allow lazy instantiation and late binding Meta-data is decoupled from implementationo Service properties can be queried without the need for creating

service instance

45

Services

But most importantly, services are dynamico Static dependencies are handled by the module layer,

runtime dynamics are realized through serviceso Services can be enabled/disabled/substituted dynamically at runtime

OSGi Framework

package dependency (static)

service dependency (dynamic)

However, there is a challengeo Fully dynamic systems require a different way of thinking

Services may come and go at any moment in time• Java programming not for faint hearted

Multiplicity of service providers

46

OSGi Compendium Services

Log ServiceEvent AdminHttp ServiceUser AdminConfiguration AdminDeployment AdminMetatype ServicePreferences ServiceWire Admin

Initial ProvisioningUPnPDeclarative ServicesAuto ConfigurationApplication AdminDMT AdminMonitor AdminDevice AccessIO Connector

OSGi uses services to extend its functionalityo One or more service components ‘bundled’ together

implementing a specific functionality

o Provided separately from the OSGi coreo Many services are defined as mandatory for OSGi framework

providers Application architecture can be defined by a set of deployed

serviceso Create your own profile – VM + OSGi core + required services

47

Bundling existing code

... legacy code is a challenge. Many developers saythings like: “My code is very modular”, or “My codedoesn’t depend on very much”, or “No one uses any ofmy classes except from the Foo package”Unless they are already using osgi, they are wrong.Until modularity is enforced, it is not there. John Wells, BEA

Analyze what you haveo What are the dependencies between the JARs that make up your

application OSGi bundles need manifest headerso Exported/Imported packageso Versions

Good chances are your favorite open source is already ‘OSGfied’o Check with the online repository, e.g.: http://www.springsource.com/repository/app/

48

Bundling existing code

49

Bundling existing code

Start with your existing application and all its dependent jars

Create a bundle activator (i.e.: the existing startup class or main) if needed

Put all libraries into one bundle• Super sized! • Don’t worry, it’ll get better

Having a working bundle (whatever the size) is a good baseline and allows for gradually replacing the dependent jars with bundles

Keep it working!

50

To Include or to Refer?

When to refer to a library?o External APIo Implementations can differ, e.g.: javax.naming, javax.transactiono Very large libraryo Sharing

When to include a library in the bundle?o High Cohesiono Localized library versiono Reduces number of dependencies

51

OSGi Best Practices

Raw OSGi APIs are powerful but create couplingo Isolate the use of the OSGi APIs to a minimal number of

classes Best practices is to write POJOs that are not

coupled to a frameworko There are many such application models for OSGi

OSGi Declarative Services (xml descriptors) Apache iPOJO (annotations) Spring-DM (formerly Spring-OSGi) Service Application Toolkit

Pick one of these application models after you got your existing application running

Convert the application part-by-part

52

OSGi Implementations

Several independently implemented OSGi frameworks exist today, including four that are available as open source software

o Equinox - the most widely deployed OSGi framework today owing to its use in the core runtime of Eclipse. It is also the official RI (Reference Implementation) of the OSGi spec.

o Knopflerfish - popular and mature implementation of both OSGi Release 3 and 4.1. It is developed and maintained by Makewave AB

o Apache Felix - a community implementation of the OSGi Release 4.x.It is designed particularly for compactness and ease of embedding, and is the smallest (JAR size ~360K) implementation

o Concierge - a very compact and highly optimized implementation of OSGi Release 3. This makes it particularly suited to resource-constrained platforms such as mobile phones.

53

Markets & Solutions

Enterpriseo Currently the OSGi framework is adopted by most

enterprise Java vendors for internal use as the underlying infrastructure platform

o Base for the new application server generation architectureo Oracle BEA mSA – micro-service architecture built on OSGi is the base for BEA

productso JBoss MicroContainer - replaces old JMX MicroKernelo IBM WebSphere, Lotus as well as other ~200 products run on Equinoxo JOnAS, SpringSource DM server…

o Smaller vendors, such as SpringSource and Paremus, as well as open source projects such as Apache ServiceMix and Apache Aries, are starting to promote the direct use of the OSGi programming model in developing enterprise applications, and some early adopters such as LinkedIn and VMware have started to do so

Automotive: BMW, Volvo, Bombardier…

Mobile: Nokia, Ericsson, Motorolla…

Smart Home: Bosh, Siemens, Phillips…

E-Health: Siemens Med…

54

A bit of Java politics…

JavaSE 7 (OpenJDK) long awaited featureso G1 garbage collectoro JSR 292: VM support for non-Java languageso JSR 294: Language and VM support for modular programmingo Project ‘Jigsaw’ – modularization of JDK(merged with JSR 294)

Java 7 was expected at the end of 2009 The release date was postponed to Oct. 2010 Modularity and Jigsaw is in the center of continuous debates

55

Jigsaw - Motivation

JDK is BIGo JDK 1.1 = 3.5Mbo JDK 6 = 65.2Mb (>1800% growth)o Startup time is relatively long

• 306 classes to load for a simple “Hello World”

o Difficult to evolve APIs• Deprecation has no meaning

o Depending on whole JDK version is no longer feasible

• Would like to depend on, e.g., Swing v2 and Concurrency v1.x

• Or Swing 1.x but NO higher - now v2 can break compatibility

o Doesn’t scale down(for small devices)o Doesn’t scale up(ME cannot run on SE)

56

Module (the Jigsaw way)

org/company/services/ui/module-info.java

module org.company.services.ui @1.0 {system jigsaw;requires module java-core @7;requires module java-swing @1.7.*;requires optional module commons-lang @ [2.4,3.0);requires private module org.company.services.kernel @1.0;class org.company.services.ui.Main;}

org/company/services/ui/Utils.java

public class Utils {module boolean foo(); //new access level modifier –“module”public boolean bar();}

java –m org.company.services.ui

57

Jigsaw - Summary

No support for package-level dependencies Not dynamic – requires VM restarts on installing,

updating and uninstalling a module No Lifecycle Does not offer collaboration model i.e.: services Questionable as primary development model for

your application, but… A big step forward in terms of affirmation (from Sun)

that modularity is important and currently missing from Java

Probably the shortest path for modularizing JDK as a product

58

Useful Links

OSGi Alliance http://www.osgi.org

Peter Kriens presentation http://www.infoq.com/presentations/osgi-the-foundation

Neil Bartlett’s blog http://njbartlett.name/osgibook.html

http://www.eclipsezone.com/eclipse/forums/t90365.html

Equinox http://www.eclipse.org/equinox/

Apache Felix http://felix.apache.org/site/index.html

SpringSource repository http://www.springsource.com/repository/app/