openjdk penrose presentation (javaone 2012)

Penrose@OpenJDK

Tim Ellison, IBMDavid Bosschaert, Red Hat

September 2012

Agenda

• Level settingo Introduction to modularityo Modules and the JREo The Jigsaw projecto OSGi

• Introduction to Project Penroseo Goal of the projecto Structure of the projecto Technical roadmap

• Technical worko Achievementso Observationso To be done

• Advice and call to arms

What is a module?Why do we need a module system?

• Modules facilitate software engineering & enhanced runtime capabilitieso Decomposition of complex systemso Software assemblies of pre-built moduleso Provisioning / deployment, including module repositorieso Versioning and configuration management

• The module system is used to manage the modules, including installing, activating, resolving module requirements and conflicts.

• Interacting with the module system provides a higher level abstraction than the language provides directly.

Modules in Java

• The module as a unit of encapsulation...

• The Java language already has two kinds of module: classes and packages.o classes encapsulate fields and methodso packages encapsulate classes and resources

• Problem: neither of these is a deployable unit (too granular, too tightly coupled).

• Java's unit of deployment is the JAR file.o No encapsulation, therefore not a module!

• Problem: need an entity that is a deployable unit of encapsulation.

Desirable characteristics of a module

• A module should be highly coherent

• A module should be loosely coupled to other modules.

• A module should be explicit about what it provides to other modules.

• A module should be explicit about what it depends on from other modules.

Why modules for the Java runtime (JRE)?

• Presently the JRE is a monolithic entityo in general the runtime must assume your application will use anything and

everythingo class loaders provide runtime type definition and isolationo download time and start-up time are directly impacted by the number of types

available in the runtime

• Application start-up time includes a linear search through the class path to find system and application codeo e.g. Oracle 1.7 Windows bootclass path contains nearly 20k classeso resources.jar rt.jar jsse.jar jce.jar charsets.jaro rt.jar index alone is ~1Mb

• To the JRE, your application's jars’ indexes are disjoint & unsortedo there are JRE implementation 'tricks' like look aside tables and shared classes

that can helpo class loading is a BIG time hog to amortize over the length of the run

Why modules for the Java runtime (JRE)? - continued

• Dependency managemento avoid “JAR hell” → trying satisfy competing requirements by simple path orderingo type name space comprising (unversioned) package name is often insufficiento e.g. my app depends upon foo-v2.jar and bar-v2.jar but foo-v2.jar depends

upon bar-v1.jar- classpath foo-v2.jar; bar-v2.jar; bar-v1.jar –> my app “wins”- classpath foo-v2.jar; bar-v1.jar; bar-v2.jar –> foo “wins”

• Module level visibilityo public, protected, private and package-private level visibility means some

implementation types (com.sun.) need to be marked public to be called by java. APIs.

o convention asks people not to depend upon them...

• Version controlo ability to define the compatibility of a module based on numbering schemeo different module versions for different target devices / resource goals

Introducing Project Jigsaw

• Project Jigsaw is an effort currently underway at OpenJDKo defining a simple module system for Javao modularizing the JRE itselfo originally intended as an implementation detail for the JRE / applications, but

being adopted as a Java standard (JSR)

• Working within constraints of backwards compatibilityo existing Java APIs “cannot” be changedo Some packages contain wildly different functionality

e.g. java.util contains collection hierarchy and Locale infrastructure

• Introduces language changes to define modulessrc/com/example/myclass.javasrc/com/example/myclassimpl.javasrc/module-info.java

module com.example @ 1.0 {requires jdk.base;requires foo @ 2.0;requires bar @ 2.0;

}

Introducing OSGi

• Existing, mature, standards-based modularity solution for Java applications.

• Rich modularity model based on code-level (package) dependencies, module-level dependencies, and services.

• First practical solution for effective component reuseo well supported by multiple implementations, tooling, etc.

• Millions use applications built with OSGio most application servers are based on OSGi

• No language changes.o defines info in META-INF/MANIFEST.MF

Bundle-ManifestVersion: 2Bundle-SymbolicName: com.ex.mybundleBundle-Version: 1.0.0Import-Package: com.example.barExport-Package: com.example.foo

Multiple dependency resolution

• The module system must resolve bundle dependencies, possibly in the face of multiple valid alternativeso e.g. my app depends upon (foo-v2.jar and (bar-v1.jar or bar-v2.jar)) and foo-

v2.jar depends upon bar-v1.jaro if my app touches bar first, the module system may chose bar-v2.jaro if my app touches foo first, the module system will chose bar-v1.jar

• OSGi's dynamic dependency resolution attempts to find the best fit for currently invoked bundles at runtimeo modules are apparent at runtimeo runtime activation lifecycle: installed, resolved, starting, active, stopping, uninstalled

• Jigsaw resolves module dependencies during build, installation, or runtimeo gives opportunity for load time optimizations such as pre-verify, indexing, etc.o Equinox OSGi implementation has similar optimizations.

Agenda





Project Penrose - the road to reality

• Both Jigsaw and OSGi are here to stay!o Try to get the best of both worldso It need not be a zero-sum game

• OSGi is established and will continue to be well used

• Jigsaw is underway and a key component to the Java SE plans

• Project Penrose's goal is to demonstrate interoperability between Jigsaw and OSGi

OSGi support — It must be demonstrated by prototype to be feasible to modify an OSGi micro-kernel such that OSGi bundles running in that kernel can depend upon Java modules. The kernel must be able to load Java modules directly and resolve them using its own resolver, except for core system modules. Core system modules can only be loaded using the module system’s reification API. Java Module-System Requirements

Project Penrose - structure

• Penrose is structured to allow collaboration between OSGi and Jigsaw developerso OSGi (Equinox) and Jigsaw are governed by different foundations, companies

and licenseso Need to allow separation of developers

• Two Penrose code repositorieso Main code repository is downstream from Jigsaw

§ Provides opportunity to be experimental without disrupting Jigsaw progress

§ Kept in sync with Jigsaw changes, easy to pass code upstream§ “show me” working code, rather than arguments on Jigsaw lists

o Second code repository specifically for Penrose tools, demos, etc.o These are licensed under GPLv2 + classpath exceptiono Code developed in Penrose will be delivered through Jigsaw

Project Penrose - structure

• Two Penrose mailing listso Developer mailing list (penrose-dev)

§ Conversations around coding work on main Penrose repository§ Incudes details of Jigsaw implementation and Penrose's modifications§ This is licensed GPLv2

o Discuss mailing list (penrose-discuss)§ Conversations around interoperability requirements, design and new APIs§ No implementation code is exchanged directly on this list§ This is dual licensed GPLv2 and BSD

• Java Specification Request (JSR)o Looking forward to a JSR in which design discussions can take place

Project Penrose: technical roadmap

• Goal #0 : tolerationo ensure that OSGi frameworks continue to run unmodified on a Jigsaw enabled runtimeo creating modules / bundles that have both Jigsaw & OSGi metadata on the same JAR

• Goal #1 : interoperability of module info metadatao ensure Jigsaw metadata can be enhanced with OSGi conceptso teach OSGi to read Jigsaw module infoo mapping Jigsaw metadata into OSGi format for the framework to understand, e.g.

requires Require-Bundle:⇒o resolve Jigsaw modules using the OSGi resolver

• Goal #2 : OSGi implementation exploit of Jigsaw modularityo enhance OSGi to use Jigsaw publication repositories, API, etc

• Goal #3+ : Full interopo a blend of OSGi and Jigsaw cross delegation on module phases

Agenda





Achievements in Level 0 : Toleration

• Achievement #1: pass the OSGi tests on a Jigsaw enabled runtimeo Equinox 3.7.2 – the OSGi reference implementationo OSGi 4.3 Compliance Tests – with minor patch for test case erroro Running on Windows XP SP3 & Ubuntu 11.04

• Achievement #2: run a Java application as either OSGi or Jigsaw moduleso Took Java 2D demoo Broke it into multiple functional units, and run the demo as OSGi bundles OR

Jigsaw modules

Observations: Jigsaw Module Metadata

• Jigsaw's module-info.java files compile into module-info.class fileso ultimate .jar module contains binary metadata

• Number of issues with these fileso cannot (or very hard) to extend

§ other module systems or tools may need to store additional module-related data

o binary files are not easy to read§ you need to use a tool§ or create your own .class file reader§ may make troubleshooting harder too

• Goal to create an readable, easily parsed metadata formato extensibleo expressive enough for Jigsaw and OSGi

Achievements in Level 1: Module Metadata in Penrose

• Penrose proposes using module-info.json o Plain Text module definitionso In JSON

• JSON offerso Better extensibility, additions can be made inlineo Easier to read from modular JAR files

for humansfor tools

o For performance, implementations can still cache their own binary representations

JSON Module Metadata

module-info.java (Jigsaw)

module org.astro @ 1.2 { exports org.astro;}

In the Jar file:binary module-info.class

module-info.json (Penrose)"module" : { "name" : "org.astro", "version" : "1.2", "exports" : [{ "name" : "org.astro", "org.osgi.version" : "1.2.3" }]}Note the extension on the exports

In the Jar file:plain text module-info.json

Demos

The current set of Penrose demos shows:

• Modules that use JSON metadata (no module-info.java/.class files)

• Modules that work both in JavaSE and in OSGio currently duplicate module informationo unified module information in the future

To be done Level 2/3: Deep integration with OSGi

This pilot needs to start, some areas to look at:

• Loading JavaSE modules in OSGi as bundleso OSGi will most likely be extended to understand JavaSE Module Metadata.

• Define extensions to JavaSE module metadata to cover OSGi informationo Package Versions, Package Imports, etc...

• Wiring to existing JavaSE modules in OSGi package imports

Agenda





Final words of advice

• Stop using internal APIs and implementation classeso You will be broken in a modular worldo Be modest in your API usage and we can likely help with a tailored JRE

• If you need a modularity story that works before Java 9 (expected summer 2015) then the answer is OSGio Penrose will ensure OSGi continues to work beyond Java 9 too

• OSGi has greater expressivenesso Experience of modularity has resulted in a rich set of behaviorso While you may not need it now, do you want to change system when you do need it?

• We would like to see the JSR created right now so that a Java Community Expert Group can get to work on designing the Java Module System.

• Come and participate in OpenJDK Jigsaw or Penrose projects!

Links

Penrose website:http://openjdk.java.net/projects/penrose

Penrose mailing lists:http://mail.openjdk.java.net/mailman/listinfo/penrose-devhttp://mail.openjdk.java.net/mailman/listinfo/penrose-discuss

Penrose forests:http://hg.openjdk.java.net/penrose/jigsawhttp://hg.openjdk.java.net/penrose/penrose

JSON ModuleInfo documentationhttp://hg.openjdk.java.net/penrose/penrose/raw-file/tip/doc/PenroseModuleInfoJSON.html

openjdk penrose presentation (javaone 2012)

Documents

java o

o classes

classes o

osgi o

runtime o modules

mybundle o

unsorted o

openjdk o