spring roo 1.0.0 technical deep dive

Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited.

Spring Roo 1.0.0:Technical Deep Dive

Roo Internals and Add-On Development

Ben Alex, Project Lead, Spring Roo

Copyright 2009 SpringSource. Copying, publishing or distributing without express written permission is prohibited. 2

Agenda

• Architectural Journey• Implementation• Development Mode• Building Add-Ons• Conclusion


Agenda



Design Goals

• High productivity for Java developers– Reuse their existing knowledge, skills and experience

• Eliminate barriers to adoption– Lock-in, runtime, size, development experience

• Embrace the strengths of Java– Dev-time: tooling, popularity, API quality, static typing– Deploy-time: performance, memory use, footprint


High-Impact Decisions

• Use Java– This is a tool for Java developers– It defeats the purpose if we insist on a new language

• Don't create a runtime– This avoids bugs, lock-in, adoption barriers, approvals– This avoids CPU cost, RAM cost and footprint cost

• This easily led to some sort of generation model


Prototyped Approaches

• Pluggable Annotation Processing API (JSR 269)• Generate source at build time (XDoclet-style)• Generate bytecode at build time (ASM)• Generate bytecode at runtime (ASM)• Advanced proxy-based approaches (Spring AOP)• IDE plugins

• None of them satisfied all of the design goals


Decision 1: Use AspectJ

• AspectJ ITDs for “active” generation– Active generation automatically maintains output

• Delivers compilation unit separation of concern– Easier for users, and easier for us as developers

• Instant IDE support– Reduce time to market and adoption barriers

• Other good reasons– Mature, “push in” refactor, compile-time is welcome


ITD Model

• Roo owns *_Roo_*.aj files– Will delete them if necessary

• Every ITD provider registers a “suffix” it uses– Eg “Entity” becomes *_Roo_Entity.aj– A missing ITD provider causes AJ file removal

• ITDs have proper import management– So they look and feel normal to developers– So they “push in refactor” in a natural form


Decision 2: Create A Shell

• Need something to host the “active” generation

• Need something to receive user commands

• Usability issues were of extremely high priority

• We chose an interactive shell– Highest usability (tab, context awareness, hiding, hints...)– Background monitoring of externally-made changes– Background monitoring avoided crude “generate” steps


Shell Model

• Shell will perform a “startup” scan– To handle changes made while it wasn't running

• Shell will monitor file system once started– To handle changes made while it is running

• Shell will have a set of commands registered– To handle explicit directives from the user

• Roo will never modify a *.java file except at the explicit request of a user via a shell command


That Left One Key Question...


Control Information

AspectJ ITDs

Person_Roo_ToString.aj

toString():String

Person.java

name:String

write

Person_Roo_JavaBean.aj

getName():StringsetName(String):void

readRooAdd-OnsRoo

Add-OnsRooAdd-OnsRoo

Add-Ons

Person.class

name:StringtoString():String

getName():StringsetName(String):void

AspectJCompiler

read read

writeHow will add-ons knowwhat to generate?


Metadata

• Project details– Project name, dependencies, build system...

• Java type details– Fields, methods, constructors, inheritance...

• Higher-level concepts– Entity properties, controller paths, validation rules...

• Must be automatically determined from project– With @Roo* (source level retention) where infeasible


Metadata Model

• Immutable• String-based keys (start with “MID:”)• Built on demand only (never persisted)

• Metadata can depend on other metadata– If “upsteam” metadata changes, “downstream” is told– Some metadata will want to monitor the file system

• Central metadata service and cache required


Demo #1

Shell Operation and ITD Maintenance


Agenda



Conventions We Follow

• Ensure usability is first-class• Minimize the JAR footprint that Roo requires• Dependencies only from Enterprise Bundle Repo• Relocate runtime needs to sister Spring projects• Embrace immutability as much as possible• Maximize performance in generated code• Minimize memory consumption in generated code• Use long artifact IDs to facilitate identification• Don't put into @Roo* what you could calculate• Don't violate generator predictability conventions


The Roo Distribution

• “Roo” is split into two logical components

• Base add-ons– org.springframework.roo.addon.*– Provide major out-of-the-box Roo features– Provide metadata for other add-ons

• Roo Core– org.springframework.roo.* (excluding .addon.*)– Provide services add-ons will almost always require


1.0.0 Base Add-Ons

• Backup• Bean Info• Configurable• Data On Demand• Email• Entity• Dynamic Finder• Java Bean• JMS• JPA

• Logging• Maven• Pluralization• Property Editor• Property File• Security• Integration Test• ToString• Web (various)


Role of Third Party Add-Ons

• 3rd party add-ons have a long-term role in Roo

• Minimize download size of Roo distribution• Avoid licensing issues with Roo distribution• Facilitate features of more niche interest• Separate project management and maintenance• Enable end user customization• Provide a sandbox for easy experimentation


Core Components

Project

Support(used by all)

Metadata File MonitorModel

Classpath

File Undo Shell

Process Manager


File Monitor API

• Mostly used by project and classpath modules– Rare for an add-on to directly use file monitor

• Publishes startup and “on change” events

• Events– Create– Update– Delete– Rename (if implementation supported)– Monitoring start (for each file when first found)


File Monitor Implementation

• Polling-based implementation

• Auto-scales to reduce CPU overhead

• Shell commands– “poll status” indicates scan frequency and duration→

– “poll speed” allows specification of frequency→

– “poll now” forces an immediate poll→


Metadata Identifiers (MIDs)

• Immutable strings to avoid GC cost

• Note the “instance” portion denotes a Java type– Most MIDs include this physical location and type name– Use PhysicalTypeIdentifier.getPath(..) and

getJavaType(..) to directly extract typesafe objects

MID:o.s.r.classpath.PhysicalTypeId#SRC_MAIN_JAVA?com.foo.Bar

Static MetadataIdentificationUtils.getMdClass(..) MetadataIdentificationUtils.getMdInstance(..)


Add-On

Metadata Facilities

MetadataItem

stores

queries

MetadataProvider

registers + notifies

• MetadataService can provide all metadata in the system

• MetadataService auto-evicts all downstream dependencies

• Dependency registration can be MID class or MID instance specific

MetadataDependency

Registry

MetadataCache

MetadataService


Project Abstraction

• Abstracts the user's project build system– Immutable metadata (including dependency details)– Path (eg SRC_MAIN_JAVA) mapping to directories– Operations object to modify project structures– Access to the PathResolver instance

• Maven implementation via add-on– Designed to also support Ivy in due course (ROO-91)

• Add-ons will regularly use ProjectOperations


Process Manager

• Ensures non-overlapping synchronization– Background file monitor polling thread– Shell command foreground thread

• Facilitates “atomic” operations with rollback– Notifies disk changes until disk is in stable state– Rolls back disk changes if from a failed shell command

• Provides a FileManager abstraction– Add-ons must use this abstraction for disk changes– Reads via FileManager are optional (but recommended)


Classpath API

PhysicalType

Identifier

MID:o.s.r.classpath.PhysicalTypeId#SRC_MAIN_JAVA?com.foo.Bar

PhysicalType

Metadata

PhysicalType

Category

PhysicalType

Details

ClassOrInterfaceTypeDetails ItdTypeDetails


Classpath Implementation

• JavaParser– Provides a javacc-based AST, but no binding API– Used due to licensing considerations of alternatives

• Roo offers binding, including parameterization

• JavaParser does require source code– Avoids necessity of having a compiler running– Abstractions designed to provide ASM-like *.class

introspection should it be desired in the future


Shell

• Implementations for STS and JLine

• Add-ons should– Make a “Command” class and implement CommandMaker– Delegate methods through to an “Operations” object– Annotate methods with @CliCommand– Annotate method parameters with @CliOption– Optionally use @CliAvailabilityIndicator if desired– Throw any exceptions to abort and rollback changes– Use JDK logging or return objects for console output


Building From Source

• We develop against a public Subversion repo– You can anonymously checkout the code

• There's also a public FishEye instance (plus Jira)

• Roo itself uses Maven, so it's very easy to build– Standard package, install, assembly and site goal


Mainline Development

• Checkout Roo anonymously

svn co https://anonsvn.springframework.org/svn/spring-roo/

cd spring-roo/trunk

less readme.txt

• readme.txt contains latest setup details– Add ROO_CLASSPATH_FILE variable– Create symbolic link to bootstrap/roo-dev

• It works under Windows as well, but not as well

https://anonsvn.springframework.org/svn/spring-roo/


Classpath File Maintenance

• Classpath file allows rapid execution of the shell without loading Maven to build a full classpath

• Refreshing the ROO_CLASSPATH_FILE is easy

cd /home/spring-roo/trunk

mvn clean eclipse:clean eclipse:eclipse compile

• Use the “props” shell command to verify classpath– Use the above command to update the classpath file


Development Tips

• Load Roo using “roo-dev” (not “roo”)– Uses incremental builds as per ROO_CLASSPATH_FILE

• Note roo-dev add-ons will be under ~/roo-dev

• roo-dev always starts up in “development mode”

• Debugging is supported via Eclipse/STS– Refer to readme.txt for details


Demo #2

Setting Up A Fresh Checkout


Agenda



Development Mode

• Enable via– Loading Roo via roo-dev shell script– JVM property developmentMode = true– Typing “development mode” at the roo> shell prompt

• Outputs full exception traces within Roo– Ugly for normal usage (exceptions are the normal way

that an add-ons abort from unexpected conditions)

• First line of troubleshooting add-on bugs is to ask for “development mode” exception data


Metadata For Type


Metadata For Id


Metadata Trace

• Use --level 1 for instance-specific notifications• Use --level 2 for absolutely all notifications

• Consider the number of notifications involved in adding a field to an entity that also has a JSP– File system change physical type metadata entity → →

metadata finder metadata bean info metadata → → →controller metadata jsp metadata→

– This doesn't even consider parallel and unchanged paths (eg integration test, getter/setters, menus etc)

– Metadata beyond type introspection is really useful!


Demo #3

Development Mode Demonstration


Agenda



Roo's Add-On Model

• Roo 1.0.0 offers third-party add-on support

• Very simple single classloader model

• Add-on ZIPs placed in $ROO_HOME/add-ons• JARs auto-extracted into $ROO_HOME/work

• Launch scripts add $ROO_HOME/work to classpath


User Add-On Installation

• “addon install --url file:/some.ZIP”• “addon install --url http://www.x.com/s.ZIP”• “addon uninstall --pattern some*.zip”

• Install/uninstall always does a “clean” afterwards

• “addon clean”– Manages $ROO_HOME/work as appropriate– Will request to restart shell if any changes are made– Deletions are deferred until JVM shuts down

http://www.x.com/s.ZIP


Add-On ZIP Structure

• ZIP can be created whatever way you like– Although Roo offers a Maven assembly.xml

• Directories required– “dist” contains 1+ JARs produced by an add-on– “lib” contains 0+ JARs required by the add-on– “lib” contains all transitively-necessary dependencies– Roo does not perform transitive dependency resolution

• By convention there's a readme and “legal” dir


Add-On Naming Guidelines

• Artifact IDs must be– Fully qualified (eg com.foo.bar)– Start with .com or .net or .org (due to class scanning)– End with .roo.addon (due to class scanning)

• Results in useful ZIP and JAR names like– com.my.proj.roo.addon-1.0.0.zip– com.my.proj.roo.addon-1.0.0.jar

• Roo add-on commands enforce the above– Plus offer suggestions, so don't worry too much


Add-On Creation

• It's seriously this easy:

roo> project --topLevelPackage com.my.proj.roo.addon --template ROO_ADDON_SIMPLE

roo> perform eclipse

roo> perform assembly

optional, or use mvn directly

or use Maven directly

or create a project by hand


ROO_ADDON_SIMPLE

• Depends on currently-running Roo version– Provides access to full Roo core and all base add-ons

• Creates Java types for you– These are usable types that show key Roo APIs

• Outputs a valid Maven POM and assembly.xml

• Also gives you shell scripts for classpath control


Add-On

Commands and Operations

Operations

Static FieldOptions

Operations

Static FieldOptions

ShellConverter

Command

ShellAPI

Command

Roo APIs

rego

rego


Development-Time Notes

• roo-dev uses ~/roo-dev as $ROO_HOME• roo-dev uses $ROO_ADDON_CLASSPATH_FILE

• This allows you to incrementally develop in Eclipse and not have to make a ZIP every time

• Set $ROO_ADDON_CLASSPATH_FILE via– *nix: “source /your/add-on/setup.env”– Windows: “\your\add-on\setup.bat”– Files automatically updated during “mvn compile”


Add-On Distribution

• Suggestions– Host ZIPs on your own server or Google Code SVN– Commiting to SVN simplifies HTTP distribution

• Announce via– A Tweet including #roo– A forum post at http://forum.springsource.org/– Email [email protected] with details

• I'll add it to the official list of public Roo add-ons

• A future release will revisit distribution facilities

http://search.twitter.com/search?q=#roo

http://forum.springsource.org/forumdisplay.php?f=67

mailto:[email protected]


Suggested Add-On Steps

• Learn Roo conventions from a user's perspective

• Use “project --template ROO_ADDON_SIMPLE”– Then “addon install” and try the “welcome” commands– Import it into Eclipse, study the code and customize

• Use Roo SVN to study more advanced add-ons

• Obtain help via the Spring community forums


Demo #4

Building An Add-On


Agenda



Roo's Design

• Result of considerable R&D

• Very lightweight and loosely-coupled core

• Easy to build from SVN and round-trip develop

• “Development mode” eases troubleshooting


A Growing Ecosystem

Roo Distribution

Roo Core Base Add-Ons

Free-TextSearch Reporting Scheduling

OtherSpring

projects

DTOs/Mappers

DatabaseDeployment Captcha

HundredsMore...


Thanks

• Hope you've enjoyed our “deep dive” into Roo!

• Roo community resources– Home → http://www.springsource.org/roo– Support → http://forum.springframework.org– Issues → http://jira.springframework.org/browse/ROO– Tweets → #roo hash key

• Questions?

[email protected]

http://www.springsource.org/roo

http://forum.springsource.org/forumdisplay.php?f=67

http://jira.springframework.org/browse/ROO

http://search.twitter.com/search?q=#roo

mailto:[email protected]

spring roo 1.0.0 technical deep dive

Technology

express written permission

spring roo copyright

user roo

itd model roo

footprint copyright

shell model shell

steps copyright

infeasible copyright