implementing groovy domain-specific languages - s2g forum - munich 2010
DESCRIPTION
TRANSCRIPT
© 2010 SpringSource, A division of VMware. All rights reserved© 2010 SpringSource, A division of VMware. All rights reserved
Design Your Own Domain-Specific Languagewith Groovy!
Guillaume Laforge
Groovy Project Manager
lundi 22 mars 2010
Guillaume Laforge
• Groovy Project Manager
• JSR-241 Spec Lead
• Head of Groovy Developmentat SpringSource / VMWare
• Initiator of the Grails framework
• Co-author of Groovy in Action
• Speaker: JavaOne, QCon, JavaZone, Sun TechDays, Devoxx, The Spring Experience, SpringOne, JAX, Dynamic Language World, IJTC, and more...
2
lundi 22 mars 2010
!Groovy is a dynamic language for the JVM
• with a Meta Object Protocol
• compiles directly to bytecode
• provides seamless Java interoperability
!Groovy was created in 2003, is hosted at Codehaus, and is under the Apache license
!Relaxed grammar deriving from Java 5
• annotations, generics, static imports, enums, varargs...
• borrowed good ideas from Ruby, Python, Smalltalk
• flat learning curve for Java developers
3
Groovy in a few words
lundi 22 mars 2010
The Context
lundi 22 mars 2010
Subject Matter Expers, Business Analysts...
lundi 22 mars 2010
HAICAN HAS STDIO?I HAS A VARIM IN YR LOOP UP VAR!!1 VISIBLE VAR IZ VAR BIGGER THAN 10? KTHXBYEIM OUTTA YR LOOPKTHXBYE
Developer producing LOLCODE
lundi 22 mars 2010
Lots of Languages...
lundi 22 mars 2010
And in the end... nobody understands
each other
lundi 22 mars 2010
Expressing Requirements...
9
lundi 22 mars 2010
DSL: a potential solution?
!Use a more expressive language than a general purpose one
!Share a common metaphore of understanding between developers and subject matter experts
!Have domain experts help with the design of the business logic of an application
!Avoid cluttering business code with too much boilerplate technical code
!Cleanly separate business logic from application code
!Let business rules have their own lifecycle
10
lundi 22 mars 2010
Towards more readibility (1)
11
lundi 22 mars 2010
Towards more readibility (1)
11
lundi 22 mars 2010
Towards more readibility (1)
20%
11
lundi 22 mars 2010
Towards more readibility (2)
12
lundi 22 mars 2010
Towards more readibility (2)
12
lundi 22 mars 2010
Towards more readibility (2)
80%
12
lundi 22 mars 2010
A collection of DSLs
! In our everyday life, we’re surrounded by DSLs
• Technical dialects
• Notations
• Business languages
13
lundi 22 mars 2010
Technical dialects
14
lundi 22 mars 2010
SQL
lundi 22 mars 2010
^[\w-\.]+@([\w-]){2,4}$
16
lundi 22 mars 2010
Notations
17
lundi 22 mars 2010
1. e4 e5
2. Nf3 Nc6 3. Bb5 a6
lundi 22 mars 2010
L2 U F-1 B L2 F B -1 U L2
lundi 22 mars 2010
Visual!
lundi 22 mars 2010
Real-life Groovy examples
!Anti-malaria drug resistance simulation
!Human Resources employee skills representation
! Insurance policies risk calculation engine
!Loan acceptance rules engine for a financial platform
!Mathematica-like lingua for nuclear safety simulations
!Market data feeds evolution scenarios
!and more...
21
lundi 22 mars 2010
Three levels of techniques
22
Flexible & malleable syntax
• scripts
• optional typing
• native syntax constructs
• parens / semi ommission
• named arguments
• BigDecimal
• operator overloading
• closures
Meta-programming
• POGO
• categories
• builders
• custom MetaClass
• ExpandoMetaClass
AST transformations
• AST traversal
• local transformations
• global transformations
• hooks into Antlr
lundi 22 mars 2010
A flexible & malleable syntax
!No need to write full-blown classes, use scripts
!Optional typing (def)
• in scripts, you can even omit the def keyword
!Native syntax constructs
!Parentheses & semi-colons are optional
!Named arguments
!BigDecimal by default for decimal numbers
!Closures for custom control structures
!Operator overloading
23
lundi 22 mars 2010
!Hide all the boilerplate technical code
• an end-user doesn’t need to know about classes
Scripts vs classes
24
public class Rule {
public static void main(String[] args) {
System.out.println(“Hello”);
}}
println “Hello”
lundi 22 mars 2010
Optional typing
25
!No need to bother with types or even generics
• unless you want to!
! Imagine an interest rate lookup table method returning some generified type:
!No need to repeat the horrible generics type info!
Rate<LoanType, Duration, BigDecimal>[]
lookupTable() { ... }
def table = lookupTable()
lundi 22 mars 2010
Native syntax constructs
26
!Lists
!Maps
!Ranges
• You can create your own custom ranges
[Monday, Tuesday, Wednesday]
[CA: ‘California’, TX: ‘Texas’]
def bizDays = Monday..Friday
def allowedAge = 18..65
lundi 22 mars 2010
!Make statements and expressions look more like natural languages
Optional parens & semis
27
move(left);
move left
lundi 22 mars 2010
! In Groovy you can mix named and unnamed arguments for method parameters
• named params are actually put in a map parameter
• plus optional parens & semis
!Corresponds to a method signature like:
Named arguments
28
take 1.pill,
of: Chloroquinine,
after: 6.hours
def take(Map m, MedicineQuantity mq)
lundi 22 mars 2010
!Main reason why financial institutions often decide to use Groovy for their business rules!
• Although these days rounding issues are overrated!
!Java vs Groovy for a simple interpolation equation
BigDecimal by default
29
BigDecimal uMinusv = c.subtract(a);
BigDecimal vMinusl = b.subtract(c);
BigDecimal uMinusl = a.subtract(b);
return e.multiply(uMinusv)
.add(d.multiply(vMinusl))
.divide(uMinusl, 10, BigDecimal.ROUND_HALF_UP);
(d * (b - c) + e * (c - a)) / (a - b)
lundi 22 mars 2010
!When closures are last, they can be put “out” of the parentheses surrounding parameters
!Signature
Custom control structures,thanks to closures
30
unless (account.balance > 100.euros,
{ account.debit 100.euros })
unless (account.balance > 100.euros) {
account.debit 100.euros
}
def unless(boolean b, Closure c)
lundi 22 mars 2010
Operator overloading
•Currency amounts
•Distance handling
•Workflow, concurrency
•Credit an account
a + b a.plus(b)
a - b a.minus(b)
a * b a.multiply(b)
a / b a.divide(b)
a % b a.modulo(b)
a ** b a.power(b)
a | b a.or(b)
a & b a.and(b)
a ^ b a.xor(b)
a[b] a.getAt(b)
a << b a.leftShift(b)
a >> b a.rightShift(b)
+a a.unaryPlus()
-a a.unaryMinus()
~a a.bitwiseNegate()
31
15.euros + 10.dollars
10.kilometers - 10.meters
taskA | taskB & taskC
account << 10.dollars
account += 10.dollars
account.credit 10.dollars
lundi 22 mars 2010
The MOP (Meta-Object Protocol)
lundi 22 mars 2010
Groovy’s MOP
!All the accesses to methods, properties, constructors, operators, etc. can be intercepted thanks to the MOP
!While Java’s behavior is hard-wired at compile-time in the class
!Groovy’s runtime behavior is adaptable at runtime through the metaclass
!Different hooks for changing the runtime behavior
• GroovyObject, custom MetaClass implementation, categories, ExpandoMetaClass
33
lundi 22 mars 2010
!All instances of classes created in Groovy implement the GroovyObject interface:
!A GO can have “pretended” methods and properties
GroovyObject
34
get/setProperty(String name)
invokeMethod(String name, Object[] params)
propertyMissing(String name)
methodMissing(String name, Object[] params)
get/setMetaClass(MetaClass mc)
lundi 22 mars 2010
MetaClass
!The core of Groovy’s MOP system
!MetaClasses can change the behavior of existing third-party classes — even from the JDK
35
invokeConstructor()
invokeMethod() and invokeStaticMethod()
invokeMissingMethod()
getProperty() and setProperty()
getAttribute() and setAttribute()
respondsTo() and hasProperty()
lundi 22 mars 2010
!A DSL for MetaClasses!
!To avoid repetition of Type.metaClass, you can pass a closure to metaClass { ... }
!The delegate variable in closure represents the current instance, and it the default parameter
ExpandoMetaClass
36
MoneyAmount.metaClass.constructor = { ... }
Number.metaClass.getDollars = { ... }
Distance.metaClass.toMeters = { ... }
Distance.metaClass.static.create = { ... }
lundi 22 mars 2010
The Builder pattern
lundi 22 mars 2010
A builder for HR
38
softskills {
ideas { capture 2
formulate 3
} ...}knowhow { languages {
java 4
groovy 5
} ...}
lundi 22 mars 2010
A builder for HR
38
softskills {
ideas { capture 2
formulate 3
} ...}knowhow { languages {
java 4
groovy 5
} ...}
lundi 22 mars 2010
Builders
!Builders are...
• a mechanism for creating any tree-structered graph
• the realization of the GoF builder pattern at the syntax level in Groovy
• simply a clever use of chained method invocation, closures, parentheses omission, and use of the GroovyObject methods
!Existing builders
• XML, Object graph, Swing, Ant, JMX, and more...
39
lundi 22 mars 2010
The clever trick
!GroovyObject#invokeMethod() is used to catch all non-existing method calls in the context of the builder
!The nesting of closures visually shows the level of nesting / depth in the tree
! builder.m1(attr1:1, attr2:2, { builder.m2(..., {...}) } becomes equivalent to builder.m1(attr1:1, attr2:2) { m2(...) {...} }
thanks to parens omission
40
lundi 22 mars 2010
Adding properties to numbers
!Three possible approaches
• create a Category
• a category is a kind of decorator for default MCs
• create a custom MetaClass
• a full-blown MC class to implement and to set on the POGO instance
• use ExpandoMetaClass
• friendlier DSL approach but with a catch
41
lundi 22 mars 2010
With a Category
42
class DistanceCategory {
static Distance getMeters(Integer self) {
new Distance(self, Unit.METERS)
}}
use(DistanceCategory) {
100.meters
}
! Interesting scope: thread-bound & lexical
!Have to surround with “use”
• but there are ways to hide it
lundi 22 mars 2010
!Works for the class hierarchy for POJOs, and a flag exists to make it work for POGOs too
!But the catch is it’s really a global change, so beware EMC enhancements collisions
With an ExpandoMetaClass
43
Number.metaClass.getMeters = {->
new Distance(delegate, Unit.METERS)
}
100.meters
lundi 22 mars 2010
!Groovy 1.6 introduced AST Transformations
!Compile-time == No runtime performance penalty!
44
Transformation
Compile-time Metaprogramming
lundi 22 mars 2010
Compile-time metaprogramming
!With metaprogramming, Groovy’s able to modify the behaviour of programs... at runtime
!Groovy 1.6 introduced AST Transformations
• AST: Abstract Syntax Tree
• Ability to change what’s being compiled at compile-time!
• No runtime impact!
• Lets you change the semantics of your programs!
• Nice way of implementing patterns and removing boiler-plate technical code
!Two kinds of transformations: global and local
45
lundi 22 mars 2010
AST Transformations
!Two kinds of transformations
• Global transformations
• applicable to all compilation units
• Local transformations
• applicable to marked program elements
• using specific marker annotations
46
lundi 22 mars 2010
AST Transformations in Groovy 1.6
!Several (local) transformations finds their way
•@Singleton — okay, not really a pattern :-)
•@Immutable, @Lazy, @Delegate
•@Newify
•@Category and @Mixin
•@PackageScope
• Swing’s @Bindable and @Vetoable
• Grape’s @Grab
!Let’s have a look at some of them
47
lundi 22 mars 2010
The @Singleton anti-pattern
48
!The evil Java singleton
!In Groovy now:
!A lazy version also:
public class Evil {
public static final Evil instance = new Evil();
privavte Evil() {}
Evil getInstance() { return instance; }
}
@Singleton class Evil {}
@Singleton(lazy = true) class Evil {}
lundi 22 mars 2010
@Immutable
49
!To properly implement immutable classes
• No mutators (state musn’t change)
• Private final fields
• Defensive copying of mutable components
• Proper equals() / hashCode() / toString() for comparisons, or for keys in maps, etc.
@Immutable class Coordinates {
Double lat, lng
}def c1 = new Coordinates(lat: 48.8, l
ng: 2.5)
def c2 = new Coordinates(48.8, 2.5)
assert c1 == c2
lundi 22 mars 2010
@Lazy, not just for lazy dudes!
50
!When you need to lazily evaluate or instantiate complex data structures for class fields, mark them with the @Lazy annotation
!Groovy will handle the boiler-plate code for you!
class Dude {
@Lazy pets = retrieveFromSlowDB()
}
lundi 22 mars 2010
@DelegateNot just for managers!
51
!You can delegate to fields of your class
•Think multiple inheritance
!Damn manager will get all the praise...
class Employee {
def doTheWork() { "done" }
}class Manager {
@Delegate Employee slave = new Employee()
}def god = new Manager()
assert god.doTheWork() == "done"
lundi 22 mars 2010
Global transformations
52
! Implement ASTTransformation
!Annotate the transfo specifying a compilation phase
!For discovery, create the file META-INF/services/org.codehaus.groovy.transform.ASTTransformation
!Add the fully qualified name of the class in that file
@GroovyASTTransformation(phase=CompilePhase.CONVERSION)
public class MyTransformation
implements ASTTransformation {
public void visit(ASTNode[] nodes, SourceUnit unit)
{ ... }
}
lundi 22 mars 2010
Local transformations
53
!Same approach as Globale transformations
!But you don’t need the META-INF file
! Instead create an annotation to specify on which element the transformation should apply
@Retention(RetentionPolicy.SOURCE)
@Target([ElementType.METHOD])
@GroovyASTTransformationClass(
["fqn.MyTransformation"])
public @interface WithLogging {...}
lundi 22 mars 2010
Example: the Spock framework
54
!Changing the semantics of the original code
!But keeping a valid Groovy syntax
!Check out http://www.spockframework.org
@Speckclass HelloSpock {
def "can you figure out what I'm up to?"() {
expect:
name.size() == size
where:
name << ["Kirk", "Spock", "Scotty"]
size << [4, 5, 6]
}}
lundi 22 mars 2010
Various integration mechanisms
!Java 6’s javax.script.* APIs (aka JSR-223)
!Spring’s language namespace
!Groovy’s own mechanisms
!But a key idea is to externalize those DSL programs
• DSL programs can have their own lifecycle
• no need to redeploy an application because of a rule change
• business people won’t see the technical code
55
lundi 22 mars 2010
Java 6’s javax.script.* API
!Groovy 1.6 provides its own implementation of the javax.script.* API
56
ScriptEngineManager mgr =
new ScriptEngineManager();
ScriptEngine engine =
mgr.getEngineByName(“Groovy”);
String result =
(String)engine.eval(“2+3”);
lundi 22 mars 2010
Spring’s lang namespace
!POGOs (Plain Old Groovy Objects) can be pre-compiled as any POJO and used interchangeably with POJOs in a Spring application
!But Groovy scripts & classes can be loaded at runtime through the <lang:groovy/> namespace and tag
!Reloadable on change
!Customizable through a custom MetaClass
57
<lang:groovy id="events"
script-source="classpath:dsl/eventsChart.groovy"
customizer-ref="eventsMetaClass" />
lundi 22 mars 2010
Groovy’s own mechanisms
!Eval
• for evaluating simple expressions
!GroovyShell
• for more complex scripts and DSLs
!GroovyClassLoader
• the most powerful mechanism
58
lundi 22 mars 2010
Eval
!Simple mechanism to evaluate math-like formulas
59
Eval.me ( ‘3*4’)
Eval.x (1, ‘3*x + 4’)
Eval.xy (1, 2, ‘x + y’)
Eval.xyz(1, 2, 3, ‘x * y - z’)
lundi 22 mars 2010
GroovyShell
!A Binding provides a context of execution
• can implement lazy evaluation if needed
!A base script class can be specified
60
def binding = new Binding()
binding.mass = 22.3
binding.velocity = 10.6
def shell = new GroovyShell(binding)
shell.evaluate(“mass * velocity ** 2 / 2”)
lundi 22 mars 2010
GroovyClassLoader
!Most powerful mechanism
• could also visit or change the AST
• scripts & classes can be loaded from elsewhere
• more control on compilation
61
GroovyClassLoader gcl =
new GroovyClassLoader();
Class clazz = gcl.parseClass(
new File(“f.groovy”));
GroovyObject instance =
(GroovyObject)clazz.newInstance();
instance.setMetaClass(customMC);
lundi 22 mars 2010
Externalize business rules
!Although Groovy DSLs can be embedded in normal Groovy classes, you should externalize them
!Store them elsewhere
• in a database, an XML file, etc.
!Benefits
• Business rules are not entangled in technical application code
• Business rules can have their own lifecycle, without requiring application redeployments
62
lundi 22 mars 2010
A few considerations
63
lundi 22 mars 2010
Start small, with key concepts Beware over-engineering!
lundi 22 mars 2010
Grow your language progressively
lundi 22 mars 2010
Get your hands dirty Play with the end-users
lundi 22 mars 2010
Let your DSL fly, it’s not yours, it’s theirs!
lundi 22 mars 2010
Tight feedback loop Iterative process
lundi 22 mars 2010
Stay humble, You can’t get it right the 1st time.
Don’t design alone at your desk Involve the end users from the start
lundi 22 mars 2010
Playing it safe in a sandbox
lundi 22 mars 2010
Various levels of sandboxing
!Groovy supports the usual Java Security Managers
!Use metaprogramming tricks to prevent calling / instanciating certain classes
!Create a special GroovyClassLoader AST code visitor to filter only the nodes of the AST you want to keep
• ArithmeticShell in Groovy’s samples
71
lundi 22 mars 2010
Test, test, test!
!Don’t just test for nominal cases
• Explicitely test for errors!
!Ensure end-users get meaninful error messages
72
lundi 22 mars 2010
Question & Answers
73
lundi 22 mars 2010
Picture credits
http://www.flickr.com/photos/featheredtar/2305070061/
http://www.thedailygreen.com/cm/thedailygreen/images/WT/christmas-tree-with-gifts-flipbook.jpg
http://www.flickr.com/photos/chicoer2001/188468490/
http://www.flickr.com/photos/olibac/4054644737/
http://www.flickr.com/photos/epsos/3384297473/
http://media.techeblog.com/images/clapboard.jpg (clap)
http://www.flickr.com/photos/diegocupolo/3614879332/ (flower power)
http://www.flickr.com/photos/oskay/237442629/sizes/m/ (danger)
http://www.partybox.co.uk/data/partyimages/250x250/6ftcutoutaustinpowers.jpg (austin powers)
http://www.flickr.com/photos/27663074@N07/3413698337/ (jeepers creepers)
http://www.flickr.com/photos/wheatfields/420088151/sizes/l/
http://www.flickr.com/photos/therefromhere/518053737/sizes/l/
http://www.flickr.com/photos/romainguy/230416692/sizes/l/
http://www.flickr.com/photos/addictive_picasso/2874279971/sizes/l/
http://www.flickr.com/photos/huangjiahui/3127634297/sizes/l/
http://www.flickr.com/photos/25831000@N08/3064515804/sizes/o/
http://www.flickr.com/photos/lanier67/3147696168/sizes/l/
http://www.flickr.com/photos/ktb/4916063/sizes/o/
http://www.flickr.com/photos/nathonline/918128338/sizes/l/
http://www.flickr.com/photos/kevinsteele/39300193/sizes/l/
http://commons.wikimedia.org/wiki/File:Brueghel-tower-of-babel.jpg
http://commons.wikimedia.org/wiki/File:Platypus.jpg
http://www.flickr.com/photos/joaomoura/2317171808/sizes/l/
http://www.flickr.com/photos/wiccked/132687067/
http://www.flickr.com/photos/timsamoff/252370986/sizes/l/
http://www.flickr.com/photos/29738009@N08/2975466425/sizes/l/
http://www.flickr.com/photos/howie_berlin/180121635/sizes/o/
http://www.flickr.com/photos/yogi/1281980605/sizes/l/
http://www.flickr.com/photos/dorseygraphics/1336468896/sizes/l/
http://www.flickr.com/photos/xcbiker/386876546/sizes/l/
http://www.flickr.com/photos/pietel/152403711/sizes/o/
http://www.flickr.com/photos/forezt/192554677/sizes/o/
http://keremkosaner.files.wordpress.com/2008/04/softwaredevelopment.gif
http://www.jouy.inra.fr
http://www.flickr.com/photos/ejpphoto/408101818/sizes/o/
http://www.flickr.com/photos/solaro/2127576608/sizes/l/
http://www.flickr.com/photos/biggreymare/2846899405/sizes/l/
http://www.flickr.com/photos/wwworks/2222523978/ (earth)
http://static-p3.fotolia.com/jpg/00/01/64/30/400_F_1643044_YrBQoPnt0SC5gHAueG0bhx20yCSL42.jpg (trafic light)
http://aldaria02.a.l.pic.centerblog.net/lz2levrz.jpg (griffon)
http://www.flickr.com/photos/geishaboy500/104137788/ (soap)
Matriochka http://lang.russe.free.fr/images/matriochka.jpg
74
lundi 22 mars 2010