modularizing your grails application with private plugins - springone 2gx 2012
TRANSCRIPT
Modularizing Your Grails Application
with Private PluginsKen Liu
IEEESpringOne 2GX 2012
17 Oct 2012
Wednesday, October 17, 12
About Me
• Web development with Java since JDK 1.0.2 (applets, anyone?)
• Groovy/Grails since 2009
• Lead developer for IEEE Spectrum
• Always been a closet fan of dynamic languages (starting with Perl)
Wednesday, October 17, 12
About IEEE• Institute of Electrical and Electronics
Engineers
• Widely known for standards - 802.11
• World’s largest technology professional association - 400,000 engineers in 160 countries
• Publishes nearly a third of the world’s technical literature in EE/CS
Wednesday, October 17, 12
About IEEE Spectrum• Flagship publication of IEEE
• General interest technology magazine with focus on EE topics
• 2012 National Magazine Award winner
Wednesday, October 17, 12
IEEE Spectrum website
• http://spectrum.ieee.org
• ~1.5M PV/mo
• Frequent referrals from HN, Reddit, /.
• 100% Groovy/Grails since 2009 - both front-end and CMS
• Akamai/Weblogic/Oracle stack
• Award-winning coverage of ROBOTS!
Wednesday, October 17, 12
Agenda
• Grails plugins overview
• Grails plugin development basics
• Application architecture with private plugins
• Lessons learned
Wednesday, October 17, 12
To fork or not to fork?
• “We need a new website...”
• “You must use the standard corporate Enterprise CMS system.”
• “Can’t you guys just make a copy of the existing code and tweak it?”
• DANGER!
Wednesday, October 17, 12
From “big ball of mud” to plugin architecture
Front EndControllers/Views/
Services(e.g. article, blog pages)
Domain classes(e.g Article,
BlogPost)
CMS Controllers/
Views/Services
Quartz Jobs(e.g. sitemap generation)
Spectrum TaglibUtility classes
(src/**)UrlMappings
Test classesFront End & CMSimages, CSS, JS
Where do we go from here?
Wednesday, October 17, 12
• Grails plugins overview
• Grails plugin development basics
• Application architecture with private plugins
• Lessons learned
Agenda
Wednesday, October 17, 12
Plugin systems and component models
• Plugin systems are now commonplace
• Firefox, Eclipse, Jenkins, JEdit
• Package managers, OSGi, Ruby on Rails Engines
• Component-oriented software?
• Coarse-grained high-level reuse, separation of concerns
Wednesday, October 17, 12
Grails plugins in a nutshell
• Tightly integrated into Grails system
• Can implement common functionality in plugins that is overridden by application
• Can create new Artifact types
• Interact with Spring context
• Integrated with build system
Wednesday, October 17, 12
Grails plugins in a nutshell
• Central public plugin repository
• Version management
• Dependency management
Wednesday, October 17, 12
Grails - plugins inside
• Many parts of Grails itself are built as plugins
• Some default plugins can even be swapped out
• Tomcat -> Jetty
• GORM/Hibernate -> Redis GORM
Wednesday, October 17, 12
Public Grails plugins• 850 public plugins in the central
plugins repository
• Search on plugin portalhttp://grails.org/plugins/
• Quickly bootstrap new applications
• Spend some time nowsave some time later
• Follow @grailsplugins for updates
Wednesday, October 17, 12
Selected Grails plugins
Spring Security - user authenticationCache - caching using annotationsQuartz - job scheduling (like cron)DB Migration - controlled schema changesMail - send email with JavaMailResources - web performance optimizationConsole - in-app interactive consoleCodeNarc - static code analysisSpock - enable Spock unit testing framework
Wednesday, October 17, 12
What can go into a plugin?
• Really, anything
• Standard Grails artifacts: GSPs, Controllers, Services, i18n, TagLibs - just like in app
• Define custom Grails artifact types: see Quartz plugin “Jobs”
• Plugins can depend on other plugins
Wednesday, October 17, 12
Agenda
• Grails plugins overview
• Grails plugin development basics
• Application architecture with private plugins
• Lessons learned
Wednesday, October 17, 12
Creating a new plugin
grails create-plugin myplugin
Wednesday, October 17, 12
Creating a new plugin
Same structure as a Grails application, but with an extra plugin descriptor file
Wednesday, October 17, 12
Plugin Descriptorclass MypluginGrailsPlugin { // the plugin version def version = "0.1"
// the version or versions of Grails the plugin is designed for def grailsVersion = "2.1 > *"
// the other plugins this plugin depends on def dependsOn = [:]
def doWithDynamicMethods = { ctx -> // TODO Implement registering dynamic methods to classes (optional) }
def doWithApplicationContext = { applicationContext -> // TODO Implement post initialization spring config (optional) }
Wednesday, October 17, 12
Excluded plugin files
Grails excludes certain files from packaging
/grails-app/conf/BootStrap.groovy/grails-app/conf/BuildConfig.groovy/grails-app/conf/Config.groovy/grails-app/conf/DataSource.groovy(and any other *DataSource.groovy)/grails-app/conf/UrlMappings.groovy/grails-app/conf/spring/resources.groovy/web-app/WEB-INF/**/web-app/plugins/**/test/**
Wednesday, October 17, 12
Running a plugin
• A plugin is a Grails app!
• grails run-app from plugin dir
• excluded files loaded during run-app
• useful for testing (test-app works too)
Wednesday, October 17, 12
Packaging a plugin
• grails package-plugin
• creates a zip file that can be installed & distributed
Wednesday, October 17, 12
Installing a plugin
from local directory:grails install-plugin /path/to/grails-example-0.1.zip
from URL: grails install-plugin http://myserver.com/plugins/grails-example-0.1.zip
in BuildConfig.groovy (“in-place” plugin):// Uncomment this only for local development - do not check in!grails.plugin.location.'spectrum-core' = '../spectrum-core'
Wednesday, October 17, 12
Plugin Development
• Easy: create Controllers, Services, Views, Taglibs, etc.
• Specify resource paths <g:resource>
• Can alter Spring context
• Hook into build system, reload events
Wednesday, October 17, 12
• Grails plugins overview
• Grails plugin development basics
• Application architecture with private plugins
• Lessons learned
Agenda
Wednesday, October 17, 12
Public vs. Private plugins
• Highly cohesive
• Maximize reuse
• Sometimes provide “horizontal slices” - e.g. Spring security core
• Most don’t depend on other plugins
Public plugins
Wednesday, October 17, 12
Public vs. Private plugins
• Proprietary code
• Application or domain-specific
• Code reuse within an organization
• Can be coarse-grained, heavyweight
• Private forks of public plugins
• Enable interesting options for application architecture
Private Plugins
Wednesday, October 17, 12
From “big ball of mud” to plugin architecture
• Grails applications tend to become unmanageable over time
• Convention-based directory trees(e.g. /grails-app/views )
• Rapid development/scaffolding
• Need “separation of concerns”
Wednesday, October 17, 12
From “big ball of mud” to plugin architecture
Front EndControllers/Views/
Services(e.g. article, blog pages)
Domain classes(e.g Article,
BlogPost)
CMS Controllers/
Views/Services
Quartz Jobs(e.g. sitemap generation)
Spectrum TaglibUtility classes
(src/**)UrlMappings
Test classesFront End & CMSimages, CSS, JS
Spectrum application before refactoring
Wednesday, October 17, 12
Reusable domain model
• Reuse a set ofDomain classes between applications
• Different applications, same DB instance
• Hibernate not included in plugins by default
• Consider Hibernate caching issues
DB
Primary webapplication
Domain Model plugin
API endpoint application
Domain Model plugin
Reporting application
Domain Model plugin
Wednesday, October 17, 12
Refactored with plugin
Front EndControllers/Views
Core Domain classes
CMS Controllers/
Views/Services
Quartz Jobs
Front End Taglib
Utility classes(src/**)
Front EndUrlMappings
Test classes
Front Endimages, CSS, JS
CMS test classes
CoreUrlMappings
Spectrum Core PluginSpectrum Application
CMSimages, CSS, JS
Common Front End Controllers
Core Services
CMS Taglib
Depends on other plugins: spring-security-core, quartz, searchable, ckeditor, and others
Wednesday, October 17, 12
Different web sites, common backend code
spectrum-corePlugin
Spectrumwebsite
10.4+
The Institutewebsite
3.0
Wednesday, October 17, 12
Same Controller, different Views
FrontendArticleController#show()
Spectrum applicationshow.gsp
The Institute applicationshow.gsp
Wednesday, October 17, 12
Same Controller, different Views
• Define common Controller in plugin
• View resolution: application first, then plugin
• Can specify plugin using<g:render plugin=”myplugin”>
• The promise of MVC fulfilled!
Wednesday, October 17, 12
Plugin UrlMappings• UrlMappings.groovy excluded from plugin
build
• *UrlMappings.groovy merged into application mappings
• Application can override plugin mappings
• Enables “mini application” in plugin
• Can’t remove unwanted mappings, only override
Wednesday, October 17, 12
Config.groovy• Config.groovy is excluded from plugin
• Might want to fix certain config properties - won’t need them in application Config
• Manually merge config in doWithSpring
• application.config.merge(new ConfigSlurper().parse(application.classLoader.loadClass('MyPluginConfig')))
Wednesday, October 17, 12
Extending plugindomain model
Domain classes in plugins can be
extended in application
Content classDomain model plugin
Webinar extends ContentApplication
Wednesday, October 17, 12
Publishing private plugins
• Publish to internal corporate Maven repo - e.g. Artifactory or Nexus
• Release plugin - installed by default
• Old-style svn-based repos supported, but not recommended for Grails 2
• Customize plugin metadata with internal Maven groupId
• Publish from Jenkins
Wednesday, October 17, 12
Declaring private plugin dependencies
repositories { grailsPlugins() /* other default repos here */
/* point to internal corporate Maven repo */ mavenRepo 'http://mvn.ieee.org/nexus/content/repositories/spectrum/'}
plugins { /* other plugin dependencies here */ compile 'org.ieee.spectrum.plugins:spectrum-core:2.0-SNAPSHOT'}
BuildConfig.groovy (Maven/Ivy dependencies)
Wednesday, October 17, 12
Day-to-day development
• Use “in-place” plugin (exploded plugin) in your local dev workspace (BuildConfig.groovy)
• Private plugins tend to be updated along with application features
• Beware of potential breaking changes(in other applications)
• Define deprecation strategy and use @deprecated javadoc
Wednesday, October 17, 12
Refactoring to plugin
• Determine candidates for refactoring
• Create new empty plugin & plugin descriptor
• Move Grails artifacts (e.g. Controllers, Services, Views, etc.) into new plugin
• Move dependent files from /src
• Move related test files into new plugin• Remove hard coded paths to resources
• Run your functional test suite
Wednesday, October 17, 12
Refactoring to plugin
• Start small - don’t create a whole bunch of plugins from the outset
• Consider migrating version history
Wednesday, October 17, 12
Super-project with pluginsOne application with several in-place plugins
/myapplication /grails-app ... /pluginA /grails-app /pluginB /pluginC
grails.plugin.location.pluginA = 'pluginA' // path to plugingrails.plugin.location.pluginB = 'pluginB'grails.plugin.location.pluginC = 'pluginC'
In BuildConfig.groovy:
Wednesday, October 17, 12
Public website,private back end
Deploy public and private versions of application with shared code
DB
Business-facing
application
Core application
plugin
Public-facingapplication
Core application
plugin
PublicInternet
Private intranet
Wednesday, October 17, 12
Vertical slices / mini Grails applications
• Plugins can be self contained
• UrlMappings + Controllers + Domains + Views + Services = Grails application
• Remove views -> “white label” application
• Embed a small application within another application
• Vertically slice your application into plugins
Wednesday, October 17, 12
Plugins as a platform
Clinical Data Management Platform plugin
Clinical data management server
application
Clinical data review application
Data Captureplugin
Data Cleaningplugin
Data Validationplugin
Super-plugin to combine plugin
dependencies into a platform or
domain-specific framework
Wednesday, October 17, 12
Sensitive code protection
Data Ingestionplugin
Data Analysisplugin
“secret sauce”
Full application
DB
Shell application (external team)
Split applicationinto separate parts to isolate key algorithms from external vendor
Wednesday, October 17, 12
• Grails plugins overview
• Grails plugin development basics
• Application architecture with private plugins
• Lessons learned
Agenda
Wednesday, October 17, 12
Domain/Schema management
• Changes to domain model need to be carefully managed
• Avoid dbCreate=update
• Adopt DDL file naming convention:spectrum-core-2.1.sqlspectrum-core-2.2.sql
• Bootstrap-triggered migrations
• DB Migrations plugin (Liquibase)
Wednesday, October 17, 12
Config.groovy changes over time
• Easy to add a new entry to Config.groovy and then forget about it
• Group together plugin config properties into a closure
• Don’t forget about external config files and differences in various environments
Wednesday, October 17, 12
Maven/Ivy/build issues
• Maven/Ivy hell > JAR hell
• SNAPSHOT resolution error-prone
• See JIRA for bugs related to plugins
• Avoid creating many small plugins
• Strange problems? Look at your build.
Wednesday, October 17, 12
Plugin versioning and branching
• Define version numbering strategy
• Be prepared to branch/merge
• Be sure to update plugin versions and tag your code consistently
• Just use Git - merging & cherry-picking are sweet
Wednesday, October 17, 12
Code Rigidity
• Inflexibility to change/afferent coupling
• Avoid breaking changes (duh)
• Solution: automate your tests, catch breaking changes early
Wednesday, October 17, 12
Continuous Integration
• Use Jenkins to trigger builds of applications that depend on your private plugin
• Run Plugin test suite with plugin build
• Publish snapshots to Maven repo from Jenkins
Wednesday, October 17, 12
Plugin namespace conflicts
• Plugin artifacts all share same namespace with each other
• Potential for class name collisions, especially with “conventions”
• Planned enhancement for Grails 2.2
Wednesday, October 17, 12
Plugin resources
• Plugin static resources (images, CSS, JS, etc.) packaged in a path corresponding to plugin name & version - /plugins/myplugin-1.0/*
• path changes between dev mode and WAR - <g:resource> handles correctly
• May need to redeploy with each plugin version bump
• Don’t accidentally share sensitive files!
Wednesday, October 17, 12
Component interfaces
• Grails plugins don’t have any kind of external “interface” or contract besides Plugin Metadata
• Up to you to make the rules(not a bad thing)
Wednesday, October 17, 12
Release strategy for 2013 website redesign
Success!
spectrum-core Plugin2.0, 2.1, 2.2
Spectrumwebsite
10.8, 10.9, 10.10
The Institutewebsite
3.x
Spectrumwebsite
4.0 - 2013 release
Wednesday, October 17, 12
Takeaway
• Plugins are easy to create and use
• Private plugins are a safe, robust way to modularize Grails applications
• Private plugins enable coarse-grained reuse of application components
• Configuration management and OO principles are important (as always)
Wednesday, October 17, 12
Additional resources
• @grailsplugins - Grails plugin updates - https://twitter.com/grailsplugins
• Grails reference docs chapter on plugins - http://grails.org/doc/latest/guide/plugins.html
• Inside Grails: The Build System and Plugin Management - http://www.slideshare.net/skillsmatter/grails-internals-ggug-dec-2009
• GGUG: Grails Plugins - Lessons to Learn http://skillsmatter.com/podcast/java-jee/grails-plugins
Wednesday, October 17, 12
Questions?
Wednesday, October 17, 12
Thanks!
feedback is appreciated.
@kenliu
Wednesday, October 17, 12