Dynamic Groovy Edges

Download Dynamic Groovy Edges

Post on 15-Jan-2015




3 download

Embed Size (px)


Spring and Groovy to create dynamic integration edges and regression isolation.


<ul><li> 1. 1 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges May 21st, 2012 Prepared for: Triangle Java Users Group (TriJUG) </li></ul><p> 2. 2 Who Am I? Well Java Architect with ICF Ironworks Adjunct Professor Started with HTML and Lotus Notes in 1992 In the interim there was C, C++, VB, Lotus Script, PERL, LabVIEW, etc. Not so much an Early Adopter as much as a Fast Follower of Java Technologies Learned Java 1.1 in 1997, J2EE in 1999 Alphabet Soup (MCSE, ICAAD, ICASA, SCJP, SCJD, PMP, CSM) LinkedIn: http://www.linkedin.com/in/iamjimmyray Blog: http://jimmyraywv.blogspot.com/ Avoiding Tech-sand 3. 3 Managing Change and Achieving Regression Isolation Dynamic Groovy Edges 4. 4 JEE Application Architecture Arranged into layers When there are N layers: There are N-1 layer junctions There are (N-1) X 2 layer edges In this diagram there are three layers, two layer junctions, and four layer edges. We control edges 5. 5 The Issue with Layers Often written by someone else You often have little control or influence over the implementation of the layer to which you must integrate You control your side (edge) of the interface junction Notorious change zones Changes occur at different paces 6. 6 Keeping Up the Pace Pace Layering Approach to Application Design http://www.doozer.com/developer-exchange/entry/accelerating-innovation- by-adopting-a-pace-layered-application-strategy Complex systems can be decomposed into multiple layers, where the layers change at different rates. The fast layers learn, absorb shocks and get attention; the slow layers remember, constrain and have power. One of the implications of this model is that information architects can do what they have always doneslow, deep, rich work; while tagging can spin madly on the surface. If it is worth keeping, it will seep down into the lower layers. Donna Maurer 7. 7 When Layers Change In Java web applications, regression analysis and testing are usually tied to the boundaries of the application or deployment artifacts: JARs, WARs, EARS etc. Upstream and downstream dependencies also come into play Not to be confused with functional testing of changed components If you redeploy the entire WAR or EAR, you could find yourself testing that entire application boundary. As well as its integration junctions (critical patches not withstanding) 8. 8 Detecting Changes How do we detect changes to web services as soon as possible? XMLUnit/JUnit Tests Easily written in Groovy Hudson scheduled Ant tasks DIFFERENCE CODE:19:CHILD_NODELIST_LENGTH_ID:Comparing 2 nodes with different numbers of children Control(prod) Node: Target(dev) Node: 9. 9 Demo Detecting Changes Groovy Console JUnit XMLUnit Apache 2.2 Tomcat 6.0.35 10. 10 CI Using Groovy XmlUnit Tests Wrap Groovy script with Ant Execute in Hudson build Use Gradle with the Groovy Plugin Run your Groovy test within Gradle build 11. 11 Managing Change How do we better manage the change between the layers? Understand that we cannot stop change Reduce regression testing needs by shrinking regression boundaries Prepare our teams for the changes with more dynamic reaction techniques Start with Groovy beans OrStart with Java beans and move to Groovy as needed Detect changes as early as possible, and before they impact downstream systems 12. 12 Groovy Edges Build your edges with Groovy (or other dynamic JVM language) objects Groovy can be written as a script and parsed at run- time and compiled into Java byte-code and then loaded by the Groovy class loader. Groovy scripts can contain Groovy and Java syntax in the script file (*.groovy). Groovy can use the same Java APIs that are used by traditional Java code. Groovy adds features that make it easier to work with XML (primary language of choice for most integrations) Markup Builder XML Slurper 13. 13 Groovy Markup Builder Allows developers to use builder patterns to construct XML def writer = new StringWriter(); def xml = new MarkupBuilder(writer); xml.'ser:writeEmployee'('xmlns:ser':"http://service.rjug.ironworks.com/"){ employee{ dateOfBirth(emsRequest.getEmployee().getDateOfBirth()) firstName(emsRequest.getEmployee().getFirstName()) lastName(emsRequest.getEmployee().getLastName()) middleName(emsRequest.getEmployee().getMiddleName()) employeeId(emsRequest.getEmployee().getEmployeeId()) hireDate(emsRequest.getEmployee().getHireDate()) address{ addressLine1(emsRequest.getEmployee().getAddress().getAddressLine1()) 14. 14 A Word About Hard-Coding (or not) When is hard-coding not really? Our internal beliefs have shifted slightly. How about when it is in Groovy? Especially when that Groovy is treated as content and not part of a rigid deployment architecture? Still have to work within the boundaries (constraints) of the JVM 15. 15 Groovy XML Slurper Groovy construct for efficient partial XML processing. EmployeeManagerServiceResponse emsResponse = new EmployeeManagerServiceResponse(); GPathResult response; try{ response = new XmlSlurper().parseText(rawResponse.getWriter().toString()); emsResponse.getEmployee().setFirstName(response.'return'.firstName.text()); emsResponse.getEmployee().setLastName(response.'return'.lastName.text()); 16. 16 Groovy XML Slurper Another Example slurper.jobList.job.each { it.docs.doc.each { doc = new DocDto(); docs.add(doc); doc.setDocumentId(new BigInteger(it.documentId.toString())); //doc.setDcmntNbr(new BigInteger(it.dcmntNbr.toString())); doc.setOriginalFilename(it.originalFilename.toString()); doc.setPhasecode(it.phasecode.toString()); doc.setRcvdDt(it.rcvdDt.toString()); doc.setStatuscode(it.statuscode.toString()); doc.setStepcode(it.stepcode.toString()); 562132403073245.pdf422009-01-10 00:00:00.0ABC 17. 17 A Word About Groovy Slurping XML Payloads No longer tied to JAXB This is uncomfortable for some traditional Java developers Can use Groovy closures on collections (even nested collections) Easy to enable/disable getter calls, especially if Groovy is stored outside of rigid deployment structure (CMS, Apache, etc.) This is scary to some Still requires Configuration Management rigor Its about risk tolerance It could be that this approach is better suited for non-production environments within certain organizations 18. 18 Groovy XML Slurper Trade-offs Use the Groovy XML Slurper when you do not have to parse the entire XML Slurper uses paths expressions as apposed to constructing a DOM Path expression might be executed multiple times Could be slower in larger, more complex XML There are also differences when editing XML The XML Parser allows for edits inline The XML Slurper would require another slurp after changes are applied. The XML Slurper is well suited for simple web service clients Allows direct access via getters to return object graph 19. 19 Groovy Edge Component Web Service Client For SOA interfaces implemented with web services, web service clients are primarily the target edge component for dynamic updating. Using Groovy to write these web services clients makes it easier to update clients. Features can be added or removed and quickly redeployed via content changes, and not necessarily Java deployments Spring will reload the Groovy web service client when it changes This provides a great solution for prototyping in an Agile environment. For MVC containers, the controller is the edge component that is up for Groovy-enizing, Groovy-ization, etc. 20. 20 Spring Beans 21. 21 Groovy Location Property groovy.url=http://localhost/groovy/EmployeeManagerServiceImpl.gro ovy Not only that, but Store your properties outside of the WAR/EAR, reachable via URL 22. 22 Loading Properties http://localhost/properties/service.properties 23. 23 Demo Web Services Provider Simple JAX-WS Bottom-up web service Environment: Apache2.2 Tomcat 6.0.35 Java 1.6 JRE JAX-WS RI 2.2 JAXB 2.2.3 SoapUI 4.0.0 Spring Data Layer MongoDB Data Repository MongoDB NoSQL Database (Upcoming TriJUG Event: 07/16/12 Deep Mistry Mongo DB) 24. 24 Demo Web Services Consumer Simple Spring WS Client Environment: Apache2.2 Tomcat 6.0.35 Java 1.6 JRE Spring Framework 3.0.x Spring WS Groovy Web Services Framework 25. 25 This Is Not Bleeding Edge Yet, it is still new to a lot of Spring users January 2009: http://www.ibm.com/developerworks/java/library/j- groovierspring2/index.html At the NVSS NFJS conference in Reston in April 2012, there were still sessions on Spring and Groovy integration with dynamic beans NFJS Due in Raleigh, NC August 24-26 http://www.nofluffjuststuff.com/home/main 26. 26 Spring Bean Configuration Issues Spring needs to be able to locate Groovy script content in order to load configurations. ERROR - ContextLoader.initWebApplicationContext(220) | Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'employeeManagerService' defined in file [C:Toolsapache-tomcat-6.0.16w ebappsDynamicGroovyTutorialWEB-INFclassescontextservice_ws.xml]: Cannot resolve reference to bean 'employeeManagerServiceImpl' while setting bean property 'groovyImpl'; 27. 27 Spring Bean Config With Content Proxy Use a Content Proxy The Content Proxy is deployed into a separate WAR and addressable via a URL (perhaps a RESTful web service) that would retrieve the dynamic Spring Bean configuration scripts If the scripts are not found in script repository, the Content Proxy can load the scripts from a backup site or even from inside the application boundaries Default Location. The Content Proxy is a layer of indirection to better manage the link between your Spring application and the content management system that you may be using. Content Proxies are also very helpful when scripts (or other content) are loaded from a CMS through an API that would not link directly to the Spring Bean configurator. Provides a layer of indirection and intelligence for Cache Forward Architectures. 28. 28 Avoiding the Race Condition Using a ServletContextListener allows you to specify how long the application will try to seek the ContentProxy (or other URLs) in other WARs that still may be spinning-up, before it lets the rest of the application load. Reduces the race-condition introduced by URLs across multiple WARs 29. 29 ServletContextListener web.xml Configuration proxyUrlhttp://localhost/proxy.htmlstopInterval60000com.ironworks.rjug.container.listener.ContentProxyListenerclass&gt; 30. 30 Inline Groovy You can, but why would you? package com.ironworks.framework.scripting; import org.springframework.core.io.ResourceLoader; import org.springframework.scripting.ScriptSource; import org.springframework.scripting.support.*; public class URLScriptFactoryPostProcessor extends ScriptFactoryPostProcessor{ @Override protected ScriptSource convertToScriptSource(String beanName, String scriptSourceLocator, ResourceLoader resourceLoader) { if (scriptSourceLocator.startsWith(INLINE_SCRIPT_PREFIX)) { return new StaticScriptSource( scriptSourceLocator.substring(INLINE_SCRIPT_PREFIX.length()), beanName); } else if (scriptSourceLocator.startsWith("classpath")) { return new ResourceScriptSource(resourceLoader.getResource(scriptSourceLocator)); } else{ return new URLScriptSource(resourceLoader.getResource(scriptSourceLocator)); } } } 31. 31 Spring Bean Delegating Construct Use a Spring Bean Delegate construct as an alternative to ServletContextListener and ContentProxy for handling potentially unreliable or dynamic URLs during application startup Deploy JAXB version of services, but leave them open to replacement by Groovy, without needing another deployment Still able to use Dynamic Groovy Edges, if JAXB version of service client requires updating 32. 32 Spring Bean Delegate Construct Code public final class DelegatingService implements Service { private List&gt; delegates = new ArrayList&gt;(); @Override public RESPONSE execute(final REQUEST request) { for (Service service : delegates) { if (!(service instanceof AvailabilityStatus) || ((AvailabilityStatus) service).isAvailable()) { return service.execute(request); } } throw new UnsupportedOperationException( "No service implementation available to execute request"); } public void setDelegates(final List&gt; delegates) { this.delegates = delegates; } } 33. 33 AvailabilityStatus Interface public interface AvailabilityStatus { /** * Indicate whether this service instance is available. * * @return true if the service is available, false otherwise. */ boolean isAvailable(); } 34. 34 Spring Delegate Construct Code Explanation DelegatingService is substituted into the class attribute of the bean definition Possible delegates are added to the property list as objects DelegatingService looks for viable services in the list of delegates Could be services that implement the AvailabilityStatus interface, and are available (isAvailable()) Beans that implement AvailabilityStatus, but are not available our by-passed This allows Groovy substitutes to be enabled/disabled quickly Could be services that do not implement AvailabilityStatus 35. 35 Some Tradeoffs for Groovy Approach Groovy Web Clients Pros Easier to program with Getters and Setters approach to accessing and mutating XML objects for request and response Markup Builder XML Slurper Cons Lose auto-generation of object graphs via JAXB Lose tighter integration to WSDL/XSD and makes it harder to track down changes. Its not for every changemore complex WSDLs, possibly with nested object architectures, would not be good candidates for Groovy re-write 36. 36 What About All Those Classes? Almost every technical decision comes with a trade off When using dynamic Groovy with the approach we have outlined, obviously we are generating more non-setup, Groovy classes in the JVM. Every time Spring detects a changed script, Groovy will parse it and load another class. Depending on your JVM, GC settings and JVM memory settings, you could experience OOM issues with PermGen 37. 37 Does Your JVM Have a PermGen? IBM: No PermGen, classes exist in the native C-Heap Oracle/WebLogic JRockit: No PermGen, classes exist in the native C-Heap Oracle/Sun Hotspot: Yes, PermGen in Java Heap Prior to Java 5, the Hotspot JVM assumed that classes were forever Then came CMS (Concurrent Mark Sweep) class unloading settings 38. 38 What Can We Do With PermGen? Use GC settings for the CMS Collector Java 1.6: -XX:+CMSClassUnloadingEnabled Java 1.5: -XX:+CMSPermGenSweepingEnabled Both require CMS GC enabled This may not be suitable for all applications Increase size (not really desirable) Switch to 64 bit, if you have not already Increase JVM bounce frequency (not really desirable) Not really a good long term strategy 39. 39 Dont Use PermGen. Some of our projects use JRockit While we do not see the same sensitivity to PermGen in our dynamic environment, occasional...</p>