1006 z2 intro complete
DESCRIPTION
Z2 Environment OverviewTRANSCRIPT
© ZFabrik Software KG 2010
The z2 Environment- introduction -
© ZFabrik Software KG 2010
… the solution to humane life-cycle management for Java solutions:
Self-updating from source,
Cost-effective and highly productive,
Extremely easy to distribute,
Transparent for developers and supporters,
Auditable and versioned,
Standards friendly and extensible
The z2 Environment is...
© ZFabrik Software KG 2010
Motivation
Operating a continuous integration infrastructure is complex and costly
Deployment of application servers and applications is complex and time consuming - developers tend to delay integration
Operating developer build infrastructure is error-prone, complex, and hard to debug
So Don't Do It!
© ZFabrik Software KG 2010
Runtime is fully capable of preparing all source code and configuration for execution – no external toolset required.
Re-use of standard versioned store that holds everything that defines a system's content – no external configuration required.
Store- and file system based integration with development tools – no complex development tool integration required.
Approach
© ZFabrik Software KG 2010
MachineMachineMachine
Architecture: Deployment
Many <home>s share one root component repository - usually implemented over a subversion repository
Every <home> runs a <home> Java process that manages one or more worker Java processes, as defined by a home layout
Worker processes run whatever participates in their target states
One <home> may run over several component repositories, in particular a dev repo that is defined over a workspace folder structure
<home>
Worker 1 Worker 2
manage
Check for updates
Repository
e.g. subversion, a workspace,...
© ZFabrik Software KG 2010
Cus
tom
/Ext
ensi
ble
z2 im
ple
men
tatio
n
Architecture: In-process
Servlet/JSP
Jetty
Ker
nel (
bina
ry)
Resource Management
Component Model &Component Factories
Pro
gra
mm
ing
Mod
els
& C
onve
n tio
ns
JRE
App
licat
ions
DB Migration
Migrate4j
Java Builder
ECJ
SVN Repo
SVNKit
JNDI Naming
OSGi Bundles
Equinox
Timer/Jobs
Worker Processes
Data Sources
System States
© ZFabrik Software KG 2010
Components in z2
The component concept captures anything stored in the repositories that z2 understands at runtime, for example
Web Applications
Worker process configurations
Java modules
System states
At runtime, in particular during development, changes in the repositories translate to component state invalidations (e.g. unloading of a Web application)
Repository
C1 C2
Cn
Worker Process
Componentsas defined inthe repo
C1C1C1C1
C1C1C1
C2
C1C1C1C1Cn
Web App
Java Classes
JDBC connection pool
depends
e.g. a Web App
e.g. Java e.g. a datasource
corresponds
Connection props read from C2
Connection props read from C2
Source files readfrom C1 and compiled
Source files readfrom C1 and compiled
Web resource in WebContent folder of Cn
Web resource in WebContent folder of Cn
© ZFabrik Software KG 2010
Synchronization is about letting go...
When synchronizing, changes in the repository (since the last synchronization) are detected and translated into component invalidations
Invalidations observe declared and implicit dependencies
Invalidations are orchestrated by the <home> process, so that the life-cycle of worker processes is also subject to change detection
After invalidation, the <home> process will try to have all worker processes attain their target states again
Repository
C1 C2
Cn
Worker Process
Componentsas defined inthe repo
C1C1C1C1
C1C1C1
C1
C1C1C1C1Cn
Web App
Java Classes
JDBC connection pool
depends
e.g. a Web App
e.g. Java e.g. a datasource
1
2
3
Example: Detection of modification triggers
invalidation chain
Example: Detection of modification triggers
invalidation chain
© ZFabrik Software KG 2010
System States
System states abstract operational modes of a system into configuration objects
Examples are:
de.travelhands/up → Start everything needed for the Travelhands front end
environment/jobWorkerUp→ target state for batch jobs
Components (in particular system states) can participate in states and depend on states: “run” before attaining a state, or a state must be attained before the component “runs” resp.
state
de.travelhands/up
state
environment/up
state
de.travelhands/up_db
state
de.travelhands/up_dev
depends
participates
part
icip
ates
schema
de.travelhands/tenantDB
participates
web
de.travelhands.app/web
participates
web
de.travelhands.admin/web_tests
participates
Example: Travelhands state graph
Example: Travelhands state graph
© ZFabrik Software KG 2010
Scenarios...
© ZFabrik Software KG 2010
<home>* Installation and Operation
Best practice: Keep bootstrapping core distribution in same repository as the base system
Install and update <home> by issuing simple Subversion client commands
Core installation is a matter of seconds
All further configuration is stored in Subversion and loaded on-demand
As all configuration and code is in a versioned store, configuration changes are tracked, secured and can be reverted at any time.
Checked in core dist(~7 MB)
Checked in core dist(~7 MB)
svn co …svn up ...svn co …svn up ...
<home> 1 <home> 2 <home> 3
© ZFabrik Software KG 2010
Development & Modification
Have a running local <home> for testing and provisioning of binary
Check out projects (say A) to modify from the Subversion repo into an Eclipse Workspace
Use the Eclipsoid plugin to “arm” those projects and to satisfy all project dependencies
Keep modifying, sync, and test until done.
Note: There is no deployment
Commit or revert changes
“Disarm” projects. Done
check
outA
A' sync A from here
sync all but A from here
Project inEclipse workspace
<home>
Sourcerepo
© ZFabrik Software KG 2010
SCMSCM
SCM
System Creation and Cloning
A system is completely defined (incl. configuration) by a repository URL (and hence a repository)
New systems are created by basic repository operations (e.g. branch)
When cloning, only essential interfaces with the outer world, e.g. database URLs, must be adapted
Every system constitutes a complete development branch
Updates are promoted between repositories via Transports
Shipping means to deliver a repository dump
branch
copyOR
© ZFabrik Software KG 2010
Transport
Transports work via tool-supported creation of delta-workspaces that allow reviewing of changes before committing to the transport target (in particular when considering configuration)
Transports promote changes between repositories (and hence systems)
A standard transport route is helpful but not restrictive: cross-transport are a necessary reality
Transports can be used for remote upgrades as well
prepare
commit
revi
ew
prepare
commit
revi
ew
© ZFabrik Software KG 2010
Support
As the runtime is a faithful representation of its repository, there is generally no doubt about sources when debugging
Since adding another <home> is trivial, debugging without interfering with production runtimes is easily achieved
As every system is a potential development branch, emergency fixes can be applied on the spot
In general: Fixes can be supplied as unified diffs
ProductionSupport
debug, emergency fix
Part of system
check
© ZFabrik Software KG 2010
Support WorkflowsBug fixing flow with z2:1. Analyze the bug report: What system?
2. Install local <home> for that system, possibly on site
3. Debug and fix on local <home>
4. Test fix. Submit fix. Transport fix or provide diff
5. Customer imports fix and tests again
6. Propagate fix to other releases via transport
Traditional bug fixing flow:1. Analyze the bug report: What release?
2. Identify patch level of product or module
3. Find code line for that release
4. Verify/Install suitable app server
5. Build application for given version (e.g. tag)
6. Deploy application
7. Debug, apply fix and test
8. Submit and build application again
9. Copy binaries to customer – hope that it does not contains lots of other changes
10. Convince customer to install new application and test
© ZFabrik Software KG 2010
Positioning...
… w.r.t. to ANT and Maven
© ZFabrik Software KG 2010
In Principle...Sources
Developers
Define what'shappening there
Modify
Test,
debug,
support
Dev
, Q
A,
or P
r od.
Run
time
© ZFabrik Software KG 2010
So, Where is the Problem?
Java application servers do not live on sources but rather on binaries
It is up to the development organization to (somehow) provide those binaries and hence... … find out what binaries are affected by a change … find out what binaries belong to one solution … keep development environments up-to-date … integrate changes (continuously) … live with delays between commit and run
© ZFabrik Software KG 2010
Deployables
The “classical” approach
Dev
. Ru n
time
Sources
Dev
elop
ers
(wor
ksp a
ce)
Check o ut and
comm
it change s
Nightly build(everything?)
deploy/push(everything?)
Deployables
Dev build(what?)
QA
. R
u ntim
e
debug
deploy/push(what?)
central
local
© ZFabrik Software KG 2010
Deployables
The “classical” approach
Dev
. Ru n
time
Sources
Dev
elop
ers
(wor
ksp a
ce)
Check o ut and
comm
it change s
Nightly build(everything?)
deploy/push(everything?)
Deployables
Dev build(what?)
QA
. R
u ntim
e
debug
deploy/push(what?)
central
local
How “equal” are these?How “equal” are these?
How “hard” is it to get thisup to date?
How “hard” is it to get thisup to date?
What are the sources of what you are debugging (but others change)?
What are the sources of what you are debugging (but others change)?
How “hard” is it to introduce a
structural change?
How “hard” is it to introduce a
structural change?
© ZFabrik Software KG 2010
The “classical” approach
Ok for monolithic projects (only few deployables)
Breaks for modular systems (too complex to keep in sync: Due to the “push” approach, it's up to the developer to keep things straight)
Breaks for large projects (turn around times too long)
Summary: Simple but doesn't scale.
© ZFabrik Software KG 2010
The Maven Way
Dev
. Ru n
time
Sources
Dev
elop
ers
(wor
ksp a
ce)
Check o ut and
comm
it change s
ContinuousIntegration
Deployables
QA
. R
u ntim
e
debugde
ploy
/pus
h
SNAPSHOT Repo
Release Repo
Dev Build(mvn)
rele
ase
depl
oy/p
ush
deploy/push
depoy
central
local
© ZFabrik Software KG 2010
The Maven Way
Dev
. Ru n
time
Sources
Dev
elop
ers
(wor
ksp a
ce)
Check o ut and
comm
it change s
ContinuousIntegration
Deployables
QA
. R
u ntim
e
debugde
ploy
/pus
h
SNAPSHOT Repo
Release Repo
Dev Build(mvn)
rele
ase
depl
oy/p
ush
deploy/push
depoy
central
local
Who maintains all this
infrastructure?
Who maintains all this
infrastructure?
How hard is it to keep the
dependency version vector right?
How hard is it to keep the
dependency version vector right?
How “equal” are these?How “equal” are these?
How “hard” is it to get thisup to date?
How “hard” is it to get thisup to date?
What are the sources of what you are debugging (but others change)?
What are the sources of what you are debugging (but others change)?
binary release management requires
extreme care
binary release management requires
extreme care
© ZFabrik Software KG 2010
The Maven Way
Maven is built around versioning, versioned dependency and release processes for binaries
It does not address the system consistency side of things
It is geared towards a distributed, independent community of producers of libraries
It does not care about source consistency Summary: Useful for the community, much less
interesting for solution providers.
© ZFabrik Software KG 2010
And in z2
Sources
Debug
Pull what has changedsince the last pull
Pull from workspace with prio
Dev
. R
u ntim
eQ
A.
Ru n
time
central
Dev
elop
ers
(wor
ksp a
ce)
Check o ut and
comm
it change s
Pull what has changed
since the last pull
© ZFabrik Software KG 2010
In z2
A runtime that updates itself on-demand, according to changes in source repositories
Developer runtime takes developer workspace into account with preference
The runtime is a faithful representation of the repository – no question where sources are.
Almost like a scripting environment
© ZFabrik Software KG 2010
Outlook
What could be done but we did not need to so far:
1. z2 as life-cycle tool over any Java EE server: z2 manages update checking and invalidations Instead of start/stop perform deploy/undeploy
2. Component Repository over Maven Repository Integration of existing assets
3. z2 for PHP et al Consistent dev and production environment
© ZFabrik Software KG 2010
More Information
Try it out at:
www.z2-environment.de
Contact us at: