SOFTWARE AS A SERVICE – FIRST THINGS FIRSTDamon Wilder Carr, CTO agilefactor
‘A pattern language approach to empowering stakeholder communications’
Overview
An attempt to
eliminate the
Industry Fluff
while not being
overly high-
level
Concise
statements on
what you should
demand for
SaaS business
driven value
The problem with so many TLAs
Even the SaaS acronym is commonly thought to be SOA. This is not necessarily true!
The common sense bootstrap to SaaS
Which is not so common…..
A meta model view of the ‘Easy’
The ‘Blank Whiteboard’ view
SaaS Expected Results
A meta model view of the ‘hard’
The ‘Constrained yet Optimized’ Evolving State
Your Components have the GAC, what do your services have?
The Combined Dimensions
Service Repository and Governance
Common Feedback and Concerns
The problem with so many TLAs
The ‘Buzz’ Factor
and those who
like confusion
Why a lack of
shared
understanding is
now (and has
been) systemic to
Software
Engineering
Why we need to
resolve this
internally
I am using SaaS on purpose
For some, SaaS means ‘external vendors who manage their services which you integrate with yours’
For others, it means a philosophical approach in how you view the role of software strategically and architecturally across all domains (this is the view I am proposing we use, not SOA as ‘Software as a Service’ is incredibly specific if we agree it is (grin)
Our industry suffers from severe ambiguity in the core areas that require clarity
Ask 10 technologists to define Service Oriented Architecture and you’ll likely get 10 different answers
This encompasses not just SOA but its constituent parts such as ‘Domain Driven Design’ (and there are some who would argue my statement that DDD is even a constituent.
We have no ‘Webster's Dictionary’ that all have agreed to follow
Therefore we can only do this at our organization. This is the pattern language
Everyone ends up being very confused and ‘stick’
Before we can get stuck our goal is to avoid this predictable problem by proactively understanding it
The common sense bootstrap to SaaS – Not so common actually
A lesson from the
Gang of Four
A statement of
immediate goals
related to a
common
understanding
A team that shares
a vision with the
business
stakeholders using
A COMMON
language is
incredibly powerful
The largest impact for Design Patterns are not about code, UML Diagrams or technical details
What then?
These are important but have less to do with why this is so transformative to teams
It’s all about the words.
They create succinct and unambiguous language for teams to reach increased bandwidth, however most technologist think in different terms
In one phrase (or even word) you can create a definitive and concrete shared understanding of a best practice for a very tangible item, across business domains, practice areas, etc.
A Pattern Language as the Artifact of Shared Understanding
To address the
problems teams
face in
communication, we
use the term
‘pattern language’
to describe our
immediate
evolving solution
The common sense bootstrap to SaaS – Not so common actually
In technical areas like SaaS that are ‘trendy’ and often ill-defined, there are strong motivatig factors for vendors, consultants, etc. to MAKE THIS MUCH HARDER THEN IT IS
Not so long ago it was very hard, but now with the third-generation offerings fully available SaaS is not nearly as mystical.
But it is nearly impossible with people/organizations confusing everyone with overloaded definitons
If this presentation had one point, this is it:
We must create a clear and concise language and set of semantics (collectively called a Pattern Language) which attack all ambiguity and clarity in this often cloudy area
The TLA ‘SOA’ is practically useless now so I use SaaS instead. Full definitions are of course key to this presentations and requested clarifications are expected
Stakeholders in SaaS cannot execute ‘actionable’ steps due to confusion on design, development, architecture, etc. related to a clear pattern language FIRST.
OUR IMMEDIATE TO DO:
Produce a ‘Pattern Language’ (with easily accessed references typically web based) for the SaaS space (leveraging others work as well as our own, which is always evolving).
We will achieve a level of immediacy and an elimination of ambiguity in what we mean when we communicate across all levels of the enterprise (just like our services! – grin)
A meta model of the concepts
‘Blank Whiteboard’ Unconstrained Drivers
SaaS Expected Results
Interoperate cross-platform in complex
workflows with little to no effort or concern
Key SaaS Differentiation
Dramatic End-State Reduction for Software
‘Overhead’ tasks Most work becomes
strategic and benefits all systems
Empower enterprise to achieve astounding
agility to address issues, large and small (with
IT as a true strategic asset, never a cost
center)
Ability to choose third-party technology
based on ‘fitness to need’ rather then
‘constraints of exiting systems’
On occasion SaaS is
providing more then is
actually needed
(especially in a phased
approach)
We all need to be clear
exactly why all this
effort is being done.
Almost all companies
benefit but benefits are
ROI based
SaaS Expected Results
Short term ROI is impacted by additional effort, with
mid-long term ROI showing dramatic improvement
Typically the largest ROI for non-SaaS is the initial release
of a new ‘application’ if it effectively solves the business
need (in its limited scope)
With SaaS the concept of ‘Application’ is dramatically
changed, and rather then a first release always providing
the largest benefit, the ROI should increase for subsequent
release as they (more then not) will address a far broader
scope (more detail to come)
Elimination of the ‘impedance mismatch’ between the business experts and the software professionals
CRITICAL POINT: The real provider of this benefit is now almost always a precursor to an SaaS imitative. This is a core enabler of SaaS (the Domain Driven Design with supporting ‘component’ API) - more to come.
This ‘Domain Driven Design’ precursor is now a given as the two predominant SaaS APIs are now (finally) native object-orientation and component lessons (WCF from Microsoft and SCA from a Java Consortium) – We will not get much into the implementation today I suspect
Remember that success in SaaS interactions is quite different then any traditional RPC or Distributed Object Framework. Although the new APIs allowing seamless OO practices to build SaaS, remember this power can ‘empower you to fail’.
We will cover specifics on that topic in great detail in future discussions
SaaS Expected Results
In addition to the other
areas, these should be
in our thoughts as a
good measure (when we
are uncertain) about
decisions.
SaaS Expected Results
SaaS is a process, not a single end-state
The stages of evolution to ‘get to’ SaaS often provide many of the same benefits as SaaS itself
The maturity level of ‘when’ to evolve to the final stage is non-trivial and assumes a vast level of competencies across many complex areas
MEASURE: Most development – code even –becomes understandable by your most advanced business analysts (or we are probably doing something) – MUCH more to come
It is common to have a subgroup of the development team focus more on the infrastructural issues and a (typically larger) team to be far more business facing (but this is not always the case)
For example, the infrastructure team would develop a far deeper mastery of the enabling SaaS API typically when compared to the more ‘SaaS Composite Delivery’ team. This term is more accurate then ‘Application’
As all core individuals driving solutions have created a (ring a bell?) COMMON PATTERN LANGUAGE it becomes frictionless and all metrics are positively impacted
SaaS Expected Results
Probably the
least understood
of the benefits
and the most
confused
Is absolutely
vital to SaaS
success
While your systems are ‘individual’ and ‘autonomous’ as required, however they also are fully empowered to engage in collaboration with any other service
Example: A third-party CRM system which has an entrenched role for the marketing department can seamlessly participate in complex messaging workflows across separate systems (regardless of platform) via service interfaces
The finance department (which uses a Linux based internally developed system running J2EE) would like to build predictive revenue models using the new algorithms created by the recent PhD hire out of Columbia.
These require detailed demographic information that only exists in the CRM system. In addition we must extend their proprietary browser interface (now showing its age) to not perform well but not expose the user to any service invocations. We must invoke services directly from the browser using Ajax (a first for this application) to provide acceptable performance
SaaS Expected Results
All data must be real-time from ‘systems of record’
As real-time data is processed and becomes aged, it will be stored as Business Intelligence information in a SQL Server 2005 Analysis Services repository
We want to leverage existing service interfaces and implementations, with a key change: We want messaging to be asynchronous and sent via our MQ Series environment – This must leverage the same work but ‘inject’ the channel type via external configuration
Company executives will consume these as temporal graphs in their Executive Portal – SharePoint 2007 MOSS’. Also, key performance indicators which are derived from this OLAP repository are calculated for monitoring status
Here we have Asynchronous queued communication, real-time high performance messaging and coordination
A meta model of the concepts – Real World Concerns
The Combined Dimensions
Your Components have the GAC, what do your services have?
For those who succeed with SaaS, their success can quickly become their downfall.
Why? They number of services are unmanageable, undiscoverable, and dependencies break left and right. In short?
It’s precisely the same issues we had in COM, and that the GAC addresses (side by side versioning for example is a critical capability in SaaS)
Service Repository
Versioning is critical, and multiple contracts (same idea as a .NET component version which is strongly signed) running concurrently is very common
This is a logical organization (usually top down) for cataloging, finding and consuming your service assets as well as much more (to be continued)
The number of services should explode over time, as everything becomes a part in the larger whole. Without a repository you will find yourself unable to use the SaaS Paradigm (by your very success in it!)
If consumers cannot find what they need quickly and logically, the very core reasons for having services will often spin out of control.
Service Repository and Governance
“Development organizations have found they must evolve new
compliance and incentive systems to ensure their SOAs are built on
stable foundations.”
“Without governance, unfettered ad-hoc development of services
threatens to undermine the promised benefits of SOA, such as
improved productivity through service reuse and better alignment
with the business.”
“Instead, SOA initiatives may lead organizations in a backwards
direction, pouring development dollars into a black hole. “
‘Weak governance haunts SOAs’, David Longworth
http://www.looselycoupled.com/stories/2005/gov-dev0606.html
Also See:
www.looselycoupled.com/opinion/2006/stanek-gov0517.html
Service Repository and Governance
Does that previous quote sounds familiar? It sure does to me!
Due to (I am guessing) the few companies who have reached this level of maturity now, this is a subject that is amazingly glossed over far too often
What is an example? UDDI is the most common (a free service in Windows 2003) and available (for years now) right is Visual Studio.
Service Repository
This is precisely what has killed most Object Oriented reuse initiatives
In spite of this rich history of failure, many are repeating it with reckless abandon
It’s far better to ‘just get something running’ as your repository (say the UDDI Service in Windows 2003) and evolve as required, as UDDI is the backbone of almost all candidate repositories.
UDDI is now at Version 3.0 which Microsoft does not yet support
NOTE: When we have this problem we will not only be prepared, it will be a very nice indication of our success
Service Composition Strategy Guidance
Service Composition as a Rule, not an Exception
Most of the time you will combine services into composite solutions
It will matter little to you the technical details about the services (such as platform, API, etc) as your interaction will likely be purely as ‘just a set of class instances and types’
A dedicated discussion will likely follow on just this point, especially around
Composite Security Contexts, Credentials, Single Sign-On, Authentication, etc.
Inter-Service Transactional Scope
Inter-Server Workflow/Orchestration (especially now with easy to use GUI tools)
Service Composition Strategy Guidance
For a real
measure of SaaS
success, just see
how much you
are composing
rather then
building new
services.
As services are
built and your
repository grows
this should
dominate your
development
efforts
To succeed in SaaS you cannot avoid spending a lot of time composing services
What are some guiding principles around this?
Service Characteristics that Help you Decide
How much can a service be reused? If not a lot is there a better design or further decomposition to get more reuse – a constant question to ask yourself in SaaS.
What is the type of logic that the service primarily performs? Just like objects and the single responsibility rule, this loosely holds for your lowest level services (as will be briefly discussed)
Another view of reuse is ‘How could this service possible be used across other areas we are not thinking of’?
Every bit of effort becomes driven by these kind of thoughts, as your benefit soon becomes far less effort to do very hard tasks. It’s a matter of ‘I know we have most of what we need already, I need to search the repository for a few items, but we can deliver this new capability soon. In fact, this could also be leveraged by our marketing department as they have a use case defined which we will end up covering 90% of. We will extend this to cover 100% and kill 2 birds.
Hey, I just realized with these changes, this can become a service we can use in three more areas! We just freed up 20 hours of planned work!
Drivers for the ‘Three Service Composition’ categories
This work happens
during your creation of
the Domain Driven API
which is a necessary
precursor to full SaaS
(they actually happen in
parallel typically, with
shifting focuses)
Remember these all
evolve via Agile
iterations, a lot of
feedback from the users,
and quality checkpoints
on all code via tangible
items like Continuous
Integration
To succeed in SaaS you cannot avoid spending a lot of time composing services
What are some guiding principles around this?
Service Characteristics that Help you Decide
How much can a service be reused? If not a lot is there a better design or further decomposition to get more reuse – a constant question to ask yourself in SaaS.
What is the type of logic that the service primarily performs? Just like objects and the single responsibility rule, this loosely holds for your lowest level services (as will be briefly discussed)
Drivers for the ‘Three Service Composition’ categories
Want to start now on
something? Know that a
Mock Testing Framework
is necessary at some point
sooner rather then later.
Which tool? Try NMock2
from ThoughWorks. Ask
me to pair-program with
you to jump start this
powerful technique.
NOTE: In our model, this
is the box called ‘Best
Practices’ in the lower
half.
Another view of reuse is ‘How could this service possibly be used across other areas we are not thinking of’? Most effort is driven by these thoughts, as you
experience less effort to do much more.
It’s a matter of ‘I know we have most of what we need already and I need to search the repository for a few items.
We can deliver this new capability leveraging that much sooner now.
EXAMPLE: A complex cross-domain transactional service (use case defined) which we have reuse on 90% of.
The incremental 10% must be carefully considered so the net benefit is far greater then the work would otherwise be.
In other words, almost all work (at least has the potential) to be strategic, and all effort should be considered in these terms, not ‘just get it done’
Task Services
This is an oversimplification
most likely, however these often
become the first ‘shared pattern
language’ words that are used in
the beginning
These are blatant rip-offs of the
Domain Driven world we will
inhabit, and in fact, many
decisions around these items are
made there.
The areas which differ
significantly are when we need
to do fundamental
transformations due to a
necessary Entity API that simply
does not translate to a service
(almost always because it is
from a vendor, as we have the
ability to avoid this)
Task Services
In many ways these can be though of as ‘use case’ driven as your use cases will start ignoring your existing system boundaries (otherwise why do all this!)
Another way to think of this is as a ‘Controller’ however that is not recommended due to conflicts with patterns like MVC.
Just getting use cases that ignore your boundaries will allow discovery of your services. This is typically a critical early driver of service definition from Business Analysts
They are the top level of the tree, and almost always call two or more lower level services
These are the simplest form of orchestration, can serve as the transaction coordinator, and the injector of specific capabilities into lower level services via configurations in the repository or elsewhere
]This is almost always decomposed much further using whatever makes sense for your enterprise
This also tends to link back to how you model your repository
In short? These are very important (again depending on the pattern language and various design decisions)
Entity Services
Now is a good time
to point out that we
are not addressing
ANYTHING to do
with the user
interface.
It is assumed that we
can support just
about anything
thrown at us.
Entity Services
This is NOT CRUD! Even without SaaS our work here is fundamentally object oriented. SaaS is MOSTLY object oriented but not to the same extent.
Why? Services are very clear in their separation of Data and Process definition, unlike a class. However at least this work is done via Interfaces instead of the previous requirement to build separate XML Schema definitions for contracts (more to come)
We are not simply wrapping rows in a table (with some exceptions where they happen to match – probably due to over normalization)
By focusing on working out our Domain Driven Entity API, we can now see why it is a precursor. Almost without exception this will be leveraged by Entity Services.
Entity Services
Almost without exception today, the ‘Best Practice’ used from the service layer up to the UI is the Model View Presenter Pattern (to be discussed in detail) with Dependency Injection and Inversion of Control (again, we will cover all that later)
Why not skip the Domain Driven API?
It turns out that solving those problems overlap extensively with the same problems (except cross-platform concerns for the most part)
A Domain API is a ‘Best Practice’ for the code services execute to fulfill their contracts
Often Entity Services are similar to the Entity Classes (with a transformation into the Service Messaging Requirements)
Most organizations will benefit immediately from the Domain API so this is yet another way to
Create the CRITIAL feedback loop
Create the shared language between the Business and I.T.l
Seamless fit the process of ‘delivery highly visible value in iterations as often as possible’.
We gain immense value from the feedback of a ‘Pre-SaaS’ Entity design that will directly impact and improve SaaS work.
Entity Services
We will likely focus
extensive energy
on the Domain
Driven API which
will driven (mostly)
the Entity Services
Task Services come
when we have a
base, and utility
services are often
dynamically
injected
For example: If you do not need interoperability (at least not in the short term but likely will add it later) in a use case but do need maximum performance due to real-time concerns, you will want the option to leverage this Domain API.
Indeed this is THE SAME API that your services use, they are just a layer wrapping them (basically)
In the real-world, no company always invokes EVERY ASSET via SaaS or via a Domain API. It’s always some point on a continuum. Rather then ignore this, we embrace it!
Utility Services
All the stuff you know you need to do
Logging, Security, and any cross-cutting infrastructural concern would go here
It is assumed we will leverage the best practice of an Inversion of Control container using Dependency Injection
We have many ‘cross-cutting’ areas which all services will need
Security, Authentication, etc…
Logging
Audit Trail
Services dedicated to these areas can be called ‘Utility’ services or ‘Common’ services
BEST PRACTICE: As will be demonstrated, we recommend Injecting these dependencies using an Inversion of Control contain (like Castle Windsor)
This is a huge win as this is a standard practice in any Domain API development and even Object Oriented development
It is only recently with the advent of true OO APIs for SaaS that we can use the best practices learned in that space and apply them to the SaaS space. This is no accident, as vendors needed to lower the bar and leverage the expertise developers already had and let them apply it to SaaS.
This all used to be so much harder and less fun
Common Reactions….
Can this all really be done?
In short? ABSOLUTELY! Once things are as tangible as creating a class, Interface, etc. that our in-line with our standards and Pattern Language, it will likely become second nature.
INCREMENTAL EVOLUTION
many checkpoints that will reap rewards almost immediately
CHANGING A CULTURE IS FAR HARDER THEN CHANGING TECHNOLOGY
Although everyone is different there is now a clear evolution path set which very clear stages and checkpoints
It’s rare the team that succeeds in the face of overwhelming information overload
How are services ‘orchestrated’ by some coordinating entity? How can we involve the business analysts in defining these orchestrations?
How are atomic (transactional) operations handled across fundamentally incompatible systems?
Common Reactions….
How can a service possibly work with no change across ‘run-time’ injected details like the actual method of sending? In short? Our pattern language and design standards which will dictate
how to accomplish this with a little planning and the definition of common practices our services will share
After we have made significant progress on the Domain Driven API Model (a step before we ‘jump in the deep end’ with SaaS), how do we evolve this to a SaaS model? It just so happens you are now ‘killing two birds’ by evolving this way.
Again my previous comments discuss the possible downsides which we address via our standards and pattern language.
Let’s say we reach a latter stage milestone such as completing work on our Domain API and we find we really do not need the incremental next steps that SaaS provides? That is more common then most think. It’s a business decision, not a
technical one typically.
Common Reactions….
What are the new skills as a developer I must learn to be effective in this area? Now? Surprisingly few… Not long ago the burden was rather daunting in
many cases (WSE 3.0, XML Schemas, etc.)
Exactly how are the messages you describe so different from the API calls we make today or the distributed component calls I have used before? This is one of the most important areas we will drill down on
Can I really do work that is almost 100% business driven and ignore so much of the nasty details of services? This used to be far more of the work then any of the business logic! You’ll see (grin)
How will my IDE (Visual Studio for example) help me work in this new paradigm? Will I have to use other tools as well? How will I even know I am building solutions that are in line with all this?
THANK YOU!