The tension between agile and architecture

Useful definitions on software design and architecture

Peter HendriksIT Architect at Info Support B.V.peterhe@infosupport.com@PeterHendriks80blogs.infosupport.com/peterhe/

Agile and architecture: cats and dogs?

Agile and architecture are often considered cats and dogs. Many "classic" software architecture methods are considered an enemy of agile principles: often describing heavyweight, upfront documents and -decisions, and a hierarchy with architects wielding all technical decision power and responsibility.

Although there are some new "agile architecture" concepts out there, these typically only address small parts of the problem and often require significant skill to practice correctly. There is even the notion that architecture is not needed anymore when applying agile practices.

But what is "architecture" anyway?

Architecture: a concept, process and role

Like so many software development terms, "architecture" is not a very well defined thing. For starters, the term architecture is used for wildly different categories:

The architecture concept: the notion that certain aspects and design choices of a system are more important and fundamental.

The architecture process: the way architecture concerns are addressed in the way teams

The architect role: the person considered responsible for architecture

We’ll look at each category to establish how "agile friendly" architecture actually is.

A definition of architecture: the concept

"Worries about the hard stuff - whatever that organization thinks is hard“- Martin Fowler and Ralph Johnson, thought leaders in agile and design, on architecture

"Design is the structure or behavior of an system whose presence resolves or contributes to the resolution of forces on that system.“- Grady Booch, one of the fathers of modern software design, on design

"All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where

significant is measured by cost of change.“- Grady Booch, now on architecture

Architecture versus design

A design has correlated, but different goals. Design may be needed to allow communication, collaboration and complex problem solving. Since all architecture is design, these goals matter for architectural significant design decisions as well.

Design often exists in multiple levels of detail, with the lowest level being the code. High level design is often considered the architecture. Most of the time this makes sense: high level design naturally contain a lot of decisions that are hard to change after the system is built.

Considering high level design as the only architecture is risky: you might want to do too much of it, while missing decisions with a high cost of change at lower levels.

Architecture in the context of system design

Design

Design Design

Design

Design

Achievability

Future

Stakeholder needs

Environment

System

An example model of system design, relevant forces, and architecture

Performance

Features

Monitoring & controlSecurity

Operational costs

Existing systems

Common practices

Licensing deals

Corporate standards

Technology capabilities

Framework support

Team skills Project budget (time/money)

Team changes

Technology changes

Productivity

Extendable

Laws

The need for the architecture concept in agile

Architecture-as-a-concept assumes that certain parts of a system will be hard to change. It also assumes that design decisions here will be considered more significant, because of the greater short-term risk and possible long-term limitations for the evolution of the system.

It's safe to say these assumptions are still valid for systems built in an agile fashion. Common agile practices, like automated testing and short iterations, drive down the overall cost of change. In that sense, agile reduces the amount of architecture needed. But there is no silver bullet here, there will still be "hard stuff". Making wrong decisions here will still seriously hurt or kill an agile project.

Agile practices, like working in small teams, releasing early and often or being testable, often add extra demands on the system design, even at the architecture level. Also, some architectural design decisions, like applying modularity, may reduce cost of change and support other important agile values. In order for agile to be a successful approach to building a system, the architecture must support it.

Where is the problem?

Agile is about embracing change. You want to change because the old goal has become suboptimal. You want to build “the right thing”.

Architecture is about recognizing that some design decisions are expensive to change. It will be suboptimal to change those often. You want to build “the thing right”.

Architecture is a natural effect of building complex systems, and as such, also a factor for agile teams. Agile and architecture can complement each other very well, building “the right thing in the right way”.

However, there is a tension here. When we look at architecture as a process, this becomes more apparent.

Defining the architecture process

If we consider that architecture is about design decisions that have a high cost of change, then an architecture process should strive to lower the amount of these decisions, and the rate of change on these decisions once they are committed to.

As a secondary goal, the process should help to identify what changes will be more costly to make, making the software development process more predictable and improving the confidence of the team.

Lowering rates of change and long term predictions does not feel very agile. It isn't. In a way, this is where the ideals of agile meet the

boundaries of practical reality.

Why change a design decision?

Reason for change:

• The decision was wrong; the resulting system does not work

• A change in a force invalidates the decision

• A more optimal decision is found

Some countermeasures:

• Investigate unclear and changing forces

• Consult existing experience and expertise

• Early evaluation of the decision

• Delay the decision

• Add abstractions that are more stable

Considering these goals, it becomes apparent that the architecture process should involve all

disciplines in a software development team.

Investigating and negotiating stable insight is key to predict whether the design decision will last

for a longer period.

Testing design assumptions as soon as possible and fixing problems before building an entire

system on them can help immensely.

Planning the system evolution around tough choices, or adding abstractions, can postpone or

alleviate effects of change.

Deliberate vs accidental architecture

During the evolution of a system, not all design decisions with a high cost of change are deliberately made using an architecture process. This design that just happens is often called “accidental architecture”.

Accidental architecture can be a judgment- or communication problem, but also a learning effect of building the system.

We should expect that while the system is being built, we may periodically need to evaluate which design decisions matter most, and if existing decisions need adjustments.

Agile places heavy emphasis on feedback early and often. This is a big help for an architecture process. We can continuously

evaluate design decisions using the real system as it’s built. Also, we learn what matters for decisions that come up later.

Tools and practices for architecture processes

There are many established software architecture tools and practices available. These vary from complete and specialized process frameworks like TOGAF to various notations and metamodels, like UML and ArchiMate, to best practices, like design patterns and the SOLID design principles.

There are design tools, like Enterprise Architect, or code analysis tools like Structure101, that can be very useful, but require skill to use. Often, a whiteboard is used, easy for everyone to participate, without the distractions of having to operating a complex tool while thinking about a difficult problem.

Personally, I believe all these methods and tools can be applied in an agile fashion, if used in a small package form. However, they are often marketed as big and overarching things. This results in a lot of resistance in agile teams.

The role of the architect

So who is driving the architecture process? In the "classical" architecture process, an architect creates a detailed Big Design Up Front (BDUF). The architect is a senior specialist, who knows what's the best way to build the system and anticipates how design forces will behave during the lifetime of the system. He/she designs the system before the "construction" team starts, so they don’t have to wait during the period needed to create the BDUF. Then, the process aggressively limits any deviation to the original architecture.

As an industry, we have learned that this is an naïve approach. It assumes an all-knowing architect, a completely predictable future, and a team that just reads a document and then knows what to do.

Even in an agile team, this definitely seems like a candidate for a specialist role, especially for larger systems, where the amount of architecture rapidly increases and the stakes are much higher.

However, consider the architecture process we've just discussed. There is complex decision making, multi-discipline collaboration and communication, and specialized design methods and tools. Experience with both the problem domain and the technologies used to build the system are essential to effectively predict and communicate design decisions and their effects.

Is the architect part of “the team”?

Agile focuses on small, empowered teams. In most cases, the biggest challenge for an architect is collaborating with everyone involved creating the system. From a team perspective, the best way to collaborate is to be part of the team.

An architect ideally does hands-on work. Reading and writing code, for instance, is useful for a software architect to feel the effect of design decisions. It is also a great way to earn the respect of the team. Being part of the team makes it easier to do hands-on work.

In larger enterprises, the architecture may span multiple teams and systems. In this case, the architect operates on a different level, often called enterprise architecture, outside the team. Usually, this architecture is perceived different, with other main stakeholders and design goals, but still is very important for a team building a system.

For an architect, trust is a key aspect of being successful. Face-to-face communication and close collaboration, whether as formal part of the team or not, helps to build

trust, and provides a lot of learning opportunities.

Wrap-up

• Software architecture is all about design decisions with a high cost of change• This is the nature of complex systems; while agile reduces cost of

change overall, it does not eliminate the need for critical elements to stay stable

• Architecture in your process is about managing these decisions• This involves your entire team, and is needed throughout the lifetime

of the system

• Architects should not be positioned as an all-knowing oracle• But shaping a good architecture for a large enough system requires

responsibility, expertise and experience that should not be considered a common skill set

• Architecture is essential for agile success• Systems built using agile need to work, and need to last, too• Many agile practices rely on advanced architecture design