refactoring, emergent design & evolutionary architecture

Refactoring, Emergent Design &

Evolutionary Architecture

by Brad AppletonCreated July 2009

(last updated, October 2010)

ingRefor

Refingact

oracting

ingactor

actingRef

orRefing

orRef

ingact

Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton 2

Presenter InformationBrad Appleton

25+ years in industry – last 20 w/large Telecom.Agile Leader, Coach & Manager

Scrum, XP, SAFe, LeSS, DADALM/SCM/DevOps Solution Architect

Git, JIRA, Confluence, TFS, RTC, Jenkins, CI/CDLeading adoption & scaling of Agile development for

large organization since 2000.Co-author: Software Configuration Management

Patterns (Addison-Wesley 2002), Agile CM Environments (column in CM Journal), Reviewer for The Agile Journal

Refactoring, Emergent Design & Evolutionary ArchitectureBrad Appleton

Outline• Part I: Refactoring for Agility

Refactoring Overview How to Refactor Refactoring to Patterns & Principles

• Part II: Refactoring @ Scale Emergent Design Technical Debt Restructuring Evolutionary Architecture Incremental Design

• Part III: Special Topics & Further Info Refactoring Legacy Code Refactoring Databases & Web Content Refactoring Test Code Additional Resources (Books & Articles)

3

Part IRefactoring for Agility

ingRefor

Refingact

oracting

ingactor

actingRef

orRefing

orRef

ingact


Outline for Part I• Overview of Refactoring

Refactoring Defined Refactoring Myths & Misconceptions The Practice of Refactoring Why Refactor?

• How to Refactor Simple Code & Clean Code The Refactoring Cycle Code Smells & Refactorings When to Refactor?

• Refactoring to Patterns & Principles Design Attributes / Code Qualities Design Principles & Design Patterns Other Acronyms of Simple/Agile Design

• Summary & Resources for Part I

5


Overview of Refactoring• Identifies design-maintenance issues (“code smells”)

that typically represent violations of known principles of good design.

• Incrementally and iteratively applies a set of design improvement techniques (“refactorings”).

• The goal is to minimize complexity & duplication in order to maximize simplicity & ease-of-change.

• Encourages the “right” design details to emerge “just-in-time” with minimal guesswork/rework.

• Scaling-up includes the use of periodic restructuring, initial & incremental design (“just enough”), and evolutionary architecture.

6


Refactoring DefinedRefactoring (noun): a change made to the internal

structure of software to make it easier to understand and cheaper to modify without changing its observable behavior.

Refactoring (verb): to restructure software by applying a series of refactorings without changing its observable behavior.

— Martin Fowler, Refactoring: Improving the Design of Existing Code

“A change to the system that leaves its behavior unchanged, but enhances some nonfunctional quality – simplicity, flexibility, understandability, performance.”

— Kent Beck, Extreme Programming Explained

7

http://www.informit.com/articles/article.aspx?p=360842

http://www.informit.com/title/0321278658


Refactoring DistilledRefactoring is a disciplined technique for the

restructuring of an existing body of code, altering its internal structure without changing its external behavior. Its heart is a series of small behavior preserving

transformations. Each transformation (called a ‘refactoring’ ) does only a

little, but a sequence of transformations can produce a significant restructuring.

Since each refactoring is small, it's less likely to go wrong.

The system is kept fully working after each small refactoring, reducing the chances that a system can get seriously broken during the restructuring.

Source: Martin Fowler, www.refactoring.com

8

http://www.refactoring.com/


Refactoring Myths

9


Refactoring is NOT …“Rework” – redesigning things that could, and should, have been

designed correctly in the first place.

Design “gold-plating” – work that adds no business value, and merely serves to stroke the egos of perfectionists who are out of touch with business reality.

Miscellaneous code “tidying” – the kind that is “nice to have,” but should only happen when the team has some slack-time, and is a luxury we can do without, without any serious consequences.

A license to “hack” – avoiding any and all initial design & analysis and instead jumping straight to coding with no “real” design.

Reengineering – large-scale restructuring that requires a concerted effort over the course of several weeks/months to re-write or re-architect significant parts of the system.

Derived from: Patrick Wilson, Continuous Refactoring and the Cost of Decay

10

http://patrickwilsonwelsh.com/?p=10


Refactoring IS …A systematic approach to source-code “hygiene” that minimizes

the chances of introducing bugs Improving the design of the code after it has been writtenA behavior-preserving transformation of source-code structureThe process of simplifying & consolidating a work-product by

making several, small, successive revisions focused on: preserving correctness, removing redundancy, revealing thoughts & intentions, and … improving clarity & conciseness.

A disciplined way of making changes while exposing the project to significantly less risk.

An effective means to address the economic reality of software growth/complexity by reducing & amortizing its cost throughout the daily business of development & maintenance activities.

11


The Practice of RefactoringThe process of refactoring involves:

the removal of duplication & dependencies the simplification of complex logic/structure, and the clarification of unclear code & design intentions

Each step is small and simple, even simplistic. You move a field from one class to another, pull some code out of

a method to make into its own method, and push some code up or down a hierarchy.

To refactor safely, you must run tests [preferably automated] to ensure your changes didn't break anything. You will have more confidence to refactor and make other design

changes if you can quickly run automated tests.Refactoring in small steps helps prevent introducing defects.

Most refactorings take only seconds or minutes to perform. Even large restructurings are implemented in small steps.

Sources: Martin Fowler, Joshua Kerievsky

12


Why Refactor?Lots of reasons … Make it easier to add new code Improve the design of existing code Increase readability/understanding of code Make coding less annoying/frustrating Prevent “design decay” Clean-up/Simplify messy code Find & fix design-principle violations Reduce debugging time Incorporate learning about the application

13


How to RefactorThe overarching goal of refactoring is to:• minimize complexity & duplication in order to • maximize simplicity & ease-of-change.

When we refactor we are striving to produce code that is clean and simple.

Clean Code follows “Uncle” Bob Martin’s The Boy Scout Rule: When touching any code, leave the codebase cleaner

than you found it!Simple Code follows Kent Beck’s Four Rules of

Simple Code …

14


Simple Code

15


The Rules of Simple CodeSimple Code …1. Passes all the tests2. Contains no duplication3. Clearly reveals & expresses the developer’s

intent4. Minimizes the number and size of entities

classes/modules methods/subroutines

Note: these are in order of highest priority first!

16


Clean Code

17


The Rules of Clean CodeKeeping the Code “Clean” means ...

We must continuously simplify our code (following the rules of simple code)

We must continuously clean-up the code, and not tolerate messes, spills, or “smells” in our code.

We must not regress into bad habits.—Joshua Kerievsky, Introduction to Refactoring to Patterns

• Rules for Clean Code come from the book of the same name by Robert C. Martin (a.k.a. “Uncle Bob”) et.al. Clean Code: A Handbook of Agile Software Craftsmanship

• Also see Uncle Bob’s Clean Code Tip of the Week series of articles online

18




http://www.informit.com/promotions/promotion.aspx?promo=136914


The Refactoring Cycle

19


The Refactoring Cycle

Check for “Code smells”

RefactorRe-test

After newly added/changed code passes its tests, you:1. Check it for “Code smells” 2. When you find one, apply

the specific refactoring that resolves it!

3. Re-run automated tests to ensure nothing broke (fix any “breakage” immediately)

4. Repeat until the affected code is smell-free!

20


Code Smells

21


Code Smells“Any symptom in the source code of a computer program

that indicates something may be wrong. It generally indicates that the code should be refactored or the overall design should be reexamined.”

• A Code Smell is a hint that something might be wrong, not a certainty.

• Major approaches to programming “hygiene”: Pragmatic: code smells should be considered on a case by case

basis Purist: all code smells should be avoided, no exceptions Possible bad practice to a pragmatist, but a sure sign of bad practice

to a purist

• Beware of Code Stenches

22


Code Smell Examples• Duplicate code or Duplicated method - a code, method,

function, or procedure that is very similar to another• Large method or class - has grown too large• Lazy class - a class that does too little• Feature envy - a class that uses methods of another class

excessively• Inappropriate intimacy - a class that has dependencies on

implementation details of another class• Refused bequest - a class that overrides a method of a base

class in such a way that the contract of the base class is not honored by derived class

• Contrived Complexity - forced usage of overly complicated design patterns where simpler design would suffice

• Code not actually ever used

23


Categories of RefactoringsSmall Refactorings

(De)Composing methods Moving features between objects Organizing data Simplifying conditional expressions Dealing with generalization Simplifying method calls

Larger Refactorings/Restructurings Tease apart inheritance Extract hierarchy Convert procedural design to objects Separate domain from presentation

Each category contains as many as a dozen or more refactorings, most of which are catalogued at http://refactoring.com/catalog/

24

http://refactoring.com/catalog/


RefactoringsSome refactorings from real projects

Add a Parameter Extract Method Inline Method Inline Class Extract Interface Move Field Move Method Parameterize Method Rename Method Method Parameter to Constructor Parameter

Composite Refactorings (often to Design Patterns)

See http://refactoring.com/catalog/ for an up-to-date list (and the “Refactoring to Patterns” catalog too)

25

http://refactoring.com/catalog/

http://www.industriallogic.com/xp/refactoring/catalog.html


What to do if …?I spot a “smell” that is not already known or

catalogued? Convince yourself it really is a “smell” by trying to

identify its ill-effects and/or the design principle it appears to violate

There is no specific known/catalogued “refactoring” for what I think I need? Check again, just to be sure you didn’t miss it Consult a co-worker Use known design principles and/or

software modifiability tactics to guide your judgment Check for known design patterns to resolve that

specific problem for your particular scenario

26


When to Refactor? While adding functionality While fixing a bug While reviewing code After coding the same/similar thing for the

third time (to “factor out” the duplication)A.k.a., The Rule of Three: 3 strikes and you refactor.

After the third time you deferred refactoring a change, for any reason [The Rule of Three, again]

Before the end of the iteration if you haven’t been following The Rule of Three

27


Refactoring Continually“It's best to refactor continuously (as a normal

part of each coding activity) rather than in phases or increments. When you come across code that is in need of improvement,

improve it! On the other hand, if you must finish a feature before a demo

scheduled for tomorrow, finish the feature and refactor later.

Business is well served by continuous refactoring, yet refactoring must coexist harmoniously with business priorities.” ―Joshua Kerievsky, Introduction to Refactoring to Patterns

“In my view, refactoring is not an activity you set aside time to do separately from implementation activities. Refactoring is something you do all the time in little bursts.” ―Martin Fowler, Refactoring: Improving the Design of Existing Code

28





When NOT to Refactor? When the build is broken or tests don’t pass When it would compromise meeting an

impending deadline or commitment When the code in question really just needs to

be re-written “from scratch” When it would modify code/interfaces that could

significantly impact/break other work: Published/public interfaces and protocols Database schemas/tables/operations

Sometimes we must defer refactoring for later and/or plan for subsequent restructuring

29


Refactoring to Patterns & Principles

Software Design Principles & Design Patterns are the underlying foundation for Refactoring:

Code smells (a.k.a “code pathologies”) Signal a possible violation of design principles Suggest which refactoring may be needed

Refactorings Correct a design principle violation (at least partially) Converge toward common design patterns

Design Patterns Reconcile forces among conflicting design concerns Restore balance between competing design principles

Design Principles Lead us to attain desired design qualities/attributes

30


Design Attributes/Code Qualities

31


Design Attributes/Code Qualities

Qualities of Highly Maintainable Software: Loose Coupling & High Cohesion Hierarchy (Structural Decomposition) Abstraction, Encapsulation & Modularity Sufficiency, Parsimony and Primitiveness Readability Testability Modifiability Serviceability

32

http://www.pragmaticprogrammer.com/ppllc/papers/1998_01.html

http://www.netobjectives.com/resources/downloads/Encapsulation_1stPrinciple_OODesign.pdf

http://www.faqs.org/docs/artu/modularitychapter.html


Design Principles

33


Design Principles: SOLID, SoC, DRY, Shy

The SOLID Principles of Object-Oriented Design:

SRP : The Single Responsibility PrincipleOCP : The Open-Closed PrincipleLSP : The Liskov Substitution PrincipleISP : The Interface Segregation PrincipleDIP : The Dependency Inversion Principle

The SoC Principle: Separation of Concerns — separate interface from implementation, policy from mechanism, behavior from construction, commands from queries, levels of abstraction, ...

The DRY Principle: Don’t Repeat Yourself (Eliminate Duplication), Single Point of Truth (SPOT)

The “Structure-Shy” Principle: (“Tell, Don’t Ask!”), The Law of Demeter, Principle of Least Assumed Knowledge

34


Other Acronyms of Simple/Agile Design

OAOO – Say Things Once And Only Once (restatement of the DRY principle)

DTSTTCPW – Do The Simplest Thing That Could Possibly Work! (restatement of the KISS principle)

YAGNI – You Aren’t Gonna Need It!The LRM Principle: Defer Commitment of

Irreversible Decisions to the Last Responsible Moment!

BDUF – Big Design Up-Front! (vs. JEDI)JEDI – Just Enough Design Initially/In-front!DDD – Domain-Driven Design

35


Design Patterns

36


Design PatternsA Design Pattern is …• A recurring solution to a common problem in a given context and

system of forces.• A named "nugget" of instructive insight, conveying the essence of a

proven solution to a recurring problem in a given context amidst competing concerns.

• A successfully recurring "best practice" that has proven itself in the "trenches."

• A literary format for capturing the wisdom and experience of expert designers, and communicating it to novices.

Design patterns …• Establish a common terminology and professional vocabulary for

discussing recurring design problems & their reusable solutions• Raise the “level of thinking” in our communication and

understanding about software design• Are context-specific applications of general design principles

37


Design Patterns ExamplesExamples of well-known Design Patterns:• Creational: Abstract Factory, Builder, Factory Method,

Prototype, Singleton• Structural: Adapter, Bridge, Composite, Decorator, Façade,

Flyweight, Proxy• Behavioral: Chain of Responsibility, Command, Interpreter,

Iterator, Mediator, Memento, Observer, State, Strategy, Template Method, Visitor

Patterns & pattern catalogs are available for:• Concurrent, Parallel & Distributed Systems, Network Architecture,

User-Interaction Design, Secure Programming, Database Access, Real-time/Embedded, Information Design, Software Product-Lines, J2EE (and core J2EE), dotNET, EA, SOA, and Many others

38

http://www.cs.wustl.edu/~schmidt/patterns-ace.html


http://en.wikipedia.org/wiki/Interaction_design_pattern

http://www.securitypatterns.org/patterns.html


http://www.eventhelix.com/RealtimeMantra/PatternCatalog/

http://interface.fh-potsdam.de/infodesignpatterns/patterns.php

http://www.sei.cmu.edu/productlines/spl_patterns.html

http://java.sun.com/blueprints/patterns/

http://www.corej2eepatterns.com/Patterns2ndEd/

http://www.designpatternsfor.net/

http://martinfowler.com/eaaCatalog/

http://www.soapatterns.org/

http://hillside.net/patterns/onlinepatterncatalog.htm


Summary: Refactoring for Agility Successively applies small behavior-

preserving transformations to eliminate code smells

Based on proven design principles and patterns for achieving maintainability & modifiability

Good automated testing is a prerequisite

Refactoring is not rewriting, rework or restructuring

With refactoring, we continuously invest nominal effort to reduce the risk & cycle-time of changes The goal is to minimize complexity & duplication in

order to maximize simplicity & ease-of-change.

39


Summary: Refactoring for Agility

When practiced in a highly disciplined manner, refactoring promotes:

Sufficient functionality Simple & clean code Supple design Serviceable software Sustainable team velocity

40


Resources: Code Smells• Introduction to Code Smells and Refactoring from TestingEducation.org • Code Smells and Refactoring to Patterns, by Josh Kerievsky • Refactoring and Code Smells • The original WikiWiki CodeSmells page• Martin Fowler's Code Smells page • A Catalogue of Bad Smells in Code, by Martin Fowler and Kent Beck • A Taxonomy of Bad Code Smells • CodingHorror.com's list of Code Smells • Smells to Refactorings Quick Reference, by Josh Kerievsky • Gene Garcia's mapping of CodeSmells to Refactorings • Using Static Analysis Tools to Identify Code Smells, IBM

developerWorks article by Paul Duvall

41

http://www.testingeducation.org/pt/Refactoring-smells.pdf

http://www.testingeducation.org/

http://www.cs.siue.edu/wwhite/CS325/CourseNotes/07_Kerievsky0111.ppt

http://www.industriallogic.com/


http://wiki.swelab.info/selabwiki/images/4/49/20070523.ppt

http://c2.com/xp/CodeSmell.html





http://martinfowler.com/bliki/CodeSmell.html

http://sis36.berkeley.edu/projects/streek/agile/bad-smells-in-code.html

http://www.soberit.hut.fi/mmantyla/badcodesmellstaxonomy.htm

http://www.codinghorror.com/blog/archives/000589.html

http://www.codinghorror.com/blog/archives/000589.html

http://www.industriallogic.com/papers/smellstorefactorings.pdf





http://wiki.java.net/bin/view/People/SmellsToRefactorings




http://www.ibm.com/developerworks/library/j-ap07088/

http://www.testearly.com/


Resources: Design Principles• S.O.L.I.D. Principles e-Book (free)

http://www.lostechies.com/content/pablo_ebook.aspx • An O-O Primer, http://www.rgoarchitects.com/Files/ooprimer.pdf• The Principles of Object-Oriented Design:

http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod • Design Principles and Patterns,

http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf and http://www.oodesign.com/

• The Art of SoC (Separation of Concerns), http://www.ctrl-shift-b.com/2008/01/art-of-separation-of-concerns.html

• Composed Method & Single-Level of Abstraction, http://www.ibm.com/developerworks/java/library/j-eaed4.html

• Tell, Don’t Ask, http://www.pragprog.com/articles/tell-dont-ask• O-O in One Sentence: Keep it Shy, DRY, and Tell the Other Guy!

http://media.pragprog.com/articles/may_04_oo1.pdf

42

http://www.lostechies.com/content/pablo_ebook.aspx

http://www.rgoarchitects.com/Files/ooprimer.pdf

http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod

http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf

http://www.oodesign.com/

http://www.ctrl-shift-b.com/2008/01/art-of-separation-of-concerns.html

http://www.ibm.com/developerworks/java/library/j-eaed4.html

http://www.pragprog.com/articles/tell-dont-ask

http://media.pragprog.com/articles/may_04_oo1.pdf


Resources: Design PatternsOnline Resources:• Wikipedia on Design Patterns • The Patterns Home page: hillside.net/patterns• Design Patterns Reference and Patterns Tutorials• Patterns & Software: Essential Concepts & Terminology

Books:• Design Patterns, by the “Gang of Four” (the book that started it all)• POSA Series of books on Pattern-Oriented Software Architecture• Head First Design Patterns, from the O’Reilly Head First Series• Refactoring to Patterns; by Joshua Kerievsky• Agile Software Development, Principles, Patterns, and Practices

by Robert C. Martin• Implementation Patterns; by Kent Beck • Design Patterns Explained (2ed); by Alan Shalloway, James Trott

43

http://en.wikipedia.org/wiki/Design_pattern

http://www.hillside.net/patterns

http://www.oodesign.com/

http://www.cs.wustl.edu/~schmidt/tutorials-patterns.html

http://docs.bradapp.net/patterns-intro.html


http://www.cse.wustl.edu/~schmidt/POSA/

http://oreilly.com/catalog/9780596007126/

http://oreilly.com/store/series/headfirst.html






Resources: Other Agile Design Slogans• XP Simplicity Rules, http://c2.com/cgi/wiki?XpSimplicityRules • Essential XP: Simple Design,

http://www.xprogramming.com/xpmag/expEmergentDesign.htm • DTSTTCPW -

http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork • OAOO - http://c2.com/cgi/wiki?OnceAndOnlyOnce • YAGNI - http://en.wikipedia.org/wiki/You_Ain%27t_Gonna_Need_It • LRM - http://stevesmithblog.com/blog/delaying-decisions/,

http://vig.pearsoned.co.uk/samplechapter/0321150783.pdf • BDUF - http://en.wikipedia.org/wiki/Big_Design_Up_Front • JEDI - http://www.featuredrivendevelopment.com/node/507 • DDD - http://www.domaindrivendesign.org/

44

http://c2.com/cgi/wiki?XpSimplicityRules

http://www.xprogramming.com/xpmag/expEmergentDesign.htm

http://c2.com/cgi/wiki?DoTheSimplestThingThatCouldPossiblyWork

http://c2.com/cgi/wiki?OnceAndOnlyOnce

http://en.wikipedia.org/wiki/You_Ain't_Gonna_Need_It

http://stevesmithblog.com/blog/delaying-decisions/

http://vig.pearsoned.co.uk/samplechapter/0321150783.pdf

http://en.wikipedia.org/wiki/Big_Design_Up_Front

http://www.featuredrivendevelopment.com/node/507

http://www.domaindrivendesign.org/

Part IIRefactoring @ Scale

ingRefor

Refingact

oracting

ingactor

actingRef

orRefing

orRef

ingact


Outline for Part II• Scaling-up!• Emergent Design• Technical Debt

Technical Debt – Tips & Truths Restructuring Technical Debt

• Restructuring Refactoring vs. Restructuring Restructure Periodically Architecture Smells & Modifiability Tactics When to Plan for Restructuring?

• Evolutionary Architecture Incremental Design JEDI (Just Enough Design Initially) Supple Design & Domain-Driven Design

• Resources for Part II

46


Scaling-Up


Scaling-UpTo scale refactoring for larger projects, additional

techniques & concepts must be added to the mix. Note that this is “in addition to” (not “instead of”)

RefactoringIn-the-Small

Refactoring @ Scale(adds the following)

Small, Fast & Frequent Refactorings

Larger, Periodic & Planned Restructurings

Emergent Design Incremental Design & Evolutionary Architecture

Deferred Refactoring Restructuring & Technical DebtCode Smells Architecture SmellsDesign Principles & Patterns

Software Modifiability Tactics

Simple/Clean Code Supple/Domain-Driven Design


Emergent Design

49


Emergent DesignEmergent Design is a fancy name for the resulting

design that “emerges” from the synergy of combining Refactoring together with TDD, Continuous Integration and Automated Testing.“As a result, you’ll find the balance of work changes. Design, rather than occurring all up front, occurs continuously

during development. The cumulative effect of these small changes can radically

improve the design. You learn from building the system how to improve the design. The resulting interaction leads to software with a design that stays

good as development continues. It is the exact reverse of the normal notion of software decay.”

― Martin Fowler, Refactoring: Improving the Design of Existing Code

A lack/lapse of discipline in any of these technical practices will result in the accrual of technical debt!

50



Technical Debt

51


Technical Debt [a.k.a. Design Debt]

Technical debt occurs when our software becomes difficult or risky to change, and takes increasingly more time and effort to evolve.

• It represents the accumulated amount/cost of rework that will be necessary to correct and/or recover from the deviation between:

the current design of the system versus … a design that is minimally complex yet sufficiently complete to

ensure correctness & consistency for timely delivery.• This effort grows more than linearly over time as a system

becomes bigger and more complex.• The economic impact of technical debt is directly related

to the cost of complexity and its resulting “friction” against the velocity of the development team.

52


Technical Debt – Real CostsThe New Push to Measure Software’s True Cost,

from ComputerWorld.com, October 2010:• The idea that software acquires a "technical debt" that is paid

in real dollars is getting new attention and research: The National Science Foundation approved a $465,000

research grant last year on technical debt Gartner Research just released its study on the subject of

"IT debt“, and puts the current IT debt bill at $500 billion worldwide, and says that it will double in five years to $1 trillion.

A company that makes software quality tools, Cast, just released a study gleaned from customer software evaluations that puts the cost of technical debt at $2.82 per line of code. For the average-size software application of 374,000 lines of code, this amounts to just over $1 million in technical debt.

53

http://www.computerworld.com/s/article/9190198/The_new_push_to_measure_software_s_true_cost?taxonomyId=11

http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0916699

http://www.gartner.com/it/page.jsp?id=1439513

http://www.computerworld.com/s/article/352022/Gartner_Warns_of_App_Maintenance_Debt_?intsrc=print_latest

http://www.castsoftware.com/Resources/Materials/News/PR_092810.aspx


Technical Debt – Tips & Truths• Incurring technical debt means your velocity slows

down and you will deliver less value• The cost of getting out-of-debt is compounded over

time: the longer you wait, the faster it grows • If you plan to incur technical debt, the persons

responsible must have a workable plan to pay it off!• “Interest only” payments won’t improve things• Pay early, pay often, and pay-as-you-go. (The only

other options are bankruptcy or death.) • Remember: Those with the worst debt problems often

have the most difficulty imagining a life without borrowing!derived from http://agileinaflash.blogspot.com/2009/02/technical-debt.html

54

http://agileinaflash.blogspot.com/2009/02/technical-debt.html


Restructuring Technical DebtIf we accrue a non-trivial amount of technical debt, we

can’t simply “refactor” it away. Paying it off typically requires restructuring efforts (or even

reengineering) that must be planned. Iteration plans must accommodate specific tasks for these

restructuring efforts (or even be dedicated to restructuring). Ignoring it, or deferring it for very long is not a viable option!

“All successful software gets changed. So if we think we’re working on code that will be successful … we need to keep it easy to change.Anything that makes code difficult to change is technical debt. Just like any other debt, the cost of paying off technical debt gets more and more expensive over time. …Technical debt drives the total cost of software ownership relentlessly higher … eventually we will have to pay it off or the system will go bankrupt.”

―Mary Poppendieck, Leading Lean Software Development 55

http://www.poppendieck.com/llsd.htm


Restructuring

56


Overview of Restructuring• Identifies higher-level issues (“architecture smells”)

that typically represent violations of known principles of good software architecture & design.

• Periodically applies larger-scale “refactorings” and/or many small refactorings that were previously deferred.

• The goal is to “pay down technical debt” in order to limit the increasing costs of accumulated complexity.

• Typically requires a concerted effort that must be separately planned.

• Uses not only design patterns/principles, but also architectural patterns/principles, as well as software modifiability tactics.

57


Refactoring vs. RestructuringThe following are not valid examples of refactoring:

● The system being broken (unbuildable) for a couple of days while someone is “refactoring”

● Reworking the structure of an entire document or web site/portal

● A concerted effort by one or more people over several days/weeks to rework and consolidate the code/design structure

All of the above are restructuring:Restructuring is any rearrangement of parts of a whole. It's a

very general term that doesn't imply any particular way of doing the restructuring.

Refactoring is a very specific technique, founded on using small behavior-preserving transformations (themselves called refactorings).

―Martin Fowler, Refactoring Malapropism

58

http://martinfowler.com/bliki/RefactoringMalapropism.html


Restructure Periodically• Restructuring is often associated with absent or

neglectful refactoring and/or design.• But … Any large software project spanning multiple

teams eventually needs restructuring. Even in the presence expert-level architecture, design &

continuous refactoring This is just a reality of software evolution/entropy

• Therefore … Large software projects should assume that periodic restructuring will be necessary, and should plan accordingly to: Clean-up accumulated code-smells and apply numerous

refactorings that were deferred but are now sorely needed, Address architecture smells by applying restructurings, patterns

and modifiability tactics that have broader impact.

59


Architecture Smells

60


Architecture SmellsSmells in Dependency Graphs•Obsolete Classes•Tree-like, Static Cycles•Visibility of Dependency Graphs

Smells in Inheritance Hierarchies•Type Queries, List-like Hierarchy•No Subclass Method Redefinition•No Polymorphic Assignments•Parallel or Too Deep Hierarchy

Smells in Packages•Unused Packages•Cyclic Package Dependencies•Packages Too Small / Large•Package Hierarchies Unbalanced•Package Hierarchies Not Clearly Named

Smells in Subsystems•No Subsystems•Cycles between Subsystems•Subsystem Too Large / Small•Subsystem-API Bypassed•Subsystem-API Too Large•Overgeneralization

Smells in Layers•No Layers, Too Many Layers•Strict Layers Violated•Upward References Between Layers (Cycles between Layers)

• Inheritance between Protocol-Oriented Layers

•References between Vertically Separated (non-adjacent) Layers

Source: Refactoring in Large Software Projects, by Martin Lippert & Stefan Roock

61

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0470858923.html


Software Modifiability Tactics

62


Software Modifiability TacticsLocalize Changes (increase cohesion)

Maintain Semantic Coherence Anticipate Expected [types of] Changes Generalize the Module Limit Possible Options Abstract Common Services

Prevent Ripple Effects (reduce coupling) Hide Information Maintain Existing Interfaces Restrict Communication Paths Use an Intermediary

Defer Binding-time (defer decision-making) Run-time Registration Configuration Files Polymorphism/Delegation Component Replacement Adhere to Defined Protocols Source: SEI Technical Report CMU/SEI-2007-TR-002

63

http://www.sei.cmu.edu/publications/documents/07.reports/07tr002.html


When to Consider/Plan Restructuring?

Consider whether restructuring effort is needed …• During Planning (of course)

Release planning / Sprint planning Of “special” iterations (e.g., “spikes”, architecture, etc.) Of stabilizing (“hardening”) iterations

• During Retrospectives Iteration retrospectives Inter-team (“joint”) retrospectives [Scrum-of-Scrums level]

• After Reviews After a Sprint review After a Technical/Release/Program review

• After three iterations without any restructuring

64


Evolutionary Architecture

65


Evolutionary ArchitectureSoftware Architecture concerns infrastructure elements that must exist before you can begin execution. Architecture is about things that are hard to change later, it is

difficult to allow an architecture to emerge. For large projects, this includes high-level organization of the system

into functionality/elements that will be allocated to separate teams.However, just because we can't allow architecture to emerge doesn't mean that it can't evolve. If we create an initial, flexible architecture and take special care to

not create an irreversible decision, then we can allow it to evolve over time as new concerns appear.

— Neal Ford, Evolutionary Architecture and Emergent Design

Key techniques of Evolutionary Architecture include: Deferring Irreversible Decisions to the “Last Responsible Moment”

(LRM Principle) Architectural “Spike” (a.k.a. Architectural “Deep Dive”) Architecture Iteration and/or Spike Iteration

66

http://www.ibm.com/developerworks/java/library/j-eaed1/index.html?S_TACT=105AGX02&S_CMP=EDU


Incremental DesignDesign evolves incrementally, iteration by iteration,

based on current business priorities and discovered technical limitations.“Invest in the design only what is needed to comfortably support the current

iteration.” —Kent Beck & Cynthia Andres

Incremental Design …• Does not prohibit thinking about higher-level design.• Does encourage planning in detail only what will be

constructed soon.• Focuses on “Just Enough, Just-In-Time” :

Specifying too much detail too soon causes more rework later. But doing less now and saving the rest for later should not require

significantly more work later than it would today. We must do Just Enough Design Initially to attain the right balance

of anticipation and adaptation.

67


Just Enough Design Initially (JEDI)

68


Just Enough Design Initially (JEDI)Initial design (before coding) is still necessary.

This use of “JEDI” was coined by Stephen Palmer as part of Feature-Driven Development (FDD)

Basic rule of thumb to tell when “JEDI” is achieved: At iteration-scope, when, after one pass through the iteration

backlog, modeling in small groups does not produce any new classes or associations of real significance.

At task/TDD scope, when we have defined enough structure & interface(s) to know specifically what code to write/test, precisely where to write it, and exactly how to invoke it.

Techniques of “the JEDI way” include: Collaborative Domain Modeling and Color Modeling [from FDD] Supple Design techniques & patterns Domain-Driven Design (a.k.a. DDD -- see domaindrivendesign.org) Design Blitz & other Agile Modeling techniques (see agilemodeling.com

)

69

http://www.featuredrivendevelopment.com/node/507

http://www.step-10.com/SoftwareProcess/FeatureDrivenDevelopment/introduction.html

http://en.wikipedia.org/wiki/UML_colors

http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture30.pdf


http://en.wikipedia.org/wiki/Domain-driven_design

http://www.domaindrivendesign.org/

http://agilemodeling.com/

http://www.agilemodeling.com/


Supple Design

70


Supple Design & DDDDomain-Driven Design (DDD) approaches modeling the core logic of the software by focusing on the domain. The basic idea is the design should directly reflect the core business

domain and domain-logic of the problem to solve This helps understanding the problem as well as the implementation

and increases maintainability of the software.

DDD uses common principles and patterns as "building blocks" to model & create a “supple design” Supple: pliant, malleable, limber, yielding or changing readily. The design is firm yet flexible, with structure and intent both clearly

conveyed and deeply realized by the code.

Patterns of Supple Design include: Intention-Revealing Interfaces, Ubiquitous Language, Side-Effect-

Free Functions, Assertions, Conceptual Contours, Standalone Classes, Closure of Operations, Declarative Style

71

http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215


Resources: Evolutionary Architecture

• Neal Ford's series of IBM developerWorks articles on Evolutionary Architecture and Emergent Design

• Chris Sterling's book on Architecture in an Agile Organization• Presentations by Ryan Shriver at theagileengineer.com• Dean Leffingwell's writings on Agile Architecture • OOPSLA '09 workshop on "Architecture in an Agile World"• Software Architecture and Agile Software Development - An Oxymoron? by

Philippe Kruchten • Agile Architecture - How much is enough?, by Eoin Woods• The Agile Architect site (including the role of the agile architect)• Agile Architecting by Erik Philippus • Scott Ambler's Scaling Agile Development via Architecture• Lean Software Architecture, Jim Coplien & Gertrud Bjornvig• Architecture Meets Agility, by Hakan Erdogmus• Systems Engineering and Architecting Challenges: Application to Agile Devel

opment, by Murray Cantor

72

http://www.ibm.com/developerworks/views/java/libraryview.jsp?search_by=evolutionary+architecture+emergent+design:

http://architectureinanagileorganization.com/

http://www.theagileengineer.com/public/Presentations/Presentations.html

http://www.theagileengineer.com/

http://scalingsoftwareagility.wordpress.com/category/agile-architecture/



http://mysite.verizon.net/dennis.mancl/oopsla09

http://csse.usc.edu/csse/event/2009/Arch-Workshop/presentations/Kruchten%20090608%20agile%20architecture%20usc.ppt

http://www.eoinwoods.info/doc/SA2008-Woods-Agile-Arch.pdf

http://www.agilearchitect.org/agile/index.asp

http://www.agilearchitect.org/agile/role.htm

http://www.improvement-services.nl/Agile%20Architecting%20-%20Friends%20or%20Foes.pdf

http://www.agilejournal.com/articles/columns/column-articles/146-scaling-agile-development-via-architecture

http://www.leansoftwarearchitecture.com/

http://www2.computer.org/cms/Computer.org/ComputingNow/homepage/2009/0809/rW_SW_ArchitectureMeetsAgility.pdf

http://www.csi.stevens-tech.edu/Keynote%20&%20Invited/Murray%20Cantor.pdf

http://www.csi.stevens-tech.edu/Keynote%20&%20Invited/Murray%20Cantor.pdf


Resources: Emergent Design andIncremental Design• Emergent Design: The Evolutionary Nature of

Professional Software Development, by Scott L. Bain• Neal Ford,

Evolutionary Architecture and Emergent Design series of articles on IBM developerWorks

• Is Design Dead? by Martin Fowler• The Mikado Method, by Daniel Brolund & Ola Ellnestam

73


http://www.tinyurl.com/nf-ead

http://www.martinfowler.com/articles/designDead.html

http://mikadomethod.wordpress.com/


Resources: Restructuring

• InfoQ: Michael Stal on Architecture Refactoring• Refactoring in Large Software Projects: Performing Compl

ex Restructurings Successfully, by Martin Lippert, Stephen Roock (also an earlier version online)

• The Mikado Method and Refactoring Large Software Systems

• Object-oriented Reengineering Patterns, by S. Demeyer, S. Ducasse & O. Nierstrasz (freely downloadable online)

74

http://www.google.com/url?sa=t&source=web&ct=res&cd=4&url=http://www.infoq.com/interviews/michael-stal-on-architecture-refactoring&ei=FZdlSszqAYfENsTIlZUB&usg=AFQjCNGgGxdn_zhxA-a0Gfyrq9JnVa_N9w&sig2=hws48fbHt5iLDvR3KVjlWA







http://www.martinlippert.org/publications/review/LargeRefactoringsReviewVersion.pdf

http://mikadomethod.wordpress.com/

http://www.methodsandtools.com/archive/archive.php?id=98

http://scg.unibe.ch/download/oorp/


Resources: Technical Debt• Technical Debt and Design Death, by Kane Mar and Michael James• Design debt economics, by John Elm, IBM developerWorks, June 2009• The Agile Executive articles on Technical Debt, by Israel Gat• Managing Technical Debt and 10X Software Development - Technical Debt,

by Steve McConnell• Managing Technical Debt; by Tom Brazier • Repaying Technical Debt; by Simon Baker • Software Debt and Technical Debt, by Chris Sterling, author of

Managing Software Debt (also see video presentation) • Software Entropy: Don’t Tolerate Broken Windows, and

Zero-Tolerance Construction by Andy Hunt and Dave Thomas• Monetizing Technical Debt, and other InfoQ articles on technical debt• Technical Debt - How not to ignore it! by Henrik Kniberg • Continuous Refactoring and the Cost of Decay, by Patrick W. Welsh

75

http://www.danube.com/system/files/CollabNet_WP_Technical_Debt_041910.pdf

http://www.ibm.com/developerworks/rational/library/edge/09/jun09/designdebteconomics/index.html

http://theagileexecutive.com/category/technical-debt/

http://theagileexecutive.com/category/technical-debt/

http://www.construx.com/Page.aspx?hid=2801

http://blogs.construx.com/blogs/stevemcc/archive/2007/11/01/technical-debt-2.aspx

http://accu.org/index.php/journals/1301

http://www.think-box.co.uk/blog/2005/11/repaying-technical-debt.html

http://architectureinanagileorganization.com/software-debt/

http://architectureinanagileorganization.com/chapter-07-technical-debt/


http://video.yahoo.com/watch/4754803/12699073

http://www.pragprog.com/the-pragmatic-programmer/extracts/software-entropy

http://media.pragprog.com/articles/sep_02_zerotol.pdf

http://www.infoq.com/news/2010/03/monetizing-technical-debt

http://www.google.com/search?q=%22Technical+Debt%22+site:www.infoq.com

http://www.google.com/search?q=%22Technical+Debt%22+site:www.infoq.com

http://www.crisp.se/henrik.kniberg/presentations/agile2008/Technical-Debt-how-not-to-ignore-it.pdf

http://patrickwilsonwelsh.com/?p=10


Resources: Modifiability Tactics

• Modifiability Tactics, SEI @ CMU Technical Report CMU/SEI-2007-TR-002 http://www.sei.cmu.edu/publications/documents/07.reports/07tr002.html

• Modifiability Tactics, Chapter 5 section 3 of Software Architecture in Practice http://www.tar.hu/softarchpract/ch05lev1sec3.html

• Understanding Architectural Patterns in Terms of Tactics and Models http://www.sei.cmu.edu/news-at-sei/columns/the_architect/2007/08/architect-2007-08.htm

• Design and Tactics, Lecture notes from an Aalborg University course on Database and Software Architecture http://www.cs.aau.dk/%7Emly/dbsa07/L8_Design_and_Tactics.pdf

76



http://www.tar.hu/softarchpract/ch05lev1sec3.html

http://www.tar.hu/softarchpract/ch05lev1sec3.html

http://www.sei.cmu.edu/news-at-sei/columns/the_architect/architect.htm

http://www.sei.cmu.edu/news-at-sei/columns/the_architect/architect.htm

http://www.sei.cmu.edu/news-at-sei/columns/the_architect/2007/08/architect-2007-08.htm

http://www.cs.aau.dk/~mly/dbsa07/L8_Design_and_Tactics.pdf

http://www.cs.aau.dk/~mly/dbsa07/

http://www.cs.aau.dk/~mly/dbsa07/L8_Design_and_Tactics.pdf


Resources: Supple Design & DDD• Wikipedia page on DDD: http://en.wikipedia.org/wiki/Domain-driven_design • DDD Intro Article: http://www.infoq.com/articles/ddd-in-practice and

http://www.typo3-media.com/blog/domain-driven-design-introduction.html

• DDD – a Brief Introduction:http://www.typo3-media.com/blog/domain-driven-design-introduction.html

• DDD: Supple Design Patterns http://www.cs.colorado.edu/~kena/classes/6448/s05/lectures/lecture30.pdf

• DDD Pattern Summaries:http://domaindrivendesign.org/resources/what_is_ddd

• Free mini-eBook: Domain-Driven Design Quickly http://www.infoq.com/minibooks/domain-driven-design-quickly

• Yet another free mini-eBook: Step-by-Step Guide to DDD http://dddstepbystep.com/

• See also http://domaindrivendesign.org/ and http://www.infoq.com/domain-driven-design

77

http://en.wikipedia.org/wiki/Domain-driven_design

http://www.infoq.com/articles/ddd-in-practice




http://domaindrivendesign.org/resources/what_is_ddd

http://www.infoq.com/minibooks/domain-driven-design-quickly

http://dddstepbystep.com/

http://domaindrivendesign.org/

http://www.infoq.com/domain-driven-design

Part III

Special Topics and Further Information/Resources

ingRefor

Refingact

oracting

ingactor

actingRef

orRefing

orRef

ingact


Refactoring & Legacy CodeFrom Michael Feathers’ book Working Effectively with Legacy Code (WELC) : Legacy Code is defined as any existing code that does not have

adequate automated test coverage.

Recommends an incremental approach for legacy code: Refactor & automate tests for legacy methods/subroutines whenever

you are already “in the neighborhood” to modify them for an existing task in the current iteration.

Don’t attempt a comprehensive & thorough “legacy clean-up” all at once (no “magnum opus” grand-redesign-in-the-sky)

Break-up larger/significant “legacy clean-up” efforts into smaller chunks and defer to the next planned restructuring.

79



Refactoring DatabasesSee the following:• Refactoring Databases: Evolutionary Database Design, by Scott

W. Ambler & Pramodkumar J. Sadalage• Refactoring SQL Applications, by Stephane Faroult, Pascal

L'Hermite• Catalog of database Refactorings, by Scott Ambler et.al.• Evolutionary Database Design, by Martin Fowler• Agile Database Refactoring with Hibernate, by Gilad Buzi, Kelley

Glenn, and Jonathan Novich.

80

http://databaserefactoring.com/

http://oreilly.com/catalog/9780596514976/

http://www.agiledata.org/essays/databaseRefactoringCatalog.html

http://www.agiledata.org/essays/databaseRefactoringCatalog.html

http://www.martinfowler.com/articles/evodb.html

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Resources/tabid/3161/Default.aspx


Refactoring Web Content and Test CodeSee the following:• Refactoring HTML: Improving the Design of Existing

Web Applications, by Elliotte Rusty Harold

• xUnit Test Patterns: Refactoring Test Code, by Gerard Meszaros

81






Sample Test Code RefactoringsMinimize Data:

Remove things from the fixture until we have a Minimal Fixture.Inline Resource:

Move contents of an external resource into fixture setup logic of the test.Make Resource Unique:

Make the name of any resources used by a test unique.Replace Dependency with Test Double:

Break dependency by replacing a depended-on component with a Test Double.

Extract Testable Component: Extract logic to test into a separate component that is designed for

testability and is independent of the context in which it is run.Setup External Resource:

Create an external resource within the fixture setup logic of the test rather than using a predefined resource.

Source: www.xunitpatterns.com

82

http://www.xunitpatterns.com/


Online Resources• The Refactoring homepage - www.refactoring.com • Refactoring to Patterns Catalog• Refactoring: Small Steps Guaranteed to Help You Clean Up Your Code,

by C. Keith Ray.• Introductory Refactoring Articles• Podcasts, “Refactoring: Part 1 and Part 2,” a talk with Martin Lippert.• Refactoring Checklist, by Michael Nielsen.• Michael Stal on Architecture Refactoring, by Niclas Nilsson.• Introduction to Refactoring to Patterns, by Joshua Kerievsky.• Code Refactoring Guidelines, by Federico Cargnelutti.• Smells to Refactorings table, from wiki.java.net • Refactoring with Code Metrics, Andrew Glover• Refactoring Thumbnails, from www.refactoring.be • Continual Refactoring, by Paul Duvall• Refactoring blogs, books, forums, sample chapters, videos, tools and

tutorials.

83

http://www.refactoring.com/

http://www.industriallogic.com/xp/refactoring/catalog.html


http://www.deitel.com/ResourceCenters/Programming/Refactoring/Introductions/tabid/3159/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Podcasts/tabid/3160/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Blogs/tabid/3154/Default.aspx




http://www.deitel.com/ResourceCenters/Programming/Refactoring/SampleChapters/tabid/3162/Default.aspx

http://blog.fedecarg.com/2008/09/14/code-refactoring-guidelines/




http://wiki.java.net/

http://www.ibm.com/developerworks/java/library/j-cq05306/

http://www.refactoring.be/thumbnails.html

http://www.refactoring.be/

http://www.ibm.com/developerworks/java/library/j-ap07088/

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Blogs/tabid/3154/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/RefactoringBooks/tabid/3155/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/ForumsandGroups/tabid/3158/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/SampleChapters/tabid/3162/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Videos/tabid/3166/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Tools/tabid/3163/Default.aspx

http://www.deitel.com/ResourceCenters/Programming/Refactoring/Tutorials/tabid/3165/Default.aspx


BooksRefactoring: Improving The Design of Existing Code; by

Martin FowlerRefactoring to Patterns; by Joshua KerievskyEmergent Design: The Evolutionary Nature of Professional

Software Development, by Scott L. BainClean Code: A Handbook of Agile Software Craftsmanship;

by Robert C. MartinWorking Effectively with Legacy Code; by Michael FeathersxUnit Test Patterns: Refactoring Test Code, by Gerard

MeszarosDomain-Driven Design: Tackling Complexity in the Heart of

Software; by Eric EvansImplementation Patterns; by Kent Beck

84











Books

85

Backup Slides

[for reference material]


What Motivates us to Refactor?• Make it easier to add new code.

When adding new functionality to a system, we have a choice:a) implement without regard to how well it fits the existing design,b)modify existing design to easily accommodate the new feature.

With a), we incur design debt, which can be paid down later by refactoring [or more significant restructuring].

With b), we analyze what must change to best accommodate the new feature and then make the necessary changes.

• Improve the design of existing code. By continuously improving the design of code, we make it easier

and easier to work with. Continuous refactoring involves constantly sniffing for code smells

and removing them immediately (or soon after) If you get into the hygienic habit of refactoring continuously, you'll

find that it is easier to extend and maintain code.

Source: J. Kerievsky, Introduction to Refactoring to Patterns

87




What Motivates us to Refactor? [2]

• Gain a better understanding of code. Sometimes we look at code and have no idea what it does or

how it works. Even if someone could stand next to us and explain the code,

the next person to look at it could also be totally confused. If the code isn't clear, it's an odor that needs to be removed

by refactoring, not by “deodorizing” the code with a comment.

• Make coding less annoying/frustrating. Sure, we can say we refactor to remove duplication, to

simplify or clarify the code. But what actually compels us to refactor?

Sometimes it is emotions. Face it – it’s just plain frustrating to have to fix or extend code that is poorly structured, overly complex, and difficult to understand/navigate.


88




Refactoring and EvolutionWorking incrementally changes “I wish I had thought of

this sooner” to “Now I get it; let's do it this way.”It is a fundamentally stronger position to be in.• Evolution requires change ― refactoring is a disciplined way of

making change while exposing the project to significantly less risk.

• It allows us to introduce design at a later period, especially through the concept of refactoring to open-closed.

• Knowing you can do this reduces the ratio of design to over-design.

— Scott L. Bain Emergent Design: The Evolutionary Nature of Professional Software Development, May 2008

89



Test-Driven RefactoringTesting also plays an altogether different role in

refactoring; it can be used to rewrite and replace old code. A test-driven

refactoring involves applying test-driven development to produce replacement code and then swap out old code for new code (while retaining and rerunning the old code's tests).

Just remember that the "reimplement and replace" technique, as performed by using test-driven refactoring, is another useful way to refactor.

While it tends to be most helpful when you're designing a new algorithm or mechanism, it may also provide an easier path than applying low-level or composite refactorings.


90




7 Deadly Symptoms of Rotting Design

1. Rigidity – make it hard to change 2. Fragility – make it easy to break3. Immobility – make it hard to reuse4. Viscosity – make it hard to do the right thing5. Needless Complexity – over design6. Needless Repetition – error prone7. Not doing any

Source: Robert C. Martin, http://www.rgoarchitects.com/Files/ooprimer.pdf &http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf

91

http://www.rgoarchitects.com/Files/ooprimer.pdf

http://www.objectmentor.com/resources/articles/Principles_and_Patterns.pdf


The Laws of Software EvolutionA series of 8 laws formulated by Lehman &

Belady starting in 1974, including: The Law of Continuing Change The Law of Increasing Complexity The Law of Continuing Growth The Law of Declining Quality

The laws can be summarized as follows: A system that is being used undergoes continuing

change or degrades in effectiveness. A computer program that is changed, becomes less

and less structured. The changes increase the entropy and complexity of the program.

Source: Wikipedia on “Lehman’s Laws of Software Evolution”

92

http://en.wikipedia.org/wiki/Lehman's_laws_of_software_evolution


Dimensions of software complexityHigher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance

Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance

Highermanagement complexity - Large scale - Contractual - Many stake holders - “Projects”

Lowermanagement complexity - Small scale - Informal - Single stakeholder - “Products”

Defense MIS System

Defense Weapon SystemTelecom

Switch

CASE Tool

National Air TrafficControl System

Enterprise IS(Family of ISApplications)

CommercialCompiler

BusinessSpreadsheet

IS ApplicationDistributed Objects

(Order Entry)

Small ScientificSimulation

Large-ScaleOrganization/Entity

Simulation

An average software project - 5-10 people - 3-9 month duration - 3-5 external interfaces - Some unknowns & risks

EmbeddedAutomotive

Software

IS ApplicationGUI/RDB

(Order Entry)

Source: Walker Royce

93


Managing Software ComplexityIn No Silver Bullet, Fred Brooks defines two

categories of complexity: Essential complexity: This is the inherent complexity of

the problem domain. There is no avoiding it. (It comes with the territory.)

Accidental complexity: This is the complexity of the solution. It is a byproduct of the systems, languages, and frameworks we use. (“We have met the enemy, and it is us!”)

• In principle, accidental complexity can be reduced by changing the system. Refactoring reduces accidental complexity!

94

http://en.wikipedia.org/wiki/No_Silver_Bullet

http://en.wikipedia.org/wiki/Essential_complexity

http://en.wikipedia.org/wiki/Accidental_complexity


Accidental ComplexityThree things tend to spawn accidental complexity.• Design debt -- just-in-time hacks to code because of schedule or

other external pressures.• Duplication, violations of the DRY (Don't Repeat Yourself)

principle. Duplication is the single most insidious diminishing force in software

development because it manages to creep into so many places without developers even realizing it..

Duplication harms projects because it resists attempts to make structural changes or refactor toward better code.

• Irreversibility. Any decision you make that cannot be reversed will eventually lead to some level of accidental complexity.

Irreversibility affects both architecture and design, although its effects are both more common and more damaging at the architectural level. Try to avoid decisions impossible or cumbersome to reverse.

Source: Neal Ford, Evolutionary Architecture and Emergent Design

95



Why do we call it “Debt”?

Ward Cunningham's financial metaphor of design debt works far more effectively than technical language (e.g., “refactoring”) to influence management.

Due to ignorance or the urge to "not fix what ain't broke" too many developers & teams spend too little time paying down design debt. if you don't pay off a debt, you incur late fees. If you don't pay your late fees, you incur higher late fees. The more you don't pay, the worse your fees and payments

become. Compound interest kicks in, and as time goes on, getting out of

debt becomes an impossible dream.The same is true with design debt.

Source: Joshua Kerievsky, Introduction to Refactoring to Patterns

96




What Causes Technical Debt?It is a difficult, delicate and dynamic balancing act to achieve the necessary and sufficient amount of design to implement only the essential complexity required by system.

We lack perfect knowledge/certainty, and can’t always get it right the first time we try:

Sometimes we knowingly (and under pressure) do something the "quick & dirty" way, with the intent to “clean it up” later.

Sometimes we attempt too much to soon, delving into details of the requirements when we and our customers/users haven't yet learned enough about the true needs of the system.

Sometimes we unknowingly violate design principles, resulting in undesirable dependencies that make code or other work-products hard to change.

Sometimes we neglect to properly "tend" to the design and don't give it the necessary amount of ongoing care and feeding needed to keep it "fit" and "supple."

97


Principal & Interest• Every developer becomes aware of the concept of

technical debt, whereby you make compromises in your design for the sake of some external force, such as schedule pressure. Technical debt resembles credit card debt: you don't have

enough funds at the moment, so you borrow against the future. Similarly, your project doesn't have enough time to do something

right, so you hack a just-in-time solution and hope to use some future time to come back and retrofit it.

• Unfortunately, many managers don't seem to understand technical debt, causing resistance to revisiting past work.


98



Software Entropy & ComplexityBuilding software isn't like digging a ditch.

If you make compromises when you dig a ditch, you just get uneven width or unequal depth. Today's flawed ditch doesn't prevent you from digging a good ditch tomorrow.

But the software you build today is the foundation for what you build tomorrow. Compromises made now for the sake of expediency cause entropy to build up in your software.

Entropy is a measure of complexity, and if you add complexity now because of a just-in-time solution to a problem, you must pay some price for that for the remaining life of the project.


99



Technical Debt & Interest• Let's say that you want to add new features to an

existing, long-running project. These new features have a certain inherent complexity to them. However, if you have technical debt already, you must work

around those compromised parts of the system to add new features.

• Thus, the cost for additions mirrors the financial metaphor.

• Figure 2 shows the difference between the effort required to add a new feature in a cleanly designed system (for example, one with little or no technical debt), versus a typical system that contains a lot of technical debt.


100



Technical Debt & Interest [2]Difference in effort required to add a new feature in a cleanly designed system, versus a typical system that contains a lot of technical debt.


You can think of the inherent complexity as the principal, and the extra effort imposed by previous expedient shortcuts as the interest.

101
