© 2007 grady booch read’n writ’n ‘rithmetic … & cod’n grady booch ibm fellow

© 2007 Grady Booch

Read’nWrit’n

‘Rithmetic… & Cod’n

Grady BoochIBM Fellow

© 2007 Grady Booch2

NUMB3RS @ http://www.cbs.com/primetime/numb3rs/


“One Hour,” NUMB3RS @ http://www.cbs.com/primetime/numb3rs/

QuickTime™ and aH.264 decompressor

are needed to see this picture.


Software in the Mainstream

An entire generation has grown up believing that the Internet has always existed

Software-intensive systems pervade the interstitial spaces of society


Our civilization runs on software.

Bjarne Stroustrup


You have riches and freedom here but I feel no sense of faith or direction. You have so

many computers, why don’t you use them in the search for love?

Lech Walesa

Epstein, J., Yale Book of Quotations, New Haven, Connecticut: Yale University Press, 2006.


Software-intensive systems can amplify human intelligence

– They cannot replace human judgment

Software-intensive systems can fuse, coordinate, classify, and analyze information

– They cannot create knowledge

The Promise

Software-intensive systems can amplify human intelligence

Software-intensive systems can fuse, coordinate, classify, and analyze information

•


Not everything we want to build can be built

– There exist pragmatic theoretical and technical limits

Not everything we want to build should be built

– There exist moral, economic, social, and political limits

The Limits

Not everything we want to build can be built

Not everything we want to build should be built

•


Software-intensive systems are perhaps the most intellectually complex artifacts created by humans

– Software is invisible to most of the world

– The best software is simple, elegant, and full of drama as manifest in the cunning patterns that form its structure and command its behavior

The Beauty


•


1994 Chaos Report

16.2% of software projects are successful

31.1% of software projects are outright failures

52.7% of software projects are challenged

http://www.standishgroup.com


2006 Chaos Report

35% of software projects are successful

19% of software projects are outright failures

46% of software projects are challenged



Reasons for Improvement

Better project management

Iterative development

Emerging Web infrastructure



Forces in Software


Software Engineering Body of Knowledge

Project of IEEE, released in 2004

Knowledge areas

– Requirements - Configuration management

– Design - Management

– Construction - Process

– Testing - Tools and methods

– Maintenance - Quality

http://www.swebok.org


Computing Curricula

Project of ACM, first released in 1968

Major subjects

– Information structures and processes

– Information processing systems

– Methodologies

– Mathematical sciences

– Physical and engineering sciences

Curriculum 68, Communications of the ACM, Vol. 11 (3), March 1968.


Computing Curricula (Computing Topics)

Data structures

Programming languages

Models of computation

Computer design and organization

Translators and interpreters

Computer and operating systems

Special purpose systems

Numerical mathematics

Data processing and file management

Text processing

Computer graphics

Simulation

Information retrieval

Artificial intelligence

Process control

Instructional systems

Elementary analysis

Linear algebra

Differential equations

Algebraic structures

Curriculum 68, Communications of the ACM, Vol. 11 (3), March 1968.

Theoretical numerical analysis

Methods of applied mathematics

Optimization theory

Combinatorial mathematics

Mathematical logic

Number theory

Probability and statistics

Operations analysis

General physics

Basic electronics

Circuit analysis and design

Thermodynamics and statistical mechanics

Field theory

Digital and pulse circuits

Coding and information theory

Communication and control theory

Quantum mechanics


Historical Perspectives

Computing as mathematics

Computing as engineering and design

Computing as art

Computing as science

Computing as a social science

Computing as interdisciplinary

Goldweber, M., et al, Historical Perspectives On The Computing Curriculum, ACM SIGCUE Outlook, Vol. 25 (4), October 1997.


Computing Curricula

Project of ACM and IEEE, released in 2005

Major disciplines

– Computer engineering

– Computer science

– Information systems

– Information technology

– Software engineering

http://www.acm.org/education/curricula.html



Algorithms and complexity

Architecture

Discrete structures

Graphics and visual computing

Human-computer interaction

Information management

Intelligent systems

Net-centric computing

Operating systems

Programming fundamentals

Social and professional issues

Software engineering




Programming fundamentals

Iterative programming

Algorithms and complexity

Computer architecture and organization

Operating system principles and design

Operating systems configuration and use

Net Centric principles and design

Net Centric use and configuration

Platform technologies

Theory of programming languages

Human-computer interaction

Graphics and visualization

Intelligent systems

Information management theory

Information management practice

Scientific methods

Legal/professional/ethics/society

Information systems development

Analysis of business requirements

E-business


Analysis of technical requirements

Engineering foundations for software

Engineering economics for software

Software modeling and analysis

Software design

Software verification and validation

Software evolution

Software process

Software quality

Computer systems engineering

Digital logic

Embedded systems

Distributed systems

Security issues and principles

Security implementation and management

Systems administration

Management of information systems organization

Systems integration

Digital media development

Technical support


K-12 Model Curriculum

Project of ACM, released in 2003

Sructure

– Foundations of computer science (K-8)

– Computer science in the modern world (9-10)

– Computer science as analysis and design (10-11)

– Topics in computer science (11-12)

http://csta.acm.org/Curriculum/sub/ACMK12CSModel.html


Missing Curriculum

Estimation, procrastination, and motivation

Programming and testing

Tools

Documentation

Users and User Interface

Delegation and interpersonal skills

Meetings

Health and appearance

Finance and future

http:engtech.wordpress.com/2007/03/01/missing_programmer_curriculum/


Estimation, Procrastination, and Motivation

Personal estimation: why it is going to take more than two weeks to hit your deadline

Why cramming for deadline is effective at school but not in the workplace

The Internet: do you really need it for your job?

The passion of the code: loving what you do

Metrics: what isn’t measured, isn’t done

Focus on the important, not the immediate



Programming and Testing

Why we don’t’ teach scripting languages even though you’ll use them more often than Fortran, Pascal, C, or Java

All programming is maintenance programming

Why re-writing from scratch is always the worst choice

Verification: if it wasn’t tested it doesn’t work

Automation vs doing it by hand: what is more efficient?

Hardening: the only bug-free code is old code

The turtle and the rabbit: fast and lazy vs slow and diligent



Tools

Don’t blame the tools (even though the tools suck)

Don’t reinvent the wheel: leveraging open source

How version control will save your life

Text editors: why notepad isn’t enough

Bug tracking: more than Post-It notes

Back up strategy: painful but necessary



Documentation

Diagrams, diagrams, diagrams: a picture is worth a thousand words

English: not just for managers

Automation document generation: why documentation should be tied to code

Abbreviation: why abbreviating four to five helps no one



Users and User Interface

Keep it simple, stupid

Cognitive overload

Users are people, not problems

Support isn’t a four letter word (it’s seven)

Servicing interrupts: getting work done while supporting co-workers



Delegation and Interpersonal Skills

Passing the buck and shifting the blame

Making small talk when you have nothing in common

Why co-workers are companions, not competition

Trust issues: don’t assume that what your co-worker gave you works

Bribing IT: getting your computer problems fixed in a reasonable time



Meetings

How to avoid meetings

Who to invite to meetings

How to stay awake in meetings when they are unavoidable

Shutting up: keeping meetings productive when you have nothing useful to say

Why design by committee doesn’t work



Health and Appearance

Staying in shape even though you’re sedentary

Eating healthy when you have fast food every day

Hygiene: friend of foe?

World Wide Wassup: meeting your mate on the Internet

I am pretty: why do into consulting or marketing



Finance and Future

Planning for retirement

Public companies vs start ups: pros and cons

How much of the company do VCs own?

Stock options dos and don’ts

How to evaluate benefits compared to salary



Handbook Of Software Architecture (Genres)

Artificial intelligence

Commercial and non-profit

Communications

Content authoring

Devices

Entertainment and sports

Financial

Games

Government

Industrial

Legal

Medical

Military

Operating systems

Platforms

Scientific

Tools

Transportation

Utilities

http://www.booch.com/architecture


Lessons Learned

Essential skills

– Abstraction

– Collaborative development

Metaskills

– Fundamentals

– The ability to learn

– Passion, beauty, joy, and awe


The Limits of Software

Laws of physics

Laws of software

Challenge of algorithms

Difficulty of distribution & concurrency

Problems of design

Importance of organization

Impact of economics

Influence of politics

Limits of human imagination

Fundamental

Human•


Laws of Physics

The speed of light

– Has pragmatic implications for distributed systems

Relativistic effects

– There is no such thing as absolute time

Quantum effects

– There are theoretical as well as practical limits to information density

Thermodynamic effects

– Software is weightless/containers for software are not

– Computation dissipates heat


Laws of Software

Sometimes we can’t do it

– Halting problem

– Gödel's theorem/highly non-computable problems

Sometimes we can’t afford to do it

– Sorting problem/Towers of Hanoi/chess

– Exponential time

Sometimes we just don’t know

– Traveling salesman/scheduling/bin packing

– NP complete

Harel, D., Computers, Ltd: What they Really Can’t Do, Oxford, England: Oxford University Press, 2003


Challenge of Algorithms

Compression

Photorealistic rendering

Speech recognition

Simulation

Knowledge representation

Intimate/massive parallelism



Halo 3 @ http://www.bungie.net •


Difficulty of Distribution/Concurrency

A distributed system is one in which the failure of a computer you didn’t even know existed can render your computer unusable (Leslie Lamport)

The average developer does not have, as a core competency, the ability to develop secure, concurrent, and distributed systems


Problems of Design

The entire history of software engineering is that of the rise in levels of abstraction

The limitations of human understandability

Building for resilience

Sometimes worse is better

The discovery of patterns


Importance of Organization

All meaningful development is formed by the resonance of activities that beat at different rhythms– The activities of the individual developer– Social dynamics among small sets of developers– Dynamics among teams of teams

Work products and work flows Parallel development The cacophony of stakeholders The tension of high and low ceremony processes


Impact of Economics

Performance = Effort or time

Complexity = Volume of human-generated code

Process = Methods, notations, maturity

Team = Skill set, experience, motivation

Tools = Software process automation

Performance = (Complexity) (Process) * (Team) * (Tools)


Influence of Politics

Success as defined by the software development team is sometimes misaligned with success as defined by the management team

Software as a strategic weapon

Software as a pawn


Limits of Human Imagination

The visionaries

– Alan Kay, Danny Hillis, Marvin Minsky, Rodney Brooks, Bill Joy, Bill Gates, Adele Goldburg, Tim Berners-Lee, Nicholas Negroponte…

The dreamers

– Arthur C. Clarke, Neal Stephenson


Software developmenthas been, is, and will remain

fundamentally hard


What We Know

Fundamentals

– Craft crisp and resilient abstractions

– Maintain a good separation of concerns

– Create a balance distribution of responsibilities

Process

– Grow a system’s architecture through the incremental and iterative release of testable executables


What We Know

Reuse

Patterns

Languages

Architectural codification of certain domains


Representing Software Architecture

Logical View

End-user Functionality

Implementation View

Programmers Configuration management

Process View

PerformanceScalabilityThroughput

System integrators

Deployment View

System topologyCommunication

Provisioning

System engineering

Conceptual Physical

Use Case View

Kruchen, P. The 4+1 Model View, IEEE Computer, vol. 12 (6), November 1995


Gallery of Software Architecture: Air Traffic Control

Gallery of Software Architecture @ http://www.booch.com/architecture/architecture.jsp?part=Gallery


Gallery of Software Architecture: C3I



Gallery of Software Architecture: Games



Gallery of Architecture: Games



Gallery of Software Architecture: Google



Gallery of Software Architecture: Pathfinder



Gallery of Software Architecture: Speech Recognition



Movements in Web-centric Architectures

Simple documents

Colorful clients

Simple scripting

Rise of middleware

Rise of simple frameworks

Emergence of dynamic frameworks

Semantic web


Handbook of Software Architecture

No architectural reference exists for software-intensive systems

Goals of the handbook– Codify the architecture of a large collection of interesting software-intensive

systems

– Study these architectural patterns in the context of the engineering forces that shaped them

– Satisfy my curiosity

Handbook of Software Architecture @ http://www.booch.com/architecture


Preservation Of Classic Software

No comprehensive and intentional activity has yet been undertaken to preserve the industry’s seminal software artifacts

There are a number of reasons to act now

– Many of the authors of such systems are still alive

– Many others may have the source code or design documents for these systems collecting dust in their offices or garages

– Time is our enemy

Computer History Museum @ http://www.computerhistory.org


How We Got Here

1910s beginning of automation

1920s beginning of expansion

1930s beginning of dependence

1940s beginning of von Neuman machines

1950s rise of the machines

1960s rise of the languages and methods

1970s death of the mainframe

1980s age of the personal computer

1990s age of the Internet and new methods

2000s retrenchment


The Current State

The typical software-intensive system is

– Continuously evolving

– Connected, distributed & concurrent

– Multilingual and multiplatform

– Secure

– Autonomic

Most systems are actually systems of systems

– Services and other messaging mechanisms dominate


Where We Are Going

2010s age of transparency

2020s total dependence

2030s rise of the machines


The Future State

Every advance leading to the future state of the world requires the presence of software yet-unwritten as of today


Languages & Algorithms

Most programmers still write algorithmic snippets in the context of a sea of objects

Legacy XML, Java, C++, and UML persist

Some algorithmic breakthroughs have emerged

– Searching massive quantities of information is still a bit of a struggle

Domain-specific frameworks are mainstream

••


Platforms

Moore’s law has died

The typical personal computer contains multiple processors, a petabyte of main memory, an exabyte of external memory, and untethered terabit connectivity

Virtual high resolution displays dominate; 3D windows, mice, gestures, and voice are the usual mechanisms for interaction

Form factors will change such that most personal computers will be wearable or embedded; most software is embedded in devices

••


Operating Systems & Middleware

Operating systems have largely been commoditized

Middleware that does transaction isolation, load balancing, resource management, and data access still dominates

– but it too has largely been commoditized, forcing the platform vendors to keep growing the value pile

••


Connection

More than ever, the network is the computer

– Monolithic -> client/server -> Web -> grid

Network access is a global utility

Not everything is an enterprise system, but most applications are connected to several

••


Security

New kinds of cybercrime have arisen

– Unlimited piles of money still do not yield secure systems

– Air gaps are still not enough

Rolling failures still plague some systems

••


Autonomics

No computer has yet passed the Turing Test (but we have come close)

Most interesting systems exhibit signs of agency and self-repair

•



Star Trek@ http://www.startrek.com •


Developer Experience

Most developers have grown up believing that the Internet has always existed

Most programming occurs on the edge of a system and in the interstitial spaces among systems

Most programming is now done by domain-specific developers who only incidentally know how to program

There have been only incremental improvements in programmer productivity and the programming model– CLI-> IDE -> XDE -> CDE

••


The Contradiction


– The Internet provides an environment where terrorists and other criminal enterprises can operate with little fear of detection

The Internet changes the way that individuals communicate and businesses collaborate

•Internet Mapping Project @ http://www.cheswick.com/ches/map/index.html


The Contradiction


– The Web creates a greater opportunity for the exploitation of children as well as fraud and theft directed against individuals and organizations

The Web provides unprecedented mechanisms for social networking

•Britney Spears @ http://www.myspace.com/britneyspears


The Contradiction


– Software-intensive systems can erode personal privacy and other basic human rights

Software-intensive systems permit real time and distributed access to information

•WeatherLink @ http://www.davisnet.com


The Contradiction


– Email and the aging of digital archives threatens the preservation of history

Email and other software-intensive mechanisms increase the velocity of communication

•Project Gutenberg @ http://www.gutenberg.org


The Contradiction


– Piracy disrupts the economic underpinnings of traditional media companies and can dilute the intellectual property of artists

Software-intensive systems create new forms of artistic expression

•Halion @ http://www.steinberg.net


The Contradiction


– Software-intensive systems are at the center of a new generation of offensive and defensive weapons

Software-intensive systems enable and accelerate scientific research

•Entrez Genome Project @ http://www.ncbi.nlm.nih.gov/entrez


The Contradiction


– The complexity of software-intensive systems continues to grow; this complexity impacts its users as well as the stakeholders who develop, deploy, operate, and evolve them

Software is a part of the very fabric of civilization, living in its interstitial spaces

•

Annual U.S. Patent Applications/AwardsAnnual U.S. Patent Applications/Awards

ApplicationsAwards

350

300

250

200

150

100

50

0 1910 1920 1930 1940 1950 1960 1970 1980 1990 2000

New or modified source lines of code per year per developer & cumulative

0

100,000,000,000

200,000,000,000

300,000,000,000

400,000,000,000

500,000,000,000

600,000,000,000

700,000,000,000

800,000,000,000

194519481951195419571960196319661969197219751978198119841987199019931996199920022005

New or modified source lines of code peryear

Cumulative source lines of code


It is a tremendous privilege to be asoftware professional

it is also a tremendous responsibility

•


Read’nWrit’n

‘Rithmetic… & Cod’n

Grady BoochIBM Fellow

© 2007 grady booch read’n writ’n ‘rithmetic … & cod’n grady booch ibm fellow

Documents

software slide

software projects

humans software

best software

information slide

humans slide

bjarne stroustrup slide

outright failures