Your speaker

David Zifferwww.ProjectPro.com630-879-0620Connect w/me on LinkedIn

The coming revolution in software productivity:

Eliminating useless variation in software development

Part 0:A visionof a not-too-distant future

Imagine we are cruising along on our current course. As software managers we spend our days thinking about .Methodologywhere we hope to gain maybe a 5% gain in productivity

and .Libraries and Frameworkswhere we hope to gain maybe another 5% gain in productivity

and .Toolswhere we hope to gain maybe another 5% gain in productivity

Then one amazing day .Someone comes up with a fundamentally different way of building applications.

Instead of gradual, fractional changes in productivity, people are now able to write apps in half the time, or maybe even one third the time.

A revolution takes place.

Everything we were doing before becomes irrelevant in the face of this change.

The only problem is .This new system and all its documentationare written in Hindi.

Or maybe Mandarin.

But dont worry.Your future is bright.Orange, that is.

Part 1:The past

Hardware: a billion-to-one improvement in fifty years1960: 16 bytes per cc2010: 16 GB per cc

Hardware gains were made by active management of products & processes1950196019701980199020002010

In comparison, software progress is pretty lackluster1975: C2010: C#

Software management focuseson three types of changesMethodology (Waterfall, Agile, etc.)Waterfall & Agile are 40 years apartbut unlike core planes vs. LSI, people still make a case for using eithersuggesting that the efficacy is something less than spectacularLibraries (and frameworks)completely eliminate the task of coding, but only for particular functionsthey do not fundamentally change the production efficiency of the code that still must be writtenTools (particularly compilers)These are the only software innovations that have fundamentally changed the efficiency of writing code

Arguably there have been onlythree significant gains in software productivity,and all are in the area of languages1950s: Assembly language replaces machine language(1000 to 1) 1960s: Structured languages replace assembly language(100 to 1) 1980s: Object-oriented enhancements improve structured languages(10 to 1)

Language changes are getting fewer,further apart, and less effective1950s: machine language -> assembly1000 to 1 improvement at the dawn of computing 1960s: assembly -> structured languages 100 to 1 in ten years 1980s: structured -> object oriented10 to 1 in twenty years 2010: its been 30 years and nothing much new yet

Part 2:The present

Software today is produced in the manner of pre-assembly-line carsSmall groups of craftsmen build apps one at a time from the ground up. Every app is improvised to some great degree. Different construction methods are used at various times even within the same app, according to the tastes of the individuals working on it, i.e. there is plenty of useless variation.

Small groups of craftsmen build apps19101980

Theres plenty of improvisation19102010

And theres plenty of useless variation, leading to expensive development, poor reliability& high cost of ownership

Useless design variation is the rule.Even within the same app we are likely to findmultiple implementations of:database table structure (keys, status fields, etc.) table-access functions in the database (insert, update, delete, read) database-access functions in the application (ditto) business logic structure includingdata transformation (between database and UI)validationerror handling user interface structure

Enforcement of coding standards tends to fail:Training in the use of standards costs money. Enforcement of standards costs money. Use of standards is generally at the option of the developer, who may not subscribe to them. The value of standards is apparent only to those who understand long-term profitability. Most importantly: use of standards imposes a guaranteed extra short-term effort that must be tolerated in order to potentially benefit in the possibly distant future.

Since we dont tend to have standards,our concept of quality control tends to center around post-mortem inspection:A code review performed after a lot of bad code has been written is expensive and ineffective:Why didnt we help the programmer up front to produce what was expected?We probably cant afford to rewrite the code and even if we can, were late. A testing regime can only tell us whether the code happens to pass a limited set of tests. Testing does not tell us whether the code is well constructed, i.e.:whether we can impute from the design whether the code will pass tests that we cant afford to runwhether new staff can be easily assimilatedwhether the code can be changed easily without fear of breakage.

Something has to changeMost programmers do not have the training or experience to efficiently or correctly build most of the components of most apps. Unless we stop assigning them such tasks we cannot improve productivity significantly. Giving programmers frameworks and libraries is not enough; this does not fundamentally change the way applications are built. Changing & refining methodologies is at best an exercise in low expectations.

Part 3:The future

Meaningful productivity gains are always implemented through technical managementNon-technical management cannot possibly create or enforce innovations in programmer behavior. Since productivity-enhancing revolutions are always highly technical in nature, non-technical managers cannot implement them. Management by the feature list and schedule actually impedes productivity improvement by forcing technical managers & programmers to improve products and processes at their own risk.

Henry Ford was technical managerNo non-technical or non-automotive person could ever have conceived of the automotive assembly line"1913 - Trying out the newassembly line" By an unknown photographer,Detroit, Michigan, 1913 National Archives and Records Administration, Records of the Bureau of Public Roads

Steve Jobs is a technical managerNo non-technical or non-computer person could ever have conceived of (or stolen) Apples many innovations.

Apples least memorable years were under the direction of a non-technical CEOYou cannot substantially improve a complex technical product or product line by merely applying veneer and promotion

What is the potential here?Prior to structured languages, every programmer had the latitude to design his own parameter-passing conventions. Consequently, conventions were typically different from app to app or even within the same app from function to function. Program writers consumed a lot of time creating such conventions. Program maintainers spent a lot of time decoding such conventions.Consider just one aspect of the change fromassembly language to structured languages

Lets shoot for the ideal the data modelthe business rulesand the user interface In an ideal world programmers would have to think about only three things that are fundamentally different across applications:How do we get there?

by getting rid of the real world data modeling database access in the database database access in the application business logic technology error handling UI component management UI design In the real world programmers are mostly consumed with reinventing:So how do we get rid of that?

First we must reconcile ourselves to an unpleasant realityAssemblers prevented programmers from diddling with address bits, but rewarded them by letting them modify programs without rewriting every instruction containing an address. Structured languages hid the vast majority of underlying code from the programmer, but improved productivity by vastly reducing the amount of code written by hand. OO language enhancements are largely voluntarily and depend heavily upon programmer competence; consequently they have had less impact.Significant software productivity gains always come at the cost of developer freedoms

Innovations that do not restrict programmer freedoms have relatively little impactLibraries create tremendous economies by eliminating the writing of some code entirely, but only for the functionality that is implemented by the library. Methodologies arguably have little impact on overall productivity.

We get to the ideal by eliminating useless variationin software development:A new paradigm must define a standard program architecture to be used as the basis of the vast majority of common applications (i.e. database-based business apps). A complete prototype of the standardized architecture must be the basis of application development. It is not enough to tell or show programmers how to do things; rather they need templates. To conform the prototype to the specific needs of each application (different schemas, business rules, user interfaces), the prototype must contain code generators that generate customized code in a standardized way.

continuing The prototype and its generators must support standard conventions for constructing:database tablesdatabase codedata-layer tables & related componentsbusiness rule structureuser interface structure (especially, keeping business rules out of the UI) The prototype should contain complete implementations of standard components that perform generalized functions (such as an object-relational-mapper [ORM]).

Keep in mind Modern car factories do not look exactly like Henry Fords original assembly line. His specific implementation was not the point. The point was that the assembly line specified a very detailed plan for building a car, far more specific than anyone had previously imagined. The assembly line demonstrated the fantastic economies that could be obtained through detailed standardization.The objective here is not to build a system that survives for all time and defines application structure forever:

Recommended Reading

RAP Standard Four-Tier Architecture

Generating a table-- Personcreate table TBcrmPerson(--#include "PrimaryKey.sql"-- non-indexed fieldsDateOfBirthDTDateTime,Genderchar(1) null,GovtIdNumbervarchar(20) null,

--#include "CommonFields.sql")

Describing a table row const string Description = DESCRIPTION const string Description = "person '[GovtIdNumber]; const string Description = "person '[" + DataTables.TBcrmPerson.ColumnNames.GovtIdNumber + "]'";For every table, RAP generates:You could change this to:But we actually do this:

Using a foreign key in a description const string Description = DESCRIPTION const string Description = "name '[First] [Middle] [Last] of {PersonId};RAP generates:You could change this to: const string Description = "person name '[" + DataTables.TBcrmPersonName.ColumnNames.First + "] [" + DataTables.TBcrmPersonName.ColumnNames.Middle + "] [" + DataTables.TBcrmPersonName.ColumnNames.Last + "]' of {" + DataTables.TBcrmPersonName.ColumnNames.PersonId + "}";And actually we write:

Describing cascading behaviorFor every relationship, you describe what happens to the children when the parent is deleted:and what happens to the parent when the children are deleted:

Get RAP at www.ProjectPro.com

RAP is a LinkedIn group

Your speaker

David Zifferwww.ProjectPro.com630-879-0620Connect w/me on LinkedInYou cant inspect quality into a product. W. Edwards Deming