control systems

+

Open Loop / Closed Loop Project Controls

How to distinguish the difference between Open Loop Control and Closed Loop Control and why this is important when discussing project management, software development, and making decision in those domains.

I write to discover what I think.— Daniel J. Boorstin, Librarian of Congress

Version 2.0, 7.15.14

Copyright ©, 2014, Niwot Ridge LLC, Performance-Based Project Management®


+ Introduction 2

Since the earliest days of the computer software industry, managing of software development projects has been fraught with uncertainty and risk. While the technical content of software products and the technical methods used to build them have changed over time, the fundamental issues that determine the success or failure of software projects have remain constant. The fundamental management mistakes have remained the same.


+These Management Mistakes Are Simple and at the Same Time Complex 1.  We don’t know what Done looks like in units of measure

meaningful to the decision makers.

2.  We don’t have a credible path to reaching Done that provides some level of confidence to those paying for the project, that the needed capabilities will arrive on or before the need date, at or below the expected cost.

3.  Missing visibility to the resources needed to fulfill bullet #2.

4.  Failed to capture the uncertainties that create risk to fulfilling bullet #2.

5.  Failed to provide the means of measuring progress to plan, based on physical percent complete, again in units of measure meaningful to the decision makers.

3


+ Purpose of Management Control System

4

Project Controls is the – Management actions, either preplanned to achieve the desired result, or taken as a corrective measure prompted by the monitoring process. Project controls is concerned with the metrics of the project – quantities, time, cost, and other resources. As well project revenues, delivered customer value and cash flow. These metrics start by focusing on the delivery of business or mission Value in exchange for the Cost of that Value.


5 "You've got to be very careful if you don't know where you're going, because you might not get there."


Any notion of not having a plan is nonsense, just ask Yogi


+Overview of Project Controls

!  Without a desired delivery date, target budget, and expected capabilities, a control system is of little interest to those providing the money at business level. !  There is no way to assure those needs – date, budget, capabilities – can

be met with the current capacity for work, efficacy of that work process, or budget absorption of that work.

!  With a need data, target budget, and expected capability outcome, a control system is the basis of increasing the probability of success. !  These targets are the baseline to steer toward !  Without a steering target the management of the project is Open Loop

!  With need for a project control system, this briefing addresses the two choices: !  Closed Loop Control – where the output signal has direct impact on the

control action. !  Open Loop Control – where the output signal has no direct impact on

the control action.

6


+ Open and Closed Loop Control

!  A control system where the output signal has direct impact on the control action.

!  In a closed loop system the error signal, which is the difference between the input and the feedback is fed to the controller to reduce the error and bring the output of the system to a desired value.

!  For each reference input, feedback (difference between actual state and desired state) is used to take corrective action.

!  A control system where the output signal has no direct impact on the control action.

!  In an open loop system, the output is neither measured nor fed back for comparison with the input.

!  For each reference input there is a fixed operating condition.

Closed Loop Control Open Loop Control

7


+Open Loop Control

!  An Open-Loop system, is a non-feedback system, where the output – the desired state – has no influence or effect on the control action of the input signal.

!  In an Open-Loop control system the output – the desired state– is neither measured nor “fed back” for comparison with the input.

!  An Open-Loop system is expected to faithfully follow its input command or set point regardless of the final result.

!  An Open-Loop system has no knowledge of the output condition – the difference between desired state and actual state – so cannot self-correct any errors it could make when the preset value drifts, even if this results in large deviations from the preset value.

8


+Open Loop Control for a Clothes Dryer !  We set the controller (the timer) on the dryer to run for a

specific time.

!  The dryer creates heat (output) and runs until the timer says stop.

!  When the elapsed time is complete the dryer stops.

!  The clothes may or may not be dry depending on the allotted time and temperature of the dryer.

9


+Open Loop Paradigm

!  A process is controlled through the actions of !  Work performed at the capacity for that work

!  Time period of the work performed

!  This process has a desired outcome not connected with the actual desired state

10

!  In the clothes dryer example – the actual desired state is dry clothes.

!  The timer (controller) is a proxy for the desired state

!  The timer is not measuring the actual state and comparing it to the desired state

!  The timer is an Open Loop control system.


+Formal Representation of Open Loop Control

11

! 

! 

! 

G1 =θ1θi, G2 =

θ2θ2, G3 =

θ0θ2

G1 ×G2 ×G3 =θ1θi×θ2θ1×θ0θ2

=θ0θi

Gain, G( ) = θ0θi


+Closed Loop Control

!  A Closed-loop Control System, is a feedback control system which uses the concept of an open loop system as its forward path but has one or more feedback loops between its output and its input.

!  The reference to “feedback,” means some portion of the output is returned “back” to the input to form part of the systems excitation.

!  Closed-loop systems are designed to automatically achieve and maintain the desired output condition by comparing it with the actual condition.

!  It does this by generating an error signal which is the difference between the output and the reference input.

!  A “closed-loop system” is a fully automatic control system in which its control action being dependent on the output in some way.

12


+Closed Loop Control

!  The desired dryness is set by the user

!  The dryer turns on the heat and starts drying the cloths

!  A sensor measures the dryness of the cloths and determines if it is necessary to keep drying or not

!  When the cloths reach the desired dryness the dryer stop

13


+Closed Loop Control Paradigm

!  The closed loop control has three elements !  The desired state !  The actual state !  The error signal – difference between desired and actual – used

to control the drying function

!  This is the steering signal to keep the system headed in the desired direction – DRY CLOTHS

!  Without the desired state, there can be no steering signal – different between desired and actual

14

LOOP WHILE (IF State=“no DRY, Continue, ELSE Stop)!

!  The desired state of the cloths is defined to the control system !  Make my cloths dry !  Sense “dryness” !  Stop drying when the desired state of “dryness” is

reached


+Formal Representation of Closed Loop Control !  G represents the open-loop gain of the control system and is

the forward path !  H represents the gain in the sensor or measurement system in

the feedback path !  Output of the system = G × Error !  Θe is also input to feedback loop G !  Output f rom summing point equal Error = Input – H × Output

15


+ Key Differences Between Open and Closed Loop Before Proceeding

!  Controller has some knowledge of the output condition

!  The desired condition is not present in the control loop – hence the Open Loop

!  Any corrective action requires an operator input to change the behavior of the system to achieve a desired output condition

!  Controller has some knowledge of the output condition

!  The desired condition is compared to the actual condition to create an error signal !  Which is the difference

between the input signal (the desired dryness) and the output signal (the current dryness)

!  Closed loop means feedback not just for recording the output, but for comparing with the desired state to take corrective action

Open Loop Control Closed Loop Control

16


+ Key Behaviors of Open Loop and Closed Loop Controller

!  No comparison between actual output condition and the desired output conditions

!  Has no regulation or control action over the output condition

!  Each input condition determine a fixed operating condition for the controller

!  Changes or disturbances in external conditions does not result in a direct output change unless the controller and manually altered

!  Output condition errors are adjust by changes in the controller function by measure difference between output and desired condition

!  Output conditions are stable in the presence of an unstable system.

!  Reliable and repeatable output performance results from corrective actions taken from the error signal

Open Loop Control Closed Loop Control

17


+Control Loops and Software Development What data do we have? What data to we need? What questions can be answered with this data? What decisions can be made from answers to these questions using the data? How can we better manage the project to success?

18


+Let’s Tell a Good Story†

!  The Setting – we work in an enterprise IT environment, a product development company, or on a mission critical software project.

!  The Protagonist – Those providing the money need information to make decisions

!  The Imbalance – it’s not clear how to make decisions in the absence of information about, the cost, schedule, and technical outcomes from those decisions.

!  Restoring the Balance – when a decision is made, it needs to based on the principles of microeconomics, at least in a governance based organization. The decision

!  Recommended Solution – start with a baseline estimate of the cost, schedule, and technical performance. Execute work and measure the productivity of that work. !  Using these measures to calculate the variance between planned and actual. !  Take management action to adjust the productivity, the end date, the budget

– using all variables produce a new Estimate To Complete to manage toward

This is a closed loop control system

19

† This approach is guided by Cliff Atkinson’s Beyond Bullet Points, where telling the story is the basis of successful capabilities based planning, process improvement, writing a winning proposal, and or a credible white paper.


+The Microeconomics of Software Development†

!  Economics is the study of how people make decisions in resource-limited situations. This definition of economics fits the major branches of classical economics very well.

!  Microeconomics is the study of how people make decisions in resource-limited situation on a person scale. It deals with decision that individual and organizations make on such issues as how much insurance to buy, which word processor to buy, what prices to change for pro ducts and services, which path to take in a project.

!  Throughput the project lifecycle, these decision making opportunities. Each decision impacts the future behavior of the project and is informed by past performance and the probabilistic and statistical processes of the underlying project activities.

!  To make an informed decision about the project, estimates are made using this information

20

† Microeconomics applied to projects is a well understood and broadly applied discipline in cost account and business strategy and execution. Decision making based on alternatives, their assessed value and forecast cost. Both these values are probabilistic. Microeconomics is the basis of Real Options and other statistical decision making. Without this paradigm decision are made not knowing the future impact of those decisions, their cost, schedule, or technical impacts. This is counter to good business practices in any domain.


+First Look At Sampled Performance Data and Software Control Methods !  In an Open Loop system, the Output condition has no

direct impact on the control action. !  Sampled data from past performance provides information

about the outputs of the process !  No error signal is created between the actual condition and

the desired condition !  This is the definition of Open Loop Control

!  To have a Closed Loop control system !  The Output condition must impact the control actions !  This impact is created by an error signal calculated by the

difference between the desired condition and the actual condition.

!  This feedback is then used to make changes in the system to assure the output condition meets the desired condition.

21


+What Does This Mean For Software Development? !  We have a planned completion date for any important

project. !  This is a critical assumption. !  Without a need date or target budget, the discussion of

controlling projects is of little interest. !  The actual condition is compared to the desired condition

(the planned completion date) to assess if that date can be met. !  If we produce work at the current rate (actual condition) will we

be able to meet the desired condition (planned completion date).

!  The critical aspect for Closed Loop is a desired condition. !  Without that, the control process is Open Loop and the

completion date will be what it will be. !  With a Closed Loop, the corrective actions to complete on

needed date are defined by the difference between actual and desired.

22


+Economic Analysis Techniques for Decision Making Under Uncertainty !  Under complete uncertainty

!  Maximax rule

!  Maximum rule

!  Laplace rule

!  Expected value techniques – estimating probabilities of occurrence of each outcome

!  Buying information based on Epistemic Uncertainty !  Prototyping

!  Little Bets in an agile paradigm

!  How much information is enough is answered with Bayes’ Law

23


+We Have Some Data We Have An Estimate Now We Need To Decide With the project underway, we need to manage toward a needed completion date, for a planned cost, and a needed set of capabilities. How can we do that in an Open Loop system?

24


+Example of Open Loop Control

!  Perform work and write down the actual performance

!  Use this data to forecast future performance

!  Without a target – desired – performance the actual performance can’t be used to generate the error signal needed to take corrective action.

25

Example data from http://softwaredevelopmenttoday.blogspot.com/2014/07/what-is-capacity-in-software.html of sampled development projects for number of stories per sprint.


+ Principles of Open Loop in software development

The question is: !  Is the measured capacity for work sufficient to meet the

needed capacity for work in the project paradigm? !  How can we control the project so we show up on time, on

budget, with the needed capabilities? !  Measuring actuals and using these to forecast this capacity

is Open Loop, since there is no desired performance. !  With the need date, target cost, and expected capabilities,

we need targets to steer toward. !  These targets are not the actual capacity, but the needed

capacity.

Your job is to learn what that capacity is and limit the work to capacity! (Credit to Mary Poppendieck (PDF, slide 15) for this quote).

26


+ What Can We Learn With Open Loop Control?

27

1 3 2 3 2 3 4

2 1

4 3 2 3 4

1

5

2 3 4 3 2 1

4 6

9 11

14

18 20 21

25

28 30

33

37 38

43 45

48

52

55 57

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Actual Stories / Iteration

Cum Actual Stories / Iteration

The actual stories per iteration measured from

project

The cumulative stories per iteration measured from

project

We know what rate we are running at because we record that data. But we don’t want rate we should

be running at to show up on time

We have not steering target to compare our

actuals against


+ What Can We Learn With Closed Loop Control

28

0

10

20

30

40

50

60

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Actual Stories / Iteration Cum Actual Stories / Iteration Planned Stories / Iteration Cumm Planned Stories / Iteration

Actual cumulative stories per iteration measured

from project

Desired cumulative stories per iteration

measured from project

Difference between desired stories per iteration and actual stories per iteration provides steering

information, to tell project if they’ll make the planned completion date given “baseline”

We know what rate we are running at because we record that data. And we don’t want rate we should be running at to show up on time.


+ Summary of two Previous Charts

!  Stories per iteration recorded as project proceeds.

!  Cumulative stories per iteration recorded as project proceeds.

!  What number of storied per iteration should the project be produced to stay on schedule and cost (work effort = cost)?

!  Don’t know – there is no “desired” number for steering.

!  Stories per iteration recorded as project proceeds.

!  Cumulative stories per iteration recorded as project proceeds.

!  What number of storied per iteration should the project be produced to stay on schedule and cost (work effort = cost)?

!  The difference between “desired” and “actual” shows shortfall or overage of stories per iteration to show performance of project

Open Loop Closed Loop

29


+ Process Check for Open Loop Control Systems applied to Software

30

Closed Loop control is based on an Open Loop control. To close the loop, we must establish a target performance number – the desired capacity for work. This desired capacity for work may or may not be the actual performance for work. We need to answer: " At the actual rate, will we be over budget when we finish on

the needed date? " At the current rate will we show up late? " At the current rate, will we be able to deliver the needed

capabilities as planned?


+Wrap Up "  Without a plan, you won’t recognize done when it arrives "  Without a plan, you don’t know where you’re going until you

arrive at the destination, out of money and out of time. "  Without a plan you can’t measure progress other than

consumption of time and money

31


+ Project Success Depends on Many Things

!  When you drive the project in the absence of a desired outcomes – the project goal – without units of measures meaningful to the decisions maker,

Some technical, some managerial, some political, some economic.

32

!  Is like driving in the rear view mirror. !  It can be done, but you don’t know

you ran over anything until you do.

!  Close Loop Control provides the headlights to see where you’re going and what’s in the way of your progress

!  Open Loop Control has no desired outcome in terms of delivery date, cost, and needed capabilities, defined before you start and during you trip, so you’ll only know where you’re going when you arrive.


+Bibliography

!  Software Engineering Economics, Barry Boehm, Prentice Hall

!  Software Cost Estimation with COCOMO II, Barry Boehm, et al, Prentice Hall

!  COSMIC: Guideline for Sizing Business Software, Version 3, http://www.etsmtl.ca/Unites-de-recherche/GELOG/accueil

!  “Avoid Software Project Horror Stories: Check the Reality Value of the Estimate First!”, Harold van Herringen, ICEAA 2014,

http://documentcenter.meetingcaddie.mobi/Events//2ffb743d-7bc3-492c-b756-cfb6eeea0110/Documents/ef7d120c-c386-471a-9f86-bac6bc7a80f7.pdf

See Nederlandse Software Metrieken Association, www.nesma.org and International Software Benchmarking Standards Group, www.isnsg.org, and Common Software Measurement International Consortium, www.cosmicon.com

!  How to Lie with Statistics, Darrell Huff, W. W. Norton, 1954

!  Facts and Fallacies of Software Engineering, Robert L. Glass Addison-Wesley, 2003.

!  Probability Method for Cost Uncertainty Analysis: A Systems Engineering Perspective, Paul R. Garvey, CRC Press, 1999

33


+Bibliography

!  “Factors Affecting Duration And Effort Estimation Errors In Software Development Projects,” Ofer Morgenshtern, Tzvi Raz, and Dov Dvir, Working Paper No 8/2005, Henry Crown Institute of Business Research, Israel. http://recanati-bs.tau.ac.il/Eng/?CategoryID=444&ArticleID=747

!  “An Empirical Validation of Software Cost Estimation Models,” Chris F. Kemerer, Research Contributions, Management of Computing, Communications of the ACM , may 1987, Volume 30, Number 5.

!  “Software Engineering Economics,” Barry Boehm, TRW Defense Systems, Redondo Beach, CA

!  “Software Cost Estimation Methods: A Review,” Vahid Khatibi and Dayang N. A. Jawawi, Journal of Emerging Trends in Computing and Information Sciences, Volume 2, No. 1, 2010.

!  “What We Do and Don’t Know About Software Development Effort Estimation,” Magne Jorgensen, IEEE Software, March 2014.

!  “Review of Studies on Expert Estimation of Software Development Effort,” M. Jorgensen, Simula Research laboratory, 2002.

34


+Bibliography

!  Protecting Software Development Projects Against Underestimation, Eduardo Miranda, Alain Abran, École de technologie supérieure - Université du Québec

!  “Improving Subjective Estimates Using Paired Comparisons,” Eduardo Miranda, IEEE Software, January/February 2001.

!  “Improving Estimations in Agile Projects: Issues and Avenues,” Luigi Buglione, Alain Abran, Software Measurement European Forum – SMEF2007, Rome (Italy), May 8-11, 2007.

!  “Estimating the Test Volume and Effort for Testing and Verification & Validation,” Alain Abran, Juan Garbajosa, , Laila Cheikhi1, First Annual ESA Workshop on Spacecraft Data Systems and Software - SDSS 2005, ESTEC, Noordwijk, European Space Agency, Netherlands, 17-20 October 2005.

35

control systems

Technology

niwot ridge llc

fixed operating condition

software engineering economics

closed loop control

planned completion date

open loop control

software development projects

loop control system