edar training fall 2015

Information Technology Solution Engineering

Engineering Decision Analysis and Resolution (EDAR)

Spring 2015

Rodney DuBosePatrick McQuinn

Engineering Decision Analysis and Resolution | Solution Engineering

Agenda

• High Level Overview• Procedure Overview• Exercise• Contact

Engineering Decision Analysis and Resolution | Solution Engineering2

EDAR High Level Overview

• Training Objective• EDAR Purpose• When EDAR Is Not Done Well• EDAR High Level Overview

• Relationship to other processes• EDAR Specific Goal• EDAR Specific Practices• EDAR Work Products


Training Objective

• To increase understanding of the Engineering Decision Analysis and Resolution (EDAR) procedure

• Institutionalize repeatable engineering design and development processes and procedures


EDAR Purpose

The purpose of EDAR is to analyze possible decisions using a formal evaluation procedure that evaluates identified alternatives given established criteria.


When EDAR Is Not Done Well• It is unclear who is authorized to make what decisions.

• Decisions are made subjectively.

• Rationale is unavailable when needed to understand an earlier decision.

• Too few choices are considered for major decisions.

• Missing a more optimal solution costs time, money, credibility, and perhaps even the whole project.


High Level OverviewRelationship to Other Processes

• EDAR is a Technical Solution (TS) Procedure

• EDAR is a CMMI Support Process Area Process

• The CMMI Support Process Area covers the activities that support product development and maintenance.


High Level OverviewRelationship to Other Processes


EDAR Specific Goal


SG1Evaluate Alternatives

Decisions are based on an evaluation of Alternatives using established criteria.

EDAR Specific Practices


SG1

Evaluate Alternatives

SP 1.1 Use The EDAR GuidelineSP 1.2 Establish Evaluation CriteriaSP 1.3 Identify Alternative SolutionsSP 1.4 Select Evaluation MethodsSP 1.5 Evaluate Alternative SolutionsSP 1.6 Select Solutions

EDAR Work Products


Problem Statement

Use the EDAR Guidelines

SP 1.1

Recommended Solutions

SelectSolutions

SP 1.6

Evaluation Results

Evaluate Alternative Solutions

SP 1.5

Methods

Select Evaluation Methods

SP 1.4

Proposed Alternatives

Identify Alternative Solutions

SP 1.3

Criteria

Establish Evaluation

Criteria

SP 1.2

Agenda



EDAR Process Steps

1. Use the EDAR Guideline

2. Establish Evaluation Criteria

3. Identify Alternative Solutions

4. Select Evaluation Methods

5. Evaluate Alternative Solutions

6. Select Solutions


EDAR Step 1 ActivitiesUse the EDAR Guidelines


Role Description

Definition of Responsibility

Lead analyst •Obtain the EDAR Guidelines.

•Determine whether or not the stated problem falls within the scope of the EDAR Guidelines.

•Communicate the decision to conduct a formal evaluation.

Start Obtain the EDAR Guidelines.

A1

Use the EDAR Guidelines.

A2

Communicate the decision.

A3

Problem within

Guideline scope?

StopNo

Yes

1

ProblemStatement

EDAR Step 1 ActivitiesUse the EDAR Guidelines


Activity Steps RolesA1: Obtain the EDAR Guidelines

1. Download the EDAR Guidelines

Lead Analyst

A2: Use the EDAR Guidelines

1. Determine whether or not the stated problem falls within the scope of the EDAR Guidelines.

2. If not then stop.3. If yes, then proceed to step A3.

A3: Communicate the Decision

1. Communicate the decision to conduct a formal evaluation.

EDAR Guidelines, Summary

1. General

2. Insertion of New Technologies

3. Acquisition of New Hardware/Software Products or New Tools


EDAR Guidelines, cont., 1/4

1. General Guidelines – Any project engineering effort may employ formal evaluation methods and is welcome to use the EDAR procedure by accessing the IT PAL and executing the EDAR Procedure that is associated with Engineering Technical Solution. The procedure is generic enough that it will apply to any need to conduct a formal evaluation of technical solution alternatives. Each use of the EDAR procedure requires that evaluation criteria be established for the specific issue being evaluated. It is well known that evaluation criterion will vary depending on the specifics of the alternatives being evaluated.



2. Guidelines for Insertion of New Technologies – Whenever new technologies are being considered for introduction into the IRS computing environments, a formal evaluation is required. The function, organization, or project considering the new technology is required to engage Enterprise Architecture who will lead, participate, or give guidance in the conduct of the EDAR activities. In the event the evaluation will also select new HW/SW products or tools, Solution Engineering will also be engaged to either participate in the EDAR activities or give guidance as to the follow-on conduct of a Product Integration activity.


3. Guidelines for Purchase of New Hardware/Software Products or New Tools – Whenever new HW/SW products or tools are being considered for purchase and introduction into IRS computing environments, a formal evaluation is required. The function, organization, or project is required to engage Enterprise Architecture who will advise as to whether or not they should be involved in the EDAR activities. Solution Engineering must also be engaged for participation in the EDAR activities and to determine whether or not a Product Integration activity will be conducted.



EDAR Step 2 ActivitiesEstablish Evaluation Criteria


Role Description


Lead Analyst • Define the criteria for evaluating alternative solutions for the specific problem to be solved.

• Define the scale of relative importance for ranking evaluation criteria.

• Assign the weight of each criterion.• Document the rationale for the selection

of evaluation criteria.

Roles and Responsibilities 1

2

Document the rationale for the

selection of evaluation criteria.

A7

Assign the weight of each criterion.

A6

Define the scale of relative importance for

ranking evaluation criteria.

A5

Define the criteria for evaluating alternative

solutions for the specific problem to be

solved.

A4

EDAR Step 2 ActivitiesEstablish Evaluation Criteria


Activities Steps Roles

A4: Define the criteria for evaluating alternative solutions for the specific problem to be solved.

1. Define criteria for evaluation of the problem to be solved.

2. Consult with subject matter experts for project, programmatic, security, and vendor aspects as applicable.

3. Document the evaluation criteria.

Lead Analyst

A5: Define the scale of relative importance for ranking evaluation criteria.

1. Define a numerical scale to be applied in assigning relative importance to the criteria.

2. Define the numerical scale from lower relative importance to higher relative importance. i.e. lower numbers to be applied to criteria of lesser importance, higher numbers to criteria of greater importance.

A6: Assign the weight of each criterion.

1. Assign a relative weight percentage to each criterion.NOTE: the sum of all criteria weights must equal 100%.

A7: Document the rationale for the selection of evaluation criteria.

1. Document the rationale for criteria selection.

EDAR Step 3 ActivitiesIdentify Alternative Solutions


Role Description


Analysts • Discover candidate alternative solutions.

• Document proposed alternatives.

Roles and Responsibilities2

3

Discover candidate alternative solutions.

A8

Document proposed alternatives.

A9

EDAR Step 3 ActivitiesIdentify Alternative Solutions



A8: Discover candidate alternative solutions.

1. Perform discovery to uncover similar previous alternative analyses both inside and outside the organization to expose alternative candidates and barriers to implementation.

2. Discover relevant existing trade studies and lessons learned from previous formal decisions.

3. Identify priorities of relevant stakeholders and technical/logistics challenges using evaluation criteria.

4. Conduct interviews and workshops with relevant stakeholders to solicit and uncover candidate alternatives.

5. Use available research tools (e.g. Forrester).

Analyst(s)

A9: Document proposed alternatives.

1. Document the proposed alternatives.

EDAR Step 4 ActivitiesSelect Evaluation Methods


Role Description


Lead Analyst

• Select the evaluation method(s).

• Determine the measures needed to support the evaluation method(s).

• Document the selected valuation method(s).

Roles and Responsibilities3

4

Select evaluation method(s).

A10

Determine the measures needed to support the evaluation

method(s).

A11

Document the selected evaluation method(s).

A12

EDAR Step 4 ActivitiesSelect Evaluation Methods



A10: Select evaluation method(s).

1. Select methods based on the following:• The problem statement.• The level of the availability of and confidence in

the information used to support the method.• The ability to focus on the issues at hand without

being overly influenced by side issues.

Lead AnalystA11: Determine the

measures needed to support the evaluation method(s).

1. Develop measures for each proposed method that will, when collected, aid in the selection of the most appropriate evaluation method(s) through the analysis of the method’s impact on cost, schedule, performance, and risks.

A12: Document the selected evaluation method(s).

1. Document the selected evaluation method(s) and supporting justification.

EDAR Step 5 ActivitiesEvaluate Alternative Solutions


Roles and Responsibilities Role Description Definition of ResponsibilityLead Analyst • Identify the Decision Making Authority

• Identify the stakeholders• Select key personnel (Analysts) to perform

the process• Distribute the formal report, if required

Analyst • Perform the evaluation and provide a recommended solution (one person or a team, one of whom is designated leader)

• Request additional resources if required.

Higher Authority • Allocate additional resources, if required• Discuss the EDAR results with the Lead

Analyst

Stakeholders • Provide information as requested by the EDAR team to perform the analysis of alternative solutions

Decision Making Authority

• Receive the documented evaluation results

4

5

Develop evaluation artifacts.

A14

Evaluate the assumptions related to the evaluation criteria and the evidence that support

the assumptions.

A15

Evaluate the identified alternatives.

A16

Document the evaluation results and recommend alternative

solutions.

A17

Distribute the evaluation results and

recommendations.

A18

Prepare for evaluation of proposed alternative solution(s).

A13

EDAR Step 5 ActivitiesEvaluate Alternative Solutions – 1/3



A13: Prepare to evaluate proposed alternative solutions.

1. Identify Analyst(s)2. Identify Stakeholders Lead Analyst

3. Acknowledge Roles Analyst(s)

Stakeholders

A14: Develop evaluation artifacts.

1. Populate the EDAR evaluation matrix with evaluation criteria, criteria weights, identified alternatives and selected method(s).

2. Develop man-hour estimates for development and operations.

3. Develop cost estimates for development and operations.

4. Develop expected technology end of life estimate.

Analyst(s)

A15: Evaluate the assumptions related to the evaluation criteria and the evidence that support the assumptions.

1. Develop assumptions and supporting arguments.

2. Document assumptions related to evaluation criteria.

Analyst(s)

EDAR Step 5 ActivitiesEvaluate Alternative Solutions (cont. 2/3)



A16: Evaluate the identified alternatives.

1. Using the method(s) selected, and evaluation criteria, evaluate the identified alternative solutions.

2. Request additional funding if necessary. 3. Assign raw scores using the numerical

scale developed in EDAR step 2 for each criterion relative to each alternative. Record scores in the evaluation matrix.

4. Calculate the value of each criterion relative to each alternative. (value=(raw score) *(criterion weight) Record values in the evaluation matrix.

5. Calculate the rank of each alternative solution by summing the values of each alternative’s criteria. Record ranks in the evaluation matrix.

Analyst(s)

6. Disposition requests for additional funding.

Higher Authority

EDAR Step 5 ActivitiesEvaluate Alternative Solutions (cont. 3/3)



A17: Document the evaluation results and recommend alternative solution(s).

1. Document the evaluation results. 2. Document the recommended

alternative solution(s). Analyst(s)

A18: Distribute evaluation results and recommendations.

1. Receive and distribute the completed EDAR evaluation documentation form.

Lead Analyst

2. Receive the evaluation results. Decision Making Authority

EDAR Step 6 ActivitiesSelect Solutions


Role Description



• Select solutions.• Document and

communicate the recommended solution.

Roles and Responsibilities

End

5

Select Solution(s).

A19

Document and communicate the recommended solution.

A20

EDAR Step 6 ActivitiesSelect Solutions



A19: Select solutions.

1.Examine the documented evaluation results and recommended solutions.2.Accept security risks associated and documented with the recommended solutions.3.Instruct the Lead Analyst to repeat EDAR steps required to mitigate identified risks or concerns.4.Select solutions.


A20: Document and communicate the recommended solutions.

1.Document the solutions selected.

Agenda



Exercise Problem StatementTo build and install a solution to the following:To build and install a solution to the following:

•Business NeedBusiness Need: To process paper checks in a timely basis.: To process paper checks in a timely basis.

•Business CaseBusiness Case: Every day, IRS Taxpayer Assistance Centers, Consolidated : Every day, IRS Taxpayer Assistance Centers, Consolidated Campuses, and Revenue Officers accept tax payments in the form of paper Campuses, and Revenue Officers accept tax payments in the form of paper remittances (i.e., checks, money orders). As these IRS entities are not equipped to remittances (i.e., checks, money orders). As these IRS entities are not equipped to fully process and deposit the remittance, so paper checks must be controlled, fully process and deposit the remittance, so paper checks must be controlled, packaged, and transported via overnight mail to a Submission Processing Center. packaged, and transported via overnight mail to a Submission Processing Center. This adds days to the time a payment can be deposited, and creates more employee This adds days to the time a payment can be deposited, and creates more employee effort than would otherwise be necessary. effort than would otherwise be necessary.

•Customer RequirementsCustomer Requirements::• Process Paper checks at the point of receipt.• Maintain a copy of the check and associated information in a central

repository.• Deposit funds.


Exercise: Procedure Step 1Exercise: Procedure Step 1Use the EDAR GuidelinesUse the EDAR Guidelines

Questions for determination:Questions for determination:•Does the Problem Statement indicate that a new technology Does the Problem Statement indicate that a new technology will be introduced into the IT enterprise? (Answer: NO)will be introduced into the IT enterprise? (Answer: NO)

•Does the Problem Statement indicate that new HW/SW will be Does the Problem Statement indicate that new HW/SW will be introduced into the IT enterprise? (Answer: NO)introduced into the IT enterprise? (Answer: NO)

•Does the Project Management believe that a EDAR is indicated Does the Project Management believe that a EDAR is indicated justified by the EDAR guideline General clause? (Answer: Yes)justified by the EDAR guideline General clause? (Answer: Yes)


Exercise: Procedure Step 2/3Exercise: Procedure Step 2/3Selection Criteria and Alternative Selection Criteria and Alternative SolutionsSolutions

Some questions for discussion:Some questions for discussion:• What Criteria should be used to select the approach used?What Criteria should be used to select the approach used?• What are some alternative approaches to meet the customer What are some alternative approaches to meet the customer

requirements?requirements?• Consider:Consider:• What technologies are Available?What technologies are Available?• What can be reused?What can be reused?• Operations Concept for approachOperations Concept for approach

• What are the alternative solutions?What are the alternative solutions?


Exercise: Procedure Step 2/3 Identify Criteria and Alternatives

• Derived Criteria:• Cost, Schedule, Risk, Performance, EA Compliance,

Maintainability• (NOTE: Criteria used in this exercise are arbitrary but

plausible.)• For the purpose of the exercise assume the approach uses the

following 3 alternatives:• Current System: (Keeping current system)• New Centralized System• Modify Existing ISRP System

• Make sure all requirements are allocated to each alternative


Weight the Evaluation Criteria

Overview• Weighting criteria involves assigning a ranking or relative

weight to each criterion to reflect its importance to the customer.

• NOTE: The term “weight” as used in the statistical evaluation of criteria is a verb, not a noun. This usage is common to the EDAR procedure and has become a best practice in the discipline.


Weight the Evaluation Criteria, Cont.

Analytic Hierarchy MethodThe Analytic Hierarchy Method (AHM) is a decision-making process

developed by Dr. Thomas L. Saaty, University of Pittsburgh, in the early 1980s. Rather than prescribing a "correct" decision, the AHM helps decision makers find one that best suits their goal and their understanding of the problem. It provides a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions. This section shows how to derive weighting factors (W) for the selection criteria using the Analytic Hierarchy Method. The Analytic Hierarchy Method is the preferred method for deriving weighting criteria and consists of the following steps that may be implemented on a standard PC-based spreadsheet.


Weight the Evaluation Criteria, Cont. Sliding Pair-Wise Comparison Matrix

• Perform Pair-Wise Comparison of Selection Criteria• The basis of the weighting process is to perform a pair-wise comparison of all of the selection criteria.

For example, suppose you have six selection parameters: cost, schedule, performance risk, EA compliance, and Maintainability. You can construct a 6x6 matrix as follows:


NOTE: 1.Number of Pair-Wise Comparisons = (N2 –N)/22.Number of Pair-Wise Comparisons increases exponentially with the number of criteria. (Motivation to keep the number of criteria to a minimum!)


•Assess the relative priority of one parameter to the other for each pair, using the sliding pair-wise comparison scale shown below.

Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 2Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 3Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 1 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 3Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 2 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 4Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 3 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 5Criteria 4 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6Criteria 5 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Criteria 6


• Assess the relative priority of one parameter to the other for each pair, using the sliding pair-wise comparison scale shown below.



• The numerical values on the scale have the following meanings:1: Equal importance2-4: More important5-7: Much more important7-9: Extremely more important

• If the two parameters in a pair are of equal importance, designate the 1 on the scale. If one is more important than the other, designate the appropriate number (as defined above) on the side where the more important parameter exists. For instance, if cost is more important than schedule, designate the 4 nearest the cost, as shown in the next slide.




Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 ScheduleCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 PerformanceCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 PerformanceSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskPerformance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.EA Compl. 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.

The completed sliding pair-wise comparison matrix:

Weight the Evaluation Criteria, Cont.The Priority Matrix


Cost Schedule Performance Risk EA Compl. Maintain.

Cost 1 4 3 4 1/5 1/4

Schedule 1/4 1 1/5 1/4 1/5 4

Performance 1/3 5 1 5 1/5 5

Risk 1/4 4 1/5 1 1/5 1

EA Compl. 5 5 5 5 1 5

Maintain. 4 1/4 1/5 1 1/5 1

Complete the Priority MatrixWhen the pair-wise comparisons have all been completed, enter the results into the priority matrix using the following rules: *The cells along the diagonal of the matrix (pairing a parameter with itself) are always assigned a value of 1 .*If the two parameters are rated as equally important, enter the value 1 into the cell;*If the parameter along the row in the matrix is more important than the parameter in the column, enter the value (2 to 9) into the cell;*If the parameter in the column is more important than the parameter in the row, enter the reciprocal of the value (1/2 to 1/9) into the cell.

Weight the Evaluation Criteria, Cont.The Priority Matrix


Cost Schedule Performance Risk EA Compl. Maintain.Cost 1 4 3 4 1/5 1/4Schedule 1/4 1 1/5 1/4 1/5 4 Performance 1/3 5 1 5 1/5 5 Risk 1/4 4 1/5 1 1/5 1 EA Compl. 5 5 5 5 1 5 Maintain. 4 1/4 1/5 1 1/5 1 Column Total 10.83 19.25 9.60 16.25 2.00 16.25

Normalize the Priority MatrixNormalization is the statistical method used to equalize the scale used to measure each criteria parameter. In this case the scale chosen for normalization is 1 or 100%. To normalize the priority matrix, sum each column as shown below.

Weight the Evaluation Criteria, Cont.Calculate the Weights


Cost Schedule Performance Risk EA Compl. Maintain. WeightCost 0.092 0.208 0.313 0.246 0.100 0.015 16.24%Schedule 0.023 0.052 0.021 0.015 0.100 0.246 7.62%Performance 0.031 0.260 0.104 0.308 0.100 0.308 18.50%Risk 0.023 0.208 0.021 0.062 0.100 0.062 7.91%EA Compl. 0.462 0.260 0.521 0.308 0.500 0.308 39.29%Maintain. 0.369 0.013 0.021 0.062 0.100 0.062 10.44%

Normalization 1.000 1.000 1.000 1.000 1.000 1.000 100.00%

• Next, divide the value in each cell by its column total. • For example, in the cell Cost:Schedule: (4)/(19.25) = 0.208

• Compute Weighting Factors• The weighting factor (W) for each parameter may now be computed by summing along

the parameter's row in the matrix and dividing by the number of selection parameters, N = 6 (in this case).

• W (cost) = (0.092 + 0.208 + 0.313 + 0.246 + 0.1 + 0.15)/6 = 0.1624• W (schedule) = (0.023 + 0.052 + 0.021 + 0.015 + 0.1 + 0.246)/6 = 0.0762• W (performance) = (0.031 + 0.26 + 0.104 + 0.308 + 0.1 + 0.308)/6 = 0.185• etc…

• The sum of the weighting factors should add up to the whole number one. [NOTE: Multiply the weighting factors by 10 if you want them on a 10-point scale or 100 for percentage.]

Weight the Evaluation Criteria, Cont. Checking the Consistency


Check Consistency• Occasionally, the pair-wise comparisons may not be consistent. Instead of the sample values used

above, suppose that you obtained the following results:Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Schedule

Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Performance

Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskCost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Cost 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.

Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Performance

Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskSchedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Schedule 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 RiskPerformance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Performance 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 EA Compl.Risk 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.EA Compl. 9 8 7 6 5 4 3 2 1 2 3 4 5 6 7 8 9 Maintain.

• Here, cost is more important than schedule and equally as important as performance. Schedule is also rated as equally as important as performance. This result seems inconsistent. If Cost > Schedule and Schedule = Performance, then logically you should have Performance > Schedule. Instead, you have Schedule = Performance. Some inconsistency is to be expected when comparing such diverse parameters as schedule, performance, and cost, and reflects subjective human judgment, which cannot be held to such mathematical precision as: if a = b and b = c, then a = c. The question is how such inconsistencies affect the derived weighting factors (W).

Identify the Alternatives


Discover candidate alternative solutions.

•Current System

•New Centralized System

•Modify Existing ISRP System

Select Evaluation Methods• Overview• Using the previously identified alternative solutions, select the evaluation methods to be

used during the evaluation step. The below example matrix is intended to aid in the selection of methods.


Evaluation Method

Alt 1 Current ISRP System

Alt 2 New Centralized System

Alt 3 Modified ISRP System

Cost Schedule Perf. risk EA Compl. Maintain.

• Example evaluation method types for typical IT Problems:• Testing• Modeling and simulation• Engineering Studies• Manufacturing studies• Extrapolations based on field experience and prototypes• Judgment provided by an expert or group of experts (e.g. Forrester)

Scoring the Alternatives• Overview• The purpose of this exercise is to present a method for scoring the

alternative solutions given that the units and ranges of the evaluation criteria vary widely.

• The evaluation criteria consist of a diverse set of parameters, such as cost, schedule, and performance. The units of these evaluation criteria vary widely. For instance, schedule may be expressed as time consumed by humans in hours, while cost is given in dollars.

• The method used in this exercise is called “Normalization”


Scoring the Alternatives, Cont. Calculate the Score

• Alternatives Scoring Method: NormalizationNormalization involves taking a ratio between the value of a given alternative and

the alternative with the best value. If the best value is defined as a maximum, as for schedule in the table below, the ratio is given by the following formula. In this case, alternative 1 has the best value (200 hours).

• If the best value is defined as a minimum, as for cost in the table, the ratio is given by the following formula. For cost, alternative 1 has the best value ($600K).

S (alternative n) = (alternative n value)/(best alternative value) • If the best value is defined as a maximum, as for performance in the table, the

ratio is given by the following formula. For cost, alternative 1 has the best value ($600K). S (alternative n) = (best alternative value)/(alternative n value)

• In either case, the alternative with the best value will receive a score of 1, and all other alternatives will receive a score between 0 and 1. If you wish to operate on a 10-point scale, multiply the normalized score by 10.


Scoring the Alternatives, Cont. Calculate the Score


Selection Criteria

Alt 1 Current ISRP System

Alt 2 New Centralized

SystemAlt 3

Modified ISRP System

Value (S) Score Value (S) Score Value (S) ScoreCost 600K$ 1 2.2M$ 0.27 1.2M$ 0.5Schedule 200hrs. 1 2000hrs. 0.1 1000hrs. 0.2Performance 21Mbps 0.84 25Mbps 1 22Mbps 0.88Risk 300K$ 1 1.4M$ 0.21 700K$ 0.43EA Compl. 200hrs. 1 2000hrs 0.1 1600hrs. 0.13Maintain. 2.6M$/yr 0.54 1.4M$/yr. 1 1.7M$/yr. 0.82

Alternatives Analysis Matrix• Alternatives Analysis Matrix• The Alternatives Analysis Matrix presents all of the information required

by the Decision Making Authority to select a solution for recommendation. Include this matrix in the EDAR Report.


Alternatives Analysis

CriteriaWeigh

t

Alt 1 Alt 2 Alt 3Scor

e ValueScor

e ValueScor

e Value

Cost W1 S11 V11=(W1*S11) S21 V21=(W1*S21) S31 V31=(W1*S31)

Schedule W2 S12 V12=(W2*S12) S22 V22=(W2*S22) S32 V32=(W2*S32)

Performance W3 S13 V13=(W3*S13) S23 V23=(W3*S23) S33 V33=(W3*S33)

Risk W4 S14 V14=(W4*S14) S24 V24=(W4*S24) S34 V34=(W4*S34)

EA Compl. W5 S15 V15=(W5*S15) S25 V25=(W5*S25) S35 V35=(W5*S35)

Maintain. W6 S16 V16=(W6*S16) S26 V26=(W6*S26) S36 V36=(W6*S36)

Rank: R1=∑ V11:V16 R2 =∑ V21:V26 R3 =∑ V31:V36

Normalized Rank: = R1/ ∑ (R1:R3)*100% = R2/ ∑( R1:R3)*100% = R3/ ∑( R1:R3)*100%

Alternatives Analysis Matrix,Rank the Alternatives


Alternatives Analysis

Criteria WeightAlt 1

Current ISRP SystemAlt 2

New Centalized SystemAlt 3

Modified ISRP System

Score Value Score Value Score ValueCost 0.1624 1 0.1624 0.27 0.043848 0.5 0.0812Schedule 0.762 1 0.762 0.1 0.0762 0.2 0.1524Performance 0.185 0.84 0.1554 1 0.185 0.88 0.1628Risk 0.791 1 0.791 0.21 0.16611 0.43 0.34013EA Compl. 0.3929 1 0.3929 0.1 0.03929 0.13 0.051077Maintain. 0.1044 0.54 0.056376 1 0.1044 0.82 0.085608

Rank: 2.320076 0.614848 0.873215Normalized Rank: 61% 16% 23%

Select Solutions

• Select solutions. 1. Current ISRP System (Rank 61%)

2. Modified ISRP System (Rank 23%)

3. New Centralized System (Rank 16%)

• Document and communicate the recommended solutions.



Questions


Contact for EDAR

edar training fall 2015

Documents