Exploration Systems Engineering: Technical Performance Measures Module

Technical Performance Measures Module

Exploration Systems Engineering, version 1.0

Exploration Systems Engineering: Technical Performance Measures Module 2

Module Purpose: Technical Performance Measures

♦  To define Technical Performance Measure (TPM).

♦  To show how TPM trends are used to predict delivered system performance.

♦  To describe how TPMs are used to monitor project progress and, when compared with standard resource contingency values, highlight when corrective action should be considered.

♦  To provide example TPMs from current NASA development projects.

Exploration Systems Engineering: Technical Performance Measures Module 3

Technical Performance Measures ♦  TPMs are measures of the system technical performance that

have been chosen because they are indicators of system success. They are based on the driving requirements or technical parameters of high risk or significance - e.g., mass, power or data rate.

♦  TPMs are analogous to the programmatic measures of expected total cost or estimated time-to-completion. There is a required performance, a current best estimate, and a trend line.

♦  Actual versus planned progress of TPMs are tracked so the systems engineer or project manager can assess progress and the risk associated with each TPM.

♦  The final, delivered system value can be estimated by extending the TPM trend line and using the recommended contingency values for each project phase.

♦  The project life trend-to-date, current value, and forecast of all TPMs are reviewed periodically (typically monthly) and at all major milestone reviews.

Exploration Systems Engineering: Technical Performance Measures Module 4

Tracking Technical Performance Measures

♦  Tracking TPMs and comparing them against typical resource growth provides an early warning system designed to detect deficiencies or excesses.

♦  Contingency allocations narrow as the design matures. ♦  TPMs that violate their contingency allocations or have trends that do not

meet the final performance should trigger action by the systems engineer.

Mass Allocation

Mas

s

Concept CDR PDR Test Time

Launch

35%

20%

15%

5%

Mass Contingency Plan

2%

Contingency violated, decisions are needed! Is the trend dependable and no action is needed? Act now to avoid more drastic action in the future?

Current Best Estimate Trend

Today

Exploration Systems Engineering: Technical Performance Measures Module 5

Chandra Mass TPM System Requirements Review to Launch

Exploration Systems Engineering: Technical Performance Measures Module 6

Design Contingencies

♦  Design contingencies are largest during concept exploration and uniformly shrink as the project matures. For example, mass contingencies are typically 35% at SRR, 20% at PDR, 15% at CDR and 2% at the launch readiness review.

♦  Why? Contingencies are used to account for development risks, interface uncertainties, and less than perfect design fidelity. As the design becomes more established and the team has greater confidence in their estimates for resource use or system performance, less contingency is needed.

♦  The trends of past, successful projects have been used to create guidelines for new projects.

♦  Why not carry even more contingency? Say 50% mass contingency at PDR to cover an even greater range of possible risks against system mass. With greater contingencies there is less allocation for the design - greater contingencies make the design problem harder. So there is a balance between contingency for risk management and allocation for design flexibility.

Exploration Systems Engineering: Technical Performance Measures Module 7

Contingency Guidelines for Common TPMs For Different Project Phases

Exploration Systems Engineering: Technical Performance Measures Module 8

JWST Key Technical Performance Measures

♦  Observatory Mass Margin ♦  Observatory Power Margin ♦  Observing Efficiency ♦  OTE Wave-front Error ♦  Wave-front Error Stability ♦  Strehl Ratio ♦  Sensitivity ♦  Image Motion ♦  Stray Light Levels ♦  Cryogenic Thermal Margins ♦  Commissioning Duration ♦  Data Volume / Link Margin ♦  Momentum Acceleration

James Webb Space Telescope (JWST)

Exploration Systems Engineering: Technical Performance Measures Module 9

JWST TPM - Mass

Exploration Systems Engineering: Technical Performance Measures Module 10

JWST TPM – Mass Reserve

Exploration Systems Engineering: Technical Performance Measures Module 11

265 265 260 244

614590 590 592592 559 549 549549 546 552 552552 556 556 556549 549 549459459

555 555 555555 555 555 555

248 248

47 47 47 46

111112 112128 128 128126 126 126 110107 107 107 106106 106 104 10410485 85

135135 135 135 135138 138

45 45108 108 69 86

15 38 38 20 20 53 65 65 65 84 81 81 81 78 78 78 87 87 87196 196

50 50 50 50 50 47 47

17 17

400 400

0 030807 807

802 802

859 864 864 867867 879 840 840840 840 841 841 841 841841 841864 864 864728 713 713713 728728 706 657 657657 657

158 158154 154

146 147147 143 143 144173 172 172 172169 169 169 169170 170 162 162162

128 125 125125 120 120 115104 104 104 104135 135144 144

95 89 89 90 0 0 64 65 65 65 67 67 67 67 66 66 41 41 41

26 44 44 44 34 34 61 121 121121 12133 33 33 33

33 33 33 3333 33 33 33 33 33 33 33 33 33 33 33 33 33 33

33 33 33 33 33 33 33 33 33 33 33

566 566 610398

34189189 189279 266 212 212 212 212212 212 212 212212 212 196 196196

169 169 16916955 55 55 55 55 154 154

30

0

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

1/1/

05

2/1/

05

3/1/

05

4/1/

05

5/1/

05

6/1/

05

7/1/

05

8/1/

05

9/1/

05

10/1

/05

11/1

/05

12/1

/05

1/1/

06

2/1/

06

3/1/

06

4/1/

06

5/1/

06

6/1/

06

7/1/

06

8/1/

06

9/1/

06

10/1

/06

11/1

/06

12/1

/06

1/1/

07

2/1/

07

3/1/

07

4/1/

07

5/1/

07

6/1/

07

7/1/

07

8/1/

07

9/1/

07

10/1

/07

Wat

ts

ISIM Estimated ISIM Contingency ISIM Reserve Cryocooler EstimatedCryocooler Contingency Cryocooler Reserve Spacecraf t Estimated Spacecraf t ContingencySpacecraf t Reserve OTE Estimate OTE Contingency OTE Reserve5 Y r Management Reserve

JWST TPM – Power!

Notes: 5/05: ISIM allocation changed to 740 W 12/06: Power Margin being carried as Load Margin not Solar Array Margin (Golden Rules Compliance) 4/07: Solar Array Capability decrease due to 1 wing baseline 8/07: Cryocooler separated from ISIM, Solar Array Capability increased

6 year Power System Capability = 1826 Watts Spacecraft + OTE Allocation (882 + 50) = 932 Watts ISIM + Cryocooler Allocation (310+430) = 740 Watts Power Margin (Estimate vs. Allocated) = 25 %

Exploration Systems Engineering: Technical Performance Measures Module 12

Module Summary: Technical Performance Measures

♦  TPMs are measures of the system technical performance that have been chosen because they are indicators of system success.

♦  The trends of past, successful projects have been used to create contingency guidelines for new projects.

♦  Tracking TPMs and comparing them against typical resource growth provides an early warning system designed to detect deficiencies or excesses.

♦  TPMs that violate their contingency allocations or have trends that do not meet the final performance should trigger action by the systems engineer.

♦  The final, delivered system value can be estimate by extending the TPM trend line and using the recommended contingency values for each project phase.

♦  There is a balance between contingency for risk management and allocation for design flexibility. This balance is apparent since contingency allocations shrink as designs mature.

Exploration Systems Engineering: Technical Performance Measures Module

Backup Slides for Technical Performance Measures Module

Exploration Systems Engineering: Technical Performance Measures Module 14

Technical Performance Measures

♦  TPM Basics •  Parameter for meeting key requirements and constraints.

•  Sound engineering parameter that is always tracked regardless of mission, such as mass margin or milestone achievements.

•  TPMs are usually tracked over the development life cycle of a project.

•  TPM trends over time usually compare a planned profile with the actual profile…planning is very important in order to meet specified targets.

•  TPMs are usually reported monthly or quarterly in management/engineering status meetings.

♦  TPM Sources •  Responsible NASA Center guidance (e.g., GSFC STD-1000 “The

Golden Rules”) •  Industry Practices •  Mission-specific risk assessments

Exploration Systems Engineering: Technical Performance Measures Module 15

JWST TPM - Mass

265 265 260 244

614590 590 592592 559 549 549549 546 552 552552 556 556 556549 549 549459459

555 555 555555 555 555 555

248 248

47 47 47 46

111112 112128 128 128126 126 126 110107 107 107 106106 106 104 10410485 85

135135 135 135 135138 138

45 45108 108 69 86

15 38 38 20 20 53 65 65 65 84 81 81 81 78 78 78 87 87 87196 196

50 50 50 50 50 47 47

17 17

400 400

0 030807 807

802 802

859 864 864 867867 879 840 840840 840 841 841 841 841841 841864 864 864728 713 713713 728728 706 657 657657 657

158 158154 154

146 147147 143 143 144173 172 172 172169 169 169 169170 170 162 162162

128 125 125125 120 120 115104 104 104 104135 135144 144

95 89 89 90 0 0 64 65 65 65 67 67 67 67 66 66 41 41 41

26 44 44 44 34 34 61 121 121121 12133 33 33 33

33 33 33 3333 33 33 33 33 33 33 33 33 33 33 33 33 33 33

33 33 33 33 33 33 33 33 33 33 33

566 566 610398

34189189 189279 266 212 212 212 212212 212 212 212212 212 196 196196

169 169 16916955 55 55 55 55 154 154

30

0

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

1/1/

05

2/1/

05

3/1/

05

4/1/

05

5/1/

05

6/1/

05

7/1/

05

8/1/

05

9/1/

05

10/1

/05

11/1

/05

12/1

/05

1/1/

06

2/1/

06

3/1/

06

4/1/

06

5/1/

06

6/1/

06

7/1/

06

8/1/

06

9/1/

06

10/1

/06

11/1

/06

12/1

/06

1/1/

07

2/1/

07

3/1/

07

4/1/

07

5/1/

07

6/1/

07

7/1/

07

8/1/

07

9/1/

07

10/1

/07

Wat

ts

ISIM Estimated ISIM Contingency ISIM Reserve Cryocooler EstimatedCryocooler Contingency Cryocooler Reserve Spacecraf t Estimated Spacecraf t ContingencySpacecraf t Reserve OTE Estimate OTE Contingency OTE Reserve5 Y r Management Reserve

Exploration Systems Engineering: Technical Performance Measures Module 16

JWST TPM – Strehl Ratio!

Science Requirement: L1-14 The Observatory, over the field of view (FOV) of the Near-Infrared Camera (NIRCam) shall be diffraction limited at 2 micrometers defined as having a Strehl Ratio greater than or equal to 0.8.

Definition: The modern definition of the Strehl ratio is the ratio of the observed peak intensity at the detection plane of a telescope or other imaging system from a point source compared to the theoretical maximum peak intensity of a perfect imaging system working at the diffraction limit. This is closely related to the sharpness criteria for optics defined by Karl Streh. Unless stated otherwise, the Strehl Ratio is usually defined at the best focus of the imaging system under study.

Exploration Systems Engineering: Technical Performance Measures Module 17

JWST TPM – Wavefront Error!