weiler slideshow

IT Acquisition Advisory Council (IT-AAC)A non-partisan think tank, 501.C3

Roadmap for Sustainable IT Acquisition Reform

Leveraging non-traditional expertise and benchmarked standards of practices That exceed CCA & Section 804 Mandates

Honorable John Grimes, Former OSD CIOJohn Weiler, Managing Director, [email protected]

Dr. Marv Langston, IT-AAC Vice Chair [email protected] Carroll, IT-AAC Vice Chair

www.IT-AAC.org 703 768 0400

904 Clifton Drive * Alexandria * Virginia 22308

www.IT-AAC.org * (703) 768-0400

mailto:[email protected]

http://www.it-aac.org/

501.C Non-Profit Research Institute IT-AAC Proprietary © 2008- 2010 All Rights Reserved703-768-0400 www.IT-AAC.org

™

2

Senior Exec Briefing SummaryAssuring Business Value from every IT $ Spent

Purpose

Today's Situation

Our Proposal to Assist

Way Forward Recommendation

Predictable Outcomes

“Together, these steps will help to catalyze a fundamental reform of Federal IT, which is essential to improving the effectiveness and efficiency of the Federal Government” White House, OMB Director


™

3

Think Tank Purpose

Acquisition Practitioners and workforce will need commercial methods, access to deep industry expertise and emerging standards of practice to overcome common failure patterns and cultural impediments that have prevented previous attempts to achieve following objectives:

Speed -- achieve 6-12 month cycle times vice 7-8 years (early pilots prove this is possible)

Incremental development, testing, and fielding -- vice one "big bang"

Actionable Requirements -- Sacrifice or defer customization for speed and COTS/OS utilization - Leverage established standards of practice and open modular platforms

Meet DoD's wide-range IT needs -- from modernizing C2 to updating word processing software

Focused on Outcomes and Operational Effectiveness - Health IT, InfoSharing, Cyber Security, Consolidated IT Infrastructure, Business Systems

To provide the Decision Makers with an alternative set of resources and expertise needed to guide the establishment of a “best in class” set of IT Acquisition Processes and Governance Structure.

“You can’t solve today’s problems with the same thinking that got you there” Albert Einstein


™

4

Information Technology Evolution

Info

rmat

ion

Dri

ven

Cap

abili

ty

• Central computer center, slow turn around• One size fits all• Limited reuse of application modules

1950 1960 1970 1980 1990 2000 2010 2020

1. Centralized - Mainframe

2. Client/Server - Decentralized

3. Internet - Cloud

• PC enabled and network• Software distributed in both server and client computers

• Heavy focus on software development and point to point integration

• Virtualized compute; global network enabled, plug & play

• IT Infrastructure decoupled from Applications

• COTS & OSS Integration, Software as a Service

Adding functional capability has become easier with each new wave

We are in early stages of Wave 3 information technology

Mainframe and Client-Server waves remain in place

Waves represent many co-dependent technologies, matured over time

But enterprise infrastructure gaps & vulnerabilities have become more critical

DoD is using 1970s acquisition processes; to acquire Wave 3 IT capability

IT-AAC understands IT Acquisition Dilemma Wave 3 Solutions can’t be acquired using MilSpec processes…


™

5

Understanding the Many Dimensions of the IT Acquisition Lifecycle

IT Acquisition Building Blocks:

Governance and Oversight: how an enterprise supports, oversees and manages IT programs and on-going portfolio. SOA as defined in the commercial market is governance tool not technology. DoD5000 and BCL represent the current approaches.

Decision Analytics: enables effective Program Management and Value Stream Analysis execution. As most of these sub-processes are designed to improve decision making, a relative new discipline has evolved (since 86), that addresses the human and cultural challenges in decision making. Decision Analytics is the discipline of framing the essence and success criteria of each gate in the acquisition lifecycle. It brings focus to the high risk areas of a program, and reduces analysis/paralysis.

Requirements Development: Actionable requirements must be constrained by the realm of the possible. With pressures to do more with less, we must embrace mechanisms that force a relative valuation/impact of the gap/capability, with clearly defined outcomes

Architecture: This is one of the most critical elements of the acquisition lifecycle, as it should represent all stake holder agreements. The market embrace of SOA is not about technology, but a refocusing of the EA on service level management and data. A good architecture is a lexicon that links requirements, technologies and acquisition strategy.

Technology Assessment: Understanding the limitation of technology early in the process is key. Without a clear view of the “realm of the possible” validated by real world results, we often find ourselves in high risk areas and over specification. Market research must be done early to help users constrain requirements and embrace the inherent business practices that codify. Recognizing that 70% make up of every IT application is vested in IT infrastructure (netcentric, cloud, SOA), it is critical to establish a common infrastructure/infrastructure standard by which all applications can share. The most prolific is ITIL to date.

Business Case Analysis: Demonstrating the business value of technology investments, based on evidenced based research and lifecycle cost. This is a core requirement of Clinger Cohen Act.

Performance Based Acquisition and Metrics: Software as a Service and SOA portent a new dynamic for acquisition of IT (health IT, cyber, business systems), that brings focus to Service Level Agreements (SLAs), Software as a Service (SaaS) and SL Management. If the previous activities do not directly feed the acquisition strategy or provide mechanisms for contractor accountability, all is lost.

“IT Reform is about Operational Efficiency and Innovation”


™

6

Today’s Situation -- as highlighted by the HASC Panel on Defense Acquisition Reform

Studies of both commercial and government IT projects have found some disturbing statistics;

Only 16% of IT projects are completed on time and on budget.

31% are cancelled before completion.

The remaining 53% are late and over budget, with the typical cost growth exceeding the original budget more than 89%.

Of the IT projects that are completed, the final product contains only 61% of the originally specified features.

As was pointed out in testimony before the Panel, the traditional defense acquisition process is “ill-suited for information technology systems. Phase A is intended to mature technology; yet information technologies are now largely matured in the commercial sector”. Weapon system acquisition processes are often applied to IT systems

acquisition, without addressing unique aspects of IT. “the weapon systems acquisition process is optimized to manage production risk and does not really fit information technology acquisition that does not lead to significant production

quantities.” Defense Acquisition Panel, HASC


™

7

Understanding IT Acquisition Reform Laws

MilSpec must give way to Industry Best Practices: SOA, Agile, COTS

HR5136: ‘‘Implementing Management for Performance and Related Reforms to Obtain Value in Every Acquisition’’. Requires:(1) Determine clear performance metrics for specific programs from the start;(2) Foster an ongoing dialogue during the technology development process between the system developers and thewarfighters;(3) Promote an open architecture approach that allows for more modularization of hardware and software;(4) Develop a plan for how to strengthen the IT acquisition workforce;(5) Implement alternative milestone decision points that are more consistent with commercial product development for IT;(6) Develop a process for competitive prototyping in the IT environment;(7) Develop a new test and evaluation approach that merges developmental and operational testing in a parallel fashion;(8) Place greater emphasis on the up-front market analysis; and(9) Conduct a rigorous analysis of contracting mechanisms and contract incentive

Clinger Cohen Act Requires: (1) Streamline the IT Acquisition Process(2) Change business processes (BPR), not COTS(3) Favor COTS/OSS over custom development. (4) Build business case and acquire based objective assessment criteria(5) Use architecture for driving investment decisions(6) Favor standards and best practices over MilSpec approaches


™

8

Congressional Action to Reform IT Acquisition: 2009 NDAA Sec804

“The Secretary of Defense shall develop and implement a new acquisition process for information technology systems. The acquisition process developed and implemented pursuant to this subsection shall, to the extent determined appropriate by the Secretary--

be based on the recommendations in chapter 6 of the March 2009 report of the Defense Science Board Task Force on Department of Defense Policies and Procedures for the Acquisition of Information Technology; and

be designed to include--

– early and continual involvement of the user;

– multiple, rapidly executed increments or releases of capability;

– early, successive prototyping to support an evolutionary approach; and

– a modular, open-systems approach”

Congress and DSB made these recommendations based on early adoptions by AF, Navy, USMC, and BTA of alternative methods like the Architecture Assurance Method (AAM), a risk management framework designed to improve decision making and assure stake holder value. AAM incorporates by reference industry best practices like SOA, ITIL, and Evidenced Based Research.


™

9

OMB Guidance on IT Reformfrom OMB’s 25 Point Plan

Align the Acquisition Process with the Technology Cycle

Point 13. Design and develop a cadre of specialized IT acquisition professionals .

Point 14. Identify IT acquisition best practices and adopt government-wide.

Point 15. Issue contracting guidance and templates to support modular development

Point 16. Reduce barriers to entry for small innovative technology companies"


™

10

OMB’s View of Federal ITFundamentally Broken!


™

11

Sources of EvidenceFailure to fix IT is costing the tax payer $40Billion a Year

“We are buying yesterday’s technology tomorrow in the rare instances we are successful ”

DSB IATF: “DoD reliance on FFRDCs is isolating it from sources of new technologies, and will hinder the departments ability to get the best technical advise in the future”

AF Science Advisory Board 2000: PMs need greater access to real world lesson learned and innovations of the market to mitigate risk and cost overruns. PMs frequently enter high risk areas due to limited access to lessons learned from those who have already forged ahead.

CMU SEI Study 2004: The DoDAF alone is not effective for IT architectures, lacks business view, performance metrics or means of avoiding over specification. DoDAF (C4ISR) was developed by Mitre and IDA in 1986 to provide DoD with a systems engineering documentation tool for existing system implementations. 2009 NDAA Sec 803 : Government needs a high integrity knowledge exchange by which innovations of the market can be objectively assessed.

DSB 2009: Weapons Systems Style Solution Architecture and Acquisition Processes take too long, cost too much, recommend establishing a separate IT Acquisition market that is tuned for the fast paced market.

IT-AAC 2009: Major IT Programs lack senior leadership support, and have few vested in the success. All participants, including oversight, must be incentivized in meeting program goals and outcomes.

BENS RPT on ACQUISITION 2009: DoD needs independent architecture development that is not compromised by those with a vested interest in the outcome. FAR OCI rules must be better enforced.

NDAA Sec 804 2010: DoD will establish a modular IT Acquisition process that is responsive to the fast paced IT market.

"Weapons systems depend on stable requirements, but with IT, technology changes faster than the requirements process can keep up," he said. "It changes faster than the budget process and it changes faster than the acquisition milestone process. For all these reasons, the normal acquisition process does not work for information technology.” DepSec Bill Lynn statement at the 2009 Defense IT Acquisition Summit hosted by IT-AAC


™

12

Federal IT Acquisition Root Causes compromising mission effectiveness and costing tax payer $40B/year

1. IT Acquisition Ecosystem Ineffective:

– Missing incentives & metrics, redundant oversight, vague accountability, ineffective governance (MOE, SLA) puts focus on compliance vs outcomes.

– Programs spending up to 25% on compliance without any reduction in risk.

2. Good laws (CCA, OMB 119, FAR, Sec804) lack enforcement:

– Frequently compounded by Ad-hoc Implementations and MilSpec methods.

– DODAF, JCIDS, NESI, LISI were designed for Weapons Systems, compete with standards and orthogonal to Industry Best Practices.

3. Conflict of Interest unenforced, optimal resources and expertise overlooked:

– FAR prohibits Contractors with vested interests in implementation should not use “Chinese firewalls” to bypass rules or gain unfair advantage.

– Optimal resources in IT Program planning, market research, and solution engineering overlooked, inhibiting access to real world best practices and innovations of the market. Standards bodies & non-profit research institutes under utilized.

“Insanity is continuing the same process over and over again and expecting different results” Albert Einstein


™

13

Root Cause (cont.)

4. FFRDCs/SIs are Stifling Innovation and Decision Making due to Analysis/Paralysis:

– Traditional Sis/FFRDCs are insulated from IT innovations and commercial best practices.

– PMs lacks effective outreach/research capabilities needed to inform the requirements and acquisition lifecycle. Lacks timely access to innovations of the market, commercial expertise, or benchmarked best practices and lessons learned.

– Small Businesses, Innovators and Public Service entities (.edu, .org, SDOs) are under utilized, threatening Open Systems and Open Architecture efforts.

5. MilSpec Acquisition Processes in conflict with Open Systems, best practices and drive “design to spec” approach ( in spite of CCA and NDAA Section 804 directive to the contrary):

– MilSpec Requirements (JCIDS), Architecture (DoDAF), Tech Assessment (TRL/C&A), Business Case Analysis (AoA), Procurement (DoD5000) and Enterprise Management (CMM) processes are inconsistent with fast paced IT market (in spite of Paperwork Reduction Act, CCA, Section 804 and OMB A119 directives)

– Section 804 call Open Process cannot be implemented using the same resources and expertise that created the current MilSpec processes

6. Budgeting (POM) approaches drive stove pipe solutions:

– Frequently undermining ability to establish common & interoperable infrastructure services which accounts for 70% of every IT program buy. Concepts like SOA, Cloud Computing and Service Level Management cannot be embraced without a change in the above.


™

14

Critical Success Factors for Sustainable IT Acquisition Reform

Root causes analysis derived from 15 years of studies, suggests the following critical success factors for sustainable IT Acquisition Reform. Any new process will need to meet the following litmus test:

Must replace each of the existing IT Acquisition lifecycle building blocks (per DSB report) and address the unique challenges of the fast paced IT market (JCIDS, DODAF, DOD5000, NESI)

Must be derived from commercial best practices (CCA)

Must avoid MilSpect by leverage existing investments and capabilities (CCA, NTTAA)

Should favor processes already proven in the market

Should be based on a consensus based standard (OMB A119)

Must be modular, services oriented (NDAA Section 804)

Should be measurable, repeatable and sustainable, with supporting training, education and mentoring (HR 5013)


™

15

IT Reform Way Forward

Adapting Agile Acquisition Standards and Benchmarked Commercial Best Practices


™

16

Resources for Sustainable IT Reformfor when failure is not an option

Interoperability Clearinghouse (ICH)• Repository of reusable Best Practices Frameworks (process and solution architectures)• Conflict Free Research Coop dedicated to operationalizing Clinger Cohen Act directives• Means of accessing wide range of SMEs and community of practices outside the Defense Industrial Complex

Acquisition Assessment Method (AAM)• Decision Analytics Tool for IT centric AoA, EoA, BCA, Risk and Technical Assessments

• Measurable, repeatable and sustainable method to enable cost avoidance and savings

• Incorporates by reference: SOA best practices, IT Infrastructure Libraries (ITIL) and Evidenced Based Research (EBR)

Solution Architecture Innovation Lab (SAIL)• Virtual Lab by which innovators can validate their solutions

• Solution Architecture patterns for e-Gov, IT Infrastructure, Cyber-Security & Health IT

• Means of tapping existing testing and implementation resources for rapid deployment

IT-Acquisition Advisory Council (IT-AAC)• A non-partisan Government and industry think tank created to drive sustainable IT Acquisition Reform

• Leverages expertise from academia, standards bodies, innovators and COIs

• Provide an interchange for senior level leadership interchange

™


™

17

Partner with DAU to create a Mentoring and Training Curriculum


™

18

Assessment of Alternative IT Acquisition Processes

MilSpec Acquisition Processes

Assessment against Sec 804 Criteria

Alternative Acquisition Process

Assessment against Sec 804 Criteria

Where successfully applied

Decision Analytics

Ad hoc, not formalized

Largest gap in IT Lifecycle

Acquisition Assurance Method (AAM)

Open, Successfully piloted, modular

AF, Navy, USMC, BTA, GSA, DISA,

Requirements Development

JCIDS, IT Box Not tuned for COTS, SOA, OSS Market

Value Stream Analysis w/ Agile Development

Exceeds criteria US TRANSCOM, DISA, CIA

Architecture DoDAF Systems Engineering Method

Missing Metrics, Infrastructure View, Stake holder perspectives

OMB FEA RMs SEI SMART

Strong evidence, Services Based

PTO, DOC, GPO, GSA, DOI, DOT, DHS

Technology Assessment:

TRL Assessment

IT Matures at a very fast rate

AF Solution Assessment Process (ASAP)

COTS/OSS Focused, support BPR

AF, USMC, BTA, Navy CANES, PTO, GPO, GSA

Risk & Cost Management

Analysis of Alternatives,

Time consuming, not aligned with industry B.P.

ASAP/AAM BCA

BTA ERAM

Effective w/ COTS based sys Limited risk mgt

AF, Navy, USMC, BTA

Governance and Oversight

DoD 5000 Bus Capability Lifecycle (BCL)

Milestone based, not effective for IT

ICH Clinger Cohen Act Guide

Integrated SOA best practices

BTA, OSD HA, Navy,


™

19

Resource Optimization Considerationsyou cannot outsource risk or critical thinking

1. FFRDCs: Best suited for govt unique R&D and Weapon Systems Source Selection.

2. Standards Development Orgs (SDO), Trade Associations: Source of standardizations among suppliers, ISVs. Effective source for market communications and outreach.

3. Research Institutes, Labs & Academia: Excellent source of low cost research, piloting of emerging technologies not yet proven in the market. Effective in IT & acquisition training.

4. Consultancies, A&AS Firms: Excellent for IV&V and source selection if free of vendor relationships or implementation interests. Can mitigate OCI issues in acquisition.

5. Innovators, ISVs, Open Source: The engine of innovation. Most effective and efficient way of filling common industry IT gaps. Great source of customer case studies and best practices.

6. System Integrators: Optimized for large scale implementation and outsourcing. Have significant economies of scale and technology usability insights.


™

20

Resource Guide for Effective IT AcquisitionBased on Clinger Cohen Act and FAR Guidance

Partner Type

SDLC Phase

FFRDC User Groups, Communities of Practice

Standards development orgs, trade associations

Research Institutes, Labs & Academia

Consultants, IV&V, A&AS Firms

Innovators, Tech Mfg, Open Source

System Integrators

Requirement, Gap Analysis

Only when no other company can support (4).

OMB Lines of Business offers Critical Role (6,7)

SDOs = Primary driver for open systems. Conflict free structures (2,3)

Provide Conflict free structure and economies of scale (2,6)

Limited access to industry lessons learned.

Great source for customer use cases, lessons learned.

FAR OCI Rules limit participation

Architecture and Planning, Mkt Research

Only when no other company can support (4)

Agency CxOs provides critical guidance (2, 3)

Provide standards of practice, not support

Principle source of expertise

Primary source of expertise

FAR OCI rules limit participation

FAR OCI rules prohibit direct support

PMO & IV&V Support

Only when no other company can support (4)

Not inherently governmental

Play supporting role

Optimized for this area

Key role FAR OCI rules prohibit participation

FAR OCI rules prohibit participation

Material Solution Engineering

Forbidden (4) Not inherently Governmental

Support role Support role Provide developmental

Primary partnership area

System Impl., Maint, & Support

Forbidden (4) Not inherently Governmental

Forbidden Lack Resources & Expertise

Internal IV&V for Prime contract reduces risk.

Provider of key technologies

Primary partnership area


™

21

Critical Success Factorsfor Sustainable IT Acquisition Reform

Clinger Cohen Act, FAR, and NDAA Sec804 directives cannot be accomplished with the same thinking that got us their. FFRDCs are prohibited from competing with industry and therefore the least effective resource for IT programs. The CSF include;

Agile Methods derived from benchmarked commercial best practices (CCA, Sec804)

Leverage existing investments and innovations of the market while avoiding competing with industry or duplicating what already exists (Economy Act)

Utilize public service institutes and standards bodies over FFRDCs (FAR)

Should be based on Open, consensus based methods (OMB A119)

Must be modular, services oriented (NDAA Section 804)

Should be measurable, repeatable and sustainable, with supporting training, education and mentoring (HR 5013)

15 years of studies suggest the following critical success factors for sustainable IT Acquisition Reform. An “Open” IT Acquisition process will still need to conform to the rule of law (non-MilSpec):


™

22

Repeatable Patterns of Success when agencies tap IT expertise outside the Defense Industrial

Complex!

Navy: Assessment of AFLOAT Program – CANES SOA & Security Strategy

Eliminated hi-risk Requirements by 23%, $100Ms in potential savings

USAF: Streamlined COTS Acquisition Process. Applied to Server Virtualization.

Established optimal arch with ROI of 450% & $458 million savings

USAF: Procurement of E-FOIA System using AAM

Completed AoA, BCA, AQ Selection in just 4 months.

USMC: AoA and BusCase for Cross Domain, Thin Client Solutions

Greatly Exceeded Forecasted Saving in both analysis and acquisition

GSA: Financial Mgt System consolidation using AAM.

Moved FMS from OMB “red” to “green”. Eliminated duplicative investments that saved $200M

BTA: Assessment of External DoD Hosting Options using AAM

$300 million in potential savings with minimal investment

BTA: Apply AAM to complete AoA and BCA for DoD SOA Project

Reduced pre-acquisition cycle time and cost of Analysis by 80%

(4 months vs 18)

GPO: Developed Acquisition Strategy for Future Digital System

Led to successful acquisition and implementation on time, on budget and 80% cheaper than NARA RMS

JFCOM: MNIS Evaluation of Alternatives for Cross Domain Solutions

Evaluated 100’s of Options in 90 days, enabling stake holder buy in and

source selection.

“. the concept of the Interoperability Clearinghouse is sound and vital. Its developing role as an honest broker of all interoperability technologies, no matter what the source, is especially needed. Such efforts should be supported by any organization that wants to stop putting all of its money into maintaining archaic software and obtuse data formats, and instead start focusing on bottom-line issues of productivity and cost-effective use of information technology.” OSD Commissioned Assessment of Interop. Clearinghouse (Mitre 2000)


™

23

AF Solution Assessment Process (ASAP) Case Study; Thin Client/Server Based Computing

Using ASAP/AAM enabled a 60day turn around to complete the following mandatory tasks;

1. Converted Requirements to Services/Capabilities Gaps

2. Established Measures of Effectiveness and Source Selection Eval Criteria

3. Conducted Baseline Assessment (cost/value of current portfolio)

4. Market Research (Realm of the Possible) that reaches real innovators and associated lessons learned.

5. Conduct Evaluation of Alternatives (apples to apples) using Evidence Based Research

6. Lite Weight Business Case Analysis for Investment Justification


™

24

Scope of ASAP Phase 2 EffortAdapting ICH’s Architecture Assurance Method

ASAP Phase 1 (June 08 - Jan 07)

TA Root Cause Analysis

Integration of AF TA best practices and ICH Architecture Assurance Method

ASAP Process development documentation

Facilitated TA Value Stream Analysis

ASAP Phase 2 (Aug 07 – Sept 07)

SBC Capability Determination

SBC Capability Prioritization

SBC Solution Architecture & Feasibility Assessment

SBC Business Case Analysis (lite)

ASAP Process maturity assessment

TA-ASAP Phase 3, Operationalize an Enterprise Process (4 FTEs) pending funding approval)

SBC Procurement Documentation Development

SBC Source Selection

TA-ASAP Management Oversight and Governance Build Out

TA-ASAP Process Integration (XC EA, XC PfM, XC CBA, AQ OTD)

Estimated TA resource allocation is 4-5 FTEs to support an AF enterprise wide implementation


™

25

ASAP Project Milestone

June 14 July 1 Aug 2 Sept 1 Oct 1

Project Plan

Kick Off Meeting

Data Call Results

MAJCOM Data Call

Announced to AF

ASAP Completed

BCA Completed

ASAP Assessment

PeriodASAP Report

Data Collection on Capabilities AFCA On-board

Capability Determination

Capability Prioritization

Feas./Arch. Assessment

BCA

ASAP Artifacts

Capabilities Determination - data

collection capabilities list 3 weeks

Capabilities Prioritization less than a week Assessment of technology Products 1 day BCA (operated in parallel with ASAP 5 weeks from start or

1 week after ASAP Assessment Remaining time is being applied to final

task on evaluation of the ASAP process itself

ASAP Process ExecutionHERE


™

26

Outcomes

OutcomesOutcomes

NOGO

ArchitectureAssessment

NOGO

GO

IndependentAudit

Lab

INNOVATION

SOLUTION

COMMODITY

CUSTOM

Type 1 Technical Solutions - COTS (Portal)Type 2 Major Program Solution (DoDAF/5000)Type 3 Technical Solutions - Custom (Guard)Type 4 Product Evaluation - Innovation Type 5 Product Evaluation - Commodity

T1

T2

T3

T4

T5

GO

SelectionAssessment

IndividualAudit

NetworkCertification

IndividualAudit


SelectionAssessment


SelectionAssessment

IndividualAudit


Cat-1: Value, Mature, Funded

Cat-2: Value, Mature, To Be Funded

Cat-3: Value, Immature

Cat-4: No Value

Wait

Invest

PROCUREMENT

Feasibility Assessment

KPP/CSF




IndependentAudit


KPP/CSF

Lab

ArchitectureAssessment Independent

Audit

Lab

SelectionAssessment

OutcomesOutcomes

OutcomesOutcomes

COTS


COTS

Selection Outcomes

Pre-Tech Assessment Technology Assessment Phase Implementation Phase






STEP-1 STEP-2 STEP-4

BCASTEP-3

Combined

ASAP Phase 2 Project Plan Innovation Thread

Use Cases Volumes/Timings

System Behaviors

BCA Templates and Models

Probable Cost Mode

Actual vs Planned

We R Here

Process; TA-ASAP Process ModelASAP

Artifact; TA Sequencing Diagram


™

27

SBC Capabilities before ASAP

1Reduce time to deploy infrastructure1aReduce the time to deploy new applications across entire command within hours.1bReduce the time to reassign client locations 1cReduce the time to stand up new organization/office/unit2Reduce infrastructure cost2aReduce equipment accountability cost asset mgt2bReduce software license cost 2cReduce number of report of survey asset mgt2dReduce power consumption (HVAC)2eReduce touch maintenance throughout installation2gReduce number of CSAs/SAs2iReduce technology refresh cost3Improve Reliability, Availability Survivability (RAS)3aReduce mission downtime caused by CMI/Intrusions3bReduce mission downtime caused by loss PC-resident data3cProvide for data backup/recovery services for clients.3dAllow workstation recovery within 15 minutes from a remote location to include applications, user data, and operating system3eProvide improved reliability and availability of computing resources, services4Work within current Security Management Posture4aReduce Vulnerabilities4bImprove patch compliance5Provide support for AF Use Cases5aProvide support for client type – Baseline5bProvide support for client type – Functional5cProvide support for client type – Non-Standard5dProvide support for client type – Standalone5eProvide support for client type – Remote5fProvide support for client type – Unmanaged6Support SBC storage strategy6aProvide server-side storage of System data and/or system images6bProvide server-side storage of enterprise data6cProvide server-side storage of user data and/or system images6dProvide server-side storage of user application6eProvide server-side storage of enterprise data application7Support Infrastructure Requirements7aMaintain current bandwidth/network loads (min 10 GB to max 100GB user profiles, 100 MB to the desktop)7bProvide consistent capability, whether rich or thin, with differing capabilities based on Active Directory rights/groups7dProvide support for the Common Access Card (CAC)/DOD Public Key Infrastructure (PKI) logon8Improved Manageability8aProvide for remote manageability of desktop8bProvide support for all business and mission applications, including bandwidth sensitive applications8cProvide for a client computing environment solution that scales over the AF enterprise 8dAllow use of a diverse mix of hardware end devices in a heterogeneous environment 8eIncrease IT service availability to the mobile/pervasive user 9Provide the same user experience (irrespective of client; rich or thin client).

Jumbled – Needed a Standardized Methodology

To Establish A Repeatable and Executable TA process


™

28

SBC Capabilities Tie Back to Mission Capability GAPS

Mission Capability Gaps

FROM CJCSI 3170.01F , NCOE JIC

1. Improved Operational Efficiencies asset management, system administration capacity management, manpower efficiencies, patch compliance

2. Improved ability to deploy/modify new infrastructure

within hours

3. Improved Mobility

as supported by a pervasive SBC infrastructure

4. Improved security

loss or theft of physical storage at the edge

CJCSI 3170.01F - Joint Chiefs of Staff Instruction under the Joint Requirements Oversight Council (JROC)NCOE JIC - Net-Centric OperationalEnvironment under Joint Integrating Concept

Provide the same user experience (irrespective of client; rich or thin client).

91

Improved Manageability81

Support Infrastructure Requirements72

Support SBC storage strategy61

Provide support for AF Use Cases53

Work within current Security Management Posture44

Improve Reliability, Availability Survivability (RAS)31

Reduce infrastructure cost21

Reduce time to deploy infrastructure12

High level CapabilityNoMission Capability

SBC Capability

Derived From: ConstellationNet Architecture, USAFE Pil0t, NetCent RFI, AF Data Call, Industry Studies and Data

Process; Capability DeterminationASAP

Artifact; Hi-Level Capability Matrix


™

29

Established Weighted CriteriaFor an Objective Decisions - 1

WT No. Capability Importance 150 1 Reduce time to deploy infrastructure 1a Reduce the time to deploy new applications across entire command within hours. 1 1b Reduce the time to reassign client locations 1 1c Reduce the time to stand up new organization/office/unit 1 150 2 Reduce infrastructure cost 2a Reduce equipment accountability cost asset mgt 4 2b Reduce software license cost 1 2c Reduce number of report of survey asset mgt 4 2d Reduce power consumption (HVAC) 1 2e Reduce touch maintenance throughout installation 1 2g Reduce number of CSAs/SAs 1 2i Reduce technology refresh cost 1 50 3 Improve Reliability, Availability Survivability (RAS) 3a Reduce mission downtime caused by CMI/Intrusions 1 3b Reduce mission downtime caused by loss PC-resident data 3 3c Provide for data backup/recovery services for clients. 1 3d Allow workstation recovery within 15 minutes from a remote location to include

applications, user data, and operating system 2

3e Provide improved reliability and availability of computing resources, services 3 50 4 Work within current Security Management Posture 4a Reduce Vulnerabilities 1 4b Improve patch compliance 1 50 5 Provide support for AF Use Cases 5a Provide support for client type – Baseline 1 5b Provide support for client type – Functional 1 5c Provide support for client type – Non-Standard 3 5d Provide support for client type – Standalone 2

Process; Capability PrioritizationASAP

Artifact; Weighted Capability Matrix


™

30

Established Weighted CriteriaFor an Objective Decisions - 2

5e Provide support for client type – Remote 3

5f Provide support for client type – Unmanaged 5 125 6 Support SBC storage strategy 6a Provide server-side storage of System data and/or system images 1 6b Provide server-side storage of enterprise data 1 6c Provide server-side storage of user data and/or system images 1 6d Provide server-side storage of user application 1 6e Provide server-side storage of enterprise data application 1 125 7 Support Infrastructure Requirements 7a Maintain current bandwidth/network loads (min 10 GB to max 100GB user profiles,

100 MB to the desktop) 1

7b Provide consistent capability, whether rich or thin, with differing capabilities based on Active Directory rights/groups

1

7d Provide support for the Common Access Card (CAC)/DOD Public Key Infrastructure (PKI) logon

1

150 8 Improved Manageability 8a Provide for remote manageability of desktop 1 8b Provide support for all business and mission applications, including bandwidth

sensitive applications 4

8c Provide for a client computing environment solution that scales over the AF enterprise

1

8d Allow use of a diverse mix of hardware end devices in a heterogeneous environment

1

8e Increase IT service availability to the mobile/pervasive user 2 150 9 Provide the same user experience (irrespective of client; rich or thin

client). 1

Process; Capability PrioritizationASAP

Artifact; Weighted Capability Matrix


™

31

Assessment Results Quantify Risks

Data Faith - R

Reduce

tim

e to

deplo

y

infrast

ruct

ure

Reduce infrastructu

re

cost

Impro

ve R

elia

bility

,

Availa

bility

Surv

ivability

(RAS)

Work

within

current Security

Managem

ent Postu

re

Pro

vid

e s

upport for AF

Use C

ases

Support S

BC s

tora

ge

strate

gy

Support Infrastructu

re

Requirem

ents

Impro

ved M

anageability

Pro

vide th

e s

am

e u

ser

exp

erience

(irre

spect

ive o

f

clie

nt;

rich

or th

in c

lient).

Score

Value Factors 15% 15% 5% 5% 5% 13% 13% 15% 15%

Softgrid 1.67 3.00 3.40 1.50 0.73 1.40 1.00 1.56 1.00 1.67Ardent 2.33 3.15 3.40 3.00 1.53 1.40 1.33 2.11 2.00 2.23ClearCube 1.67 2.23 1.30 2.50 2.07 1.40 2.00 2.78 4.00 2.48Wyse 1.00 1.92 1.30 1.50 2.80 1.00 2.33 4.22 5.00 2.67 CCI/HP 1.67 2.23 1.30 2.50 2.07 1.40 2.00 2.78 4.00 2.83Citrix 1.00 1.92 1.30 1.50 2.80 1.00 2.33 4.22 5.00 3.03

Blue = Essential 1 - 1.99

Green = Desirable 2 - 2.99

Yellow = Less Desirable 3 - 3.99

Red = Undesirable 4 - 5.00

The results showed how the combined Use Cases could be several by a single product suite. SoftGrid scored the highest due to it’s ability to provide the same user experience even though the range of score was not statistically different and all evaluated where in the low risk range. Overall, SoftGrid had the best solution to cover the entirety of the use case including speed to deploy and react, and manageability. However, SoftGrid has the least potential to reduce infrastructure costs.

Clearly there are tradeoffs. Citrix and Wyse solution had the most potential to reduce infrastructure costs.

CCI/HP, Citrix, ClearCube and Wyse had the most impact on reliability availability, and survivability

Citrix and Wyse were the least capable in manageability. Ardence was the least capable in speed to deploy and react.

ASAP quickly provided clarity to what seamed as a difficult decision which was being clouded by no one commercial solution being optimal to all capabilities and that ‘80%’ solutions could be rapidly analyzed and then procured and implemented.

Revisiting the Assigned Weights may change the scoring outcome

- Click on Table -

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

Process; Feasibility & Architecture Assessments

ASAP

Artifact; Solution Risk Assessment Report


™

32

Use Cases Provide Context for Tech Assessment

Connected –at work, persistent networkStandalone –must work without Internet (e.g. laptop)Remote Access –at home or in hotel

NIPRNet

SIPRNet

JWICS

Baseline – Standard Desktop Configuration

Functional – GCCS, Communications, Logistics, Finance, etc.

Nonstandard –Photoshop, Developer Tools, etc.

Subscriber Connection Security Domain Services/Applications

Each connection type is availablefor each security domain, subscribing to a

variety of services/applications. This producesmany user types and a complex scenario.

Of 635,000 clients identified by the MAJCOMs, 269,000 were Baseline Users and 278,000 were Functional User (86%) which are the bulk of Mission Support users

Can be viewed as two roles:

Mission Support Users who are supportable by the standard availability design of the network, e.g. ERP or training.

Mission Critical Users who must work even when there is a network outage, e.g. mobile, remote or standalone user (use cases)

USE CASES

Single Domain Only

Process; Capability DeterminationASAP

Artifact; Solution Use Cases


™

33

BCA for SBC must define business value to AF

Question Answer

Does the SBC improve AF effectiveness?

SBC improves security – loss or theft of physical storage at the edge by 100% SBC improves ability to deploy/modify new infrastructure from weeks to hours SBC improves operational efficiencies – asset management, system administration, capacity

management, manpower efficiencies, patch compliance by as much as 75% on a 4 year TCO SBC improves Mobility – as supported by a pervasive SBC infrastructure but requires a 20%

investment in infrastructure migration. This translate into increase first year cost to 200%

What is the ROI and TCO? Breakeven in 2nd year ROI 468%

Where and when should it be implemented?

Target implementation – Baseline and Functional office user (non-power users) representing 250,000 to 500,000 initial SBC recipients. Recommend phased implantation.

Process; Business Case AnalysisASAP

Artifact; BSA Report


™

34

ITCC’s Standard Desktop fulfills 1st Step in launching SBC

Unmanaged PC PC environment where software installation and setup, software maintenance, and asset management are local to the client.

SBC/ Managed PC PC environment where software installation and setup, software maintenance, and asset management are managed centrally at the server-side where all data relevant to these services are housed.

SBC/ Thin Client A thin client is characterized by a workstation that does not provide local storage and performs only local execution of specialized applications. A thin client is a lightweight

workstation that contains a standard operating system with the capability to execute local applications. A variation of a thin client can be a workstation that does not provide local

storage and performs no local execution of applications, referred to as an ultra-thin client.

SBC – Server-side Data Strategy

AF

St ep

1

Industry Data:Migration to Managed PCs (SDC) will provide over 50% of SBC saving

Industry Data:Migration to Thin Client (will provide a an additional 50% of SBC saving

AF

Step

2

Industry Data:Categorization is necessary in a BCA to determine how much additional savings can be obtained in thin client type platforms


™

35

TCO AssumptionsDerived From Burton Group, Gartner, IDC, Mitre & ICH Sources

Direct cost of thin client at 21% of Unmanaged PC costs - Burton Group

Migration Cost are 20% of the implementing the total cost - IDC

Managed PC add management server to the Unmanaged PC environment - Burton

The ratio between direct (fixed ) costs and indirect (variable costs) are 50:50 in Unmanaged PC environment – Gartner, Mitre

The ratio between direct (fixed ) costs and indirect (variable costs) are 65:35 in Managed PC environment – Gartner, Mitre

The ratio between direct (fixed ) costs and indirect (variable costs) are 74:26 in Thin Client environment – Gartner, Mitre

Strategy – replace a fourth of the workstation each year inline this technology refresh cycle Variable cost will increase each year in line with increasing SBC population

Data Faith - R

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable


™

36

MAJCOM Response Total Clients – 635,000

Mission Support

Use Case (000)

1. Baseline 269 42%

2. Functional 278 44%

3. Nonstandard 11 2%

4. Standalone 13 2%

5. Remote 57 9%

6. Other 7 1%

Connection (000)

A. NIPRnet 553 87%

B. SIPRnet 67 11%

C. JWIC 15 2%

Data Faith - R

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

Process; BCA & Volumes BehaviorsASAP

Artifact; BCA Segregation by Use Cases


™

37

AFSPC ANG PACAF AETC USAFE AMC ACC AVGReduce touch maintenance throughout installation 50% 7% 70% 40% 50% 43%Reduce power consumption 50% 2% 80% 50% 46%Reduce command-wide deployment times for new applications 15% 80% 48%Improve patch compliance 50% 35% 100% 30% 80% 59%Reduce number of CSAs 50% 25% 55% 100% 40% 54%Reduce equipment accountability cost 50% 25% 100% 70% 10% 51%Reduce technology refresh cost 57% 85% 71%Reduce software license cost 25% 30% 50% 35%Increase IT service aavailability 25% 25% 25%Increase IT service availability to the mobile user 25% 10% 60% 32%Deploy new applications across entire command within hours. 0%Rapidly reassign client locations 500% 80% 70% 217%Rapidly stand up new organization/office/unit 50% 80% 75% 68%Reduce equipment accountability cost 40% 70% 55%Potential software license cost savings 0% 0%Reduce number of report of survey 5% 100% 20% 42%No reduced mission downtime caused by CMIs 0% 100% 100%Reduce mission downtime caused by loss PC-resident data 30% 100% 65%Increase ITservice availability 10% 10%Compatibility with functional applications 97% 97%

Potential Efficiencies As Reported by MAJCOMs

Data Faith - Q

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

Process; BCA & Volumes BehaviorsASAP

Artifact; BCA Segregation by Use Cases


™

38

SBC Implementation Alternatives

Alternative 1 – SBCs implement SBC/TC devises ad hoc into an Site SBCs existing environment creating a mixed desktop environment

Alternative 2 – SBCs implement SBC/TC across a large user Use Case community type to single or small no of networks SBCs (minimum of 2500 clients to take advantage of the operational efficiency occurring in the migration cost only at those NOC centers applicable to the

migration. This strategy replaces PC at their normal refresh cycle with SBC devices there by not increasing the investment cost with early PC replacement cost. .

Alternative 3 – SBCs implement SBC/TC as a major NIPRNet Enterprise service within the AF, occurring all migration costs SBC upfront and a strategy of replacing PC at their normal refresh cycle with SBC devices. This an enterprise approach and all use case would be incorporated into the plan.

Recommended


™

39

Analysis of Alternatives Lays out Multiple Viable Options

Highest Impact at least cost

Very HighMediumLow Cost to Implement

Very HighHighIsolated Improved security

Very HighHighIsolated Improved Mobility

Very HighHighIsolated

Improved ability to deploy/modify new infrastructure

Very HighHighIsolated

Improved Operational Efficiencies

Enterprise SBCUse Case SBCsSite SBCsMission Gaps

Process; Analysis of AlternativesASAP

Artifact; AoA Semented by Use Cases


™

40

TCO Analysis 1 – 250,000 SBC/TC

Units 250,000

Unmanaged PC Managed PC Thin ClientDirect Cost - 1 Unit 500$ 504$ 393$

Direct cost - 250K Units 125,000,000$ 126,000,000$ 98,278,503$

In-Direct cost - 250K Units 125,000,000$ 69,300,000$ 24,569,626$

Migration Costs -$ -$ 24,569,626$

4 yr TCO 437,500,000$ 299,250,000$ 184,272,193$

4 yr TCO per SBC Client 2,500$ 1,613$ 885$

SBC Year 1 (25%) Year 2 (25%) Year 3 (25%) Year 4 (25%) TCODirect Cost 24,569,626$ 24,569,626$ 24,569,626$ 24,569,626$ 98,278,503$

In-Direct Cost 6,142,406$ 12,284,813$ 18,427,219$ 24,569,626$ 61,424,064$

Migration Cost 24,569,626$ 24,569,626$

Annual Costs 55,281,658$ 36,854,439$ 42,996,845$ 49,139,251$ 184,272,193$

Unmanaged PC Unmgd PC Annual 62,500,000$ 93,750,000$ 125,000,000$ 156,250,000$ 437,500,000$

SBC Saving 7,218,342$ 56,895,561$ 82,003,155$ 107,110,749$ 253,227,807$

Managed PCManaged PC Annual 48,825,000$ 66,150,000$ 83,475,000$ 100,800,000$ 299,250,000$

SBC Saving (6,456,658)$ 29,295,561$ 40,478,155$ 51,660,749$ 114,977,807$

Breakeven Year is 2nd yearROI 468% benefit/investment

Summary

Savings $114,987,807

Investment $24,589,626

ROI 468%

Breakeven 2nd Year

Data Faith - R

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

Process; BCAASAP

Artifact; Total Cost of Ownership


™

41

SBC ASAP Lessons Learned

1. TA-ASAP requires clear mandate (like AQ OTD initiative). TA-ASAP Governance and Management Processes need to be put in place (4-5 FTEs) to effect process and cultural change (mentoring, facilitation, mgt oversight)

2. With HQ XC Management Support and Facilitate Outreach, TA cycle times can approach 30 days, but.. Supply Chain/time to market savings may be hindered due to disconnects with requirements and acquisition processes (reduce redos).

3. Centers need XC mentoring support in executing TA-ASAP. MAJCOMs cannot produce quality requirements or capability gaps and need help in meeting exit criteria. (VSA and ASAP training issue)

4. Current RFI approach that relies on FSI’s ability to capture real world implementation best practices from ISVs is a non-starter. ICH needs to better utilize it research coop of 38 COIs and direct access to the ICH network of domain experts.

5. SAF/XC needs to partner w/ AQ and establish a more dynamic TA feedback loop to expose MAJCOMs and Centers to capabilities/developments/results (KM issue).


™

42

TA-ASAP Action ItemsTeed up, on contract and awaiting funds

SBC Pilot (2 FTEs, 30 days)

1. Present SBC Results to C2 GOSG for action

2. Move decision into Acquisition Stage. Develop SBC Acquisition Ready Documentation (unclear).

TA-ASAP Rollout (XC = 4 FTEs, AQ = TBD)

1. Present revised TA-ASAP Plan to Process Council for AF enterprise deployment approval. Address Cyber Command utilization.

2. Coordinate w/ AQ to integrate and de-conflict w/ emerging tech acquisition policy/processes (OTD/NESI).

Work AQ Partnership

3. Coordinate w/ XC process owners to integrate and de-conflict; EA, PfM, KM, & SOA Game Plan.

4. Establish and Issue Appropriate TA-ASAP Mgt Oversight, Policy and Process guidance (XC, AQ, Centers, MAJCOMs).

5. Establish TA-ASAP mentoring and training to assure effective use of AF resources.

6. Build out ICH’s research coop to leverage existing industry expertise and experiences to reduce cycle times and risk.


™

43

How ASAP Met the ChallengeExecutable, Measurable, Repeatable

Proved out as a sound IT Assessment methodology standard that will mitigate deployment risk. using products used: Capability Determination, Capability Prioritization and Feasibility Assessment

Validated Maturity of SBC Market

Demonstrated a capabilities-based technology assessment method

Established a body of standard artifacts that are important for future efforts

Demonstrated a rapid AOA Approach for COTS

Overall - Provides the AF with a standardized COTS Assessment Process that will increase the efficiency and technical robustness across the spectrum of future technology assessment


™

44

TCO Analysis 2 – 500,000 SBC/TC

Units 500,000

Unmanaged PC Managed PC Thin ClientDirect Cost - 1 Unit 500$ 504$ 393$

Direct cost - 500K Units 250,000,000$ 252,000,000$ 196,557,006$

In-Direct cost - 500K Units 250,000,000$ 138,600,000$ 49,139,251$

Migration Costs -$ -$ 49,139,251$

4 yr TCO 875,000,000$ 598,500,000$ 368,544,386$

4 yr TCO per SBC Client 2,500$ 1,613$ 885$

SBC Year 1 (25%) Year 2 (25%) Year 3 (25%) Year 4 (25%) TCODirect Cost 49,139,251$ 49,139,251$ 49,139,251$ 49,139,251$ 196,557,006$

In-Direct Cost 12,284,813$ 24,569,626$ 36,854,439$ 49,139,251$ 122,848,129$

Migration Cost 49,139,251$ 49,139,251$

Annual Costs 110,563,316$ 73,708,877$ 85,993,690$ 98,278,503$ 368,544,386$

Unmanaged PC Unmgd PC Annual 125,000,000$ 187,500,000$ 250,000,000$ 312,500,000$ 875,000,000$

SBC Saving 14,436,684$ 113,791,123$ 164,006,310$ 214,221,497$ 506,455,614$

Managed PCManaged PC Annual 97,650,000$ 132,300,000$ 166,950,000$ 201,600,000$ 598,500,000$

SBC Saving (12,913,316)$ 58,591,123$ 80,956,310$ 103,321,497$ 229,955,614$

Breakeven Year is 2nd yearROI 468% benefit/investment

Summary

Savings $229,956,614

Investment $49.139,251

ROI 468%

Breakeven 2nd Year

Data Conf. - R

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

V

R

Q

U

Very Reliable

Reliable

Questionable

Unreliable

Process; BCAASAP

Artifact; Total Cost of Ownership


™

45

AFCA Tech Assessment Team Scoring of SBC Results - 1

No.

Capability SCORING 1 to 5: 1 is least risk C

CI/

HP

Cle

arC

ub

e

Cit

rix

Ard

en

t

So

ftg

ris

Wy

se

1 Reduce time to deploy infrastructure 1a Reduce the time to deploy new applications across entire command within hours. 2 2 1 3 2 1 1b Reduce the time to reassign client locations 1 1 1 2 2 1 1c Reduce the time to stand up new organization/office/unit 2 2 1 2 1 1

2 Reduce infrastructure cost 2a Reduce equipment accountability cost asset mgt 2 2 2 3 3 2 2b Reduce software license cost 3 3 4 4 4 4 2c Reduce number of report of survey asset mgt 2 2 2 3 3 2 2d Reduce power consumption (HVAC) 3 3 2 4 4 2 2e Reduce touch maintenance throughout installation 3 3 1 3 1 1 2g Reduce number of CSAs/SAs 2 2 1 3 3 1 2i Reduce technology refresh cost 2 2 1 3 3 1

3 Improve Reliability, Availability Survivability (RAS) 3a Reduce mission downtime caused by CMI/Intrusions 1 1 1 3 3 1 3b Reduce mission downtime caused by loss PC-resident data 2 2 1 3 3 1 3c Provide for data backup/recovery services for clients. 1 1 1 3 3 1 3d Allow workstation recovery within 15 minutes from a remote location to include applications,

user data, and operating system 1 1 1 5 5 1

3e Provide improved reliability and availability of computing resources, services 1 1 2 3 3 2

4 Work within current Security Management Posture

4a Reduce Vulnerabilities 3 3 2 3 1 2 4b Improve patch compliance 2 2 1 3 2 1 5 Provide support for AF Use Cases

5a Provide support for client type – Baseline 1 1 1 1 1 1 5b Provide support for client type – Functional 2 2 4 2 2 4 5c Provide support for client type – Non-Standard 2 2 5 1 1 5 5d Provide support for client type – Standalone 5 5 5 4 1 5 5e Provide support for client type – Remote 4 4 4 3 1 4 5f Provide support for client type – Unmanaged

Process; Feasibility and Architecture Assessments

ASAP

Artifact; Scoring Template Results


™

46

AFCA Tech Assessment Team Scoring of SBC Results - 2

No.

Capability SCORING 1 to 5: 1 is least risk C

CI/

HP

Cle

arC

ub

e

Cit

rix

Ard

en

t

So

ftg

ris

Wy

se

6 Support SBC storage strategy 6a Provide server-side storage of System data and/or system images 1 1 1 1 1 1 6b Provide server-side storage of enterprise data 2 2 1 2 2 1 6c Provide server-side storage of user data and/or system images 2 2 1 2 2 1 6d Provide server-side storage of user application 1 1 1 1 1 1 6e Provide server-side storage of enterprise data application 1 1 1 1 1 1

7 Support Infrastructure Requirements 7a Maintain current bandwidth/network loads (min 10 GB to max 100GB user profiles, 100 MB to

the desktop) 3 3 3 1 1 3

7b Provide consistent capability, whether rich or thin, with differing capabilities based on Active Directory rights/groups

2 2 3 2 1 3

7d Provide support for the Common Access Card (CAC)/DOD Public Key Infrastructure (PKI) logon

1 1 1 1 1 1

8 Improved Manageability 8a Provide for remote manageability of desktop 2 2 1 3 3 1 8b Provide support for all business and mission applications, including bandwidth sensitive

applications 2 2 5 1 1 5

8c Provide for a client computing environment solution that scales over the AF enterprise 3 3 4 3 1 4 8d Allow use of a diverse mix of hardware end devices in a heterogeneous environment 4 4 5 3 2 5 8e Increase IT service availability to the mobile/pervasive user 4 4 4 3 2 4

9 Provide the same user experience (irrespective of client; rich or thin client). 4 4 5 2 1 5


ASAP

Artifact; Scoring Template Results


™

47

AFCA Tech Assessment Intermediate Results

Imp

orta

nce

CC

I/HP

Cle

arC

ube

Cit

rix

Ard

ent

So

ftgr

id

Wys

e

CC

I/HP

Cle

arcu

be

Cit

rix

Ard

ent

So

ftgr

id

Wys

e

150 1 Reduce time to deploy infrastructure 1.67 1.67 1.00 2.33 1.67 1.00

1aReduce the time to deploy new applications across entire command within hours.

1 2 2 1 3 2 1 0.67 0.67 0.33 1.00 0.67 0.33

1b Reduce the time to reassign client locations 1 1 1 1 2 2 1 0.33 0.33 0.33 0.67 0.67 0.33

1c Reduce the time to stand up new organization/office/unit 1 2 2 1 2 1 1 0.67 0.67 0.33 0.67 0.33 0.33

150 2 Reduce infrastructure cost 2.23 2.23 1.92 3.15 3.00 1.92

2a Reduce equipment accountability cost asset mgt 4 2 2 2 3 3 2 0.62 0.62 0.62 0.92 0.92 0.622b Reduce software license cost 1 3 3 4 4 4 4 0.23 0.23 0.31 0.31 0.31 0.312c Reduce number of report of survey asset mgt 4 2 2 2 3 3 2 0.62 0.62 0.62 0.92 0.92 0.622d Reduce power consumption (HVAC) 1 3 3 2 4 4 2 0.23 0.23 0.15 0.31 0.31 0.152e Reduce touch maintenance throughout installation 1 3 3 1 3 1 1 0.23 0.23 0.08 0.23 0.08 0.082g Reduce number of CSAs/SAs 1 2 2 1 3 3 1 0.15 0.15 0.08 0.23 0.23 0.082i Reduce technology refresh cost 1 2 2 1 3 3 1 0.15 0.15 0.08 0.23 0.23 0.08

WT

Raw Scoring Weighted Scoring

CapabilityNo.

Imp

orta

nce

CC

I/H

P

Cle

arC

ub

e

Cit

rix

Ard

en

t

So

ftg

rid

Wyse

CC

I/H

P

Cle

arcu

be

Cit

rix

Ard

en

t

So

ftg

rid

Wyse

150 1 Reduce time to deploy infrastructure 1.67 1.67 1.00 2.33 1.67 1.00

1aReduce the time to deploy new applications across entire command within hours.

1 2 2 1 3 2 1 0.67 0.67 0.33 1.00 0.67 0.33

1b Reduce the time to reassign client locations 1 1 1 1 2 2 1 0.33 0.33 0.33 0.67 0.67 0.33

1c Reduce the time to stand up new organization/office/unit 1 2 2 1 2 1 1 0.67 0.67 0.33 0.67 0.33 0.33

150 2 Reduce infrastructure cost 2.23 2.23 1.92 3.15 3.00 1.92

2a Reduce equipment accountability cost asset mgt 4 2 2 2 3 3 2 0.62 0.62 0.62 0.92 0.92 0.622b Reduce software license cost 1 3 3 4 4 4 4 0.23 0.23 0.31 0.31 0.31 0.312c Reduce number of report of survey asset mgt 4 2 2 2 3 3 2 0.62 0.62 0.62 0.92 0.92 0.622d Reduce power consumption (HVAC) 1 3 3 2 4 4 2 0.23 0.23 0.15 0.31 0.31 0.152e Reduce touch maintenance throughout installation 1 3 3 1 3 1 1 0.23 0.23 0.08 0.23 0.08 0.082g Reduce number of CSAs/SAs 1 2 2 1 3 3 1 0.15 0.15 0.08 0.23 0.23 0.082i Reduce technology refresh cost 1 2 2 1 3 3 1 0.15 0.15 0.08 0.23 0.23 0.08

50 3 Improve Reliability, Availability Survivability (RAS) 1.30 1.30 1.30 3.40 3.40 1.30

3a Reduce mission downtime caused by CMI/Intrusions 1 1 1 1 3 3 1 0.10 0.10 0.10 0.30 0.30 0.10

3bReduce mission downtime caused by loss PC-resident data

3 2 2 1 3 3 1 0.60 0.60 0.30 0.90 0.90 0.30

3c Provide for data backup/recovery services for clients. 1 1 1 1 3 3 1 0.10 0.10 0.10 0.30 0.30 0.10

3d

Allow workstation recovery within 15 minutes from a remote location to include applications, user data, and operating system

2 1 1 1 5 5 1 0.20 0.20 0.20 1.00 1.00 0.20

3eProvide improved reliability and availability of computing resources, services 3 1 1 2 3 3 2 0.30 0.30 0.60 0.90 0.90 0.60

50 4 Work within current Security Management Posture 2.50 2.50 1.50 3.00 1.50 1.50

4a Reduce Vulnerabilities 1 3 3 2 3 1 2 1.50 1.50 1.00 1.50 0.50 1.004b Improve patch compliance 1 2 2 1 3 2 1 1.00 1.00 0.50 1.50 1.00 0.50

50 5 Provide support for AF Use Cases 2.07 2.07 2.80 1.53 0.73 2.80

5a Provide support for client type - Baseline 1 1 1 1 1 1 1 0.07 0.07 0.07 0.07 0.07 0.075b Provide support for client type - Functional 1 2 2 4 2 2 4 0.13 0.13 0.27 0.13 0.13 0.275c Provide support for client type - Non-Standard 3 2 2 5 1 1 5 0.40 0.40 1.00 0.20 0.20 1.005d Provide support for client type - Standalone 2 5 5 5 4 1 5 0.67 0.67 0.67 0.53 0.13 0.675e Provide support for client type - Remote 3 4 4 4 3 1 4 0.80 0.80 0.80 0.60 0.20 0.805f Provide support for client type - Unmanaged 5 0.00 0.00 0.00 0.00 0.00 0.00

125 6 Support SBC storage strategy 1.40 1.40 1.00 1.40 1.40 1.00

6aProvide server-side storage of System data and/or system images

1 1 1 1 1 1 1 0.20 0.20 0.20 0.20 0.20 0.20

6b Provide server-side storage of enterprise data 1 2 2 1 2 2 1 0.40 0.40 0.20 0.40 0.40 0.20

6cProvide server-side storage of user data and/or system images

1 2 2 1 2 2 1 0.40 0.40 0.20 0.40 0.40 0.20

6d Provide server-side storage of user application 1 1 1 1 1 1 1 0.20 0.20 0.20 0.20 0.20 0.20

6e Provide server-side storage of enterprise data application 1 1 1 1 1 1 1 0.20 0.20 0.20 0.20 0.20 0.20

125 7 Support Infrastructure Requirements 2.00 2.00 2.33 1.33 1.00 2.33

7aMaintain current bandwidth/network loads (min 10 GB to max 100GB user profiles, 100 MB to the desktop)

1 3 3 3 1 1 3 1.00 1.00 1.00 0.33 0.33 1.00

7b

Provide consistent capability, whether rich or thin, with differing capabilities based on Active Directory rights/groups

1 2 2 3 2 1 3 0.67 0.67 1.00 0.67 0.33 1.00

7dProvide support for the Common Access Card (CAC)/DOD Public Key Infrastructure (PKI) logon 1 1 1 1 1 1 1 0.33 0.33 0.33 0.33 0.33 0.33

150 8 Improved Manageability 2.78 2.78 4.22 2.11 1.56 4.22

8a Provide for remote manageability of desktop 1 2 2 1 3 3 1 0.22 0.22 0.11 0.33 0.33 0.11

8bProvide support for all business and mission applications, including bandwidth sensitive applications

4 2 2 5 1 1 5 0.89 0.89 2.22 0.44 0.44 2.22

8cProvide for a client computing environment solution that scales over the AF enterprise

1 3 3 4 3 1 4 0.33 0.33 0.44 0.33 0.11 0.44

8dAllow use of a diverse mix of hardware end devices in a heterogeneous environment

1 4 4 5 3 2 5 0.44 0.44 0.56 0.33 0.22 0.56

8e Increase IT service availability to the mobile/pervasive user 2 4 4 4 3 2 4 0.89 0.89 0.89 0.67 0.44 0.89

150 9Provide the same user experience (irrespective of client; rich or thin client). 1 4.00 4.00 5.00 2.00 1.00 5.00 4.00 4.00 5.00 2.00 1.00 5.00

WT

Raw Scoring Weighted Scoring

CapabilityNo.


ASAP

Artifact; Near Term Results


™

48

SBC ASAP Pilot FocusTA-ASAP Governance,

Mgt Reporting, and Oversight pending



Feasibility Assessments

Architecture Assessment

Selection Assessments

Network Certification

Outcomes

Entry Criteria

Management Processes

Knowledge ExchangeProcesses

Req

ues

tP

roce

sses

Exit Criteria

Res

ult

s

ASAP continues to require refinement – HPTs:

– Tool standardization– Library standardization– Cataloging

standardization– Collection Methods

ASAP requires active:– Training, feedback and

mentoring– Further streamlining

ASAP requires governance over:

– Capabilities – Business and Military

– Policies– Processes– Procedures– Reports– Measures and Metrics

SBC ASAP Pilot was a partial implementation of ICH’s Architecture

Assurance Method

SBC ASAP

Pilot Focus


™

49

Back Up SlidesBack Up Slides


™

50

Ph

ase 3

Ph

ase 3

Ph

ase

2P

hase

2P

hase 1

Ph

ase 1

What Congress and OMB are Encouragingto assure Sustainable IT Acquisition Reform

1. Recommend using IT-AAC to Establish Measures of Effectiveness: tap alternative resources and expertise to provide critical resource support to the DepSec and IT Acquisition Task Force to establish performance metrics. Guide Task Force in establishing Governance Structure and Incentives for Sec804 and Operational Efficiencies in terms of process, culture, incentives and mentoring.

2. Brief out Root Cause Analysis: of current acquisition ecosystem (processes, culture, acqu resources and incentives) with public/private partners. Repurpose existing studies developed by objective sources; GAO, DSB, AF SAB, BENS, CSIS, IAC/ACT, ICH, IT-AAC, RAND, Battelle, NDIA. Conduct impact assessment and cost of maintaining status quo. Establish Critical Success Factors

3. Direct OMB and DoD to utilize IT-AAC to effect transformation. Build out IT-AAC Leadership Forums to identify existing capabilities, expertise, and emerging standards of practice. “804 Solution” must address weakness of all acquisition lifecycle processes; requirements (JCIDS), architecture (DoDAF), tech assessment (TRL), acquisition strategy, source selection, decision analytics (oversight).

4. Require Agencies to leverage IT-AAC Benchmark IT Standards of Practice: Document emerging IT Requirements, Architecture, Assessment & Acquisition standards of practices, approaches, processes, processes standards that have already been proven in the market. Reduce cost and risk of “build from scratch” or “reshaping broken processes”. Identify high risk programs where new processes can be piloted.

5. Institutionalize New IT Acquisition “Think Tank” that addresses Section 804, HR 5013 process implementation, training and piloting of the new IT Acquisition process. Mentor high profile IT programs ( who are already looking for change) through new 804 process; TMA’s EHR, DEEMs, Army FCS, DISA NECC, AF SOA, etc.

6. Work with DAU to establish IT Acquisition training curriculum and mentoring program. Build out DAU’s IT Clearinghouse to capture benchmarked industry best practices and proven innovations of the market.


™

51

Pro

cess

Pro

cess

Tech

nolo

gy

Tech

nolo

gy

Peop

leP

eop

le

How Agencies Can Respond to achieve Efficiencies and Cut Waste

1. Workforce Empowerment: Establish robust IT Acquisition Training and Mentoring program with the IT-AAC that builds on DAU/IT-AAC Partnership. Build out Best Practices Clearinghouse with reusable acquisition decision templates and solution architectures already proven in the market

2. Change Management: Establish Incentives that bring focus to Outcomes with supporting Measures of Effectiveness and StakeHolder Forums that align IT with mission outcomes.

3. Portfolio Assessment and Market research: Identify the capability gap. Align with real world metrics and service levels that reflect the realm of the possible. Partner with non-profits, SDOs, and other public service entities that increase the aperture of innovation.

4. Establish Shared Infrastructure Services: Define core IT infrastructure capabilities & services that can be widely leveraged (shared services), via SOA, IT Infrastructure, Cloud Computing

5. Adopt Open Architecture and Agile Acquisition Processes: Identify and eliminate legacy processes and policies that are no longer relevant to IT Acquisition outcomes. Establish streamlined set of methods & tools based on proven evidence to deliver.

6. Transform Acquisition Ecosystem: Streamline and integrate current IT Lifecycle Processes by focusing on outcome, metrics and decision analytics. Establish Technology Advisory Council or Grey Beard Council that improves transparency and leverages proven expertise.


™

52

IT-AAC Accomplishmentscan significantly reduce time, risk and cost

1. Established an alternative, conflict free think tank composed of the worlds top minds and most respected public service entities.

2. Completed Root Cause Analysis of Reoccurring Failure Patterns in DoD IT Acquisition and their devastating impact, derived from; over 40 major studies, 2 surveys, 121 interviews, 21 Leadership Workshops and 4 conferences.

3. Benchmarked Industry IT Architecture & Acquisition Best Practices and Common Failures, 10 of Fortune 50.

4. Completed agency pilots of alternative IT Acquisition processes covering; requirements, architecture, tech assessments, business case analysis, and source selection.

5. Standardized Agile Acquisition Framework, documentation, and case studies for rapid adoption

6. Partnership with DAU, which established an alternative IT Acquisition Training Curriculum.

"It is not a great mystery what needs to change, what it takes is the political will and willingness, as Eisenhower possessed, to make hard choices -- choices that will displease powerful people both inside the Pentagon and out” Defense Secretary Robert Gates