practice symposium sessionsipmconference.org/wp-content/uploads/2012/06/practice-symposia... · ps...

IPMW 2014 26th International Integrated Program Management Workshop

Practice Symposia

Agile Integration Track

Symposia Coordinator: Dale Gillam

PS 01 Agile and EVM Do Work Together, Here’s How… Andrew Rymaruk, L-3 National Security Systems (L-3 NSS) Barbara Phillips, Eagle One Solutions, Inc. (EOSi) Evan Francis, L-3 National Security Systems (L-3 NSS)

PS 05 The EVM + Agile Anthology

Ray Stratton, Management Technologies PS 09 GAO’s Scheduling Best Practices Applied to Agile Setting

Karen Richey, Government Accountability Office Jennifer Leotta, Government Accountability Office

PS 01 - Agile and EVM do work together, Here’s How…

Are you tired of only hearing the theory of applying EVM with Agile Software Development? Do you want to know if the combined application of both methodologies “actually” works? Well it does! In this session we will provide a real world example of how Agile and EVM requirements on an L-3 National Security Solutions (NSS) contract (w/ DCMA surveillance) have been married together to form a robust performance management solution for over a year. This session will cover the contract’s implementation and organization of both EVM and Agile solutions with specific emphasis on rolling wave planning, customer role, and most importantly, how the data and metrics generated are actually used to manage cost, schedule, scope, as well as continuously improve future predictability. Additionally, within this session the added complexity of DCMA surveillance will be covered.

Barbara Phillips, PMP, EVP, Eagle One Solutions, Inc. (EOSi)

Phone 703-626-4752 Email [email protected]

Ms. Phillips is Director of Performance Management Solutions at Eagle One Solutions, Inc. (EOSi), a small business specializing in performance management, acquisition and contraction management, business process management, human capital management, IT project management, strategic planning, cost/schedule analysis, and program assessment support. For RCAS, Ms. Phillips developed the RCAS EVM Implementation Plan and initial

mailto:[email protected]

WBS at start up and continues to support the program through DCMA Surveillance. She directs and leads consulting assignments to develop and implement earned value management systems (EVMS) for contractors and large-scale Federal programs. She has trained hundreds of participants in the specifics of project management best practices with an emphasis on EVM. Ms. Phillips is a recognized leader in the EVM community. Prior to joining EOSi, Ms. Phillips was the EV Program Architect at L-3 STRATIS' Center for Performance Management. In this role, she initiated and deployed a strategy for the development and implementation of sustainable earned value management systems across the division.

Evan Francis, L-3 National Security Systems (L-3 NSS)


Mr. Francis is currently a L-3 National Security Systems (NSS) Fellow and NSS Lead Agile Engineer, and the RCAS Program Lead and Chief Software Architect on the Reserve Component Automation (RCA) Program. On RCA Mr. Francis is the RCAS CAM and is responsible for eight Agile scrum teams and all support staff for a large integrated software system. Mr. Francis has over 25 years of information technology and technology management experience spanning the commercial and defense industries. His prior experience includes a variety of management roles such as Information Technology Director, Program and Project Management, as well as technical roles such as Chief Software Engineer and Chief Software Architect. Mr. Francis has thirteen years of experience in Agile project management in both the commercial and defense industries, where he was responsible for introducing Agile development methods into multiple organizations including creating tailored methodologies, managing organization change, and mentoring Agile teams from team inception to true Agile institutions. As the L-3 NSS Lead Agile Engineer, Mr. Francis acted as project manager, lead technical engineer and chief author of the L-3 Agile Scrum Enterprise (LASER) methodology used on numerous contracts including RCA.

Andrew Rymaruk, L-3 National Security Systems (L-3 NSS)


Mr. Rymaruk is Program Control Manager for the Reserve Component Automation (RCA) Program at L-3 National Security Solutions. In this capacity from contract start in Fall 2013, he is responsible for the implementation and architecture of EVM processes and procedures, Control Account Managers (CAM) training, change execution and documentation, IPMR generation, DCMA surveillance preparation, and maintenance of WBS and WBSD. Mr. Rymaruk helped lead L-3 through a successful Integrated Baseline Review (IBR) in late Fall 2013. In his current position Mr. Rymaruk is responsible for a team of four highly skilled EVM analysts. He is proficient in using EVM and Scheduling tools including; EVMax, BOEMax, Cobra, Open Plan Professional, and MS project.

PS 05 The EVM + AGILE Anthology



What have we learned about the marriage of EVM and Agile? Does each project have to rediscover what works and does not work? Where are the lessons learned, best practices, and past mistakes documented? At last year’s Integrated Program Management Conference (IPMC) a team of volunteers was formed with experience in applying EVM to Agile and Agile to EVM. They were tasked to document their lessons, ideas, and cautions in The EVM + AGILE Anthology”. This document discusses how the Agile approach needs to be different and Agile terminology. It also describes the linkage between Agile stories, story points, and EVM. An Agile EVM Gold Card is included as well as a mapping of Agile roles to EVM roles. It also discusses managing the EVM baseline with the needed flexibility of Agile. This presentation will highlight this fifty page document and its key contents.

Ray W. Stratton, PMP, EVP, Management Technologies

Phone 714-318-2231. Fax 714-529-2961 E-mail: [email protected]

Mr. Stratton, PMP®, EVP®, is founder and president of Management Technologies, an earned value management training and consulting firm. He provides his clients EVM training, ANSI 748 process engineering, third party EVM assessments, and application of EVM to their projects and programs. He has taught earned value management to US and foreign governments, educational institutions, and commercial and defense companies. He is the author of The Earned Value Management Maturity Model®, published by Management Concepts, and Ray Stratton’s Earned Value Professional (EVP) Exam Study Guide. Mr. Stratton is also the editor of the monthly “The EVM Newsletter™”

His flexible and tailored EVMS approach has provided small to medium sized projects rapid, low cost, ANSI 748 EVMS implementations by exploiting existing project controls processes and tools.

Ray W. Stratton has over twenty five years experience as a software program manager with a major aerospace defense firm. While there he managed the development of radar, communication, and command and control systems. Mr. Stratton retired from the Naval Air Reserve at the rank of Captain. He last commanded an Engineering Unit that supported a variety of studies and projects on behalf of the Naval Air Systems Command, laboratories, and ranges.

When not involved with clients, Mr. Stratton volunteers his time piloting his aircraft for Angel Flight West in the transport of financially needy medical patients from distant rural locations to urban specialized treatment centers.

PS 09 – GAO’s Scheduling Best Practices Applied to Agile Setting

The Government Accountability Office (GAO) is responsible for assisting the Congress in its oversight of the federal government, including agencies’ stewardship of public funds. To use public funds effectively, the government must meet the demands of today’s changing world by employing effective management practices, including measuring program performance. In


addition, legislators, government officials, and the public want to know whether government programs are achieving their goals and whether the estimated costs and schedules are reliable. In May 2012, GAO published an exposure draft of the GAO Schedule Assessment Guide which presents ten best practices associated with developing and maintaining a high-quality, reliable schedule. In finalizing the Schedule Guide, GAO is adding an appendix that addresses the applicability of scheduling best practices in an Agile Development method environment. This presentation will discuss issues that need to be considered when evaluating program schedules that are using Agile practices to develop software. The purpose of the appendix is to identify common misconceptions about scheduling for Agile projects and dispel them. Therefore, the Agile appendix in the GAO Schedule Assessment Guide will describe the applicability and benefits of scheduling best practices for Agile projects with various considerations including identifying key differences between Agile data / metrics and traditional scheduling.

Karen Richey, Assistant Director, Government Accountability Office (GAO)

Phone 202-512-4784 E-mail [email protected]

Karen Richey is an Assistant Director for the Applied Research and Methods Team at the Government Accountability Office (GAO). She is a senior cost analyst responsible for performing cost, schedule, and Earned Value Management analyses to support audit findings on a wide range of government programs. Certified by the National Defense University as a Chief Information Officer (CIO), Karen is also Level-III certified in the field of cost estimating and financial management. She has 23 years experience in the fields of cost estimating, scheduling, and EVM analysis.

Before joining the GAO, Karen was a cost analyst for the Department of the Navy where she performed earned value management and developed independent cost estimates for major weapon programs and automated information systems. Karen holds a degree in Statistics and Mathematics from the University of South Carolina. In the past few years, she has developed and delivered a one day training class for auditors on EVM as well as co-authoring articles in The INTOSAI IT Journal and the Navy Comptroller Magazine.

In June 2009, Karen received an Award of Excellence from the Federal Capital Planning and Investment Control (CPIC) and an Award of Merit from the Association for the Advancement of Cost Engineering International for leading the development of the GAO Cost Estimating and Assessment Guide. In March 2011, she received a CPIC Best Educational Contribution award for providing an outstanding contribution in training and educational tools for CPIC and related fields

Jennifer Leotta, Senior Research Analyst, Government Accountability Office (GAO)


Jennie Leotta is Senior Research Analyst for the Applied Research and Methods Team at the Government Accountability Office (GAO). She is a responsible for performing cost, schedule, and Earned Value Management analyses to support audit findings on a wide range of government programs. Certified by the International Cost Estimating and Analysis Association,



Jennie is also Level-III certified in the field of cost estimating and financial management. She has 9 years experience in the fields of cost estimating, scheduling, and EVM analysis.

Before joining the GAO, Jennie was a senior operations research analyst for the Department of Homeland Security where she performed proposal evaluation reviews and developed and reviewed cost estimates for major programs and high priority projects. She has also worked as an operations research analyst at the Navy and a survey statistician at the US Census Bureau. Jennie holds a B.S. degree in Quantitative Finance from James Madison University and an M.A. degree in Economics from George Mason University.

Practice Symposia Effective Program Start-Up Track

PS 13 Panel on Current Trends & Lessons Learned with Government Contracts Contracting Officers, NCMA Tysons

PS 17 Program Start-up Best Practices

Dale Gillam, SAIC PS 21 Building In Upfront Integrity for Successful Program Start-up

R. Scott Brunton, Lewis Fowler, LLC.

PS 13 - Panel on Current Trends and Lessons Learned in Effective Contracting

Contracting establishes the environment that governs the relationships between the end customer, supplier(s), oversight authorities, and other key stakeholders.

This session is a facilitated panel discussion hosted by the National Contract Management Association’s (NCMA’s) Tysons VA Chapter to share lessons learned and best practices in effective contracting and to discuss current trends and the way forward with Federal Government contracts.

In this session, we will discuss in an open forum:

• Learn about new regulations and how it affects doing business with the Federal Government

• Small Business concerns • Lessons Learned within Effective Contracting

Panelists include representatives providing their personal point of view from the Federal Government, Industry and small Business

Moderator: Crystal Glenn, Century Link, and NCMA Tysons Chapter President


Crystal Glenn has more than ten years experience in various career fields, including Finance and Accounting, Contract Administration, Marketing, and Commercial Insurance. Currently, she is a Program Control Consultant at CenturyLink Government Services and previously employed as a Project Control Manager for Science Applications International Corporation (SAIC) and Program Controller for the National Institute of Aerospace (NIA). She has worked on various Government and International programs managing budgets and scheduling of $100M+ with earned value requirements providing various project and financial reporting. She works closely with the program managers, program team members, contracts, program control team, accounting and finance to ensure program requirements are met and overall team collaboration.


Crystal currently serves as President for National Contract Management Association (NCMA) Tysons Corner chapter. She previously served as Treasurer, VP, Mentoring Program, and Chair of Programs for the Tysons and Hampton Roads chapters. Crystal also completed the elite NCMA Inaugural Contract Management Leadership Development Program (CMLDP) as an honor graduate. She is a certified Project Management Professional (PMP) with the Project Management Institute (PMI).

She earned a Bachelors degree in Business Administration with an International Business emphasis and a Minor in Accounting from Marist College. Crystal received a Board of Trustees scholarship from Hawaii Pacific University, where she earned a Masters of Business Administration (MBA) with an International Business concentration; and she was the first student at the university to complete the one-year MBA program.

Panelist: Debra W Scheider, Head of Contracting Activity and Director of Contracting for the National Reconnaissance Office


Debra W. Scheider is the Senior Procurement Executive, Head of Contracting Activity and Director of Contracting for the National Reconnaissance Office. Appointed in 2004, she is responsible the execution of all NRO contracting and contract policy. An acquisition professional for over 30 years, she is an expert in major system contracting and has firsthand experience in all facets of acquisition including high-tech, cutting edge, highly classified contracts. Her responsibilities include cradle to grave implementation of classified and unclassified contracting.

She designs and implements NRO acquisition policy through the NRO’s FAR like supplement the NRO Acquisition Manual (NAM). Her responsibilities include acquisition training of over 4,400 students and the performance of over 100 source selections per year through the NRO Acquisition Center of Excellence (ACE). She is responsible for management of the contract officer workforce, industrial property management, all leasing, and government purchase card programs. As the NRO Deputy Senior Manager for Services she designed a framework to more effectively manage requirements and cost for service contracts. Ms. Scheider is also the Procurement Integrity official, and the Suspension and Debarment Official for the NRO.

She holds a B.S. from American University, an MBA from VA Tech, and a MPP from the University of Maryland. She serves on the NCMA Board of Directors, is a Certified Professional Contract Manager, and Level 2 Program Manager.

Panelist: Jean Marceau Lohier, CPCM, CFCM, Sr. Contracts Manager for AT&T Government Solutions, Inc


Jean Marceau Lohier is currently a Sr. Contracts Manager for AT&T Government Solutions, Inc. He started his contracting career in 2001 with the Academy for Educational Development (AED now FHI 360) and has been a member of NCMA since 2004. Mr. Lohier served as



Director of Training, Mentor and Instructor for the Tysons Corner Chapter for the last two years. He is also an Instructor Contractor with INL/CAP Training – U.S. State Department

Jean volunteers as VP - Professional Growth & Development (NCMA Tysons Corner Chapter). Nationally, he has presented at conferences and has several articles published in the Contract Management Magazine. Mr. Lohier holds two post-bachelor Degrees: one from Georgetown University (GU) in Paralegal Studies and the other from the University of Virginia (UVA) in Contracts and Procurement Management. He is also a U.S. President’s Volunteer Award Winner Gold (2014) and Silver (2013).

Panelist: Christine Wade, Founder and Managing Partner of CaaSus


Christine Wade is a federal regulatory specialist with nearly 15 years advising and providing solutions to the federal government and government contractors, alike. She is the founder and Managing Partner of CaaSus where she provides outsourced federal compliance expertise via the Compliance as a Service™ model. A Certified Federal Contracts Manager (CFCM) and Project Management Professional (PMP), Ms. Wade provides federal regulatory, cost accounting, audit and full contract life-cycle support to clients in a variety of industries including aerospace, defense, civilian, healthcare and insurance, pharmaceutical, manufacturing, utilities and advertising. She specializes in detailed knowledge of Federal Acquisition Regulations (FAR) and agency supplements. Before founding CaaSus, Ms. Wade served government contracting clients through both Big Four firms and regional-based firms. In these roles, she led investment practices, supported Government Contractor Due Diligence activities, drafted policies, procedures and other internal control documentation, provided technical training for industry associations and clients, acted as a liaison with agencies such as DCAA and DCMA, and led system assessment and process re-engineering and remediation efforts. She has led service-oriented teams, managed complex projects and programs, developed and mentored project teams, published technical papers, and developed custom training for clients. Ms. Wade earned her Bachelor of Arts degree in Government and Foreign Affairs from the University of Virginia.

Panelist: Randall Burleson, Director of GWAC Contracting Operations Division, General Services Administration (GSA)


Randall Burleson is currently the Director of GWAC Contracting Operations Division of GSA. Randall began his career in contracting over twenty five years ago and served much of that time working in industry. Randall worked for small 8(a) as well as some of the Nation’s largest defense contractors.



Since leaving industry Randall has worked for the Department of Homeland Security as both a Contracting Officer and the Component Acquisition Executive for the DHS Office of Health Affairs. He also worked for the Department of Interior prior to coming to GSA. Randall has a unique perspective in that he has served as a Contracting Officer, COR, CAE and as an industry Contracts Manager. Randall is a big proponent of Small Business and collaboration between the Government and Industry. He has been a NCMA member over 15 years, he is a CPCM, and has written several articles for NCMA’s Contract Management magazine. Randall’s avocation is a law enforcement tracker. He currently serves as Coordinator for a regional Task Force as well as serving with Warren and Frederick County Sheriff’s Offices. He is married and has four children. PS 17 - Program Start-up Best Practices This session will be used to share the program start-up best practices that have been implemented and continue to evolve at SAIC. These start-up practices are part of SAIC’s Integrated Program Management (IPM) solution to support programs during the entire planning and execution lifecycle. The information will be shared in a way so that participants can learn from our program start-up experiences and improve practices in their own organizations. Information will be shared from the perspective of the organization and from that of the program teams. Mr. Gillam will be joined by two senior program leaders overseeing two of SAIC’s largest and most challenging programs. Eric Mackiewicz, Business Manager, will share the lessons learned and best practices from SAIC’s largest prime contract, a >$2.5 Billion primarily Firm Fixed Price (FFP) effort to provide custom IT solutions and support services. Rick Keeney, Program Manager, with share lessons learned and best practices from his leadership on the same program and then as he transitioned to lead a new >$300M IT support services contract. These programs include significant FFP work and voluntary election to implement SAIC’s Earned Value Management System (EVMS) and mature scheduling practices.

Dale Gillam, SAIC, Corporate EVM and Scheduling Focal Point


Dale Gillam is a Program Manager and the Earned Value Management (EVM) and Scheduling Focal Point for SAIC. Mr. Gillam’s expertise is in program start-up, planning, scheduling, and EVM, with an emphasis on implementing Integrated Program Management (IPM) solutions. Mr. Gillam is responsible for policy and practice definition, oversight and compliance, customer interface, and program implementation. Mr. Gillam has overseen hundreds and led dozens of IPM solutions for many U.S. Department of Defense and Civilian Agency customers. The solutions have spanned the spectrum from million dollar task orders, to several hundred million dollar services and staff augmentation contracts, to multi-billion dollar developmental contracts across all contract types. Mr. Gillam serves on the Governing Boards of the National Defense Industrial Association’s Integrated Program Management Division (NDIA IPMD), the College of Performance Management (CPM), and the National Contract Management Association (NCMA) Tyson’s Chapter. Mr. Gillam engages on these Boards to contribute to improving the depth in each


discipline while also improving the integration across the disciplines to support better program outcomes.

Eric Mackiewicz, SAIC, Senior Director, Program Management


Over thirteen years of experience in business management, project management and quantitative analytic task management and support. Mr. Mackiewicz is currently the Business Manager for $2.5B+ Department of State Vanguard 2.2.1 contract which is part of the Department's Information Technology (IT) consolidation program, responsible for engineering, designing, securing, operating and maintaining the critical IT infrastructures. Mr. Mackiewicz’s recent focus has been on structuring the financial and schedule baselines and maturing the baseline through a rigorous annual planning process. The Vanguard 2.2.1 program is managed through a myriad of monitoring and controlling tools, reports and forums, including an internal EVM approach to ensure detailed insight into program performance. Previous experience includes working in the Netherlands and managing two multi-national teams on the $100M+ Active Layer Theatre Ballistic Missile Defence System Engineering and Integration contract.

Rick Keeney, SAIC, Senior Program Director


Over 35 years of program/project/engineering management experience for high technology, high risk programs for both US and non-US commercial and government customers. Mr. Keeney is currently the Program Executive leading IT Operations and Business Services for the Federal Retirement Thrift Investment Board (FRTIB) who administers the Thrift Savings Plan (TSP), a tax-deferred defined contribution plan similar to private sector 401(k) plans which provides Federal employees the opportunity to save for additional retirement security. There are about 4.7M participants and over $400B, which is the largest managed fund in the world. Previous contracts included systems for the Governments of Greece and Egypt, including security systems and combat training systems, as well as work with the intelligence community. Prior to joining SAIC, he was vice president and senior program director for companies in the Baltimore area, where he led program teams in numerous fixed price/cost type, high technology, hard deliverable system developments and deployments with contract values up to $230M which included deployments at >900 global locations.

PS 21 - Building In Upfront Integrity for Successful Program Start-Up Successful program execution critically depends on the integrity of underlying data sources and systems being capable of producing meaningful information and reporting to guide program leadership in decision making. It is during this preparation phase; prior to program start-up, that focus centers on establishing and validating these financial and schedule data repositories, assessing their integrity, and



verifying the systems’ ability to deliver valid data-driven reporting and dashboards. Has your business environment experienced challenges in establishing such a credible information foundation before start-up? This presentation will reflect on how one engineering technology company took on this challenge by establishing a vision of success focused on the goal of being the provider of information truth. A review of the systems approach taken, the challenges faced with integrating data from disparate sources, and establishment of fallout reporting to provide real-time visibility of gaps occurring during execution. Key learnings from this start-up experience will be shared that can provide a context for increasing your organizations opportunity for a successful program start-up.

R. Scott Brunton, Lewis Fowler LLC


Scott is a Senior Consultant with Lewis Fowler LLC, a technology consulting and strategic project management company dedicated to delivering benefit to clients in the Aerospace & Defense, Oil & Gas and Enterprise Commercial vertical markets. He provides consulting services that enable cost effective Program Portfolio Management and Project Delivery processes to meet/exceed customer needs and to deliver on-time, best-valued and risk managed relationships and solutions. He provided integrated schedule, technical and financially accountable strategic driven business portfolio processes and skills that reveal and deliver unique customer capabilities to expand market penetration, and deepen value awareness of core processes and execution methodologies. Recent engagements include regional-facing Oil & Gas Project Management Office (PMO) leadership to provide portfolio management, financial cost/forecasting and schedule reporting throughout the life-cycle of project effort, and training to regional resources. Prior to joining Lewis Fowler, Scott delivered leadership and management expertise in a variety of sectors, including public utilities, healthcare, defense/aerospace proposal management and commercial product/marketing development He holds advanced technical and business academic degrees, is multiply credentialed with professional associations, licensed as a Professional Engineer and enjoyed teaching at several higher learning institutions.


Practice Symposia IMP and IMS Track

PS 02 Integrating a High Impact Summary Level Program Schedule Paul Bolinger, Humphreys & Associates

PS 06 GAO’s Schedule Assessment Guide Jason Lee, U.S. Government Accountability Office (GAO)

Juaná Collymore, U.S. Government Accountability Office (GAO) PS 10 Finding and Pruning Leaps of Logic in CPM Network Diagram

Jim Peter, John Hopkins Applied Physics Lab Kelvin Murray, Westall Murray International, Inc. PS 14 To Complete Performance Index – Based on Schedule Remaining Duration (TCPIRD)

Michael Nelson, Project Management Office at the INL Zane Mickelson, Idaho National Laboratory (INL)

PS 18 Schedule Margin…er Buffer, I mean Reserve, umm…Contingency Yancy Qualls, Bell Helicopter Textron

PS 22 IMP/WBS: Getting it Right is Paramount

Gordon Kranz, Earned Value Management, PARCA

PS 02 - Integrating A High Impact Summary Level Program Schedule

There are inexpensive and effective tools and techniques available now to integrate the arcane schedule information form the large network based tools to high impact executive level summary schedules. . This session will outline how the project team can turn the thousands of lines of IMS into a vibrant graphic story of the program that will make their case quickly and with impact. Too often, the chance to capture the attention of company and customer executives is squandered. Most often this high level view of the project is a hand drawn “cartoon” in Power Point or Excel or merely a roll-up or collapsed version of the IMS; a series of long bars. These methods are unwieldy, inefficient, and do not walk the intended reader through the project flow or point out the decision points, key dependencies, or particularly interesting parts of the program This presentation is based on years of experiences dealing with the IMS, weaknesses of the top level schedule, scheduling software, major programs, and more recently building the story to help sell major proposals. Attendees will see full blown examples of summary schedules and learn how tool availability and expertise can yield an impressive executive level schedule.

Paul F. Bolinger , Engagement Director, Humphreys & Associates

Phone 509-951-2459 E-Mail [email protected]


Paul has been an Engagement Director with Humphreys & Associates for 20 years working within client company teams on major proposals and major programs as part of the EVMS and IMS efforts. He has been involved in several EVMS implementations but more importantly to this topic, he has been a hands-on project scheduler and schedule team leader building and maintaining major network logic schedules. His recent work on major proposals and re-plans of programs has helped him spot the weaknesses in the most common approach to the Summary Schedule and to develop best practices and methodologies to make the Summary Schedule one of the main tools for the program/proposal team.

PS 06 - GAO’s Schedule Assessment Guide

The Government Accountability Office (GAO) is responsible for assisting the Congress in its oversight of the federal government, including agencies’ stewardship of public funds. In May 2012, GAO published an exposure draft of the GAO Schedule Assessment Guide which presents ten best practices associated with developing and maintaining a high-quality, reliable schedule. Developing the scheduling concepts introduced in the GAO Cost Estimating and Assessment Guide, the GAO Schedule Assessment Guide presents them as ten best practices associated with developing and maintaining a reliable, high-quality schedule. The ten best practices are intended to be used by auditors and managers to help ensure that the program schedule is reliable. The reliability of the schedule in turn determines the credibility of the program’s forecasted dates for decision making.

Jason T. Lee, Assistant Director, U.S. Government Accountability Office (GAO)

Jason Lee is an Assistant Director for the Applied Research and Methods team at the U.S. Government Accountability Office (GAO), where he is responsible for evaluating cost estimates and project schedules generated by programs across myriad agencies. Jason has fourteen years of experience in the analysis, parametric estimation, and evaluation of costs and master schedules associated with large federal government acquisition programs. He is currently leading the effort to produce GAO’s Schedule Assessment Guide, which provides criteria and best practices to help managers and auditors ensure that a program schedule is reliable. Before joining GAO in January 2009, Jason was a cost analyst with Technomics, Inc for eight years, where he developed independent cost estimates, cost assessments, and cost methodology research primarily for Department of Defense cost agencies and program offices. Prior to his work as a cost analyst, he was an operations research analyst for the Naval Center for Cost Analysis. Jason holds an undergraduate degree in Statistics from Virginia Tech and a graduate degree in Applied Economics from The Johns Hopkins University. In 2012, Jason received a meritorious service award from GAO for his leadership in developing schedule reliability criteria for auditors.

Juaná Collymore, PMP, PSP, Senior Schedule Analyst, U.S. Government

Accountability Office (GAO)

Juaná Collymore is a senior schedule analyst for the Applied Research and Methods team at the U.S. Government Accountability Office (GAO), where she is responsible for evaluating cost processes and performing schedule analyses generated by programs across a myriad of federal agencies. She has 15 years experience in the fields of schedule estimating and technology

auditing and holds certifications as a Project Management Professional (PMP) and Planning and Scheduling Professional (PSP).

Before joining the GAO in November 2012, Juaná was a Master Scheduler with Chevo Consulting, where she provided subject matter expertise in proper scheduling for major federal IT investments. Before IT consulting, she managed schedules and project controls for various local major construction projects including the Woodrow Wilson Bridge, Dulles and National Airports and Fort Belvoir Community Hospital.

Juaná has a Bachelor of Science degree in Civil Engineering from Howard University. She spends her non-GAO time cooking, exercising and spending time with family and friends.

PS 10 - Finding and Pruning Leaps of Logic in a CPM Network Diagram

The link from Task #1 to Task #4 is what we are calling a “Leap of Logic”. Leaps bypass true network logic. At best leaps are confusing and misleading; at worst leaps inject computational errors when the leap, or the links in the bypassed chain, have lag.

The leap rate in real schedules is between 2% and 20% with an average of 4%. Leaps get injected unintentionally and are impossible to detect manually. You won’t notice leaps in a list of predecessors and successors, and you can’t see them in a real network diagram because they are buried. We have created an optimized recursive topological search algorithm implemented in Visual Basic in Microsoft Project that traces all paths in the network and flags all the leaps of logic. The user then simply filters on the flagged leaps and deletes the ones that don’t affect the dates. The remaining leaps require manual inspection to determine what if any underlying engineering principle makes the leap sensible. Commercial tools are becoming available.

Jim Peter, Johns Hopkins Applied Physics Lab


Jim Peter has been an engineer at the Johns Hopkins Applied Physics Laboratory since 1986. Jim has a Bachelor’s Degree in Electrical Engineering, a Master’s Degree in Computer Science, and a Master’s Degree in Technical Management. Jim works in APL’s Space Department Scheduling and Earned Value Management Section. On major aerospace projects staff are responsible for designing project reporting systems; generating Work Breakdown Structures and Work Packages; coordinating the creation and updating of critical path activity networks;


coordinating fabrication efforts; preparing detailed and summary status reports; and presenting both the methodology and those reports to customers. Jim is also an inventive programmer with Visual Basic for Applications in Microsoft Project. Since 1992 Jim has also guest lectured in the JHU Technical Management Master’s Degree Program. Since 2003 Jim has taught project management overview courses for the Hopkins Business of Medicine Program, for a Fort Meade Defense Agency, at the Secret Service Training Center; and at the University of Maryland in Catonsville.

Kelvin Murray - Westall Murray International, Inc.


Kelvin Murray has over 30 years of experience leading projects and organizations across six continents, and now delivers training, consulting, staffing and management support services to clients worldwide. A Senior Project Management instructor for Global Knowledge, he has helped hundreds of individuals prepare for and pass the PMP exam. Under his pseudonym of “Sir Ganttalot®” Kelvin is the author of the booklet, “Project Risk Management – a Rapid Rollout Methodology”. As Sir Ganttalot, Kelvin also appears on YouTube providing informative and entertaining on-demand training and education on troublesome Project Management and Microsoft Project issues. Kelvin’s educational background includes the Royal Air Force Academy, England, Aberdeen and Herriott-Watt Universities in Scotland, and Agape Seminary.

PS 14 - To Complete Performance Index – Based on Schedule Remaining Duration (TCPIRD)

As a project nears completion it becomes more difficult to forecast completion dates for key milestones such as project turnover and project completion with reasonable precision. Historically forecast dates for these key milestones are overly optimistic. The units for calculating SPICUM are dollars, but the variable of interest is time, or days. This usually results in unpleasant announcements to both the company’s senior management and the customer and surprises by necessary schedule extensions. The question asked, and usually unanswered, by these executives is, “How is it possible that for several months the project is forecasting that it will meet the baseline date, and then within one month the project is forecasting a significant schedule extension and cost increase for these key milestones?”

This work shop presents a method for calculating future schedule performance efficiency in days. The process is similar to calculating the To Complete Performance Index (TCPI) for either the BAC or EAC. Due to the similarity it is called TCPIRD where RD is “Remaining Duration.” Similar to the other TCPI indices, the TCPIRD identifies the future schedule efficiency required to meet the forecasted remaining duration of the project. TCPIRD is used to provide an early identification of potential schedule forecast issues. It is calculated as a comparison of the original duration for un-started activities to the updated forecast activity duration. Join us in this workshop for a discussion on this exciting concept.

Michael Nelson presently serves as a Project Manager for the Idaho National Laboratory (INL). He has over 40 years’ experience providing leadership in the Project Management, Project


Controls and Operations disciplines. His experience has taken him into several different geographical areas supporting Owners and Contractor’s in both the private and DOE communities. He was responsible for providing leadership and management to achieve successful execution of a variety of projects including multi-million dollar construction projects, Operations management for a company providing specialized support teams in the international arena, IT processes, software, and work flows, and supporting a major environmental cleanup project.

Michael Nelson, Project Manager, Idaho National Laboratory


Mr. Nelson’s current assignment is as a Project Manager in the Project Management Office at the INL. In this position he leads efforts for reviewing and improving procedures, processes and training; assessment activities; and business processes and systems improvements Michael has a Bachelor of Science degree in Engineering Management from the University of Idaho.

Zane Mickelsen, Project Manager, Idaho National Laboratory


Zane Mickelsen works for the Idaho National Laboratory (INL) as a Project Manager in the construction arena. Prior to transferring into Project Management, Zane’s experience included serving as a Project Controls Specialist for the INL Project Management Office. In this capacity he provided analysis and input to support project performance reviews, assurance assessment activities, and process improvement initiatives. It was during his assignment in Project Controls that Zane searched for a method to assess schedule performance quantified in days rather than dollars. This search led to the development of the TCPIRD concept.

Zane’s love for project management grew from his project management experience in the family-owned road construction and mining business. While working on multi-million dollar highway expansion and city revitalization projects, Zane developed solutions to many of the business’ project management challenges including resource scheduling, cost allocation strategies, and tracking project performance. Zane holds a Bachelor of Business Administration degree in Accounting and a Master of Business Administration in Management.

PS 18 - Schedule Margin…err Buffer, I mean Reserve, umm…Contingency

Is Schedule Margin the same as Schedule Buffer? Is Schedule Buffer the same as Schedule Reserve? And how are any of these different than Schedule Contingency? Meanings and usages differ from company-to-company, government agency-to-government agency, and guidance document-to-guidance document. This presentation will propose clear and specific definitions for each term, as well as an overview of suggested best practices for their calculation and management.

Yancy Qualls, Bell Helicopter Textron



Phone 817-280-8306 Fax: (817) 278-8306 E-Mail [email protected]

Yancy Qualls has over 20 years of experience as a full-time scheduler in the Aerospace and Defense industry, with the last 15 being spent at Bell Helicopter. Yancy currently holds the position of Manager, Integrated Program Scheduling. In this capacity he sets company policy and provides training for all schedulers supporting both military and commercial aircraft in areas of design, test, and production.

Yancy is a frequent presenter and panelist at national conferences and is the co-lead of NDIA’s Planning and Scheduling Working Group (PSWG). Yancy was also a co-writer of the Planning and Scheduling Excellence Guide (PASEG) and the Predictive Measures Guide.

Yancy earned a B.S. in Industrial Engineering from Texas A&M University and an MBA from Texas Christian University. In addition, Yancy holds both the PSP and PMI-SP professional scheduling certifications.

PS-22 IMP/WBS: Getting it Right is Paramount

The Integrated Master Plan (IMP) and Work Breakdown Structure (WBS) are communication tools that inform team members and stakeholders what is being done and who is responsible for it. The IMP and IMS are critical to performing effective planning, scheduling, and execution of work efforts. Setting them up in the correct way to perform and manage the work is imperative. In this workshop, participants will discuss IMP and WBS structures and best practices to establish program expectations and define success criteria and establish end to end plan integrity.

Gordon M. Kranz, Deputy Director, Earned Value Management, PARCA

Phone (703) 697-3703 E-mail [email protected]

Mr. Gordon M. Kranz is the Deputy Director for Earned Value Management Performance Assessments and Root Cause Analyses in the Office of the Assistant Secretary of Defense for Acquisition. Mr. Kranz’s office serves as the Department of Defense focal point for all policy, guidance, and competency relating to Earned Value Management. Earned Value Management (EVM) is one of DoD’s and industry’s most powerful integrated program management tools used by the government and industry program managers and their teams to support decision making as they navigate the day-to-day constraints and risks that all DoD programs face.

Prior, Mr. Kranz held positions as the Executive Director, Engineering and Analysis, for the Defense Contract Management Agency (DCMA) and the Director, Systems and Software Engineering, in the Office of the Deputy Under Secretary of Defense (DUSD) for Acquisition and Technology (A&T). In these roles, Mr. Kranz provided policy, training, and tools required to perform systems engineering, software engineering, earned value management, manufacturing and production, quality, system assurance, risk management, and supply chain predictability.



Mr. Kranz has 30 years of defense acquisition experience, including 16 years in private industry as a senior technical lead and program manager and 10 years as an acquisition program manager for the United States Air Force.

Mr. Kranz received a Bachelor of Science degree in Electrical Engineering from North Dakota State University, a Master of Science degree in Electrical Engineering from the Air Force Institute of Technology, and is certified by the Project Management Institute as a Project Management Professional.

Practice Symposia Technical Readiness and IBRs Track

PS 03 GAO’s Technology Readiness Assessment (TRA) Best Practice Guide John Ortiz, U.S. Government Accountability Office

PS 07 The IBR: Your Insurance Policy for a Sound Baseline Jon Fleming, NASA’s Marshall Space Flight Center

PS 11 The Phased Approach to an Integrated Baseline Review

Eleanor Haupt, Earned Value Associates Deric Petty, PPMS

PS 03-Technology Readiness Assessment (TRA) Best Practices Guide

The U.S. Government Accountability Office (GAO) is responsible for, among other things, assisting the Congress in its oversight of the federal government, including agencies’ acquisition programs and projects. Federal agencies spend billions of dollars each year to develop, acquire, and build major systems, facilities, and equipment. Managing these complex acquisitions has been a challenge for federal agencies. For more than a decade, GAO has reported that using effective management practices and processes to assess how far a technology has developed and how it has been demonstrated are keys to evaluating its readiness to be integrated into a system and managing risk in the federal government’s major acquisitions.

To address the challenges of technology development efforts for integration into larger capital acquisitions, the GAO TRA Team has formed a community of technology experts from across the federal government, industry, nonprofits, and academia to develop a best practice guide. This TRA guide, “the Guide,” will be a companion to GAO’s Cost Estimating and Assessment Guide and to GAO’s Schedule Assessment Guide.1

TRAs have been in widespread use at the Department of Defense and National Aeronautic and Space Agency since the early 2000s. Since then, other agencies, including the Departments of Energy and Homeland Security, have adopted the practice to help manage their own technologically complex system acquisitions. Few other agencies have guides for assessing a technology’s readiness for insertion into larger acquisition programs, and no generally accepted approach for assessing technology beyond using technology readiness levels (TRL) has been adopted for governmentwide use.2 This Guide is intended to help fill those gaps.

With this Guide, GAO intends to establish a methodology based on best practices that can be used across the federal government for evaluating technology readiness. Existing TRA guidance in government agencies and industry may include similar strategies for assessing

1GAO, Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009), and GAO, Schedule Assessment Guide: Best Practices for Project Schedules, GAO-12-120G (Washington, D.C.: May 2012). 2TRLs is a scale of nine levels used to measure a technology’s progress, starting with paper studies of the basic concept and ending with a technology that has proven itself in actual usage on the intended product. Table 1 describes TRLs.

technology readiness, but no widely held or accepted process exists for doing so. The Guide relies heavily on DOD and NASA, pioneers in government technology readiness assessments, for terminology and examples. In addition, the Guide borrows from credible resources, materials, and tools developed by experts and organizations and try to capture the current thinking on technology readiness and maturity.

Finally, the Guide intends to give TRA practitioners, program and technology managers and governance bodies throughout the federal government the tools to increase their understanding of technology readiness assessment, technology maturity, and to identify potential concerns. Agencies that have developed their own guidance can use the Guide to support their own practices. Agencies that have not yet developed their own policies can use it to begin establishing their own guidance. Finally, as a companion to GAO’s cost and schedule assessment guides, this Guide can help GAO and other oversight organizations evaluate agencies’ bases for their conclusions and decisions about technology readiness.

PS 07 - The IBR: Your Insurance Policy for a Sound Baseline

So the contract has been awarded to the prime contractor, they have established a baseline, and are now ready to proceed to the execution phase. You’re now ready to start receiving the baselined Integrated Master Schedule (IMS), Contract Performance Reports (CPR’s), etc. from your prime contractor, right? Was an Integrated Baseline Review (IBR) conducted? If not, how do you know if the baseline is achievable? Were all of the stakeholders and Control Account Managers (CAMs) involved in the baselining process? Is the IMS capable of producing credible critical path(s)? These questions and more need to be asked during the IBR process. The big question is: Does the project team have confidence that the Performance Measurement Baseline (PMB) is achievable and is there cost, schedule, and technical integration? This presentation will cover what exactly and IBR is, and what it is not. We will discuss the NASA requirements and best practices for conducting an IBR and when it typically occurs based on the contract award date; training the NASA IBR team; assigning roles and responsibilities; information requirements, data gathering, and preliminary assessments; conducting a mock IBR and CAM discussions; all the way to actually conducting the on-site IBR at the contractor facility; as well as follow-on activities. In addition, we will review example NASA forms, reports, and other documents.

Jon Fleming, MTS/NASA

Phone 256-544-0137 Fax 256-837-6896 E-Mail: [email protected]

Mr. Fleming is a Project Management Specialist at NASA’s Marshall Space Flight Center. In this capacity, he performs project management consulting for the Agency and Marshall, responsible for assisting in the development of Center/Agency training material, as well as, training Center personnel in various Program Planning & Control (PP&C) related areas, such as: Work Breakdown Structure (WBS), Project Scheduling, Earned Value Management (EVM), Microsoft Project, Project Analysis, etc.

He also supports program/project planning; conducts schedule assessments; analyzes EVM data; conducts IBR training and facilitates reviews; supports the EVM Capability Project developing and implementing an Agency-wide solution for doing EVM on in-house projects;


reviews and edits Agency and MSFC handbooks, requirements, guidelines, and directives documentation regarding PP&C; works closely to support the mission of improving PP&C at Marshall; and has assisted with JPL’s EVMS Validation Review, among other aerospace contractors.

Prior to this position, Mr. Fleming has held positions from a program analyst to a project manager. He has over 19 years of diversified project management experience supporting NASA, DoD and commercial projects. Mr. Fleming He has a Bachelor of Science Degree from Athens State University in Management of Technology, and is a certified Project Management Professional (PMP) from the Project Management Institute (PMI), and actively promotes project management excellence at NASA.

PS 11 - The Phased Approach to an Integrated Baseline Review

The Integrated Baseline Review (IBR) is normally conducted as a single event, over a period of several days. Prior to the IBR, the contractor spends a considerable amount of time planning the work scope, schedule, and budget baseline in detail, but this planning is done internally without customer discussions. This can result in some replanning effort if the baseline does not adequately support the customer’s requirements. However, a phased approach to the IBR results in better communication and collaboration with the customer. In this presentation, the presenters will share their recent experience with conducting a phased IBR with their government counterparts, on a major defense program. They will present the phased approach methodology, lessons learned, and best practices so that others may adopt this as a best practice.

This session will be led by Eleanor Haupt, Earned Value Associates, and Deric Petty, PPMS.

Deric Petty, MBA, EVP, PMP, PPMS International


Mr. Petty is a Principal Earned Value Management Consultant for PPMS, International. He is a seasoned Project/Earned Value Management Professional that has worked with both government and commercial scopes of work over fourteen-years in a myriad of industries. He began his career in the Aerospace and Defense industry where he was the Program Controls Administrator on the Bell/Boeing V-22 Osprey program. In 2006, Mr. Petty was hired by a Medical Device company to act as the Program Manager on a portfolio of projects. This portfolio included the design, development, and manufacturing of several versions of a blood pathogen inactivation device. Soon thereafter, Mr. Petty began consulting for PPMS International providing assistance on the following: Program Management, Program Controls, Cost Engineering, Scheduling, Earned Value Management Systems (EVMS) and their implementation, financial reporting (weekly/monthly/quarterly performance), Risk Analysis and Forecasting.

Mr. Petty is a certified Project Management and Earned Value Professional, holds an MBA with a concentration in Finance from the University of North Texas, and earned his Bachelor’s


degree in Economics and Finance from the University of Texas at Dallas. Mr. Petty is a veteran of the United States Air Force (USAF). He is also an active member of the College of Performance Management (CPM), as well as, the National Defense Industrial Association (NDIA)

Eleanor L. Haupt, EVP, Senior Partner, Earned Value Associates LLC


Eleanor is a senior partner with Earned Value Associates LLC, a training and consulting company dedicated to improving earned value management. Eleanor provides customized support for EVM implementation and policy development to private and public clients. As part of EVMS implementation for a client, she conducts gap analyses and continuous improvement assessments, as well as compliance and integrated baseline reviews. Eleanor has developed and presented a wide variety of training materials covering the full spectrum of EVM, and is also a certified wInsight instructor. She is a frequent speaker, trainer, and keynoter at national and international EVM conferences. Eleanor is the author of several guides and manuals, and has been a major contributor to other guides. Prior to entering the public sector, she had a successful career within the Department of Defense for over 26 years, and was a leading expert on several key teams. She worked for the U.S. Air Force, Navy and Army, and has supported numerous programs in financial and cost management, specializing in earned value management and integrated management. Eleanor was the EVM focal point at Aeronautical Systems Center and responsible for policy development, guidance, training, and direction for the application of EVM on all Air Force programs at the center. Eleanor served as Executive Vice President and President of the PMI College of Performance Management, and also served on the Education Committee.

Professional Certifications: Earned Value Professional, Level III DAWIA certification in Budget, Cost Estimating and Financial Management; Professional Designation in Cost Analysis and Price Analysis

Education: Bachelor degree, Capital University, Columbus OH.


Practice Symposia Surveillance Reviews Track

PS 15 Surveillance as a Proactive Part of Battle Rhythm Marty Doucette, Pilgrim Companies, Inc.

PS 19 Earned Value Management – How to Pass a Government Audit

David Eck, Dixon Hughes Goodman Tom Shanahan, ProjStream

PS 23 Managing with the Best Possible Data: Enabling Effective and Risk-Based EVMS Across Industry

Ivan Bembers, NRO, Earned Management Center of Excellence Ed Knox, NRO, Earned Management Center of Excellence

PS 15 – Surveillance as a Proactive Part of Battle Rhythm

EVMS surveillance, whether triggered by external requirements, internal policy or both, can be an opportunity for organizational embracing of Integrated Program Management best practices. The realization of this opportunity originates in program management’s determination that the most efficient and effective means to schedule/budget success is by doing things right, just once, the first time and on time. This Practice Symposium session focuses on the premise of surveillance criteria as the basis of inputs and outputs in a program’s normal monthly battle rhythm. Attendees will receive a surveillance best practice assessment tool and sample templates that will assist them in reasonably identifying key surveillance points in the normal performance of integrated program management with the goal of transitioning from a ‘compliance’ to a ‘best practice’ team mentality. Included in the presentation will be discussions of auditing as a form of program support, determination of surveillance traceability readiness, starting or improving a battle rhythm, learning curve factors, surveillance LREs, and PM/IPT adjusting to changing internal and external surveillance capabilities expectations.

Marty Doucette, CTO, Pilgrim Companions, Inc.


Marty Doucette is the Chief Technology Officer of Pilgrim Companions, Inc. and has over 25 years of project management and EVMS implementation experience. Marty began his project management experience in television program production which led to his authoring of Microsoft Project for Dummies and Digital Video for Dummies. He was introduced to Earned Value Management in 2000 when contracted by a large DoD contractor to assist in their preparation for an IBR. Subsequently, Marty wrote, taught and developed software applications to assist first time users of wInsight and Microsoft Project in programs being newly exposed to DFARS EVMS requirements. Marty currently provides surveillance and IPMR support to large DoD contractors. He is a voting member of the NDIA Integrated Program Management Division and at this time serves on the Surveillance Update Work Group. Marty and his family reside in Indianapolis, IN.


PS 19 - Earned Value Management – How to Pass a Government Audit

Over the last several years, we have been working with companies to improve their earned value management systems (EVMS). We have helped many companies work government reported deficiencies and obtain approved EVMS. We have identified our top risk areas and have developed best practices to address these high risk areas. We plan to present those areas, processes to work the high risk areas, and strategies on working and resolving system deficiencies with the government. Our discussion will include:

• Reconciling Actual Cost of Work Performed (ACWP) to accounting records (especially SAP and Oracle)

• Processes for controlling overheard • Estimate at Completion (EAC) development and reconciliation (especially with other

financial data) • Processes for scheduling and budgeting work • Adequate processes for monitoring and documenting baseline changes (change control) • Adequate documentation of variance analysis • Processes for monitoring and documenting actions on variances • Reconciling subcontract cost performance report data to ACWP and accounting records • Determining impact of non-compliances with the 32 EVM criteria • Assessing adequacy of cost performance reports • Processes to work and resolve government reported deficiencies We will also present examples where companies were able to use EVM data to identify problems early and take action to minimize schedule, technical and cost problems.

David Eck, Director, Dixon Hughes Goodman

Phone 214-334-3233 FAX 703-970-0401 Email [email protected]

David has more than 31 years of experience working in the government accounting field. He has extensive knowledge and experience in all aspects of government laws and regulations, including the Federal Acquisition Regulations, individual agency supplements to the FAR, the Truth in Negotiations Act, the Cost Accounting Standards, and the Office of Management and Budget (OMB) Circulars. Prior to working with Dixon Hughes Goodman, David worked over 30 years with the Defense Contract Audit Agency (DCAA) at field, regional and headquarters offices. Over the last several years, he was the Regional Director of DCAA’s Central and Mid-Atlantic Regional Offices and responsible for auditing over 100 major contractors and several thousand smaller contractors. He has participated extensively in the development of changes to the FAR, DFARS, and the OMB Circulars, more recent noting his participation in the development of the new Contractor Business Systems DFARS rule. David was also responsible for development and implementation of DCAA’s contract audit policy, including the Contract Audit Manual and standard audit programs. He led the development of the Parametric Estimating Manual, which contributed significantly to improving estimating techniques to price


government contracts. He has worked extensively with DCMA and companies to improve earned value management.

Tom Shanahan, MBA, PMP, EVP, ProjStream


Tom Shanahan is the founder of ProjStream and is a recognized thought leader and visionary within the project management software industry. Mr. Shanahan has over 15 years of experience in earned value management (EVM), contract administration, scheduling, cost engineering and project management. Tom has a proven track record across multiple industries including Aerospace and Defense, Government, Energy and EPC. Tom has lead many successful EVMS implementations and has experience in the validation process.

PS 23 - Managing with the Best Possible Data: Enabling Effective and Risk-Based EVMS Across Industry In an effort to promote high quality data standards and to ensure contractor EVMS compliance, the National Reconnaissance Office (NRO) Earned Value Center of Excellence (ECE) has been conducting Surveillance Reviews (SRs) with its contractors for more than a decade. During that time period, the ECE has maintained a complete database of guideline incidences during the SR process and developed trends regarding type and severity of each Corrective Action Request (CAR). During the past year, using the NRO Systems Engineering Risk Management framework, the ECE has initiated a Risk Assessment on all current and future CARs. This process is designed to help the NRO and Industry better understand current problems and to fully assess the solutions that are presented when closing out a SR with appropriate follow-up action. This process is also being used to generate a roadmap for future reviews. This session will present the NRO ECE Corrective Action Risk Assessment process.

Ivan Bembers, NRO, Earned Management Center of Excellence Mr, Bembers is the Chief of the National Reconnaissance Office Earned Value Management Center of Excellence in the Cost and Acquisition Assessment Group. He is responsible for managing EVM system acceptance and surveillance reviews, facilitating Integrated Baseline reviews, and supporting programs use of EVM across the enterprise. He is a founding member of the Intelligence Community (IC) EVM Council that was established in 2004. Prior to his current position, he served in the same function in another defense intelligence agency and has worked on cross-functional teams across the IC. In 2011, he was the recipient of the Director of National Intelligence Certificate of Distinction to the Intelligence Community’s Earned Value Management System Team for meritorious service in developing and implementing evaluation and surveillance guidance and practices. Additionally, Mr. Bembers has supported various DoD agencies, worked in industry in corporate finance and contract/procurement positions, and served in the US Army.


Ed Knox, NRO, Earned Management Center of Excellence Mr, Knox supports the NRO Earned Value Center of Excellence. He has been as the forefront of schedule analysis and has developed numerous analysis tools for the NRO including the Scheduled Wizard, a BEI Assessment tool, and a Schedule Comparison module. He is also the chief architect in developing the current NRO CAR database and the ongoing risk assessment process. Mr, Knox has been with Tecolote Research since 1989.

Practice Symposia Implementation Considerations Track

PS 04 EVM Adoption: Making the Transition from Old to New Candi Randolph, Charter Performance Management Group, LLC

PS 08 EVMS Tool Evaluation

Ted Hibsman, Leidos EVMS Center of Excellence PS 12 The Mystery of Rolling Wave Planning

Michael R. Nosbisch, Project Time & Cost, Inc.

PS 16 Program Management Dashboard: How to be Intelligent about Business Intelligence

Tom Sullivan, PMP, Herren Associates Steve Sheamer, Herren Associates

PS 20 A Tale of Two Contractors: A Comparison of EVM Practices Keyi Oborokumo, Booz Allen Hamilton

PS 24 EVMS Implications of Best Intentions and Misdirection

Mark Infanti, Sr. SM&A, Inc. PS 04 - EVM Adoption: Making the Transition from Old to New

You’ve planned the EVM implementation effort and executed the plan. Procedures are documented and personnel are trained but when the rubber meets the road you’re seeing more business-as-usual than new process implementation. In this session, we will explore common barriers that prevent institutionalization of new EVM processes and how to realign the implementation focus towards EVM adoption.

Candi Randolph, PMP, Charter Performance Management Group, LLC


Candi Randolph is co-founder of Charter Performance Management Group and has over 25 years of Project Management and EVMS implementation experience. Candi spent over 20 years in the Aerospace and Defense industry where she supported business operations, managed the design, development and implementation of Earned Value Management Systems, and managed large-scale IT programs. She currently provides EVMS consulting and training services to companies implementing Earned Value Management Systems and teaches project management classes including a Project Management Professional (PMP®) Certification Boot Camp. Candi holds a B.S. degree in Computer Science and is certified by the Project Management Institute as a Project Management Professional.

PS 08 - EVMS Tool Evaluation at Leidos

Regardless of the size and type of your earned value programs, it will likely be advantageous to use some form of automated tools to satisfy earned value requirements and manage your


programs effectively. Particularly when larger and more complex programs are taken on, the type of tool used may make a significant difference in management ability and cost efficiencies. In this session, we discuss the process and criteria used by Leidos in approaching the evaluation and selection of an earned value tool. Addressed as parts of the process are identifying the scope and requirements for tool application, the types and size of programs to which it will be applied, evaluation criteria, evaluator roles, and an evaluation schedule. Addressed as part of the criteria are cost, amount of support required, ease of installation and implementation, scalability, ease of use, past experience/history, and capabilities/features. The session will provide attendees with a roadmap for conducting their own EV tool evaluations and a framework for making informed tool decisions and selections.

Ted Hibsman, Leidos EVMS Center of Excellence

Phone 858-826-4431 Fax 858-826-2700 E-mail [email protected]

Ted has over 25 years of experience in project management and controls and Earned Value Management Systems (EVMS), including project scheduling and cost integration. He has experience on both sides of contracts, having served in the U.S. Air Force as a project and procurement officer for DoD Space Shuttle payload integration programs and in-space mission operations; he has also worked as a contractor for TRW, a NASA Jet Propulsion Laboratory (JPL) contractor, Hughes Aircraft Company, Raytheon, SAIC, and now Leidos. His experience extends across DoD, Intelligence, and other government agency programs. In EVMS, Ted has several years’ experience working “hands on” as an EVMS lead on programs of a wide range of value and contract type. He also has several years’ experience as an instructor of EVMS policies, processes, and program and tool implementation. Most recently, he works as a senior EVMS manager in the Leidos EVMS Center of Excellence, responsible for update/maintenance of the EVM System Description, proposal/program implementation assistance, and internal/joint surveillance.

Ted was certified as a PMI Project Management Professional (PMP) and AACE Earned Value Professional (EVP) in 2011, and as a PMI Agile Certified Practitioner (PMI-ACP) in 2014. He received a degree in Systems Engineering from UCLA and studied graduate-level business at the University of San Diego (USD). He currently works for Leidos in San Diego, California.

PS 12 - The Mystery of Rolling Wave Planning

For the most part, an EVMS that complies with the ANSI/EIA 748 Standard is meant for cost reimbursable contracts that are “developmental” in nature. This implies that the scopes of work for these contracts are not very well defined, which would seem at odds with the fundamental EVMS concept/requirement to develop a comprehensive performance measure baseline (PMB) within 120 days of contract award. For the most part, this requirement can only be met by the use of the “rolling wave planning process,” which emphasizes detailed planning only for near term work scope elements, while far term elements are planned in summary-level detail. Despite this key role in the overall process, however, rolling wave planning is still not well understood by both government agencies and the contractors that support them, to the point where key components are often times not allowed within the constructs of the program-specific


system(s) being implemented. This presentation will provide recent examples of this, but more importantly will seek to more effectively define the rolling wave planning process and the rationale for using it correctly within an ANSI-compliant EVMS.

Michael R. Nosbisch, Project Time & Cost, Inc.

Phone 562-896-0374 Fax 562-628-5501 E-mail [email protected]

Michael has over 26 years of experience in project management and controls within both the government and private sectors. He has worked for several of the leading engineering, procurement, and construction (EPC) firms in the industry, to include URS Corporation, Jacobs Engineering, Kiewit, and Parsons Corporation. In 2009, Michael joined a leading management consulting firm, SM&A, and was responsible for their successful EVM Compliance and Consulting practice. In May of 2011, he joined Project Time & Cost as Managing Principal, Western Region, and EVM Practice Lead, based in Long Beach, CA. Michael is also an adjunct professor at the University of Southern California, teaching graduate level courses in heavy construction estimating and scheduling.

Michael was certified as a PMP in 1999. He has also been an active member of AACE International since 1999, earning numerous professional certifications as well as serving President of the Association from 2011 to 2012. In addition, Michael is an active member of both the Integrated Program Management Division of NDIA and the Energy Facility Contractor’s Group’s (EFCOG’s) Project Management Working Group (PMWG).

PS 16 - Program Management Dashboards: How to be Intelligent about Business Intelligence

Management dashboards are becoming increasingly prevalent in government, commercial and non-profit performance management processes and nearly all program management tools are incorporating dashboard capabilities into their software. Despite their popularity, many program dashboards fail to provide meaningful insights that aid decision makers and drive performance accountability. It takes proper planning and a structured approach to ensure dashboards effectively measure program performance. A properly planned dashboard transforms raw data into meaningful information, allowing decision makers to investigate the root cause of performance issues. Unfortunately, many programs are measuring things for the sake of measuring and are failing to establish meaningful targets that enable accountability. Without properly establishing meaningful goals and measurement criteria, many organizations are investing in dashboards that do not provide any strategic benefit.

This presentation’s objective is twofold. The first is demonstrating the importance of the planning process, including a well-defined approach for establishing an effective metrics plan with actionable objectives, measurements, and reporting mechanisms. We will explain how this process is a critical step in establishing a program dashboard and why dashboard projects rarely succeed without this upfront analysis. Secondly, we will demonstrate real-world examples of dashboards and clearly demonstrate the difference between an executive scorecard, dashboard, metric and report. We will explain how successful program dashboards


tie together the right pieces of performance management data and display them in simple to use formats. The approaches and concepts discussed in this presentation will provide the audience with an understanding of what is required to ensure that your investment in program management dashboards does not go to waste.

Tom Sullivan, PMP, Herren Associates

Phone 202-509-0470 Fax 703-607-7186 Email [email protected]

Tom Sullivan is an Associate at Herren Associates and combines seven years of industrial and commercial experience developing financial models and dashboards. Working with numerous clients in the Natural Gas value chain, he developed financial models and dashboards to aid those in the C-suite to make investment decisions. He developed and currently manages the Navy Maritime Energy Portfolio tool to analyze energy initiatives and develop optimized implementation plans. Mr. Sullivan holds a BBA in Business Administration from the University of Notre Dame (’06) and MBA in Finance from Loyola University Chicago (‘09).

Steve Sheamer, PMP, CCEA®, LSSBB, Herren Associates

Phone 202-549-3320 Fax 703-607-7186 Email [email protected]

Steve Sheamer is a Senior Associate at Herren Associates and has experience supporting both government and commercial programs including the F-35 Joint Strike Fighter, Littoral Combat Ship and Navy ERP Program. He has designed and implemented more than a dozen management dashboards and reporting tools across multiple organizations with many lessons learned along the way. Mr. Sheamer has both a master's and bachelor's degree in industrial engineering and is a Certified Cost Estimator/Analyst, Project Management Professional (PMP), Lean Six Sigma Black Belt, and holds a certificate in earned value management from PMI.

PS 20 - A Tale of Two Contractors: A Comparison of EVM practices

In this real-world case study, we will compare two different approaches to $100M ANSI-Compliance required Earned Value Management. Both contractors were awarded similar IT government scopes. One contractor successfully executed the project and under-ran its budget by 5%, while the second contractor exceeded its budget by 30%. What are some of the lessons learned from these two cases? What best practices did the contractor who executed the project on time and under budget adopt, and what went wrong with the other contractor who exceeded its budget significantly?

We will examine 5 areas within project management; resource management, scheduling, EVM methodologies, and cost management, where each contractor took different approaches. What were the results and impact of each approach? Finally, we will discuss the lessons learned from both approaches, and how we can apply them to future projects to attain successful project execution.

Keyi Oborokumo, PMP Booz Allen Hamilton





Keyi Oborokumo has 7+ years of experience in project cost control. Keyi is a Senior Consultant with Booz Allen Hamilton, providing project management oversight with a focus in Earned Value Management and Risk Management. She performs evaluation and assessment of cost, schedule, and technical performance on multiple IT projects, identifies early warnings for cost and schedule risks, and provides assessment for multiple Integrated Baseline Reviews. Prior to joining Booz Allen in 2012, she worked for Lockheed Martin. At Lockheed Martin she served as a financial lead, developing and pricing proposals, monitoring program expenditures against budget, forecasting, reviewing commitments, analyzing profit and revenue, and implementing Earned Value Management approaches and methodologies.

PS 24- EVMS Implications of Best Intentions and Misdirection

Throughout the lifecycle of a program, stakeholders make decisions based on what they believe is in the best interest of the program. Occasionally these decisions are in conflict with and impact the contractor’s (validated) EVMS. However, if these decisions result in findings during a third party (DCMA) review, the contractor is held responsible; even if the contractor is following customer direction.

This presentation highlights three key drivers to this issues: misuse of Management Reserve (MR), problems with Over Target Baselines (OTB), and program direction that is not contractual. Specifically covered are the approaches contractors should employ to mitigate and resolves these challenges.

We will discuss in detail the responsibilities of the government contracting officer, the contractor’s contracting officer, the government PM, the contractor PM and the contractor EVM focal point. This presentation addresses what you should do and where to seek help if you or your program receives direction that is in conflict with your approved EVMS.

Mark Infanti, Sr. EVM Consultant, SM&A Inc.


Mark’s experience with Earned Value Management began in 1975 in the aerospace industry. Over the past 30 years, he has been in the project management field as a project analyst, EVM consultant, technical sales support for EVM products, implementation consultant, project controls manager at a major EPC company and marketing manager for EVM tools. In these positions he has made presentations at conferences, presented EVM and scheduling training around the world, and supported professional organizations such as CPM, PMI, NDIA, & AACE.

He has developed EVM system descriptions and procedures to meet customer needs, trained users and supported automated system implementation for major engineering and design projects. Mark has lead several DOD and DOE clients through successful EVMS validations.

Mark has presented hundreds of training classes to both the public and private sectors on 3 continents to contractors and government agencies and wrote a book on EVM that was used to train a project management software company team in the 1980’s.


Mark holds a degree in Business Management from the University of Texas at El Paso.

Practice Symposia NDIA IPMD Track

Coordinator: Dale Gillam

NDIA 01 - Overview of NDIA IPMD Initiatives Carol Boser, Lockheed Martin Space System Company (SSC)

NDIA 02 - Role-Based Performance Management: The Team Makes Things Happen

Gordon M. Kranz, Earned Value Management, PARCA NDIA 03 - Application of EVM in PMNC

John Kanicsar, Orbital Sciences Corporation Matthew Schmidbauer, Orbital Sciences Corporation

NDIA 04 - IPMD Predictive Measures Guide

Bill Altman, Battelle Sung Soon Stultz, Rockwell Collins NDIA 05 - IPMD Clearing House Working Group Pete Wynne, Lockheed Martin Space Systems Company Mellissa Glbert, Orbital Sciences Corporation NDIA 06 - Civilian Agencies and Industry Working Group (CAIWG) Update Jerald G. Kerby, NASA NDIA 01 - Overview of NDIA Integrated Program Management Division (IPMD) Initiatives

The Integrate Program Management Division of NDIA is the primary forum for building strong Industry and Government working relationships to promote integrated program management -- the disciplined performance management methodology of planning and executing programs to optimize outcomes. With renewed energy, the IPMD has broadened their focus to serve as a thought leader and source of best practices for program performance management. Gaps and deficiencies in the current state of practice are addressed head-on with sponsorship of joint industry / government Working Groups in specific focus areas such as Production, Planning & Scheduling, Services and Sustainment and, most recently initiated, Agile Development and Subcontracting. By working jointly with PARCA and DoD oversight agencies on EVMS-related policies and activities, the IPMD provides industry a voice in driving consistent practices, achieving approved EVM systems and avoiding surprises to industry. Through close collaborate with non-DoD federal agencies, disparate EVM interpretation and documentation emanating from multiple sources is resolved.

Carol S. Boser, Director of Program Performance Management (PPM), Lockheed Martin Space System Company (SSC)

Carol Boser is the Director of Program Performance Management (PPM) with Lockheed Martin Space System Company (SSC) responsible for PPM implementation specifically focusing on developing and institutionalizing consistent and robust Earned Value Management practices

across SSC. As a member of the Programs Management organization, Carol is responsible for establishing effective PPM policy, assisting programs in implementation and providing training, development, selection and assessment of all CAMs across SSC. In January 2014, Carol was elected as the Chair of the NDIA Integrated Program Management Division, the primary industry and government forum for integrated program management using earned value management (EVM) and related processes. Previously, Carol served as LM Corporate Vice President of Program Performance. She was responsible for enterprise-wide improvement in program performance by driving a transformational change in program management practices including application of proven techniques in performance measurement baseline control, earned value, risk and opportunity management, and program planning. From 2004 to 2008, Carol was Director, Programs and Technology for Lockheed Martin’s Electronic Systems Business Area, where she established and maintained performance oversight of Electronic Systems programs, ensured that technology investment and relevant strategic alliances were both effective and efficient, and supported new business pursuits. Carol served as Chairman of the Electronic Systems Program Management Council (PMC), comprised of over 1,200 professional program managers with annual sales of ~$12B. Carol holds an Executive Masters of Business Administration (MBA) degree from the Crummer Graduate School of Business at Rollins College and a Bachelor’s degree in Physics from Florida Atlantic University. NDIA 02 - Role-Based Performance Management: The Team Makes Things Happen Each member of the program team plays an integral role in the success or failure of the program. Knowing who is accountable for what is imperative on any program team. The role-based competency model describes the integrated program management skills required by each member of the program. In this workshop participants will discuss the roles of the individuals on the program team and how they contribute to integrated program management.

Gordon M. Kranz, Deputy Director, Earned Value Management, PARCA


Mr. Gordon M. Kranz is the Deputy Director for Earned Value Management Performance Assessments and Root Cause Analyses in the Office of the Assistant Secretary of Defense for Acquisition. Mr. Kranz’s office serves as the Department of Defense focal point for all policy, guidance, and competency relating to Earned Value Management. Earned Value Management (EVM) is one of DoD’s and industry’s most powerful integrated program management tools used by the government and industry program managers and their teams to support decision making as they navigate the day-to-day constraints and risks that all DoD programs face. Prior, Mr. Kranz held positions as the Executive Director, Engineering and Analysis, for the Defense Contract Management Agency (DCMA) and the Director, Systems and Software Engineering, in the Office of the Deputy Under Secretary of Defense (DUSD) for Acquisition and Technology (A&T). In these roles, Mr. Kranz provided policy, training, and tools required to perform systems engineering, software engineering, earned value management, manufacturing and production, quality, system assurance, risk management, and supply chain predictability.


Mr. Kranz has 30 years of defense acquisition experience, including 16 years in private industry as a senior technical lead and program manager and 10 years as an acquisition program manager for the United States Air Force. Mr. Kranz received a Bachelor of Science degree in Electrical Engineering from North Dakota State University, a Master of Science degree in Electrical Engineering from the Air Force Institute of Technology, and is certified by the Project Management Institute as a Project Management Professional. NDIA 03 - Application of Earned Value Management (EVM) in the Manufacturing, Production and Naval Construction Environment (PMNC) The value of EVM on a development program is practically a given in today’s environment. The application of the thirty –two criteria on a production program, in a manufacturing environment or in naval construction may require modifications to a system description or process that may not be readily obvious: the critical path for fifteen units is not a development program concept. From the Executive Summary of the NDIA White Paper “Earned Value Management in a Production Environment” issued in October 2011. “While the historical EVM emphasis has been on DoD development contracts, DoD production contracts, sustainment contracts and many commercial or non-DoD projects also utilize EVM. However, because of the historical development project emphasis, the majority of government and industry personnel have EVM experience emanating from that type of effort. As such, when EVM is applied to a production project there is often confusion or misunderstandings and misinterpretations related to the unique terminology, processes or management practices that take place in a production environment relative to a development one.” The NDIA IPMD PMNC Working Group has taken the task of identifying and providing explanations and guidance regarding the differences when using the DCMA Cross Reference Checklist as guidance for System Description compliance. This session will review the history and possible future of those efforts.

John Kanicsar, Orbital


John has been associated with Program Management and Earned Value in the aerospace industry since 1978. His initial exposure to cost and performance reporting was with Goodyear Aerospace in the Reconnaissance Image Analysis business area. Successful accomplishments in these areas lead to assignments as a program manager with the Aeromechanical product group for: canopies for the F-15 and T-46 ; Mobile Photographic Processing and Analysis Systems for various FMS customers and a multiyear contract for the Launch Control Shelter for the Tomahawk Ground Launch Cruise Missile. During the1990’s industry restructuring, John joined Honeywell’s Space & Satellite Systems > Assignments included the following product areas: reaction wheel / control motion gyro assemblies, used to provide attitude control for satellites; roll ring assemblies used for data or power transfer; solar and thermal arrays and Multiplexer-Demultiplexers used on the International Space Station (ISS).


He joined the Performance Management Association (PMA) in 1992 and was president of the state chapter when PMA joined with Project Management Institute (PMI) in 1999. John served on the PMA national board of directors during the joining period. John joined Orbital Sciences Corporation, Launch Systems Group (LSG) in February of 2002. Current duties include; coordination of Joint Surveillance, Internal Surveillance, oversight of EV training, EVM point of contact with the resident DCMA team and owner of LSG’s System Description. John is also leading the NDIA’s Performance, Manufacturing and Naval Construction (PMNC) Working Group.

Matthew Schmidbauer, Orbital

Phone: 480-722-3307 Email [email protected] After 10 years in the Financial Planning Industry, Matthew joined Orbital Sciences Corporation, Launch Systems Group (LSG) in 2012. Working as a Financial Analyst on the Ground-Based Midcourse Defense (GMD) Program he quickly transitioned to the Earned Value Management Team. Currently he supports; DCMA Joint Surveillance Reviews, Internal Surveillance, Earned Value Training, System Description Changes, and acts as a liaison between Program Offices & the local DCMA team. Matthew is also a member of National Defense Industrial Association (NDIA), and College of Performance Management (CPM). NDIA 04 - IPMD Predictive Measures Guide The Predictive Measures Guide, recently approved for publication by the NDIA Integrated Program Management Division (IPMD) membership, is intended to provide a “menu” of metrics that organizations can use to assess the status of programs. The guide used, as a starting point, the study on predictive measures conducted by the NDIA ICPM (Industrial Committee on Program Management) in 2008 which resulted in a PowerPoint presentation, but was not widely publicized. This guide began with a re-assessment of the original study and its proposed 24 measures, adding some additional measures and deleting others, and documenting the measures as a more usable Word document in a standard format. The guide identifies 35 predictive metrics. The intended audiences for this guide are organizations (government and industry) that are looking for standard approaches to manage programs. This guide is not intended to necessarily provide a new set of standards that would be required to assess program performance, but instead provide a “menu” of typical measures that could be applied. Each organization should decide which measures are most appropriate for their environment and select only those measures suitable for them. This presentation will cover how the guide was developed and an overview of the guide itself.

Bill Altman, Program Manager, Battelle

Phone 614-327-4342 Email [email protected] Bill Altman returned to work at Battelle as a Program Manager for Special Projects. He retired from Battelle in September 2010 after 23 years of service rising through the ranks as Program Manager, Senior Program Manager and Vice President of the Program Management Office in Battelle Corporate Operations (BCO). While leading project management at Battelle, he was



also responsible for the Proposal Operations Center. Bill provided interface to the organization on issues that deal with proposal and project management. Bill brings over 30 years of experience in managing and conducting advanced development programs and in the evaluation of major bid opportunities. He received certification as an Earned Value Professional (EVP) from the AACEI. Mr. Altman received a BS (Physics) from Drexel University and a MSEE from the University of Pennsylvania under a Ford Foundation Fellowship. He completed all requirements for a PhD from the University of Maryland (except for the thesis).

Sung Soon Stultz, Rockwell Collins

Phone 319-295-3849 Email [email protected] Ms. Sung Soon Stultz is the Director Program Planning and Control (PP&C) for the Government Systems Business Unit of Rockwell Collins, Inc. Ms. Stultz is responsible for providing overall leadership and guidance to Government Systems in its Project Management policies and procedures, disciplines and evaluations on all Government Systems projects and programs. Ms. Stultz joined Rockwell Collins in 1979 and has served in various functions at Rockwell Collins including Industrial Engineering, Technical Director, Lean Director, Pricing Director and Business Systems Analysis with major focus on systems implementation to satisfy Earned Value Management Systems, Finance and Engineering. Ms. Stultz received a Bachelor of Science degree in Systems, Science and Mathematics from Washington University in St. Louis and a Master of Business Administration from University of Iowa. NDIA 05 - IPMD Clearing House Working Group Practice Symposia The IPMD Clearing House Working Group was created to provide a forum for industry’s EVMS and other integrated program management practitioners to raise and discuss related issues. The goal is to identify common issues, gather facts and supporting data, formulate industry positions, and to develop recommendations for issue resolution. This will be an open forum for industry to raise and discuss related issues. No two sessions are ever the same.

Pete Wynne - Lockheed Martin Space Systems Company and Past Chair, NDIA IPMD


Pete Wynne is a member of the Lockheed Martin Space Systems Company (SSC) Program Performance Management team providing expertise and guidance on all EVMS matters. Pete advises programs on effective and efficient implementation of PPM/EVM through application of proven techniques and guides the SSC policy board responsible for maintaining a compliant EVM system description. Most recently, Pete was Corporate Director of Program Performance Operations responsible for the development of Lockheed Martin’s corporate PPM / EVM policies and processes and for interfacing with government agencies on LM’s implementation of EVM principles. Prior to



joining the corporate staff, Pete was Director, Program Finance, at Lockheed Martin Aeronautics, where he established and maintained the Aeronautics Earned Value Management System across diverse manufacturing sites delivering airlift and fighter aircraft and advance development products. Pete has also served as Chairman of the NDIA IPMD.

Mellissa Gilbert – Orbital

Phone 480-814-6936 Email [email protected] Mellissa Gilbert is a Director of Program Finance for Orbital Sciences Corporation. She has over twenty-five years of experience in both commercial and government sectors in the areas of financial accounting, cost accounting, financial analysis, and earned value management. Currently, she is the managing director of the Launch Systems Group Program Finance organization. Her organization is responsible for performing earned value management and financial analyses for government and non-government contracts totaling over $400M annual revenue. NDIA 06 - Civilian Agencies and Industry Working Group (CAIWG) Update The CAIWG is a joint government and industry working group. The goal of the CAIWG is to promote best practices and sharing of ideas that demonstrate real value to Civilian Agencies and Industry. This is accomplished by reaching out widely to all segments of the project management and acquisition communities to share information and socialize results. Currently, the CAIWG has three initiatives that are being worked by the various subgroups; 1) Program/Project management across civilian agencies, 2) EVM reciprocity agreements, and 3) Scalability of EVM. This presentation will provide an overview of the CAIWG, such as objectives, goals, etc. It will also provide a status of the sub-team activities and the expected outcomes.

Jerald G. Kerby, NASA

Phone 256-544-3243 Fax 256-544-1390 E-Mail [email protected]

For the past 24 years, Jerald has worked for NASA at the Marshall Space Flight Center (MSFC). Currently, he is the NASA Earned Value Management (EVM) Program Executive. In this role, he serves as the Chairperson for the NASA EVM Working Group (EVMWG) that supports the implementation of EVM throughout NASA, along with the development of EVM policy and handbooks within the Agency. He also is responsible for providing both classroom and hands-on EVM and data analysis training to the project management community at NASA and MSFC. In addition, Jerald led the development of a fully compliant EVMS used by NASA projects to comply with the OMB Circular A-11 requirements. He currently serves as the Co-chairperson for the Civilian Agency Industry Working Group (CAIWG). He has served as a speaker at many Project Management Institutes (PMI) and NASA Project Management conferences throughout the years. In addition, he has co-authored a chapter in the book entitled ‘Applied Project Management for Space Systems’. He co-authored an article that was published in the Spring 2005 edition of the Academy Knowledge Sharing (ASK) magazine entitled ‘Putting EVM to the Test’. He also received the prestigious NASA Outstanding Leadership Medal. Jerald has a Bachelor of Science Degree from Athens State University. He has a MS Degree from the Florida Institute of Technology (FIT). He is also a Certified Public Accountant (CPA).



Practice Symposia ICEAA Track

The International Cost Estimating and Analysis Association (ICEAA) is an international non-profit organization dedicated to advancing, encouraging, promoting and enhancing the profession of cost estimating and analysis by providing members opportunities to learn, teach, collaborate, present and network in a collegial environment devoted to professional growth and success. Presentations in this track will provide background in software cost estimating, integration of cost and schedule risk, as well as cost estimating in an integrated program management environment. Attendees can earn Continuing Education Units (CEUs) for attending sessions in this track Coordinator: Megan Jones COST 01 A Comprehensive CES and BCA Approach for Lifelong Learning

Kevin Cincotta, ICF International Dr. Darcy Lilley, Air Force Air Mobility Command COST 02 Bayesian Parametrics: How to Develop a CER with Limited Data and Even Without Data Dr. Christian Smart, Missile Defense Agency (MDA) COST 03 An Update to the Use of Function Points in Earned Value Management for Software Development

Daniel French, Cobec Consulting, Inc. Michael Thompson, Cobec Consulting, Inc.

COST 04 Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition Programs

Alan Gideon, Booz Allen Hamilton COST 05 The NASA Project Cost Estimating Capability

Andy Prince, NASA/Marshall Space Flight Center Brian Alford, Booz Allen Hamilton

COST 06 Trust but Verify – An Improved Estimating Technique Using the IMS Eric Lofgren, Technomics, Inc. COST 01 - A Comprehensive CES and BCA Approach for Lifelong Learning The Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization. AMC ELO is initially focusing on training. Training is generally represented as only one line within a cost estimate. We developed a first-of-its-kind comprehensive Cost Element Structure (CES) for training. The CES decomposes training into its core elements. This paper presents the training CES, and preliminary results of the BCA.

Kevin Cincotta, Technical Director, ICF International

Email: [email protected] Kevin Cincotta joined ICF as a Technical Director in November 2013. His primary areas of expertise are cost analysis, statistics, and training. He is the Financial Analysis Task Lead for ICF’s support of the AMC Enterprise Learning Office. Prior to that, Kevin worked at Technomics, where his clients included the DAU, DHS, DASA-CE. In addition, Kevin created the Technomics Training Institute, which trains junior and mid-level costs analysts.

Dr. Darcy Lilley, Chief Learning Officer, Air Force Air Mobility Command

Email: [email protected] Darcy Lilley serves as Chief Learning Officer for the Air Mobility Command (AMC) supporting over 132,000 Airmen worldwide. She is leading the effort to transform AMC into a premier learning organization, achieve learning through optimum approaches, and develop Mobility Airmen into life-long learners who demonstrate institutional Air Force competencies with a positive approach to managing their own learning. She holds a Master’s in human resource development from Webster University and BS from San Diego State. COST 02 - Bayesian Parametrics: How to Develop a CER with Limited Data and Even Without Data When I was in college, my mathematics and economics professors were adamant in telling me that I needed at least two data points to define a trend. You may have been taught this same dogma. It turns out this is wrong. You can define a trend with one data point, and even without any data at all. A cost estimating relationship (CER), which is a mathematical equation that relates cost to one or more technical inputs, is a specific application of trend analysis. The purpose of this paper is to discuss methods for applying parametrics to small data sets, including the case of one data point and the case of no data. The only catch is that you need some prior information on one or more of the CER’s parameters. For example, consider a power CER with one explanatory variable. The slope of the equation can be interpreted as an economies of scale factor. As such, it is typically between 0 and 1. When using weight as the explanatory variable, rules of thumb are 0.5 for development cost, and 0.7 for production cost. Bayes’ Theorem can be applied to combine the prior information with the sample data to produce CERs in the presence of sparse data. This paper discusses Bayes’ Theorem, and applies it to linear and nonlinear CERs, including ordinary least squares, log-transformed ordinary least squares, and other methods.

Dr. Christian Smart, Director of Cost Estimating and Analysis, Missile Defense Agency (MDA)

Email: [email protected]

Dr. Christian Smart is the Director of Cost Estimating and Analysis for the Missile Defense Agency (MDA). He is responsible for overseeing all cost estimating activities developed and produced by the agency, and directs the work of over 100 cost analysts. Prior to joining MDA Dr. Smart worked as a senior parametric cost analyst and program manager for NASA contractors. Dr. Smart spent several years overseeing improvements and updates to the NASA/Air Force Cost Model and developed numerous cost estimates, cost models and techniques that are used by Goddard Space Flight Center, Johnson Space Center, Marshall




Space Flight Center, and NASA HQ. In 2010, he received an Exceptional Public Service Medal from NASA for his contributions to the Ares I Joint Cost Schedule Confidence Level Analysis and his support for the Human Space Flight Review Panel led by Norm Augustine. He has won seven best paper awards at ISPA, SCEA, and ICEAA conferences, including five best overall paper awards. Dr. Smart was named the 2009 Parametrician of the Year by ISPA. He is an ICEAA certified cost estimator/analyst (CCEA). Dr. Smart is a past president of the Greater Alabama Chapter of SCEA, a past regional VP for SCEA, and is the managing editor for The Journal of Cost Analysis and Parametrics. Dr. Smart earned bachelors degrees in Economics and Mathematics from Jacksonville State University, and a Ph.D. in Applied Mathematics from the University of Alabama in Huntsville. COST 03 - An Update to the Use of Function Points in Earned Value Management for Software Development In this follow up presentation to their 2013 IPM/ICEAA presentation, describing the successful implementation of an EVM methodology for a software development project utilizing the International Function Point User Group (IPFUG) function point software sizing metric. The process developed by the authors addressed this by application of the International Standards Organization (ISO) approved function point software sizing methodology to measure EVM during the course of the software development lifecycle.

Daniel B. French, Principal Consultant, Cobec Consulting Inc.

Email [email protected] Dan French has over fifteen years of experience in a variety of Information Technology roles including analyst, developer, tester, project manager, software metrics and software project estimation. Additionally, he has developed CMMi and ISO compliant process improvement and software estimation processes. An IFPUG member and Certified Function Point Specialist (CFPS) since 2000, Dan currently serves on the FSSC. He is also a PMP as well as a member of ICEAA. Michael Thompson, Director, Cost and Performance Management, Cobec Consulting, Inc.

Email [email protected]

Mike Thompson has been a part of the cost estimating, analysis, scheduling, and earned value community for over 30 years and has worked for a number of defense contractors either as an employee or consultant, as well as various government agencies. Currently Mike is supporting the FAA a Director, at Cobec Consulting. His responsibilities include providing guidance, as a Subject Matter Expert, on EVM and schedule matters. COST 04 - Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition Programs This paper modifies the 30 year old Asher-Maggelet model of future program acquisition cost to account for trends in Department of Defense cost outcomes by using recent platform-specific median cost growth outcomes. An exploratory analysis of cost trends from the 1960s to the present shows that the original model significantly over-estimates cost, making it unusable. The relationships between accurate individual program cost estimates, contract negotiations, and overall portfolio/department risk are discussed.



Alan Gideon, Senior Systems Engineer, Booz Allen Hamilton


Alan Gideon is a senior systems engineer with Booz Allen Hamilton, currently assigned to the Department of Defense. His 20 years in the U.S. Navy included ship operation, design, repair, and construction. Mr. Gideon holds a B.Sc. in Mechanical Engineering from UNM, and a SM. in Naval Architecture and Marine Engineering and OCE from MIT. He is a Ph.D. candidate at The George Washington University; his chief area of research is risk management. COST 05 - The NASA Project Cost Estimating Capability The paper provides a detailed description of the capabilities and shortcomings of the NAFCOM architecture and how what we learned from NAFCOM, plus new estimating requirements, is leading NASA to develop the Project Cost Estimating Capability (PCEC). The requirements for the PCEC are discussed, followed by a description of the PCEC architecture. Finally, the paper provides a vision for the future of NASA cost estimating capabilities.

Andy Prince, Manager, Cost Engineering Office, NASA/Marshall Space Flight Center


Andy Prince is the manager of the Engineering Cost Office at NASA/Marshall Space Flight Center. His office is responsible for parametric cost estimates and analyses, maintenance of the REDSTAR Library, data collection and analysis, and cost model development. Andy has received numerous awards during his career, including the SCEA Cost Estimator/Analyst of the year.

Brian Alford, Operations Research Analyst, Booz Allen Hamilton

Email [email protected] Brian Alford is an operations research analyst at Booz Allen Hamilton currently supporting the Marshall Space Flight Center's Engineering Cost Office as a cost analyst. His area of expertise focuses primarily around designing and implementing models for conducting cost and decision analyses in support of portfolio- and platform-level decision making. He has a Bachelor's degree in Physics, a Master's degree in Applied Mathematics, and holds his CCE/A certification. COST 06 - Trust but Verify – An Improved Estimating Technique Using the Integrated Master Schedule What’s the state of the Integrated Master Schedule (IMS) today? Analysis of actual contract data suggests declining task performance, numerous binding constraints, unraveling logic, and optimistic forecasts. These factors in part lead to a failure of IMS predictive power. By analyzing IMS development across submissions, this paper will show that early performance to baseline is a good measure of realized slip and provides the framework for a new estimating technique.




Eric Lofgren, Cost Analyst, Technomics, Inc.

Email [email protected] Eric Lofgren is a cost analyst for Technomics, Inc. His analytical focus has been in the field of Earned Value Management (EVM), in particular developing new data visualization techniques for OSD CAPE. Mr. Lofgren has also supported other EVM related tasks, such as support to Program Offices, CAPE analysts on Defense Acquisition Board (DAB) reviews, GAO Record of Analyses on contract cost/schedule estimates, and a Congressional study on the Space Industrial Base.


practice symposium sessionsipmconference.org/wp-content/uploads/2012/06/practice-symposia... · ps...

Documents