1.Complexity Challenges in the Integrationof Systems and Organizations Does Systems Engineering need an Overhaul?NASA PM Challenge 20121

2. AgendaComplexity Challenges in the Integration ofSystems and Organizations Does Systems Engineering Need an Overhaul? Looking at Complexity from the Outside In Complexity & Teams Dialogue 2 3. Does Systems Engineering Need an Overhaul?Michael C. Lightfoot NASA Langley Research Center, Hampton, VA PM Challenge 2012, Orlando, FL February 22, 20123 4. Systems Engineering is Being Placed Underthe MicroscopeThere is a growing number of engineering communities who are askingtough questions about the current practice of Systems Engineering.Tough Questions: Why do the current SE processes, if rigorously applied, not guaranteeus safe, effective, robust systems delivered on time and within budget? What is it about our methods, processes and tools that seems to fail innewsworthy fashion when we attempt to design and build large-scalesystems. Has our SE system somehow evolved to become a system that defiesour control?Why the tough questions now?4 5. Systems Engineering TrendsSystem Size and Complexity has increased:One Example*: F-16, 15 Subsystems, 103 InterfacesF-35, 130 Subsystems, 105 InterfacesOrganizations: Size increase (100s to 1000s), most likely global teams, different cultures w/ different incentives multiple companies, many reporting structures, sometimes competing incentivesSubsystems that were once modular in design are nowirreducibly entwined (tightly coupled)Many systems are one of a kind (NASA) or limited quantityproductions* Data courtesy of United Technologies Research Center:https://www.fbo.gov/download/9cb/9cb78f01aa9db1fe92e093e786bc6733/Abstraction_Based_Complexity_Management_Final_Report_Dist_A.pdf5 6. Increases in Aerospace Systems ComplexityPaul Eremenko, DARPA, META Program6 7. Large- Scale Complex Engineered Systems7 8. Characteristics of Large-Scale Complex Engineered Systems Increased Engineering Complexity Highly-coupled interfaces, many of which are only discovered during integration & testing or system operation. Design Cycles are Longer and More Complicated Significant Cost and Risk Extremely high political and monetary risk Low tolerance for failures or degraded performance Public fear of catastrophic failure is high Limited opportunities to experiment (trial and error) Very Large, Dispersed Engineering Organizations Yet organizations are expected to function synergistically Coordination and data exchanges are greater in frequency and volume of data. Unlike the early days of SE, no one Chief SE is able to keep the entire system view in his/her head.8 9. Classes of Engineered Systems (Relative Comparisons, Not Rigorous Definitions)Simple System: Consist of few parts, Small number of interfaces Interactions well understood & well controlled, Typically used as building blocks for more sophisticated parts & componentsComplicated Consist of many parts, components, subsystemsSystem: Moderate to large number of interfaces Interactions/reactions understood for controlled cases Vigilant control required to properly construct V & V is the basis to accept/reject bad parts, components, subsystems Global system behavior is mostly predictable; Part decomposition & analysis leads to reasonable global property predictionsComplex Can possess extreme numbers of parts, components, subsystemsSystem: Extreme numbers of interfaces- sometimes impossible to identify Interactions understood for limited number of highly controlled cases but mostly unknown due to dynamic adaptations Vigilant control often exercised but system sensitivity is nonlinear & dependent on initial conditions (path dependent). Current analysis tools are poor predictors of system behavior Complete system V & V not possible. Global system behavior can be emergent (reductionist approaches fail) 9 10. Complicated System Example Star Caliber Patek Phillipe mechanical watch. We understand: how it is constructed, the required tolerances, the order of assembly. Each component works in unisonto accomplish a global function: keeptime precisely. We can take a reductionist path todefine the smallest required partsand can further write equations ofmotion to predict the performanceand functionality of the watch. 10 11. Complex Systems Through a Complexity Science Lens Dynamical systemsDynamical/non-linear Highly-coupled System response is non-linear & sensitive to initial conditions Consist of many parts, components, or subsystems (agents) that interactAdaptive with each other & the environmentCan be Self- organizing They learn & adapt their behaviors to survive If the adaptation strategy is good they continue to exist If the strategy is bad or non-existent they cease to existGlobal behaviorshappen without a They can move from an ordered to disordered statecentralized controller unpredictably, and can be self-organizing Reductionist No centralized controller approaches do notdescribe global behaviors Knowledge of the inner workings of each agent typically shed no information about the global behavior/response of the system 11 12. Examples of Complex SystemsDynamical/non-linear Ant colonies Rain forests Highly-coupled Communities where you live U.S. Power Grid The World Wide WebAdaptive The Stock Marker Propagation of infectious diseasesCan be Self- organizing The Global Economy (financial system collapse 2008) The Occupy ?? Protest Groups Multinational corporationsGlobal behaviorshappen without a The NASA employees and contractors who supported thecentralized controllerConstellation ProgramThe various engineering organizations that developed specific flight Reductionist hardware for Pad Abort Activities approaches do notThe NASA PM and SE groups that supported Constellationdescribe global behaviorsComplex Systems can be Technical(Engineered),Biological, Social or some combination12 13. Domains of ComplexitySocial TechnicalComplex Engineering OrganizationsSocio- TechnicalCreating Complex Engineered Systems 13 14. Why is a Complexity Science FrameworkImportant to the SE Community? Current SE processes consists of experientially-based guidance. Although this guidance is tailorable, it is not deterministic. There currently is no theory, nor science of system engineering thatenables us to predict the efficacy, resilience or robustness of the systemswe produce. Our gut tells us that organizations impact the products we create butwe have no analytical tools to express the relationship between the two. A complexity science framework encourages us to question theexistence of dynamical relationships where we formerly assumed no orlinear relationships existed. This includes interactions between socialsystems and technological systems. Many of the basic tenets/tools of complexity science are quite familiarto engineers that work in dynamical systems (chaos, non-linear behavior,neural networks, genetic algorithms, graphical modeling & simulationtools tools, etc.)14 15. What are the building blocks needed to grow a competency in Complex Engineered Systems? ?????? Listen, Share and Solve Explore, Understand, problems across Integrate social systems disciplines & use new complexity into our decision tools in novel ways.making & SE processesHolistic Systems Uncertainty-BasedStatistical Thinking &Thinking Modeling and SimulationProbabilistic Uncertainty[ embrace non-linearity ]Tools and Techniques Analysis15 16. Potential Domain InfusionsScience ofTrans-disciplinarySocio-Technical EngineeringSystems ScienceSocialTechnicalComplex EngineeringOrganizationsSocio- Technical Creating ComplexEngineered SystemsEngineering of Systems EngineeringActivities16 17. NSF/NASA Workshop on Design of Large-scale Complex Engineered Systems February 7-8, 2012 Arlington, Virginia Organizers:Steven McKnight, NSF Vicki Crisp, NASAChristina L. Bloebaum, NSF Anna-Maria McGowan, NASAGeorge Hazelrigg, NSFMichael Lightfoot, NASA Paul Collopy, University of Alabama, Huntsville 17 18. Workshop OverviewObjective: Examine the challenges unique to large-scale complex engineeredsystems Examine how we can better prepare for a future of growing systemcomplexity?Four Topic Areas Explored:1. New approaches to system complexity by framing it through a complexity science lens.2. Current developments in design science and how might they help us in designing within the SE process.3. Awareness of what is known in organization science and how the engineered product is a function of the organization.4. How decision science can provide a more rigorous approach to decision making in large-scale project teams. 18 19. Who Attended the NSF/NASA Workshopon The Design of Large-Scale ComplexEngineered Systems? A total of ~115 people in attendance Government:NSF, NASA, DoD (ODASD, AFRL, AFOSR, ONR, NRL, ARL), V-DOT Academia (25): University of Illinois at Urbana-Champaign, University of Minnesota, George Mason University, University of Maryland, Northwestern University,University at Buffalo SUNY, Purdue University, Schulich School of Business, York University, North Carolina State University, Georgia Institute ofTechnology, Pennsylvania State University, Texas A&M University, Oregon State University, Stevens Institute of Technology, Johns HopkinsUniversity, University of Virginia, University of Michigan, University of Florida, Brigham Young University, Massachusetts Institute of Technology, Iowa State University, Stanford University, George Washington University, Mills College Industry & Others:Lockheed Martin, Boeing, MITRE, SpaceWorks, Global Project Design, Google, NAE and others Disciplines Represented:Engineering, Social Science, Cognitive Science, Organization Science,Anthropology and Economics 19 20. My Workshop Takeaways Systems Engineering as practiced is laden with human decision making whichcould be enhanced by the understanding & practice of decision science SE needs to embrace nonlinearity and embrace a future where the systems webuild will not be fully testable (within the current practice of V&V). In order to better design & build large-scale complex engineered systems of thefuture we need to 1st build better relationships between: Complexity Science Researchers Engineering Design Science Researchers Organizational Science Researchers Systems Engineers (PM+SE) Optimization Researchers S & T Leaders within Government Agencies Government participant agreed to form a Community of Practice to exploit uniquestrengths that NSF, NASA, and DoD can bring to the challenge of large-scalecomplex engineered systems.20 21. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul? Looking at Complexity from the Outside In Complexity & Teams Dialogue21 22. Looking at Complexity from the Outside Ina fresh look including outside our current processes Ed Rogan NASA PM ChallengeFebruary 22, 2012 www.gpdesign.com | info@gpdesign.com 23. What isComplexity Complexity means different things in different technical and professional contexts We encounter most of them in practice A common language accessible to non-experts would be usefulSlide 23Global Project Design 2012 www.gpdesign.com 24. Complexity as length of a bit stringCompressibleBit(Kolmogorov, 1965)Strings What is the shortest computer program thatwill output a given bit string? Simple: 0101010101010101 Slightly more complex:3.14159265358979323846. A definition of randomness: a string thatcannot be compressed to any shorter Slide 24 computer program Global Project Design 2012www.gpdesign.com 25. Complexity as time required to find a solution Complexityofto a computational problem (e.g. factoring aComputation large composite number, scheduling, routing)(Cook, 1971) If we can verify an answer quickly (timebounded by a polynomial function of the inputlength), can we also find an answer quickly? Probably not in all cases. P = NP?. $1 million prize remains unclaimedfor solving. Slide 25 Global Project Design 2012www.gpdesign.com 26. Paradox: how can unpredictable behaviorComplexDynamics result from the laws of classical physics? Examples: celestial mechanics (Poincare, 1895), fluid dynamics (Lorenz, 1963) Nonlinear terms amplify small differences in boundary or initial conditions Solutions to compressible Navier-Stokes equations exhibit qualitative changes in behavior with changes in a parameter (e.g. Reynolds number, Mach number) bifurcation, strange attractors, and chaos.Slide 26Global Project Design 2012www.gpdesign.com 27. Large, highly interconnected networkedOthersystems withsystemscomplex Hybrid (discrete-continuous or digital-analog)dynamicssystems Example: brains. 80 - 100 billion digital-analog/analog-digital converters Up to 10,000 inputs to a single converter Emergent behaviors: decision-making, attention Slide 27 Global Project Design 2012 www.gpdesign.com 28. Example: stock marketsComplexInteractions Assume decision makers are rational. All available in information about the value of a stock is reflected in its price. Decision-How can stock markets crash? Making Decision-makers are not always rational (noise traders,prospect theory, risk of arbitrage). Sometimes, buyers and sellers in the stock market choosenot to reveal all of the information that they know about thevalue of stocks. Or, decision-makers as a group may have more knowledgethan they have as individuals (muddy children puzzle). Anevent may make this information common knowledge. When previously hidden information becomes commonknowledge, behavior of many (rational) buyers and sellers Slide 28 can (and does) change quickly. Global Project Design 2012 www.gpdesign.com 29. Software (Kolmogorov and P = NP?) Summary:What Digital-analog interconversion (hardware-softwareComplexitiesinterfaces)do we face inEngineering Nonlinearity (fluids and structures) Systems? Large systems with many interfaces, dependencies, orcouplings Human decision makers Sharing or exchange of knowledge and information Risk and uncertaintyComplex systems have elements we haventconsidered in the past Slide 29 Global Project Design 2012www.gpdesign.com 30. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?Looking at Complexity from the Outside In Complexity & Teams Dialogue30 31. Complexity & TeamsWhat multi-disciplinary research showsabout behavior in socio-technical systems Bryan MoserNASA PM ChallengeFebruary 22, 2012www.gpdesign.com | info@gpdesign.comSlide 31 32. Who is GPD? Technology Leaders from Complex Global Industries U. Tokyo: Graduate School of Frontier Sciences The Design of Global Projects Rapidly prototype and adjust plans Predict coordination activity Drive attention to interactions of value GPDs Methods & Experience Visual Modeling of integrated socio-technical architecture Behavior based simulation including global factors 15 years of case experience in industry: 3/ month globally GPDs Partnership Agenda Measures of Coordination by Humans in the Loop Leverage observation and massive sensing Practicability of new techniquesSlide 32Global Project Design 2012www.gpdesign.com 33. Models of organization have shifted fromShiftingcentrally controlled mechanical systems to Models ofdynamic organisms with distributed,Organizationadaptive, and behavior based subsystems. planning and forecasting, organizing,commanding, coordinating, and controlling(Fayol, 1916) structure, hierarchy, authority, roles(Weber, 1924) as systems with boundaries, goals,incentives, behaviors (Simon, 1962) differentiation, formalization, complexity,centralization, span of control, rules,procedures (Burton, 1995 and others) Slide 33 Global Project Design 2012www.gpdesign.com 34. Work as a Product Development viewed as a Socio-socio-technical system if we includeTechnicalHumans in the Loop System People do work, process information, and interact as part of an organization Individuals allocate attention based on behaviors within limited capacity Organizations with architecture exhibit emergent behavior (e.g. exception handling, quality)Slide 34Global Project Design 2012www.gpdesign.com 35. Engineering What if an IT system allowed in Socio- teams with access to all information? technicalsystems all processes clearly written? workflow software to support tasks? If requirements, work packages, anddependencies are clearly written andassigned, is performance guaranteed?What about human performance duringcomplex work isnt addressed above? Slide 35 Global Project Design 2012 www.gpdesign.com 36. Coordination Coordination is the activity to managedependencies. What portion of your weekly effort isspent coordinating? What happens to items in your inboxwhen it overflows?Manufacturing has shown for decadesthat managing human attention is a key:if we over-automate, quality drops Slide 36 Global Project Design 2012www.gpdesign.com 37. Three activities to realize aArchitecturesubsystem. Three teams. &Coordination Independent activities.Where will coordinationoccur? Dependent activities.Where coordination? Changed pattern of rolesand dependence, yetscope and resourcesunchanged.Where Coordination?The demand and supply of coordination activity are drivenby the integrated architecture of the project. Slide 37 Global Project Design 2012www.gpdesign.com 38. Architecture Teams have structure. & QualityWhat coordination is this?What impact on performance? Exception Handling Dependent activities.Why does capacity of Team_1now matter?And in this case? What if:Teams in different time zones?Teams speak different nativelanguages? Organization attributes matter. They can beobserved, measured, and their impacts predicted. If we do not explicitly predict, we assume that Slide 38 teams behave according to (our) past experience. Global Project Design 2012www.gpdesign.com 39. Case Can one predict coordination and its Example impact on a programs likely duration, cost, and risk? If we can see impacts of complexity ahead of time, what might we do to: Reduce scope? Re-organize the system? Re-system the organization?Slide 39Global Project Design 2012www.gpdesign.com 40. Coordination Coordination, Low Utilization, is Predicted & Rework Predictedin a Complex Not only the amount of Industrial coordination, but when, where, Case and WHY it is demanded 40 Global Project Design 2012www.gpdesign.com 41. Resources: Teams Size, Location, Calendar, AbilitiesArchitectural choices within a Architecture: Dependence, Roles, OBS, WBS & PBS Scenarios: complex program are a leverfor better performance Externals: Targets, Start, Supplier DeliveryBasis of Scope: Activities, Direct Work, ComplexityProject Model 30 Change25201510 # of scenarios 5 0 WKSP 1 day 1 WKSP 1 day 2 WKSP 1 day 3WKSP 2 WKSP 3WKSP 4Workshop Session Global Project Design 2012www.gpdesign.com 42. Interfaces Does an interface allow reduced or no interaction? In what horizon? Is an interface a call to interact? Should a team pay more attention to the interface? What happens when an interface breaks? Our teams need to be engaged, to interact and respect that which we yet dont know or will discoverSlide 42Global Project Design 2012www.gpdesign.com 43. Normally work cultures evolve over time to Design ofalign behaviors, promote learning, and controlProjects in a risk. Complex Through a stable career one knew which interactionsEnvironment mattered, and others were on the same page. Today coordination and risk arise in unexpectedplaces Attention to coordination is not soft. Theseare real attributes of time, cost, and quality Cases in complex global industrial programsconfirm observed behaviors and sufficientpredictability Design of a project architecture can weighteam capacities and strengths, coordination,and flexibility Slide 43 Global Project Design 2012www.gpdesign.com 44. AgendaComplexity Challenges in the Integration ofSystems and OrganizationsDoes Systems Engineering Need an Overhaul?Looking at Complexity from the Outside InComplexity & Teams Dialogue44 45. Wrap Up 1. Organizations and the technicalsystems we engineer are more andmore complex 2. SE as framed today is strained andneeds to respond 3. Changes are needed which includeobservation, analysis, and integrationof socio-technical dimensionsSlide 45Global Project Design 2012www.gpdesign.com 46. Dialogue Does this hypothesis resonate with your experiences and practices? Do you agree that SE needs to evolve What tough questions should we be asking and researching about the SE process? What are best practice examples in current programs which recognize of these real world behaviors? How was the SE process adapted?Slide 46Global Project Design 2012 www.gpdesign.com