achieving a culture of safety, security, and sanity · ecosystem, organization, team, and...

© 2012 Carnegie Mellon University

Achieving a Culture of Safety, Security, and Sanity

Mike Konrad September 27, 2012 10th WOCS

2 © 2012 Carnegie Mellon University

Outline

Attention/Motivation Chain in Knowledge Work 1. Organization (CMMI; Collaboration) 2. Team (Collective Intelligence; TSP) 3. Individual (Attention; Rudeness; etc.) 4. Ecosystem (Final link)


The Attention/Motivation Chain in Knowledge Work -1

Real-time “equations” for knowledge work: • State [N+Δ] = F (State[N], Activity[N],…) • Activity [N+Δ] = F (Goal[N], Capability[N], Support[N],…) • Goal [N+Δ] = F (Goal[N], Perception[N], Support[N],…) • Capability [N+Δ] = F (Capability[N], Learning[N],…) • Learning [N+Δ] =

F (Perception[N], Activity[N-Δ], Theories[N-2*Δ], Motivation[N],…) • Perception [N+Δ] =

F (State[N], Capability[N], Support[N], Attention[N], Motivation[N],…) • Through Learning and Support, granularity of control (Δ -> 0) can be partially improved.

Abbreviations used above:

• Support is work aids: Theories and Technology; as well as Assets and Culture. • Theories are learned patterns of causality (i.e., activity => state change) and answer “why?” for:

policies, rules, principles, practices, processes, methods, and procedures. • Technology extends human capability (in perception, skills) through tools and environment (e.g., Web). • The ellipses (“…”) indicates other important factors, e.g., perhaps Kahneman’s “Heuristics and Biases”


According to [Dan Pink’s book “Drive”] we are motivated by: • Seeing progress made when working on a task • Receiving commendation for our contributions • Gaining more autonomy, self-control • Engaging in tasks greater than ourselves

According to the previous slide, both Motivation and Attention affect Perception and Learning. And Attention is a function of:

• Goals, Capability, Support, Motivation, and unexpected conditions Therefore, what each person does is strongly “conditioned” by context:

• Individual, team(s), organization(s), and ecosystem(s) • If any of these four are weak, then goals or the means to achieve them also suffer; and

thus attention and motivation may be diminished or misdirected. • Thus, we can imagine a “Chain:” an Attention/Motivation Chain

To assure some overarching quality attribute, e.g., safety, all four “links” of the Chain must be made strong and resilient.

The Attention/Motivation Chain in Knowledge Work -2 (The key roles of Attention and Motivation in knowledge work)


Ecosystem Organization Team Individual Results

At any point in time:

-what each knowledge worker does is a function of their motivation and attention

-their attention can be focused on only one non-routine situation at a time

Their goals, theories, and capabilities are determined by what happens at each “link.”

Goals, Theories, and Technology, etc. “flow” to the Individual



Ecosystem •Has Founding or Emerging goals/values/principles -(by which appropriateness of governance can be evaluated)

•Has mechanisms for: -establishing organizations -obtaining access to shared assets and limited resources -making laws and regulations -raising and addressing grievances -ensuring accountability -funding governance operations -modifying goals/values/principles -conducting joint studies and work -implementing recommendations -identifying new assets and resources to be protected -communications

•Has culture: -norms of personal and business conduct (e.g., “Pittsburgh left,” dress codes)

Organization •Governs and Operates by attending to: •1) Defining the business: •- Mission, Values, Business model & strategy

•2) Designing & Building the business:

•- People (Management, Staff)

•- Competences •- Process Assets •- Tools and work environments

•- Units, Functions, Work Teams, Supply Chain

•3) Aligning & sustaining the business:

•- Learning captured in processes & training

•- Monitoring performance •- Motivating (e.g., through recognition)

•- Tracking trends, threats, and technology

•- Key Stakeholder relationship management

Team •Sustains Discipline by attending to: •1) Defining the work (e.g., in team charter)

•2) Designing the work: •- Customer interactions (to determine needs etc.)

•- Decisions •- Cooperation, Coordination, and Sharing

•- Operational processes and protocols

•- Measures & Analyses (e.g., progress, predictors)

•- Work environment •3) Managing the work •- Building and training the team

•- Launching the work •- Risks •4) Motivating & Coaching team and its members

Individual Knowledge Worker •Attends to: •1) Fulfilling role(s) on team •- Understanding & Designing the work

•- Doing the work •- Measuring and tracking the work (immediate clear feedback)

•- Reporting on the work (e.g., progress, issues)

•2) Managing attitudes toward the work (because it affects what you attend to, what you remember, report, and notice about your work)

•- Objectivity •- Trust & Beliefs •- Biases •- Motivations •3) Reflecting on and learning from the work

•- Manage to personal and professional goals

The Four-Fold Design Problem in Knowledge Work


Res

ults

(Saf

ety)


The Attention/Motivation Chain in Knowledge Work -5 In summary, safety in the ecosystem might be approached in this way: • Determine the family of situations that may need to be attended to • Establish a theory of how safety (and other quality attributes) will be achieved • Determine which processes and technologies will implement that theory • At the ecosystem, organization, team, and individual levels:

– Assign roles and responsibilities (RR) to the processes and technologies – Allocate resources, assets, and technologies in support of those RRs – Provide training – Ensure incentives are aligned – Monitor and evaluate execution

• But individuals and teams have authority to make their own decisions in unanticipated situations (the theory can be extended later on)

• Validate the theory: adjust the theory or its implementation as needed


Outline



CMMI in a Nutshell

CMMI is an organized collection of characteristics of effective processes (as identified by its users) that provides guidance for improving an organization’s ability to manage the acquisition, development, maintenance, and delivery of products or services.

CMMI places proven approaches into a structure that • helps an organization (or appraisal team) identify critical gaps in its processes • establishes priorities for improvement • helps implement these improvements

Organizations and Lead Appraisers must use the same criteria to determine what is a “critical gap in the processes” (i.e., “missing characteristics of effective processes”).

• Those process characteristics are documented in CMMI models • Their scoring is documented in the SCAMPI appraisal method.


CMMI Product Suite

CMMI Models oCMMI for Acquisition oCMMI for Development oCMMI for Services

SCAMPISM (Standard CMMI Appraisal Method for Process Improvement) oClass A (results in ratings) oClass B (deployment) oClass C (approach)

Training o Introduction to CMMI oAdvanced training courses

Training

Models

SCAMPI


Most Reported Appraisals per Year by Country

(Performed and reported to the SEI by June 30, 2012.)


CMMI V1.3 Foreign Language Translation Status

Language Status (for CMMI-DEV V1.3) Chinese Simplified underway Chinese Trad. completed Dutch completed French completed German completed Japanese completed Korean underway Portuguese delayed Spanish completed

Language Status (for CMMI-SVC)

Arabic underway Korean underway

(Report date: July 31, 2012)


What CMMI Says About Safety and Security -1

Capability Maturity Model Integration (CMMI) Version 1.3 (2010) consists of 3 models: • CMMI for Acquisition (CMMI-ACQ) • CMMI for Development (CMMI-DEV) • CMMI for Services (CMMI-SVC)

CMMI models describe what disciplined organizations do to: • acquire, develop, deliver (and sustain) software-intensive systems

and services Safety and security are mentioned in all three models.



Quality Attributes (QAs) is a key concept in CMMI • Definition:

–A property of a product or service by which its quality will be judged by relevant stakeholders. QAs are characterizable by some appropriate measure.

–QAs are non-functional, such as timeliness, throughput, responsiveness, security, modifiability, reliability, and usability. They have a significant influence on the architecture.

Thus, safety and security are QAs.



QAs are the focus in the engineering-related process areas of CMMI. Engineering systems begins with understanding requirements. CMMI-DEV covers how QA requirements (including safety and security) are:

• elicited (including from whom) • analyzed • prioritized • allocated • addressed in system architectures • addressed in lower-level designs and implementations • verified, prototyped, simulated, and validated



Why are safety and security addressed at such a high level in CMMI? • It is rare that only safety or security are important.

– A set of QAs, as well as safety and security, are often critical. • A holistic approach is needed to successfully engineer a system.

– Solution approaches that favor some QAs may disfavor others (e.g., trading future modifiability for improved performance).

– Or reinforce others, e.g., security => (often) safety: • security vulnerabilities => system availability, functionality, or

performance issues => safety issues – Working with QAs requires a deep understanding of such dependencies – Is basis for selecting architectural styles, patterns, and tactics

• Therefore, CMMI practices, likewise, encourage taking a holistic approach • See the SEI Webinar: http://www.sei.cmu.edu/library/abstracts/webinars/Capability-Maturity-Model-Integration-

and-Architecture-Centric-Engineering.cfm

http://www.sei.cmu.edu/library/abstracts/webinars/Capability-Maturity-Model-Integration-and-Architecture-Centric-Engineering.cfm

http://www.sei.cmu.edu/library/abstracts/webinars/Capability-Maturity-Model-Integration-and-Architecture-Centric-Engineering.cfm



But safety and security are also addressed explicitly in CMMI, e.g.: • Organizational processes need to comply with relevant standards

– Addressed in the Organizational Process Focus process area and References (Appendix A): • Assurance Focus for CMMI • NDIA Engineering for System Assurance Guidebook • ISO 27001 Information Technology – Security Techniques

A threaded discussion on information system vulnerabilities appears in the Measurement and Analysis process area. Security is a consideration in work environment standards and in training (Organizational Process Definition and Organizational Training process areas). Safety and security are mentioned as considerations in project/work planning in all three models (specific practices 1.1 and 2.3).



What does this imply? • While CMMI does address safety from the essential system perspective,

explicit guidance is not found in CMMI. • Knowledge of safety-specific methods, techniques, and tools is found

elsewhere.

But perhaps something even more fundamental is missing from CMMI and indeed many quality-related standards, models, and techniques that is just as essential to a safety culture. This presentation will bring a few of these to your attention. We begin with “collaboration” and then move to the other links in the Attention/Motivation Chain.


A Collaborative Culture [Morten Hansen, “What Leaders Need to Know About Collaboration”]

Summarizing Principles: 1. Collaborate with purpose: achieving better results! 2. Culture of collaboration must originate from the top (CEO) 3. Set up systems and processes to encourage collaboration 4. Allocate “decision rights” to people; but if there is no consensus

then the most senior person “closes the decision” 5. Create valuable collaboration by promoting constructive conflict 6. Organize diverse collaborative teams for a broader spectrum of

ideas; seek talented individuals in the periphery of the organization 7. At “close of decision” strive for unity in execution among all

participants; this is achieved by recognizing common goals 8. Maintain transparency of effects of decisions so that costs and

benefits can be openly assessed 9. Politics is poison – avoid secret aliances/attacks to prevent disunity

Also, see [HBR Insight Center “Making Collaboration Work”]


Outline



Definition of Team in CMMI

team A group of people with complementary skills and expertise who work together to accomplish specified objectives.

A team establishes and maintains a process that identifies roles, responsibilities, and interfaces; is sufficiently precise to enable the team to measure, manage, and improve their work performance; and enables the team to make and defend their commitments.

Collectively, team members provide skills and advocacy appropriate to all aspects of their work (e.g., for the different phases of a work product’s life) and are responsible for accomplishing the specified objectives.

Not every project or work group member must belong to a team (e.g., a person staffed to accomplish a task that is largely self-contained). Thus, a large project or work group can consist of many teams as well as project staff not belonging to any team. A smaller project or work group can consist of only a single team (or a single individual).


Team Performance [Herb Brooks, www.herbbrooksfoundation.com]

Herb Brooks: “All-star teams fail because they rely solely on the individual's talent. [But] The Soviets win because they take that talent and use it inside a system that's designed for the betterment of the team. My goal is to beat 'em at their own game.” Lou Nanne: “Beat the best team in the world? Gold medalists in '64, '68, '72, '76? Pretty lofty goal, Herb.” Herb Brooks: “Well, Lou, that's why I want to pursue it.”

Herb Brooks coached the 1980 Winter Olympics American ice hockey team (of amateurs) to beat the Soviets (and then Finland) to take the gold medal. Another quote: “This team isn't talented enough to win on talent alone.”

http://www.herbbrooksfoundation.com/


Importance of Team “Collective Intelligence” [Anita Williams Woolley, www.sciencemag.org/content/330/6004/686/suppl/DC2] There is an analog to IQ for groups, called “c” (for collective intelligence, the ability to perform successfully on a wide variety of tasks) that can be measured and used to predict future group performance.

• c accounts for about 40% of the variation in task performance across tasks. c correlates (both strongly and positively) with three factors: • average "social sensitivity" of group members (how well group members

perceive each others' emotions) • equality in distribution of conversational turn taking (e.g., everyone takes

turns to speak) • proportion of females in the group (perhaps due to first bullet)

Surprisingly, the following were inferior predictors (group tasks): • maximum or average individual group member intelligence • group cohesion, motivation, and satisfaction • some personality types such as maximum extraversion

http://www.sciencemag.org/content/330/6004/686/suppl/DC2


Integrated Teams [Dan Pink, “Drive”] When integrated teams are needed, having the right expertise on the team is critical. In particular, the integrated team should have expertise for all important dimensions of the: • Problem • Solution space


High-Reliability Teams (and Organizations) [Weick and Sutcliffe, “Managing the Unexpected: Resilient Performance…”]

In their reviews of case studies of high-reliability organizations, Weick and Sutcliffe identified five key characteristics of teams engaged in life-critical operations: • Preoccupation with failure • Reluctance to simplify interpretations • Sensitivity to operations • Commitment to resilience • Deference to expertise

While these characteristics may be very useful to teams more generally, Weick and Sutcliffe argue they are critical for high-reliability teams.

Also see [CERT Resilience Management Model].


TSP Principles [Yoshihiro Akiyama, TSP 2009 Symposium] Use a self-directed team to manage knowledge work. • Knowledge work must be managed by the team and individuals who

actually do the work. • The TSP launch process creates a self-directed team. • A detailed plan [for next 3-4 months] is developed before committing

to objectives to management and the customer. • Team management is accomplished through TSP weekly meetings

and management reporting.


TSP Launch Process [Yoshihiro Akiyama, TSP 2009 Symposium]

1. Establish product and

business goals

2. Assign roles and define team goals

4. Build overall and

near-term plans

5. Develop the quality

plan

6. Build

individual and

consolidated plans

7. Conduct risk

assessment

8. Prepare

management briefing and

launch report

Launch postmortem

9. Hold management

review

3. Produce development

strategy and process

A qualified coach guides the team through a defined process to develop its plan and to negotiate that plan with management.

Ref. SEI Course: “Leading a Development Team”


Outline



What Are Some Ways to Improve Individual Performance (Including Safety)? • Protect Attention—your most precious asset

• Kill Rudeness before it kills your performance

• Have the Right Mindset

• Improve Willpower for Follow Through

• Be on a socially-intelligent team

• Be wary of limits on expert intuition

• Develop discipline


Attention [Konrad 2011, Schenker] Summary Observations on attention (from years playing Kakuro [Konrad] and Bridge [Schenker]) • If you succeed at a difficult task, willpower improves, and maintaining focus is

easy. • But when you're failing, it takes much more willpower to maintain attention. • Distractions lead to a high rate of making mistakes

– When switching contexts, many errors happen unconsciously • Hunger and fatigue increase susceptibility and decrease focus control AND

LEARNING • What you can do when distracted:

– Remove the source – Remove yourself (work to clear your reaction to the distraction from your

mind so you can regain and sustain your task focus upon your return). • Process can aid you by:

– Making things more routine that otherwise would grab our attention – Helping establish sufficient context for process launches and handoffs.


A Kakuro Study – Effects of What You Drink

The Kakuro* study on the next few slides examined a few of the many factors that affect attention. • I was the subject—and the experimenter. • Analyses also provided by Chris Little, now with Deloitte Touche Tohmatsu

Ltd. but previously a graduate student at Carnegie Mellon University.

*Originally called “Cross Sums” and introduced in Dell puzzle magazines in about 1950.


Cross Sums (Kakuro) Sample: Trial 11, Puzzle 7

From: July 25, 2009 at 2:36 AM


Cross Sums Notes/Comments Transcription

Below is a raw transcription of the information found on the subject’s notes from Trial 11 (late evening shown only). Problem 7 is highlighted in pink (also, see puzzle on previous slide). Here the subject indicated the highest level of tiredness, a “3,” on account of how he felt and the number of “micronaps” that he recorded as experiencing; this is not surprising given that he started the problem at 2:36 am.


Z-score by Attempt vs. Drink

To allow comparing solution times across puzzles of varying difficulty (6 puzzles each completed 12 times), problem data was normalized by converting into z-scores. At left, z-scores have been separated according to whether water was consumed (0.0) or sugary drink (1.0). As can be seen, puzzle solution times after consuming sugary drink are: • closer to the mean (p < 0.05) • perhaps faster (p < 0.10) than when water alone is consumed.


Drink Vs. Performance and Learning • The following boxplots display z-scores by scenario (6 scenarios total)

– Scenarios were separated in time to reduce learning across scenarios. – Left scenario (3x): late evening water only; followed by afternoon of sugary drink. – Right scenario (3x): late evening of sugary drink followed by afternoon of water only.

scenarioTime of Day

d1/2=S/Wd1/2=W/S3pm9pm3pm9pm

3

2

1

0

-1

-2

Z-sc

ore

Boxplot of Z-score Perhaps, this is saying: sugary drink improves: 1) performance (faster solution times for the evening of scenario two) and 2) learning (afternoon of scenario 2 [water] shows as much improvement as afternoon of scenario 1)


Some of the Related (and More Formal) Research

“Sugaring the decision,” The Economist, March 29, 2008, p. 105-106. Knudsen. "Fundamental Components of Attention," Annual Review of Neuroscience 30 (1), p. 57-78. • What is known “today” about the mechanisms of attention.

Chabris, Chris and Dan Simons, “The Invisible Gorilla - How our Intuitions Deceive Us,” Random House 2009.

• Our subconscious in real time “fills in” what we perceive around us with default information and this also happens during recall; later on a subsequent recall, we can’t distinguish what was actually experienced vs. the default at the time of the experience vs. the default from earlier recall events. This limits not just our understanding of our past but what we learn for the future.

Gilbert, Daniel T. “How Mental Systems Believe,” American Psychologist, Feb. 1991, p 107-119. • A rushed pace, stress, and fatigue cause us to not question unreliable

sources of information and instead to treat them as actual valid information and integrate them into what we believe we experienced.


Rudeness is a Performance Killer [Amir Erez and Christine Porath, “Effects of Rudeness on Task Performance and Helpfulness”] Rudeness originating 3 ways (victim to an authority figure, witnessing a third-party offender, and imagining a situation where someone was rude) consistently resulted in the same effects: • diminished performance on both routine and creative tasks (including

cognitive tasks like solving anagrams; identifying ways to use a brick were fewer and more violent)

• priming to think more violently (performance on brick task) • decreased helpfulness to others

These effects stubbornly persisted for an extended period of time with victims continuing to feel angry and annoyed. Part of the reason for the drop in performance seems to be the victim has part of their working memory occupied with what happened leaving less available for other tasks. Rudeness easily spreads (is contagious), priming others to think and act violently.


Right Mindset [Carol Dweck, HBR 283: The Right Mindset for Success] Bottom Line: Successful people tend to be the ones that have a “Growth Mindset (GM).” “They believe their talents and abilities can continue to be developed; not worried about how smart they are, how they'll look, what a mistake will mean, they challenge themselves and grow from that.” To develop a GM:

• Expect to stretch beyond your Comfort Zone and take reasonable risks; don’t limit yourself to things you are already good at over and over again.

• Value process, reward process. • Take on big but reasonable challenges, pursue them doggedly, engage your

team and stakeholders. Leading others to develop a GM: don’t praise talent or ability, reward process (the effort, grit, strategies, doggedness, resilience, bouncing back, trying something new when previous thing didn’t work) and not just a successful outcome.


Follow Through – Willpower [Roy F. Baumeister, “Science Friday 123011 Hour 2 Resolutions”]

There is something essential for follow through. You have one stock of it. It generally decreases with use, but increases with eating and rest; what is it? Willpower

•Definition: capacity for volitional action, self-control


Follow Through – How to…

Summarizing Principles for managing willpower [Baumeister, Bregman]: • Start by committing to small, achievable goals

– Set specific targets about your commitment (what, when, where, how). – Develop your capability (and capacity) to tackle bigger commitments

• Maintain a limited # of commitments. – As you attend to one commitment, your willpower gets depleted

• Failure leaves one demoralized – Improve your resilience by envisioning failure in advance and what you

should do if failure happens. • Create a supporting process and environment (nudging you along).

– When the time comes to follow through, don’t let your minds “sabotage your aspirations.” (You can train your mind to ignore “the bait.”)

– Make yourself accountable to a friend. • As you learn what works, adjust to improve consistency of performance. • Nutrition matters (ah, sugar!)


Expert Intuition – Developing It; Relying on It -1 [Kahneman and Klein “Conditions for …;” Ruggieri “Confessions of a Surgeon…;” Gazzaniga, “Who’s In Charge …”]

Herb Simon: skilled intuition = recognition of patterns stored in memory Subjective experience is not a reliable indicator of judgment accuracy; on the contrary its compellingness often blinds us! Intuitions arise from “System 1,” which operates mostly unconsciously. • Such intuitions engage the same part of the mind as “the Interpreter,” which

continually integrates what information it has and delivers a coherent tale so that we see ourselves as “one person.”

• Effortlessly and immediately, we become aware of a judgment • But anchoring biases the sample that is brought to mind • And attribute substitution replaces the issue we were to address with an issue

for which there is a readily available answer • Also, faulty statistical intuitions survive both training and experience


Expert Intuition – Developing It; Relying on It -2 [Kahneman and Klein “Conditions for …;” Ruggieri “Confessions of a Surgeon…;” Gazzaniga, “Who’s In Charge …”]

Under what conditions can one develop expert intuition within a domain? • The domain is predictable with opportunities to learn its “regularities” • Prolonged practice accompanied by immediate, accurate feedback

“Expert intuition” should also engage our deliberate, reflective, System 2. We must question our intuitions and be wary of “heuristics and baises” (e.g., consider base rates of outcomes associated with those of the situation; and mitigate against anchoring and attribute substitution). How long does it take to be an expert surgeon?

• 5-7 years residency training • 10 years to learn how to operate, fine details, more confident • 10 more years on how to operate in very tough situations • 10 more years on when Not to operate, because can hurt patient


Individual Differences [Paulk] DeMarco and Lister • Count on the best people outperforming the worst by about 10:1 • Count on the best performer being about 2.5 times better than the median

performer

Process discipline improves quality by 79% and productivity by 12% while decreasing variability. The bottom quartile with disciplined processes outperforms the top quartile when undisciplined – but the top performers remain at the top when using disciplined processes.

– Paulk, “The Impact of Process Discipline on Personal Software Quality and Productivity,” ASQ Software Quality Professional, March 2010.

43


PSP* Quality Trends By Ability Quartile [Paulk, “The Impact of Process Discipline …”]

44

0

10

20

30

40

50

60

70

1A 2A 3A 4A 5A 6A 7A 8A 9A 10A

PSP Assignment

Defe

ct D

ensi

ty in

Tes

ting

BQB M2T M2TQ

PSP C

*PSP = Personal Software Process


Decision Making - References

People have systemic biases and utilize faulty heuristics when making decisions. • Kahneman, Thinking, Fast and Slow, 2011

[Be sure to read the second half of the book, which re-interprets the “power” of intuition and even the book title itself!]

• Hastie and Dawes, Rational Choice in an Uncertain World: The Psychology of Judgment and Decision Making, 2001

• Gladwell, Blink - The Power of Thinking Without Thinking, 2005 [Be sure to read the second half of the book, which re-interprets the “power” of intuition and even the book title itself!]

• LeGault, Think! Why Crucial Decisions Can’t Be Made in the Blink of an Eye, 2006

45


Outline



What Can We Do At the Ecosystem Level?

There might literally be thousands of stakeholders with unique perspectives and skills that might be relevant to the continued vitality of the larger ecosystem. But there may be: • No central command. • Few incentives. • “Social loafing” (a lack of cooperation going unobserved) • Little trust • Spam, noise, trolls, destructive behavior

In the slides that follow, we’ll look at some approaches that are being tried to effectively engage a large ecosystem.


Nudge the Ecosystem -1 [“Nudge,” by Richard H. Thaler and Cass R. Sunstein] The problem addressed by the book is how do you guide a large population to exercise good decisions when they are often confused, stressed, hurried, or indifferent? Thaler’s and Sunstein’s “golden rule:” Offer nudges that are most likely to help and are least likely to inflict harm…for decisions: • That are difficult and rare • That offer no prompt feedback • Where aspects of the situation cannot be easily translated into

terms that will be understand.


Nudge the Ecosystem -2

How implemented: apply these basic tactics (they spell “NUDGES(S)”): • iNcentives

– Analyze who uses, benefits, chooses, pays, profits – Not just monetary: e.g., peer recognition (award, chair, special committee) – (See Dan Pink’s key motivators)

• Understand mappings (explain alternatives; provide guidance) • Defaults (determine which option is generally the best; opt-in people) • Give feedback (specific, measurable, actionable feedback on how well the

person is doing) • Eliminate errors (poka yoke) • Structure complex choices (guide the human through making a sequence of

careful eliminations and evaluations) And another “S:” Social influences: make the desired behavior easily recognizable and socially transparent

• creates peer pressure (Perception) and a mindset of “following the crowd”


Platform-Based Ecosystems -1

Several ecosystems have arisen over the past few years that leverage mobile technology and many app developers to create an ecosystem that has grown exponentially, gaining hundreds of millions of users. For example, Google’s Android “open and free” mobile platform consists of: • Open Handset Alliance – responsible for sustaining the Android ecosystem • Android Open Source Project – for contributors of source code • Compatibility Definition Document – hardware and software requirements for

device makers (to ensure that third-party apps can run on their device) • Google Play – what device makers want to be able to place on their device—

access to all media and the apps • SDK – APIs give third-party developers full access to a phone’s capabilities


Platform-Based Ecosystems -2

Can we use (or mimic the general approach taken by) Google Android, Apple, Facebook, and others to engage everyone potentially affected by our products and services? • By engaging all roles in the ecosystem of our products and services (users,

operators, trainers, coaches, repairers, etc.), we are more likely to be made aware of: – Situations (and scenarios) where safety might be compromised that might

otherwise be overlooked – Additional products and services that might address safety needs and risks – Design ideas – Issues or risks associated with design ideas

Also, see [Henry Chesbrough “Open Innovation”].


“Structuring” Discussions in an Ecosystem -1

Challenges with orchestrating deep discussions in ecosystems include: • Understanding the big picture: argumentation revisits older topics, so

time obscures the relationships and similarities • “Balkanization of conversation”—people like to dialogue with those

just like them • Varying quality of the argumentation (a lot of bias, noise, trolls, group

think) Klein recommends “Argument Mapping,” in which a few rules enforced by a moderator allows deep discussions to emerge (be organized). • Being tested to determine how well such an approach can scale up. • See [Mark Klein’s “Deliberatorium,” MIT Center for Collective

Intelligence]


“Structuring” Discussions in an Ecosystem -2

Other approaches involve identifying topical structure (see [Blei “probabilistic topic models”]). • Probabilistic topic models have been applied on a very large

scale (a million documents or more) and can be applied in real-time and under a variety of assumptions about the topical structure of the discourse.

• Unclear whether such approaches will support the deep discussions characteristic of asserting and exploring claims of safety (and other quality attributes).


Ecosystem Organization Team Individual Results

Assuring the safety of our products and services requires an integrated approach (spanning levels) to establishing goals, theories, and technology (and assets and culture) so that all developers, users, installers, trainers, etc. have the motivation and attention they need in situations where safety issues and risks arise. All four levels (ecosystem, organization, team, individual) are still not well understood.

The Attention/Motivation Chain in Knowledge Work • Goals, Theories, and Technology, etc. “flow” to the Individual

• What each knowledge worker does is a function of their motivation and attention

Conclusion


Contact Information

U.S. Mail: Software Engineering Institute Carnegie Mellon University 4500 Fifth Avenue Pittsburgh, PA 15213-3890

World Wide Web: http://www.sei.cmu.edu SEI Fax: 412-268-5758

Mike Konrad Software Engineering Process Management Telephone: 412-268-5813 Email: [email protected] http://www.sei.cmu.edu/about/people/mdk.cfm

http://www.sei.cmu.edu/productlines

mailto:[email protected]

http://www.sei.cmu.edu/about/people/mdk.cfm

achieving a culture of safety, security, and sanity · ecosystem, organization, team, and...

Documents