esign heory for ystems that upport mergent …wesrac.usc.edu/wired/bldg-7_file/markus.pdf · is...

Markus et al./Design Theory to Support EKPs

MIS Quarterly Vol. 26 No. 3, pp. 179-212 /September 2002 179

SPECIAL ISSUE

A DESIGN THEORY FOR SYSTEMS THAT SUPPORT EMERGENT KNOWLEDGEPROCESSES1

By: M. Lynne MarkusManagement DepartmentBentley CollegeWaltham, MA [email protected]

Ann MajchrzakSchool of Business AdministrationUniversity of Southern CaliforniaLos Angeles, CA [email protected]

Les GasserGraduate School of Library and

Information ScienceUniversity of Illinois at

Urbana-Champaign501 East Daniel StreetChampaign, IL [email protected]

1Robert Zmud was the accepting senior editor for thispaper.

Abstract

This paper addresses the design problem ofproviding IT support for emerging knowledge pro-cesses (EKPs). EKPs are organizational activitypatterns that exhibit three characteristics in com-bination: an emergent process of deliberationswith no best structure or sequence; requirementsfor knowledge that are complex (both general andsituational), distributed across people, andevolving dynamically; and an actor set that isunpredictable in terms of job roles or prior knowl-edge. Examples of EKPs include basic research,new product development, strategic businessplanning, and organization design. EKPs differqualitatively from semi-structured decision makingprocesses; therefore, they have unique require-ments that are not all thoroughly supported byfamiliar classes of systems, such as executiveinformation systems, expert systems, electroniccommunication systems, organizational memorysystems, or repositories. Further, the develop-ment literature on familiar classes of systemsdoes not provide adequate guidance on how tobuild systems that support EKPs. Consequently,EKPs require a new IS design theory, asexplicated by Walls et al. (1992).

We created such a theory while designing anddeploying a system for the EKP of organizationdesign. The system was demonstrated through


180 MIS Quarterly Vol. 26 No. 3/September 2002

subsequent empirical analysis to be successful insupporting the process. Abstracting from theexperience of building this system, we developedan IS design theory for EKP support systems.This new IS design theory is an important theo-retical contribution, because it both providesguidance to developers and sets an agenda foracademic research. EKP design theory makesthe development process more tractable fordevelopers by restricting the range of effectivefeatures (or rules for selecting features) and therange of effective development practices to amore manageable set. EKP design theory alsosets an agenda for academic research by arti-culating theory-based principles that are subject toempirical, as well as practical, validation.

Keywords: IS design theory, IS development,emergent knowledge process, knowledgemanagement

ISRL Categories: AH05, AC04, FA, HA, HD

Introduction

A perennially interesting research topic in the ISfield is how to effectively develop new systems.The topic is interesting because, as IT developsand technical knowledge grows, IT is applied tonew application areas that were not previouslybelieved amenable to IT support. In the process,new kinds of systems and new developmentmethods are also created.

For example, when IT was first applied to clericalrecord keeping, the waterfall development methodemerged to handle the challenges of buildinggood transaction processing systems (TPS). Inthe late 1960s, when IT was increasingly appliedto managerial reporting and decision making, anew class of systems (decision support systemsor DSS) and a new development approach(iterative development) were devised (Keen andScott Morton 1978). Subsequently, as packagedindividual productivity tools became popular, anew development strategy evolved (Cusumanoand Selby 1995; Grudin 1991a, 1991b). Similarly,

the emergence of executive information systems(EIS) as a distinct class of applications warrantedthe creation of a new development approach(Watson et al. 1997).

Walls et al. (1992) used the name �IS designtheories� to refer to an integrated prescriptionconsisting of a particular class of user require-ments, a type of system solution (with distinctivefeatures), and a set of effective developmentpractices. Thus, there are design theories forfamiliar system types, like DSS, TPS, EIS, etc.The benefit of an IS design theory, according toWalls et al., is to articulate the boundaries withinwhich particular design assumptions apply. ISdesign theories make the design process moretractable for developers by focusing their attentionand restricting their options, thereby improvingdevelopment outcomes. In addition, IS designtheories inform researchers by suggesting test-able research hypotheses.

For example, DSS design theory is a contributionto the IS field because it signals systemdevelopers to do things differently than they wouldwith TPS. In contrast to the advice given to TPSdevelopers, DSS design theory tells developersnot to specify decision-making problems asprocedural processes and not to try to specify alluser requirements in advance of starting systemdevelopment. In so doing, DSS design theorymakes the DSS design problem more manage-able for developers, and it gives researchers abasis for making predictions about DSS usepatterns and impacts. Similarly, software packagedesign theory (Cusumano and Selby 1995; Grudin1991a, 1991b) helps developers cope with a situa-tion in which they do not have access to users fordetermining requirements, as they would if theywere developing an in-house TPS.

In the spirit of Walls et al., this paper proposes anew IS design theory for a class of user require-ments we call emergent knowledge processes(EKPs). Emergent knowledge processes areorganizational activity patterns that exhibit threecharacteristics in combination: �deliberations� withno best structure or sequence; highly unpre-dictable potential users and work contexts; and


MIS Quarterly Vol. 26 No. 3/September 2002 181

information requirements that include general,specific, and tacit knowledge distributed acrossexperts and non-experts. Examples include basicresearch, new product development, strategicbusiness planning, and organization design. Weargue that our design theory is a contribution tothe IS literature because EKPs represent animportant class of design situations that have notyet been adequately served by existing types ofsystems and their associated design theories.

The plan of the paper is as follows. In the theo-retical background section, we discuss IS designtheory and theorizing, describe a theoreticallybased conceptualization of emergent knowledgeprocesses, and explain why a new IS designtheory is needed. Next, we provide backgroundabout the development effort that served as thestimulus for our design theory, and we present thedesign theory as a set of principles that offerguidance to developers. We then explain whyEKP design theory represents a contribution to theIS literature, and we discuss its generalizability.Finally, we present an agenda for future researchand discuss implications for practitioners.

Theoretical Background

In this section, we first discuss IS design theoryand design theorizing. Next, we describe atheoretically based conceptualization of emergentknowledge processes. Finally, we explain why anew IS design theory for EKPs is needed.

Introduction to Design Theorizing

Although �IS design theory� is a term that couldrefer to general systems theory and the rela-tionship between developers, clients, and users(Churchman 1979) in the abstract, Walls et al.(1992) used the term in a very particular way torefer to solutions for specialized classes of ISdesign problems, usually given such labels asTPS, DSS, EIS, etc. According to Walls et al., anIS design theory is a package of three interrelatedelements: a set of user requirements, a set of

system features (or principles for selecting systemfeatures), and a set of principles deemed effectivefor guiding the process of development. By ad-dressing all three elements in conjunction, an ISdesign theory can be thought of as a completepackage of guidance for designers facing parti-cular sets of circumstances.

An IS design theory, as explicated by Walls et al.,has two distinctive characteristics: it is based intheory, and it provides guidance to practitioners.The theory underlying an IS design theory(referred to as �kernel theory�) may be an aca-demic theory (e.g., organizational psychology) ora practitioner theory-in-use (Sarker and Lee2002). Kernel theory enables formulation of empi-rically testable predictions relating the designtheory to outcomes like system-requirements fit.For example, it should be possible to establishempirically that the application of DSS designtheory to a particular set of requirements producesbetter results than applying TPS design theory tothe same requirements.

At the same time, IS design theories arenormative theories. That is, they are prescriptiveand evaluative, rather than solely descriptive,explanatory, or predictive. Because IS designtheories are intended to give guidance to devel-opers, they must not only pass scientific tests ofexplanatory or predictive power, they must alsopass the tests of practice: Does the system work?Does the system do what it was supposed to do?Is the system elegant and aesthetically appealing?(Florman 1994).

It should be emphasized that the IS design theoryapproach of Walls et al. is not a radical departurefrom established IS practice and theorizing. Itsprimary contribution is to formalize, justify, andextend the traditional IS practice of labelingsystem types (e.g., TPS, DSS, GSS, ESS, EIS),describing their characteristic features, and pre-scribing an effective development approach. Thevalue of an IS design theory is to reduce devel-opers� uncertainty by restricting the range ofallowable system features and developmentactivities to a more manageable set, therebyincreasing the reliability of development and thelikelihood of success, and to stimulate research.



This paper outlines a design theory for systemsthat support EKP. As mentioned above, a ISdesign theory in the Walls et al. sense consists ofthree interrelated elements: (1) a set of userrequirements derived from kernel theory, (2) prin-ciples governing the development process, and(3) principles governing the design of a system(i.e., specifying and implementing its features). Inthe next section, we present our kernel theory.

Emergent Knowledge Processes

Work that is to be supported by informationtechnology is generally described in terms of thecharacteristics of the process by which work isperformed (Keen and Scott Morton 1978), thecharacteristics of users and their work context(e.g., Markus and Keil 1994), and users� infor-mation requirements (e.g., related to their criticalsuccess factors, cf., Rockart 1984). An IS designtheory must address all three characteristics.

The first characteristic, process, has traditionallybeen described in terms of the concept of struc-ture. For example, Keen and Scott Mortondescribed three degrees of structure: highlystructured, semi-structured, and unstructured.They labeled as �semi-structured� processes likebrand management, cash management, andmanagement exception monitoring and distin-guished them from �unstructured� processes, suchas basic research and the concept definitionphase of new product development.

Pava (1983) subsequently pointed out that it is notaccurate to say that some processes merely lackstructure. After all, as Keen and Scott Mortonexplained, the degree of structure in processeslike cash management can increase over time asmore becomes known about them. However, forthe managerial decision-making processes ofinterest to Pava, increased structure is neitherpossible nor desirable, because it might introducerigid, stereotyped responses where creativity andflexibility are needed. Such unstructurable pro-cesses have been referred to in terms of humansense-making: building knowledge in a recursive,participatory, and evolutionary manner (Boland

and Tenkasi 1995). Because the term unstruc-tured suggests that structuring is possible anddesirable, whereas the term emergent does not,we believe that emergent is a better label for manyknowledge processes.

An example of an emergent process is newproduct development, which has been describedas a series of trial-and-error experiences in whichthe developer iterates recursively betweenproblem-finding and solution evaluation (Bhat-tacharya et al. 1998; Iansiti 1992). Similarly,strategy-making has been described as assem-blages of deliberations, with unpredictable triggersand fluid courses, evolving organically as thesituation changes (Pava 1983). Finally, the pro-cess of organization design has been charac-terized as one of identifying solutions throughanalyses of the contingencies, limits, andtradeoffs of alternative organizational patterns andthen adapting these patterns when circumstanceschange (Litterer and Jelinek 1983).

In short, we find the term unstructured insufficientto characterize the processes of new productdevelopment, strategy-making, and organizationdesign. Instead, we refer to them as emergentprocesses, in which problem interpretations,deliberations, and actions unfold unpredictably.

The second factor addressed by system designersis the user. Most in-house IS development pro-cesses (such as for DSS and ES) assume that theuser type is known in advance (Grudin 1991a,1991b; Poltrock and Grudin 1994), permittingsystematic requirements analysis. The unpredict-ability of emergent processes means that it isnearly impossible for a system developer to knowin advance the kinds of people who will be calledinto a deliberation, when they will be called in, orwhy. In addition, because emergent processesoften involve high-level professional and technicalpersonnel, the actors have a high degree ofautonomy in how they do their work. They canresist the imposition of standard routines and newtechnologies (Davenport et al. 1998; Frenkel et al.1999). Therefore, designers of systems to sup-port emergent processes do not have the luxury ofsystematic requirements analysis; they must plan



for very infrequent use of support tools (maybeonce only) and they cannot even assume that theintended users will want, or can be required, touse their support systems.

An example of �unknown users� arises in strategicplanning. For example, Mintzberg (1994)describes strategic planning as �big strategiesgrowing from little ideas� in strange places atunexpected times. Thus, almost anyone in anorganization (e.g., line managers, strategicplanners, IS specialists) could initiate the processof strategic planning, and almost anyone is acandidate user of a strategic planning supporttool. Further, the strategy formulator could workalone or in collaboration with others in theorganization. Similarly, in the concept definitionphase of new product development, it is oftenimpossible to specify in advance what kinds ofexperts will be needed to solve a problem. At thesame time, it would be counterproductive to limitinvolvement to the initial members of a newproduct development team (Clark and Fujimoto1991). To address these issues, new productdevelopment teams routinely pull in experts ofmany types on an as-needed basis. Finally, in thecase of organization design, many different eventscan trigger the process (Litterer and Jelinek 1983).Some of these triggers are quite rare. As a result,many different kinds of actors can be involved�manufacturing engineers, product developmentengineers, managers, HR specialists, shop floorworkers, external consultants�but no one groupwould always be involved, and most people wouldparticipate in organization design very infre-quently. Further, the different types of potentialprocess participants have considerable autonomyin deciding how to approach the task (such aswhether to use experts or even a support tool).

In short, emergent processes are characterized byhighly unpredictable user types and work con-texts. The unpredictability extends to when andwhy the process is performed and whether sup-port tools will be used.

A third factor considered by system developers isusers� information requirements. The informationrequirements of knowledge-intensive emergent

processes are quite different from those of semi-structured business processes. First, in manysemi-structured business processes, such asbrand management, users require systems thatanalyze numeric data presented in tables andgraphs. By contrast, in emergent processes,users must often search for the information theyneed from documents that are poorly indexed andstored. (See Blair [1984] on the different storageand retrieval capabilities of documents versusdata.) Second, as many authorities have noted,much of the knowledge involved in sense-makingprocesses is tacit, not explicit (Weick 1995). As aresult, it is difficult to capture and share. Third,knowledge-intensive emergent processes have ahigh level of expert knowledge content. Thismeans that, when tacit knowledge can be madeexplicit, it cannot easily be represented numeri-cally, but must instead be represented as if-thenrules (Baligh et al. 1996), as cases (El Sawy andBowles 1997), or as text. And, because non-expert users may not understand expert jargon,the knowledge-base must be translated into termsnon-experts can understand (Markus 2001).Finally, in most knowledge-intensive emergentprocesses, knowledge is distributed across manydifferent people (Hutchins 1991). Some of thedistributed knowledge is local (e.g., conditions ina certain geographic locale), and some is general(e.g., scientific knowledge). Unless distributedexpertise can effectively be brought togetherduring what Hutchins calls �local design activities,�knowledge will be incomplete, and action faulty(Cannon-Bowers et al. 1993).

The challenging knowledge requirements of emer-gent knowledge processes can be seen in severalexamples. For instance, Keen and Scott Mortondistinguish between the information required formanagement control and the intelligence requiredfor strategic planning. Strategic planning requiresexternal information, primarily qualitative, with awide scope and a future time horizon. Aggregateand approximate information is of greater valuethan detailed and highly accurate data (Mintzberg1994). In addition, a great deal of intuition and�feel� is involved in managerial decision-making(Mintzberg 1994). Similarly, expert knowledge isa significant factor in new product development



groups (Bhattacharya et al. 1998), and new pro-duct development knowledge evolves dynamically(Couger 1996). New product development exper-tise comprises both codified general knowledge(e.g., scientific rules from physics, mechanics, andchemistry) and tacit local knowledge (e.g., con-cerning such factors as how materials �breathe� inlocal weather conditions) (Cross 1997). Similarly,the organization design process draws upongeneral scientific knowledge from disparate disci-plines (including organizational behavior, organi-zation design, systems theory, sociotechnicalsystems design, political behavior, engineeringdesign, and incentive structures) as well as localknowledge, distributed across various actors,concerning union rules, management politics, andthe capabilities of individual workers (Litterer andJelinek 1983).

In sum, then, knowledge-intensive emergent pro-cesses have challenging information require-ments. They require knowledge and expertise inapplying the knowledge. They require tacit andexplicit knowledge, general and contextual knowl-edge. Because knowledge is distributed, theyrequire knowledge sharing.

Summarizing across the characteristics of pro-cess, user, and information needs, we define anemergent knowledge process as an organizationalactivity pattern characterized by (1) an emergentprocess of deliberations with no best structure orsequence, (2) an actor set that is unpredictable interms of job roles or prior knowledge, and(3) knowledge requirements for general andspecific distributed expertise. Semi-structuredprocesses may have some of these attributes to alesser extent, but EKPs are differentiated byhaving all three characteristics to a significantextent. As we show below, these characteristicsmake it difficult for EKP support system designersto use IS design theories that were developed forsemi-structured decision-making problems.

Why Is a New Design TheoryNeeded for EKPs?

Many researchers have commented on the poor fitbetween the requirements of processes with the

characteristics of EKPs (e.g., creative problemfinding, new product development) and existing ITapplication types, such as executive informationsystems (EIS) and expert systems (ES). Forexample:

� Davenport et al. (1996) argued that, �Theabstract and unstructured inputs to and outputsfrom knowledge work processes...make appli-cations of [information] technology more diffi-cult� (p. 55).

� Stein and Vandenbosch (1996) concluded that,�Advanced IS, in particular ES and EIS, provideample opportunities for higher-order organi-zaitonal learning�that is rarely exploited.�

� Vandenbosch and Huff (1997) interviewed 36executive users of EIS and found that only nineof them used their systems for the creative workof scanning (i.e., browsing data to understandtrends and relationships and to challengefundamental assumptions). One problem withEIS was the absence of a predefined modelbased on expert knowledge of how the datacould and should be used.

� Shneiderman (1998) lamented that softwaretools have had little success in supportingcreative problem solving. Although EIS haveuseful features like information visualization anddynamic queries, they are insufficient as toolsfor creativity because they rarely allow trying outall permutations of a problem, combining ideas,and rapid prototyping.

� Kivijarvi and Zmud (1993) and Alavi (2000)suggested that ES and DSS requiring codifiedknowledge will not be successful in domainscharacterized by subjectivity and complexity.ES require the problem space, the designobjectives, and guidelines for addressing theproblem space to be defined in advance(Kivijarvi and Zmud 1993), which is difficult todo with EKPs.

� Todd and Benbasat (2000) recently concludedthat DSS have successfully supported only theproblem solving tasks associated with decision



making. However, problem finding�includingthe tasks of uncovering the underlying decisionproblem, gathering relevant information about it,and diagnosing it�is �fuzzy, difficult, and notamenable to technical support� (p. 4).

This poor fit seems to stem from threedisconnects between EKP requirements andfamiliar system types (e.g., DSS, EIS, ES).

The first disconnect concerns the EKP require-ment that general expert knowledge (such asabout human resource management) must becontextualized when making decisions in parti-cular local conditions (such as extreme conflict orpoor morale) (Vandenbosch and Huff 1997).Systems that support semi-structured decisionmaking (DSS and EIS) lack expert knowledgerepositories and contextualizing translation rules,thereby inhibiting creative problem finding andsolution generation (Todd and Benbasat 2000).Expert systems do include general expert knowl-edge, but they may not support contextualization,and they often sacrifice the flexibility needed forprocess emergence. The difficulty of reconcilingthe needs for general knowledge, contextualknowledge, and flexible support leads manyauthorities to claim that the best way to supportEKPs is through personal and electronic com-munication systems or though document databasesystems with user-supplied content, such as LotusNotes (Davenport et al. 1996). But without thediscipline provided by validated expert knowledge,organizations as well as communities of practicecan degenerate into self-deluding �groupthink�(Brown and Duguid 1998).

A second disconnect is that expert systems, DSS,and EIS are all designed for a known usercommunity (Watson et al. 1997), that is, a knowntype of user. They do not adapt gracefully to theEKP characteristic of shifting user types withwidely differing knowledge requirements�a factorthat has also been noted in organizationalmemory systems (Markus 2001). For example, asystem that successfully supported in-house tech-nical support consultants was not successfullyredeployed among external support providers, andthe knowledge-base had to be significantly refined

and simplified before it could be successfully usedby external customers (El Sawy and Bowles1997).

Third, knowledge workers are supported today,not with a dearth of tools, but with too many toolsand with tools that are not integrated. According toa recent Gartner report (Hayward 2000), knowl-edge workers have access to expert systems,decision support systems, executive informationsystems, organizational communication systems,organizational knowledge repositories, and collab-orative tools. This �tool glut� has two negativeresults. First, because the tools are not integratedinto the work process, knowledge workers exhibitconstrained and stereotypical work behaviors(Hayward 2000): A fair portion of a knowledgeworker�s day involves �junk computing� (Guthrieand Gray 1996)�managing tools, instead ofgetting work done. Second, because the tools arenot integrated with each other, knowledge workersmay not use an essential tool, such as a reposi-tory of expert knowledge. For instance, Markusand Keil (1994) found that, when provided withtwo unintegrated systems�one for product pricequotation and another for ensuring high qualityproduct configurations�a computer company�ssalespeople did not use the configuration tool.Consequently, product configuration quality didnot improve.

As a result of these three disconnects, existing ISdesign theories for DSS, EIS, ES, and groupwaredo not meet all three EKP requirements (process,user, knowledge) simultaneously. Further, whilegeneralized system development methodologiesdo exist, they do not satisfy the criteria for an EKPdesign theory, because they are not tailored toEKP requirements. Developers might be able toapply sociotechnical systems theory (Mumford1995; Taylor and Felten 1993), participativedesign (Emery 1993; Greenbaum and Kyng 1991),or contextual design (Beyer and Holtzenblatt1998) to produce an effective system for a specificEKP, but the solution would only be applicable toa particular place, context, and time. Similarly,soft systems methodology was intentionallydevised not to generate generalizable solutions(Checkland 1984). By contrast, an IS design



theory for EKPs should, according to Walls et al.(1992), define system design and developmentprinciples that generalize to the entire class ofEKPs.

In short, familiar types of systems such as DSS,EIS, ES, and groupware do not individuallysupport all the requirements of EKPs. Applyingthem to an EKP in combination without integratingthem (to the work process and with other tools) isnot an adequate solution for reasons of usabilityand likely effectiveness. Applying a general ISdesign methodology would solve a particular EKPdesign problem, but it would not result in a generalsolution applicable to the class of EKPs. Conse-quently, a new IS design theory is neededspecifically for EKPs.

In such a new EKP design theory, two types ofdesign principles are inextricably intertwined:principles governing the development or selectionof system features and principles guiding thedevelopment process (Walls et al. 1992). Forexample, developing a system that supportsexecutives in responding to new strategic oppor-tunities requires the design principle of vigilance(Walls et al. 1992). Vigilance is enabled in avigilant information system (VIS) by such featuresas anticipatory alerts, rather than post hoc excep-tion reports. Designing a system that embodiesvigilance on the part of users requires certaindevelopment principles, that is, practices bydevelopers. Rather than, for example, focusingon users� stated information requirements, thedeveloper of a VIS would need to observe execu-tives� response thresholds and triggers. An ISdesign theory for EKPs represents a similarinterweaving of design and development prin-ciples related to EKP requirements�an inter-weaving that is apparent in our presentation ofEKP design theory in the next section.

The Top Modeler Case andan EKP Design Theory

In this section, we present our design theory forEKPs in the context of the case that prompted us

to develop it. We describe the background of theTOP Modeler system and present the designprinciples that we evolved during TOP Modeler�sdevelopment.

TOP Modeler Background

TOP stands for �Technology, Organization, andPeople� integration. The system called TOPModeler was developed to support the process oforganization design in manufacturing organi-zations. TOP Modeler was funded with a $3million grant from the National Center for Manu-facturing Sciences and included the activeinvolvement of four companies: Hewlett-Packard,General Motors, Digital Equipment Corporation,and Texas Instruments. (Each company dedi-cated one person to the project full time for threeyears.) The second and third authors of thispaper managed the system development effort.The first author was only peripherally involved indevelopment (she conducted an assessment ofrequirements early in the project), providingpsychological and emotional distance from theproject for reflection and identification of lessonslearned.

Organization design is a critical process formanufacturing organizations, since it is known tobe associated with good or poor performance onsuch measures as productivity, cost, quality, andcycle time. Few organizations have been able toinfuse organization design expertise throughouttheir manufacturing operations because of thecharacteristics of EKPs (process emergence,unpredictable user types and use contexts, anddistributed expert knowledge). Few software toolsto support the process exist: evaluations indicatedthat extant tools lacked either a solid base inscientific knowledge about organization design orsupport for a sociotechnical systems perspectiveon organization design. For example, the tooldeveloped by Burton presents rules for organi-zational structure but little specific guidance forredesigning information systems or productionand process technologies (Baligh et al. 1996).Consequently, the funding organizations weremotivated to underwrite the development of TOPModeler.



Structurally, the TOP Modeler system has threemain components: a knowledge-base of thescientific knowledge about organization design, aninference engine, and an interface for data inputand analysis display. The knowledge-base wasderived from sociotechnical systems theory(Cherns 1976; 1987; Trist and Murray 1993).Academic experts were commissioned to sum-marize the empirical literature on specificelements of an organization (e.g., work design,information technology, skills) into sets of rulesspecifying the appropriate design elements fororganizations under a wide variety of conditions.The knowledge content of the system wasreviewed and vetted for practical usability througha series of consensus-building meetings held overtwo years with representatives from the fourorganizations.

After the knowledge-base was developed, it wasevaluated against the eight verification require-ments for expert systems: competency, complete-ness, consistency, correctness, testability, rele-vance, usability, and reliability (Vermesan 1997).The rules were quantitatively validated using datagathered during intensive three-day site visits to93 electronics manufacturing companies in theU.S. This massive validation effort (Majchrzak1997) demonstrated that the knowledge-base wasable to predict statistically significant differencesbetween manufacturing firms achieving higherversus lower levels of throughput time, a criticalmeasure of effectiveness for electronics manu-facturing firms.

In addition to the knowledge-base validationstudy, there are several other indicators of systemsuccess. First, the system was deployed on timeand within budget. Second, the TOP Modelersystem was eventually commercialized by TOPIntegration, Inc. Third, the system has been usedin over two dozen �real use� situations of organi-zation redesign, involving over three dozen naïveusers in more than 10 organizations (Majchrzakand Finley 1995; Majchrzak and Gasser 2000).

In a structured evaluation of 19 users (Borys andMajchrzak 1999), the system was deemed helpfulin fostering new learning about organization and

sociotechnical systems design and in changingthe behavior of people involved in organizationredesign decisions. In addition, users reportedreassessing their business strategies, clarifyingbusiness issues, learning more about theirorganizations, and achieving consensus on impor-tant organization design issues. One use of TOPModeler occurred in a high technology subsidiarythat was considering moving its operations fromSingapore to Thailand. The system-assistedevaluation revealed a number of weaknesses inthe Thai plant, and the planned move wasstopped. Another case involved a proposed jointventure between a U.S. manufacturer and aChinese counterpart. Use of the system helpedsurface previously unrecognized differences in thestrategic directions of these two organizations.The joint venture was postponed until the dif-ferences could be resolved.

In short, the evidence suggests that TOP Modelerwas successful in supporting organization design.Therefore, the story of TOP Modeler is a worthyexemplar for purposes of design theorizing aboutemergent knowledge processes.

The Stimulus for an EKP DesignTheory for TOP Modeler

According to Walls et al. (1992), one researchstrategy appropriate for design theory building isaction research, coupled with iterative hypothesisdevelopment. The action research process startswith requirements derived from kernel theoriesand hypothesized design and development prin-ciples that meet these requirements. The hypo-thesized principles are then used as the basis forspecifying system features. Once developed anddeployed, the use and impacts of the system canbe observed. If the results are not as expected,new hypothesized principles are generated, asystem version instantiating the new principles isdeveloped, deployed, and evaluated, and thecycle is repeated. A desired outcome of actionresearch is better theory.

The TOP Modeler project followed this actionresearch strategy. We started with a kernel



theory. Then, over an 18-month period, thedevelopment team repeatedly intervened into theorganizational design activities of the involvedcompanies, deploying prototypes that testedvarious assumptions about how organizationaldesign work is done, observing how usersresponded, and iterating. Finally, we articulatedour learning in the form of a new IS design theory.

At the outset of the TOP Modeler project, we didnot use EKP kernel theory, since we had not yetdeveloped the EKP concept. Instead, we initiallyconceptualized organizational design as a semi-structured decision-making process, followingKeen and Scott Morton�s (1978) prescriptions.1

We therefore believed we could effectively userelevant design theories�the theoretical andpractical literature on how best to build decisionsupport systems, expert systems, and executiveinformation systems (e.g., Watson et al. 1997).That literature led us to expect that we couldidentify a target group of users, achieve con-sensus about their requirements, incorporateexperts� knowledge about how the task was bestperformed and what information was relevant, anddesign a system that matched the work process.(Table 1 summarizes our initial conceptualizationof organization design and the corresponding ISdesign theory.)

Over time, however, we learned that the IS designtheory of semi-structured decision-making pro-cesses was inapplicable to the organizationdesign process. The last column of Table 1summarizes the problems we encountered whileattempting to apply such a design theory. (Theseproblems are discussed more fully in the nextsection.) As a result, we were forced toreconceptualize (1) the requirements of theorganization design process (as those of anemergent, rather than a semi-structured, knowl-

edge process), (2) the features of a system thatwould adequately support the work of organizationdesign (as a support system that combinesgeneral and contextualized knowledge and knowl-edge sharing capabilities, rather than a DSS,ESS, or EIS), and (3) the process of developingsuch a system (as emergent, rather than iterative).The three related elements of our recon-ceptualization, summarized in Table 2, underlie anew IS design theory for EKPs.

EKP Design Theory Principles

This section describes our design theory as a setof six combined design and development prin-ciples for EKPs. We present each principle as weevolved it from a design theory for semi-structureddecision-making processes to a design theory forEKPs. By describing the evolution of our theory inthe context of the TOP Modeler story, our intent isto capture the richness of our design theory in away that simple verbal guidelines to designerscannot. In addition, exhibit boxes provide detailsabout how the TOP Modeler system and devel-opment process exemplify the design principles.Despite this contextual presentation of EKPdesign theory, our claim is that the theory repre-sents a general strategy for the class of problemswe call EKPs.

Principle #1: Design for CustomerEngagement by Seeking Out Naïve UsersEarly on, we attempted to create a detailed por-trait of TOP Modeler�s potential users and theirinformation requirements: Which occupationalgroups would use the system? What did theyknow, need to know, and not know? How werethey likely to use the system? What were theimplications for the tool�s functionality, interface,and support requirements?

Interviews with representatives from the foursponsoring companies revealed that the potentialuser set included manufacturing engineers, HRspecialists, shop floor workers, and generalmanagers, as well as expert organization develop-ment consultants and academics. The interviews

1Decision making requires systematic data searchingand subjective analysis. Decision making can beimproved by providing IT functionality for data retrieval,reporting, and display. Strategic planning is an exampleof semi-structured decision making, with the charac-teristics of low accuracy, future vs. present time horizon,infrequent performance, and need for qualitative andwidely scoped information.



Table 1. Initial IS Design Theory and Problems Encountered While Attempting toApply it

Initial RequirementsOrganization design is asemi-structured expertdecision-making process

Initial IS Design TheoryThe solution is an expert decision-supporting system, developed viaan appropriate iterativedevelopment methodology

Problems Encountered WhileAttempting to Apply Initial ISDesign Theory

Users and Their Work Context

� Organization designershold identifiable jobroles known in advance

� System must support theconsensus needs of a knownuser community, determinedthrough use of a method likeRAD

� No identifiable user group;anyone or everyone couldperform organization design atany time

Users� Information Requirements

� Expert organizationdesign knowledge willbe useful to layorganization designers

� System must represent experts�knowledge of organizationdesign as if-then rules withprescriptions for action

� Lay organization designers donot use rules and do not followprescriptions for action

� Expert organizationdesigners see theorganization as aunified whole

� Knowledge-base should providea single, cross-functional view ofthe organization

� Lay organization designersreinterpret knowledge throughtheir own functional lenses anddon�t seek to involve othergroups or get their input

The Process

� Expert organizationdesigners follow aprescribed, semi-structured process

� System must incorporate theprescribed process and restrictad hoc organization designers tothe accepted process

� No single process is acceptedamong experts; lay organizationdesigners can, and do,circumvent prescribedprocesses

further revealed that we were unlikely to find acommon fund of knowledge about organizationdesign among the potential user groups. Further,the range of desired or expected uses of TOPModeler was quite broad, including:

� Generate design alternatives quickly� Evaluate designs� Conduct sensitivity analysis� Identify impacts of strategic objectives� Identify organizational and people changes

needed for new product and process tech-nologies

� Analyze competitors� capabilities� Certify suppliers� Facilitate user involvement in design� Facilitate learning by a sociotechnical design

team about integration of technology, people,and organization factors

Moreover, most of these uses would occur onlyrarely and unpredictably.

We also learned that there might be resistance tousing the system by both potential hands-on usersand managers. For example, HR specialists view



Table 2. Revised IS Design Theory for EKPs

Revised RequirementsOrganization design is an emergentknowledge process

Revised IS Design TheoryThe solution is an EKP support system, developedvia an emergent development methodology

Users and Their Work Context

� Specific types of users cannot be iden-tified; it cannot be assumed that users willbe knowledgeable, trained, or motivated,nor can it be assumed that training anduse will be mandated

1. System must be self-deploying; developersshould conceptualize each user-system inter-action as a customer engagement process andrepeatedly seek out �naïve� users through aprocess of �onion-layering� the design team

Users� Information Requirements

� Lay organization designers need expertknowledge translated into a form they canuse, involving multiple types of tradeoffanalyses with clear implications for action

� Lay organization designers cannotimplement system-recommended actionsonline; they must convince others toimplement organization design changesoffline

2. System must translate expert knowledge intoactionable knowledge for non-experts; devel-opers should expect to need many functionalprototypes, instead of a few nonfunctionalprototypes

3. System must induce users to take offline action;developers must observe and strive to changeusers� offline, as well as online, action

� Lay organization designers must beinduced to consider knowledge aboutother functional areas and to develop aholistic conception of the organizationdesign process

4. System must integrate expert knowledge withlocal knowledge sharing; multiple neededfunctionalities must be integrated rather thanadded

The Process

� The organization design process isemergent, with many process triggers,many process flows and tradeoffanalyses, and many motivations amongorganization designers

� Many changes in the process, expert andspecific knowledge, and user-systeminteraction must be expected

5. System must implicitly, not explicitly, guide users�deliberations in desirable directions, withoutrestricting them to a prescribed process;developers should use a dialectical developmentprocess instead of a consensus-seekingapproach

6. System must be extremely flexible; developersshould componentize everything, including theknowledge-base

organization design as their unique expertise, andmanagers expressed concern about obtainingrecommendations about organizational designfrom shop floor workers and supervisors. Finally,the interviews indicated that, despite the com-plexities of organization design knowledge, users

were unlikely to accept any training, either in theorganization design process or in system use.The system had to stand totally on its own!

These interviews led us to think of our system asneeding to be self-deploying among a population



of reluctant, even hostile, naïve users of everyconceivable stripe. An encounter between thesystem and a naïve user is like a cold sales callon a negatively predisposed prospect: the systemhad to convert the prospect through a seductiveprocess of customer engagement�without theadditional support of in-person training, coaching,or consulting.

This principle of self-deployment and customerengagement goes far beyond mere �user-friendliness��a pervasive design guideline in theDSS literature (e.g., Watson et al. 1997). User-friendliness, which we incorporated into thesystem from the beginning, did not addresspeople�s lack of incentives to use the system (cf.,Markus and Keil 1994), nor did it counteract anynegative perceptions of the activity the systemwas designed to support. Therefore, we concep-tualized a three-stage customer engagementprocess to make our system self-deploying:induce naïve users to try the system, provideimmediate benefits, and encourage people to staywith the system long enough to complete asociotechnical analysis.

The first stage required a system that inducednaïve users to try it. Early on, we elicited knowl-edge from academic scholars and expert prac-titioners of organization design and attempted torepresent their expert knowledge as the expertsrepresented it. For instance, we initially designedTOP Modeler to improve organizational effective-ness, since that objective is the focus of muchrelevant academic literature. However, throughaction research with potential non-expert organi-zational designers (e.g., engineers, shop floorworkers), we found that the objective of improvingoverall organizational effectiveness did not compelthem to use a support system. Non-experts wouldonly try a support system if it were focused on thegoal of improving specific performance metrics(e.g., throughput times, quality, or new productdevelopment ramp-ups) of immediate practicalinterest to them. We further learned that textualrepresentations of knowledge (e.g., documentsdescribing lessons learned and best practices) didnot induce use: these documents were timeconsuming to read and digest. In short, if we

wanted to induce naïve users to try out thesystem, we needed to model the system on acomputer game, with color-coded evaluations ofthe human side of manufacturing technology,while providing benchmarks to shows about howusers� organizations �measured up� to others.How TOP Modeler was designed for this firststage of customer engagement is shown inExhibit 1.

The second stage of the customer engagementprocess, we found, required users to acquireimmediate benefits from using the system. Theyneeded to have their intuitions confirmed quicklyor learn something they did not already know. Ifnot, they quickly abandoned the new tool.Exhibit 2 presents how TOP Modeler did this.

The third stage of the customer engagementprocess involved encouraging users to stay withthe system long enough to complete a thoroughsociotechnical analysis. In TOP Modeler, userswere encouraged to stay by initializing all systemvalues to �no�; that is, the default organization wasshown to contain none of the required organiza-tion features. Only by working through a thoroughanalysis could many gaps be removed, allowingthe user to focus on those that must be fixed.

The three-stage customer engagement modelrequired a major shift in the development process.Initially, the development team followed a tradi-tional user-centered methodology. The projecthad an Executive Steering Committee (high-levelmanagers from NCMS and each of the fourcompanies and the second and third authors asproject leaders), a Development Team (fivecomputer scientists), and a Domain Team (full-time user representatives from each company).The Domain Team�s original responsibility was torepresent potential users and review prototypes inan iterative development methodology, as sug-gested by DSS design textbooks (e.g., Watson etal. 1997), using user-centered techniques such asjoint design meetings and cooperative prototyping(Greenbaum and Kyng 1991).

We learned that this development process wasnot compatible with the design principle of self-



Figure 1. The Ferris Wheel

Exhibit 1: How TOP Modeler Induces Naïve Users To Try It

Potential gaps in users� knowledge were signaled by using color-coding to portray an undesirable initialstate. For example, TOP Modeler�s primary interface is the Ferris Wheel, shown in Figure 1. The FerrisWheel presents 12 sets of organization design features laid out around the outside of a circle, wherethe inside is composed of business strategies (21 process variance control strategies and sevenmanufacturing organization objectives). The 12 feature sets include norms, skills, customerinvolvement, discretion, organizational values, employee values, production process characteristics,reporting structure characteristics, technical system characteristics, performance measurement andrewards, information resources, and production tasks. (Each feature set in turn contains from five to30 separate specific features. For example, a �Skill� feature was �Reading and Writing Capabilities�.)

The layout of the Ferris Wheel was intended to convey to naïve users the entire array of appropriateconsiderations in successful organization design, which we hoped would stimulate their desire to learnabout concepts unfamiliar to them. Next to each feature set on the Ferris Wheel were color-coded�thermometers,� indicating the percentage of features in the set matched to benchmarked best practicesgiven the business strategies designated by the user. Thus, TOP Modeler was designed to capturenaïve users� attention quickly.



Figure 2. Example Tradeoff Matrix

Exhibit 2. How TOP Modeler Provides Immediate Benefits

Tradeoff matrices provided immediate feedback. The tradeoff matrices were organized around 12 setsof organizational features, with the specific features of each set as column headings and the businessstrategies as row headings. An example tradeoff matrix is shown in Figure 2. As users inputted a valuefor an organizational feature, the system evaluated that input for alignment with the organization�sbusiness strategy, shown in green. If not aligned, the cell immediately turned red. This feedbackencouraged users to continue using the system.

deploying customer engagement. The DomainTeam soon became so knowledgeable about thesystem (because we were involving them in dailyconversations and weekly prototyping) that theylost their representativeness as �naïve users�;they had effectively �crossed cultures� andbecome developers. Therefore, the developmentprocess continually needed new potential users tointeract with the evolving system. Otherwise, wecould not accurately assess the system�s potentialto attract users who had not been previouslyexposed to it. Exhibit 3 describes how TOPModeler�s development process continuallysought out naïve users.

Principle #2: Design for KnowledgeTranslation Through Radical Iterationwith Functional PrototypesThe literature on expert system development (e.g.,Durkin 1997; Nikolopoulos 1997) recommendsmatching the structure of a knowledge-base to theknowledge representation of domain experts.Much organization design knowledge is repre-sented in the scientific literature as if-then heu-ristics for predicting organizational success. Forexample, Galbraith�s (1977) work on informationprocessing yields rules like: �If an organizationexperiences high input uncertainty, then jobsshould be designed with a high degree of discre-



Exhibit 3. How the Development Process Continually Sought Naïve Users

The Domain Team�s responsibilities were shifted from representing users to managing a process thatwas called �onion layering.� As the naivety of Domain Team members faded (and they became lesslike new customers and more like partners and developers), they were asked to add a new layer ofusers who would test-drive prototypes and provide feedback. As that second layer of naïve usersbecame too enculturated into the development process to maintain their naïveté, the Domain Team wasasked to add yet another new layer of naïve users. By the end of the project, four layers of �naiveusers� had been involved in the project. By never actually replacing the previous users, we were ableto maintain their commitment to the project while simultaneously enlarging our user community. Ourdevelopment process then required not only a user group, but also a continuously replenished set ofnaive users.

Exhibit 4. How Top Modeler Translates Expert Knowledge into Tradeoffs

The tradeoff matrices were developed from the 1,500 if-then rules of organization design experts. Eachcell in each matrix is essentially three rules: one rule for each of the four possible outcomes in each cell(hurts, helps, critical, or neutral). The matrices are designed so the user can trade off differentoutcomes: �If I don�t provide this skill, how much will it hurt me?� versus �If I provide this skill, how muchwill it help me?� For example, the matrix shown in Figure 2 indicates that a particular business manageris providing inadequate discretion to workers, so that workers� output is being hurt. The manager canthen evaluate various options: provide the degree of discretion required, set more realistic performancegoals for the unit if discretion is not provided, or live with the fact that lack of discretion is hurting theunit�s performance.

tion to accommodate, react, and resolve thisuncertainty.� (See Majchrzak [1997] for a discus-sion of the scientific literatures used in develop-ment of TOP Modeler.)

Initially, then, we represented the TOP Modelerknowledge-base as a set of over 1,500 if-thendecision rules. To invoke a rule, a user providedcompany-specific knowledge about inputs (the �if�part of the if-then clause). This rule-based way ofrepresenting the knowledge had excellent pre-dictive ability in tests comparing computer-basedpredictions to those of human experts. In addition,the other knowledge-based software package fororganization design (Baligh et al. 1996) used arule-based knowledge representation. Finally,academic experts hired to review the knowledge-base found that the rule-based format greatlyfacilitated evaluation.

While the rule-based representation worked forexperts, observation showed it did not match non-experts� representations of organization designknowledge. Practitioners did not follow a set ofrules to arrive at a single prescribed solution.Instead, they compared and traded off alternativesolutions and eventually arrived at hybrid com-binations that could not easily be traced back to afinite set of rules. Sometimes they proposed asolution and traced its probable impacts; some-times they started from company constraints tosee what solutions were possible; and sometimesthey balanced many variables simultaneously andsearched for an optimal solution in a wayreminiscent of Pava�s (1983) description of delib-erations. This observation suggested to us thatlay users represent their knowledge as tradeoffsfor action, rather than the if-then rules of experts.The need for a way to translate expert knowledge



Exhibit 5. How the Development Process Used Functional Prototypes in RadicalIteration

Every Friday, Domain Team members downloaded a fully functional prototype from an FTP site andobserved as someone in their organization worked with TOP Modeler to conduct a real use case oforganization design analysis (an activity that took several hours). The Domain Team members reportedback to us early the following week on how the analysis had gone. (How did the user use the system?What were the stumbling blocks, questions raised, or results that the user found of immediaterelevance?) From this feedback, we were able to determine which aspects of the system were morelikely to meet the needs of the unpredictable user community.

As an example, one Domain Team member (from her company�s corporate engineering staff)downloaded the software on Friday and flew to a plant on Monday to meet with the plant manager andplant engineering staff for an organizational analysis. Upon her return, she reported to the DevelopmentTeam the questions and frustrations of the plant personnel. Based on her input, changes were madeto the analysis formulae. Another Domain Team member made the system available to a shop floorsupervisor on Friday. The following Monday, we were informed that the supervisor had worked overthe weekend, performed an organizational analysis of her production operations, and presented theresults to management!

into actionable tradeoffs for non-experts led todevelopment of the tradeoff matrices knowledgerepresentation. Exhibit 4 explains how tradeoffswere represented and used in TOP Modeler.

Again, this shift in system design corresponded toa shift in our system development approach. Ourradical iteration approach differed from the tradi-tional prototyping approach in several ways. First,functional prototypes were used. Initially, asrecommended by many decision support experts(Sprague and Carlson 1982; Watson et al. 1997),we interviewed potential users and encouragedtheir feedback on nonfunctional, throwaway proto-types of screens, reports, and interfaces. Wefound, however, that users� feedback on our earlynonfunctional prototypes was of limited value.The task of organization design had never beforebeen supported by a system. Users could nothypothesize how they might use an organizationdesign support system from the limited evidenceof simulated system features.2 Second, in our

radical iteration approach, users evaluated func-tional prototypes by working with the systemthrough real �use cases� of organization designanalysis, rather than hypothetical ones. Third, wecontinued iterating far more times than is custo-mary with prototyping. During an 18-month peri-od, over 70 functional prototypes were generated.(This activity was supported by the system�scomponent-based architecture, described below.)

Each prototype was fully functional and speci-fically designed to shed light on some aspect of ITsupport for organization design. We tested alter-native interfaces, different representations of theknowledge-base, multiple gap analysis formulae,different ways of providing method guidance, andalternative explanation styles, among othersystem features. Exhibit 5 provides examples ofhow the development process worked.

In sum, we learned that, when designing foremergent knowledge processes, asking people toreview nonfunctional prototypes or participate inconference room pilots or �organizational games�is not sufficient to guide development, becausethese techniques involve hypothetical contexts.People infrequently perform EKPs, and they don�t

2Zmud et al. (1993) have similarly commented on thedifficulty of IS design when users must project into ahypothetical future.



really know how they might act when using anonfunctional hypothetical support system; theycan only demonstrate how they actually do actwhen confronted with a working system and a realuse case. Using functional prototypes allowed usto intervene directly in the work process andobserve which aspects of the system worked andwhich did not. Further, our radical iteration stra-tegy was suited to the novelty of providing ITsupport for the emergent organization designprocess.

Principle #3: Design for Offline ActionWhen we first began observing users working withfunctional prototypes, users reported that TOPModeler�s tradeoff analyses were useful. Butwhen we observed users offline (in organizationdesign discussions in their organizations), wefound they rarely acted on the information gene-rated in TOP Modeler analyses. Overwhelmed bythe sheer number of organization design optionsTOP Modeler considers (each with costs, benefits,interdependencies, feasibilities), users did notknow what actions to take, and as a result theyignored TOP Modeler output when they madeorganization design decisions.

In this way, we learned that it was not enough toinduce naïve users to try TOP Modeler (Principle#1) and to translate expert knowledge intoactionable tradeoffs for lay users (Principle #2),we also had to induce naïve users to do thingsdifferently�to act offline on the basis oforganization design knowledge generated online.Our observations of meetings in which organi-zation design decisions were made revealed thatlay organization designers spent a great deal oftime trying to prioritize actions that would reducethe gaps between existing organization structuresand desired organization performance measures.Should we provide training first, or should weinvest in new equipment and do training later? Isit more important to initiate cultural change now,or wait until we�ve fixed our basic productionproblems? Therefore, to induce offline behaviorchange, we needed to provide support in TOPModeler for action prioritization.

We initially expected that a single actionprioritization approach would work for all potentialusers. But just as lay organization designers havean extremely wide range of relevant backgroundknowledge, they also prioritize actions in verydifferent ways. Some people prioritized actions byfocusing on the biggest performance gaps first.Others focused on gaps related to their highestpriority business objectives. Still others focusedon gaps that could only be resolved with aparticular solution. Therefore, providing severalways for users to prioritize actions to close gapswas necessary to our goal of influencing users�offline actions. Exhibit 6 indicates how TOPModeler fulfilled this design principle.

Our recognition of the need for this design prin-ciple accompanied a major shift in how weconceptualized the system development process.Our initial view of the development process wasformalized in the contract with NCMS and thepartner companies. The contract spelled out ourresponsibilities to supply a system that wouldfacilitate non-experts� use of expert organizationaldesign knowledge. We came to realize, however,that we needed to supply the system-facilitatedapplication of expert organization design knowl-edge to real organization design problems. If TOPModelers� users did not actually take offline action,the goal of better organizational design decisionswould not be achieved. Thus, instead of evalu-ating our performance in terms of what the systemdid and what users did when working with thesystem, we began evaluating our performance bywhat users did offline. Exhibit 7 describes how thedevelopment approach focused on offline action.

Principle #4: Integrate Expert Knowledgewith Local Knowledge SharingWe initially thought of organization design as aholistic organization-wide decision that could beinformed by expert knowledge. Because improvingorganizational effectiveness requires coordinatedinputs from human resource specialists, industrialengineers, manufacturing technology specialists,shop floor workers, supervisors, etc., as well asexpert knowledge, we believed that an organi-zation design support system should synthesize



Figure 3. Summary Gap Panel

Exhibit 6. How TOP Modeler Facilitates Offline Action

TOP Modeler allowed users to prioritize their offline actions in three different ways. First, for those userswho prioritized gaps by focusing on the worst cases first, the color-coded thermometers of the FerrisWheel were helpful, because they indicated where the biggest gaps were located (areas where therewere the most red thermometers).

Second, for those who focused on gaps related to their highest priority business objectives, we provideda �Summary Gap� panel that color-codes gaps based on the number of business objectives they affect.The example depicted Figure 3 involved production technology that did not allow shop floor operatorsdiscretion in three areas: dynamically changing work priorities, improving work procedures, and settingperformance standards. As indicated at the top of the screen, only four out of the nine possible areasof discretion were relevant given the manufacturing unit�s business strategies. Therefore, the fact thatthree out of four relevant areas of discretion were negative for the selected business strategies givesthis gap a higher priority than it might otherwise have had.



The third way users prioritized gaps was by attending to �show stoppers��gaps that could not beeliminated by making compensatory organizational changes. For these users, the Design Model wasdeveloped. (Figure 4 shows the Design Model.) If, for example, the workforce of a unit is missing anecessary skill (such as �repair�), the model indicates that the skill might be provided by makingavailable someone who has the skill (a repairman); thus, this missing skill is not a show stopper anddoes not have to be fixed immediately through training. By contrast, as shown in Figure 4, failing toreward workers for the right outputs is rarely compensated by other features, and thus this gap is ashow stopper.

Figure 4. Design Model



Exhibit 7. How the Development Process Focused on Offline Action

We stopped asking users: What did you learn when you used the system? Why did you press thosebuttons? What will you do with that information? Instead, we started observing what they actually didin their organizations after they used TOP Modeler. We sent Domain team members and developersto watch design meetings before, during, and after TOP Modeler usage. By focusing on offlinebehavior change, the development process took many unexpected turns. One example was ourlearning the three different ways people prioritized TOP Modeler�s gaps. Another example involved ourrecognizing a potential new user group for the system: maintenance workers. Observing maintenanceworkers using TOP Modeler indicated many translation problems between the manufacturing languageand the maintenance language. This caused the Development Team to consider how the language inthe system could be generalized beyond the manufacturing environment. We then made changes inthe knowledge-base to broaden definitions and facilitate translation.

expert and diverse local knowledge inputs into asingle, consensus perspective. We believed thatthe components of a traditional expert supportsystem�knowledge-base, inference engine, andinterface�would promote such a synthesis.

Over the course of the project, we learned thatusers had very different perspectives onorganization design, shaped by their positions andjob requirements. Shop floor workers did notshare the concepts or the concerns, let along theknowledge, of manufacturing engineers, yet theinput of all is needed for a good organizationdesign. Unfortunately, we observed that func-tional prototype users rarely felt the need toconsider others� points of view. Despite theirlimited understanding of many organization designissues, they did not solicit inputs from people inother functional areas. Further, they interpretedthe system�s knowledge-base through the lensesof their functional specialties. As a result, theyperformed incomplete analyses and producedineffective solutions to organization designproblems.

Consequently, we recognized that the compo-nents of a traditional expert support system�knowledge-base, inference engine, and interface�were not enough. We had to integrate theexpert knowledge-base with system design fea-tures that might promote knowledge sharingamong organizational members in different func-

tional areas. In this way, we came to see thatsuccessful emergent knowledge support systemsmust represent a fusion of multiple �system types.�They are not just decision support or expertsystems but also knowledge sharing systems.We also learned that unstructured communicationsystems are not the only effective way to supportemergent knowledge processes: expert knowl-edge repositories combined with knowledgesharing features are a good solution, too.Exhibit 8 describes the features of TOP Modelerthat supported local knowledge sharing.

In keeping with this design principle, the devel-opment process changed. Initially, most of ourfocus was on translating expert knowledge intoactionable knowledge for users. As we increas-ingly focused on users� offline behavior andobserved their lack of interest in others� localknowledge, we began designing to influence theiroffline knowledge sharing as well as their priori-tization of actions to resolve gaps. We observedwho users communicated with before and afterusing TOP Modeler and who they involved inorganization design deliberations.

Principle #5: Design for ImplicitGuidance Through a DialecticalDevelopment ProcessWe initially assumed that the way to support layorganization designers with TOP Modeler was to



Exhibit 8. How TOP Modeler Encourages Local Knowledge Sharing

We redefined the terms in the knowledge-base more generally so that people from different functionalareas could make locally relevant interpretations. We tried to increase opportunities for shop floorworkers and supervisors to participate in organization design decisions by building into the knowledge-base the concepts most relevant to them�workers� skills, work norms, and responsibilities for 144specific shop floor tasks. We added annotation capabilities that made it possible for users to documentinputs and analyses. We added the capability to save and e-mail analysis results to others for evalua-tion and as a catalyst for discussion. We added the capability to save partial inputs into a case so that,for example, managers could suggest business objectives and ask workers to complete a redesign thatmet those objectives. We provided full-screen visuals for projection to promote the use of the systemand its analyses in meetings. And we redesigned the system technically so that it could be easilydeployed on multiple platforms, whether on the shop floor or in the boardroom.

guide (Silver, 1991) them explicitly through theprocess that expert organization designers use.The envisioned process included the followingsteps: objectively conduct a compete socio-technical analysis, submit the analysis to co-workers for deliberation, evaluate the effects ofeach organizational dimension on business stra-tegies and on other dimensions, and collectivelydecide on a single course of action. To supportthis explicit process, we constructed a �roadmap�metaphor for TOP Modeler�s interface design.

However, we found the principle of explicit guidancefaulty in two respects. First, expert organizationdesigners did not themselves follow such aroadmap. Because of the emergent nature oforganizational design, expert organization de-signers need process flexibility. Second, explicitprocess guidance did not work with non-experts.Some TOP Modeler users deliberately circum-vented the roadmap and refused to conduct athorough sociotechnical analysis. For example,one manager was observed to use the system toprove that his organization was already properlystructured by focusing on only one aspect of theknowledge-base, when a more comprehensiveanalysis would have revealed numerous gaps.Further, since the system at that time did notpromote knowledge sharing across functionalareas, there was no self-correcting mechanism forusers� failures to do a good analysis. As designers,we viewed this and similar examples as evidence ofdesign failure, and we sought a better solution.

Eventually, we realized that the autonomy ofknowledge workers makes explicit processguidance risky and failure-prone. We had no wayto ensure that they would conduct completeanalyses or engage their co-workers in delibera-tions about the meanings of terms, interpretationsof findings, and evaluations of alternative actions.So, instead of guiding users explicitly, we guidedthem implicitly. A key design decision was toreplace the roadmap with the Ferris Wheel as theinterface metaphor, thereby encouraging fulleranalysis. To promote deliberations, we addedextensive explanations. How TOP Modeler impli-citly guided users toward fuller sociotechnicalanalyses and deliberations with co-workers isexplained in Exhibit 9.

Arriving at the Ferris Wheel interface required usto change the way we dealt with conflictingrequirements, such as the conflict between non-experts� need for guidance in organization designand their undeniable autonomy in system use.We initially pushed for consensus, as recom-mended in the 50 IS development textbooksreviewed by Salzman and Rosenthal (1994).However, we found that pushing for consensussharply limited what people would be able to dowith the system. We then tried a principle ofproviding �both-and capabilities� that gave theappearance of reconciling the conflicting require-ments. For example, we developed a button thatswitched between the roadmap and a �quick-start�option. But this solution did not meet our objec-



Figure 5. Example Explanation



Exhibit 9. How TOP Modeler Guides Users Implicitly

To guide users to conduct more complete sociotechnical analyses, we created the Ferris Wheel as theinitial screen and main interface to TOP Modeler. In the center of the Ferris Wheel were businessstrategies, surrounded by the 12 sets of organizational design features. We intended this represen-tation to imply that all 12 organizational design features must be aligned with business strategy;therefore, one should not selectively attend to one feature and ignore the others. User testing revealedthat the Ferris Wheel interface did indeed accomplish the goal of conveying an intuitive grasp of theneed for comprehensive sociotechnical analyses.

Further, because we set default values on the Ferris Wheel to zero, the thermometers on the Wheelinitially showed red. This motivated users to continue their analysis until they could resolve the worstorganizational gaps (the red thermometers). For example, Figure 1 presents results from a unit thatwas generally following best practices to achieve the unit�s business strategies. However, the thermo-meters (dark gray in the figure, but red on the screen) clearly indicate inadequate involvement ofcustomers. The businessman confronted with this evidence of under-performance was challenged totake action offline to improve the situation.

To encourage deliberations among co-workers, we precisely defined each term in a way that requiredinformation from people knowledgeable about the particular organizational design feature. For example,TOP Modeler posed questions about norms (behavior expected of employees) encouraged bymanagement. Management encouragement of a norm was defined in terms of: mangers� verbal andoperational sanctions, managers� demonstrations of the norm, and specific examples of norm com-pliance observable on the last shift. This definition encouraged the user to seek out shop floor workersto determine if specific examples of a particular norm had been demonstrated recently. As a result, theywere often drawn into deliberations.

In addition to detailed definitions of terms, explanations for every result were provided. Figure 5 showsan example of an explanation. These explanations were specifically designed to encourage discussionand deliberations.

tives for guiding users� offline actions. In the end,we adopted a dialectical approach to development(Churchman 1979; Truex et al. 1999) that enableda more fundamental resolution of the conflictingrequirements.

In a dialectical development process, contra-dictions in requirements are not viewed as hurdlesto be overcome. They are actually the mechanismby which effective systems support for EKPs iscreated. The emergent design process is one inwhich the designer responds creatively to theunexpected failures of a prototyped design featureto achieve its desired effect. Exhibit 10 describesthe dialectical nature of our development process.

Principle #6: Componentize Everything,Including the Knowledge-baseFrom the outset, we had planned a component-based architecture that would isolate the knowl-edge-base from the inference engine and inter-face. Componentization has long been recom-mended by DSS and ES developers as an aid tosoftware construction and maintenance (Nikolo-poulos 1997; Sprague and Carlson 1982).

What we had not anticipated was the extent ofcomponentization required in TOP Modeler.Because the organization design process wasemergent, we continually encountered new userswith new use cases, thwarting every attempt we



Exhibit 10. How the Development Process Was Dialectical

The dialectical approach used in the development process for TOP Modeler consisted of several steps.First, we clearly articulated apparent contradictions between user requirements as dilemmas, with validarguments in support of each side. We then selected one side of the dilemma, developed an approachfor effectively satisfying the needs epitomized by that side, and then created a prototype exemplifyingthe approach. To assess how successful we had been, we used criteria appropriate to that side of thedilemma. The roadmaps are an example of a solution for the �provide guidance to non-expert users�side of the dilemma.

Next, we went through the same process for the other side of the dilemma. For example, we developedan interface that provided essentially no guidance by allowing users to click through the knowledge-base as they saw fit. We then identified the underlying regularity, structure, or domain content that wascommon to the solutions for both sides of the dilemma and incorporated both the solutions and thecommonality into the final product. Our creation of the Ferris Wheel interface illustrates the results ofsuch a dialectical approach.

Exhibit 11. How TOP Modeler Demonstrates Radical Componentization

Within the interface component, we had the following components: a matrix viewer (used with theTradeoff Matrices discussed above), a gap viewer, the Ferris Wheel viewer, roadmaps, and a DesignModel viewer. Within the inference engine, we had the following components: a constraint satisfactionsystem and a �gap� aggregator (for computing aggregated values that appeared in the Ferris Wheelthermometers). Within the knowledge-base, we had the following components: the Tradeoff Matrices,the more complex Design Model, definitions of terms and concepts, and explanations. Even theTradeoff Matrices had components, corresponding to the 12 sets of organizational features for eachbusiness strategy. Finally, system explanations were componentized as well. For every object in TOPModeler�every screen, every organizational feature, every gap result�there was a componentizedexplanation composed of three parts: definition, functional explanation, and contextual explanation. Inall, there were over 21,000 explanations. Figure 6 shows a schematic view of TOP Modeler�scomponent-based architecture.

made to stabilize the knowledge, inferenceengine, and interface. In reaction, we proceededto componentize virtually every aspect of thesystem. Exhibit 11 demonstrates the extent ofcomponentization in TOP Modeler.

The componentized architecture ensured that, asdomain knowledge evolved, the system wouldevolve with it. As components were modified, theycould be dynamically �plugged into� the genericsystem structure to very quickly create a new,testable system for user evaluations. This archi-

tecture allowed us to create and experiment withover 70 significantly different system versions in aperiod of 18 months�a pace of innovation notcommonly found in ES, DSS, or EIS development(Watson et al. 1997).

In addition, the componentized structure allowedfor easy post-development modifications to thesystem. For example, changing explanations ofconcepts and relationships was easily done by adesignated knowledge-base maintenance person:changes are made in a Word document and the



Interface toNew PluggableKnowledgebases

AnalysisLogic

Object-Oriented Query Generator

Object-OrientedUser Interface

Users

[Relational Model Meta-Knowledgebase]

KnowledgebaseConstruction

Tools

Designer(s)of New

Knowledgebases

Excel +MS Word

OLE Objectsas EditingInterfaces

DeliveredSystem

CaseFile

KBFile

KBFile

KBFile

AlternativePluggable

KnowledgebaseCollection

for Multiple Domains

Stable CoreSystemFramework

Relational Model KnowledgebasesTheory and Org-Wide Data

Data from Users

Theory and Org-Wide Data

Data from UsersCaseFile

Figure 6. Top Modeler System Architecture

system is recompiled with the new Word docu-ment. Similarly, changes to the matrices can bemade by simply updating an Excel spreadsheet.3

Our strategy of radical componentization in thedesign of TOP Modeler had significant effects onthe development process. Much of the time indevelopment team meetings was spent onworking out critical assumptions about how thecomponents would work together. Often, theassumptions only became clear when they wereviolated by experience. For example, thedeveloper of the matrix viewer assumed a certainregularity in the database that was not initiallyintended, but was initially present. As devel-

opment proceeded, that regularity was removed,and problems with the matrix viewer arose. Muchdiscussion ensued: Should the matrix vieweraccommodate lack of regularity in the database,or should the database be made more regular?To surface such issues as soon as possible, weinstituted at least weekly integration builds, apractice employed in numerous open-sourcedevelopment projects and later popularized byMicrosoft (Cusumano and Selby 1995; DiBona etal. 1999; Raymond 1999).

Radical componentization also allowed radicalresponses to changes in user requirements as ourknowledge about IT support for EKPs grew. Forexample, two-thirds of the way through thedevelopment effort, the radical componentizationin TOP Modeler made it possible for us to replacethe roadmap interface with the Ferris Wheel. Thissystem modification was so major that theExecutive Steering Committee discouraged it,

3Since changes are so easy to make in TOP Modeler, amajor concern is that the scientific grounding of theknowledge-base can be lost. Thus, a user group isgenerally organized in a company to monitor recom-mended changes to the knowledge-base.



believing that it would be too difficult to implementon time. The fact that the change was completedon time and within budget attests to the robust-ness of the system architecture and the flexibilityof our development process.

In sum, the emergent nature of EKPs ensuresunexpected problems and opportunities through-out the system development effort. Therefore,structuring the system and the development effortto pursue the unforeseen is a way to incorporategreat ideas into the system design�even in theproject�s waning hours.

Contribution of EKPDesign Theory

To recapitulate, we identified a class of designproblems we call emergent knowledge processes.This class of problems has different process, user,and knowledge requirements from those of semi-structured decision-supporting systems. In addi-tion, this class of problems is not adequatelysupported by existing system types (e.g., DSS,EIS, ES, groupware, etc.) either alone or inunintegrated combination. Therefore, a new ISdesign theory for systems that support EKPs isneeded. Such a design theory matches principlesguiding the selection of system features andprinciples guiding the development process withthe unique user requirements of EKPs. Figure 7summarizes our EKP design theory.

We argue that this EKP design theory is animportant theoretical contribution, first, because itaddresses the IT support and development pro-cess requirements of an important class of humanactivities. In addition to organization design,EKPs include the important processes of strategicplanning, new product development, basicresearch, and indeed academic theory building(Weick 1989). More instances of this class mayeventually be identified. An example is intellectualcapital management (Stewart 1997). By showinghow one process in the class can be successfullysupported with IT, we pave the way for furtherattempts to develop IT support for EKPs. A

plausible example is the application of knowledgemining technologies to the development ofacademic theory.

Second, our EKP design theory is an importanttheoretical contribution, because it shows how thefeatures of familiar system types (such as DSS,EIS, ESS, etc.) can be effectively integrated (notjust added) to accomplish effective support.Through its emphasis on integrated support, ourEKP design theory helps resolve the considerable(and to our mind unproductive) disagreement inthe knowledge management field (Fahey andPrusak 1998) about whether the best approach tosupporting EKPs is via a high-tech �contentful�system, such as a case-based reasoning tech-nology (El-Sawy and Bowles 1997), or via a low-tech communication-type system (i.e., usersupplied content), such as video conferencing oremail (Brown and Duguid 1998).

Third, our EKP design theory is an importanttheoretical contribution, because it shows how ISdevelopment practices need to be modified for thespecial requirements of EKPs. For example, areview of over 50 IS development textbooks(Salzman and Rosenthal 1994) articulates thedevelopment principle of striving for consensusamong user requirements. How can this principlebe applied to EKPs when specific user rolescannot be targeted? In such a situation, Salzmanand Rosenthal recommend that, �rather thantrying to achieve consensus� (p. 184), developersneed to conduct a tradeoff analysis to determinethe real and legitimate needs of different parties.This recommendation suggests that the designgoal is a single, privileged solution, to which otherneeds are subordinated. But the many differentevent triggers of EKPs permit no single approachto prevail. Instead, EKP design theory recom-mends dialectical development as a way to designsystem features that reconcile, rather than tradeoff, conflicting requirements.

Fourth, our new IS design theory for EKPs is animportant theoretical contribution, because it bothprovides guidance to developers and sets anagenda for academic research. EKP design theorymakes the development process more tractable



Characteristics of Emergent KnowledgeProcesses (Kernel Theory)1. It is nearly impossible to predict in advance

who will participate in the process andwhich tools they will use.

2. Knowledge is distributed and includes bothgeneral expertise and local context knowledge

3. The process is emergent

Requirements for IT Support of EKPs1. Systems cannot target specific user roles, depend

on training, or assume motivation to use the tool2. Systems must accommodate complex, distributed,

and evolving knowledge-bases3. Systems must support an unstructurable, dynamically

changing process of deliberations and tradoffs

EKP Support System Design and Development Principles1. Design for customer engagement by seeking out naïve users2. Design for knowledge translation through radical iteration

with functional prototypes3. Design for offline action4. Integrate expert knowledge with local knowledge sharing5. Design for implicit guidance through a dialectical develop-

ment process6. Componentize everything, including the knowledge-base

Effective EKP Support System

Characteristics of Emergent KnowledgeProcesses (Kernel Theory)1. It is nearly impossible to predict in advance

who will participate in the process andwhich tools they will use.

2. Knowledge is distributed and includes bothgeneral expertise and local context knowledge

3. The process is emergent

Requirements for IT Support of EKPs1. Systems cannot target specific user roles, depend

on training, or assume motivation to use the tool2. Systems must accommodate complex, distributed,

and evolving knowledge-bases3. Systems must support an unstructurable, dynamically

changing process of deliberations and tradoffs

EKP Support System Design and Development Principles1. Design for customer engagement by seeking out naïve users2. Design for knowledge translation through radical iteration

with functional prototypes3. Design for offline action4. Integrate expert knowledge with local knowledge sharing5. Design for implicit guidance through a dialectical develop-

ment process6. Componentize everything, including the knowledge-base

Effective EKP Support System

Figure 7. A Design Theory for Systems That Support Emergent KnowledgeProcesses



for developers by restricting the range of allowablefeatures (or rules for selecting features) anddevelopment practices to a more manageable set.EKP design theory represents a total solution tothe design problem by correlating user require-ments (based on academic theories or prac-titioners� theories-in-use) with system design anddevelopment principles. Moreover, as an ISdesign theory (Walls et al. 1992), EKP designtheory provides a set of general principles to solvea class of business problems, rather than a uni-que set of system features to solve a unique busi-ness problem. Thus, our design theory representsan advance from the largely atheoretical practiceof requirements-driven software development.

EKP design theory also sets an agenda for aca-demic research by articulating theory-basedprinciples that are subject to empirical, as well aspractical, validation. Just as Keen and ScottMorton (1978) made a contribution to the IS fieldby alerting researchers and developers to theexistence of a new design situation that theylabeled DSS, and just as Silver (1991) extendedand formalized DSS design theory, so we toohope to make a contribution with our identificationof the requirements, system features, andeffective development practices of EKPs.

In the next section, we detail an agenda forresearch inspired by our theory. However, as ourdesign theorizing was stimulated by our experi-ence with a single revelatory case, we need toaddress the generality of our contribution.Because IS design theories are theories (Walls etal. 1992), our design principles can be restated asa set of hypotheses to be tested empirically inother situations where the same theoreticalconditions hold (e.g., in work contexts charac-terized by emergent knowledge processes). Ineffect, our generalizability argument is that,because organization design has the charac-teristics of the general class of emergent knowl-edge processes, applying our EKP design anddevelopment principles to other specific EKPs(e.g., strategic planning) will result in new systemsthat successfully support those processes. Onlythe accumulated weight of empirical evidence willestablish the validity of this hypothesis.

Conclusions

In this final section of our paper, we considerfuture research directions and practical impli-cations.

Agenda for Future Research

As part of the MIS Quarterly themed issue ontheory development, our conceptualization is onlyas good as its implications for further research.Two primary lines of future research flow from ourdesign theory.

An obvious first step on the research agenda is tovalidate the design theory. Validation questionsinclude:

� Do the requirements for IT support for EKPsoutlined in Table 2 constitute a complete set?Are there alternative sets of requirements thatalso fit the kernel theory of EKPs?

� Can other development teams follow the EKPdesign and development principles to producesuccessful systems?

� Do systems that satisfy the principles outlined inTable 2 successfully enhance the outcomes ofEKPs? In particular, is it possible to quantifythe benefits of EKP support?

� Is it possible to demonstrate that the perfor-mance effects of using EKP support systemsare really attributable to the systems instead ofto �placebo� or �Hawthorne� effects?

� Are EKP support systems effective in allcontexts? In the case of TOP Modeler, bothHewlett-Packard and General Motors were pilotsites. Despite the vast differences in organiza-tion structure and culture in these two organi-zations, the tool was successfully used in both.Does that suggest that organizational structurehas limited effects on EKP support system use?

� Do the requirements and principles apply onlyto EKPs and EKP support systems, or can theyalso be usefully employed in the case of other



knowledge work processes (e.g., semi-struc-tured decision making)? Put differently, is thereany downside to providing more support forsemi-structured decision-making than hasproved successful in the past?

Beyond validation, the specific principles listed inTable 2 suggest additional challenging researchissues.

� The Principle of Designing for CustomerEngagement by Seeking out Naïve Users: Howcan an EKP support system be designed toencourage use, if users are not required even totry it? What kinds of novel system implemen-tation strategies might be appropriate for EKPs?

� The Principle of Designing for KnowledgeTranslation Through Radical Iteration withFunctional Prototypes: With EKPs there mustbe both deliberation and action. How can anEKP support system achieve an appropriatebalance�not sending people off to makebusiness changes willy-nilly, but not contributingto �analysis paralysis� either?

� The Principle of Designing for Offline Action:This principle raises difficult ethical issues forthe system designer. What behaviors can andshould one support? For instance, one clientwanted TOP Modeler to determine how manypeople could be laid off in a manufacturing worksetting. How far can and should the designergo to support business �needs�?

� The Principle of Integrating Expert Knowledgewith Local Knowledge Sharing: Recent ad-vances in text mining, directory standards, and�living portal� technologies suggest that muchmore can be done in the way of content-richsupport for knowledge workers. However,crafting the right kinds of tools and integratingthem with effective social interaction patternswill call for interdisciplinary research effortsinvolving computer scientists, anthropologists,and IS specialists.

� The Principle of Implicit Guidance: Tools likeTOP Modeler send very strong signals to peopleabout what they should do. This type of guid-

ance can interfere with organizational authoritysystems. For example, in one of our pilotorganizations, a shop floor supervisor obtaineda copy of TOP Modeler, conducted an analysisof her organization, and reported to her man-agement that her organization�s performancewas hindered by her lack of control overmaterial handling resources. Management wastaken aback at her boldness in making organi-zation design proposals �outside of her purview�and promptly forbade her and all shop floorsupervisors to use the tool. What responsibilitydo IT designers have to help organizationsmanage the unintended social and politicalconsequences of using their tools?

� The Principle of Radical Componentization:This principle is intended in part to keep an EKPsupport system alive�updated, current, main-tained. But who �owns� the job of keepingexpert knowledge current? In-house expertswho might dilute the scientific validity by servinglocal agendas? Or expensive third parties?

Finally, the overall EKP development strategy ofradical iteration and observation of naïve usersworking with functional prototypes on real usecases also raises challenging research questions.Our development process involved weekly itera-tions with functional prototypes (70 in total) thatwere field tested by users in actual use casesunder the careful observation of developers. Thisprocess required significant user involvement, farbeyond the normal demands of the systemdevelopment life cycle. In addition, it required thedevelopment team to place an independentobserver into the client/user organization. Therequired resource commitments were clearly veryhigh. While this approach has been used whenpiloting collaborative KMS for new product inno-vation (Majchrzak et al. 2000), the high resourcerequirements suggest the need for research onwhen such a development strategy is absolutelyessential.

Implications for Practitioners

For practitioners, our EKP design theory has twomajor implications. First, the design theorizing



approach offers benefits over an atheoreticalrequirements-driven system development ap-proach. With the feature list approach, devel-opers are often faced with hard choices aboutwhich features to cut when time or budget runsshort (Cusumano and Selby 1995). In addition,the traditional approach provides designers withlittle guidance about what to do if particularfeatures fail to gain user acceptance or to producedesired offline outcomes. The design theorizingapproach yields general principles to guide thesystem developer. Since there are always alter-native features capable of satisfying generaldesign principles, our approach allows the devel-oper both guidance and flexibility.

Second, EKP design theory suggests that there isgreat potential scope for EKP support systemsthat combine both expert knowledge and localknowledge sharing. As of yet, no packagedapplications or development tools for an inte-grated EKP support system exist�so suchsystems must be painfully developed from scratchin those areas most likely to yield strategicorganizational payoffs. However, we believe ourexperience in the case of TOP Modeler�and thegeneral principles we derived from it�providedevelopment teams with guidance on where andhow to start.

Acknowledgements

We would like to thank the organizers and parti-cipants of the Oklahoma Workshop in May 2000,as well as MIS Quarterly reviewers for their helpfuladvice on earlier versions. We would also like tothank the many people who made TOP Modelerpossible.

References

Alavi, M. �Managing Organizational Knowledge,�in Framing the Domain of IT Management:Projecting the Future Through the Past, R. W.Zmud (ed.), Pinnaflex Educational Resources,Cincinnati, OH, 2000.

Baligh, H. H., Burton, R. M., and Obel, B.�Organizational Consultant: Creating A Use-

able Theory for Organizational Design,� Man-agement Science (42:12), 1996, pp. 1648-1662.

Beyer, H., and Holtzenblatt, K. ContextualDesign: Defining Customer-Centered Systems,Morgan Kaufmann Publishers, Inc., SanFrancisco, 1998.

Bhattacharya, S., Krishnan, V., and Mahajan, V.�Managing New Product Definition in HighlyDynamic Environments,� Management Science(44:11), 1998, pp. 50-64.

Blair, D. C. �The Management of Information:Basic Distinctions,� Sloan Management Review(26:1), 1984, pp. 13-23.

Boland, R., and Tenkasi, R. �Perspective Makingand Perspective Taking in Communities ofKnowing,� Organization Science (6:4), 1995, pp.350-372.

Borys, B., and Majchrzak, A. �User Concerns inAdapting Organization Design Theory toOrganization Design Practice: Results of aLongitudinal Field Study� University of SouthernCalifornia Technical Report, Los Angeles, 1999.

Brown, J. S., and Duguid, P. �Organizing Knowl-edge,� California Management Review (40:3),1998, pp. 90-111.

Cannon-Bowers, J. A., Salas, E., and Converse,S. �Shared Mental Models in Expert TeamDecision Making,� in Individual and GroupDecision Making, J. J. Castellan (ed.),Earlbaum, Hillsdale, NJ, 1993.

Checkland, P. Systems Thinking, Systems Prac-tice, Wiley, Chichester, 1984.

Cherns, A. B. �The Principles of SociotechnicalDesign,� Human Relations (29), 1976, pp. 783-792.

Cherns, A. B. �Principles of SociotechnicalDesign Revisited,� Human Relations (49), 1987,pp. 153-162.

Churchman, C. W. The Systems Approach, Dell,New York, 1979.

Clark, K. B., and Fujimoto, T. Product Devel-opment Performance, Harvard Business SchoolPress, Boston, 1991.

Couger, J. D. Creativity and Innovation in Infor-mation Systems Organizations, InternationalThomson Publishing, New York, 1996.

Cross, N. �Descriptive Models of Creative Design:Application to an Example,� Design Studies(18), 1997, pp. 427-440.



Cusumano, M. A., and Selby, R. W. MicrosoftSecrets: How the World�s Most Powerful Soft-ware Company Creates Technology, ShapesMarkets, and Manages People, Free Press,New York, 1995.

Davenport, T. H., DeLong, D. W., and Beers, M.C. �Successful Knowledge Management Pro-jects,� Sloan Management Review, 1998, pp43-57.

Davenport, T. H., Jarvenpaa, S. L., and Beers, M.C. �Improving Knowledge Work Processes,�Sloan Management Review, 1996, pp. 53-65.

DiBona, C., Ockman, S., and Stone, M. (eds.).Voice from the Open Source Revolution,O�Reilly and Associates, Cambridge, MA, 1999.

Durkin, J. �Expert System Development Tools,� inThe Handbook of Applied Expert Systems, J.Liebowitz (ed.), CRC Press, Boca Raton, FL,1997.

El Sawy, O. A., and Bowles, G. �Redesigning theCustomer Support Process for the ElectronicEconomy: Insights from Storage Dimensions,�MIS Quarterly (21:4), December 1997, pp. 457-483.

Emery, M. Participative Design for ParticipativeDemocracy, Australian National University,Canberra, Australia, 1993.

Fahey, L. , and Prusak, L.. �The Eleven DeadliestSins of Knowledge Management,� CaliforniaManagement Review (40:3), Spring 1998, pp.265-276.

Florman, S. C. The Existential Pleasures of Engi-neering (2nd ed.), St. Martin�s Press, New York,1994.

Frenkel, S. J., Korczynski, M., Shire, K. A., andTam, M. On the Front Line: Organization ofWork in the Information Economy, CornellUniversity Press, Ithaca, NY, 1999.

Galbraith, J. Organization Design, Addison-Wesley, Reading, MA, 1977.

Greenbaum, J., and Kyng, M. (eds.). Design atWork: Cooperative Design of Computer Sys-tems, Earlbaum, Hillsdale, NJ, 1991.

Grudin, J. �Interactive Systems: Bridging theGaps Between Developers and Users,� IEEEComputer (24: 4), 1991a, pp. 59-69.

Grudin, J. �Systematic Sources of SuboptimalInterface Design in Large Product DevelopmentOrganizations,� Human-Computer Interaction(6:2), 1991b, pp. 47-196.

Guthrie, R., and Gray, P. �Junk Computing,�Information Systems Management (13:1), 1996,pp. 23-28.

Hayward, S. �Knowledge Work Also NeedsDesign,� Research Note SPA-11-1992, GartnerGroup, Stamford, CT, July 2000.

Hutchins, E. �The Social Organization of Distri-buted Cognition,� in Perspectives on SocialShared Cognition, L. B. Resnick, J. M. Levine,and S. D. Teasley (eds.), American Psycho-logical Association, Washington, 1991.

Iansiti, M. Technology Integration: Exploring theInteraction Between Applied Science andProduct Development, Harvard BusinessSchool, Cambridge, MA, 1992.

Keen, P. G., and Scott Morton, M. DecisionSupport Systems: An Organizational Perspec-tive, Addison-Wesley, Reading, MA, 1978.

Kivijarvi, H., and Zmud, R. W. �DSS Imple-mentation Activities, Problem DomainCharacteristics, and DSS Success,� EuropeanJournal of Information Systems (2:3), 1993, pp.159-168.

Litterer, J. A., and Jelinek, M. �Design as aSetting for Useful Research,� in ProducingUseful Knowledge for Organizations, R. H.Kilmann (ed.), Jossey-Bass, San Francisco,1983.

Majchrzak, A. �What To Do When You Don�tHave It All: Toward a Theory of SociotechnicalDependencies,� Human Relations (50:5), 1997,pp. 535-565.

Majchrzak, A., and Finley, L. �A Practical Theoryand Tool for Specifying Sociotechnical Require-ments to Achieve Organizational Effectiveness,�in The Symbiosis of Work and Technology, J.Benders, J. de Haan, and D. Bennett (eds.),Taylor and Francis, London, 1995, pp. 95-116.

Majchrzak, A., and Gasser, L. �TOP Modeler,�Information, Knowledge, Systems Management(2:1), 2000, pp. 95-110.

Majchrzak, A., Rice, R. E., Malhotra, A., King, N.,and Ba, S. �Technology Adaptation: The Caseof a Computer-Supported Inter-OrganizationalVirtual Team,� MIS Quarterly (24:4), December2000, pp. 569-600.

Markus, M. L. �Toward a Theory of KnowledgeReuse: Types of Knowledge Reuse Situationsand Factors in Reuse Success,� Journal of



Management Information Systems (18:1), 2001,pp. 57-93.

Markus, M. L., and Keil, M. �If We Build It TheyWill Come: Designing Information SystemsThat Users Want To Use,� Sloan ManagementReview, Summer 1994, pp. 11-25.

Mintzberg, H. The Rise and Fall of StrategicPlanning, MacMillan, New York, 1994.

Mumford, E. Effective System Design andRequirements Analysis, Macmillan Press,London, 1995.

Nikolopoulos, C. Expert Systems, Marcel Dekker,New York, 1997.

Pava, C. Managing New Office Technology: AnOrganizational Strategy, Free Press, New York,1983.

Poltrock, S. E., and Grudin, J. �OrganizationalObstacles to Interface Design and Devel-opment: Two Participant Observer Studies,�ACM Transactions on Computer-Human Inter-action (1:1), 1994, pp. 52-80.

Raymond, E. S. The Cathedral and the Bazaar:Musings on Linus and Open Source by anAccidental Revolutionary, O�Reilly and Asso-ciates, San Francisco, 1999.

Rockart, J. F. �Engaging Top Management inInformation Technology,� Sloan ManagementReview, (25:4), 1984, pp. 3-17

Salzman, H., and Rosenthal, S. R. Software byDesign, Oxford University Press, New York,1994.

Sarker, S., and Lee, A. S. �Using a PositivistCase Research Methodology to Test ThreeCompeting Theories-In-Use of Business Pro-cess Reengineering,� Journal of the AIS (2:7),2002 (online).

Shneiderman, B. �Codex, Memex, Genex: ThePursuit of Transformational Technologies,�International Journal of Human-Computer Inter-action (10:2), 1998, pp. 87-106.

Silver, M. S. Systems that Support DecisionMakers: Description and Analysis, Wiley, NewYork, 1991.

Sprague, R. H., and Carlson, E. D. BuildingEffective Decision Support Systems, Prentice-Hall, Englewood Cliffs, NJ, 1982.

Stein, E. W., and Vandenbosch, B. �Organi-zational Learning During Advanced System

Development: Opportunities and Obstacles,�Journal of Management Information Systems(13:2), 1996, pp. 115-136.

Stewart, T. A. Intellectual Capital: The NewWealth of Organizations, Doubleday, New York,1997.

Taylor, J. C., and Felten, D. F. Performance byDesign, Prentice-Hall, Englewood Cliffs, NJ,1993.

Todd, P., and Benbasat, I. �The Impact of IT onDecision Making: A Cognitive Perspective,� inFraming the Domains of IT Management, R.Zmud (ed.), Pinnaflex, Cincinnati, OH, 2000.

Trist, E., and Murray, H. (eds.). The SocialEngagement of Social Sciences, Volume II,University of Pennsylvania Press, Philadelphia,1993.

Truex, D. P., Baskerville, R., and Klein, H.�Growing Systems in Emergent Organizations,�Communications of the ACM (42:8), 1999, pp.117-123.

Vandenbosch, B., and Huff, S. L. �Searching andScanning: How Executives Obtain Informationfrom Executive Information Systems,� MISQuarterly (24:1), March 1997, pp. 81-99.

Vermesan, A. I. �Foundation and Application ofExpert System Verification and Validation,� inThe Handbook of Applied Expert Systems, J.Liebowitz (ed.), CRC Press, Boca Raton, FL,1997.

Walls, J. G., Widmeyer, G. R., and El Sawy, O. A.�Building an Information System Design Theoryfor Vigilant EIS,� Information Systems Research(3:1), 1992, pp. 36-59.

Watson, H. J., Houdeshel, G., and Rainer, R. K.Building Executive Information Systems andOther Decision Support Applications, Wiley,New York, 1997.

Weick, K. E. Sensemaking in Organizations,Sage Publications, Thousand Oaks, CA, 1995.

Weick, K. E. �Theory Construction as DisciplinedImagination,� Academy of Management Review(14), 1989, pp. 516-531.

Zmud, R., Anthony, W., and Stair, R. �The Use ofMental Imagery to Facilitate Information Iden-tification in Requirements Analysis,� Journal ofManagement Information Systems (9:4), 1993,pp. 175-191.



About the Authors

M. Lynne Markus is Trustee Professor ofManagement at Bentley College. Markus hasover 20 years experience teaching, conductingresearch, and consulting in the information sys-tems field. Her current research interests includethe strategic positioning of electronic market-places, the challenges of B2B systems integration,and inter-organizational change managementaround B2B e-business initiatives. Recent publi-cations include �Toward a Theory of KnowledgeReuse: Types of Knowledge Reuse Situationsand Factors in Reuse Success� (Journal ofManagement Information Systems, 2001), �ThePerformance Impact of Quick Response and Stra-tegic Alignment in Specialty Retailing� (InformationSystems Research, 2000), and �What Makes aVirtual Organization Work�Lessons from theOpen Source World� (Sloan Management Review,2000). Markus holds a B.S. in Industrial Engi-neering from the University of Pittsburgh and aPh.D. in Organizational Behavior from CaseWestern Reserve University. She is a member ofthe editorial board of MISQ Executive and aformer member of the MIS Quarterly editorialboard.

Ann Majchrzak is Professor of InformationSystems at the Marshall School of Business at theUniversity of Southern California. Her researchinterests focus on the design of computer-facili-tated work environments, including the human,technology, and organizational context. Recentpublications include �Radical Innovation WithoutCollocation: A Case Study at Boeing-Rocketdyne�(MIS Quarterly, 2001), winner of the SIMInternational Paper Award Competition, �Tech-

nology Adaptation: The Case of a Computer-Supported Inter-Organizational Virtual Team� (MISQuarterly, 2000), winner of the MIS QuarterlyPaper of the Year Award, and �GeneratingTestable STS Theory� (Journal of Engineering andTechnology Management, 2001). She won the2001 Best Paper Award for the Academy ofManagement Organizational Communication andInformation Systems Division for a study ofknowledge management and reuse in innovativeenvironments.

Les Gasser is Associate Professor at the Grad-uate School of Library and Information Science,University of Illinois at Urbana-Champaign. Hismajor scientific interest has been to understandthe technical and social aspects of organization-scale information processing, in both theory andpractice. This work includes investigations of thenature of social-level knowledge, action, andlearning; organization-scale knowledge-pro-cessing frameworks such as intelligent multi-agentsystems; and concurrent object-based computing.Much of his practically oriented work has had afocus on manufacturing organizations and onissues of enterprise-integration for continuouslylearning, agile firms. He has published over 50technical papers and five books in these areas.Gasser has been on the faculties of ComputerScience, Systems Management, and IndustrialEngineering at the University of Southern Cali-fornia, and has held visiting faculty posts at theUniversity of Paris and the Ecole des Mines deParis. He received his B.A. in English Literature,magna cum laude, from the University of Massa-chusetts in 1976, and his M.S. and Ph.D. degreesin Computer Science from the University ofCalifornia, Irvine, in 1978 and 1984, respectively.

esign heory for ystems that upport mergent …wesrac.usc.edu/wired/bldg-7_file/markus.pdf · is...

Documents