The Alzheimer’s Disease Research Network and the Uniform Data Set1

Douglas R. Austrom2, Ph.D., Betty Barrett3, Ph.D., Elizabeth Merck4, Pamela A. Posey4, D.B.A., Bert Painter5, Ramkrishnan V. Tenkasi6, Ph.D.

INTRODUCTION Alzheimer's disease (AD) is recognized as a public health crisis worldwide (IADRP, 2013). AD is a complex neurodegenerative disease and the leading cause of dementia among the elderly people (Evans et al., 1989). Currently, there are approximately five million AD cases in the United States and about 35 million cases worldwide (Alzheimer's Disease International, 2009. The focus of this case study is on the Uniform Data Set (UDS), a longitudinal database on Alzheimer’s patients, and the 29 Alzheimer’s Disease Centers (ADC) that submit their data to the UDS and actively collaborate in the ongoing maintenance, development and research utilization of the database. The ADCs are based in major medical institutions across the United States. They have a multi-decade track record of collaborative research and a networked and virtual approach to the scientific study of AD. The central coordinating mechanism for the ADCs and the UDS is the National Alzheimer’s Coordinating Center (NACC), which is located at the University of Washington. The NACC coordinates data collection and supports collaborative research among the ADCs.

Interviews were conducted with individuals from the NACC, the NIA, and the ADCs about the virtual work processes, the challenges facing participants in these virtual environments, and about the coordination mechanisms used to resolve those problems in order to co-create knowledge. These data were then analyzed to determine what coordinating mechanisms were used to deal with and overcome the barriers to knowledge development and dissemination that exist in virtual organizations. This research site and project is one of three research sites included in our study of virtual R&D. We used an exploratory case study methodology and a sociotechnical systems analysis framework to assess the factors that contribute to the effectiveness virtual R&D projects. This case study focused on two main research questions:

1. What distinguishes effective deliberations and forums from those that are ineffective in virtual R&D projects?

2. What are the major variances or knowledge barriers in virtual R&D projects?

1 This material is based upon work supported by the National Science Foundation under grant number NSF OCI 09-43237. Any

opinions, findings and conclusions or recommendations in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

2 Indiana University

3 Massachusetts Institute of Technology 4 Sociotechnical Systems Roundtable

5 Modern Times Productions

6 Benedictine University

SCAN OF THE ALZHEIMER’S DISEASE RESEARCH NETWORK

National Institute of Aging and the Alzheimer’s Centers In 1984, leaders at the National Institute of Aging (NIA) recognized that Alzheimer’s Disease and related dementias were catastrophic diseases (Butler, 2005) and decided to create a national, interdisciplinary research program focused on the causes and the course of Alzheimer’s disease. With remarkable foresight, they believed that a network of centers “could create the necessary infrastructure to promote longitudinal clinical-pathological studies; integrate basic and clinical research; standardize clinical assessment tools, methods, and clinical trials; and establish national data banks to share resources for clinical, neuropathological, and genetic studies” (National Institute of Aging, 2005).

The NIA originally funded five Alzheimer’s Disease Research Centers (ADRC) in 1984 and five additional centers in 1985. In 1986 the centers collaborated to establish the Alzheimer’s Disease Patient Registries (ADPR) and the Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). CERAD standardized the definition, assessment, and profile of AD and the criteria for the diagnosis of AD that are still widely used today.

The scientists and clinicians at these 10 ADRCs provided the foundation of a network that today reaches across the country and includes ADCs at 29 major medical institutions (see figure 1 for the geographic distribution of these centers).

Figure 1. Geographic Distribution of the 29 NIA-Funded Alzheimer’s Disease Centers

The ADCs conduct basic research on the mechanisms of AD and translational research to improve diagnosis of the disease and improve care for AD patients. In addition, they support

the patients and their families with the actual diagnosis, medical management of the disease, and information about the disease and available services and resources.

The ADCs are a multidisciplinary mix of clinicians, researchers, and administrators; for example, neuropathologists, neurologists, neuropsychologists, geriatricians, geriatric psychiatrists, radiology and imaging technicians, nurse practitioners, research technicians, social workers, clinical psychologists, epidemiologists, biostatisticians, data base managers, and other computer specialists. While the ADCs are required as a condition of funding to have five common cores -- Clinical, Neuropathology, Data, Education, and Administration -- this does not preclude them from establishing other core clinical or research areas such as neuroimaging and genetics.

National Centers and Initiatives In addition to the ADCs, the AD research network includes several NIA-funded centers: the National Cell Repository for Alzheimer’s Disease located at Indiana University, the Alzheimer's Disease Education and Referral Center in Bethesda, Maryland, coordinates and the National Alzheimer’s Coordinating Center located at the University of Washington. The history of these centers is intertwined with that of the ADCs. In 1989, the NIA created the National Cell Repository for Alzheimer’s Disease (NCRAD), a genetics cell repository. NCRAD banks DNA and cells, builds a database of family histories and medical records, and provides researchers with cell lines and DNA samples from normal control subjects and people with well-documented AD. The U.S. Congress created the Alzheimer's Disease Education and Referral (ADEAR) Center in 1990 to compile and disseminate “current, comprehensive, unbiased” information concerning AD for health professionals, people with AD and their families, and the public (NIA, 2013). In addition to providing answers to specific questions about AD, free publications about the disease -- diagnosis, related disorders, risk factors, treatment, caregiving tips, home safety tips—and training materials, the ADEAR Center can provide information on ongoing clinical trials and referrals to the ADCs and other local services. Collaboration on multi-site clinical trials for the cholinesterase inhibitor, tacrine, led to the creation of the Alzheimer’s Disease Cooperative Study (ADCS) in 1991. The ADCS coordinates a consortium of ADCs and affiliated organizations that conduct large clinical trials on promising compounds designed to improve cognitive functioning, slow the rate of decline, delay the onset of the disease. In 1997, the NIA established the Minimum Data Set (MDC), which was originally housed at Rush Presbyterian Hospital in Chicago to compile data on patients and control subjects enrolled in the ADCs. The MDS included brief clinical and demographic information on over 74,000 patients and control subjects enrolled in the ADCs. While still valuable, the data were assembled retrospectively and were primarily cross-sectional rather than longitudinal and minimal by design. Also, the ADCs did not follow a standardized protocol when these data were originally

collected, even though the data were reported to NACC in defined data elements and the missing data rate was as high as 30%. National Alzheimer’s Coordinating Center (NACC) and the Uniform Data Set (UDS) Key individuals at the NIA and in the larger AD research community recognized the need for a common database with longitudinal data. They believed that this could be best accomplished by creating a coordinating center for the ADCs. National Alzheimer’s Coordinating Center (NACC) was established at the University of Washington by the NIA in 1999 to facilitate collaborative research among the ADCs (NACC website, 2010). The MDS was transferred to the NACC and the first task of the center was to reduce the error rate in the MDS to less than two percent. In 2002, the MDS was expanded to include neuropathological data from autopsies. In collaboration with neuropathology researchers, a new data form was put together that linked clinical diagnoses and pathology findings. In the early 2000’s, the NIA created an External Advisory Committee to make recommendations on the future of ADC program. They recommended expanding and standardizing clinical data collected at the ADCs and stored at the NACC. The NIA formed an ADC Clinical Task Force to plan this expansion, define an expanded data set, and standardize clinical evaluation for all ADCs. The ADC Clinical Task Force, working closely with the NACC, met regularly for three years to define the content of the new Uniform Data Set (UDS), develop common data collection templates, and construct a relational database. The ADCs agreed upon the UDS in early 2005. The NACC piloted the standardized data collection templates in the summer of 2005 and the common protocols were formally implemented that September.

The ADCs enroll subjects in various ways: referrals from clinicians, self-referrals by patients themselves or concerned family members, active recruitment through community organizations, and volunteers who wish to serve as healthy controls. Subjects visit the ADCs annually and are assessed by clinicians, neuro-psychologists, and other ADC research personnel. As many as 18 standardized forms are used and 725 data points are collected at each visit including socio-demographics, family history, dementia history, neurological exam findings, functional status, neuropsychological test results, clinical diagnosis, and imaging tests.

ADC research assistants enter the data from these standardized forms in the ADC’s local database. Data managers at the ADCs monitor the quality of the local data before submitting it electronically to the NACC each month. Data base managers at the NACC also conduct data checks on the data they receive from the Centers before adding it to the UDS. The NACC makes payments of $29,000 per year to the ADCs to cover the costs associated with submitting their data.

A primary goal of the UDS is to accumulate a large-scale pool of data which all qualified scientists can use to discover better diagnostic criteria and treatment options for patients suffering from AD and related neurodegenerative diseases. Table 1 provides a description of the data available from the NACC As of June 2013, there are 28,444 subjects in the UDS

longitudinal database, 74,397 subjects from the original Minimum Data Set, and 13,279 subjects from the Neuropatholgy Data Set.

Table 1. Description of the Data Available from the NACC as of June 20137

Minimum Data Set

(MDS) Uniform Data Set (UDS)

(LONGITUDINAL) Neuropathology

Data Set (NP)

Years covered 1984 - 2005 Sept. 2005 - present 1984 - present

Study subjects Enrollees followed at ADCs (with & without dementia)

Enrollees followed at ADCs (with or without dementia)

Subjects who died and underwent autopsy

Approx. # of subjects*

74,397 28,444 13,279

Approx. # of variables

67 725 85

Method of data collection

Mainly abstracted retroactively from ADC medical records

Collected prospectively by clinicians, neuro-psychologists, and other ADC research personnel, using up to 18 standardized forms at each visit.

Standardized neuropathology form, completed by neuropathologist

Time period covered for each subject

Mainly status on last ADC visit; some variables also capture initial-visit status

Initial visit and each annual follow-up visit, plus milestones such as death or dropout

Status of brain at autopsy

Topics covered

Demographics, cognitive status, clinical dementia diagnosis, selected clinical manifestations, comorbid conditions, MMSE score, vital status, primary neuropathological diagnosis (if died and had brain autopsy)

Socio-demographics on subject and informant, family history, dementia history, neurological exam findings, functional status, neuropsychological test results, clinical diagnosis, whether imaging testing done, ApoE genotype

Demographics, date of death, primary and secondary neuropathological diagnoses, presence/absence of neuropathological features of most major dementias, APOE genotype, brain weights

The NACC also has limited funds to support collaborative research projects that involve at least three ADCs as well as research projects proposed by junior investigators that use the UDS or Neuropathology data.

7 NACC, 2013, https://www.alz.washington.edu/WEB/data-descript.html

Any researcher can request access to the NACC’s database for basic and translational research. The requests are reviewed by a methods group to make sure that they are legitimate. NAC’s staff will also format data if needed, and occasionally give analytic support. The NACC received more than 700 requests in 2010. Since its inception, the NACC has been instrumental in 590 publications as of June 2013:8 191 by external researchers using NACC data, 100 by NACC personnel, 173 from NACC funded projects, 34 based on NACC-funded secondary analysis, 72 from NACC–ADC collaboration in external studies, and 20 that relied on indirect NACC support. The UDS project and the NACC have received some acclaim in the popular press as an exemplar of research collaboration (Kolata, 2010).

Figure 2. Structure of the ADC Committees

While the central role of the NACC is enabling coordination and collaboration among the 29 Alzheimer’s centers through the ongoing maintenance, development and research utilization of the UDS, the NACC also organizes the face-to-face (F2F) meetings of the ADC core directors and ADC steering committees. These meetings are held twice a year in conjunction with the annual conferences of American Neurological Association and the American Academy of Neurology. The structure of the ADC committees is shown in Figure 2. The members of the executive committee and the steering committees are elected. The NACC Director and the NIA’s Program Director are ex-officio members of the Date Core steering committee. The Clinical Task Force is comprised of five members who are elected by the members of the Clinical Core Steering Committee, six members who are appointed by the NIA and NACC, and three members who serve as liaison to other programs. Major Research Initiatives Involving the ADCs This collaborative infrastructure has provided the foundation for several major research programs that either directly or indirectly involve the ADC’s, the UDS, and the NACC.

8 NACC June 2013. https://www.alz.washington.edu/cgi-

bin/broker64?_service=naccnew9&_program=naccwww.pubrep1.sas&TYPEF=DISPLAYIDS

The Alzheimer’s Disease Genetics Consortium (ADGC) Genome Wide Association Study (GWAS) was formed in 2003 to collaboratively use the collective resources of the AD research community to identify variability in genes that influence susceptibility to AD. The ADGC is a partnership between the NACC, NCRAD, the ADCs, and the Children's Hospital of Philadelphia (CHOP). NACC identifies the subjects from each ADC who are eligible for the GWAS, tracks the samples that have been sent to NCRAD for DNA banking, and reimburses the ADCs for their costs. NCRAD sends DNA for genotyping to CHOP, which sends the genotype data to the ADGC. Finally, NACC provides the ADGC with the HIPAA research-limited phenotypic data, which it pairs with the genotype data.

Because AD is such a complex disease with significant phenotypic heterogeneity, it is a greater challenge for researchers to identify the genetic variants associated with AD necessitating a combination of larger studies and novel approaches such as GWAS. (University of Pennsylvania School of Medicine, 2010). The Alzheimer’s Disease Neuro-Imaging Initiative (ADNI) was launched the following year with approximately 50 research sites in the United States and Canada. It is a public-private partnership with the NIA and the National Institute of Bioimaging and Bioengineering, 22 companies, and two Foundations including the Foundation for the National Institutes of Health (FNIH), which administers non-government sources of funds such as the Alzheimer's Association. ADNI data (images, cerebrospinal fluid, and blood samples) are stored anonymously in a central repository and all qualified researchers can access the electronically stored images and test results as soon as data are available.

The success of these collaborative multi-center research programs in the United States has provided a model and infrastructure that has been scaled internationally. To that end, North American ADNI (NA-ADNI) served as a founding member of World Wide ADNI (WW-ADNI), a collaborative effort of scientists from around the world. It is now the umbrella organization for neuroimaging initiatives being conducted by NA-ADNI, European ADNI (2005), Japan ADNI (2007), Australian ADNI (2006), Taiwan ADNI (2011), Korea ADNI, China ADNI (2011), and Argentina ADNI (2012). Brazil ADNI (2014) is a future site. Since WW-ADNI was beyond the funding mandate of the NIA, the Alzheimer’s Association, a major charitable organization dedicated to addressing AD, provided some of the initial funding for this global initiative.

In line with NA-ADNI, the goals of WW-ADNI are to better understand the progression of mild cognitive impairment and Alzheimer's disease, to standardize data collection methods so that data from all sites can be combined and provide a worldwide picture this disease. WW-ADNI is unique because the clinical, neuropsychological, imaging, and biological data are all available to the scientific community at no cost so that scientists around the globe can use this information for their own research purposes.

The Dominantly Inherited Alzheimer Network (DIAN) is another international partnership with 10 research institutions in the United States, United Kingdom, and Australia researching a rare form of AD caused by a gene mutation. DIAN was established in 2008 with a grant from the NIA.

It has received additional funding from foundations, charitable donations, and the German Center for Neurodegenerative Diseases. The DIAN coordinating center is located at Washington University in St. Louis, which is also an ADC site. Figure 3 provides a high level map of the AD research network and the tightly coupled relationships between the ADCs, the centers and projects, and these collaborative research programs.

Figure 3. Mapping the Alzheimer’s Disease Research Network

Our focus, based on our initial scan and mapping of the AD research network/system, is the Uniform Data Set (UDS) project. It has played a central role in the emergence of this far-reaching collaborative research enterprise yet is sufficiently bounded so that we could meaningfully gather data on the key deliberations, both virtual and face-to-face, involved in creating it, implementing it, revising it, and ensuring timely and accurate data submissions from the ADC’s. It also involved a high degree of virtuality, multi-disciplinary activity, and varying levels of complexity and uncertainty in the work to be done and the processes required to complete the work.

These initial interviews also helped us confirm where to locate the UDS project on the six-stage R&D continuum9 that we have been using in our research on virtual R&D projects. As shown in Figure 4, the six stages are demarcated by the degree of task uncertainty and complexity.

Figure 4: R&D Continuum10 with Location of Case Study Projects

Based on the initial scan and the results of the interviews, we concluded that creating, implementing, and subsequently revising the UDS involved a degree of uncertainty characteristic of the initial development stages, or D1 and D2 on the R&D continuum. Once implemented, the activities involved in maintaining the UDS and ensuring timely submissions of accurate data were more routinized and akin to the start-up and scale-up stages (D3 and D4) on the R&D continuum. We used the differences in the degree of complexity to sort the major deliberations that the interviewees identified as either the periodic deliberations involved in designing, implementing,

9 R&D has been characterized as an intrinsic learning system (Purser et al., 1992) with multiple stages. Each stage

is defined by the degree to which participants do or do not know the “what” (objective) or the “how” (method or means) of their knowledge development and synthesizing activities. These stages form a developmental continuum that ranges from projects with high uncertainty in which participants don’t know what is the final or real objective in concrete terms and don’t know how to operationalize it – to projects with low uncertainty in which participants know what they need to achieve and also know how to achieve it operationally.

10 Carolyn Ordowich (personal communication, March 26, 2009) adapted the R&D continuum from a research portfolio model originally developed at Bell Laboratories (Mashey, 2008; Revkin, 2008).

R

1

R

2

D

1

D

4

D

2

D

3

and revising the UDS or the ongoing deliberations needed to ensure the quality and timeliness of the data submitted to the UDS.

Major Periodic Deliberations to Design, Implement, and Revise the UDS The NACC’s mission is to encourage and support more effective collaboration among ADCs across the United States. The success of the UDS has been a critical contributor to achieving that mission, both symbolically and practically. The UDS is the result of innumerable deliberations that have occurred since 1992, and probably many years before the planning of the UDS was formally initiated. As noted above, the process of designing and then launching the UDS took three years and required considerable effort on the part of numerous individuals from the ADCs, the NACC, and the NIA. This work was entrusted to the Clinical Task Force and various sub-committees comprised of subject matter experts on the main elements of the UDS. The key deliberations, participants, values divergence and knowledge barriers, and the coordinating mechanisms involved in designing, implementing, and revising/upgrading the UDS are summarized in Table 2. While the NACC and UDS have facilitated greater collaboration in the field of AD research, the task was fraught with challenges. Some of the ADC scientists disagree with the fundamental premise that standardized data collection and a uniform data base approaches is a more effective and productive way to conduct research on AD. It is not clear whether this constitutes a knowledge barrier due to lack of common frame of reference or due to divergent values. One person we interviewed observed that the UDS approach violated what he understood about the scientific method; namely that hypotheses and research questions should determine the measures and data to be collected, and not the other way around. Another interviewee described the disagreement this way: The trade off is the perceived value of data from tens of thousands of people collected uniformly versus the value of 29 centers pursuing individual and local and maybe smaller consortium expertise (Personal interview, 2011). It is understandable that the scientists at the ADCs placed a high value on autonomy and the discretion to conduct research on AD as they deemed most appropriate. As some of the most eminent scientists in the field of AD researchers – demonstrated, in fact, by their ability to secure NIA funding for their ADCs -- they undoubtedly believe that they deserve this latitude. There is also the question of the cost of the UDS and the NACC, especially when research funds are limited. Creating an integrated system that allows collection, distribution, and shared use of the data is also expensive, and not everyone believes that it is the best use of limited research funds. One center director that we interviewed commented on the fact that more than one grant was necessary to create and maintain the UDS.

Table 2. Major Periodic Deliberations to Design and Revise the UDS

Topics and Issues

Participants

Values Divergence/ Knowledge Barriers

Coordinating Mechanisms

Fundamental premise of the UDS

ADC Directors and Scientists, NIA, NACC PI

Autonomy and research independence versus standardized methods and measures Lack of shared frame of reference

Transcendent purpose, Threat of ADC not being renewed Latitude to pursue personal research agenda as well as UDS

Design, build support, and agree for the key elements of the UDS:

Standardized clinical evaluation protocol for assessing and diagnosing AD and other dementias

Common clinical data points

Common data collection forms

Relational database

Clinical Task Force, discipline-specific sub-committees that solicit input and feedback from ADC researchers, NIA PM, and NACC PI

Lack of a common frame of reference (competing theories on the mechanisms of AD) Failure to use knowledge from all disciplines

Multiple opportunities to provide input and influence decisions Transcendent purpose, Credible and influential network builders and leaders ADC steering committees NACC NIA

Revise UDS to replace proprietary instruments with non-proprietary instruments; anticipate and incorporate scientific breakthroughs

Clinical Task Force plus discipline-specific sub-committees that solicit input and feedback from Center staff

Sunk costs in longitudinal data already collected versus rising licensing costs and lost data due to inability to extend licenses to non-ADCs Failure to use knowledge from all disciplines

Multiple opportunities to provide input and influence decisions; Transcendent purpose; Credible and influential network builders, peer leaders; ADC steering committees; NACC; NIA

These deliberations occurred in informal discussions with peers at their own ADC and other ADCs and with individuals from the NACC and the NIA and in more formal settings such as the semi-annual directors meetings as well as numerous scientific meetings. The forums occurred both face-to-face (F2F) and virtually. There were three factors that allowed a workable resolution of the divergent values in this deliberation; first, the compelling mission that the AD researchers share, the cure and eradication of this devastating disease; second, the latitude to pursue an ADC-specific research agenda along with the common agenda of the UDS; and third the financial incentives of continued funding for the ADC Countless deliberations were also required to design, build support, and agree on the key elements of the UDS which included a standardized clinical evaluation protocol for assessing and diagnosing AD and other dementias, a common set of clinical data points, common data collection forms, and a relational database for the UDS.

This was arguably the most challenging set of deliberations. Reaching agreement on the value of common standards in most decentralized organizations is conceptually a straightforward task. But in practice, people are only supportive of common standards and common approaches if the chosen standards resemble their current practices. The scientists and ADCs had all invested considerable, time, money, and professional credibility to conduct the clinical evaluations and gather the data in ways that best supported their research agenda and theories on the mechanisms of AD. Designing, building support, and reaching agreement on which clinical evaluation protocol to use, which clinical data points to collect, how to configure the data collection forms, and how to construct the optimal relational data base to satisfy ADCs required an iterative process of building commitment to, or at least, compliance with the UDS.

The Clinical Task Force surveyed the ADCs to find out what protocols, data points, and measures they were using at that time. They sought people’s input and feedback throughout the process. In this set of deliberations, the knowledge barriers were due to divergent values, lack of common frame of reference, the failure to use the full range of knowledge resident in the ADC network. For example, some of the disciplines such as data managers did not believe that their input was fully utilized in the final design of the UDS.

The UDS is evolving, and these changes are constrained by the choices made at the outset. The early decisions shaped the current form of the data set, and the well-established longitudinal database made it difficult to make alterations to the UDS. But several of the instruments used in the UDS are proprietary and their use is governed by agreements that have become quite restrictive; in particular, the license fees for these instruments have increased dramatically. Here is how one interviewee described this challenge (Personal interview, 2011):

Almost all sites would use a Mini Mental State at some in their characterization of patients. Well, it turns out that many of these instruments that we use, particularly the neuropsychological instruments, have been licensed by commercial firms. So we had to enter into licensing agreements. Actually NACC, the National Alzheimer’s Coordinating Center, is the one that developed these licensing agreements. And the companies varied in their willingness to do this. But some of them were really very restrictive. They’d had a bad experience with one university or another, and they didn’t want them to be part of our licensing agreement. But in the end of it all, it is that we do have blanket licensing agreements for all Alzheimer’s Disease Centers to use the uniform data set in ADC participants. If you’re not in one of the ADCs, the licensing agreement doesn’t extend to you; you would have to go to the individual companies and develop your own licensing agreement with each company that has an instrument in the UDS. It’s a tremendous problem. It turns out that the licensing fees keep going up each year. It’s absurd to us – I’m sorry, but this is a real problem for us. It’s absurd for us because many – I think I mentioned the first centers began over 25 years ago, and we tried to adhere to using the same instruments that they’ve been using for 25 years, so we didn’t ask to throw all of their longitudinal data sets out of the window. So we’re using in many instances, forms or versions of these cognitive tests that have been subsequently revised. So the forms we use are no longer being manufactured, except for us, and the companies – we’re the only users and they keep raising the rates on us, even though they’re selling other versions, newer versions of

these forms. So it’s been a real problem. Many people have asked for access to the UDS; we’re able to give them the parts, the components, that are not licensed, but most of the cognitive tests are licensed, so it’s a real, real problem. And talk about evolution, we have a current task force, primarily made of neuropsychologists, that is trying to develop a new neuropsychological battery for the UDS that would be comprised of tests that are unlicensed, so that we could then disseminate the entire UDS without these restrictive licensing agreements.

One researcher remarked that standardization is also holding back change. The concern raised is that changing the items or definitions of elements in the items can damage long-term studies by creating uncertainty about the meaning of the data. Another concern raised was the cost of change in terms of the reorganization of the data set, the forms used to gather the data, and staff needed to make the changes. The Clinical Task Force and subject area sub-committees conducted a lengthy set of deliberations to develop and pilot a non-proprietary battery of instruments. Most of these deliberations were conducted virtually using email, teleconferences, and occasionally videoconferences. They also met in person at the Directors meetings. They experienced a similar knowledge barrier revising the UDS as they did developing the original UDS; namely, the failure to use the knowledge and expertise of all of the disciplines involved in the ADCs. For example, one interviewee commented that they did not believe that any data managers from the ADCs were included in the sub-committees. But the need to anticipate scientific developments and modify the UDS accordingly is an ongoing challenge. In a field as vitally important as AD research, there is unrelenting pressure to stay on the forefront of scientific discoveries and technological advances. As one interviewee stated (Personal interview, 2011),

. . . it’s a hard row to hoe because a lot of that literature is in computing journals and engineering journals and linguistic journals, and it’s hard to get the mainstream biomedical community to sort of turn the battleship around. It’s an interesting time we live in. But I think one of the amazing transformations that’s going to happen is that hopefully with the emphasis on the electronic media we will begin to realize that there’s a tremendous power in aggregating data electronically, but actually the electronic capture of data doesn’t just have to occur in a clinic. It can actually occur anywhere at any time.

These anticipatory deliberations can conceivably involve all member of the AD Research network; especially the formal and informal network leaders. These deliberations have occurred and will continue to occur both within the formal structure and the informal influence network.

Major Ongoing Deliberations to Maintain the UDS

The major ongoing deliberations required to maintain the UDS and ensure data quality and timely submissions to the UDS are summarized in Table 3. Implementing the original UDS as well as revised versions of the UDS involved deliberations to educate ADC personnel regarding conducting standardized clinical evaluations, collecting the common clinical data points, using the common data collection forms, and inputting the data into the UDS’s relational database. The Clinical Task Force and the NACC developed detailed user manuals for these activities which they made available to the ADCs on their website. While most of these deliberations were conducted virtually, there were discussions regarding implementation and revisions with the relevant Core Directors at the semi-annual Director’s Meetings. The NACC also created an online bulletin board that the ADCs could use to submit questions and get answers on data collection issues. In the early days of the UDS implementation, there was considerable traffic on the bulletin board about how to deal with various scenarios. If necessary, the NACC forwarded the question to the Clinical Task Force for a decision. Because they have been collecting UDS data for several years, the bulletin board is relatively quiet; that is, until the NACC introduces minor modifications to the UDS. The number of questions posted on the online bulletin board is contingent on the complexity of the modifications. We were informed in our interviews that the decisions to make minor modifications to the UDS are made centrally by the NACC and the Clinical Task Force and then communicated by group emails to the ADCs. However, the group emails may not be sent to all parties impacted by the decision. For example, if it is a clinical issue, data core staff may not be copied on the changes. This may lead to downstream problems when the data manager attempts to enter the modified data formats. Attempting to implement and maintain a uniform data set across 29 geographically dispersed, stand-alone ADCs is challenging. More specifically, while the Clinical Task Force and the NACC standardized the outputs with the common data forms, they have not standardized the local processes. The lack of standardization in how different types of expertise are integrated and deployed across the ADCs was an issue that surfaced during our interviews. Each ADC has its own unique clinical processes for assessing and diagnosing AD and collecting and managing the data. The data is collected and analyzed at the ADCs by multidisciplinary groups of people: clinicians, neurologists, statisticians, data managers, neuropsychologists, nurses and research staff. The Centers have differing professional categories and responsibilities for their staff. For example, some centers include the statisticians and data managers in the team and others view them as support staff only leading to varying degrees of involvement in deciding how best to code the items for the database. One statistician described their role this way: “I’m the outsider looking in, I guess.” By comparison, the data manager at another ADC meets with the clinical staff twice a week to review patient charts from which the data to be submitted to the UDS is generated.

Table 3. Major Ongoing Deliberations to Maintain the UDS

Topics and Issues

Participants

Forums and Degree

of Virtuality

Values Divergence/ Knowledge Barriers

Coordinating Mechanisms

Implement UDS and major revisions to the UDS

NACC staff, Clinical Task Force, ADC clinical core and data core staff

Online users manuals and bulletin board for questions, emails, and F2F updates at Directors meetings

Lack of knowledge due to new or updated standards

Well-defined procedures, online manuals, F2F updates and online

Implement minor modifications to the UDS

NACC staff, Clinical Task Force, ADC clinical core and data core staff

Emails and online bulletin board

Failure to share knowledge with all relevant parties

NACC specifies requirements

Consistency of diagnosis, accuracy and completeness of data forms

ADC clinical core staff and data core staff

Email, F2F, hard copies of the data forms

Lack of common frame of reference due to diversity of disciplines and ADC-specific processes

NACC provides specific requirements

Quality and timeliness of data submitted to ADC by ADCs

ADC staff (data mgmt. staff, clinical staff, Core and Center Directors), NACC staff and PI, and NIA PM

Primarily virtual (EDI, monthly reports, emails, and telephone if an issue needs to be escalated

Familiarity of staff with data forms; Failure to share data/knowledge in timely fashion due to differing priorities

NACC uses error tracking procedures; ADC’s commitment to data integrity

While decisions about team inclusion are made in each center, the NACC and UDS might benefit from some analysis of the tasks that are vital to ensuring consistency of the data set. In this situation, statisticians and data managers make decisions about the definitions that are core to interpretation of the data. Also, the degree of co-location differs from center to center. At some ADCs, staff members are co-located while at other ADCs they are in separate departments that are housed in different buildings and located on other areas of the campus. The degree of geographic dispersion affected both the amount of interaction and the modes of interaction. These factors impact the reliability of the UDS. To mitigate these risks to the UDS, both the ADCs and the NACC have evolved a process that they follow each month to ensure data quality. The data core staff at the ADCs conduct quality checks on their data before they submit it via electronic data interchange to the NACC each month. Once they receive the data from the ADC, NACC’s data managers and programmers conduct a standardized set of quality checks and then distribute a report to the Center Directors, Clinical Core Directors, and the Data Core Director. These reports will typically be shared with other Center staff such as the data core manager, clinical core leader, scheduling assistants, and so on. If the NACC discovers any data discrepancies, they are listed in these reports to the ADCs. ADC staff will compare the data

they submitted with what the NACC reports and then email questions to the programmers and data managers at NACC. The NACC reports also include questions regarding what they consider to be inconsistencies in the data. One example involved how a Center’s clinicians classified subjects as depressed. When asked if a subject exhibited depressive symptoms, the clinicians said yes. When asked in a later question if the patient was depressed, the clinicians indicated that s/he was not. The data managers asked the clinicians for clarification and were told that the subjects had signs of depression, but they were not comfortable diagnosing the subject as clinically depressed because they did not know if the person had a history of clinically diagnosed depression. Here is how one of the interviewees described another virtual deliberation that their ADC had with the NACC (Personal interview, 2011): “We have been back and forth with them (NACC) on how they clarify follow-up rates and how they count who is being followed in what time window. The time window is where we really, really struggled with how they ended up with different rates reported from how we thought they should be. But ultimately their decision is the one we have to follow.” The issue of follow-up rates was particularly relevant because the ADCs must report a specific number of patients enrolled in their Center’s data set each month. Failure to do so signals potential problems and is an important consideration in future funding. There were probably several reasons that an ADC might be tardy in subject follow-up or timeliness of their data submission: the disagreement on the calculation of follow up rates, the amount of effort required to be fully up-to-date, and conflicting priorities for how they wanted to spend their time. The original deliberations occurred initially with email requests from the NACC to the ADCs. If the ADCs failed to comply or were slow to comply, the deliberations escalated to telephone calls from the NACC’s Director to the Center’s Director. If this did not produce the desired results, it would be followed by telephone calls from the NIA’s Program Manager. Given the threat of losing center funding, subject follow-up and timeliness of data submissions are less problematic now than when the UDS was originally implemented. DISCUSSION AND CONCLUSIONS This case provides illustrative insights into the complexity of deliberations involved in virtual R&D projects. It is generally acknowledged that the UDS, and by extension the deliberations involved in

creating it, implementing it, maintaining it and revising it, have been successful. An August 2010, New York Times article reveals that while scientists may have had initial concerns about such a collaborative effort, they have started to recognize the benefits. One researcher quoted in the article noted “it’s not science the way most of us have practiced it in our careers. But we all realized that we would never get biomarkers unless all of us parked our egos and intellectual property noses outside the door and agreed that all of our data would be public immediately.” This attitude has prevailed and the Alzheimer’s project has become a model for other disease states such as Parkinson’s disease.

Similarly, the following observation from our interviews attests to the success of the UDS (Personal interviews, 2011): “I think the metric of success is the extent to which the data we’re collecting is getting used and advancing scientific productivity. We look at how many different proposals have come forward to use the data and how many publications have come out that have used the data inside of the grant. I think that’s the main reason we’re doing this. It is not just collecting data for collecting data sake, but actually to answer the questions. The first couple of years were just about getting the data collected in a format that would be useable, and now we’re beginning to see the fruits of that labor, if you will.” Coordinating Mechanisms The effectiveness of the UDS deliberations is due in part to the coordinating mechanisms that provided the context for the key stakeholders to make the important trade-offs on a reasoned and ongoing basis. The theory of organizational information processing and mutual meaning making (Galbraith, 1974; Boland and Tenkasi, 1995; Malhotra & Majchrzak, 2004; Weick, 1995, 1979; Daft & Lengel, 1986) suggests that the structural mechanisms for coordination must provide the means to address the level of task uncertainty (what) and the complexity of achieving the task (how) in order to effectively co-create knowledge (Nonaka, 1994; Boland and Tenkasi, 1995). Sabherwal (2003) condensed several typologies of coordinating mechanisms into a continuum of four main types: informal mutual adjustment, formal mutual adjustment, plans, and standards. Figure 5 superimposes these four coordinating mechanisms onto the R&D continuum in light of the levels of task complexity and uncertainty. The key stakeholders in the AD research network generally knew what to do to create, implement, and subsequently revise the UDS. How to do it, especially how to build support for the UDS given the divergent values and priorities of the 29 geographically dispersed ADCs, was more challenging and complicated (stages D1 and D2 on the R&D continuum). The coordinating mechanisms for these deliberations relied heavily on formal mutual adjustment and to a lesser, but no less important, degree on informal mutual agreement. The key structural elements of the formal mutual adjustment included the national coordinating structure (NACC), the ADC steering committees, the semi-annual Directors meetings, the Clinical Task Force and its sub-committees, the AD center model, and the NIA. Arguably the most powerful coordinating mechanism has been the compelling mission that is shared by the entire AD research community: eradicating Alzheimer’s disease. This underscores both the informal and informal coordinating mechanisms and is the basis for coordination of efforts requiring shared goals and shared plans.

Figure 5. Coordinating Mechanisms Across the R&D Continuum

The NACC has played a particularly important structural role in the formal mutual adjustment involved in creating the UDS. The NACC has provided the requisite infrastructure for effective and efficient deliberations within this virtual inter-organizational domain. In so doing, it serves the key functions of a referent organization, which Trist (1983) identified as the coordination of relationships and activities, appreciation of emergent trends and issues, and infrastructure support including providing resources, sharing information, and conducting special projects. As mentioned above, the NACC organizes the semi-annual, F2F meetings of the ADC core directors and ADC steering committees. It provides funding to the ADCs to offset the cost of submitting their data to the UDS, it funds collaborative research projects, and it assists scientists seeking to use the UDS in their research. The distinctions between informal and formal mutual adjustment are not so much binary, as incremental or additive, in that the necessary conditions for effective informal mutual adjustment such as trust and respect, also provide a positive foundation for more formal mechanisms of mutual adjustment. The foundation for informal mutual adjustment among the ADCs and the NACC was a dense network of collegial relationships among participants in the AD research network that had developed over several years of interacting with each other at conferences and symposiums, in professional organizations on editorial boards, and on review panels for the NIA and other funding agencies. In addition, many of the ADC core directors and center directors serve on the Advisory Boards of two to three other ADCs, much like the interlocking directorates in the private sector. This level of F2F interaction has certainly served to reduce some of the complexity in the working relationships that might otherwise have adversely affected the efficacy of the deliberations in this virtual R&D project; that is, the members of the teams tasked with creating

the UDS were familiar with each other and had already developed socially derived norms of interaction. Further, the participants who were elected or selected to serve on the formal coordinating mechanisms – the Clinical Task Force the working sub-committees, and the ADC Steering Committees are generally highly regarded members of the AD research network who possessed considerable source credibility based on their acknowledged expertise and perceived trustworthiness (Gilbert et al, 1998). An interviewee described the qualities of these individuals as follows:

The task forces were comprised of people that were well recognized as being leaders and reasonable people. I think that those are the two characteristics of the people on the task forces; they’re not only leaders, they’re politically astute. They’re open. They’re “just reasonable” is the best way to describe them. They’re not difficult to deal with. Because they commanded confidence from the various other members that were involved in contributing data, they’ve been successful in driving each other forward.

Another informal and formal mechanism for mutual adjustment was the presence of several very effective network leaders and boundary spanners from the NIA, the NACC and the ADCs. These network leaders or boundary spanners serve a ‘reticulist’ (Friend et al., 1974) role that has been critical to the creation and implementation of the UDS from vision to inception. Alter and Hage (1993) describe reticulists as “individuals who engage in networking tasks and employ methods of co-ordination and task integration across organizational boundaries” (p. 46). Similarly, Webb (1991) describes reticulists as “individuals who are especially sensitive to and skilled in bridging interests, professions and organizations” (p. 231). Both are apt descriptions of the key network leaders in the AD research network who have provided an effective balance of diplomacy, negotiation, persuasion (Rhodes, 1999), and when necessary confrontation to ensure the success of this project and the AD research network. Once the UDS had been established, the NACC and the ADCs generally knew what to do and how to do it both conceptually and operationally (stages D3 and D4 on the R&D continuum, Figure 4). The tasks and deliberations required to maintain the UDS have become more routinized and are less uncertain and less complex. These deliberations are coordinated with plans and standards including specific requirements, delivery schedules for data submissions, and financial incentives. In fact, coordination by standards is the fundamental premise of the UDS with its standardized clinical evaluation protocol for assessing and diagnosing AD and other dementias, common data points, standardized data collection forms with detailed coding manuals, and error tracking procedures. And with regards to the UDS, the relationship between the NACC and the ADCs, while still collegial, has generally become more hierarchical and prescriptive. As the interviewee stated above: “… ultimately their decision is the one we have to follow.” The findings from this case study on coordinating mechanisms support Sabherwal’s (2003) assertion that more informal, communications-oriented coordinating mechanisms are suitable when uncertainty is greater, and that the more impersonal and formal, control-oriented

mechanisms are a better fit when the level of task uncertainty is lower. Virtuality and Communication The nature of communication, whether F2F or technologically mediated and the choice of forums for the deliberations, were also impacted by the levels of task complexity and uncertainty. The deliberations to create, implement and revise the UDS relied on high levels of input and participation directly for the members of the Task Force and the sub-committees and for others via surveys, bulletin boards, and direct feedback to the Clinical Task Force and its sub-committees. These discretionary coalitions used emails, teleconferences, and occasionally videoconferences for their deliberations. However, when they had a particularly difficult decision to make they usually made these in F2F meetings. As a member of one of the working groups stated: “One of the things I demanded in my job on the toolbox has been face-to-face meetings of my team at regular intervals. We’ve been doing everything by email and phone conferences, but there is nothing like a face-to-face meeting.” It is also interesting to note that even though there were several information and communication technology (ICT)-mediated vehicles for providing input, the ADC Directors often used the public forums in their semi-annual meetings to raise issues or share their concerns.

The communication involved in the ongoing deliberations to maintain the UDS are predominantly technologically mediated: that is, the data is submitted electronically to the relational data base; questions for clarification are posed and answered on an online bulletin board; and discussions between the ADCs and the NACC regarding submitted data are conducted almost exclusively by email and only in exceptional cases by telephone. Even the day-to-day operations of the ADCs rely heavily on information and communication technology. Here is how an interviewee described communications within their ADC (Personal interview, 2011): Communications are partly face-to-face. They’re partly by email. They’re partly by a system that my data management group created, which is online. It’s partly by electronic reports that are also up on the Web. They’re generated each evening. And (in our center) they’re partly through an online data management request system that we built, where you couple communications with data management. In sum, the routine deliberations characteristic of the D3 and D4 stages in the R&D continuum were mostly virtual and technologically mediated. In comparison, the deliberation topics that were more uncertain and complex (at the D1 and D2 stages in the R&D continuum), especially those that involved important and sensitive trade-off decisions, tended to be addressed in F2F interactions. Culture of Collaboration The NIA has been very explicit in its intention to foster an environment of collaboration across multiple disciplines and the network of ADCs. Tony Phelps, NIA Program Director, described the NIA’s vision for an ethos of collaboration on the 20th anniversary of the ADCs as follows:

When the ADC program was created, NIA’s leaders hoped that an environment of cooperation would stimulate AD researchers to seek new pathways to scientific discovery and to share their findings. The program has done just that. In the collaborative atmosphere of the centers, specialists in biomedical, behavioral, pathological, and clinical science are studying the causes, and possible prevention of AD, and developing new lines of multidisciplinary research. As scientists at the centers uncover the complexities of dementia, they spark a certain friendly competition with other research scientists throughout the world. Their brilliance and enthusiasm creates an excellent training ground for up and coming investigators, and inspires others to devote their energies to AD research (National Institute of Aging website, 2005)

The following comment from our interviews provide anecdotal evidence that the NIA’s vision of cooperation and ‘friendly competition’ is being realized (Personal interview, 2011):

I think we’re beginning to see young investigators and more established investigators getting grants to do secondary data analysis, using the information, planning GWAS studies, hooking up to other investigators within the group who have large data sets that they’ve contributed, looking to see if they have other data that’s not necessarily part of the UDS, but would help to augment analyses that they’re doing. I think we’re seeing a lot of that now, and that will definitely be the metric for success going forward. It’s becoming more and more important.

This researcher’s comment also captures the balance of competition and cooperation that now characterizes the ADC network:

I would say cooperation is really the key behind what we’re doing right now in terms of the UDS. And I think because we’re all part of the same project, it doesn’t make any sense to be competing with one another because we all have the same common goal whether we like it or not. We’ve got to collect the data for this protocol. Now, we may compete in other ways in terms of other aspects of scientific research, but for this protocol and for this project, we’re all on the same page. So we all have to work together for a common good. I think it’s that spirit that we’re all one family for this project, so we have got to figure out how we’re going to work together.

Building and maintaining the UDS and this collaborative network of researchers has been a social task as well as one of technical or professional alignment. Admittedly there have been pressures that would normally work against cooperative effort such as competition for funding and recognition such as the Nobel Prize for Medicine, ego, professional jealousy and such. It appears that these effects have been mitigated in large part by appealing to the compelling mission and genuinely shared goal of eradicating AD, as well incentives such as continued funding for one’s center. The UDS has played a significant role in fostering and extending the culture of collaboration in the AD research network. It has also played a role in other major collaborative research

initiatives both in the United States and now globally, including the Dominantly Inherited Alzheimer Network, the Alzheimer’s Disease Genetics Consortium Genome Wide Association Study, the Alzheimer’s Disease Neuro-Imaging Initiative (ADNI), and now, World Wide ADNI. The UDS is a brilliant example of the benefits to be gained from virtual collaboration across research sites and disciplines. Together, all of these contributors have created a powerful instrument to help accelerate the identification of new treatments, preventions and ultimately, a cure for this insidious disease.

References (to be completed)

Alzheimer's Disease International. World Alzheimer Report 2009. www.alz.co.uk/research/world-report

Asaro, F. 2011. Universal Co-opetition: Nature's Fusion of Competition and Cooperation. Betty Youngs Books.

Brandenburger, A. and Nalebuff, B., 1997. Co-Opetition: A Revolution Mindset That Combines Competition and Cooperation, New York: Doubleday Currency

Dominantly Inherited Alzheimer Network, http://www.dian-info.org/

Evans, D.A., Funkenstein, H.H., Albert, M.S., Scherr, P.A., Cook, N.R., Chown, M.J., Hebert, L.E., Hennekens, C.H., and Taylor, J.O. 1989. Prevalence of Alzheimer's disease in a community population of older persons. Higher than previously reported. JAMA. Nov 10; 262(18): 2551–2556.

Friend, J., Power, J., and Yewlett, C. 1974. Public Planning: The Intercorporate Dimension, London, Tavistock.

Gilbert, D.T., Fiske, S.T., Lindzey, Gardner, eds. (1998). The Handbook of Social Psychology. Oxford University Press.

Kolata, G., 2010, Sharing of Data Leads to Progress on Alzheimer’s, August 12, 2010, New York Times.

Mashey, J. (2008, December 3). What can we learn from Bell Labs about managing energy R&D? Retrieved from http://bravenewclimate.com/2008/12/02/hansen-to-obama-pt-iv-where-to-from-here/#comment-3023.

National Institute of Aging, 2013, Alzheimer’s Disease Research Centers. Retrieved from http://www.nia.nih.gov/alzheimers/alzheimers-disease-research-centers

National Institute of Aging, 2013. Retrieved from http://www.nia.nih.gov/alzheimers/about-adear-center

Purser, R.E, Pasmore, W.A., Tenkasi, R.V., 1992. The Influence of Deliberations on learning in new product development teams, Journal of Engineering and Technology Management, 9, pp. 1-28.

Revkin, A. (2007, December 12), ‘R2-D2’ and other lessons from Bell Labs. New York Times. Retrieved from http://dotearth.blogs.nytimes.com/2008/12/12/r2-d2-and-other-lessons-from-bell-labs/.

2013 www.alz.washington.edu WEB NACCsum.pdf

Alzheimer’s Association re WW Adni http://www.alz.org/research/funding/partnerships/ww-adni_overview.asp

2010 University of Pennsylvania School of Medicine http://alois.med.upenn.edu/adgc/

Williams, P. 2010. Special agents: The nature and role of boundary spanners. Paper presented at the ESRC Research Seminar Series: Collaborative Futures: New Insights from Intra and Inter-Sectoral Collaborations, University of Birmingham.

Appendix A - Methodology

We incorporated the diagnostic steps of open sociotechnical systems design to gather data and analyze the impact of virtuality on the AD research network: conduct an initial scan of the system, map the system, analyze the technical subsystem, and analyze the social subsystem (Pava, 1983). We did this in two stages. First, we conducted an initial scan of the system, the AD research network, and used this preliminary information to map the system and choose a specific project to investigate in greater depth. We then conducted a series of interviews to analyze the technical and social subsystems. Conducting an Initial Scan and Mapping the System The purpose of an initial scan is to discern the mission or goals of the system and the governance processes and coordination mechanisms that enable collaboration in pursuit of the mission. The mission and governance system provide the impetus for a self-regulating system of players who define and iteratively evolve the technical subsystem in terms of the key deliberations or issues they need to address in order to achieve the mission. We conducted the initial scan of the AD research system/network by reviewing available documentation on the major research initiatives and key stakeholders, by attending and observing the Fall 2010 ADC Directors meeting in San Francisco, and by interviewing the Principal Investigator of the National Alzheimer’s Coordinating Center (NACC) and the NIA’s Program Director for the Alzheimer’s Disease Centers. This afforded us a deeper understanding of the mission of the AD research network and the emergence of the ADCs, the NIA-funded centers, and other major research programs, initiatives and projects such as the UDS, ADNI, GWAS, NCRAD, and DIAN. Analyzing the Technical Subsystem

Pava (1983) described deliberations as “equivocality reducing events”, i.e. choice points that are critical to work systems involving knowledge generation and knowledge utilization. From this general description, Purser (1990) defined deliberations in product development as “social interactions in which knowledge is exchanged to define or solve a problem, make a decision, or implement a solution”. However, deliberations are not simply the equivalent of decisions or meetings; rather, they are sense-making exchanges (Weick, 1995), communications and reflections in which people engage to reduce the equivocality of a problematic issue. A deliberation is identified by the existence of an equivocal topic, which is explored in different types of forums, involving a particular group of participants who contribute important information or take-away important information. Deliberation analysis assesses the values and perspectives of participants within forums, and “the interpretative dynamics among interdependent parties who must forge a discretionary coalition” (Pava, 1983) to make intelligent trade-offs from their respective values, priorities, and cognitive orientations (Tenkasi, 1994; 2000).

Deliberations connect people together to meaningful issues much like linear work processes connect people together to a common outcome. For people who work virtually and may never see one another as they work together, this connection to an issue may be vital to their performance.

Deliberations in knowledge work such as R&D can be viewed in terms of intellectual bandwidth (Nunamaker et al. 2001, 2002, Qureshi et al. 2002) and the ability to mobilize intellectual assets in deliberations to create value. This model provides a framework for measuring the extent to which an organization can create value from its intellectual assets by looking at two key elements in deliberations. The first is the process of understanding the data and available information and translating it into knowledge. The second addresses the interdependence of efforts and whether it is primarily an individual work mode, a collected work mode and the sum of individual work, a coordinated work mode in which there is sequential interdependence or a concerted work mode in which all work in concert to produce joint deliverables.

Analyzing the Social Subsystem The social system is defined in terms of discretionary coalitions that are needed to conduct the deliberations effectively. These coalitions make the important trade-offs in creative work that are made necessary by the presence of useful but inherently divergent values and perspectives. For example, in traditional research environments, scientists typically compete against one another for limited grant money and to publish articles in top journals, neither of which enable the effective functioning of coalitions in a virtual project. The social system design does not try to eliminate differences, but to create a mutual understanding and a common orientation such that trade-offs can be settled on an intelligent and ongoing basis. Coalitions are to nonlinear work what work groups or teams are to more routine work. Roles and responsibilities can be defined for the parties involved in the coalitions as well as other changes in the coordinating mechanisms in a way that supports and rewards the sort of integrative perspective necessary to successful coalition functioning.

In classic sociotechnical systems analysis, the focus is on addressing variances in work processes and performance. However, in the knowledge work environment of R&D projects (co-located and virtual), variances manifest as knowledge barriers. Purser et al. (1992) identified four main categories of “barriers” obstructing and delaying collaborative knowledge development: The first was the failure to share knowledge due to lack of cooperation, missing parties, or unrealistic timeframes; the second was the lack of a common frame of reference including cognitive frame of reference barriers associated with differences in language, values, and functional expertise; third was the lack of knowledge about the work, procedures and processes, or the capabilities of virtual participants that can slow or derail progress; and the fourth was failure to utilize knowledge. In order to develop an interview frame, we performed an extensive literature review on deliberations in virtual R&D organizations. We incorporated interview questions that would be used in a sociotechnical scan of the technical subsystem in non-routine knowledge work. This included listing the major deliberations, identifying the different forums in which these

deliberations were conducted (in person, videoconferences, teleconferences, email, etc.), determining the structures of the forums (structured, semi-structured, and unstructured), identifying the participants in these key deliberations, and surfacing the variances or knowledge barriers in the deliberations. To analyze the social subsystem within which the major deliberations occurred, we developed questions that addressed issues such as the role of discretionary coalitions, key influencers or participants, means of building support for decisions, and how they managed conflicting objectives or perspectives. Since the structure of the UDS had already been established and many of the early implementation challenges had been addressed, we conducted retrospective interviews and secondary data searches to understand the development process, the scope of involvement of various personnel, and the kinds of decisions and actions taken to reach consensus on the UDS.V1. However, at the time we conducted our interviews, the NACC and the Clinical Task Force were in the process of revamping the batteries of instruments and measures for the UDS.V3 and we were able to gather contemporaneous insights on how they were conducting this task in a virtual environment. We also interviewed ADC researchers about current use, decisions, and applications of the UDS to gain a perspective on how deliberations related to it have changed over time. We followed a protocol established by the VOSS research team, but customized the questions for the ADC interviews in light of our discussions with the NACC PI and NIA’s Program Director, the insights gathered at the ADC Directors semi-annual meeting, and our preliminary mapping of the ADRN system. We also asked for their input on selecting a representative sample of individuals who have been involved in the key deliberations related to the UDS. A total of 12 telephone interviews were conducted during the winter and spring of 2010-2011 with a cross-section of ADC center directors, principal investigators, clinical core directors, neurologists, psychiatrists, biostatisticians and other members of the research teams. The interviews yielded considerable data about the technical and social subsystems. The interviews were transcribed and members of the research team analyzed the content to identify common themes regarding the major deliberations, the various forums in which they occurred, the role and relative efficacy of virtual or technology-mediated forums and face-to-face meetings, the sources of variances or knowledge barriers, and the coordinating mechanisms used to address them.