This volume reflects a range of perspectives on intercollegiate athletics. An informed and campus-wise institutional research function is critical in the development and interpretation of student-athlete academic information.

Institutional Re search : Offense, Defense, Coach, or Referee Richard D. Howard

For over 100 years, intercollegiate athletics have represented the most visible program in higher education: student-athletes and coaches are virtually the only members of our colleges and universities recognized outside of the academy. As Knapp points out in Chapter Six, “No other program of a university is subject to closer scrutiny from a wider set of interested constit- uencies, and no other program is more likely to fall prey to greater misinfor- mation and emotional distortion of fact.” This observation, in all likelihood, has been true for most of the past century. What has changed is that the focus outside the academy has expanded to include not only the outcomes of athletic contests but also the academic progress and success (or failure) of the student-athlete. The relevance of Knapp’s statement to the institutional research professional is that society’s demands for documentation about student-athletes’ academic programs are the platform from which colleges and universities are being challenged to demonstrate the integrity of their academic, as well as athletic, programs and leadership.

This is illustrated in Bradley’s discussion of the Student Right-to- Know legislation. Originally, his bill focused on the reporting of student- athlete data, but i t evolved into mandates for reporting comprehensive graduation statistics about the entire student body as well as selected sub- groups of students and for reporting campus crime statistics.

The development and interpretation of statistics describing the out- comes of athletics events are cut and dried when compared to describing (or measuring) the outcomes of academic programs. The definitions of “winning and losing” in academic programs are neither absolute nor con- sistent. As Mallette illustrates, something as seemingly “straightforward” as

NEW DIRELTIWS FOR I’.sTITITIO\AL RESEARCH. no 74 Summer 1992 @Jossey Bars Publishers 85

86 MONITORING AND ASSESSING INTERCOLLEG~ATE ATHLETICS

the calculation of graduation rates or how student-athlete is defined can significantly influence the perceptions of academic success.

For the institutional research professional whose role is to accurately and consistently report academic progress, the absence of any single or accepted set of decision rules and methodologies for comparing and report- ing student-athletes’ and all students’ academic progress is a serious prob- lem, particularly as it relates to the calculation of graduation rates. It is not always clear which methodology or definitions are “correct.” While the definitions for reporting to athletics governing bodies have evolved in re- cent years and are slowly becoming somewhat more standardized, in general it has been the perspective and agenda of the individual using the data or information that have dictated how the statistics are developed, reported, and interpreted.

Diverse Perspectives

From an institutional standpoint, two categories of constituencies demand student-athlete data and information. External constituencies, under the guise of informing and, presumably, protecting the public, are usually inter- ested in data about individuals. However, because of federal privacy laws, the “truth must be presented in the form of summary information. This protects the rights of the individual student-athlete and allows conclusions to be drawn about the “success” of specific groups of student-athletes and the institution in meeting academic goals. Internal constituencies also require these statistics to support many of the same evaluation goals of external constituencies. In addition, data and information about specific student- athletes are essential to support the educational, academic management, and leadership responsibilities of the institution’s administration, faculty, and coaches.

In this volume, student-athlete academic information needs are dis- cussed by six individuals with different perspectives on its use. Two of the authors address concerns external to the institution, and four reflect differ- ent institutional management support needs. While not exhaustive, the concerns and pressures that drive demands for student-athlete academic information and data are reflected in these discussions. A common theme of these authors is a desire for the “truth” in order to protect either the academic credibility of the institution and its students or the public.

In the chapters by Bradley and Lederman, the focus of the information needs is external to the institution. While from different points of view, their common perspective is that of informing society: both authors argue that society has the right to be informed. From a legislative perspective, Bradley presents the case that students and parents need campus-specific information to make an informed decision when choosing a college or university to attend. And members of Congress agreed with him, passing

INSTITUTIONAL RESEARCH 87

legislation requiring annual, detailed reporting of graduation rates by all colleges and universities. At this time, however, several questions remain about the final regulations concerning the collection and use of the data. Among these are the following: ( 1) What methodological procedures will be used to calculate graduation rates? (2) How will student-athlete and student be defined? ( 3 ) In what format will the data be released? (4) What will be the relationship of this report to the reports required by athletics governing bodies?

The press, on the other hand, according to Lederman, sees its role as informing the body politic and thus holding institutions “accountable.” In the case of the media, questions usually focus on a specific college or university or a specific student-athlete. The quality and level of detail of the responses will be limited by legal restrictions and the ability (or inability) or the institution to provide the requested data. Often the relationship between the media and the institution turns antagonistic if the institution will not or cannot provide the information requested. The resulting head- lines often reflect an inaccurate conclusion, drawn from incomplete infor- mation or rumor that casts the institution in a negative light. As Lederman points out, the institution may prevent this by making available to the media the types of information outlined in his chapter. When dealing with the press or other forms of the media, it is critical that the institutional research officer understand campus policies and procedures for the release of data and information to external constituencies and the legal implica- tions this release may have. If your institution does not have such policies and procedures, they should be created. It is easy to get caught up in the excitement of the moment when a well-known local or national celebrity calls personally for some insights about athletics on one’s campus.

In Chapters Four through Six, the chief executive officers (CEOs) of three campuses discuss their responsibilities to their institutions and athlet- ics programs as well as the data and information needed to monitor these programs at the institutional level. The perspectives of these three campus leaders reflect three very different types of institutions and institutional missions, and while each CEO has his own specific information needs, all three acknowledge that:

1 Competition is intense, and winning is important to their students and the institution.

2. The academic progress of the student-athlete is the primary concern of the institution.

3. There is a place for strong athletic programs on their campuses.

These three chapters provide important insights about the “unique” information that the CEO must have to effectively manage and monitor athletics programs and respond to questions raised by faculty, the public,

88 MONITORING AND ASSESSING INTERCOLLEGIATE ATHLETICS

and various regulatory or governing agencies. While traditions and local environments of the three campuses impose different expectations on their athletics programs, it would appear that much of the specific student- athlete academic information required to support each of these CEOs is the same as that usually found on a student's transcript-for example, grade-point averages, number of courses passed, progress to degree.

On the campus, maintaining the integrity of the athletic-academic relationship is operationally the responsibility of the faculty and typically falls to a faculty committee like that described by Cooper in Chapter Seven. The information and data needs of the faculty committee are similar to those of the CEO, but they include more detail and require a much more structured format and schedule. Accurate and timely information on an individual student-athlete is critical, since the faculty athletics committee is often asked to make judgments affecting the athletic and academic careers of students.

For internal management support or external reporting, i t appears that most of the student-athlete academic information can and should be generated from a common set of data elements. The difference in these user requirements, in general, is in the level of aggregation and frequency of the data and information. This does not, however, make the task of collecting the data and developing the information any easier. It does, however, help to identify the particular set of data elements that have to be collected and who should be responsible for their collection, storage, and analysis. Remember that not all perspectives have been offered in this volume; voices omitted include those of the National Collegiate Athletic Association (NCAA), athletic directors, or athletic conference offices (who, to the surprise and frustration of the editors, declined to contribute to this volume).

Role of Institutional Research

Many institutions have found that meeting the many demands for data and information regarding their athletics program is no longer an appropriate responsibility for the athletics department and is beyond reasonable expec- tations of the registrar's or admissions office. It has grown to be an institu- tional activity requiring the coordination of many units across the campus. Often, the institutional research office is charged with the responsibility for identifying the appropriate data and its sources, coordinating the col- lection of data through a number of institutional and departmental filing systems (some electronic and some paper and pencil), and compiling the reports. The case studies presented by Smith and Moss illustrate the com- plexity of this effort. It requires technical understanding of different pro- cesses and systems used to collect and store data. It also requires an institutional knowledge of the relationships between data collected by var-

INSTITUTIONAL RESEARCH 89

ious units and the academic and athletics programs and demands political and interpersonal skills. The case studies illustrate that, while the specific strategies developed to support the collection and reporting of student- athlete data and information will be institution specific, cooperation and coordination across the campus are critical to developing and maintaining reliable and accurate student-athlete academic data.

I have indicated several times that it appears that the student-athlete academic data needs identified by the authors are essentially the same, varying only in level of detail and frequency. Yet the case studies reflect institution-specific solutions to meeting the collection, reporting, and man- agement information requirements. In fact, there are two types of data that must be integrated to create the needed information: (1) academic data, and (2) athletic participation data. As pointed out by Smith and Moss, the athletics-related data were frequently found in stand-alone systems, usually within the athletics departments. The biggest challenge was finding the appropriate athletics data and converting them to a format compatible with the systems that stored academic data. In both case studies, most of the development time was spent coordinating, cajoling, threatening, or persuading individuals and offices to institute procedures that resulted in the capture and storage of the appropriate data.

The second issue is that of tracking academic data. Computerized retention and attrition systems are a relatively new tool for analyzing the academic progress of students. Academic indicators are the same for all students, athletes or nonathletes. The trick is to build into the retention system the flexibility to study specific groups of students (as Moss de- scribes), as defined by different demographic, academic, or athletic char- acteristics (Howard and Rogers, 1991). This is important, since the key evaluation parameter for academic success is usually not an absolute mea- sure but one that must be suitable for comparison with the same measures calculated for the student body as a whole or for a specifically defined comparison group.

While cooperative procedures should result in the collection and main- tenance of reliable data, it does not necessarily follow that the resulting information will be valid. As Mallette clearly illustrates, without clarification of definitions and documentation of methodologies, the real meaning of statistics or information produced about the academic progress of student- athletes will always be in doubt and, therefore, open to question. This is true for any empirical study. A significant contribution of the institutional research function is h e ongoing effort to clarify and standardize definitions and document methodological procedures for all parties concerned. To ignore or downplay this activity will eventually catch up with the institu- tion, casting doubt on the integrity of its leadership, coaches, academic and athletics programs, and usually the institutional research function.

In line with these activities on the campus, there is a parallel need at

90 MONITORING AND &SESSING INTERCOLLEGIATE ATHLETICS

the national and regional levels. Recognizing this, the Association for Insti- tutional Research in 1989 formally constituted the Data Advisory Committee (DAC). Members of the DAC, or their representatives, serve in an advisory function to several federal and national organizations that routinely survey colleges and universities. While ill-conceived surveys continue to plague our institutions, particularly in the area of academic performance, members of the committee have been instrumental in the efforts of the National Center for Educational Statistics (NCES), the NCAA, and several other agen- cies and organizations, to collect reliable and valid information. As is true on campus, data collection efforts will only be successful when conceptual clarity and technological competence guide the process.

The Future

In the short term, questions of accountability will be commonplace for institutions of higher education. This is reflected in the widespread accep- tance of Total Quality Management (TQM) strategies, where customer sat- isfaction is central to the goals and management of the organization. Universities and colleges, like other organizations, risk the loss of their standing in society if they do not develop procedures and measures to evaluate the satisfaction of their customers. Already, Congress, the NCAA, and other external governing agencies have defined what those measures are and initiated their own forms of monitoring institutional productivity and academic integrity.

Winstead discusses the implementation and implications of accredit- ation standards for certification of athletics programs. He proposes that athletics departments adopt traditional regional and professional accredit- ation procedures centering around a self-study. The process is the accepted standard in American higher education for monitoring academic and insti- tutional integrity and may help to eliminate rumor and misinformation that often surrounds the athletics enterprise. In particular, evaluation of the outcomes of the student-athletes' academic programs would address many of the questions asked by the public.

The contributors to this volume have identified specific information needs from their unique perspectives. At many institutions, the institutional research officer is key to the development of this information. Specifically, that individual's role is one of identifying and coordinating the collection of the appropriate data and creating valid information. It is important that this role not be relegated to a rote data processing function. As discussed, significant issues in the reporting of academic information have not been resolved, and standards and definitions will continue to change. In addi- tion, it is safe to assume that the visibility of intercollegiate athletics will remain high and that the demands for data and information will expand. In the final chapter, Callahan and Mallette have compiled an extensive list

INSTITUTIONAL RESEARCH 91

of resources and a bibliography designed to keep the institutional research officer and other campus officials current in the field of intercollegiate athletics. If you are responsible for developing and reporting student-athlete academic information, it is important to keep abreast of the issues, report- ing mandates, and the law.

A college or university’s intercollegiate athletics program is a visible and sensitive component of a campus about which pressures for data and information can become intense quickly. The institutional research profes- sional and other campus officials in leadership positions must be prepared to face this challenge knowledgeably and responsibly.

Athletics are a part of higher education and, for better or worse, are likely to remain so. Their value takes many forms. To the institution, an athletics program confers visibility and prestige and is a rallying point for alumni and friends. For the student-athlete, at a minimum, it provides recreation and an opportunity to compete at the highest levels; for others, it is a training ground for a potential professional career. The will to win is a part of any form of competition, whether it is to be the best academically, be awarded a grant, or win an athletic contest. The intense scrutiny of intercollegiate programs today may be the beginning of many different forms of accountability for higher education in the future. Higher education will have to develop the tools and identify appropriate measures to respond to challenge. On the campus, the institutional research personnel may have to play many roles: proactively position themselves (offense) to react to external forces (defense), make sure everybody gets the same message (coach), and objectively mediate among competing interests (referee).

Reference

Howard, R. D , and Rogers, B H. “Tracking Academic Progress Within a Complex Academic Environment.” In D. Hossler (ed.), Evaluating Student Recruitment and Retention Programs. New Directions for Institutional Research, no. 70 San Francisco: Jossey-Bass, 1991.

RICHARD D. HOWARD is director of institutional research at the University of Arizona. He has been president of the Southern Association for Institutional Research (I 983-84) and served on the executive committee of the Association for Institutional Research (1987-91).