The overall objective of this 3 year evaluation capacity building project with the AGEP leaders and departments are to:

Identify and collect quantitative indicators related to recruitment & retention & progression to the PhD in SBE fields Assess changes in SBE departmental practices Suggest qualitative indicators for use in your evaluations related to student skills development, departmental transformation, and Alliance impact Help SBE Chairs and faculty leaders better understand how to create an evaluation culture within the department related to graduate student affairs and broadening participation and success of URM

AAAS Evaluation Capacity Building Project with AGEP Leaders & SBE Chairs/Faculty Leaders

Quantitative Indicators

Recruitment

New or first time graduate students enrollees in Masters Programs and in PhD Programs

Retention

Cohort tracking of new URM enrollees in SBE, starting with 2006/2007 cohort

2 to 3 year benchmark – earned Masters or passed qualifying exams

Question

Should this data be collected for students who were in your URP/REU and entered graduate school at another insititution?

Suggested Qualitative Indicators

Assessments/Perceptions about Graduate Students Performance, Research Abilities, and Professional Behaviors

Performance assessment – course performance, qualifying examinations, and research

Faculty mentor perceptions about graduate student --- students’ academic strengths/weaknesses, understanding of the research process, analytical skills, critical thinking, science communications (writing and oral), professional behavior (research safety and ethics), interpersonal interactions, and overall progression to the PhD

Graduate student perceptions about their performance, research abilities, professional behaviors, and progression to PhD

Perceptions about Research Abilities

Conduct literature searches and reviews

Use databases

Formulate a research question

Develop a research plan, including methods, data to be collected

Use of equipment

Ability to analyze and display data

Ability to discuss experimental findings

Ability to write the research report

Perception about Professional Behavior

Class attendance & participation

On time arrivals

Research lab attendance & time spent in research lab

Departmental seminar attendance & participation & ability to ask appropriate questions

Attendance at departmental social functions

Ability to read and understand journal articles

Ability to prepare a poster abstract, poster, paper

Ability to deliver an oral presentation

Interpersonal interactions

Perceptions about Skills, Knowledge, & Environment

Understanding of science research safety and ethics

Use or research equipment

Knowledge about contemporary issues in your field

Understanding of how research impacts society

Critical thinking skills

Satisfaction with administrative issues (timely info about graduation requirements, adequate financial aid and timely disbursements)

Family Life/Work Balance (support from significant others)

Departmental Climate

Institutional Transformation Indicators

What are the sustainable changes at the administration level, in regards to SBE recruitment and progression to the PhD?

What are the sustainable changes in the SBE departmental admission/selection process?

What are the sustainable changes in SBE graduate school academic curriculum and student academic support?

What are sustainable changes as related to graduate student retention?

Psychology

801 PSYCHOLOGY, COMBINED

42.01 PSYCHOLOGY, GENERAL

802 PSYCHOLOGY, EXCEPT CLINICAL

42.03 COGNITIVE PSYCHOLOGY AND PSYCHOLINGUISTICS

42.04 COMMUNITY PSYCHOLOGY

42.07 DEVELOPMENTAL AND CHILD PSYCHOLOGY

42.08 EXPERIMENTAL PSYCHOLOGY

42.09 INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY

42.11 PHYSIOLOGICAL PSYCHOLOGY/PSYCHOBIOLOGY

42.16 SOCIAL PSYCHOLOGY

42.17 SCHOOL PSYCHOLOGY

42.99 PSYCHOLOGY, OTHER

51.2301 ART THERAPY

51.2303 HYPNOTHERAPY

51.2705 PSYCHOANALYSIS

803 CLINICAL PSYCHOLOGY

42.02 CLINICAL PSYCHOLOGY

Social Sciences

901 AGRICULTURAL ECONOMICS --01.0101 AGRICULTURAL BUSINESS AND MANAGEMENT, GENERAL, 01.0102 AGRICULTURAL BUSINESS/AGRIBUSINESS OPERATIONS, 01.0103 AGRICULTURAL ECONOMICS

902 ANTHROPOLOGY (CULTURAL AND SOCIAL) 45.02 ANTHROPOLOGY, 45.03 ARCHAEOLOGY

903 ECONOMICS -- 45.06 ECONOMICS, 52.06 BUSINESS/MANAGERIAL ECONOMICS

904 GEOGRAPHY-- 45.0701 GEOGRAPHY, 45.0702 CARTOGRAPHY

905 HISTORY AND PHILOSOPHY OF SCIENCE , 45.0804 HISTORY AND PHILOSOPHY OF SCIENCE

906 LINGUISTICS, 16.0102 LINGUISTICS

907 POLITICAL SCIENCE/PUBLIC ADMINISTRATION -- 44.04 PUBLIC ADMINISTRATION, 44.05 PUBLIC POLICY ANALYSIS, 44.99 PUBLIC ADMINISTRATION AND SERVICES, OTHER

45.09 INTERNATIONAL RELATIONS AND AFFAIRS

45.10 POLITICAL SCIENCE AND GOVERNMENT

Social Sciences continued

908 SOCIOLOGY -- 45.05 DEMOGRAPHY AND POPULATION STUDIES, 45.11 SOCIOLOGY

909 SOCIOLOGY/ANTHROPOLOGY

910 SOCIAL SCIENCES, N.E.C. (Not elsewhere classified)

04.03 CITY/URBAN, COMMUNITY, AND REGIONAL PLANNING

04.07 ARCHITECTURAL URBAN DESIGN AND PLANNING

05 AREA, ETHNIC, AND CULTURAL STUDIES

30.11 GERONTOLOGY

43.01 CRIMINAL JUSTICE AND CORRECTIONS

44.02 COMMUNITY ORGANIZATION, RESOURCES, AND SERVICES

45.01 SOCIAL SCIENCES, GENERAL (Not elsewhere classified)

45.04 CRIMINOLOGY

45.12 URBAN AFFAIRS/STUDIES

45.99 SOCIAL SCIENCES, OTHER

A short online survey of SBE departments at the participating AGEP institutions to determine practices related to graduate student recruitment, retention, academic support, and data collection and evaluation practices & uses.

Departments to be surveyed will be identified by SBE AGEPs.

During Years 1 & 3

Suggested Quantitative Data Collection Schedule

URM new graduate student enrollment data

For 2006/2007 -- December 2006

For 2007/2008 -- December 2007

For 2008/2009 – December 2008

Collection of data about 2006/2007 New Graduate Student Cohort

October 2009

Where are they now? Completed course work, earned Masters, passed qualifying

Suggested Qualitative Indicators

SBE Related Questions incorporated into AGEP Annual report

Questions to be determined with AGEP/SBE Program Directors with input from AGEP SBE community

Survey Schedule

August 2006?

August 2009?

The 2 day Alliance site visits will include meetings with AGEP PIs, PDs, staff, and evaluators, and SBE department Chairs and faculty. Alliances to be site visited will be selected with the NSF SBE and AGEP Program Directors. Site visits will be designed with Alliance Reps & Could include TA

Years 2 & 3 -- Site Visits to 2 AGEPs Each Year (October 2006 to 2008)

Evaluation Capacity Activities with SBE Faculty Chair/Leaders

Evaluation Capacity Building Workshop for SBE Faculty Chairs/Leaders in Washington DC In September 2006 (Combination focus group for AAAS and opportunity for cross Alliance chair/faculty exchanges)

Identifying research gaps related to SBE graduate education and research questions, particularly questions related to progression to the SBE PhD

Could bring in SBE professional societies to DC meeting

Use of AAAS AGEP web site for SBE Info briefs about institution evaluations

Development of the guide will begin in Year 2 and by the end of Year 2 we will post a draft version for comments by SBE Chairs and leaders in participating AGEPs. (September 2007)

The final version of the on-line guide will be posted by the end of Year 3. (September 2009)

Product --- On-line Evaluation & Assessment Guide for SBE Department Chairs and Faculty Leaders