research agenda workshop

Regional Educational Laboratory at EDCrelnei.org

1

Research Agenda Workshop

[Insert relevant information. Consider:• Alliance or group name

• Facilitator’s name(s)• Date

• Location]


Welcome and purpose

Workshop objectives:

• Engage in a collaborative process• Identify alliance research priorities• Develop a set of research

questions• Develop a coherent research

agenda for the next 3–5 years

2


Session goals

• Morning:– Review different types of research– Identify and prioritize research topics– Generate related research questions

• Afternoon:– Refine and prioritize research questions to form initial

research agenda

3


Agenda

• Introduce participants• Identify possible priority research topics• Investigate types of research and levels of evidence• Explore research questions and research agendas• Prioritize research topics• Generate research questions• Share, refine, and prioritize research questions• Put it all together into a research agenda

4


Introductions

• Name• Where you work• Your role

5


6

Alliance goal

[Insert your alliance or group’s goal. For example: The USVI College and Career Readiness Research Alliance will support the Virgin Islands Department of Education in their efforts to prevent and reduce the number of students dropping out of schools by providing applied research and analytic technical support on how best to use available data to both establish robust early warning systems and identify interventions to help improve outcomes for students at risk.]


7

Alliance goal and research topics

• Record in your workbook on page 5:– Topics that fit under your

alliance’s goal– If you are unsure of a topic,

include it in the “possible” column

• Focus on topics that are most important to you.

• You will have five minutes to think and write by yourself.


8

Small-group discussions

• Break up into small groups• On large Post-It notes, write

down the topics your group would like considered as priority topics – One topic per Post-It note


Grouping our topics

• Affinity grouping process.• What topics generated in the

groups go together?• Review the topic groupings and

name them.– Items grouped under each topic

name are now considered subtopics.

9


10

Types of research

Typology used by the REL program at IES includes:

• What’s Happening– Descriptive

• What’s Known– Descriptive

• Making Connections– Correlational

• Making an Impact– Impact/causal


11

What’s Happening

• Descriptive studies look at what is happening: trends; baselines; and experiences of individuals, groups, or programs

• Methods include secondary data analysis and document and records review

• The studies use descriptive statistics: averages, frequencies, and percentages


12

What’s Known

• Another type of descriptive study focuses on reviews of the literature and other previous research. Examples include:– Literature reviews– Meta-analyses– What Works Clearinghouse reviews

• The What Works Clearinghouse is an IES-funded initiative that provides reviews of existing education research and evaluates the credibility and reliability of the research evidence.


Making Connections

• Correlational studies look at the relationships among two or more variables or characteristics but do not imply that one causes another.

• Correlational research tests whether relationships between variables are “statistically significant,” meaning they are not likely due to chance

• Methods include analysis of existing data from administrative or other state, district, or school datasets.

13


14

Making an Impact

• Causal/impact studies usually examine questions about impact; they can build on descriptive studies

• Look for “opportunistic” possibilities

• An impact study:– Examines the effectiveness of a particular policy, program, or practice– Can be small or large– Can be used in a formative way (for example, a short-term pilot study)

or a summative way


15

Making an Impact

• Only certain types of designs can assert causal claims or claims about program impact

• REL impact studies must meet What Works Clearinghouse standards– Randomized controlled trials (RCTs) = the “gold standard”– Other designs, including quasi-experimental and matched

comparisons, may not yield unbiased estimates of impact


16

Investigating research examples

• Review the research summary you read during prework (5 minutes)

• Divide into small groups by summary and discuss (15 minutes)– What the research questions were– What data sources were used– What the research design was– What else there is to learn– What you would need to answer the

remaining questions (data, RCT, etc.)?

• Whole-group discussion (10 minutes)


17

Break

• 15 minutes


18

Researchable questions

• What is a researchable question?– Reasonable– Appropriate– Answerable– Specific

• Where do researchable questions come from?– Questions, concerns, and values of stakeholders– Important issues in the field or research literature– Professional standards or guidelines– Views and knowledge of experts– One’s own views and judgment


19

Research agendas

• What is a research agenda? – Identifies research priorities and

questions for group– Is coherent– Leads to rigorous and relevant

research that is actionable– May include current or future project

• What does a research agenda look like?– Linear– Topical


20

Example: Research agenda that Is less coherent • Topic: College readiness• Research questions:

– What are the high school dropout rates for particular subgroups of students?

– Does coaching high school English teachers impact their students’ performance?

• Studies:– Descriptive study of high school dropout rates for key subgroups– Descriptive study of policies for assigning students to college

math versus general math– RCT of the impact of classroom coaching for new high school

English teachers in five large urban districts


21

Example: Coherent, linear research agenda• Topic: High school graduation• Research questions:

– What are the high school dropout rates for key subgroups of students?

– What are the effects of alternative high school programs on students’ degree completion and graduation?

• Studies:– Descriptive study of high school dropout rates for key subgroups– Descriptive study of re-enrollment rates in traditional high

schools and education trajectories of re-enrollees– RCT of the impact of alternative programs for degree completion

on high school graduation


22

Example: Coherent, topical research agenda • Topic: Mathematics learning• Research questions:

– What is the impact of grade 8 students’ access to algebra I on their math achievement?

– How do students with disabilities perform in math?• Studies:

– RCT of the impact of student access to algebra I in grade 8– Descriptive study of math education practices for students with

disabilities– Descriptive study of math performance patterns for students with

disabilities


23

Prioritizing research topics

Narrow the list of research topics to 24 priority topics for our alliance, using a modified Focus Four:

• Brainstorm• Clarify• Advocate• Canvass


24

Example: Gender in Schools Research AllianceAlliance goal: Provide research that informs and promotes gender equity in schools

Topic Subtopic Questions TimingGirls in STEM Tracking achievement

differences

Encouraging interest and enrollment in STEM

Persistence in STEM majors

Attainment for boysStructured Inequality


Moving from subtopics to questions

25


Moving from subtopics to questions

26

Subtopic Questions Timing

Tracking achievement differences

What are gender differences in STEM achievement in K12 and how have they changed over time?Do gender differences in STEM achievement vary among districts and schools?

Encouraging interest and enrollment in STEM

What can teachers do?Are single-sex schools and classrooms better?What programs help promote STEM for girls?

Persistence in STEM majors

A fully specified research question

A general question that can be used to develop a set of more

specific research questions


27

Generating research questions

• In pairs, work for 30 minutes to generate research questions related to identified priority research topic(s).

• Think about:– What are some possible research

questions for these topics and subtopics?

– What data would be needed to research these questions?


28

Lunch break

• 45 minutes


29

Sharing questions

• Post newsprint with questions from each group

• Review the posted newsprint and consider:– What would you change or edit?– What would you delete?– What would you add?

• Briefly discuss with a partner


30

Refining questions

• Whole group (small groups if whole group is large):– Edit, add, delete, and combine questions as necessary– What kinds of studies would result from the questions

(descriptive, correlational, impact evaluation)?• In pairs, discuss:

– What is important to you? – What research topic and questions can provide a

meaningful research agenda?


31

Prioritizing research questions

• Prioritize the research questions in each priority topic area

• Focus Four process– Brainstorm– Clarify– Advocate– Canvass


32

Preliminary research agenda

• Review identified priority topics and questions– Any surprises?

• How many top-rated questions should be on the agenda? Consider:– Alliance goal– Types of research– Sequence– Short- and long-term nature of questions


Research agenda template

• Turn to research agenda template on page 19

• Use template to organize research agenda– Document and summarize the main research topics and

subtopics identified, along with research questions

33


34

Next steps

• Summary document from workshop• Iterative review and refinement of agenda• Agenda as a “living document”• Collaborative work to:

– Identify appropriate existing data or data collection for studies– Identify existing research relevant to agenda priorities and

questions– Develop study designs– Conduct research or identify research support


Thank you

• For more information contact:– [insert contact information or

customize with any special closing information]

35

research agenda workshop

Documents

research topicsrecord

applied research

research questionsput

coherent research agenda

initial research agenda

types of research agendas

orgfacilitator reviews

types of evidence