1

Using Research Connections:

A Resource Kit for Fellows and their Organizations

• American Public Human Services Association • Education Commission of the States • National Association of Child Care Resource and Referral Agencies • National Association for the Education of Young Children • National Child Care Information Center • National Conference of State Legislatures • National Governors Association • National League of Cities • Office of Head Start • The Finance Project

2

Table of Contents

Purpose of the Resource Kit 3

Introduction to Research Connections 3

Understanding Research 3

Searching the Collection 5

Browsing the Collection 12

Our Publications and Webinars 15

Datasets and State Data Tools 16

Technical Assistance 17

3

Purpose of Resource Kit The purpose of this resource kit is to provide a tool that Fellows can use in presenting Research Connections to their members and others, and to assist Fellows’ state and local members to use the Research Connections web site directly. This resource kit pulls together materials that are available in Help and other sections of the site in ways that could be useful for Fellows and their organizations. The topics covered in this resource kit include: Understanding Research; Searching and Browsing the Collection; Our Publications and Webinars; Datasets and State Data Tools; and Technical Assistance.

Introduction to Research Connections Child Care & Early Education Research Connections is operated through a partnership between the National Center for Children in Poverty at Columbia University and the Inter-University Consortium for Political and Social Research at the University of Michigan. The project is supported by the Office of Planning, Research and Evaluation and the Office of Child Care in the Administration for Children and Families, Department of Health and Human Services. The goal of Research Connections is to promote high quality research and inform policy and practice. The web site offers a free and easily searchable online collection of more than 17,000 resources including original scholarly research, government reports, fact sheets and briefs, datasets, instruments and state data tools. Additionally, Research Connections produces its own publications that synthesize research on key topics. The collection is comprehensive and drawn from a wide range of disciplines that study child care and early education. Resources in the collection examine the child care and early education experiences of children birth through age 8 and when addressing school-age child care, through age 13. Other major subjects of research in the collection include: parents and families using child care and early education services; the early childhood and school-age child care work force; child care and early education settings, including center care, Head Start, public pre-kindergarten, family child care, family-friend-neighbor care, and in home care; national, state and local child care and early learning policies. Understanding Research High quality research is critically important for informing the formulation of policy and for program development. Policy and program decisions that are not informed by sound research can have adverse effects on children and families, and may not be cost effective. However, the quality of social science and policy research can vary dramatically. Research Connections accepts all research and related documents that are disseminated in the field, without judging the quality of their design, methods, findings and general content. It is essential, therefore, that consumers of Research Connections research evaluate the quality of these studies.

4

Key Questions to Ask Was the research peer reviewed?

Peer reviewed research studies have already been evaluated by experienced researchers with relevant expertise. Peer-reviewed research is usually of high quality. A research consumer, however, should still critically evaluate the study's methodology and conclusions. The Research Connections Web site indicates whether a study has been peer reviewed.

Can a study's quality be evaluated with the information provided?

Every study should include a description of the population of interest, an explanation of the process used to select study subjects, definitions of key variables and concepts, descriptive statistics for main variables, and a description of the analytic techniques. Research Connections users should be cautious when drawing conclusions from studies that do not provide sufficient information about these key research components.

Are there any potential threats to the study's validity?

A valid study answers research questions in a scientifically rigorous manner. Threats to a study's validity are found in three areas:

Internal Validity To determine whether a research study has internal validity, a research consumer should ask whether changes in the outcome could be attributed to alternative explanations, which are not explored in the study. For example, a study may show that a new curriculum preceded a significant increase in children's reading comprehension. The study must rule out alternative explanations for the increase in reading comprehension, such as a new teacher, in order to attribute the increase in reading comprehension to the new curriculum. Studies that specifically explain how alternative explanations were ruled out are more likely to have internal validity.

External Validity To assess whether a study has external validity, a research consumer should ask whether the findings apply to individuals whose place, times, and circumstances differ from those of study participants. A study's external validity is closely related to the generalizability of the findings. For example, a research study shows that a new curriculum improved reading comprehension of third-grade children in Iowa. As a research consumer, you want to ask whether this new curriculum may also be effective with third graders in New York or with children in other elementary grades. Studies that randomly select participants from the most diverse and representative populations are more likely to have external validity.

Construct Validity To assess whether a study has construct validity, a research consumer should ask whether the study has adequately measured the key concepts in the study. For example, a study of reading comprehension should present convincing evidence that reading tests do indeed measure reading comprehension. Studies that use measures that have been independently validated in prior studies are more likely to have construct validity.

5

What about reliability? The consistency and dependability of a survey question or set of questions to gather data is important. Reliability indicates the degree to which survey questions will provide the same result over time for the same person, across similar groups, and irrespective of who collects the survey data. A reliable set of questions will always give the same result on different occasions, assuming that what is being measured has not changed during the intervening period. Additional Resources

• For tools to assess research quality see: Quantitative Research Assessment Tool and Qualitative Research Assessment Tool

• For definitions of research methods see: Research Methods • For a glossary of research terms see: Research Glossary

Searching the Collection

The Research Connections collection is made up of a range of different types of resources, including original scholarly research, policy briefs, factsheets, government reports, datasets, and instruments. By searching on a specific topic you can select various resources and create a customized tool for your specific audience.

The search box appears in the top left-hand corner of the home page and also at the top of every page above the navigation bar. To conduct a basic search, simply enter your search term into the search box. If you would like to conduct an exact phrase search be sure to enter your phrase in quotation marks, e.g. “English as a second language”.

Let’s enter in the term: QRIS, which stands for Quality Rating and Improvement System. This search will also include commonly known acronyms and phrases for this topic such as QRS, and Quality Rating Systems.

6

The results page shows us that there are 589 results for this topic as of October 2010. However, you will likely find more results over time as we are constantly adding new resources to the site.

7

On the search results page you see the title, author, publishers, and a brief description of the resource. On the right hand side you’ll see a list of ways you can filter your search results (by full text, peer reviewed journal articles, resource type, author, pub date, state, and acquisition date). Additionally, at the top of the page you will see two icons:

- Indicates the publication has been produced by Research Connections

- Indicates that the article is drawn from a peer reviewed journal

To access any of the articles, simply click on the title of the article. For example let’s select the third title that appears on this page The Child Care Quality Rating System (QRS) Assessment: New Hampshire Quality Rating System: QRS profile

8

This takes you to a page that gives you more details about the resource and then in the box on the left hand corner you’ll see a link to access the full text of the article either through Research Connections or from the original source. Clicking on either of the links downloads the article.

9

Going back to the Home Page you will see ‘More Search Options’ in the search box which allows you to do a more in-depth search on a particular topic. Let’s search for “professional development” and “classroom quality”:

We get 447 results. If you want to see the most recent articles related to this topic you can go to the ‘Filter by Pub. Date’ option on the right hand side and select 2010. We get 36 results. A few of these articles have a green asterisk next to them, which means that they are from peer reviewed journals.

10

Let’s look at Relations among preschool teachers' self-efficacy, classroom quality, and children's language and literacy gains, from May 2010. Due to copyright restrictions Research Connections often cannot provide free access to the article, however we provide some options: if you are part of a university system and your university subscribes to the journal you can link to the article directly; or you can search for an online subscription where you can purchase the article; or you can query google scholar to see if the article is posted somewhere on the internet.

Research Connections is highly integrated with links among its various resources. You will see a link to 4 ‘related resources’ on this page. Clicking on this you will see 4 different instruments that relate to the study that are listed.

11

Going back to the search results page, once you click on any of the filter options, to go back to your original results you need to click ‘remove’ at the top of the page.

Also on the search results page you will see an option to save your search and be notified of any new additions to your search results through ‘google’ ‘yahoo alerts’ and other services.

12

Browsing the Collection

Beneath the search box on the home page and next to the search box at the top of the page you will see several options to browse the collection:

Browse by Author- provides an alphabetical listing of authors to choose from

13

Browse by State- allows you to select a state to find datasets, projects, and reports for which data were collected in the selected state.

For example there are 77 studies in which Alaska was included as one (among many) states in a study. For additional state-specific research you can enter in the state name in the search box on the home page. Entering in Alaska gives us 720 results. Alaska may be the only state studied in some of these reports as opposed to one among 50 states in the larger studies that you’ll find in the Browse by State feature.

14

Browse by Topic- We have created a list of subject categories around which we have classified all the resources in the collection. You can click on any topic to see the resources related to that topic. There are also subtopic within broader subject categories. Clicking on the plus signs takes you the subtopics. Let’s select + 9. Financing & Funding

There are 731 results for this topic:

15

Our Publications and Webinars

Our Publications

In addition to collecting publications from the field, Research Connections also produces its own publications. You can access our publications through searches, but also directly by clicking on ‘Our Publications and Webinars’ on the navigation bar or by clicking on a topic under the ‘Topics of Interest’ section on the home page. Our publication topics include:

• Quality Rating Systems • Education and Professional Development of Child Care and Early Education Providers • Coordination and Integration of Early Childhood Services • Regulation of Child Care and Early Education • Health and Safety • Many others

Our Webinars

Research Connections also organizes and conducts various webinars about the site and on topical issues. By visiting the Our Webinars section (on the ‘Our Publications and Webinars’ page) you will be able to access these webinars, in addition to the various Fellows webinars.

16

Datasets and State Data Tools

In addition to publications, Research Connections also provides access to various datasets and state data tools.

There are a total of 169 datasets in the collection. To access these click on the Datasets tab on the navigation bar and press ‘Go’ in the search box:

Additionally, if you would like to analyze any of the datasets online using the software provided, select the Analyze Data Online option at the top of the Datasets page.

17

Research Connections also provides various State Data Tools that are accessible from the Datasets page.

There are two types of State Data Tools:

1) Research Connections State Data Tools- These allow you to create customized tables for a range of child care and social welfare policies. You can also compare state policies across domains.

2) 50 State Databases on the Internet- This page provides a listing of various 50 State Databases offered by other organizations. Clicking on any of the links on this page will take you directly to the database.

Technical Assistance

If you need help or Technical Assistance with searching, browsing or accessing information on the website contact us at: contact@researchconnections.org

To keep abreast of new research added to the collection, grants, events, jobs and other opportunities, join our listserv: Subscribe to News & Resources.