Methods of Research and Enquiry

Fundamental Issues with Research in Education

by Dr. Daniel Churchill

R&

D i

n I

T i

n E

du

cati

on

Content Overview

Defining research question Reviewing literature Referencing and APA Selecting Methodology Ethical issues Data Collection Analysis Validity and reliability issues Writing a Reporting

R&

D i

n I

T i

n E

du

cati

on

Let’s Check on your Group Blogs…

R&

D i

n I

T i

n E

du

cati

on

Revision--Educational Research

Educational Research explains, predicts and/or control educational phenomenon

Educational research is conducted in the following way:

Define -- a significant problem, hypothesis or question

Plan -- review literature to establish background to the study, identify methodology, plan procedure

Execute research procedures -- collect data e.g., interviews, surveys, questionnaires, observations

Analyze data -- qualitatively, quantitatively or a mixed way

Draw conclusions and recommendations

R&

D i

n I

T i

n E

du

cati

on

Revision--Overview of Approaches to ER

R&

D i

n I

T i

n E

du

cati

on

Qualitative vs. Quantitative Research

Qualitative QuantitativeDevelops hypotheses Tests hypotheses

Does not seek to control the contexts

Seeks to control the context

Researcher interacts with the participants

Researcher does not interact with participants

Involves a smaller sample Involves large number of subjects/participants for results to be statistically significant

Assumes individuality Assumes that contexts are stable, uniform and controllable

Interpretation of data Data analyses relies on statistical procedures

R&

D i

n I

T i

n E

du

cati

on

Defining research Identify the problem Understand the background of the problem Determine significance of the study – what is important of that

problem. Determine the Unit of Analysis (e.g., student, class, school…etc) Determine the Relevant Variables – relationship or variable. State the Research Questions A good research problem

Can be investigated through the collection and analysis of data

Has theoretical or practical significance It is a good problem for you

R&

D i

n I

T i

n E

du

cati

on

Possible Areas of Research on IT in Education

R&

D i

n I

T i

n E

du

cati

on

Important Research Areas (Lim & Hedberg, 2004) Professional development of teachers in ICT integration

(from pre-service to in-service) R&D of emerging technologies grounded in theories and

pedagogies Systemic processes in schools to facilitate ICT integration ICT integration in specific disciplines; that is, how the

affordances of ICT are taken up to enculturate students into discipline-specific way of thinking – e.g. teach maths is diff. from science.

ICT-mediated learning environments for low-performance students Student generated games and simulations

R&

D i

n I

T i

n E

du

cati

on

Limitations of R&D in IT in Education (Lim, 2004))

For a research to transform practices in schools, findings should be accepted by the education community

The culture of schooling does not support innovation Lack of theoretical framework for research design Research studies have been mostly at descriptive level,

lack of studies that refine and generate theories = no any suggestion or recommendation.

Lack of research-based development of tools and processes for use by practitioners – lack of implementation.

Lack of coordination of effort between research, design, development, policy and practice

R&

D i

n I

T i

n E

du

cati

on

Research Framework A framework is a particular set of rules, a particular set of rules,

ideas, or beliefsideas, or beliefs which you use in order to deal with problems or to decide what to do

A framework is a structurestructure that forms a support or frame for something

Contains all the appropriate, relevant and important conceptsconcepts related to the research study

For example, in the Law et al. (2000). Changing Classrooms & Changing Schools: A Study of Good Practices in Using ICT in Hong Kong Schools. CITE, HKU (http://sites.cite.hku.hk)

R&

D i

n I

T i

n E

du

cati

on

Framework for Research of IT in Schools

R&

D i

n I

T i

n E

du

cati

on

Framework for Research of IT in Schools

R&

D i

n I

T i

n E

du

cati

on

Reviewing literature

After selecting a research topic and formulating research problem and significance, identify key words for a literature search

R&

D i

n I

T i

n E

du

cati

on

Reviewing literature

General purposes Summary and synthesis of the state of knowledge - existing

studies and findings Evaluation - strength and limitations in methods, results and

conclusions Identifying gaps in knowledge Providing background and justification of one’s own research

Justification of the research objective There is a gap in or a need of knowledge This gap can be proposed as a research question This question should and can be answered by research Your research objective is to answer the question

Justification of the methods

R&

D i

n I

T i

n E

du

cati

on

Reviewing literature Requirements

Comprehensive coverage - literature search (e.g. electronic databases - ERIC) - search for local as well as international materials

Up-to-date e.g. you may use some original sources, but your review should include a majority of papers which are from about 1990 onwards

Depth vs. breadth i.e. it is better to cover your chosen area well, than to give an overview of a number of areas

Common Problems Too brief, too lengthy, out of proportion Missing important literature Biased, unbalanced selections or opinions Close paraphrasing and plagiarism Linkage between parts and coherence

R&

D i

n I

T i

n E

du

cati

on

Library Collections Reference collections:

Materials used in the library that provide access to information often in a summarized form. Reference collections typically include encyclopedias, statistical material, dictionaries, bibliographies, literary criticism, handbooks, and biographies. Reference collections do not circulate.

Circulating collections: Materials that can be checked out of the library. Each

library sets its own policy on how many items a patron can check-out, as well has how long the item is loaned. 

Reserve collections: Course materials set aside by faculty for student use.

Most library reserve collections are located behind the circulation desk and are limited to in-library use only.

R&

D i

n I

T i

n E

du

cati

on

Library Collections Special Collections/Archives:

Materials purchased in specialized subject areas. The library usually shelves special collection material separate from the general circulating and reference collections.

Microform collections: Material that has been miniaturized and placed on

microfilm or microfiche. Microform material must be read on special microform machines.

Electronic media collection: Collection of video and audio tapes, CD-Roms and

other electronic resources

R&

D i

n I

T i

n E

du

cati

on

Other Sources of Literature Education Index

Covers professional publications Readers Guide to Periodical Literature

covers articles from 200 widely read magazines (popular literature).

Expanded Academic Index Covers journal articles from over 1500 periodicals.

Dissertation Abstracts International Contains bibliographic citations and abstracts from

doctoral dissertations and master's theses worldwide. Psychological Abstracts

Presents summaries of studies completed in psychology, including developmental psychology and educational psychology. These two areas are of special interest to educational researchers.

R&

D i

n I

T i

n E

du

cati

on

Other Sources of Literature ERIC - the Educational Resources Information

Center collects and disseminates reports of current educational research, evaluation, developmental activity. ERIC maintains two databases searchable by computer: RIE - Resources in Education, this ERIC database

contains bibliographic citations and summaries to information not published in journals, e.g. conference presentations, technical reports, and unpublished research results. The items in this database are identified by ED numbers.

CIJE - Current index to Journals in Education, this ERIC database contains bibliographic citations and article summaries to journals. The items in this database are identified by EJ numbers.

R&

D i

n I

T i

n E

du

cati

on

Let’s Search for Something HKU Library Resources

http://lib.hku.hk/ http://lib.hku.hk/general/research/index.html http://obelix.lib.hku.hk/irms/education.html

ERIC http://searcheric.org/ http://www.eric.ed.gov

Proquest http://proquest.umi.com/login?ts=0

UMI ProQuest Digital Dissertations http://wwwlib.umi.com/dissertations/gateway

Some other places (Try Harward) http://lib.harvard.edu/e-resources/index.html

R&

D i

n I

T i

n E

du

cati

on

Other Sources of Literature Internet Search

There are many searching engines Question: Do you knows any searching engines?

Lets look at http://www.google.com Searching and filtering e.g. site: define: link: Google Scholar: http://scholar.google.com/ Google Books: http://books.google.com/

You can ask a question at Ask Jeeves:http://www.ask.com

Web 2.0 Resources: http://www.citeulike.org/ http://www.librarything.com/ http://www.connotea.org/

There is also MetaSearch Tools that use many searching engines and help you to organize search records. Lets look at the http://www.copernic.com

R&

D i

n I

T i

n E

du

cati

on

Other Sources of Information

Newspaper Organizational Records Experts in the field Professional magazines

R&

D i

n I

T i

n E

du

cati

on

Referencing If you quote someone, spell the name correctly

and make sure they appear in your reference list. Do not include references in the reference list if

they have not appeared in the text. Remember your reference listing style MUST

comply with the American Psychological Association (APA) style and must be consistent: http://www.apastyle.org/

Check out this: http://www.citationmachine.net/

R&

D i

n I

T i

n E

du

cati

on

Reviewing Articles You can use the following procedure for abstracting the

contents of articles (Gay, 1996) : Read the article's abstract or summary to see if it is a

useful Skim the entire article making a mental note of the

main topics Write the complete reference in APA style Classify and code the article according to some

system of your own devising. Put the code: on an index card, on the photocopied article (if you photocopied it), on the computer so you can sort the article abstracts in any way you wish to.

Summarize the reference by paraphrasing the essential points of the reference.

Add any thoughts that come to your mind about the article.

Indicate any statements that are direct quotations

R&

D i

n I

T i

n E

du

cati

on

Selecting Methodology Based on your research question, you can

identify your methodology A qualitative research question includes

phenomenon of interest and subjects e.g.,: A Study of challenges of a new environment

for a group expatriate teachers in a Hong Kong School What are challenges of new working environments

for expatriate teachers in Hong Kong? How perception of challenges shift through out the

first teaching semester?

R&

D i

n I

T i

n E

du

cati

on

Selecting Methodology

A quantitative research question includes variables of interest to the researcher, relationship between the variables and type of subjects involved, e.g., The relationship between intelligence and

computer use in a secondary school science class.

R&

D i

n I

T i

n E

du

cati

on

Selecting Methodology From a question you can identify a kind of quantitative

research based on the following formulas:

[variable X], [variable Y], and [variable Z] among [type of subjects] descriptive research.

The relationship between [variable X] and [variable Y] among [type of subjects] correlational research.

The effect of [independent variable not under experimenter's control] on [dependent variable] for [type of subjects] quasi-comparative research.

The effect of [independent variable X under experimenter's control] on [dependent variable Y] for [type of subjects] experimental research.

R&

D i

n I

T i

n E

du

cati

on

Data Collection What is the methodology of the research (e.g.

quantitative or qualitative)? What kind of research design will be used? What are the advantages and limitations of the design?

Who will undertake the research? What and where is a research site? What is the target population or who are the

participants? How big will sample be, how will it be selected, and will it be representative of the population?

What kinds of data are required? What instruments will be used? Do they include the

researcher? How were they selected/developed? What evidence will be given about their validity and reliability (or other standards)? Are they the best instruments to use?

R&

D i

n I

T i

n E

du

cati

on

Ethical issues Ask participants to acceptance to participant in the study

Provide the participant with Plain Language Statement containing; Information about the objectives of the study; Data collection methods; Right to withdraw from the study; Access preliminary data, analysis and report ; Explanations of the participants’ role and responsibilities will be; That the participant’s identity will not be disclosed and acronyms

will be used for his/her name, and Inform the participant that data will be used for the purpose of the

study and possible journal publications Inform the participant when collecting data When writing report the researcher will ensure that the

audience will be able to distinguish between data and interpretations.

The researcher will remain unbiased in respect to collected data and will acknowledge if any biases cannot be controlled.

R&

D i

n I

T i

n E

du

cati

on

Validity and reliability issues Concept of Validity and reliability is different

for Qualitative and Quantitative studies In Quantitative Research:

The concept of reliability has to do with how well have you carried out your research project. Have you carried it out in such a way that, if another researcher were to look into the same questions in the same setting, they would come up with essentially the same results (though not necessarily an identical interpretation). If so, then your work might be judged reliable.

Validity has to do with whether your methods, approaches and techniques relate to, or measure, the issues you have been exploring.

R&

D i

n I

T i

n E

du

cati

on

Validity and reliability issues In Qualitative Research there is a concept of Internal

Validity which is strengthened through: Prolonged involvement in the study by the researcher; Member checking -- allowing participant to scan

transcripts, field notes, preliminary analysis and report; Use multiple sources of data -- interviews, observations

reflection records, documents; Involve critical advisors to help with data analysis; Test working hypotheses by confronting the participant --

frequently move between data and analysis and allow preliminary analysis to direct further data collection;

Search and read literature for theoretical validation, and Auditing -- obtain auditing report from an independent

researcher to confirm the authenticity of data collection and analysis.

R&

D i

n I

T i

n E

du

cati

on

Writing a Reporting

The research report, thesis, dissertation, the journal article, academic text and conference paper are the main forms of communication about what the research has achieved

Report usually contains the following sections: Introduction – background of research Literature Review Methodology Results Discussion

R&

D i

n I

T i

n E

du

cati

on

Writing Academically

Academic Grammar is a HKU Web-based resource to help students with their academic assignments http://ec.hku.hk/acadgrammar/

Nuts and Bolts of College Writing http://www.nutsandboltsguide.com

PhraseBook for Writing Papers and Research http://www.wholeworldcompany.com/phrasebooks/

Writing dissertations guide

http://www.learnerassociates.net/dissthes/

R&

D i

n I

T i

n E

du

cati

on

Digital Theses -- Available in full online

http://adt.curtin.edu.au/theses/browse/by_author/all.html

R&

D i

n I

T i

n E

du

cati

on

Blog Task for next Session Complete before next session

Identify the best research oriented professional journal in the field you plan to do research in.

Review the table of contents for at least three recent issues of this journal and for each issue identify a research based article you would like to review .

Read one of the identified articles of interest to you and answer the questions about it:

What is the problem for the study? What procedures did the experimenter use for the study? What were the major conclusions for the study? How would you classify the study, according to the six types

of research studies we looked at in this lesson? Present your review in your blog Critique at least two reviews done by other

groups http://www.isiwebofknowledge.com/

R&

D i

n I

T i

n E

du

cati

on

Suggested Readings

Kozma, R. (2000). Reflections on the state of educational technology research and development. ETR&D, 41(1), 5-15.

1. Journal of Learning Sciences 2. Computers & Education 3. British Journal of Educational

Technology 4. Journal of Computer Assisted Learning 5. Etr&d educational Technology Research

and Development