itec6310 research methods in information technology instructor: prof. z. yang course website:...
TRANSCRIPT
ITEC6310Research Methods in Information Technology
Instructor: Prof. Z. YangCourse Website: http://people.math.yorku.ca/~zyang/itec6310.htmOffice: Tel 3049
2
What is Research?
There are many accepted definitions for the term “research”, for example:
Research is an active, diligent and systematic process of inquiry in order to discover, interpret or revise facts, events, behaviours, or theories, or to make practical applications with the help of such facts, laws or theories. The term ‘research’ is also used to describe the collection of information about a particular subject
3
What is Research?
Other insightful definitions of “research” are: Systematic, intensive, patient study and
investigation in some field of knowledge, usually employing the techniques of hypothesis and experiment, whose purpose is to reveal new facts, theories, or principles
Means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge
Systematic study directed toward more complete scientific knowledge or understanding of the subject studied
4
Basic and Applied Research
• Basic research– Conducted to evaluate theories
or empirical positions– Goal is to acquire information
about a phenomenon– Little emphasis on application to
real-world problems
5
• Applied Research– Evaluates real-world problem– Hypotheses may come from
theory, but goal is to apply results to real-world problem
• Substantial overlap between basic and applied research– Each type of research has
implications for the other
Basic and Applied Research (cont’d)
6
The Research Process
IdeaCasualand/or
systematicobservation
LibraryResearch
Choose an appropriate research design(experimental, correlational,etc.)
Develop idea into a testable hypothesisDeductiveReasoning
Choose an apporpriate subjectpopulation (animals, humans, etc.)
Decide on what to observe and theappropriate measures
Conduct study (pretesting, piolt work,acutal study)
Analyze data (descriptive andinferential statistics)
Report results (write paper or makepresentation)
7
Criteria for Identification of a Suitable
Research Problem1. The research problem should
address an important question so that the
answer will make a difference.
2. The research problem should advance the frontiers of knowledge by leading to new ways of thinking, suggesting possible applications, or paving the way for further research in the field.
8
State the Assumptions
• Assumptions are basic to the research problem.
• All assumptions that have a material bearing on the problem should be openly and unreservedly set forth.
• A statement of the assumptions is necessary for others to evaluate the conclusions of the study.
• A statement of the assumptions reveals what the researcher may be taking for granted with respect to the problem.
9
Benefits of Conducting a Literature Review
1. It can offer new ideas, perspectives, and approaches that may not have occurred to you.
2. It can inform you about other researchers who conduct work in the same area.
3. It can show you how others have handled methodological and design issues in studies similar to your own.
4. It can reveal sources of data that you may not have known existed.
5. It can introduce you to measurement tools that other researchers have developed and used effectively.
10
Benefits of Conducting a Literature Review (con’t)
6. It can reveal methods of dealing with problem situations that may be similar to difficulties you are facing.7. It can help you interpret and make sense of your findings and, ultimately, help you tie your results to the work of those who have preceded you.8. It will bolster your confidence that your topic is one worth studying, because you will find that others have invested considerable time, effort, and resources in studying it.
In short: the more you know about investigations and perspectives related to your topic, the more effectively you can tackle your own research problem.
11
Case study
• You are reading a literature review written by a novice researcher in your field. You notice that the researcher seems to have relied heavily on the World Wide Web in preparing the literature review. Your hunch is supported when you glance at the reference page and find a long list of Internet addresses. What goes through your mind regarding the quality and adequacy of the literature review in this situation?
12
Organization of a Research Project
1. Title Page 2. Abstract 3. Statement of the Problem 4. Background and Literature Review 5. Research hypothesis and Null hypothesis 6. Definition of research variables and operationalization 7. Research design and strategy 8. Threats to internal and external validity 9. Sampling 10. Data collection11. Data analysis and statistical tests 12. Conclusions 13. References
13
Preliminary Research Proposal
• in one or two pages• states the research question • states the purpose of the
research • sketched the initial model • discusses (explains) the initial
model • identifies pertinent background
literature
Choosing Variables for Your Study
• Research Tradition– Your variables may be similar to those
included in previous studies.– You may use previously used dependent
variables while manipulating new independent variables.
• Theory– A theory on which you are relying may
suggest certain variables to be included.
14
• Availability of New Techniques– Sometimes a new technique is
developed, allowing you to observe a variable that previously could not be observed.
• Availability of Equipment– The variables you manipulate or
observe may be limited by the equipment available to you.
Choosing Variables for Your Study
15
The RELIABILITY of a Measure
• A reliable measure produces similar results when repeated measurements are made under identical conditions.
16
• Reliability can be established in several ways– Test-retest reliability: Administer
the same test twice– Parallel-forms reliability: Alternate
forms of the same test used– Split-half reliability: Parallel forms
are included on one test and later separated for comparison
The RELIABILITY of a Measure
17
Example
Indicate for each of the types of reliability listed below whether it measures stability over time, equivalency of items, or both of these things:
– Test-retest reliability– Parallel-forms reliability– Split-half reliability
18
• An accurate measure produces results that agree with a known standard.
• A measurement instrument can be inaccurate but reliable.– The reverse cannot be true
ACCURACY of a Measure
19
• A valid measure measures what you intend it to measure
• Validity can be established in a variety of ways– Face validity: Assessment of adequacy of content.
Least powerful method– Content validity: How adequately does a test
sample behavior it is intended to measure?– Criterion-related validity: How adequately does a
test score match some criterion score? Takes two forms
• Concurrent validity: Does test score correlate highly with score from a measure with known validity?
• Predictive validity: Does test predict behavior known to be associated with the behavior being measured?
– Construct validity: Do the results of a test correlate with what is theoretically known about the construct being evaluated?
The VALIDITY of a Measure
20
Example
You have developed a comprehensive test for research methods course that covers all aspects of the course. What type(s) of validity would you recommend establishing for this measure? Why is face validity not considered a true measure of validity? How is it possible for a test to be reliable but not valid?
21
• Using Established Versus New Measures– You need not spend time assessing
validity and reliability of established measures. • However, they may not meet your research
needs.– New measures must be evaluated for
validity and reliability, which takes time and effort.• However, they may better meet your
research needs.
Choosing Your Measures
22
Scales of Measurement
• Nominal Scale– Lowest scale of measurement involving
variables whose values differ by category (e.g., male/female)
– Values of variables have different names, but no ordering of values is implied
• Ordinal Scale– Higher scale of measurement than
nominal scale– Different values of a variable can be
ranked according to quantity (e.g., high, moderate, or low self-esteem)
23
• Interval Scale– Scale of measurement on which the
spacing between values is known (e.g., rating a book on a scale ranging from 0 to 10)
– No true zero point
• Ratio Scale– Similar to interval scale, but with a
true zero point (e.g., running time of a certain algorithm)
Scales of Measurement
24
Factors Affecting Your Choice of a Scale of
Measurement• Information Yielded
– A nominal scale yields the least information. – An ordinal scale adds some crude information. – Interval and ratio scales yield the most
information.
• Statistical Tests Available– The statistical tests available for nominal and
ordinal data (nonparametric) are less powerful than those available for interval and ratio data (parametric).
– Use the scale that allows you to use the most powerful statistical test.
25
Examples
Identify the scale of measurement for each of the following variables:
• ZIP code• Software cost• SAT score• Class rank• Letter grade• Correction rate
26