sak 5090 mohd hasan selamat- chapter 5slide 1 chapter 5 planning the research project l successful...

SAK 5090 MOHD HASAN SELAMAT- chapter 5 Slide 1

CHAPTER 5Planning the Research Project

Successful research is a planned research Fail because plunged into activity with

only a partially thought-out plan & inconclusive design

Research requires a conceptualisation of overall organization and detailed plan

Should be no less totally visualized and precisely detailed


Planning the Research Project A general strategy = research design (RD) RD provides

• Overall structure for the procedures

• The data the researcher collects

• The data analyses the researcher conducts

Must plan the overall design carefully to be successful Wasted effort if half-prepared (vague ideas) Must identify RESOURCES, PROCEDURES, and DATA

always with the goal from the very beginning


All Research has a Basic Format No matter what the academic discipline Not parochial (narrow, limited) and seldom fits into neatly

packaged academic disciplines Software engineer may be compelled to stray far from the

field of SE in pursuing a research problem in user interface design

The ramifications of problem may lead into fields far from that of the original problem

Err in thinking too narrowly, in restricting problems for research to one academic area


The Basic Format of the Research Process

1. A question is posed. In the mind of the researcher, a question arises that has no known resolution

2. It’s a matter of words. The researcher converts the question to a clearly stated research problem.

3. It’s worth a guess! The researcher poses a temporary hypothesis or series of hypotheses.

4. The search is on! The researcher searches the literature for ideas that shed light on the problem and for strategies that may help to address it.


The Basic Format of the Research Process

5. Data! Hard Data! And nothing BUT the data. The researcher collects data that potentially relate to the problem.

6. How do the data fit together? The researcher arranges the data into a logical organizational structure.

7. The data speak! The researcher analyzes and interprets the data to determine their meaning

8. It’s either … or … Either the data seemingly resolve the research problem or they do not. Either they support the hypotheses or they do not.


All Research has a Basic Format Might do better to think of problems as arising

out of broad generic areas within whose boundaries all research falls:

• People – research problems relating to individuals, groups, populations, nationalities, families, ...

• Things – biological life, matter, …• Records – letters, legal documents, lists,

journals, …


All Research has a Basic Format

• Thoughts and ideas – opinions, reactions, concepts, theories, …

• Dynamics and energy – human energy and activity, metabolism, gravity, hydrologic cycles, …

Above suggestions are not all-inclusive, merely suggest the broad ramification of research possibilities

Substratum (an underlying layer or foundation) of research principles is all-inclusive


All Research has a Basic Format Physicist exploring subatomic particles and

sociologist exploring social behaviour are both employing the same research principles, although tools may differ, interpretational techniques may have little in common – approach identical

Research approaches all problems through certain methodological channels appropriate to the nature and type of data

Must differentiate research design VS research methodology


Research Planning Versus Research Methodology

Many people are confused between the two Example, statement “research in physics is

different from research in philosophy or history” Confusion between research as a process and the

methodology employed by separate academic disciplines in collecting and processing data

Genuine research follows the scientific method and exhibits the EIGHT characteristics



Data vary widely, therefore cannot employ the same methodology, e.g. blood cells vs historical facts

In planning, the researcher must choose a viable research problem and consider the nature of data and the feasible means of collecting and interpreting those data

Comparing the brain wave patterns of gifted and nongifted children may be an engaging project for research, BUT consider the following issues:



Who are willing to participate? Do we have the equipment? If so, do we have the technical skills to use it? Are we able to interpret the tracings? Do we know how to draw conclusion? It must be a practical research – something that

we must be able to handle


General Criteria for a Research Project

Universality – Could be carried out by any competent person other than the researcher, same results without prejudice to project and validity

Replication – Should be repeatable by other researchers under the same circumstances, within identical parameters and achieve comparable results

Control – All researches are conducted within an area sealed off by given parametric limitations


Criteria for Research Project Measurement – Data should be susceptible to

measurement. Easily accomplished in the physical sciences. In humanities and social sciences research, it is more difficult to quantify, measure, or evaluate

Many research efforts are wasted by going-off HALF-PREPARED

Must bring all together, early inventory of resources, your problems, and the source of data, in order to save time, money and effort


Data: Their Relationship to Design and Methodology

Without facts (data), inductive reasoning vanishes and scientific method collapses

The lifeblood of research Data is a Latin verb dare, meaning “to give” –

give information, impression or other factual data to an observer

Facts from Latin word facere, meaning “to make” – what the situation makes or manifests to the observer


What is Data? Data are manifestation of the truth rather than the

truth itself – no one has ever look upon truth itself – pure, undisguised, naked Truth

Data are merely representative, intermediate, elusive surrogate of Truth – reflects Truth as a mirror which reflects sunlight

Researchers are like those who live in dungeon seeing a beam of sunlight passes across the floor and get the idea of what the sun must be like


What are Data? Factual dungeon – never be able to see the original source

of the data – see what other people do – the behaviours they exhibit, the things they create, and the effects of their actions on others. But the actual people “inside” – those individuals we shall never know!

Research seeks, through data, what is true absolutely – the complete meaning of the data

Experienced researchers are constantly aware that the ultimate goal of research (Truth) is forever just beyond what is represented by the data, and, hence, just beyond human grasp


What are Data? The relationship between data and the ultimate

truth is shown in the diagram - various states of data

Realm of ultimate truth, the most inaccessible – can be approached by means of passing through two intermediate areas labeled The Realm of the Data


? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

The Realm of the InquisitiveMind of the Researcher

o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o oo o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o

The Region of the Secondary Data

The Region of thePrimary Data

The Barrier of the Human Senses Skills of Reading and Writing, Language Capability, and the Channels of Communication

The Dividing Line Between Primary and Secondary Data

The Impenetrable Barrier Beyond Which Lies the Absolute Truth

THE REALM OF ABSOLUTE TRUTH

The Realm of the Data


Data are Transient and Ever Changing

Data are not only elusive but also transient – exist in a split second

Example, an opinion survey on a population – tomorrow’s result will not be the same as now, people move out, move in, died, ….

Pretest and posttest conducted by a teacher in a class may not be the same class at all in six weeks, they have been growing, learning, maturing, changing


Data are Transient and Ever Changing

Example, asking people about their interests – can we be sure what they tell us is true?

They may answer honestly that represents the truth as they perceived it at that instant

May change because of factors: age, maturity, newscast, or items in newspaper

Researcher should recognize that even the most reliable, refined, carefully controlled data may have elusive (hard to find, describe or remember) quality – very volatile, evaporate


Primary Data Lie closest to the source of The Absolute Truth

underlying the phenomenon Reflect truth more faithfully than any other

approach – most valid, most illuminating, the most truth-manifesting

Impenetrable barrier exists between the Realm of Absolute Truth and the region of primary data

A beam of sunlight does not compare with seeing the sun itself


Car Accident I witnessed the full occurrence of a car veer off the

highway and careen into the ditch The driver told me that he had no awareness of the

possibility of an accident Neither of us will ever be able to determine the absolute

truth underlying the accident What are the factors? Momentary seizure? Mechanism

imperfections? Answers lie beyond an impenetrable barrier, may never be

known


Secondary Data Beyond the region of primary data lies the region

of secondary data In the dungeon – not a direct beam of sunlight but

simmering light upon the wall, reflected by mirror, distorted by imperfections

In the case of the car accident, I wrote an account stating precisely what I observed and confirmed by the driver – readers of the next day’s newspaper get the similar “reflection of the sunlight”


Barriers of The Truth The data/facts are of necessity distorted albeit by

the channels of communication, my writing skills, reading skills, and the inability of language to produce every detail

Many more barriers than in the diagram• The acuity of human senses

• The sensitivity of instruments

• The failure of language to communicate thought exactly

• The inability of two persons to witness the same event and report it precisely as duplicate accounts


Unwarranted Conclusion Even in exuberant moment of discovery, no one

has ever glimpsed the Absolute Truth, nor can we come to a knowledge of the data that reflect that Truth except through the gross and shadowy channels of our dull and imperfect senses

Be cautious and respect for words such as perhaps, it seems, one might conclude and it would appear to be


Criteria for Admissibility of Data Not all data are acceptable – can be defective –

affect the validity of the conclusion Stem from imperfections and irregularities of

nature Research should be replicable under precisely the

same conditions To regulate the precision of conditions, criteria

must be adopted, limits established, standards set up that all data must meet


Criteria for Admissibility of Data Non-conformance excluded These controls are easy in sciences Standardize data, admit only compliance – can

control the research effort and conclude with greater certainty

The criteria must be set forth clearly in proposal as well as report to ensure integrity of research

So the consumer can come to an intelligent understanding of what is being studied


Identifying Appropriate Measurement Instruments

• Sometimes we will be able to use one or more existing measurement instruments OR have to develop our own instruments

• Measurement instruments provide a basis on which the entire research effort rests• Describe any instrument used in explicit, concrete terms• Provide evidence that the instruments you use have a

reasonable degree of validity and reliability for your purposes• Validity = the extent to which the instrument measures what it

is actually intended to measure• Reliability = the extent to which it yields consistent results

when the characteristic being measured hasn’t changed


Determining the Validity of Measurement Instruments

Face validity• The extent to which, on the surface, an

instrument looks like it is measuring a particular characteristic

• Useful for ensuring the cooperation of people who are participating in a research study

• Since it relies entirely on subjective judgment, it is not, in and of itself, terribly convincing evidence that an instrument is truly measuring what the researcher wants to measure



Content validity • The extent to which a measurement instrument is a

representative sample of the content area (domain) being measured

• Used to assess people’s achievement in some area – for instance, the knowledge they have learned during classroom instruction or the job skills they have acquired in a rehabilitation program

• High content validity means that its items or questions reflect the various parts of the content domain in appropriate proportions and requires the particular behaviours and skills that are central to that domain



Criterion validity • The extent to which the results of an assessment

instrument correlate with another, presumably related measure (the criterion)

• Example, a personality test designed to assess a person’s shyness or outgoingness (introvert or extrovert) has criterion validity if its scores correlate with other tests of introversion versus extroversion

• An instrument designed to measure a salesperson’s effectiveness on the job should correlate with the number of sales the individual actually makes during the course of a business day



Construct validity • The extent to which an instrument measures a characteristic that

cannot be directly observed but must instead be inferred from patterns in people’s behaviour (such a characteristic is a construct)

• Motivation, creativity, racial bias, bedside manner – all of these are constructs, in that none of them can be directly observed and measured

• Sometimes there is universal agreement that a particular instrument provides a valid instrument for measuring a particular characteristic

• We could all agree that a ruler measures length, a thermometer measures temperature, and a barometer measures air pressure

• But whenever we do not have such universal agreement, we must provide evidence that an instrument we are using has validity for our purpose



• To demonstrate that the measurement instruments used by researchers have validity for their purposes, the following examples are used:

- A multitrait-multimethod approachTwo or more different characteristics are each measured using two or more different approaches. The different measures of the same characteristic should be highly correlated. The same ways of measuring different characteristics should not be highly correlated

- A table of specificationsTo construct a measurement instrument that provides a representative sample of a particular content domain – in other words, to establish content validity – the researcher often constructs a two-dimensional grid (table of specifications) listing the specific topics and behaviours that reflect achievement in the domain. In each cell of the grid, the researcher indicates the relative importance of each topic-behaviour combination. He or she then develops a series of tasks or test items that reflects the various topics and behaviours in appropriate proportions

- Judgment by a panel of expertsSeveral experts in a particular area are asked to scrutinize an instrument to ascertain its validity for measuring the characteristic in question

Although none of the approaches just described guarantees the validity of a measurement instrument, each one increases the likelihood of such validity.


Determining the Reliability of Measurement Instruments• Interrater reliability – the extent to which two or more

individuals evaluating the same product or performance give identical judgments

• Internal consistency reliability – the extent to which all the items within a single instrument yield similar results

• Equivalent forms reliability – the extent to which two different versions of the same instrument (e.g., “Form A” and “Form B” of a scholastic aptitude test) yield similar results

• Test-retest reliability – the extent to which the same instrument yields the same result on two different occasions


Reliability and Validity• A researcher can enhance the reliability of a measurement

instrument in several ways.1) The instrument should always be administered in a

consistent fashion; in other words, there should be standardisation in use of the instrument from one situation or person to the next

2) To the extent that subjective judgments are required, specific criteria should be established that dictate the kinds of judgments the researcher makes

3) Any research assistants who are using the instrument should be well trained so that they obtain similar results


Reliability and Validity• We can measure something accurately only when we

can also measure it consistently

• In other words, in order to have validity, we must also have reliability

• The more valid and reliable our measurement instruments are, the more likely we are to draw appropriate conclusions from the data we collect and, thus, to solve our research problem in a credible fashion


Linking Data and Research Methodology

Data are like ore – contain pieces of the truth but are in a rather unrefined state

To extract meaning from the data, we employ what is commonly called research methodology (RM)

Data and methodology are inextricably interdependent The RM to be used for a particular research problem must

always take into account the nature of the data that will be collected in the resolution of the problem

RM is merely an operational framework within which the facts are placed so that their meaning may be seen more clearly


Data and RM

• Different questions yield different types of information

• Different research problems lead to different research designs and methods, which in turn result in the collection of different types of data and different interpretations of those data

• Many kinds of data may be suitable only for a particular methodology


Data and RM

• To some extent, the data dictate the research method

• Historical data from written records of past event - cannot extract meaning by laboratory experiment. An experiment is simply not suited to the nature of the data

• Not only a true experiment constitutes a “research”


Nature of Data Two types, writings and observations

• Written records and accounts – of past happenings

• Observation for whose transmission description is the best vehicle - at the scene of occurrence

• Observation that are quantified and exist in the form of numerical concepts – in the language of mathematics

• Observation of certain differences and likeness that arise from comparison or contrast of one set of observations with another set of similar observations


Research Methodology (RM) Four kinds of data demand four discrete and

different research methodologies The data dictate the RM Unless fits a given methodology, it fails to be a

research


Research Methodology (RM) 1-1 correspondence between data and RM

• Historical method – for documentary data or literary

• Descriptive survey method (normative survey method) – data from observational situation – physically observed or through questionnaire or poll techniques

• Analytical survey method – quantitative data that need statistics to extract their meaning

• Experimental method – appropriate for data derived from experimental control or pretest-posttest design, two separate groups or one group from which data are derived at two separate intervals


Quantitative vs. Qualitative Research

• Quantitative research – is used to answer questions about relationships among measured variables with the purpose of explaining, predicting, and controlling phenomena. This approach is sometimes called the traditional, experimental, or positivist approach

• Qualitative research – is used to answer questions about the complex nature of phenomena, often with the purpose of describing and understanding the phenomena from the participants’ point of view. This approach is also referred to as the interpretative, constructivist, or postpositivist approach



• Both approaches involve similar processes (e.g., formation of one or more hypotheses, review of the related literature, collection and analysis of data). Yet these processes are often combined and carried out in different ways, leading to distinctly different research methods

• Quantitative researchers usually start with a specific hypothesis to be tested, isolate the variables they want to study, control for extraneous variables, use a standardized procedure to collect some form of numerical data, and use statistical procedures to analyse and draw conclusions from the data



• Qualitative researchers often start with general research questions rather than specific hypotheses, collect an extensive amount of verbal data from a small number of participants, organise those data into some form that gives them coherence, and use verbal descriptions to portray the situation they have studied


Qualitative vs. Quantitative Research

• A quantitative study usually ends with confirmation or disconfirmation of the hypotheses that were tested

• A qualitative study is more likely to end with tentative answers or hypotheses about what was observed. These tentative hypotheses may form the basis of future studies (perhaps quantitative in nature) designed to test the proposed hypotheses

• In this way, qualitative and quantitative approaches represent complementary components of the research process – appropriate for answering different kinds of questions

• As a result, we learn more about the world when we have both quantitative and qualitative methodologies


Qualitative vs. Quantitative Research

Summary of Qualitative vs. Quantitative Approaches

Please refer to Table 5.1 on page 96 of the textbook

Deciding whether to use a quantitative or qualitative approach

Please refer to Table 5.2 on page 106 of the textbook


Other RMs Action Research Case and Field Study Research Correlational Research Developmental Research Ex Post Facto or Causal-Comparative Quasi-Experimental Research


Other RMs Please refer to Table 5.3 on page 108 of the

textbook


Question Quantitative Qualitative

What is the purpose of the research?

To explain and predictTo confirm and validateTo test theory

To describe and explainTo explore and interpretTo build theory

What is the nature of the research process?

Focused Known variablesEstablished guidelinesPredetermined methodsSomewhat context-freeDetached view

HolisticUnknown variablesFlexible guidelinesEmergent methodsContext-boundPersonal view


Question Quantitative Qualitative

What are the data like, and how are they collected?

Numeric dataRepresentative, large

sampleStandardized

instruments

Textual and/or image-based

dataInformative, small sampleLoosely structured or

nonstandardised observations

and interviews

How are data analyzed to determine their meaning?

Statistical analysisStress on objectivityDeductive reasoning

Search for themes and categoriesAcknowledgment that

analysis is subjective and

potentially biasedInductive reasoning

How are the findings communicated?

NumbersStatistics, aggregated

dataFormal voice,scientific

style

WordsNarratives, individual quotesPersonal voice, literary style


Research Design Must answer four fundamental questions with

respect to data – no reservation• What are the data needed? – What data are

mandatory? What are their nature? Documentary? Statistical? Interview? Questionnaire? …

• Where are the data located?

• How will the data be secured?

• How will the data be interpreted?

sak 5090 mohd hasan selamat- chapter 5slide 1 chapter 5 planning the research project l successful...

Documents

successful research

research project

research process

research falls

planned research

stated research problem

research design rd

detailed slide