crafting and validation of instrument

Construction and Validation of Research

Instrument

Flerida S. PajarillagaEPS II- HRD

By the end of this seminar-workshop, the participants should be able to: Understand the usual steps in

crafting a data-gathering tool Be able to apply the principles in

question construction Conduct basic validation of created

instrument for action research

Instrument Design Designing an

instrument is challenging and a complex process.

Consider whether your variables are measurable

Consider modifying an existing instrument

TYPICAL STEPS IN INSTRUMENT DESIGN

Write different types of questionsUse Strategies for

Good Question construction

1

2

Perform a pilot test of the questions

3

Write Different Types of Questions

1

Personal, Attitudinal and Behavioral Questions

Sensitive Questions Open-and Close-

Ended Questions


Three popular Types

Other possible descriptors include:

• Exemplary, Proficient, Marginal, Unacceptable

• Advanced, High, Intermediate, Novice

• Beginning, Developing, Accomplished, Exemplary

• Outstanding, Good, Satisfactory, Unsatisfactory


Three popular Types

Behavior Did you take a semester off during any of your four years in college?______ Yes______ No

Sensitive Questions This is usually written when the respondent has “warmed up” answering neutral questions Place leading questions like: Instead of :

Have you taken drugs?

Ask:In past surveys, many men reported at some point in their lives they have taken marijuana. This could happen during adolescence or as an adult. Have you ever smoked Marijuana?

Open- and Closed-Ended Questions

CLOSE ENDED QUESTIONSThe researcher poses a question

and provides preset responses or optionsEXAMPLE:There are many reasons why adults wish to get more education. What is your most important reason for coming to adult basic education classes (Check one.)___________ To be able to help my children with their schoolwork___________ To get a better job___________ To improve myself___________ To get a high school equivalent diploma


OPEN ENDED QUESTIONSquestions for which researchers do

not provide options, participants provide responses

EXAMPLE:

Why are you attending adult education classes?____________________________________________________________________________________________________________


SEMI-CLOSED-ENDED QUESTIONShas the advantage of open-and close-ended questions

EXAMPLE:There are many reasons why adults wish to get more education. What is your most important reason for coming to adult basic education classes (Check one.)___________ To be able to help my children with their schoolwork___________ To get a better job___________ To improve myself___________ To get a high school equivalent diploma___________ Other (please comment) _______________________

Use Strategies for Good Question construction

2

Pay attention to the quality of your questions

It should be clear/ unambiguous that it does not confuse participants Consider cultural background and avoid gender-biased words

Eliminate the problems

Problem no. 1:

Do you support gun control?

_____ YES_____ NO_____ Don’t Know

Do you believe that guns do not belong in school?

_____ YES_____ NO_____ Don’t Know

UNCLEAR QUESTIONS

Problem no. 2:

Do you believe that guns and knives do not belong in school ?

_____ YES_____ NO_____ Don’t Know

Do you believe that knives do not belong in school?

_____ YES_____ NO_____ Don’t Know

MULTIPLE QUESTIONS

Problem no. 3:

Whenever violence occurs in schools, weapons are typically found in school lockers. Do you believe that students should have gun in the lockers?

_____ YES_____ NO_____ Don’t Know

Should students have guns in their lockers?

_____ YES_____ NO_____ Don’t Know

WORDY QUESTIONS

Problem no. 4:

Students should not carry weapons and not have them in their lockers. Do you Agree?

_____ Strongly Agree_____ Agree_____ Undecided_____ Disagree_____ Strongly Disagree

Should students have guns in their lockers?

_____ YES_____ NO_____ Don’t Know

NEGATIVELY WORDED QUESTIONS

Problem no. 5:

Should students pack a .45 at school?

_____ YES_____ NO_____ Don’t Know

Should students have handgun in their lockers?

_____ YES_____ NO_____ Don’t Know

QUESTIONS WITH JARGON

Problem no. 6:

How many times have you seen a student carry a handgun?

_____ 0 times_____ 1-2 times_____ 2-3 times_____ More than 3

times

How many times have you seen a student carry a handgun?

_____ 0 times_____ 1-2 times_____ 3-4 times_____ More than 4

times

OVERLAPPING RESPONSES

Problem no. 7:

To what extent do you feel that handguns are a problem at your school

_______ A great extent_______ Some_______ Not very

important_______ Not a problem at all

To what extent do you feel that handguns are a problem at your school

_______ A great extent_______ Some extent_______ Little extent

UNBALANCED RESPONSE OPTION

Problem no. 8:

How often have you seen students carry a semi-automatic weapon?

_____ None _____ 1 time_____ 2 times _____ 3 or more times

How many times have you seen a student carry a riffle at school?

_____ None _____ 1 time_____ 2 times _____ 3 or more times

MISMATCH OF THE QUESTION AND THE ANSWERS

Example no.1

Problem no. 8:

How many students have you seen carrying guns at school?

_____ 1 student_____ 2 students _____ 3 students______ More than 3

students

MISMATCH OF THE QUESTION AND THE ANSWERS

Example no.2

Have you seen students carrying guns at school?

_____ Yes _____ NoIf Yes, how many

students?_____ 1 student_____ 2 students _____ 3 students______ More than 3 students

Problem no. 9 & 10:

OVERLY TECHNICAL LANGUAGE

QUESTIONS NOT APPLICABLE TO ALL PARTICIPANTS

Example:What would be the appropriate care for children with Progeria?

What database would be most user-friendly for school-base paperless database system?

____FoxPro _____ Lotus123 _____Access

Pilot Test Questions 3

Pilot test is a procedure in which the researcher makes changes in an instrument based on feedback from small number of participants who evaluated the questions. The participants makes comments while taking the test and the researcher makes changes

Exclude this participants on the final sample of the study

ARE YOUR INSTRUMENTS RELIABLE

AND VALID?

Individual scores should be nearly the same or stable over repeated administration of the instrument and should be free from measurement errors

RELIABILITY

Types Reliability : Test re-test, Alternate Forms, Interrater and the split half

The intended test interpretation of the concept or construct that the test assumes to measure matches the proposed purpose.

VALIDITY

Types of Validity: Criterion-related (predictive and concurrent) validity, Content validity and construct validity

STEPS IN CONSTRUCTING A STANDARDIZED INSTRUMENT (Benson & Clark, 1983)

PHASE 1Planning

PHASE 2Construction

Identify purpose of test and target groups

Define Domains of the Test Review literature on construct/

variable Select item format Develop Table of Specifications Validate content – have judges

complete qualitative evaluation Revise Items

PHASE 3Quantitative Evaluation

PHASE 4Validation

Pilot Test Calculate reliability and item

analysis Revise Items

Administer second pilot test Run item analysis Repeat steps of revision Validate (until satisfied!)

STEPS IN CONSTRUCTING AN INSTRUMENT (Benson & Clark, 1983)

LET’S CREATE OUR TOOL!

WORKSHOP1. Participants will be divided into

groups according to research theme assigned

2. Select a type of tool and construct the instrument (at least 5-10 items) with the following details:

(a) Research Title/ Question (b) Participants of the Study (c) Question Items

WORKSHOP

4. Each group will have a partner who will critique (face /content validate ) their work

5. Revise the items

School Principals’ Influence on Trust

.

RESEARCH STUDY No. 1

Reference:Sheldon, D.L, Angell, M.E., Stoner, J.B. & Roseland, B.D., (2010) . School Principals’ Influence on Trust: Perspective of Mothers of Children with Disabilities. Journal of Educational Research 103, 159-170

Trust Worthy Leadership Matrix(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)

Trustworthiness is the willingness to be vulnerable to another on the basis that the other is benevolent, honest, open, reliable and competent

Trust Worthy Leadership Matrix(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)

5 Functions of a LeaderVisioning creating a trustworthy environmentModeling Continuity between word and

example. Lead quietly, exercising restraint and modesty

Coaching assist people to move forward through conversation - know when to push and to back off.

Managing delegate control, high-trust= fewer rules, Trust with parents = less time explaining actions.

Mediating Dealing with betrayal and conflict skillful in conflict management

Sample: Table of Specification

Visioning Modelling

Coaching

Managing Mediating

I encourage my team to participate when it comes decision making time and I try to implement their ideas and suggestions.

I will be the first one to follow the rules set.

I enjoy coaching people on new tasks and policies

Nothing is more important than accomplishing a goal or task.

When correcting mistakes, I do not worry about jeopardizing relationships.