test construction and evaluation
DESCRIPTION
Test Construction and Evaluation. Lizamarie Campoamor-Olegario. Purposes of Classroom Tests. Establish basis for assigning grades Determine how well each student has achieved the course objectives Diagnose student problems for remediation Determine where instruction needs improvement. - PowerPoint PPT PresentationTRANSCRIPT
Test Construction and EvaluationLizamarie Campoamor-Olegario
Lizamarie Campoamor-Olegario
Purposes of Classroom Tests
• Establish basis for assigning grades
• Determine how well each student has achieved the course objectives
• Diagnose student problems for remediation
• Determine where instruction needs improvement
Lizamarie Campoamor-Olegario
Test or Scale Development
• Step 1. Definition of Construct• Step 2. Table of Test Specifications• Step 3. Item Writing• Step 4. Content Validation including Factor Analysis• Step 5. Pilot Testing to Equivalent Group• Step 6. Item-analysis, reliability analysis• Step 7. Final Administration• Some Reminders
Lizamarie Campoamor-Olegario
DEFINITION OF THE CONSTRUCT
Step 1.
Lizamarie Campoamor-Olegario
Step 1. Definition of Construct
• Literature review
• Scientifically defined constructs
• Popular definition
• Syllabus
back
Lizamarie Campoamor-Olegario
TABLE OF TEST SPECIFICATIONS
Step 2.
Lizamarie Campoamor-Olegario
Step 2. Table of Test Specifications
• Subject: _______________
• Grade/ Year Level: _____
• Type of Test: Cognitive (Knowledge), Psychomotor (Skills), or Affective (Attitudes)?
• Testing Time: _____
• Type of Item/s: Likert? Or rating?
• Number of items: ____Lizamarie Campoamor-Olegario
Knowledge (Cognitive)
• (Source: Bloom, B., Taxonomy of Educational Objectives, 1956)
Lizamarie Campoamor-Olegario
Levels of Cognitive Domain (Knowledge)• Knowledge
– The remembering of previously learned material (recall of facts)
• Comprehension– The ability to grasp the meaning of the knowledge
being learned
• Application– The ability to use learning materials in a new way
Lizamarie Campoamor-Olegario
Levels of Cognitive Domain (Knowledge)• Analysis
– The ability to break material down into its parts so that its organizational structure may be understood
• Synthesis– The ability to combine previous experiences with
new material to form a whole new structure
• Evaluation– The ability to judge the value of material for a given purpose
Lizamarie Campoamor-Olegario
Knowledge Can students RECALL information?
Key Questions
• Who
• What
• When
• Where
• How
• How much
• Describe
• Define• Memorize• Literal questions• Which one• Name• Label• List• Reproduce• Recall
Lizamarie Campoamor-Olegario
Knowledge Can students RECALL information?
• Example of Knowledge QuestionWhich of the following are raw materials of photosynthesis? a)Water, heat, sunlight
b)Carbon dioxide, sunlight oxygen
c)Water, carbon dioxide, sunlight
d)Sunlight, oxygen, carbohydrates
e)Water, carbon dioxide, carbohydrates
Lizamarie Campoamor-Olegario
Knowledge Can students RECALL information?
• What position does Mr. Chirac hold in France?
a)President
b)Prime Minister
c)General Secretary
d)Attorney General
Lizamarie Campoamor-Olegario
Comprehension Can students EXPLAIN ideas?
Key Questions:
• Explain
• Describe in your own words
• Inferential questions
• Summarize
• What would go better
• Select the definition
• Read the graph table
• This represents
• Condense this paragraph
• What part doesn’t fit
• What are they saying
• Explain what is happening
• Give an example
• State in 5 words
• Explain what is meant
• What restriction would you add
• Translate
• Outline
• Locate
• MatchLizamarie Campoamor-Olegario
Comprehension Can students EXPLAIN ideas?
• Example of Comprehension Question• If living cells similar to those found on earth were
found on another planet where there was no molecular oxygen, which cell part would most likely be absent? a) Cell membrane
b) Nucleus
c) Mitochondria
d) Ribosome
e) Chromosomes
Lizamarie Campoamor-Olegario
ApplicationCan students USE ideas?
Key Questions: • What is this used for?• How would you use• Make a model• Tell what would happen• If…how• Demonstrate how• Construct how• Show how
• How much would there be if…
• Design a lesson• Choose the statements
that don’t apply
Lizamarie Campoamor-Olegario
ApplicationCan students USE ideas?
• Example of Application Question
• Phenylketonuria (PKU) is an autosomal recessive condition. About one in every fifty Individuals is heterozygous for the gene but shows no symptoms of the disorder. If you select a symptom-free male and a symptom –free female at random, what is the probability that they would have a child afflicted with PKU? – (.02)(.02)(.25) = 0.0001 = 0.01%, or about 1/10,000
– (.02)(.02) = 0.0004 = 0.04%, or about 1/2,500
– (1)(50)(0) = 100% = all
– (1)(50)(0) = 0 = none
– 1/50 = 2%, or 2/100
Lizamarie Campoamor-Olegario
ApplicationCan students USE ideas?
• It can be inferred from the passage the Mr. Chirac believes that the greatest impact of strict visa regulations is that they:
a) help the police to subdue French terrorist organizations
b) help the police to locate foreign terrorists operating in France
c) reduce the number of foreign terrorists entering France
d) give more power to anti-terrorist policeLizamarie Campoamor-Olegario
Analysis Do students SEE relationships?
• Whole into parts• Analyze• Research/ Survey• Group/ Categorize• Compare and Contrast• What inconsistencies,
fallacies• Arrange• What is the relationship
• Chart• What is the function of• Diagram• What conclusions• Reason for…• What does the author
believe• Investigate• Make a distinction
Lizamarie CampoamorM-Olegario
Analysis Do students SEE relationships?
• Cause for• What motive is there• Conclude• State the point of view• Separate• Graph • Differentiate• Dissect
• Distinguish fact from fiction
• fact and inference• fact from opinion• advantage from
disadvantage• good from poor reason• What persuasive
technique
Lizamarie Campoamor-Olegario
Analysis Do students SEE relationships?
• Example of Analysis QuestionMitochondria are called the powerhouses of the cell because they make energy available for cellular metabolism. Which of the following observations is most cogent in supporting this concept of mitochondrial function? a) ATP occurs in the mitochondria
b) Mitochondria have a double membrane
c) The enzymes of the Krebs cycle, and molecules required for terminal respiration, are found n mitochondria
d) Mitochondria are found in almost all kinds of plant and animal cells
e) Mitochondria abound in muscle tissue
Lizamarie Campoamor-Olegario
Analysis Do students SEE relationships?
• Which of the following best explains why countries of the Eurpoean Community have been exempted from France’s new visa regulations?
a) few terrorists operate in those countries
b) France is bound by its international treaties
c) those countries have signed the European Convention on the Suppression of
d) Terrorism
e) France needs the support of those countries in its fight against terrorism
Lizamarie Campoamor-Olegario
Synthesis: Can students combine ideas and CREATE a new entity?
Key Questions• New ways of doing• Take risks• Consider the unexpected• Pose an alternative• Hypothesize• Create/ Build/ Make• Compose• Solve
• Design/ Invent• Blend/ Combine• How else would you• Imagine• Plan• Predict• Link concepts unusual
and flexible way• What if
Lizamarie Campoamor-Olegario
Synthesis: Can students combine ideas and CREATE a new entity?
• Briefly outline an alternative strategy that France could use in order to combat terrorist attacks. Your answer should include
a) a description of the problem
b) your proposal for dealing with it
c) the likely consequences, both negative and positive, of such action.
• Your answer should not exceed 1 page. Suggested time for this question is 20 minutes. 10 points are allocated for this question.
Lizamarie Campoamor-Olegario
Evaluation – Can students make JUDGMENTS and support them?
• Evaluate quality, relevance, reliability, truth, effectiveness
• Choose and explain why• Rate/ Grade• Rank/ Order• Defend• Dispute• Criticize• Find the errors
• Editorialize• Appraise/ Judge • What fallacies,
consistencies, inconsistencies appear
• Which is more important, better, moral, appropriate, inappropriate, useful, clearer
Lizamarie Campoamor-Olegario
Evaluation – Can students make JUDGMENTS and support them?
• Example of Evaluation Question
• Disregarding the relative feasibility of the following procedures, which of these lines of research is likely to provide us with the most valid and direct evidence as to revolutionary relations among different species? a) Analysis of the chemistry of stored food in female gametes
b) Analysis of the form of the Krebs cycle
c) Observation of the form and arrangement of the endoplasmic reticulum
d) Comparison of details of the molecular structure of DNA
e) Determination of the total protein in the cell • Sources:
– http://www.nwlink.com/~donclark/hrd/bloom.html– http://www.fctel.uncc.edu/index.html
Lizamarie Campoamor-Olegario
Evaluation – Can students make JUDGMENTS and support them?
• Write a critique of the language used by Mr. Chirac, as reported in this article, in terms of its political flavor and intended audience.
• Support your discussion with relevant quotes from the passage in addition to examples drawn from other sources.
• Points will be awarded for clarity of thinking, logical argumentation and relevant examples.
• Do not exceed 2 pages. • 26 points are allocated for this question.
Lizamarie Campoamor-Olegario
Skills (Psychomotor)
Lizamarie Campoamor-Olegario
Sample Practice Questions
1. Do you eat breakfast?
____ always
____ sometimes
_____never
2. How often do you eat breakfast during the week?
____ 1 – 2 days per week
____ 3 – 4 days per week
____ 5 – 6 days per week
____ everyday
____ never
Lizamarie Campoamor-Olegario
Sample Practice Questions
3. What do you usually take to have a good breakfast? Give reason. (Please indicate number below).
FOODS REASONS
1. _____________________________________________
2. ____________________________________________
3. ____________________________________________
Reason: 1 – nutritious
2 – tastes good
3 – always available
4 – affordable
5 – others, specify ___________________Lizamarie Campoamor-Olegario
Sample Practice Questions
4. Why do you eat breakfast? (Please check all that apply to you.)
____ I feel better if I eat breakfast
____ I am hungry in the morning
____ my parents told me
____ I enjoy eating
____ I feel it is important to eat breakfast
____ others, specify ___________________
Lizamarie Campoamor-Olegario
Sample Practice Questions
5. If you skip breakfast, how do you feel at mid-morning (9:30-10:00)? Check one that best describes how you feel.
_____ Fine
_____ Angry/bored
_____ Hungry/starving
_____ Headache/sick
_____ Nervous
_____ Easily irritated
_____ Weak
_____ Others, specify: ____________
Lizamarie Campoamor-Olegario
Attitudes (Affective)
Lizamarie Campoamor-Olegario
Sample Attitude Items
• I cringe when I have to go to math class.
• I am uneasy about going to the board in a math class.
• I am afraid to ask questions in math class.
• I am always worried about being called on in math class.
• I understand math now, but I worry that it's going to get really difficult soon.
• I tend to zone out in math class.
• I fear math tests more than any other kind.
• I don't know how to study for math tests.
• It's clear to me in math class, but when I go home it's like I was never there.
• I'm afraid I won't be able to keep up with the rest of the class.
Lizamarie Campoamor-Olegario
Put a check to the number which best corresponds to your answer:
1 – strongly agree 4 - disagree
2 – agree 5 – strongly disagree
3 – neutral
Lizamarie Campoamor-Olegario
1 2 3 4 5
1. Adolescent who eats a wholesome nutritious breakfast do better in school.
2. I prefer to eat breakfast at home to have quality time with my family
3. Drinking milk is of much important as part of adolescent breakfast.
4. Skipping breakfast could lose out on an important nutritional contribution to adolescent’s total daily food intake.
5. Adolescents who take enough exercise will reduce the danger for excessive intake of calories.
6. I know that eating breakfast will improve my attendance and punctuality in class.
7. I believe that if I am able to consume a balanced meal during breakfast, I will have a “healthy day”.
8. If I eat breakfast, it is more likely that I will get the recommended dietary allowance for vitamin A, C, D, and B6, calcium, iron, zinc & folate.
9. If I have “instant noodle”, biscuit or coffee for breakfast, it will be enough to start my day.
10. Washing my hands before eating will keep me safe from food borne disease.
Put a check to the number which best corresponds to your answer:
1 – strongly agree 4 - disagree
2 – agree 5 – strongly disagree
3 – neutral
Lizamarie Campoamor-Olegario
Table of SpecificationsContent area Recall /
RecognitionSkills Comprehension/ Application
Critical Thinking/ Problem Solving
Total Allocation of marks
Biochemistry 3 12 0 15
Cells/Tissues 4 13 3 20
Genetics Reproduction
2 10 3 15
Invertebrates 4 6 0 10
Vertebrates 5 11 4 20
Plant life 2 6 2 10
Ecological 0 7 3 10
Total 20 65 15 100Lizamarie Campoamor-Olegario
ITEM WRITINGStep 3.
Lizamarie Campoamor-Olegario
Step 3. Item Writing
• 1. Keep the Test Short– Try out a longer test on a small number of subjects– If more than 10% fail to finish the test on time…
• 2. Plan to Quantify Your Results.• 3. Good tests must be based on the objectives of a
course. • 4. Each test should meet four criteria: RVDN
– Reliability– Validity– Discrimination– Non-triviality
Lizamarie Campoamor-Olegario
RVDN
• R: Does not generate random answers and consistently results in marks that reflect the skill level of each student.
• V: Measures what it intends to measure. • D: Shows clearly the differences in the levels of
achievement of the students to whom it is administered.
• N: Focuses on that which the students should know and not on that which is irrelevant to evaluation.
Lizamarie Campoamor-Olegario
Step 3. Item Writing
5. Questions must be clearly written.
6. Least complex questions should appear first
7. Reread all the items from the standpoint of the students
8. Eliminate any systematic pattern of answers
9. Guard against cultural, racial, ethnic, and sexual bias
10.Before administering the test, prepare answer keys and scoring procedures
Lizamarie Campoamor-Olegario
Test Formats
• essay
• true-or-false tests
• multiple choice tests
• completion-and-short answer tests
• matching tests
Lizamarie Campoamor-Olegario
Essay Test
• Avoid words such as what, who, when, which trigger closed responses
• Avoid letting them to answer a choice of questions
• Give definitive task to student-compare, analyze, evaluate, etc.
• Use checklist point system to score with a model answer
• Cover the student’s name and the scored items while scoring the exam
• Score one question at a time-all at the same time.
• Rearrange
Lizamarie Campoamor-Olegario
True-or-false tests
• Avoid having more than one idea in a question. • Avoid absolute terms: never, only, all, none, always• Avoid indefinite terms: in most cases, great,
sometimes, generally, some, few • Avoid double negatives. • Use exact quantitative language • Don't lift items straight from the book. • Make more false than true (60/40). • Require students to circle or underline a typed “T”
or “F”, rather than to fill in.Lizamarie Campoamor-Olegario
Multiple choice tests• In order to measure insight, avoid quoting from the textbook.
• Avoid “none of these” or “all of the above”
• Stem should be in simple, understood language; delete extraneous words.
• Make all distracters plausible
• Make all alternatives homogeneous (in subject, content, form, length, explicitness, and grammatical structure)
• Use negatively stated items sparingly. When they are used, visually emphasize the negative word.
• Present alternatives in logical or numerical order.
Lizamarie Campoamor-Olegario
Multiple choice tests
• Make each item independent of others on test.
• Need more than 3 alternatives, 4 is best.
• Ask for the best answer and use terms such as “most” and “primary” if more than one answer is partially correct.
• The reading and linguistic difficulty of items should be low.
• Use care in the repetition of words or phrases between the stem and the correct answer.
• Avoid items that reveal the answer to another item.
• Ordinarily, distracters should not overlap, subsume, or be synonymous with one another.
Lizamarie Campoamor-Olegario
Completion and short answer tests/ Identification
• Limit the number of blanks in each question.
• Generally, blanks are not inserted at the beginning of the question.
• Make sure there is only one unambiguous response for the blank.
• In numerical problems, indicate the type of units in which the answer should appear.
• State the item so that only one answer is correct
• State the items with words that students understand
• Avoid items in which the correct answer is a matter of opinion.
Lizamarie Campoamor-Olegario
Matching tests
• Organize questions together in logical groups.
• In your instructions, indicate whether or not some answers may be used more than once.
• Need 15 items or less.
• Use homogenous material in each exercise.
• Make all responses plausible.
• Put all items on a single page.
• Put response in some logical order (chronological, alphabetical, etc.).
• Avoid grammatical clues
• Provide more choices than premises
Lizamarie Campoamor-Olegario
CONTENT VALIDATIONStep 4.
Lizamarie Campoamor-Olegario
Step 4. Content Validation including Factor Analysis
• Validity denotes the extent to which an instrument is measuring what is supposed to measure.– Forms of Validity– Face Validity– Content Validity– Criterion-related Validity– Construct Validity
Lizamarie Campoamor-Olegario
Face Validity
• Does the test appear valid?
• Subjective judgment of a group generally composed of non-experts
• Includes good layout and format, clarity of directions/ instructions, non-ambiguity of items, correct grammar and sentence structure(syntax), appropriate scales, appropriate number of items, clarity of printing
Lizamarie Campoamor-Olegario
Exercises on Face Validity
The following food groups would consist of a good breakfast.
a)coffee, biscuit, instant noodles
b)fried rice, fried dilis, tomato, banana, milk
c)peanut butter, pan de sal, coffee
d)no response
e)do not know
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• Diego’s mother asked him to go to the grocery and buy ingredients for tonight’s halloween party. What is the best way for him to do to make it on time for dinner? Diego should _______ going to Trinoma.a. walk b. ride the carc. crawl
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• Which of the following causes air pollution?a. planting treesb. smokingc. eating vegetables
• Chenelyn works as a house helper in Vida Mae’s house. Which of the following will help her clean the floor easier and faster?a. vacuum cleanerb. mopc. rag
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• We take care of plants because they give us the following EXCEPTa. give us foodsb. give us shadec. make us sick
• The chef Nathan found it hard to cut the meat into small pieces. What tool is missing from the chef’s kitchen?a. rice cookerb. stovec. knife
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• If you are to make your own machine that will make work fast and easy, what will it be? Draw that machine on the space below. (5 points)
• Your community is very dirty and this cause people in your neighborhood to get sick. What can you do to help solve this problem? Draw the things that you will do to help. (5 points)
Lizamarie Campoamor-Olegario
Exercises on Face Validity
Adolescents need to consume at least ________ piece of fruit during breakfast..
a)1 piece
b)2 pieces
c)3 pieces
d)No response
e)Do not know
Lizamarie Campoamor-Olegario
Exercises on Face Validity
Uriin ang mga sumusunod na pagkain kung ito ay isda o pagkaing dagat, uri ng prutas, o uri ng gulay. Ilagay lamang sa patlang ang letra ng tamang sagot at lagyan ng (/) ang hanay kung ang pagkaing ito ay Yamang Lupa o Yamang Tubig.
Lizamarie Campoamor-Olegario
Exercises on Face Validity
Upang maging malakas at masigla, dapat ay maging aktibo. Lagyan ng star (*) ang mga gawain na nakapagpapalakas ng katawan at (x) kung ito ay hindi masyadong nakapagpapalakas ng katawan.
Lizamarie Campoamor-Olegario
Oo o Hindi. Ilagay sa puwang ang Oo kung ito ay tumutugon sa kakulangan sa pagkain o Hindi kung ito ay hindi makakatulong upang matugunan ang pangangailangan ng mag-anak sa pagkain
______1. Ang tatay ay nangingisda sa dagat.
______2. Ang mag-anak ay naglilinis ng bahay
______3. Ang tatay ay nagpuputol ng kahoy upang gamitin sa pagluluto.
______4. Ang mag-anak ay nagtatanim ng gulay sa paligid
______5. Ang bata ay tumatakbo ng matulin
______6. Ang pamilya ay nag-aalaga ng hayop
_____10. Ang mag-anak ay nagsisimba tuwing Linggo
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• _____ 19. Ang uri ng enerhiya na nagmumula sa tubig ay ang _____________. a) hydroelectric energy b) Solar Energy
• _____ 20. Pinagkukunan ito ng mga arosep, hipon, at marami pang iba. Maliban dito ito rin ang pinagkukunan ng enerhiya. a) lupa b) araw c) tubig
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• ______ 25. Mas mabuting gawing subdibisyon ang lahat ng anyo ng lupa dahil ito ay magbibigay ng maraming pera sa may-ari nito.
• ______ 27. Ang eroplano ay mas madaling bumaba sa kabundukan dahil walang gagambala dito.
• ______ 28. Kinukuha natin mula sa karagatan ang tubig na ating iniinom.
• ______ 29. Sa pagtatanim ng maraming puno ay nakaiiwas tayo sa baha.
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• 36-37. Tuwing umaga, nakikita mong sinusunog ng inyong ate ang mga basura sa inyong bakuran. Alam mong nakadadagdag ang usok mula dito sa pagdumi ng hangin. Ano ang sa tingin mo ang magandang gawin sa mga basura sa inyong bakuran?
• Ano ang pinaka gusto mong gawain o ginagawa ng isang scout? Bakit? (2 puntos)
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• Directions: Look at the pictures. Encircle the letter of the sentence that has the correct subject.
Lizamarie Campoamor-Olegario
Atty. Perez cleans the whole building.
The Chocolate Mountain Ranges are found in Bohol.
Exercises on Face Validity
• Directions: Look at each picture. Encircle the sentence with the correct predicate for the picture.
Lizamarie Campoamor-Olegario
Ms. Edna is a clerk in the hotel.
Ms. Edna is always late for work.
Ms. Edna gives out three keys for free.
Exercises on Face Validity
• 59. and 60. Penny wakes up at __________ every day. While it is still __________, she fixes the table and cooks breakfast for her children.
• a) dawn - dark• b) morning - dark• c) early - cold
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• Directions; Read the paragraph below. Write the correct word that will make the sentences correct. Choose your answers from the box below. (6 points)
• Last night, Pete cannot sleep. Mother told him to count ___________. Soon he fell asleep and had a _______.
Lizamarie Campoamor-Olegario
voice pay dream rain
day sheep boy train
Exercises on Face Validity
• Words with digraphs (ay, ai, ea, ee, oi and oy)• Directions: Encircle the letter of the answer that will
make the sentence correct.– “Do you____ before going to sleep?” Beatrix asked.
• pay b. prayc. prey
– The ______ of Wahbi’s pig is curled.• teyl b. tail c.
tale
Lizamarie Campoamor-Olegario
Exercises on Face Validity
• 13. Hindi lamang si Ate ang gumagawa ng mga _______________. Lahat kami sa pamilya ay tumutulong rin upang mas madali ang mga trabaho. a) takdang-aralinb) gawaing-bahayc) bahay-ampunan
• 19. Sina Ate Coy at Kuya John ay madalas nating matagpuan sa _______________.
• a) silid-aklatan• b) silid-aralan• c) bahay-aklatan
Lizamarie Campoamor-Olegario
Exercises on Face Validity
______2. To be more active, we should eat plenty of Go foods such as: (Upang maging aktibo, dapat tayo ay kumain ng “go foods” tulad ng): a)Bread, rice, cereals (tinapay, kanin at cereals
b)fruits and vegetables (prutas, at gulay)
c)meat, fish, and chicken (karne, isda at manok)
Lizamarie Campoamor-Olegario
Exercises on Face Validity
______3. Grow Foods are important for growth and development, examples of grow foods are: (Ang “grow foods” ay importante para sa paglaki at pag-“develop”ng muscles) a)Bread, rice, cereals (tinapay, kanin at cereals)
b)fruits and vegetables (prutas at gulay)
c)meat, fish, and chicken (karne, isda at manok)
Lizamarie Campoamor-Olegario
Matching Type. Match item A with Item B. Write your answers on the blank provided.
A B
_____ 1. Needed for proper hydration a) banana
_____ 2. Organ which produces insulin b) stomach
_____ 3. Where solid waste in the body is stored
c) digestive system
_____4. High potassium content d) water and other
fluids
_____ 5. Cause of diabetes e) lack of insulinLizamarie Campoamor-Olegario
_____ 6. The long tube between the mouth and stomach
f) mouth
_____ 7. The first part of the digestive system, where food enters the body
g) pancreas
_____ 8. Sac-like muscular organ that is attached to the esophagus
h) esophagus
_____ 9. Promotes health i) proper diet and physical activity
_____ 10. System that consists the organs of digestion
j) rectumLizamarie Campoamor-Olegario
II.Put a check (/) on the diseases of the digestive system if it is caused by the intake of dirty foods or (x) if not
______ 1. Flu ______ 6. Diabetes
______ 2. Diarrhea ______ 7. Sore throat
______ 3. Cough ______ 8. Dysentery
______ 4. Fever ______ 9. vomiting
______ 5. Stomach painLizamarie Campoamor-Olegario
Exercises on Face Validity
What principle of healthy eating is indicated in the following sentences? Put your answer on the blank provided before the number. (Anong mga prinsipyo, na nakasalungguhit, sa pagkain ng wasto ang isinasaad ng mga pangungusap sa ibaba. Ilagay ang iyong sagot sa patlang. a)Balance
b)Variety
c)Moderation Lizamarie Campoamor-Olegario
1. Eating different kinds of food everyday is important to get all the essential nutrients needed by the body. (Ang pagkain ng iba’t ibang pagkain ay nakapagbibigay ng kumpletong sustansiya na kailangan ng katawan),
2. Eating the right amount of go, grow and glow foods (Ang pagkain ng tamang dami mula sa go grow and glow foods)
3. Put less sodium such as vetsin, toyo, patis in food to prevent high blood pressure. (Maglagay lamang ng kaunting dami ng vetsin, toyo,o patis sa pagkain upang maiwasan ang pagkakaroon ng mataas na presyon ng dugo)
4. A meal should consist of rice, protein source such as fish/meat, a piece or slice of fruit, at least half cup of vegetable and a glass of milk (Ang pagkain ay dapat na mayroong kanin, piraso ng karne, isang piraso ng prutas, kalahating tasa ng gulay at isang basong gatas.
5. Too much candies, cakes and chocolates may cause you to have tooth ache. (Ang pagkain ng sobrang matatamis tulad ng kendi, keyk at tsokolate ay nakapagdudulot ng sakit sa ngipin)
Lizamarie Campoamor-Olegario
Identify the correct locomotor movement
_________ 1.A regular succession of leaps a. Run
_________ 2.To advance or move forward on foot with one foot always on the ground floor
b.Jump
_________ 3. To move fast with both feet off the ground
c.gallop
_________ 4. To slip or pass along over a surface d. Slide
_________ 5. to spring from the floor or ground f. LeapLizamarie Campoamor-Olegario
Content Validity
• The degree to which the sample of items, tasks, or questions on a test are representative of some defined universe or domain of content.
• It therefore, indicates whether or not the test items adequately represent that domain.
• Usually a table of specifications is built in order to ensure that the entire domain is represented by the items in a test.
• Expert judgment
Lizamarie Campoamor-Olegario
Content Validity
• Is the instrument comprehensive? What is not there?
• Is it logical• How adequately does the sample of items or
questions represent the content to be assessed?• Consult some experts who rate the items,
eliminating or changing those that do not meet the specified content.
• Repeat until all raters agree on the questions and answers.
Lizamarie Campoamor-Olegario
PILOT-TESTING TO EQUIVALENT GROUP
Step 5.
Lizamarie Campoamor-Olegario
Step 5. Pilot-testing to equivalent group
• If achievement test, to higher level
• Number of items times three
Lizamarie Campoamor-Olegario
ITEM ANALYSISStep 6.
Lizamarie Campoamor-Olegario
Step 6. Item-analysis
• To improve the effectiveness of test items and the validity of test scores
• Select and rewrite items on the basis of item performance data
• Item analysis provides information about:– The difficulty of the item: How easy or hard the item was
– The discriminating power of the item: How well it discriminated between high and low scorers in the test
– The effectiveness of the alternative: Whether all the alternatives served their purpose
Lizamarie Campoamor-Olegario
Steps of an Item Analysis
• S1. Order the N papers by score, placing the one with the highest score on top and continuing sequentially until on with the lowest score is on the bottom. – Note: Where two or more examinees made
the same score, order them alphabetically by surname.
Lizamarie Campoamor-Olegario
Steps of an Item Analysis
• S2. Multiply N, the total number of tests, by 0.27 and round off the result to the nearest whole number, the number is represented by n. If N is 30, n would be 8 (8.1 rounded)
• S3. Count off the n best papers from the top of the stack. This is the “high” group.
• S4. Count off the n poorest papers from the bottom of the stack. This is the “low” group.
Lizamarie Campoamor-Olegario
Steps of an Item Analysis
• S5. Determine the proportion in the high group (pH) answering a particular item correctly by dividing the number of correct answers for the high group by n, that is: Number of correct responses to the item
pH = ----------------------------------------------------------
n
• Repeat the procedure for the low group to obtain pL for each item
Lizamarie Campoamor-Olegario
Steps of an Item Analysis
• S6. To obtain an estimated item difficulty index, p (that is, the proportion of the total group that answered the item correctly), add pH to pL and divide the resulting sum by 2: pH + pL
p = ---------------
2
Lizamarie Campoamor-Olegario
• S7. To obtain a measure of item discrimination (i.e., how well this item distinguishes between the students who understand the content universe of the test and those who do not), subtract pL from pH:
• D = pH-pL
Lizamarie Campoamor-Olegario
Item Number
High or Low
Number of Correct Responses for Groups (n=8)
Proportion of Correct Responses
Item Discrimination
D = pH-pL
Item Difficulty
pH+pL
p = --------
2
1 H 7L 3
2 H 8L 6
3 H 4L 4
4 H 2L 6
n = (0.27) Nn=0.27 (30) = 8.1 (rounded off to 8) students in the high group and 8 in the low group
Item Number
Group Number of Correct Responses for Groups (n=8)
Proportion of Correct Responses
Item Discrimination
D = pH-pL
Item Difficulty
pH+pL
p = --------
21 H 7 pH=7/8=0.8
80.88-0.38 = 0.50
0.88+0.38
2
=0.63
L 3 pL=3/8=0.382 H 8 1.00 0.25 0.88
L 6 0.753 H 4 0.50 0 0.50
L 4 0.504 H 2 0.25 -0.50 0.50
L 6 0.75
Answer
Lizamarie Campoamor-Olegario
Steps of an Item Analysis
• For discrimination, the following table may be used:– 0.40 and above = very discriminating item– 0.30-0.39 = discriminating item– 0.20-0.29 = reasonably discriminating item– 0.10-0.19 = marginally discriminating item,
usually subject to improvement– Below 0.10 = unproductive item for test reliability
purposes
Lizamarie Campoamor-Olegario
Difficulty
• Items more than 90% of the students answered correctly may be too easy
• A low percentage of students answering an item may indicate students ran out of time during the test or they were unsure of the answer and didn’t want to guess.
• Items that fewer than 30% of the students answered correctly may be too difficult.
Lizamarie Campoamor-Olegario
Interpreting Item Analysis
• Good item• Potential miskey• Ambiguous items• Equal distribution to all alternatives• Alternative/s not working• Distracter too attractive• Question not discriminating• Negative discrimination• Too easy• Omit
Lizamarie Campoamor-Olegario
Good item
• * keyed correct answer• O omit or no answer
Lizamarie Campoamor-Olegario
Group A B C* D O
High 0 0 10 0
Low 4 3 7 2
Potential Miskey
Lizamarie Campoamor-Olegario
Group A* B C D O
High 1 1 10 3
Low 4 3 5 3
Ambiguous Items
Lizamarie Campoamor-Olegario
Group A B C* D O
High 6 1 7 1
Low 5 2 5 3
Equally to all alternatives
Lizamarie Campoamor-Olegario
Group A B C* D O
High 4 3 5 3
Low 3 4 4 4
Students are guessing
Alternatives aren’t working
• Each distracter has to attract at least 5% of the students• If two alternatives don't draw any students --> might
consider redoing as true/false
Lizamarie Campoamor-Olegario
Group A B* C D O
High 1 14 0 0
Low 5 7 2 0
Distracter too attractive
• Not one distracter should pull more than about half of students
Lizamarie Campoamor-Olegario
Group A B C D* O
High 7 1 1 5
Low 10 2 1 2
Question not discriminating
Lizamarie Campoamor-Olegario
Group A* B C D O
High 7 3 2 3
Low 7 2 1 5
Negative Discrimination
Lizamarie Campoamor-Olegario
Group A* B C D O
High 7 3 2 3
Low 10 3 1 1
usually means an ambiguous question
Too Easy
Lizamarie Campoamor-Olegario
Group A B* C D O
High 0 14 0 1
Low 1 13 1 1
If you dropping too many questions, you may lose content validity (specs)
Omit
• Possibilities– If near end of the test, they didn't find it because it was on
the next page (format problem)
– Test is too long
– Way too difficultLizamarie Campoamor-Olegario
Group A B* C D O
High 0 14 0 1
Low 1 13 1 1
FINAL ADMINISTRATIONStep 7.
Lizamarie Campoamor-Olegario
Step 7. Final Administration
• Usually, the run is five years
Lizamarie Campoamor-Olegario
Encourage students to evaluate the exam
(adapted from "Let Students Grade the Exam," 1987)• Did the content you expected to see appear on this
exam? • Identify the questions you never expected to see. • Were the questions clear enough that, even though you
may not have known the answer, you knew what was being asked?
• What questions confused you? • Or ask students to give a letter grade to the content,
format, and fairness of the test. Lizamarie Campoamor-Olegario
Returning Examinations
• Return test papers promptly– Within five days
– Laws governing privacy and confidentiality of student records
• Posting of grades by names, initials, student numbers
• Letting students get their test papers
• Discuss overall results
Lizamarie Campoamor-Olegario
Thank you for listening!
Lizamarie Campoamor-Olegario