educational evaluation
TRANSCRIPT
ASSESSMENT OF LEARNING 2CHAPTER 6
WEST VISAYASSTATE UNIVERSITY – EXTENSION CAMPUS ATHIMAMAYALAN CITY
JOLIETO CAPARIDA, BPE-SPEMADELYN VIDAJA, BSED - ENGLISH
Characterized as the systematic determination of merit, worth and significance of something or someone.
Characterize and appraise subjects of interest in a wide range of human enterprises, including the Arts, business, computer science, criminal justice, engineering,
foundations and non-profit organizations, gov’t., heatlthcare, and other human services.
EVALUATION is the next stage in the process
A systematic, continous & comprehensive process of determining the growth and progress of the pupil towards objectives or values of the
curriculum.(micro/classroom level)
Book 1 and most of Chapter s 1 through 5 (Advance Method Book) Concerns themselves w/ assessment
A. EDUCATIONAL EVALUATION
B. EVALUATION APPROACHES
C. EVALUATION METHODS AND
TECHNIQUES
D. THE CIPP EVALUATION MODEL
E. SUMMARY OF KEYWORDS AND
PHRASES
Joint Committee on Standards for Educational EvaluationDeveloped standards for educational programmes, personnel, and student evaluation.
Four (4) Sections1.) Utility 3.) Propriety 2.)Feasibility 4.) Accuracy
Philippine Society for Educational Research and Evaluation (PSERE) *A society which looks into educational evaluation.
A. EDUCATIONAL EVALUATION
Dept. Of Education (DepEd)They mainly set the Educational evaluation standards in the Philippines.
• Various European Institution
• More or less related to those produced by the Joint Committee in the United States.
• They provide guidelines about basing value judgmentts on
• a. systematic inquiry• b. evaluator competence and
integrity• c. respect for people, and• d. regard for the general and
public welfare.
3. Integrity/Honesty
1. Systematic Inquiry
2. Competence
4. Respect
for People
GUIDING PRINCIPLES (for evaluators)
Created by American Evaluation Association
Can be used at various levels: (Served as Benchmarks for good practices in educational evaluation)
1. Institutional Level when we evaluate learning2. Policy Level when we evaluate institutions
3.International Level when we rank/evaluate the performance of various institutions of higher learning
SYSTEMATIC INQUIRY
Evaluators conduct systematic, databased inquiries about whatever
is being evaluated.
Inquiry cannot be based on pure hearsay or perception but must be based concrete evidence and data to support the inquiry
process.
•Evaluation consulting and design•Designing and administering data collection tools•Analyzing and reporting evaluation results•Helping organizations use results in program planning
•California Instructional Technology Clearinghouse, Columbus Public Schools•The Software and Hardware Industry•Apple Computer Software Guides•Microsoft Software Guides• IBM Software Guides•Strengths: These booklets are distributed free of charge, and can be useful for learning about the software for a particular platform.•Weaknesses: Reviews are written to favor a particular platform. Reviews may be dated or not comprehensive.
The State Of Children's Software Evaluation--Yesterday, Today And In The 21st Century
COMPETENCE
Evaluators provide competent performance to stakeholders.
The evaluators must be people or persons of known competence and generally acknowledged in
the educational field.
INTEGRITY/HONESTY
Evaluators ensure the honesty and integrity of the entire evaluation
process.
As such, the integrity of authorities who conduct the evaluation process must be
beyond reproach.
RESPECT FOR PEOPLE
Evaluators respect the security, dignity and self-worth of the
respondents, program participants, clients and other stakeholders, w/ whom they
interact.They cannot act as if they know everything but must
listen patiently to the accounts of those whom they
are evaluating.
RESPONSIBILITIES FOR GENERAL AND PUBLIC WELFARE
Evaluators articulate and take into account the
diversity of interests and values that may be related to the general and public
welfare.
Believed that an INDIVIDUAL has a FREEEDOM OF CHOICE• He is UNIQUE
EVALUATION PROCESS• Guided by
Empirical Inquiry• Based on Objective
Standards
ALL EVALUATION• Based on Subjectivist
Ethics• Individual Subjective
experiences
B. EVALUATION APPROACHES
Evaluation approaches are the various conceptual arrangements made for designing and actually conducting the evaluation process.
Today, in educational setting (a. Original, b. Refinements/extensions)
1. LIBERAL DEMOCRACY1st major classification of evaluation
Anchored by House (1990)All major evaluation approaches are based on this common idealogy.
1. UTILITARIANISMWhat is Good is Defined as
that w/c maximizes the happiness of society as a
whole.
2. INTUITIONIST OR PLURALIST
No single interpretation of “the good” is assumed .
Need not be explicitly stated nor justified.
FORMS OF
SUBJECTIVIST ETHICS
EACH ETHICAL POSITION HAS ITS OWN WAYS OF OBTAINING KNOWLEDGEOR EPISTEMOLOGY
EPISTEMOLOGY(Ways of Obtaining Knowledge)
Knowledge is acquired
w/c is capable of external verification
& evidence (intersubjective
agreement)
thru methods and techniques universally
accepted and through the
presentation of data.
The Objectivist EpistemologyIs Associated with the UTILITARIAN ETHICS
The Subjective Epistemology Is Asso. w/ the
INTUITIONIST/PLURALIST ETHIC
It is used to acquire new knowledge based on
existing personal knowledge and
experiences that are (explicit) or are not
(tacit) available for public inspection.
Intersubjectivity emphasizes that shared cognition and consensus is essential in the shaping of our ideas and relations.
Intersubjectivity is "The sharing of subjective states by two or more individuals."
The Objectivist EpistemologyIs Associated with the UTILITARIAN ETHICS
The Subjective Epistemology Is Associated w/ the INTUITIONIST/PLURALIST ETHIC
Tacit Knowledge Unwritten, unspoken, and hidden vast storehouse of knowledge held by practically every normal human being, based on his or her emotions, experiences, insights, intuition, observations and internalized information.
Explicit knowledge It can be readily transmitted to others. The information contained in encyclopedias and textbooks
Used to acquire new knowledge based on existing personal knowledge and experiences that are (explicit) or are not
(tacit) available for public inspection.
House’s approach further subdivides the
epistemological approach in terms of
TWO (2) MAIN POLITICAL PERSPECTIVES1. ELITIST=An Approach in which the idea is to focus on the perspectives of managers and top echelon people and professionals.
2. MASS-BASED = An Approach in which the focus is on consumers and the approaches are participatory.
STUFFLEBEAM and WEBSTERS (1980)
Place approaches into one of
THREE(3) GROUPS ACCDG. TO THEIR ORIENTATIONToward the role of values, an ethical consideration1. THE POLITICAL ORIENTATION (PSEUDO EVALUATION)
Promotes a positive or negative view of an objective regardless of what its value actually might be.
2. THE QUESTION ORIENTATION (QUASI-EVALUATION) Includes approaches that might or might not provide answers specifically
related to the value of an object.
3. THE VALUES ORIENTATION (TRUE EVALUATION) Includes approaches primarily intended to determine the value of some
object.
Classification of approaches for conducting evaluations based on epistemology, major perspective, and orientation
Epistemology(Ethic) Major perspective
Orientation
Political(Pseudo-evaluation)
Questions(Quasi-evaluation)
Values(True evaluation)
Objectivist(Utilitarian)
Elite(Managerial)
Politically controlledPublic relations
Experimental researchManagement information systemsTesting programsObjectives-basedContent analysis
Decision-orientedPolicy studies
Mass(Consumers) Accountability Consumer-oriented
Subjectivist(Institutionalist/
Pluralist)
Elite(Professional)
Accreditation/ certificationConnoisseur
Mass(Participatory)
AdversaryClient-centered
Note. Epistemology and major perspective from House (1978). Orientation from Stufflebeam & Webster (1980).
(1st-2nd)2 pseudo- evaluation approaches presented
major perspective (House)
Orientation (Stufflebeam/W
ebster)
epistemology
When the mentioned concepts are considered simultaneously
Politically controlled
Public relation studies
Pseudo-evaluatio
n approach
es
15 evaluation approachescan be identified in terms
of
They are based on an objectivist epistemology from an elite perspective.
Approach Organizer Purpose Key Strengths
Key Weaknesses
Politically controlled
Treats Get keep or increase influence power or money
Secure evidence advantages to the client in a conflict
Violates the principle of full and frank disclosure
Public relations
Propaganda needs
Create positive public mage
Secure evidence most likely to bolster public support
Violates the principles of balanced reporting, justified conclusions and objectivity
Information obtained trhough politically controlled studies is released to meet the speacial interests of the holder.
POLITICALLY CONTROLLED
Used to paint positive image of an object.
Customers perceive value based on the experiences they received.
PUBLIC RELATIONS INFORMATION
5 of them
Accountability
When the mentioned concepts are considered simultaneously
(3rd-8th)6 Quasi-evaluation approaches use
Take an elite perspective
Experimental
research
Management
info. Sys.’
Testing progra
ms
Objectivesbased studies
Content analysis
Takes a mass perspective
Customer / Constituents Satisfaction Survey
After Sales Customers Service
Enhancing the Quality of Products and Services OfferedCreate More Services and Products that will Benefit the Public
Experimental Research
Causal relationships
Determine causal relationships between variables.
Strongest paradigm for determining causal relationships.
Requires controlled setting, limits range of evidence, focuses primarily on results.
Management information
systemsScientific efficiency
Continuously supply evidence needed to fund, direct, & control programs.
Gives managers detailed evidence about complex programs.
Human service variables are rarely amenable to the narrow, quantitative definitions needed.
Testing programs Individual differences
Compare test scores of individuals & groups to selected norms.
Produces valid & reliable evidence in many performance areas. Very familiar to public.
Data usually only on testee performance, overemphasizes test-taking skills, can be poor sample of what is taught or expected.
Objectives-based Objectives Relates outcomes to objectives.
Common sense appeal, widely used, uses behavioral objectives & testing technologies.
Leads to terminal evidence often too narrow to provide basis for judging to value of a program.
Content AnalysisContent of a communication
Describe & draw conclusion about a communication.
Allows for unobtrusive analysis of large volumes of unstructured, symbolic materials.
Sample may be unrepresentative yet overwhelming in volume. Analysis design often overly simplistic for question.
Accountability Performance expectations
Provide constituents with an accurate accounting of results.
Popular with constituents. Aimed at improving quality of products and services.
Creates unrest between practitioners & consumers. Politics often forces premature studies.
In norm-referenced test interpretation, your scores are compared with the test performance of a particular reference group, called the norm group. The norm group usually consists of large representative samples of individuals from specific populations, undergraduates, senior managers or clerical workers. It is the average performance and distribution of their scores that become the test norms of the group. – (http://www.psychometric-success.com/aptitude-tests/interpreting-test-results.htm)
Design the Experiment
Collect and Analyze Data
Draw Conclusion
Goals and Objectives are similar in that they describe the intended purposes and expected results of teaching activities and establish the foundation for assessment.There are three types of learning objectives, which reflect different aspects of student learning:Cognitive objectives: “What do you want your graduates to know?”Affective objectives: “What do you want your graduates to think or care about?”Behavioral Objectives: “What do you want your graduates to be able to do?”
(http://assessment.uconn.edu/primer/goals1.html)
Print media
Newspaper items, magazine articles, books, catalogues
Other writings
Web pages, advertisements, billboards, posters, graffiti
Broadcast media
Radio programs, news items, TV programs
Other recordings
Photos, drawings, videos, films, music
Live situations
Speeches, interviews, plays, concerts
Observations
Gestures, rooms, products in shops
For a media organization, the main purpose of content analysis is to evaluate and improve its programming. All media organizations are trying to achieve some purpose.
For commercial media, the purpose is simple: to make money, and survive. For public and community-owned media, there are usually several purposes, sometimes conflicting - but each individual program tends to have one main purpose.http://www.audiencedialogue.net/kya16a.html
When the mentioned concepts are considered simultaneously
Decision oriented
Policy studies
Consumer
-oriented
studies
3 approaches
(9th-15th)7 true evaluation approaches are included
They are based on an objectivist epistemology from an elite perspective.
Accreditation/
certification
Connoisseur
4 approaches
Based on subjectivist epistemology from an elite perspective
Adversary Client-centered
Based on subjectivist epistemology from a mass perspective
Most important questions when working with statistics is “Why are we doing this?” Proximate examples for such answers are “To find out if this new drug works better than the established ones” or “To describe the effect of inter-cropping on plant growth” while ultimate answers are “To improve medical treatment” or “To find appropriate cultivation techniques”.
Statistics are complied by an IT department and then given back to the people who initially requested them for interpretation.http://journal.code4lib.org/articles/1275
Cutting Carbon Emissions
As politicians begin to use economic arguments to support cutting carbon emissions, a new study from MIT has confirmed that the flow-on health benefits of emissions reductions could save billions of dollars. In fact, the researchers found that savings from avoiding emissions-related health problems could recoup the U.S. up to 10.5 times the cost of implementing a cap-and-trade program. The results of this study were published this week in the journal Nature Climate Change
http://inhabitat.com/cutting-carbon-emissions-will-pay-for-itself-mit-study-shows/
A service offered by companies that focuses on the internal and external needs of a business's customers. Consumer orientation establishes and monitors standards of customer satisfaction and strives to meet the clientele's needs and expectations related to the product or service sold by the business.
http://www.businessdictionary.com/definition/consumer-orientation.html
CHED ACCREDITATION IN THE PHILIPPINESThe CHED has its scheme of quality assurance when colleges and universities submit themselves to voluntary accreditation through the four accrediting agencies: the Philippine Association of Accrediting Agencies of Schools, Colleges and Universities (PAASCU), the Philippine Association of Colleges and UniversitiesCommission on Accreditation (PACU-COA), the Association of Christian Schools and Colleges (ACSC), the Accrediting Association of Chartered Colleges and Universities of the Philippines (AACCUP), all under the umbrella of the Federation of Accrediting Agency of the Philippines (FAAP).The CHED recognizes only the FAAP-certified accreditation of the four accrediting agencies-without necessarily encroaching on the academic autonomy of the latter.
http://stlinusonlineinstitute.com/yahoo_site_admin/assets/docs/CHED_ACCREDITATION_IN_THE_PHILIPPINES.67223608.pdf
Accreditation is a concept of self-regulation which focuses on self-study and evaluation and on the continuing improvement of educational quality. It is both a process and a result.As a process, it is a form of peer review in which an association of schools and colleges establishes sets of criteria and procedures to encourage high maintenance of standards of education among its affiliate members.As a result, it is a form of certification granted by a recognized and authorized accrediting agency to an educational program or to an educational institution as possessing certain standards of quality which are over and above those prescribed as minimum requirements for government recognition. Accreditation is based upon an analysis of the merits of educational operations in the context of the institution's philosophy and objectives.Membership to PACUCOA is open to all schools that are able to meet the standards and requirements of the agency.
http://www.pacucoa.ph/general_info.htm
The connoisseurship model has two major implications: holistic approach to the analysis and interpretation of data and multiple perspectives in the evaluative tasks.
http://ged550.wikispaces.com/Eisner's+Educational+Connoisseurship+Model
On being connoisseurs and criticsInformal education involves more than gaining and exercising technical knowledge and skills. It depends on us also cultivating a kind of artistry. In this sense, educators are not engineers applying their skills to carry out a plan or drawing, they are artists who are able to improvise and devise new ways of looking at things. http://infed.org/mobi/evaluation-theory-and-practice/
To this end, the approach makes use of teams of evaluators who present two opposing views (these teams are commonly referred to as adversaries and advocates).
These two sides then agree on issues to address, collect data or evidence which forms a common database, and present their arguments.
A neutral party is assigned to referee the hearing, and is expected to arrive at a fair verdict after consideration of all the evidence presented.[4]
Models for adversary evaluations, including
judicial, congressional hearing
and debate models. However, models which
subscribe to a legal-framework are most
prominent in the literature
From the first day of service, and continuing through each and every session, the unique needs of the client are at the core of our treatment model. Trained therapy professionals are dedicated to the mission of HCT and to the clients we serve. http://healthcaretherapies.net/treatment_model.php
Client-Centered Nutrition Education (CCNE) is a style of education that encourages participants to play an active role in their own learning and allows staff to act as a guide or a facilitator.
CCNE provides opportunities for group discussion, incorporates hands-on activities and, best of all, allows participants to share experiences and provide social support to each other.
CCNE makes the learning experience more fun, engaging, and meaningful, not only for participants, but also for staff.
http://www.dshs.state.tx.us/wichd/nut/ccne.aspx
• Evaluation is methodologically diverse using both
C. EVALUATION METHODS AND TECHNIQUES
DETAILED LIST OF METHODS, TECHNIQUES
AND APPROACHES
FOR CONDUCTING EVALUATION
ACCELERATED AGINGACTION RESEARCHADVANCED PRODUCT QUALITY PLANNINGALTERNATIVE ASSESSMENTAPPRECIATIVE INQUIRYAXIOMATIC DESIGN
BENCHMARKING
CASE STUDYCHANGE MANAGEMENTCLINICAL TRIALCOHORT STUDYCOMPETITOR ANALYSISCONSENSUS DECISION-MAKINGCONSENSUS –SEEKINGDECISION-MAKING
CONTENT ANALYSISCONVERSATION ANALYSISCOST-BENEFIT ANALYSISCOURSE EVALUATION
DELPHI TECHNIQUEDISCOURSE ANALYSIS
ELECTRONIC PORTFOLIOENVIRONMENTAL SCANNINGETHNOGRAPHYEXPERIMENTEXPERIMENTAL TECHNIQUES
FACTOR ANALYSISFACTORIAL EXPERIMENTFEASIBILITY STUDYFIELD EXPERIMENTFIXTURELESS IN-CIRCUIT TESTFOCUS GROUPFORCE FIELD ANALYSIS
GAME THEORYGRADING
HISTORICAL METHOD INQUIRY
INTERVIEW
MARKETING RESEARCHMETA-ANALYSISMETRICSMOST SIGNIFICANT CHANGEMULTIVARIATE STATISTICS
NATURALISTIC OBSERVATION
OBSERVATIONAL TECHNIQUES and others.
• Developed by Stufflebeam (1983)• A very useful approach in educational evaluation• Expanded to CIPPOI Outcome, Impact
D. THE CONTEXT, INPUT, PROCESS, PRODUCT (CIPP) EVALUATION MODEL
The Approach essentially systematizes the way we evaluate the different dimensions and aspects of
curriculum development and the sum/total of student experiences in the educative process.
PROCESS
In this Approach, the user is aked to go through a
series of questions.
INPUTS PRODUCT
CONTEXT
THE ‘CIPP’ MODEL OF EVALUATION
CONTEXT
What is the relation of the
course to other courses?
Is the time adequate?
What are critical or important
external factors (networks, ministries)?
Should courses be integrated or separate?
What are the links between the course and research/exten-sion activities?
Is there a need for a
course?
Is the course relevant to job needs?
What is the entering ability of students?
What are the learning skills of the students?
What is the motivation of the students/
What are the living condiions of students?
What is the students’ existing knowledge(*) (In
line WMF*)?
Are the aims suitable?Is the course content
clearly defined?
Does the content (knowledge, skills, attitudes(*) In line WMF*)
match student abilities
Is the content relevant to practical problems?
What is the theory practice relevance?
What resources/ equipment are
available
What books do the teachers have?
INPUTS
What books do the students have?
How strong are the teaching skills of the
teachers?
What time is available comparedwith the workload,
for preparation?
What knowledge, skills and attitudes, relatede to the
subject, do the teachers have?
How supportive is the classroom environment?
How many students are there?
How many teachers are there?
How is the course organized?
What regulation relate to the training?
Are the objective smart?
Do the objectives derive from aims?
INPUTScontn.
Use and apply
PROCESSWhat is the workload of
student?
How well/actively do students participate?
Are there any problems related
to teaching?Are there any
problems related to learning?
Is there an effective 2-way
communi-cation
Is knowledge only transferred to students,
or do they use and apply it?
Are there any problems w/c students face in
using/applying/analysing the knowledge and
skills?
Are the teaching and learning
process continuously evaluated?
Are teaching and learning affected by
practical/institutional problems?
What is the level of cooperation/interpersonal
relations between teachers/students?
How is disciplined maintained?
Is there one final exam ar the end or several during
the course?Has the teacher’s reputation improved or been ruined as a
result?
Is there any informal
assessment?
What is the quality of assessment (i.e. what levels of KSA
are assessed?)
What are the students’ KSA levels after the
course?Is the evaluation carried out
for the whole (*) In-line WMF*) process?
How do students use what they have
learned?
How was the overall experience for the
teachers and for the students?
What are the main ‘lessons’learned’?
Is there an official report?
PRODUCT
Those guided questions are not answered by the teacher only or by a single individual. Instead, there are many ways in which they can be answered. Some of the more common methods are
listed below.
1. Discussion with class2. Informal conversation or
observation3. Individual student
interviews4. Evaluation forms
5. Observation in class/session of teacher/trainer by
colleagues6. Video tape of own teaching
(micro-teaching)7. Organizational documents
8. Participant contract
9. Performance test10. Questionnaire
11. Self-assessment12. Written test
ASSESSMENT is the process of gathering
and analyzing specific information as part of
an evaluation. COMPETENCY
EVALUATION is a means for teachers to determine the ability of their students in
other ways besides the standardize test.
COURSE EVALUATION is the process of evaluating the instruction of a given
course.EDUCATIONAL
EVALUATION is evaluation that is conducted specifically in an
educational setting.IMMAMENT EVALUATION
opposed by Gilles Deleuze to value
judgment.
PERFORMANCE EVALUATION is aterm
from the field of language testing. It
stands in contrast to competence evaluation.
PROGRAM EVALUATION is essentially a set of
philosophies and techniques to determine if a program
‘works’
E. SUMMARY OF KEYWORDS AND PHRASES