cresst’s evaluation of the artful learning program: “findings,” contexts, and future...
TRANSCRIPT
CRESST’s Evaluation of the Artful Learning Program: “Findings,” Contexts, and
Future Explorations
Noelle Griffin,Ph.DUCLA Graduate School of Education & Information
StudiesNational Center for Research on Evaluation,Standards, and Student Testing (CRESST)CRESST Conference, September 10, 2004
Artful Learning Program
Arts-based education model: Arts infused across the curriculum
Currently implemented in 7 states, 22 schools
Four phases: Experience, Inquire, Create, Reflect
Use of Masterworks of art
Where Does ArtfulLearning Fit?
Comprehensive school reform
Instructional, organizational, assessment components
Focused on helping kids improve academic performance
Depends upon trained teachers and administrators
Theory of Action for Artful Learning
The underlying theory motivating the Artful Learning model is that the joy, discipline, and commitment required by the arts will provide a useful framework for the overall improvement of education and for individual growth
Evaluation Background
CRESST/UCLA
Beginning in 2001
Both external and internal applications
Qualitative and quantitative strategies
Emphasis: Capacity building
Evaluation Methods
Yearly implementation survey (participating schools, n=@400 per year)
Pre-post professional development surveys (on-going)
Teacher/administrator interviews (n=@30 per year)
School-level achievement data
Supplemental information
Academic Achievement
Student Engagement/Motivation
QualityImplementation
Organization And
LeadershipClassroom Instruction
Connections Processes
Professional Development/Support
District/School Context
Contextual Factors
Need for both administrative and district support
Middle/high school vs. primary
Teacher/student turnover
Role of arts teachers/specialists
Based on qualitative data
Professional Development and Support
Overall quality/utility high (quantitative and qualitative)
Consistency across levels/experience
Differences across program components
Professional Development Quality: Percent “Agree” or “Strongly Agree”
0
20
40
60
80
100
Val
id P
erce
nt
Usefulknowledge or
skills
Links topractice
Well-connectedcontent
L1
L2
Significant Pre/Post Development Increase in Reported Teacher Expertise
(MANOVA)
Level I and Level II
Artful Learning Instructional Process
Assessment Practices
Organizational/structural Practices
General Instructional Quality
How Well Did Development Prepare You to Implement Classroom Component?
Level 1
Not at all
Slightly
Somewhat
Adequately
More than adequately
Level 2
Not at all
Slightly
Somewhat
Adequately
More than adequately
How Well Did Development Prepare You to Implement Assessment Component?
Level 1
Not at all
Slightly
Somew hat
Adequately
More than adequately
Level 2
Not at all
Slightly
Somewhat
Adequately
More than adequately
Implementation Survey Multiple Regression Results: Implementation of
Artful Learning Classroom Process
Artful Learning Process
Standards Use/Understanding
Assessment Use/Understanding
High Quality Instruction
Reported Impact
Shared Mission
*
*
*
*
*
*=Significantly Predicts p<.05
Multiple Regression: No Significant Relationship to Implementation
Artful learning experience
Teaching experience/Teacher background
Parent involvement
Shared leadership
Implementation: Other Findings
Teacher evaluation of program after use is high
Continued growth of program/use over time
Assessment/parent involvement weakest components
Based on Qualitative and Quantitative Data
Additional Findings: Shared Efficacy
Teacher perceptions of school’s overall effectiveness in teaching process
Student achievement links
Group competency and task analysis components
Included in implementation survey
Multiple Regression Findings: Shared Efficacy
Shared LeadershipShared Leadership
Years ExperienceYears Experience
Grade Level TaughtGrade Level Taught
Combined Shared Efficacy
Combined Shared Efficacy
Group CompetencyGroup Competency
Task AnalysisTask Analysis
*All Significant relationships are negative*No significant relationship to other Artful learning implementation variables
Student Outcomes
Engagement and other teacher-reported outcomes
Achievement
Quantitative and qualitative consistency
Access to information – what teachers “don’t know” re: assessment results
Teacher Reported Program Impact: Student Engagement
Level of Increase
0
20
40
60
80
100
Val
id P
erce
nt
No Impact
Slightly
Moderately/Greatly
Teacher Reported Program Impact: Quality of Student Work
Level of Increase
0
20
40
60
80
100
Val
id P
erce
nt
No Impact
Slightly
Moderately/Greatly
Teacher Reported Program Impact: Classroom Tests/Assessments
Level of Increase
0
20
40
60
80
100
Val
id P
erce
nt
No Impact
Slightly
Moderately/Greatly
School-Level Achievement Data: Limitations
Cohort group issues
Dilution of effects
Test changes/lack of comparability
Insufficient information statistically
Tentative Findings: Comparative Average Growth in Students
Meeting/Exceeding Standards
GRAMMY DISTRICT COMP. SCHOOLS
READING 16 10 9
ELA 9 8 5
MATH 12 9 6
CRESST Project Synergy: QSP use in Artful Learning
QSP: Quality School Portfolio assessment use software
Address program difficulties collecting achievement data in multiple districts/states
Build project self-evaluation capacity
Potential QSP Applications for Artful Learning:
Track student-level data
Compare participating vs. non-participating students
Compare students based on “dosage”
Compare teachers based on program experience
Enduring Questions Related to Arts Education
What should we value as attainment of serious learning?
Can arts programs in schools be sustained as political and parental pressure increasingly focuses on traditional academic attainments?
What should count as quality education?
How scalable is change away from a compartmentalized view of learning?
How can such change be sustained and expanded?