so how well does this stuff work? extensive research validates use of visual tools… * reading...

So how well does this stuff work? Extensive research validates use of visual tools…

* Reading comprehension* Vocabulary acquisition* Writing fluency & ideation* Content-area learning

High-achievingTypical-achievingLow-achievingLearning Disabilities

TM

differentiated visual tools

Large N quantitative studies (true- & quasi-experimental designs) Qualitative studies Program Evaluations

Typical measures include…General performance on high-stakes testsPerformance in specific skills (writing) and vocabularyDepth / Breadth / Accuracy of new content knowledgeSocial validity (teacher & student satisfaction)

Fidelity & factors that affect it

For example….

TM


Teaches same 2nd mini-unit using traditional guided note-taking / discussion instruction

Teaches 1st mini-unit using traditional guided note-taking / discussion instruction

Teacher A

Changes in students’ knowledge about the mini-unit topic are measured at end of each mini-unit

Teacher B

32 Typical Achieving 32 Low Achieving 16 Students w/LD

WEEK 1 WEEK 2

32 High Achieving

Teaches 2nd mini-unit using

Teaches same 1st mini-unit using

TM


American History

HA

First, we measured how much new knowledge of history High Achieving students typically gain when teachers use traditional content instruction methods.

This allowed us to establish the “high water” line.

Text-based, guided note-taking / class discussion

TM

TM


American History

HAHA

TA

HA

Then, we measured how much new knowledge of history Typical Achieving students usually gain when teachers use traditional content instruction methods.

This allowed us to establish the “typical amount” line.

21% more Gap between High & Typical Achievers

High Achieving students tend to gain 21% more knowledge than do Typical Achieving students from the same lesson.

TM


American History

HAHA

TA

HA

Then, we measured how much new knowledge of history Low Achieving students typically gain.

LA

Typical-achievers typically gain 29% more knowledge from a traditional lesson than do Low Achievers.

29% more Gap between Typical & Low Achievers

TM


American History

HAHA

TA

HA

Then, we measured how much new knowledge of history students with LD typically gain from the same ”traditional” lesson.

LA

Typical-achievers typically gain 27% more knowledge from a traditional lesson than do students with LD.

27% more Gap between Typical & Students with LD

LD

TM


American History

HAHA

TA

HA

LA

So now we know what to expect in terms of differences in gains in knowledge about a topic among different types of students when business-as-usual teaching techniques are used on the same history lessons.

LD

TM


American History

HAHA

TA

HA 4 % lessWhen DVTs is used, Typical Achievers increase their learning to within 4% of what High Achievers learned when traditional instruction is used.

TM


American History

HAHA

TA

HA

LA

4 % less

When DVTs is used, Low Achievers increase their learning to within 9% of what Typical Achievers learned when traditional instruction is used.

9 % less

TM


American History

HAHA

TA

HA

LAWhen DVTs is used, Students with LD increase their learning to within 11% of what Typical Achievers learned when traditional instruction is used.

11 % less

LD

TM


American History

HAHA

TA

HA

LA

At first glance, it seems like DVTs is a powerful tool for “reducing the achievement gap.”

11 % less

LD

9 % less

4 % less

The reality is that ALL students greatly enhanced their knowledge when teachers used DVTs

TM


American History

HAHA

TA

HA

LA LD

So the effects of DVTs on the “gap” is to “raise the bar…”

All students are significantly increasing their knowledge of history…

But the gaps remain!

…and even get wider!

TM


American History

This study compared the relative impact of….

Generic Graphic Organizers

Text Resources

“Business as Usual”

VS.

…on depth, breadth, and accuracy of new history knowledge

96 11th grade students…

32 high-achieving, 32 typical achieving, 16 low-achieving, & 16 low-achieving w/ LD

VS.

Essential Understandings

VS.

Generative Idea

TM


depth / breadth / accuracy of knowledge

CLASS 1

PHASE 1 PHASE 2 PHASE 3 PHASE 4

Teacher employed a different strategy during each phase of instruction.

TEXT RESOURCES

GENERIC GRAPHIC ORGANIZERS

GENERATIVE IDEA ESSENTIAL UNDERSTANDINGS

TM



Pre & Post measures of students’

Breadth of knowledge

Depth of knowledge

Accuracy of knowledge

CLASS 1


Learning measured for each phase

TEXT RESOURCES


GENERATIVE IDEA ESSENTIAL UNDERSTANDINGS

TM



CLASS 1


2nd teacher used same strategies, but in a different order

CLASS 2

TEXT RESOURCES


GENERATIVE IDEA Visual Tools

ESSENTIAL UNDERSTANDINGS

TEXT RESOURCES


TM

GENERATIVE IDEAESSENTIAL UNDERSTANDINGS

TM



CLASS 1


CLASS 2

CLASS 3

TEXT RESOURCES




Visual Tools


Visual Tools

TEXT RESOURCES



CLASS 4



TEXT RESOURCES


Visual Tools

TEXT RESOURCES




Visual Tools

3rd & 4th teachers used same strategies, also in different orders

TM



Analysis of pre- and post-mini-unit student generated concept maps relative to pre-established criterion maps provided 3 scores (Ruiz-Primo, Shavelson & Shultz, 1997):

Accuracy of Relational Understanding ScoreBreadth of Relational Understanding ScoreDepth of Relational Understanding Score

Students orally explained links between the terms & asked to elaborate on concepts relating to the proposition identified.

3 types of oral prompts were provided to encourage elaboration: Prompts to summarize important ideas about the topicPrompts to relate or apply ideas Prompts to think about the idea in a new way (what if…)Each proposition scored 0-3, depending on whether the proposition appeared on the criterion map, whether the student’s explanation was accurate, factually correct and complete, and the degree of elaboration provided.

TM



ANOVA statistical analysis was used to determine the relative impact of each of the four instructional approaches relative to level of student achievement.

Generic G.O.s

EU

GI

Text Resources

High Achieving

Typical Achieving

Low Achieving

Learning Disabilities

Tukey’s HSD was used for post hoc analysis.

TM

TM



Tukey’s HSD was used for post hoc analysis.In other words, if a treatment worked well with one type of student, it worked equally well with the other types (& vice versa).

ANOVA of Accuracy of Knowledge Scores by Level of Student Achievement and Form of Instruction

Generic G.O.s

EU

GI

Text Resources

High Achieving

Typical Achieving

Low Achieving


… no significant differences

TM

TM




Generic G.O.s

EU

GI

Text Resources

High Achieving

Typical Achieving

Low Achieving


TM

ANOVA of DEPTH of Relational Understanding Scores by Level of Student Achievement and Form of Instruction

Significant contrasts:

Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric Gos > Text Resources

EU Visual Tools had the greatest impact on all students depth of relational understanding.

TM




Generic G.O.s

EU

GI

Text Resources

High Achieving

Typical Achieving

Low Achieving


TM

ANOVA of BREADTH of Relational Understanding Scores by Level of Student Achievement and Form of Instruction


Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric Gos > Text Resources

EU Visual Tools had the greatest impact on all students depth of relational understanding.

TM



Source Sum of Squares df Mean Square F pLevel of Ach. (A) 7990.415 2 3995.207 159.078 .000Form of Inst. (B) 740.245 3 246.748 9.825 .000A X B Interaction 48.095 6 8.016 .319 .927A X B within error 9041.321 360 25.113

Post Hoc Analysis of Depth of Relational Understanding Scores- Tukey’s HSD


Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric GOs > Text Resources

ANOVA of Depth of Relational Understanding Scores by Level of Student Achievement and Form of Instruction

Significant differences were found between the 4 techniques relative to students with and without LD. For students with LD, the EU & Generic Graphic Organizers had the greatest impact on their depth of relational understanding.

Use of traditional-text-based instruction was the least effective.

Depth, breadth, & accuracy of new 11th grade history knowledge

Source Sum of Squares df Mean Square F pLevel of Ach. (A) 1481.321 2 740.6607 126.603 .000Form of Inst. (B) 227.494 3 75.831 12.962 .000A X B Interaction 7.553 6 1.259 .215 .927A X B within error 9041.321 360 5.850

Post Hoc Analysis of Depth of Relational Understanding Scores- Tukey’s HSD


Scores Using Scores UsingEU Visual Tools > GI Visual ToolsEU Visual Tools > Text ResourcesGeneric GOs > GI Visual ToolsGeneric GOs > Text Resources

ANOVA of Breadth of Relational Understanding Scores by Level of Student Achievement and Form of Instruction

Significant differences were found between the 4 techniques relative to students with and without LD. EU Visual Tools had the greatest impact on breadth of relational understanding. Use of GI Visual Tools & traditional-text-based instruction were the least effective.

Depth, breadth, & accuracy of new 11th grade history knowledge

* Essential Understanding Visual Tools* Generative Idea Visual Tools* Traditional Guided Note-taking

Qualitative data, via semi-structured interviews of teachers and students were collected and analyzed.

Teachers taught history units employing instruction featuring each of the four instructional approaches.

* Generic Graphic Organizers

Qualitative analysis of history teachers’ & students perceptions of different types of visual tools

Sample comments:Teacher: “I could use these (Generic GOs) every day. They really help organize the material. When the students used these to take notes, it really helped them see what was important and how things relate”

Student: “The (Generic GOs) point to the main topics instead of going into long, long notes. It is better than pages of notes.”

Generic Graphic Organizers with embedded prompts about the information structure (hierarchic, compare/contrast, cause/effect, and/or sequence) were perceived as useful by teachers because they helped differentiate the curriculum and organize material.

Emergent themes…


Sample comments:Teacher: “These (EU Visual Tools) make the information real and personal to the student. It reminds them that history is interconnected and can even apply to their own lives.”

Student: ““These (EU Visual Tools) make it so that you don’t forget what it’s all about. You look at the questions, then the whole topic comes back to you and you say ‘oh yea, I remember that from our notes’.”

Emergent themes…


Essential Understanding Both teachers and students perceived that the EU Visual Tools with embedded prompts related to topic-specific essential understandings and prompts to engage in specific information processing skills help make the concepts addressed during the history instruction applicable for the students.

Generative IdeaTeachers and students viewed the Generative Idea Visual Tools somewhat differently.

Teachers perceived them as complex and perceived that their students did not understand the “big ideas”.

Sample comment:Teacher: “Maybe they would be good for 12th grade second semester students; but my students did not understand these. They did not get the big picture that was trying to be expressed.”


Sample comments:Teacher: “The concepts in these (GI Visual Tools) were not difficult, they were just new. We’ve never taught anything like this.”

Teacher: “I personally don’t have time to teach concepts not on the graduation exam.”

Teacher: “They (the students) needed more experience working with these (GI Visual Tools) and these concepts.”


Generative Idea Teachers’ value of the GI Visual Tools seemed to be impacted by:

• The novelty of teaching generative ideas;

• Perceived misalignment between generative ideas and what they perceived to be objectives in the state course of study.

Generative Idea Students, however, valued the GI Visual Tools and appreciated the direct instruction in the generative ideas related to a topic.

Sample comment:Student: “These (GI Smart-sheets) tell you what is important to understand. It spells it right out.”

TM


97%

POST

81%

PRE

11TH grade typical achievers

71%

PRE

11th grade students w/LD

17 pt gain 26 pt gain

98%

POST

TM


11th grade history vocabulary

Typical 8th graderN=20

117 words

CONTROL group of students w/LD

N = 28

EXPERIMENTAL group of students w/LD

N = 28

FLUENCY

Pretest Post-test Pretest Post-testTypical 8th grader produces an average of 117 words on high-stakes essays.

8th students with LD typically produce about 27 words on the same task.

Experimental Group Students w/LD

29 words

26 words

Control Group Students w/LD

TM


Writing fluency: 8th grade students with LD

Typical 8th graderN=20

117 words

CONTROL group of students w/LD

EXPERIMENTAL group of students w/LD

FLUENCY

Pretest Post-test Pretest Post-test

29 words

26 words

24 words

126words

+97 words more than pretest

+9 words more than typical 8th grader

Business-as-usual DVTs

TM


Writing fluency: 8th grade students with LD

9 Schools Extremely low performance

20.6%before DVTs

8 Schools Low performance

38.83%before DVTs

7 Schools Moderate

performance

58.39%before DVTs

2 Schools Good

performance

73.02%before DVTs

% of students meeting or exceeding standards BEFORE schools started implementing DVTs

TM


Impact on AYP Writing Assessment

DVTs

49.24%

DVTs

62.41%

DVTs

74.81%

DVTs

82.68%% of students meeting or exceeding standards AFTER schools started implementing DVTs

20.6%before DVTs

38.83%before DVTs

58.39%before DVTs

73.02%before DVTs

+ 23.58 pts. +16.42 pts.+ 28.64 pts. +9.66 pts.

9 SchoolsExtremely Low Performing

8 Schools Low Performing

7 Schools Moderate Performing

2 SchoolsHigh Performing

TM



So how well do these work?

Results consistently show that DVTs …FAR better than “business as usual” (control groups)

Significantly better than generic graphic organizers

Teachers and students HIGHLY value them

TM



DVTs implementation Year 1

51.4336.1

22.0

28.08

10.0 62.24

81.0

71.0

% students meeting or exceeding standards

+ 23

+ 52

+30School #1

School #2

Year 2 Year 3 Year 4Business-as-usual

+ 9

TM


Semi-rural 5th grade Alabama Writing Assessment

Non-DVTs High School 53%

DVTs High School 77%

26 pt gain

2 pt gain

DVTs implementation

51%

51%

Both groups performed at the same levels in Year 1

Year 1 Year 2


TM


Semi-rural 5th grade Alabama Writing Assessment

Year 1 Year 2 Gains

32.71 57.84 +25.13 Rural 7th GradeSuburb 7th Grade 38 61 +23.00


Business-as-Usual DVTs

TM


7th grade Alabama Writing Assessment

so how well does this stuff work? extensive research validates use of visual tools… * reading...

Documents