Using Concept Maps to Measure Changes in Knowledge

Kathy McKean, PhDKelly Langley, PhD

Oklahoma Technical Assistance Center

Why Concept Maps?

• There are so many changes going on in Oklahoma schools, we need measures that are specific to the ITQ projects.

• We need measures that don’t take a lot of time out of the PD summer experience.

• We need measures that don’t cost very much.• A measure with a solid theoretical foundation

is preferable to project-developed measures.

Original Sources

• Sarah Hough’s work: http://renew.education.ucsb.edu/downloads/ConceptMapArticlevForWeb.pdf

• http://renew.education.ucsb.edu/downloads/CMStatisticsPaper.pdf

…which was based on the work of others, such as:• Novack, J. D.(1998). Learning, creating and using

knowledge: Concept maps as facilitative tools in schools and corporations. Mahwah, NJ: Erlbaum

• Morine-Dershimer, G. (1993). Tracing Conceptual Change In Preservice Teachers. Teaching and Teacher Education, 9(1), 15-26.

The Central Concept

• For teachers:– The concept should reflect the most important

changes in knowledge you expect to see from teachers

– Can focus on pure content knowledge, pure pedagogy, or a combination of the two (e.g., “Teaching Algebra”).

– Don’t make the topic too narrow. – Try creating a concept map yourself to determine

whether you’ve selected a topic that is rich enough.

The Central Concept

• For students –– Focus on a unit, not an entire year’s work.– Select something that most of your teachers will

teach, such as a unit on functions or fractions, or force and motion. The best topics would be those that teachers plan to teach differently because of their participation in your project.

– This should not be merely a lesson, but a unit of instruction.

Concept Map Administration

• Make sure each participant has a clean sheet of paper and a PENCIL (with an eraser).

• Ask participants to place in the upper right corner (or on the reverse side) the following:– Personal Identifier– Date– Session Site (for students, Grade Level)– School District Affiliation (optional)

• Ask if participants are familiar with concept maps.• Tell them, just to make sure that we all start on the same page,

you will demonstrate a short practice one on….teenagers.• Draw the center circle with TEENAGERS in the middle. Ask

them to tell you something about teenagers. Accept answers that are appropriate and draw them in. After you have 4-6, let them know that this is one type of map, called a spider map.

TEENAGERS

Moody

Expensive!

Social

• Some people like to draw their maps other ways….pick one of the Level 1 concepts and ask them to elaborate. Take their suggestions and show elaboration.

TEENAGERS

Moody

Expensive!

Social

Talk all the time Texting

Cars

Clothes

Food!

• Tell them there are lots of other ways to show the relationships among concepts. They are free to construct whatever concept map design they like best, but it is very important that they draw a circle around each concept and draw a line to connect concepts.

• Tell them they will have 10-15 minutes to draw their maps. It’s important that they know that the expectation is that they will work on them a bit.

What NOT to do

• Pre-maps -- Do NOT tell them that they will be doing this same task later. Do not bias the pre-maps by hurrying them or by telling them “not to make their pre-maps too good.” (Yes, this has really happened.)

• Post-maps – Do NOT give them their pre-maps (or remind them of what they drew then). Do not give them more time or more encouragement than you did on the pre-map.

Quantitative Scoring (Structural Analysis)Term Definition

Concept Individual idea depicted by a circle or box

Level 1 concept A concept joined directly to the central concept

Level 2, Level 3, etc. concepts

Concepts on succeeding levels

Chunk A group of linked concepts in which the leading concept has at least 2 successors on the next level

Chain A line of concepts, that begins at Level 1 and goes to at least Level 3

TEENAGERS

Moody

Expensive!

Social

Talk all the time Texting

Cars

Clothes

Food!

Gas

Payment

Insurance

Eat out all the time

Recording Concept Map DataTeacher ID Total

Concepts (Pre)

Pre Level 1

Pre Level 2

Pre Level 3

Pre Level 4

Pre Level 5

Recording Concept Map Data (continued)

Pre-Width Pre-Depth Pre HSS Pre Chunks

Pre Chains

Pre Crosslinks

•Width – Number of concepts in the biggest level.•Depth – Number of levels in the longest chain.•HSS – Hierarchical Structure Score. • HSS = Width + Depth• Measures the complexity of the structure.•Crosslinks – Links that join 2 chunks.

Recording Concept Map Data

• You record the same things for the pre- and post-maps

• May use a pre-->post-->post design• Critical features of data collection:– Teachers use same identifiers on pre and post– Teachers put the date on their maps

• Independence in scoring – don’t look at the pre-data when you are scoring the post-maps

Last Things

• Dealing with frivolous or repetitive responses– Important that you make an a priori decision

• The occasional unscorable map• This is only the quantitative scoring– You may also wish to look at your maps

qualitatively– You should see qualitative differences if the

project leads teachers to reorganize the way they think

More Last Things

• Initially, have 2 scorers score each map– Once you are scoring consistently, you won’t need

2 scorers– You will occasionally have maps that will require 2

scorers, just because they are difficult to score• Post-It Notes are great for scoring!

Post-It Notes

ConceptsIIIIIIIVVChunksChainsCrosslinks

Table 5. Pre-post comparisons, participant concept maps. (N=344)

Score Category Mean N Std. Deviation Significance

Pair 1Pre- Total Concepts 17.64 344 10.71

YesPost- Total Concepts 24.41 344 11.73

Pair 2Pre-Width 10.56 344 5.99

YesPost-Width 14.41 344 7.18

Pair 3Pre-Depth 2.52 344 1.06

YesPost-Depth 2.92 344 1.13

Pair 4Pre-HSS 13.04 344 6.40

YesPost-HSS 17.31 344 7.43

Pair 5Pre-Chunks 3.42 344 3.46

YesPost-Chunks 4.46 344 3.09

Pair 6Pre-Chains 0.90 344 1.29

YesPost-Chains 1.41 344 1.50