TeachLivETM vs. Role-Play:

Comparative Effects on Special Educators’ Acquisition of Basic

Teaching Skills

Melanie Rees Dawson & Benjamin Lignugaris/Kraft

1st National TeachLivETM ConferenceOrlando, Florida, May 2013

Rationale A core goal of teacher preparation programs is

to bridge the gap between knowledge and practice (Allsopp, DeMarie, Alvarez-McHatton, & Doone, 2006; Dieker, Hynes, Hughs, & Smith, 2008; Hixon & So, 2009).

Training experiences are often constructed using situated learning as a theoretical foundation.

Situated Learning: Knowledge acquisition requires realistic content

and complexity Skill transfer depends on how closely practice

opportunities match the situation in which information is to be applied

(Brown, Collins, & Duguid, 1989)

RationaleSimulating classroom scenarios is one strategy for situating learning for novice teachers.

Role-playing is perhaps the oldest form of classroom simulation, dating back to the 1800’s (Brown, 1999).

Recently, virtual simulations have emerged in teacher training programs (Hixon & So, 2009). TeachLivETM is a virtual classroom that can

realistically represent the complexities that exist in actual classrooms (Dieker et al., 2008).

RationaleTeachLivETM is an example of “technology-enhanced learning” in contrast to “traditional” approaches. Three types of Technology-Enhanced Learning

used in university settings (Kirkwood & Price, 2013):

(1) Replicating existing teaching practices: delivering instruction using technology

(2) Supplementing existing teaching practices: creating additional resources or making resources/tools available for students to access at any time

(3) Transforming the learning experience: Redesigning activities to provide active learning opportunities for students

Purpose The purpose of this study was to

investigate the effectiveness of practice sessions in TeachLivETM (a technology-enhanced approach) compared to practice sessions in role-play (a traditional approach) on preservice special educators’ development of essential teaching skills.

Research QuestionsStudy 1: (Foundation teaching skills)

1) To what extent will preservice special educators, who are trained to deliver opportunities to respond (OTR) and praise, demonstrate a higher response rate for the skill practiced in TeachLivETM Gen 1 than the skill practiced in role-play with colleagues, when assessed in TeachLivETM Gen 3?

2) To what extent will teachers who practice praise in TeachLivETM Gen 1, demonstrate a higher percentage of specific praise than teachers who practice praise in role-play with colleagues, when assessed in TeachLivETM Gen 3?

Study 2: (Complex teaching skills)

3) To what extent will preservice special educators, who are trained to deliver error correction and praise around, demonstrate a higher percentage of correct steps for the skill practiced in TeachLivETM Gen 1 than the skill practiced in role-play with colleagues, when assessed in TeachLivE Gen 3?

Methods: Participants 7 teachers in an Alternative Teacher

Preparation (ATP) program Full-time special education teachers on letters

of authorization Given $150 for participating in the study Two Groups:

3 teachers practiced praise in TeachLivETM and OTR in role-play (Group 1)

4 teachers practiced OTR in TeachLivETM and praise in role-play (Group 2)

Methods: Participants

Methods: MeasuresDependent Variables OTR: The teacher asks an academic question and indicates if it is

directed to the group or an individual student

OTR rate= # OTR/mins (Goal= 4 per min)

Praise: Positive teacher statements and gestures referring to student work or behavior General: positive statements or gestures that don’t specifically

reference student work or behavior Specific: positive statements that directly reference student work

or behavior

Praise rate= # praise statements/mins (Goal= 4 per min)% Specific praise= (# specific/total praise) x 100 (Goal= >50%)

Methods: MeasuresAssessment: Immediately following training sessions conducted in either TeachLivETM Gen 1 or role-play

Generation 3 classroom Complexity:

4 academic errors (e.g. mispronouncing words, incorrect definition, poor example)

4 problem behaviors (e.g. cell phone ringing, talking out, tapping pen)

5 minute assessment session administered individually (Recorded and scored later)

Measures: ReliabilityInterobserver Agreement:

Primary Data Collector: 100% of assessment videos Second Data Collector: 31% of assessment videos

OTR:Rate: 88.1% (71.4%-100%)

Praise:Rate: 88.4% (70.8%-100%)Type: 96.5% (82.4%-100%)

IOA= Agreements/(Agreements+Disagreements) x 100

Methods: InterventionPrior to Practice Sessions: Target Skill Instruction

Video about each target skill Handout with examples/non-examples Quiz

Lesson Plans Vocabulary Lesson (words & definitions) Teaching Formats (e.g. example/non-example, definition,

sentence generation, sentence substitution) A short story that included the target vocabulary words 1 lesson plan for TeachLivETM, 1 lesson plan for role-play

Methods: Intervention

Training vs. Assessment

TeachLivETM Gen 1 or role-play TeachLivETM Gen 3

No misbehaviorsNo academic errors

4 misbehaviors4 academic errors

Training vs. Assessment

TeachLivETM Gen 1 or role-playNo misbehaviors

No academic errors1 lesson for TeachLivETM 1 lesson for role-play

A new lesson for each assessment session

4 misbehaviors4 academic errors

TeachLivETM Gen 3

Methods: DesignAlternating Treatments Design Treatments alternated between TeachLivETM and Role-Play Target skills were counterbalanced across treatments and

groups

Advantages of an Alternating Treatments Design: Quickly compare effectiveness of two treatments Does not require baseline data or withdrawal Minimizes sequence effects

(Cooper, Heward, & Heron, 2007)

TeachLivETM Role-PlayGroup 1

Praise OTR

Group 2

OTR Praise

Results

Results

Results: Specific PraisePraise

TargetedPraise

TargetedPraise

Targeted

Study 2: MeasuresNew Dependent Variables:Error Correction: When a student makes an academic error the teacher delivers a model, test, and delayed test.

Model: The teacher provides the correct answer to the question Test: The teacher repeats the initial question Delayed Test: After one or more intervening responses, the teacher

again asks the initial question.% correct Steps: (# correct steps/#possible steps) x 100

Praise Around: When a student exhibits a misbehavior that is persistent (5 s or more) or recurring (more than once) the teacher delivers the following steps:

Step 1: The teacher praises another student who is exhibiting the desired behavior. The praise statement must be specific and identify the behavior that is incompatible with the problem behavior.

Step 2: When the target student exhibits the desired behavior, the teacher praises the student using a specific praise statement that is incompatible with the problem behavior.

% correct Steps: (# correct steps/#possible steps) x 100

Study 2: Intervention

Study 2: DesignAlternating Treatments Design (with baseline)

Baseline data collected during Study 1 Treatments alternated between TeachLivETM and

Role-Play Target skills were counterbalanced across

treatmentsTeachLivETM Role-Play

Group 1

(Praise)Praise Around

(OTR)Error Correction

Group 2

(OTR)Error

Correction

(Praise)Praise Around

Study 2: Results

Study 2: Results

SurveysWhen both studies were complete, we administered two surveys: Presence Questionnaire:

Surveyed the “realness” of the Gen 1 and Gen 3 classrooms, students, interactions, and teaching scenarios. (Adapted from a survey created by Aleshia Hayes)

Social Validity: Surveyed the acceptability of the training

procedures, and compared teacher experiences in TeachLivETM and Role-Play.

Presence QuestionnaireFindings: Majority of teachers understood the students’ different

personalities Teachers were split on their perceptions of “realness” of the

classroom and students, as well as their ability to interact as they would in a real classroom.

Perceptions of “realness” depended on the level to which TeachLivETM matched their own classroom.

Overall, the teacher’s degree of buy-in was the same for Gen 1 and Gen 3.

The visual proximity to the students (zooming in and out) enhanced interactions in some situations, but may have hindered it in others.

Social ValidityFindings: The training and lesson materials were

appropriate The feedback structures were helpful TeachLivETM is more similar than role-play to

their own classrooms

Social ValidityTeachLivETM: 3 Role-play: 4

• “for a more real-life encounter”• “I enjoyed the comradery of role-play,

but it did not put you on the spot as much as TeachLivE.”

• “TeachLivE gave a variety of behaviors compared to role-play (more students).”

• “engaged the whole time”• “face-to-face interaction”• “enjoyed working with colleagues”• “practice more on academic issues than

behaviors”

Question: Which lab did you prefer? Why?

Question: If you were offered $50 for mastering a new skill in ONE session, would you choose to practice the skill in the TeachLivETM lab or the role-play lab? TeachLivETM: 5 Role-play: 1 “I don’t know”: 1

*Although teachers were split on their buy-in and preference for TeachLivETM, they recognized that is it a powerful medium for practicing teaching skills.

Discussion Overall, teachers demonstrated higher

rates and percentage of steps in the assessment session on the skills they practiced in TeachLivETM compared to the skills they practiced in role-play. These results suggest that TeachLivETM

(technology-enhanced simulation) facilitates development of essential teaching skills more effectively than role-play (traditional simulation).

DiscussionLimitations: No Training Data

Did teachers master the skill during training before generalizing to the assessment setting?

No Classroom Data To what extent do teachers demonstrate the

skills in their own classroom?

DiscussionFuture Research: Focus on TeachLivETM

Understand how many sessions it takes for teachers to become proficient with a specific skill, without being interrupted by other repertoires (multiple baseline)

Collect training data as well as assessment data Generalization to real classrooms

Regularly collect data in their own classrooms to investigate to what extent the levels of proficiency we see in TeachLivETM transfer to actual teaching.

Select more homogeneous participants with similar teaching situations to TeachLivETM

Secondary schools Teaching language arts

ReferencesAllsopp, D. H., DeMarie, D., Alvarez-McHatton, P., & Doone, E. (2006). Bridging the gap between theory and practice: Connecting courses with field experiences. Teacher Education Quarterly, 33(1), 19-35.Brown, A. H. (1999). Simulated classrooms and artificial students: The potential effects of new technologies on teacher education. Journal of Research on Computing in Education, 32(2), 307–18.Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 18(1), 32-42.Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied Behavior Analysis (2nd ed.). Prentice Hall: New Jersey.Dieker, L., Hynes, M., Hughes, C., & Smith, E. (2008). Implications of mixed reality and simulation technologies on special education and teacher preparation. Focus on Exceptional Children, 40(6), 1-20.Hixon, E., & So, H.-J. (2009). Technology’s role in field experiences for preservice teacher training. Educational Technology & Society, 12(4), 294–304.Kirkwood, A., Price, L. (2013). Technology-enhanced learning and teaching in higher education: What is ‘enhanced’ and how do we know? A critical literature review. Learning, Media, and Technology, DOI:10.1080/17439884.2013.770404

Thank You! Melanie Dawson

melanie.rees@usu.edu801-505-3290

Ben Lignugaris/Kraftben.lig@usu.edu435-797-2382