Responsive Pedagogy Questionnaire

- the developing and validation process

Bringing Teacher Education Forward

08.06.2016

Siv M. Gamlem, Volda University College

Knut Steinar Engelsen, Stord Haugesund University College

Lars M. R. Kvinge, Stord Haugesund University College

Ann Karin Sandal, Sogn and Fjordane University College

Kari Smith, Norwegian University of Science and Technology

“Responsive Pedagogy & Student Learning”- Examine relationship between teachers’ responsive

teaching (feedback practice) and pupils’ learning

outcome in Mathematics

Research question

What is the relation between

responsive pedagogy (feedback

practice) and student learning

(achievements, self-efficacy and self-

regulation)?

Mixed Methods (questionnaire, interview,

video observation, achievement test, -intervention)

Goal for this presentation:- To present the developing and validation process of the Responsive Pedagogy Questionnaire

Responsive Pedagogy – Feedback

Self-regulation skills

Self-efficacyMathematics

Pupils (9th grade)

Le

arn

ing

ou

tco

me

Responsive Pedagogy Questionnaireexamine:

- pupils’ perceptions of self regulation

skills, self-efficacy and feedback

practice in Mathematics

Self-regulation• “Self-regulated learning is an active

constructive process whereby learners set goals for their learning and monitor, regulate, and control their cognition, motivation, and behaviour, guided and constrained by their goals and the contextual features of the environment” (Pintrich & Zusho,2002, p. 64).

• Three phases: 1) forethoughts, 2) performance-volitional control, 3) self-reflection (Zimmerman, 2000).

Self-efficacy

• “Perceived self-efficacy refers to beliefs in one`s capabilities to organize and execute the courses of action required to produce given attainments” (Bandura, 1997,p.3).

• Self-efficacy beliefs are domain specific, and they differ according to physical and social context (Zimmerman, 2000; Bandura, 1997; Skaalvik & Skaalvik,

2004).

Feedback - Responsive Pedagogy

• Hattie & Timperley (2007):

• «Where am I going?»

• «Where am I?»

• «What to do next?»

• Responsive Pedagogy is the recursive dialogue

between the learner’s internal feedback and

external feedback provided by significant others

throughout the three phases of self-regulation;

forethoughts, monitoring and reflection (The RespMath

team, 2015).

A need for «bridges» – pupils’ perceptions

Achievement in

Mathematics

Feedback

practice

Self-

regulation

Self-efficacy

RPQ-M builds on validated instruments• the Norwegian version of the Cross Curricular

Competences (CCC)/PISA-survey

• the Motivated Strategies for Learning Questionnaire

(MSLQ) (Pintrich et al., 1991)

• the Achievement Emotions Questionnaire -

Mathematics (AEQ-M) (Pekrun, Goetz & Frenzel, 2005)

• the Student Perception Assessment survey (SPA) (Havnes et al. 2012)

Theoretical model of construct in questionnaire

Cluster (I): «Forethought»

• FT (18 items)

– FTSR, 4 items; FTFB, 4 items; FTSE, 10 items

– (FTFB) Forethought feedback refers to when a learner seeks answers to prospective learning goals – “where am I going?” These questions can refer to the perceptions of quality of one’s work or learning processes, depending on how they are asked and answered.4 items 4 items 10 items

Cluster (II): «Performance and control»

• PC (29 items)

– PCSR, 8 items; PCFB, 13 items; PCSE, 8 items

– (PCFB) Performance and control feedback refers to when learners seek answers to monitor learning in relation to target (learning goals) – “how am I doing?” These questions can refer to the perceptions of quality of one’s work or learning processes in relation to the target/learning goals.

8 items 13 items 8 items

Cluster (III): «Reflection»

• R (32 items)

– RSR, 7 items; RFB, 7 items; RSE, 18 items

– (RFB) Reflection feedback refers to when learners seek answers to retrospective and prospective learning goals –“what’s next?” These questions can refer to the perceptions of quality of one’s work or learning processes, depending on how they are asked and answered.

7 items 7 items 18 items

First pilot (January 2016)

• Closed-end questions

• 81 items

• Properly organized; easy to use

- Fatigue (?)

• Item stem/readability (Johnsen & Christensen, 2012)

- «Liks» (Björnsson,1968), scale 1-60 (Score 4-45)

• 4-point rating scale (Dillman, 2007)

«agree» – «partly agree» – «partly disagree» – «disagree»

First pilot-test (January 2016)

• Participants: n=10 (9th graders, one school)

– Individually answered on paper

– 2 focus-group interviews (5 participants each)

• Results:

– Rating scale: «partly agree» – «partly disagree» =

the same scale? (changed)

– Fatigue: Similar question asked «before» (add lead-in instructions in sections of questionnaire)

– Item: three removed, one divided in two

– Language: unfamiliar words (changed)

Second pilot (June 2016)

• Questionnaire: 79 items

• Demographics (at the end) (Johnson & Christensen,

2012)

– gender, parents’ education

– «value & expectations from parents» (items

integrated in questionnaire)

• 4-point rating scale (Dillman, 2007)

«strongly agree» – «agree» – «disagree» – «strongly

disagree»

• Participants: n=198 (8 removed – missing data)

Second pilot-test (June 2016)

• Test/establish reliability & validity: Coefficient

alpha (Cronbach’s alpha)

.56 .80 .91 .69 .77 .73 .78 .89 .90

.89 .86 .92

Second pilot-test (June 2016)

• Test/establish reliability & validity: Coefficient

alpha (Cronbach’s alpha)

Self-regulation Feedback Self-efficacy

.77 .92 .95

Second pilot-test (June 2016)

• Factor analysis:

– some items set are multidimensional (different

sets of item tap different construct)

• Adjust items-set; remove some items (strive for

homogeneity)

• Demographics: challenging (missing data)

– “highest education (mother/father)”

– “occupation (mother/father)”

– “grade in mathematics”

Further process

• Demographic data

– item stem?, questions?, instruction?...

• Reduce items (?) (factor analysis)

• Choose design for analysis:

– «Clusters of concept» – «theorethical model»?

Responsive Pedagogy – Feedback

Self-regulation

Self-efficacy skillsMathematics

RespMath Questionnaire- Pupils (9th grade); experimental group (n=1000)- Pupils (9th grade); control group (n=1000)

Learn

ing o

utc

om

e

The RespMath project

• Homepage:

http://prosjektsider.hsh.

no/respmath/

• Facebook: Responsive

Pedagogy in

Mathematics

• Twitter: @RespMath

- Andrade, Heidi L. (2010). Students as the Definite Source of Formative Assessment: Academic Self-Assessment and

the Self-Regulation of Learning. Nera Conference Proceedings 2010, Paper 25.

http://digitalcommons.uconn.edu/nera_2010/25

- Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioural change. Psychological review, 84(2), 191-

215.

- Björnsson, C. H. (1968). Läsbarhet. Stockholm: Liber

- Butler, D. & Winne, P. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational

Research, 65(3), 245-281.

- Hattie, J. & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.

- Havnes, A., Smith, K, Dysthe, O., & Ludvigsen, K. (2012). Formative assessment and feedback: Making learning

visible. Studies in Educational Evaluation, 38, 21-27.

- Pekrun, R., Goetz, T. & Frenzel, A.C. (2005). Achievements Emotions Questionnaire – Mathematics (AEQ-M) User’s

manual. Department of Psychology, University of Munich.

- Pintrich, P.R. (2000). The role of goal orientation in self-regulated learning. In: M. Boekaerts, P.R. Pintrich, & M.

Zeidner (Eds.), Handbook of Self-Regulation (pp.452-502). San Diego, CA: Academic Press.

- Pintrich, P.R., Smith, D.A.F., Garcia, T., & McKeachie, W.J. (1991). A Manual for the Use of the Motivated Strategies

for Learning Questionnaire (MSLQ). Ann Arbor, MI: National Center for Research to Improve Postsecondary Teaching

and Learning.

- Usher, E.L. & Pajares, F. (2008). Sources of Self-efficacy in School: Critical Review of the Literature and Future

Directions. Review of Educational Research, 78(4), 751-796.

- Zimmerman, B. J. (2000). Self-regulatory cycles of learning. In G.A. Straka (Ed.), Conceptions of Self-Directed

Learning: Theoretical and Conceptual Considerations (pp.221-34). New York: Waxmann.

- Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory into Practice, 41(2), 64-70.

References