implementation of the kssr standards-based · pdf filereltmax implementation of the kssr...

18
Implementation of the KSSR Standards-Based Assessment Simone Evans, PhD Training Fellow ELTC Division Ministry of Education Malaysia 019 623 1519 [email protected]

Upload: dangcong

Post on 30-Jan-2018

221 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

Implementation of the KSSR Standards-Based Assessment

Simone Evans, PhD

Training Fellow

ELTC Division

Ministry of Education

Malaysia

019 623 1519

[email protected]

Page 2: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 2

Table of Contents

Rationale for the RELTmax Implementation of the KSSR English Language Standards-Based Assessment (SBA) .......................................................................................................................... 3

Defining SBA ............................................................................................................................... 3

SBA for Young Learners .............................................................................................................. 3

What does SBA evaluate? ........................................................................................................... 4

Implementing SBA: The Train or Provide Path? .......................................................................... 6

Unpacking the RELTmax Implementation of SBA ........................................................................... 7

From KSSR to RELTmax Competencies ..................................................................................... 8

RELTmax Competency Descriptors ............................................................................................. 9

Example of a RELTmax Competency: SK Year 1 Competency D: Listen to Spoken Text ......... 10

Competency Descriptor Development: What constitutes evidence? .......................................... 11

Classroom Administration of SBA .............................................................................................. 12

SK Y2 Sample A Child Assessment Template: Chant ............................................................... 13

SK Y2 Sample B Child Assessment Template: Write Description or Story ................................. 14

SK Y2: Whole Class Record of Band Achievement by Child...................................................... 15

Works Cited .................................................................................................................................. 17

Page 3: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3

Rationale for the RELTmax Implementation of the KSSR English Language Standards-Based Assessment (SBA) Defining SBA ‘Standards-Based Assessment’ is the term used for Malaysia’s school-based assessment system being implemented in Years’ 1 and 2 to evaluate children’s ability to meet or achieve the ‘standards’ specified by the KSSR syllabi (both SK and SJK). School-based assessment generally refers to any evaluation of students conducted in a classroom. It is also referred to as ‘continuous assessment,’ ‘assessment for learning’ and ‘formative assessment’ in its many forms around the world (Rajput, Tewari & Kumar, 2005; Black & Wiliam, 2003). Whilst a test written and conducted within the school would fall within this definition, ‘quizzes,’ ‘tests’ and examinations are not usually included. It is also the case that the emphasis of school-based assessment is on actual, everyday teaching and learning practices. Teachers teach; learners learn, and what occurs as part of those processes is evaluated (Black & Wiliam, 2003; Webb & Jones, 2009). Although washback, the impact of assessment on teaching practices and content, is inevitable, school-based assessment attempts to avoid this, or to ensure that the effect is in a positive direction (Yu, 2007). Finally, because of ELTC’s role in enhancing English language teaching practices as the KSSR is implemented, it is worth noting that, as is the case in Malaysia, school-based assessment is often introduced in conjunction with curricula change, or innovations intended to improve teaching practices and learning outcomes (Koh, Lee, Gong & Wong, 2006; Yu, 2007). Therefore, school-based assessment plays a role as a formative instrument in realising these innovations, a summative one in evaluating them, and, in the case where it is used to focus teachers on the desired change, as a form of positive washback. SBA for Young Learners One feature of Malaysia’s SBA that differs from most others is that it evaluates young learners rather than those undertaking secondary education, although it should be noted that SBA is also being implemented in Malaysian secondary schools (Malakolunthu & Hoon, 2010). There are two major implications of this difference. First, the major cited benefit and aim of many school-based assessment programs is their impact on learning (Wiliam & Black, 1998). A type of formative assessment, these school-based programs target teaching practices and pedagogical decision-making in the development of a classroom learning culture in which students ultimately have greater responsibility for and control over their learning strategies through a ‘dialoguing’ or ‘embedding’ process (Sithamparam, 2011; Webb & Jones, 2009). It is clearly unnecessary that school-based assessment intended for young language learners be formative in nature, as the feedback processes needed to engage young children in learning, and motivate them to learn are well-documented (Crespo, 2002; Heyman, Dweck & Cain; 1992, Zentall & Morris, 2010). It follows too that the requirements of this feedback to facilitate learning (that it be effort-based, behavior-specific, and not contain criticism) mean that the results achieved and evaluated by this summative SBA system should not be communicated to the learners themselves who are particularly vulnerable to such feedback at this age, but only to other stakeholders: school management, parents, district management, and the relevant divisions of Ministry of Education (McKay, 2006).

Page 4: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 4

What does SBA evaluate? In its first iteration of SBA, the Exam Syndicate (ES) encountered negative washback in the form of the unintentional creation of an ‘alternative syllabus’ as teachers moved their focus from addressing the requirements of the KSSR to addressing the SBA descriptors themselves (ES Representative, personal communication, January 9, 2012), an extremely common occurrence with any form of assessment which appears to change the content of what is taught long before it affects teaching practices themselves (Cheng, 1999; Yu, 2010). To counter this, the SBA descriptors were re-released: the later version precisely mapping to the KSSR syllabi (Kementerian Pelajaran Malaysia, 2012a, 2012b, 2012c & 2012d). To understand the SBA, therefore, we must first briefly examine the KSSR in light of the literature on and terms used within syllabus design and assessment. As noted above, the KSSR is referred to as a ‘standards’ based syllabus: a commonly used term. For the purposes of SBA, however, it is worth considering the syllabi in terms of their being ‘competency-based’ and ‘text-based.’ The hallmark of a competency-based syllabus is that it focuses on learners being able to do things with language, rather than to know about it (Findley & Nathan, 1980). All the standards within the KSSR are expressed in this language. “[Pupils will be] Able to listen to, say aloud and recite rhymes or sing songs with guidance” (Bahagian Pembangunan Kurikulum, 2011b, p. 14) is one such example. Additionally, the KSSR can be said to be a ‘text-based’ syllabus. A ‘text’ is a whole piece of language which is complete in itself, such as a story, a poem, a chant, a dialogue or a description. A text can be heard, spoken, written or read (Feez & Joyce, 1998; Metropolitan East Disadvantaged Schools Program, 1989), and the process of reading, saying, listening to or writing a text is known as ‘negotiating’ (Savignon, 1991), constructing or deconstructing it. The KSSR and KBSR syllabi require learners to negotiate a range of texts such as those noted above (BPK 2003, 2011a) as do the Ujian Pencapaian Sekolah Rendah (UPSR) examinations conducted at the end of Year 6 (brochure, a dialogue, a story, and either a process description or a chart or graph). Each competency statement or ‘descriptor’ is then linked to a text or a part of a text as can be seen in Table 1 below. All Table descriptors 1-6 are taken directly from the KSSR SJK Tahun 1 syllabus (BPK, 2011b). The right-hand column contains the KSSR Standard Number. Table 1: Competency-Text Relationship within the KSSR (SJK Tahun 1, BPK, 2011b)

KSSR

1. Able to take part with guidance in a performance based on nursery rhymes & songs 4.3.2 a.b.

2. Able to produce simple creative works with guidance based on nursery rhymes & songs 4.3.1 a.b.

3. Able to recite nursery rhymes and sing action songs with correct pronunciation and rhythm.

4.1.2

4. Able to listen to, say aloud and recite rhymes or sing songs with guidance 4.1.1

5. Able to listen and respond to stimulus given with guidance: (rhythm & rhyme, voice sounds, alliteration)

1.1.1.d-f

6. Able to enjoy nursery rhymes and sing action songs through non-verbal response: (environmental sounds, instrumental sounds, body percussion)

1.1.1.a-c

Key: Competency Complete Text

Part of Text (Textual Component)

Page 5: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 5

Descriptors 1-4 relate to the learners’ ability to negotiate (perform, produce, recite, sing, listen, say aloud) the whole text (rhyme, song). Descriptors 4-6 relate to the learners’ ability to negotiate (non-verbally respond to, listen to, say aloud, sing and recite with guidance, parts of the whole text (rhyme, song). By teaching systematically from the phoneme level (Descriptor 6) to the whole text level (Descriptor 1) the children gradually acquire the skills they need to be able to produce (perform, recite, sing) the whole text (performance-based on nursery rhymes and songs). For this reason, the overarching descriptor (1: Able to take part with guidance in a performance based on nursery rhymes & songs) is termed the ‘competency’ and the skills/textual components that children need to be able to do to achieve that competency are known as ‘competency descriptors’ (2-6). If we now return to the original question of what the KSSR Standards-Based Assessment is intended to evaluate, the answer is simple. Each competency within the KSSR syllabi is mapped to a corresponding ‘Band Descriptor.’ The Band Descriptors require teachers to provide ‘evidence,’ (teachers need to see and record) that each child is able to hear, say, read or write each competency or Band Descriptor (KPM, 2012a, 2012b, 2012c & 2012d). Examining Table 2 below reveals that when a child provides evidence that they are “able to take part with guidance in a performance based on nursery rhymes and songs” (KSSR 4.3.2ab: see BPK, 2012a, p. 25) by taking part in a performance before an audience which is witnessed and recorded by the teacher, they will have achieved “Band Descriptor B5 DL2: Perform nursery rhymes and sing songs to an audience” (KPM, 2012a, p. 14). Table 2: Mapping the KSSR (BPK, 2011b) to the SBA Band Descriptors (KPM, 2012a)

KSSR BANDS

1. Able to take part with guidance in a performance based on nursery rhymes & songs

4.3.2 a.b. B5 DL2

2. Able to produce simple creative works with guidance based on nursery rhymes & songs

4.3.1 a.b.

B5 DL1 3. Able to recite nursery rhymes and sing action songs with correct

pronunciation and rhythm. 4.1.2

4. Able to listen to, say aloud and recite rhymes or sing songs with guidance 4.1.1

B1 DL1 5. Able to listen and respond to stimulus given with guidance: (rhythm & rhyme, voice sounds, alliteration)

1.1.1.d-f

6. Able to enjoy nursery rhymes and sing action songs through non-verbal response: (environmental sounds, instrumental sounds, body percussion)

1.1.1.a-c B4 DL1

Key to the Band Descriptors or ‘Bands’ (KPM, 2012a):

B5 DL2: Perform nursery rhymes and sing songs to an audience. B5 DL1: Recite rhymes or sing songs. B4 DL1: Listen to and demonstrate understanding of the oral texts. B1 DL1: Say aloud rhymes or sing songs with guidance.

While the Band Descriptors may not be changed in any way, teachers have considerable freedom in the form the evidence they use to evaluate the children takes. This is because the Exam Syndicate provides suggestions as to what the evidence may be, both in the Band Descriptor documents themselves and on the website that Year 1 and 2 teachers have password access to (see http://apps.moe.gov.my/sppbs/index.cfm). For example, in Table 2 above, Band Descriptor B1 DL1 “Say aloud rhymes or sing songs with guidance” may be evidenced by children demonstrating that they are “able to do any of the following:

Page 6: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 6

repeat rhymes after the teacher,

sing along with the teacher, [or]

sing in groups” (KPM, 2012a, p.9). Implementing SBA: The Train or Provide Path? We now come to the question that goes to the heart of the RELTmax implementation of SBA. How do we implement SBA so that this assessment system produces the desired outcomes: improved teaching practice, the implementation of the new innovative curricula, and ultimately, better learning outcomes for children? There are two major ways in which school-based assessment is implemented across the world. The first approach is to train teachers to write their own assessment materials which will produce the necessary evidence of learning. The second approach is to provide teachers with a bank of assessment materials to achieve the same results and teach them to implement and interpret these materials (Sishi & Poliah, 2006). Despite repeated and continued attempts to pursue the former strategy of training teachers to become assessors and evaluators (see, for example, Fok, Kennedy, Chan & Yu 2006, Sithamparam, 2011), this approach consistently fails. Even with extensive and targeted training, teachers consistently prove to be poor assessment designers, regardless of whether they are designing traditional tests (Coniam , 2009) or school-based assessment materials. The reasons for this failure relate to knowledge and culture. Designing assessment tasks requires an extraordinary amount of knowledge about testing and assessment, language acquisition, how language is structured, the skills involved in negotiating a range of texts and task design itself (Hamp-Lyons, 2009; Koh, Lee, Gong & Wong, 2006, Scott, 2009). Understandably, acquiring and applying this knowledge is difficult for teachers, even should they be motivated to do so. Research has shown us that teachers have difficulty aligning their assessments to the instructional goals, thereby ensuring the validity of the assessment (Koh, Lee, Gong & Wong, 2006; Yu, 2007). There are issues designing tasks that focus on what children can do rather than on their mistakes (Lee, 2007; Lo, 2006), that produce the kind of interaction required by the assessment (Hamp-Lyons, 2009), that are not pencil and paper based, or based on children’s being able to regurgitate rote-learned knowledge, that assess a range of skills in a ‘fun’ way that does not engender stress (Goh & Mohamed, 2006) and that assess children’s higher-order thinking (Koh, Lee, Gong & Wong, 2006; Yu, 2007). Additionally, integrating assessment tasks and task types across class levels (e.g. Year 1 and 2) has been found to be problematic (Lo, 2006). In interpreting the assessment, teachers vary in their understanding and evaluation of both verbal and non-verbal learner performance (Butler, 2009; Lo, 2006), and are found to evaluate children on the basis of ethnicity, communication style, or on perceived behavioural problems unrelated to the assessment task (Lo, 2006). These issues are exacerbated by uneven affective-cognitive and proficiency development that characterises young language learners, (McKay, 2006; Scott, 2009), a particularly problematic area for Malaysia’s SBA, as there appears to an expectation that children’s demonstrated competencies will fall under one of the 6 Band ‘levels,’ brought about by some of the training on SBA done during 2011. Cultural factors play an even greater role in teachers’ failure to become assessment designers. In many Asian countries, there is a long tradition of norm-referenced, summative, high stakes examinations from primary through to the end of secondary education. This

Page 7: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 7

tradition has an enormous influence on teachers’ willingness and ability to develop school-based assessment tasks. Teachers often see school-based assessment as irrelevant to their job and student learning outcomes, as unreliable or compared to the summative assessment they are used to, or merely as additional bureaucratic paperwork imposed from above to hold them accountable (Yu, 2010). They may also see it as “another exam paper” for which they must drill students. When these perceptions are mirrored by the public, who may, in addition, not trust teachers, there is little incentive to acquire the extensive knowledge needed to implement the change (Fok, Kennedy, Chan & Yu 2006; Lo, 2006; Luyten & Dolkar, 2010). As a result of these experiences, many countries have moved to the second strategy of SBA implementation which is to supply a bank of assessment materials, prototypes or templates and provide extensive training in their use (Davison & Leung, 2009; Hamp-Lyons, 2009; Koh, Lee, Gong & Wong, 2006; Sishi & Poliah, 2006). This is also the approach taken by RELTmax in the implementation of SBA, a description of which follows.

Unpacking the RELTmax Implementation of SBA In implementing SBA, it is important to remember that the assessment is not there for its own sake. The goal of SBA is to improve learning outcomes through curriculum innovation which develops and enhances teaching practices. SBA can contribute to this in meaningful ways, but it cannot of itself bring about the change and it can also have a negative impact. Given the effect of washback and teachers’ tendency to ‘teach to the test,’ it was thought better to focus training efforts on teaching rather than on testing, since ideally SBA measures what is actually happening in the classroom and the assessment is opaque to the children. As noted above, the KSSR is text-based. However, this does not imply that teachers walk into the classroom and begin the teaching process with the whole text, be it a story, song or other text. To do so would test rather than teach. Learning takes place in a scaffolded, step by step, systematic manner. The steps are achieved through individual teaching tasks supported (scaffolded) by the learners’ peers or the teacher. Whilst a text may be approached from the top down with more proficient learners of English (such as advanced Upper Primary or Secondary learners), for younger, less proficient learners, tasks that guide learners towards the construction of a text are sequenced from the bottom up, in the following way. Reaching the top of the table means the teacher has taught one of the texts required by the KSSR syllabus. Table 3: Task Sequencing in Scaffolded Text-Based Teaching

Whole Text Level

8. Text Pattern/Organisation (Discourse Structure) 7. Context

Sentence Level

6. Form & Function (‘formulaic’ or ‘chunk’ language) Word Level

5. Write (include spelling) 4. Read 3. Say (pronounce the word 2. Hear (understand aurally) 1. Meaning (of word)

Page 8: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 8

From KSSR to RELTmax Competencies

That being the case, and given the text and competency-based nature of the KSSR syllabi, why not simply take the KSSR Competencies and Descriptors (sub-competencies) and map them to the Band Descriptors as in Table 4 below? Teachers could then be sure of achieving the Band Descriptors, and implementing the KSSR. Table 4: Teaching the KSSR as a Competency/Band Descriptor Checklist (BPK, 2011a) to the SBA Band Descriptors (KPM, 2012c)

Able to read simple texts with guidance: a) fiction b) non-fiction KSSR 2.3.1.a

BANDS B5 DB1b

Able to read a paragraph of 3-4 sentences. 2.2.4 1.1.4

B6 DL1

Able to read & understand sentences in linear and non-linear texts with guidance. 2.2.3

B3 DB2 B4 DB1

Able to read & understand phrases in linear and non-linear texts. 2.2.2

Able to read and apply word recognition & attack skills by matching words with graphics

2.2.1.a B2 DB1 B3 DB1

Able to segment words into phonemes to spell. Able to spell common sight words.

2.1.4 3.2.4

B1 DB1 B1 DB2 B1 DB4 B2 DB2

Able to recognise & articulate initial, medial & final sounds in single syllable words within a given context. Able to read and apply word recognition & attack skills by matching words with spoken words.

2.1.2 2.2.1.b

B2DB2

Able to listen to and enjoy simple stories Able to talk about a stimulus with guidance

1.1.4 4.2.2

B5 DL1

Able to listen and respond to stimulus with guidance: oral blending & segmenting 1.1.1.g B1 DB3

There are a number of reasons why implementing the SBA in this way is unlikely to succeed and these are discussed below. 1. Lack of Knowledge: To ask the teachers to follow the descriptors above is to require

the same knowledge they would need to become assessment designers. A brief look at Table 4 above assumes a knowledge of Phonics (word attack skills, segment words into phonemes to spell, articulate initial medial and final sounds), language and its structure (linear and non-linear texts), scaffolding techniques (with guidance), task design (talk about a stimulus with guidance) and a very detailed knowledge of what it means to ‘read a paragraph’ in terms of both skills and teaching methodologies that do not involve simply translating it, or children guessing at the meaning.

2. Modular Structure: The KSSR syllabus describes its modular structure as “organising

the curriculum standards under five modules” (BPK, 2011a, p.5), which means that skills related to reading are grouped together. In order to “read simple texts with guidance” (BPK, 2011a, p.18), however, learners need to speak, listen, talk, and respond, which are categorised under the Listening & Speaking and Language Arts Modules (see Table 4’s KSSR column). This also applies to reading skills when undertaking competencies within the writing module, as “skills integration is exploited strategically to enhance pupils’ development of specific language skills” (BPK, 2011a, p.5).

To go through the KSSR and reorganise it would also require knowledge of language and teaching methodologies that many teachers would not have, even if they had the time required to do this.

3. Missing Descriptors: Finally, and partially because of its modular structure, some texts

are very fully described by their competency descriptors while others are less so. For example, the KSSR syllabi make repeated references to reading and writing non-linear

Page 9: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 9

texts. As such texts are read non-sequentially, a key competency or skill in being able to do so is to identify the discourse markers in the text which tell you where you will find the information you require. If you want to know what class you must attend on Wednesday afternoon in a classroom timetable, you need to know that one column will contain Wednesday’s information, and another row, the afternoon’s time and class. Otherwise you would need to read the timetable in a linear, sequential fashion (starting Monday 8am) in order to find the information you required).

RELTmax Competency Descriptors

To address these issues, the RELTmax development team rewrote the KSSR Text Competencies, text by text, rewording each descriptor so that the linguistic, methodological, and task design knowledge is built into the overall competency. If you examine Table 5 below, for example, you will see that we have defined “reading and understanding a story” as being able to demonstrate understanding of its characters, setting, and sequence of events.” Table 5: From KSSR to RELTmax Competency (SK Tahun 1, BPK, 2011a; KPM, 2012c)

Can read and respond to a short picture story. KSSR 2.3.1.a

BANDS B5 DB1b

Can demonstrate understanding of the moral values in the story by responding to simple questions about its characters.

1.1.4 B6 DL1

Can demonstrate understanding of read sentences by matching them to realia (or pictures of realia).

2.2.1.a B4 DB1

Can demonstrate understanding of story’s sequence of events and sentences by sequencing them correctly.

2.2.3 2.2.4

B3 DB2

Can demonstrate understanding of the characters and setting of the story. 2.2.3 2.2.4

Can demonstrate understanding of written words/phrases via non-verbal response (actions or pictures).

2.2.1.a B3 DB1

Can decode (read) short, familiar, single-syllable words in the story using Phonics segmenting techniques.

1.1.1.g 2.1.2

2.2.1.b B2DB2

Can use Phonics segmenting techniques to spell familiar words they hear (in the text).

1.1.1.g 2.1.4 3.2.4

B1 DB1 B1 DB2 B1 DB4 B2 DB2

Can demonstrate understanding of word/phrase meaning via non-verbal response (actions or pictures).

2.2.1.a B2 DB1

Can make predictions about the characters, setting and sequence of events in the story by using the story’s pictures (pictorial cues).

1.1.4 4.2.2

B5 DL1

Can blend individual phonemes in short, familiar, single-syllable words. 1.1.1.g B1 DB3

There are between ten and twelve competencies for each class (depending on the Year level and school type), with each competency representing a KSSR text or text type. For example, chant and song are classed as one competency, but there are separate competencies for a short story (non-fiction, linear text) and description fiction, linear text). Any teacher who takes the RELTmax or RELTnote program is taught to teach from word level to sentence level sequentially (see Table 3 above), and are given tasks to do teach these skills as well as the experience of developing and delivering their own. If they carry out these tasks in the way we model them, they will not have to write any assessments of their own (unless they chose to do so), but can focus on teaching the KSSR in a creative, fun, engaging, systematic, learner-centred way. To aid that process, RELTmax and RELTnote tasks are mapped against each competency (such as that in Table 5 above), as can be seen over the page.

Page 10: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 10

Example of a RELTmax Competency: SK Year 1 Competency D: Listen to Spoken Text

A. Can listen to, demonstrate understanding of and respond to spoken texts. [directions, story]

BANDS KSSR RELTmax task/Evidence

Can demonstrate understanding of simple directions through physical, pictorial, verbal or written response.

B3 DL2b1 1.2.2.b 4.16 K Map *

Can demonstrate understanding through physical, pictorial, verbal or written response. B4 DL1 1.1.2

4.10 Listening: Which Building* 4.17 Where’s Sara’s cat? * 6.2 Simon Says * 6.3 Listening: Hare & Tortoise*

Can demonstrate understanding of chronological sequence and place. [story only]

B6 DL1

1.1.2

4.17 Where’s Sara’s cat? * 6.3 Listening: Hare & Tortoise* 6.7 Puppet Dialogue*

Can demonstrate understanding of the moral values in the story by responding to simple questions about its characters. [story only]

1.1.2 1.1.4 1.3.1

1 To achieve Band B3 DL2, children need to demonstrate understanding of spoken directions (see Competency D B3 DL2b), in addition to demonstrating

understanding of teacher’s instructions (See Competency K, B3 DL2a). * As with all RELTmax tasks, this task provides evidence of this competency. However, this task must be taught AFTER completing preceding tasks (e.g. 4.16 is taught after tasks 4.1, 4.2, 4.3, 4.4, 4.5, 4.6 etc to 4.15)

Page 11: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 11

Competency Descriptor Development: What constitutes evidence? Having focused on the teachers’ ability to teach to the best of their abilities and to world best practice standards, we were now able to determine what kind of evidence within these teaching practices and tasks could be used to meet the requirements of the Exam Syndicate’s Band Descriptors. As noted above, teachers (and assessment designers) have some freedom in determining the what evidence is used to demonstrate completion of each Band Descriptor. They have NO freedom whatsoever to change the Band Descriptors themselves. In determining the type of evidence that would be necessary to meet audit requirements, and as part of the process of building linguistic and teaching knowledge into the assessment instruments, a number of assumptions and interpretations were made, and these are described below. Since English language teaching is ELTC’s core business, it should be noted that these assumptions are based on current, evidence-based research into school-based assessment, language, teaching, learning and language acquisition. 1. Cautious Interpretation: We are very concerned that each teacher can evidentially

demonstrate to all stakeholders that they have fully met each Band Descriptor, and in so preparing the evidential requirements, we have erred on the side of caution.

2. Formal and Informal Talk: Unlike the KSSR, the ES Band Descriptors differentiate

between ‘casual’ and ‘formal’ conversation. In the literature, ‘casual conversation is represented by many text types (e.g. anecdotes, gossip, jokes see Eggins & Slade, 1997) and talk is not typically divided into ‘casual’ and ‘formal’ categories, but by domain, function and text (genre) e.g. transactional dialogue. There is a considerable body of literature on the differences between formal and informal talk and how they are structured. We have drawn on the seminal work by Sacks, Schegloff and Jefferson (1974) amongst others in our definition of formal and informal talk, and these explanations are contained and cited in the footnotes of the RELTmax Competency Descriptors themselves.

3. Real life classroom practice: As far as possible, evidence should mirror actual

classroom practice. When interpreting SK Tahun 1 Band Descriptor B6 DL1, “Talk about a short story with guidance,” (KPM, 2012c) for example, we sought advice on what that would mean in a Year 1 classroom, what questions would be asked and about what aspects of the story (Kalminderjit Kaur, 2009).

4. Competencies: Less is more! To make teachers’ lives easier, we have tried to

minimise the number of RELTmax Competencies teachers need to use.

5. Band Descriptors Interpretation: To better understand the descriptors, we made use of the evidence provided, and the overall Band Descriptor (e.g. SK Tahun 1 Band 6 Overall Descriptor reads: “Appreciating literary works by performing and presenting ideas using exemplary manners.” If Band Descriptors appeared similar but belonged to widely differing Overall Bands (eg B3 and B6, KPM, 2012c), we understood that the Descriptors were to be achieved by evidence at a substantially different proficiency level and we altered the textual descriptors and/or Competency as a result.

6. Socio-Linguistic Competence: We understood the “good… admirable… and

exemplary manners” specified in Bands 4, 5, and 6 to refer to socio-linguistic competence (Canale, 1983) in the construction of spoken and written texts.

Page 12: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 12

Classroom Administration of SBA Many teachers see SBA as time-consuming additional paperwork in their busy, committed lives. One of the goals of the RELTmax implementation of SBA was to make documentation and record-keeping, so necessary for government, parental and school stakeholder auditing as transparent and easy as possible. Again, teachers teach; learners learn, and what occurs as part of those processes is evaluated. There is no need to allocate additional classroom time for assessing. The children will not even know they are being assessed. Once teachers are nearing completion of a text, they make copies of the relevant template, one for each student, and take them into class (see over page). Competency Descriptors, provided they are specific (another example of expert knowledge being applied), avoid the need for teachers to make judgements about proficiency and circumvent inter-rater reliability issues (Hamp-Lyons, 2009). Children either ‘can’ or ‘cannot’ do each of the descriptors. If the teacher sees it, they tick the box/es, and move onto the next child. Children may complete all, none of some of the descriptors in each competency. Some Band Descriptors are achieved by the child’s providing evidence of having met one RELTmax Band Descriptort. Others must be achieved by completing all the RELTmax Competency Descriptors. On their return to the staffroom, teachers 1) transfer records of Band Description completion onto the Class Record (see pages 15-16), 2) cut up individual child records and place these in each child’s file for Parent Teacher Meetings and audit purposes, and 3) upload the relevant evidential information onto the Ministry of Education website designed for that purpose. These individual child and overall class achievement records provide stakeholders with the following information:

1. How many times the teacher has assessed the Competency

2. What texts they used to do so (e.g. Text = Story. The Tasik Cini Dragon.

3. If the child was absent on the day of the assessment, and if this is a regular occurrence for that child

4. How the child rates in terms of Band Achievement against other children in the class.

It is the teachers’ choice of how many times, when, what and how they will evaluate the children in the class. They can choose to write their own assessment materials or not. We hope they choose to do what they do best: teach.

Page 13: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 13

SK Y2 Sample A Child Assessment Template: Chant

Name: Hisham B. Anuar Class: 2 Arif Date: 3/9/12

Name of the text used: “May I borrow” Chant Absent

Can demonstrate an understanding of and sing or recite a song, chant or rhyme. BANDS

Can demonstrate understanding of a song, chant or rhyme through non-verbal response.

B1 DL1 Can pronounce individual words clearly (sounds and word stress).

Can use sentence stress (beat and rhythm) to clearly and accurately recite chant or sing a song, chant or rhyme.

Can match the rhythm of a song, chant or rhyme to physical actions (clapping or stamping). B1 DL2

Name: Anwar Bin Suhaimi Class: 2 Arif Date: 3/9/12

Name of the text used: “May I borrow” Chant Absent

Can demonstrate an understanding of and sing or recite a song, chant or rhyme. BANDS

Can demonstrate understanding of a song, chant or rhyme through non-verbal response.

B1 DL1 Can pronounce individual words clearly (sounds and word stress).

Can use sentence stress (beat and rhythm) to clearly and accurately recite chant or sing a song, chant or rhyme.

Can match the rhythm of a song, chant or rhyme to physical actions (clapping or stamping). B1 DL2

Name: Nasrul B Daud Class: 2 Arif Date: 3/9/12

Name of the text used: “May I borrow” Chant Absent

Can demonstrate an understanding of and sing or recite a song, chant or rhyme. BANDS

Can demonstrate understanding of a song, chant or rhyme through non-verbal response.

B1 DL1 Can pronounce individual words clearly (sounds and word stress).

Can use sentence stress (beat and rhythm) to clearly and accurately recite chant or sing a song, chant or rhyme.

Can match the rhythm of a song, chant or rhyme to physical actions (clapping or stamping). B1 DL2

Name: Walid B Rashid Class: 2 Arif Date: 3/9/12

Name of the text used: “May I borrow” Chant Absent

Can demonstrate an understanding of and sing or recite a song, chant or rhyme. BANDS

Can demonstrate understanding of a song, chant or rhyme through non-verbal response.

B1 DL1 Can pronounce individual words clearly (sounds and word stress).

Can use sentence stress (beat and rhythm) to clearly and accurately recite chant or sing a song, chant or rhyme.

Can match the rhythm of a song, chant or rhyme to physical actions (clapping or stamping). B1 DL2

Page 14: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 14

SK Y2 Sample B Child Assessment Template: Write Description or Story

Name: Hisham B. Anuar Class: 2 Arif Date: 3/9/12

Name of the text used: Animal Description Absent

Can write a simple linear text with guidance. [description, story] BANDS

Can form letters legibly and space them correctly to form words. B1 DT1

Can use Phonics segmenting techniques to spell simple familiar words. B1 DB3 B2 DT1

Can copy, adapt and personalize model sentences legibly and correctly. B3 DT2 B4 DT1

Can spell simple words, phrases and simple sentences correctly.

Can punctuate simple phrases and simple sentences correctly. B3 DT1

Can demonstrate understanding of words, phrases and sentences by unscrambling (rearranging words to form) sentences.

B3 DB2

Can demonstrate understanding of sentences by sequencing them correctly. B3 DB2

Name: Anwar Bin Suhaimi Class: 2 Arif Date: 3/9/12

Name of the text used: Animal Description Absent

Can write a simple linear text with guidance. [description, story] BANDS

Can form letters legibly and space them correctly to form words. B1 DT1

Can use Phonics segmenting techniques to spell simple familiar words. B1 DB3 B2 DT1

Can copy, adapt and personalize model sentences legibly and correctly. B3 DT2 B4 DT1

Can spell simple words, phrases and simple sentences correctly.

Can punctuate simple phrases and simple sentences correctly. B3 DT1

Can demonstrate understanding of words, phrases and sentences by unscrambling (rearranging words to form) sentences.

B3 DB2

Can demonstrate understanding of sentences by sequencing them correctly. B3 DB2

Name: Nasrul B Daud Class: 2 Arif Date: 3/9/12

Name of the text used: Animal Description Absent

Can write a simple linear text with guidance. [description, story] BANDS

Can form letters legibly and space them correctly to form words. B1 DT1

Can use Phonics segmenting techniques to spell simple familiar words. B1 DB3 B2 DT1

Can copy, adapt and personalize model sentences legibly and correctly. B3 DT2 B4 DT1

Can spell simple words, phrases and simple sentences correctly.

Can punctuate simple phrases and simple sentences correctly. B3 DT1

Can demonstrate understanding of words, phrases and sentences by unscrambling (rearranging words to form) sentences.

B3 DB2

Can demonstrate understanding of sentences by sequencing them correctly. B3 DB2

Page 15: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 15

SK Y2: Whole Class Record of Band Achievement by Child

Year 2 Arif

B1

DL

1

B1

DL

2

B1

DB

1

B1

DB

2

B1

DB

3

B1

DT

1

B2

DL

1

B2

DB

1

B2

DB

2

B2

DT

1

B3

DL

1

B3

DL

2a

B3

DL

2b

B3

DL

3

B3

DB

1

B3

DB

1

B3

DB

2

B3

DB

3

B3

DT

1

B3

DT

2

B4

DL

1

B4

DB

1

B4

DB

2

B4

DT

1

B5

DL

1

B5

DL

2

B5

DB

1

B5

DT

1

B5

DT

2

B6

DL

1

B6

DL

2

B6

DB

1

B6

DB

2

B6

DT

1

Abu Bakar B. Ahmad 2

/3

2/3

5/4

5/4

5/4

4/7

2/3

A

Anita Wong

2/3

2/3

5/4

5/4

5/4

4/7

5/4

5/4

Anwar B Suhaimi

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Azidah Bt Aziz

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Azizah Bt Hasshim

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Azmira Bt Amran

2/3

2/3

5/4

5/4

5/4

2/3

5/4

Chin Chuan Ngok 2

/3

2/3

5/4

A

A

A

A

Hisham B Anuar

2/3

2/3

A

5/4

5/4

2/3

5/4

Hazrul B. Haris

2/3

2/3

5/4

5/4

5/4

2/3

5/4

Jaya Pushani Ponnudurai 2

/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Page 16: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 16

Year 2 Arif

B1

DL

1

B1

DL

2

B1

DB

1

B1

DB

2

B1

DB

3

B1

DT

1

B2

DL

1

B2

DB

1

B2

DB

2

B2

DT

1

B3

DL

1

B3

DL

2a

B3

DL

2b

B3

DL

3

B3

DB

1

B3

DB

1

B3

DB

2

B3

DB

3

B3

DT

1

B3

DT

2

B4

DL

1

B4

DB

1

B4

DB

2

B4

DT

1

B5

DL

1

B5

DL

2

B5

DB

1

B5

DT

1

B5

DT

2

B6

DL

1

B6

DL

2

B6

DB

1

B6

DB

2

B6

DT

1

Kalminderjit Kaur

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Logeswari a/p Arumugam 2

/3

2/3

A

5/4

5/4

4/7

2/3

5/4

Nasrul B Daud

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Richard Teoh

2/3

2/3

5/4

5/4

5/4

2/3

Rosdi B. Harun

2/3

2/3

5/4

5/4

5/4

2/3

Selvanayagi Shanmugham 2

/3

2/3

5/4

5/4

5/4

A

2/3

5/4

Sheila Adelina

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Syaliana Jamaludin

2/3

2/3

5/4

5/4

5/4

4/7

2/3

5/4

Walid B Rashid

2/3

2/3

5/4

5/4

5/4

A

2/3

5/4

Wong Lindsay

2/3

2/3

5/4

5/4

5/4

4/7

2/3

A

Page 17: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 17

Works Cited

Bahagian Pembangunan Kurikulum. (2003) Huraian Sukatan Pelajaran: Kurikulum Bersepadu Sekolah Rendah. Malaysia: Bahagian Pembangunan Kurikulum Kementerian Pelajaran Malaysia.

Bahagian Pembangunan Kurikulum, (2011a). Dokumen standard kurikulum sekolah rendah: Modul teras asas: Bahasa inggeris SK: Tahun & Dua. Putrajaya: Malaysia: Kementerian Pendidikan Malaysia. Retrieved from http://www.moe.gov.my/bpk/index.php?option=com_docman&task=cat_view&gid=227&Itemid=83

Bahagian Pembangunan Kurikulum, (2011b). Dokumen standard kurikulum sekolah rendah: Bahasa inggeris SJK: Tahun, Dua & Tiga. Putrajaya: Malaysia: Kementerian Pendidikan Malaysia. Retrieved from http://www.moe.gov.my/bpk/index.php?option=com_docman&task=cat_view&gid=227&Itemid=83

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.

Black, P. & Wiliam, D. (2003). ‘In praise of educational research: Formative assessment. British Educational Research Journal, 29(5), 623-637.

Butler, Y.G. (2009). How do teachers observe and evaluate elementary school students’ foreign language performance? A case study. TESOL Quarterly, 43(3), 417-444.

Cheng, L. (1999). Changing assessment: Washback on teacher perceptions and actions. Teaching and Teacher Education, 15I, 253-271.

Coniam, D. (2009). Investigating the quality of teacher-produced tests for EFL students and the effects of training in test development principles and practices on improving test quality. System, 37, 226–242.

Crespo, S. (2002). Praising and correcting: Prospective teachers investigate their teacherly talk. Teaching and Teacher Education, 18, 739-758.

Davison, C. & Leung, C. (2009). Current issues in English language teacher-based assessment. TESOL Quarterly, 43(3), 393-415.

Eggins, S. & Slade, D (1997) Analysing casual conversation. London & New York: Cassell Feez, S. & Joyce, H. (1998) Text-based syllabus design. Sydney: National Centre for English

Language Teaching and Research, Macquarie University. Findley, C.A. & Nathan, L.A. (1980). Functional language objectives in a competency based ESL

curriculum. TESOL Quarterly, 14(2), 221-231. Fok, P-K, Kennedy, K.J., Chan, J.K-S. & Yu, F.W-M. (2006). Integrating assessment of learning and

assessment for learning in Hong Kong public examinations: Rationales and realities of introducing school-based assessment. Paper presented at Assessment in an Era of Rapid Change: Innovations and Best Practices, International Association of Educational Assessment Conference, Singapore, 2006. Retrieved from: http://www.iaea2006.seab.gov.sg/conference/programme.html

Goh, D & Mohamed, S. (2006). Engaging the learner through assessment: A study conducted with lower primary students in Singapore. Paper presented at Assessment in an Era of Rapid Change: Innovations and Best Practices, International Association of Educational Assessment Conference, Singapore, 2006. Retrieved from: http://www.iaea2006.seab.gov.sg/conference/programme.html

Hamp-Lyons, L. (2009) Principles for large-scaled classroom-based assessment of English learners’ language: An initial framework for school-based assessment in Hong Kong. TESOL Quarterley, 43(3), 524-529.

Heyman, G.D., Dweck, C.S. & Cain, K.M. (1992). Young childen’s vulnerability to self-blame and helplessness: Relationship to beliefs about goodness. Child Development, 63, 401-415.

Hui, S.K.F. & Brown, G.T.L. (2010). Contrasting teacher’s espoused and enacted classroom assessment: Exploring Hong Kong Chinese teachers’ conceptions of assessment. Paper presented at Assessment for Future Generations, International Association of Educational Assessment Conference, Thailand, 2010. Retrieved from:

Kalminderjit Kaur, (2009). Negotiating critical literacies in Malaysian classrooms. Unpublished PhD thesis. Universiti Kebangsaan Malaysia, Bangi Selangor, Malaysia.

Kementerian Pelajaran Malaysia. (2012a). Dokumen standard prestasi: Bahasa Inggeris: Untuk kegunaan Sekolah Jenis Kebangsaan Tahun 1. Retrieved from http://syazalina83.blogspot.com/search?q=prestasi&x=0&y=0

Kementerian Pelajaran Malaysia. (2012b). Dokumen standard prestasi: Bahasa Inggeris: Untuk kegunaan Sekolah Jenis Kebangsaan Tahun 2. Retrieved from http://syazalina83.blogspot.com/search?q=prestasi&x=0&y=0

Page 18: Implementation of the KSSR Standards-Based · PDF fileRELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 3 Rationale for the RELTmax Implementation

RELTmax Implementation of the KSSR Standards-Based Assessment: Simone Evans, PhD: May 2012 18

Kementerian Pelajaran Malaysia. (2012c). Dokumen standard prestasi: Bahasa Inggeris: Untuk kegunaan Sekolah Kebangsaan Tahun 1. Retrieved from http://syazalina83.blogspot.com/search?q=prestasi&x=0&y=0

Kementerian Pelajaran Malaysia. (2012d). Dokumen standard prestasi: Bahasa Inggeris: Untuk kegunaan Sekolah Kebangsaan Tahun 2. Retrieved from http://syazalina83.blogspot.com/search?q=prestasi&x=0&y=0

Koh, K., Lee, A. N, Gong, W. & Wong, H.M. (2006). Development of the Singapore prototype classroom assessment tasks: Innovative tools for improving student learning and performance. Paper presented at Assessment in an Era of Rapid Change: Innovations and Best Practices, International Association of Educational Assessment Conference, Singapore, 2006. Retrieved from: http://www.iaea2006.seab.gov.sg/conference/programme.html

Lee, I. (2007). Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning? Assessing Writing, 12, 180-198.

Lo, Y.C. (2006). Practice and challenges of school-based formative assessment. Paper presented at Assessment in an Era of Rapid Change: Innovations and Best Practices, International Association of Educational Assessment Conference, Singapore, 2006. Retrieved from: http://www.iaea2006.seab.gov.sg/conference/programme.html

Luyten, H. & Dolkar, D. (2010). School-based assessments in high-stakes examinations in Bhutan: a

question of trust? Exploring inconsistencies between external exam scores, school-ased assessments, detailed teacher ratings, and student self-ratings. Educational Research & Evaluation, 16(5), 421-435.

Malakolunthu, S. & Hoon, S.K. (2010). Teacher perspectives of school-based assessment in a secondary school in Kuala Lumpur. Procedia Social and Behavioral Sciences, 9, 1170-1176.

McKay, P. (2006). Assessing young language learners. Cambridge, UK: Cambridge University Press. Metropolitan East Disadvantaged Schools. Program (1989) A brief introduction to genre. Erskineville,

NSW: NSW Department of School Education. Rajput, S., Tewari, A.D. & Kumar, S. (2005). Feasibility study of continuous comprehensive assessment

of primary students. Studies in Educational Evaluation, 31, 328-346 Sacks, H, Schegloff, E.A. & Jefferson, G. (1974) A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language, 50(4), 696-735. Savignon, S.J. (1991). Communicative language teaching: State of the art. TESOL Quarterly, 25(2), 261-

277. Scott, C. (2009). Issues in the development of a descriptor framework for classroom-based teacher

assessment of English as an additional language. TESOL Quarterly, 530-535. Sishi, S.N.P. & Poliah, R.R. (2006). Can statistical and qualitative modes of moderation co-exist in a

model for the quality assurance of school-based assessment: A South African perspective. Paper presented at Assessment in an Era of Rapid Change: Innovations and Best Practices, International Association of Educational Assessment Conference, Singapore, 2006. Retrieved from: http://www.iaea2006.seab.gov.sg/conference/programme.html

Sithamparam, S. (2011). Dialoguing to learn through assessment in the English classroom. In Strengthening English language through teacher professional development. Proceedings of the ELTC-TED Conference, 2011 (pp. 48-57). Kuala Lumpur, Malaysia: English Language Teaching Division, Ministry of Education, Malaysia.

Yu, Y. (2010). The washback effects of school-based assessment on teaching and learning: A case study. (Doctoral dissertation). Retrieved from http://hub.hku.hk/bitstream/10722/65273/1/FullText.pdf?accept=1

Zentall, S.R. & Morris, B.J. (2010). “Good job, you’re so smart”; The effects of inconsistency of praise type on young children’s motivation. Journal of Experimental Psychology, 107, 155-163.