interrelationships among students' study activities, self-concept of academic ability, and...

APPLlED COGNITIVE PSYCHOLOGY, VOL. 7.499-532 (1993)

Interrelationships among Students’ Study Activities, Self-concept of Academic Ability, and Achievement as a

Function of Characteristics of High-school Biology Courses JOHN W. THOMAS, LINDA BOL,

ROBERT W. WARKENTIN, AND MARK WILSON University of California, Berkeley

AMY STRAGE Sun Jose State University

WILLIAM D. ROHWER, JR University of California, Berkeley

SUMMARY

This investigation focused on the interrelationships among students’ study activities, students’ self-concept of academic ability ratings, students’ academic achievement, and instructional practices in 12 high school biology courses. Using a framework derived from a previous investigation, course features were classified into those that appear to (a) place demands on, (b) support, or (c) compensate for student engagement in particular study activities. Students’ study activities, self-concept of academic ability ratings, and achievement were measured with experimenter-developed instruments. Results are reported for (a) characteristics of instruments and course features, (b) relationships between central factors of the investigation, and (c) multi-level relationships between course features and student variables. Results at the student level indicated that self-concept of academic ability and, to a lesser extent, students’ study activities were positively associated with student achievement. Students’ self-ccncept of academic ability ratings were also linked to students’ engagement in generative, proactive study activities. At the course level the supportive practices of providing challenging homework assignments and extensive feedback on student coursework were associated with student engagement in effortful, generative, proactive study activities. The provision of extensive feedback was also associated with high student achievement. Multi-level relationships were analysed using hierarchical linear modelling (HLM) analyses. These analyses revealed, for example, that in courses in which little or no feedback is given on homework assignments, the relationship between achievement and student engagement in diligent effort management activities was enhanced. Other HLM analyses were conducted to examine the mediating role of course features on the relationship between students’ self-concept of academic ability and their study activities and achievement. For example, the presence of challenging course demands was associated with an enhancement of the relationship between self-concept of academic ability and achievement whereas the presence of instructor provisions (supports and compensations) designed to reduce course demands was associated with a reduction in this relationship.

08884080/93/060499-34/$22.00 0 1993 by John Wiley & Sons, Ltd.

Received 21 August 1992 Accepted 29 January 1993

500 J. W, Thomas et al.

This investigation marks the third in a series of investigations designed to account for variations in the incidence and effectiveness of studying as a function of differences in grade level, courses, and individuals (Rohwer and Thomas, 1989; Thomas and Rohwer, 1987). These studies have as their long-term goal the development of a psychology of academic studying useful for improving the quality of students’ autonomous learning activities in and outside of the courses they take.

In a previous investigation we found students’ study activity to vary considerably across courses and grade levels (Christopoulos, Rohwer, and Thomas, 1987). Overall, the best predictor of the quality of these study activities was found to be students’ feelings of academic self-efficacy compared to their classmates (Thomas, Iventosch, and Rohwer, 1987). However, we also observed substantial course-to-course variations in students’ study practices that were not explainable in terms of personalogical factors. For example, we observed differences between high school and college students’ test preparation study practices. Despite similarities between these grade levels in the content being studied (history), the importance of tests, and the level of difficulty of the assigned reading, a significantly larger proportion of college than of high school students reported engaging in what we referred to as generative study activities (e.g. taking integrative notes, constructing summaries). Likewise, high school students reported engaging in what we referred to as duplicative activities (e.g. reading and rereading) to a far greater extent than did college students when preparing for tests (Christopoulos et al., 1987). We concluded that these grade level differences in students’ study activities were attributable to other differences we observed between these grade levels in particular features of courses, principally the tendency for tests at the college versus secondary level to be composed of items that required the integration of ideas as opposed to the reproduction of information and the tendency for secondary level but not college instructors to rehearse criterion responses prior to the test (Strage, Tyler, Rohwer, and Thomas, 1987).

The results of this investigation led us to hypothesize that course characteristics can affect students’ study activities in three ways. First, course features can pose demands on students’ study practices. Demands are features that constitute explicit requirements for students to engage in particular activities. Course demands include the amount of material to be learned, the weight of tests in determining course grades, and the degree of intellectual challenge posed by the items on teacher-developed tests. A second type of course characteristics we refer to as supports. Supports are course requirements or provisions that promote student engagement in the types of study activities that meet the demands of the course. Examples of supports from extant studies include feedback and frequent quizzes. A third category of course characteristics we refer to as compensations. Compensations are course features that reduce or eliminate demands, thus abrogating the need, on the part of students, to engage in particular study activities. Often, compensatory practices are those that provide students with the products of studying. An extreme instance of this kind of compensatory practices would be allowing students to view the exam in advance of the test date, thereby reducing the demand on students’ selective processing activities during test preparation.

Elsewhere, we have described models of study activities and of the influence of course features on the study-achievement relationship in some detail (Thomas and Rohwer, 1993). Figure 1 presents the guiding framework for our research. This framework, when applied to a particular course, leads to several overall hypotheses.

Study Activities, Self-concept of Ability, and Achievement 501

First, a substantial portion of the variance in academic achievement can be attributed

* Study Capabmties * Self-concept of

Acsdemic Ability

Feamcs * Cognitive * Effon Management

\

I * Demands u- * suppora * Cornpensapom

I I

Figure 1. Framework guiding research on the interrelationships among student characteristics, course characteristics, students’ study activities, and achievement

to the cognitive processing and effort management activities students engage in while carrying out the work of the course. Second, the quality and quantity of these activities are directly related to (a) characteristics of students, principally, their feelings of self-concept of academic ability and their study capabilities; (b) the demands, supports, and compensatory practices that define the course; and (c) interactions between these student and course characteristics.

The present study was designed to extend the principal findings of the previous investigation to science courses, to examine the mediating effect of course features and student characteristics on the study-achievement relationship, and to focus on testing practices and students’ test preparation activities. The study was also designed to improve on some of the limitations of the previous investigation. First, the present study was conducted within a single grade level, subject-matter area, and instructional unit, thus controlling for differences in the content being studied as well as the age and academic level of students. Second, in order to assess the relative effectiveness of different study activities in these courses, a standard achievement measure was developed for administration in all courses. Third, an improved study activity instrument was developed based on hierarchical models of students’ cognitive and effort management study activities (Thomas and Rohwer, 1993).

Specific course characteristic variables were selected for their presumed effect on students’ test preparation activities based on results from the previous study, infer- ences from extant literature, and results of preliminary interviews with high school science teachers conducted prior to the present study (Thomas, Strage, Bol, and Warkentin, 1990). These course characteristics include demand factors such as workload (Entwistle and Tait, 1990; Natriello, 1987), the production demands on test items (Connor, 1977; Entwistle and Tait, 1990; Rickards and Friedman, 1978), and the cognitive challenge of test items (Christopoulos et al., 1987, Haertel, 1986), support factors such as the provision of feedback to students on their coursework performance (Crooks, 1988; Duckworth, Fielding, and Shaugnessey, 1986; Kulhavy, 1977; Pressley and Ghatala, 1990), and compensatory practices such as the provision of alternative ways of obtaining a passing grade (Sanford, 1987).

The study was carried out within the genetics unit in participating high school

502 J. W. Thomas et al.

biology courses. Thus our examination of students’ study activities, self-concept of academic ability ratings, and achievement scores, as well as our examination of course features, was specific to the genetics unit. Genetics was selected as the focus of the investigation because, according to the results of a prior survey of high school science teachers (Thomas et af . , 1990), different high school instructors vary considerably in their teaching practices when administering the unit, yet the content of genetics seemed fairly standard across courses and textbooks.

METHOD

Participants

Participants consisted of 136 students enrolled in 12 biology courses from seven high schools in the San Francisco Bay Area. Each course was taught by a different teacher. Schools were selected in order to provide a representative mix of socioecono- mic groups. Courses with enrollments of advanced placement or at-risk students were eliminated from the sample.

A group of 10-25 students was randomly selected from each of the courses to serve as the student sample. Where teachers taught more than one section, participating students were selected from all sections.

Instruments and data sources

Students ‘study activities A locally developed self-report instrument, the Study AciivirJ7 Questionnaire (SAQ), was used to measure students’ study activities. The questionnaire items were constructed to correspond to the levels and dimensions of two study activity hierarchies, a cognitive and an effort management hierarchy, developed prior to the study (Tho- mas and Rohwer, 1993). The SAQ was constructed to yield data on six presumably hierarchical study dimensions and one linear dimension (time spent studying). These dimensions are described below:

Level of cognitive processing. A dimension indexing the extent to which students engage in generative or transformational processing while studying. The dimension ranges from (a) the encoding of course content, (b) the selection of important versus unimportant information, (c) the integration of information, to (d) the extension or application of information beyond the given context.

Representational level. A dimension delineating the knowledge products that serve as the ‘content’ of studying. The dimension ranges from (a) lower-level units (facts/ details), (b) mid-level units (concepts/definitions), to (c) higher-level units (main ideas/ principles).

Initiative. A dimension referring to the source of the instigation to engage in particular study activities. The dimension ranges from (a) receptive (following the directives of external sources), (b) reactive (responding to cues about what to do). to (c) proac-


tive (following internal directives for engaging in particular study activities). A high score for initiative reflects the learner’s disposition to be proactive or internally directed while studying.

Autonomous management. A dimension that measures a student’s disposition to use study time to construct or generate study aids (e.g. preparing study material, testing oneself) as opposed to doing or reviewing assigned work (e.g. reading, doing homework). A high score in autonomous management reflects the student’s disposition to exhibit control over learning resources.

Memory augmentation. A dimension indexing the extent to which students engage in activities to make the to-be-learned material more memorable. The dimension ranges from (a) no engagement, (b) duplicative activities (e.g. repeating material over and over), (c) interpretive activities (e.g. putting material in one’s own words), to (d) constructive activities (e.g. making up study aids).

EfSort management. A presumably hierarchical dimension relating to the tendency of students to match their study habit to the demands of the situation. The dimension has three components and four levels. The three components are time, concentration, and learning effectiveness. The four levels are (a) self-monitoring-the disposition to pay attention to time, concentration, or relative mastery; (b) self-regulation-the disposition to take steps to correct difficulties in time allocation, concentration, or perceived mastery; (c) planning-the disposition to take steps to adopt a strategy, in advance of studying, to deal with possible time, concentration, or learning effectiveness difficulties; and (d) evaluation-the disposition to engage, after studying, in assessment of the relative success of practices instituted to manage time, concentration, or learning effectiveness.

Time spent studying. An index of the total time that students reported studying for the course. This measure combined the time students reported studying outside of class during a typical week during the administration of the target unit with the time students reported studying outside of class for the unit test.

The SAQ was designed to assess students’ study activities within multiple study contexts. Item types associated with different levels and dimensions of the hierarchies were constructed for routine studying (e.g. doing reading assignments) and test preparation (e.g. autonomous reviewing in preparation for the unit test). Self-reported ratings within these study contexts were then combined to produce a total score for each scale.

Students’ self-concept of ucudemic ability Students’ self-concept of academic ability was assessed using a single item from the Brookover, Erickson, and Joiner (1967) Self-concept of Academic Ability Test: ‘How do you rate yourself in academic ability compared with others in this course?’ Students were asked to rate their relative ability using a five-point Likert scale ranging from ‘among the poorest’ to ‘among the best’. This method of assessing students’ self-concept of academic ability follows the practice of Covington and others (Cov- ington and Omelich, 1985; Covington, Omelich, and Schwarzer, 1986).


Student achievement A Genetics Achievemenr Test (GAT) was developed for administration to all participating students. The test was designed according to two criteria. First, it was designed to be a reasonable and fair mastery test for all students in all biology courses regardless of what textbook was used. To satisfy this standard, a content analysis was conducted using the three textbooks assigned in participating courses. Items were constructed for only those content areas and skills receiving similar weight and treatment across all textbooks. The second guideline was that the test should contain a balance of items representing two of the hypothesized hierarchical dimensions used to measure students’ study activities. Nine short-answer items were developed representing combinations of (a) level of processing (encoding/selection, integration, vs. extension) and (b) representational levels (facts, conceptsidefinitions, vs. principles).

Course characteristics There were three sources of data on course characteristics: classroom observations, teacher interviews, and document analysis. Observation logs, teacher interview forms, and document analysis procedures were developed in order to obtain information on the course demands, supports, and compensatory practices listed in Table 1. Table 1 also indicates the source of data on the different course features.

Table I . Course characteristic variables and data sources ~~ ~~ ~ ~ ~ ~

Demands Information load mean number of text pages assigned for instructional unit (TI); number

Cognitive challenge of unit test: mean level of production; mean level of processing; mean

Importance of rests: weight of unit tests in determining course grade (TI).

Pracrice opportunities: number of quizzes (TI), total number of practice items (DA). Cognitive challenge of other coursework (practice): mean level of production; mean level

of processing; mean level of representation (DA). Feedback: extent of feedback on tests; extent of feedback on quizzes; extent of feedback

on homework assignments (TI). Teacher support for review activities: provision of sample test questions for review; provision

of teacher-to-student questions during review; extra time provided for student questions during review (T1,O).

of content categories on test (DA); length of instructional unit (in days) (TI).

level of representation (DA).

Supports

Compensations Identity (verbatim overlap between test and practice ifems): total percentage of identity items

Explicit study advice: teachers’ provision of explicit advice on what and how to study (TI,O). Sufet-v ners: extra credit options; open bookhote tests (1,O).

on unit test (DA,O).

Key: TI = teacher interview, 0 =observation, DA =document analysis.

Document analysis procedures focused on items appearing on tests, quizzes, study guides, and worksheets. These items were evaluated relative to three kinds of response requirements: (a) production (recognition, short-answer recall, extended production); (b) cognitive processing (encoding, selection, integration, extension); (c) representation (factsidetails, concepts/definitions, main ideas/principles).


Procedure

All participants were pretested with the study activities instrument (SAQ) during the first few weeks of the school year. The administration of the SAQ took place on the school site during class time using Macintosh personal computers equipped with a Hypercard-based, branching program controlled by students’ responses. Groups of three to five students were given the following instructions:

The Questionnaire is designed to help us investigate what students like you do when studying for a typical high school science course. Your answers will not affect your grades in any way, and your teacher will not see your individual responses. We would like to find out something about your study practices in a science course you have recently completed. Be sure to restrict your answers to what you did in this course. Please answer these questions honestly and accu- rately.

The length of time for completing the questionnaire ranged from 20 to 40 minutes. All additional data were collected from participating courses during or just follow-

ing the unit on genetics. The timing for administration of the unit varied from course to course, ranging from 2 to 6 months following the administration of the pre-test.

All handout materials administered during the genetics unit were collected by project staff, including quizzes, homework assignments, worksheets, and test review handouts. Observers used the observation logs and audiotapes to record classroom events during the last class held prior to the genetics unit test. In all cases, class time during this period was devoted to a review of course content.

During a class session closely following administration of the test covering the genetics unit, the same students who were pre-tested on the SAQ instrument were given the SAQ a second time. The items and procedure for administering the post-test of the SAQ were identical to that for the pre-test except that (a) the directions were altered to ensure that the students responded to the items with reference to their activities during the genetics unit and (b) an additional item was included to assess students’ self-concept of academic ability.

Immediately after students completed the SAQ they were administered the nine- item genetics achievement test (GAT). This test took from 5 to 15 minutes to complete.

Soon after the completion of the genetics unit, individual project staff members interviewed participating teachers. The interview consisted of 16 questions concerning assignment, teaching, grading, and testing practices specific to the instructional unit on genetics.

Each practice and test item from collected course materials was rated by two raters, independently, for level of production, cognitive processing, and representation. Teachers’ unit tests were also scored for the number of different content categories covered by the items on these tests. The inter-rater reliability across all items was .88.

RESULTS AND DISCUSSION

Results and discussion are divided into three parts: characteristics of the instruments and course features, relationships between central variables of the investigation, and multi-level relationships.


Characteristics of the instruments and course features

Coefficient alphas computed for the SAQ scales indicated acceptable levels of internal consistency. Table 2 presents these results. Reliability coefficients for the SAQ scales ranged from .49 to .84, which is comparable with those of other self-report instruments (e.g. Weinstein, Zimmerman, and Palmer, 1990). Table 2 also presents the alpha coefficient measuring internal consistency of the GAT, which was .75.

Table 2. Reliability coefficients for measures used in the study

Measure Study activity survey dimensions* Level of processing Representational level Initiative Effort management Autonomous management Genetics achievement tesi

Reliability

.84

. I5

.84

.I1

.49

.I5

*There were too few items on the memory augmentation scale to yield valid estimates of internal consistency.

Intercorrelations between the various SAQ scales indicate that the scales were relatively independent. These correlations ranged from a high of .59 between level of processing and memory augmentation to a low of .01 between level of processing and time spent studying. The mean correlation across all scales was .26. Additional information on characteristics of the SAQ instrument can be found in Warkentin (1991).

The results presented in Table 3 indicate that there was considerable variation across the sample of courses examined on nearly all indices of instructional demands, supports, and compensations. For example, while instructors assigned an average of 2.66 pages of reading per day, the range extended from 1.13 pages to 4.60 pages per day. Teachers also varied in the extent to which they provided instructional supports for their students. For example, there was an eight-fold difference between the smallest (34) and the largest (228) number of practice exercises assigned during the genetics unit. Finally, there was considerable range in the extent to which teachers provided compensations for the demands they made on their students. For example, across courses, the percentage of items that appeared on teachers’ unit tests that were identical to items students had seen before ranged from 0 to 64 per cent.

Relationships between central variables of the investigation

Relationships between course features and achievement Given the course-to-course variation in course features observed in the sample, we were interested in determining if this variation in demands, supports, and compensatory practices was related to the quality of student learning. Table 4 presents the correlations between course features and scores on the experimenter-developed GAT. For this analysis an average achievement (GAT) score was calculated for each of the courses. These scores were then correlated with each course’s ratings on the features listed in Table 1. It is important to note that these correlational analyses


Table 3.

Course feature Mean SD Low High

Descriptive statistics for course characteristics

Demands Mean level of processing on test (encoding = 1, integration = 2, extension = 3) Mean representational level on test (facts/ details = 1, termddefinitions = 2, principles = 3) Mean production level on test (T/F = 1, MC = 2, fill-in = 3, short answer = 4, essay = 5) Number of content categories tested Weight of tests (in percentage of course grade) Mean text pages assigned (per day) Length of unit (no. of days)

Supports Mean level of processing in practice Mean representational level in practice Mean production level in practice Feedback on tests (none = 0, total score = 1, score for each item = 2, written comments = 3) Feedback on quizzes Feedback on homeworks Sample test questions for review Extra time for student questions in review Teacher to student questions in review Number of quizzes Total number of practice items

Compensations Total percentage of identity items Explicit study advice (yes = 1, no = 0) Open bookhote tests (yes = 1, no = 0) Extra credit (yes = I , no = 0)

1.47

2.21

2.31 10.67

.45 2.66

13.92

1.49 2.23 2.93

2.17 1.33 1.58 s o .25 .67

1.33 I 15.92

.22

.67

.25

.50

.21

.44

.72 2.02

.24

.77 2.23

.22

.26

.52

.94 1.07 1.31 .52 .45 .49 .98

55.50

.44

.49

.45

.52

1.09-

1.57-

1.29- 7.00-

.13- 1.13-

10.00-

1.27- 1.89- 2.35-

.00-

.00-

.O&

.O&

.or& .oo- .O&

34.00-

.O&

.O&

.00-

. 00-

2.07

3.00

4.00 14.00

3 1 4.60

17.00

2.03 2.80 3.79

3.00 3.00 3.00 1 .oo 1 .oo 1 .oo 3.00

228.00

64.00 1 .oo 1 .oo 1 .oo

involving course features are based on a small sample of courses (n=12). In light of this fact, and the exploratory nature of the study, correlations of .44 or higher (p <. 15) were considered to be potentially important and worthy of further investigation.

First, with respect to instructional demands, the results do not support the popular idea that students learn more in courses that have more stringent requirements. In fact, our findings indicate a negative correlation between achievement and both the mean representational level characteristic of teachers’ exam items ( r = -.47) and the weight of tests in determining course grades ( r = -.44). The direction of these correlations suggests that high demand conditions are associated with lower levels of student achievement.

Results for the provision of support practices revealed positive correlations between student achievement and the amount of feedback teachers provide following tests


Table 4. Correlation coefficients between course features and achievement (GAT)

Course features R

Demands Mean level of processing on test Mean representational level on test Mean production level on test Number of content categories on test Weight of tests re course grade Mean text pages assigned Length of unit (no. of days) Supports Mean level of processing in practice Mean representational level in practice Mean production level in practice Feedback on tests Feedback on quizzes Feedback on homework Sample test questions for review Extra time for student questions in review sessions

Teacher to student questions in review sessions Number of quizzes Total number of practice items Compensations Total percentage of identity items Explicit study advice Open booWnote tests Extra credit

.23 - .41* - .05

.28 - .44 - .08 - .25

- .04 - .07 -.21

.62**

.49*

.60**

. I6

-.30 -.I9

.13 - .06

-.38 - .22

. 00 - .46*

*p< . I 5; **p< .05.

(r=.62), quizzes (r=.49), and homework assignments (r=.60). The more extensive the feedback provided to students on their coursework, the higher their GAT scores.

Only one compensatory practice emerged as an important correlate of student achievement. The provision of extra credit was negatively related to GAT scores (r=-.46). The practice of giving students an option to compensate for low test grades by doing extra credit work was associated with lower student scores on the achievement test.

Relationships between study activities and achievement Data on the relationship between students’ study activities and achievement were derived by correlating students’ scores on post-test administration of the SAQ with scores on the researcher developed GAT. The results, displayed in Table 5, indicate positive correlations between achievement and student scores on the level of processing scale ( r = . 18), the representational level scale ( r= .34), and the initiative scale ( r= .30). Students who were proficient at applying the concepts and principles of genetics (a) engaged in higher levels of cognitive processing, (b) focused on higher- level units of information, and (c) were more proactive in their test preparation study activities than students who performed less proficiently on the achievement test.

Correlations obtained between student achievement and other study activity scales

Study Activities, Self-concept of Ability, and Achievement

Table 5. Correlation coefficients between study activity (SAQ) scales and experimenter-developed achievement test (GAT) scores, self-concept of academic ability (SCAAT) and GAT scores, and SAQ scales and SCAAT.

Scales Achievement (GAT) SCAAT

SAQ: Level of Processing .18* .30**

509

SAQ: Representational Level .34** .22* SAQ: Initiative .30** .35**

SAQ: Memory Augmentation . I I . I 3 SAQ: Effort Management .I2 .13 SAQ: Study time -.lo - .09 SCAAT .46** *p<.o5, **p<.Ol

SAQ: Autonomous Management .I5 - .04

were comparatively low and not statistically significant. Similarly, the relationship between achievement and the amount of time students reported studying was found to be negligible.

Relationships between self-concept of academic ability ratings and achievement Table 5 also reports the correlation between self-concept of academic ability (SCAAT) and achievement (GAT). This correlation ( r= .46) is consistent with other investigations of this relationship, as well as with high-school level results ( r = .46) from the previous study conducted in American history courses (Thomas et al., 1987). Students who rated their academic ability as among the best in their class tended to outperform other students on the experimenter-developed measure of achievement.

Relationships between self-concept of academic ability ratings and study activity engagement Table 5 also displays the results for correlations conducted between SAQ scales and SCAAT. As can be seen, a similar pattern emerges as was noted for the relationship between SAQ scales and achievement. Specifically, students’ rating of their self-concept of academic ability was found to be significantly related to three SAQ scales: level of processing (r = .30), representational level ( r = .22), and initiative ( r= .35) . Students‘who rated their academic ability among the best in the class tended to engage in study activities that can be described as generative, proactive, and focused on higher-level knowledge products.

Relationships between course features and students ‘study activities Relationships between course features and students’ study activities were examined using study activity ‘change scores’, differences between post-test and pre-test scores on the SAQ. These change scores represent a more conservative index of the effect of particular course features on students’ study activities in that this index controls for the influence of students’ prior study habits (pre-test scores). The correlations between course features and change scores on each of the SAQ scales appear in Table 6. Not all correlations between course features and SAQ change scores were examined. The correlations displayed in this table are those that have some bearing on hypotheses generated during the design phase of the study (Thomas et a f . , 1990).

There were two course demands that were related to changes on one or more study activity dimensions. The number of content categories covered on the teachers’ unit test was positively related to changes on the memory augmentation scale ( r= .48).

Tab

le 6

. C

orre

latio

n co

effic

ient

s bet

wee

n co

urse

feat

ures

and

SA

Q (p

ost-t

est-p

re-te

st)

scor

es.

SAQ

Sca

les

Leve

l of

Rep

rese

nta-

M

emor

y Ef

fort

Aut

onom

ous

proc

essi

ng

tiona

l lev

el

augm

enta

tion

man

agem

ent

Man

agem

ent

Initi

ativ

e Ti

me

Dem

ands

Mea

n re

pres

enta

tiona

l lev

el o

n

Mea

n pr

oduc

tion

requ

ired

on te

st

Num

ber o

f con

tent

cat

egor

ies

Wei

ght o

f tes

ts re

cou

rse

grad

e M

ean

num

ber o

f tex

t pag

es

Leng

th o

f uni

t in

days

Mea

n le

vel o

f pro

cess

ing

on te

st

-.33

test

-.I7

test

ed

assi

gned

- .0

4

- .3

3

.30

- .2

4

-.I5

-.23

-.34

- .48

*

.70*

* .5

3*

.25

.53*

.oo

-.27 .48*

.0

2 -

.35

- .2

1

.21

- .5

4*

.35

.40

. 00

- .3

4

- .0

9

- .3

8 .1

1 -

.27

.I 1

-.52*

-.0

2 .0

5 .2

5 Su

ppor

1s

Mea

n pr

oces

sing

leve

l, pr

actic

e

Mea

n re

pres

enta

tiona

l le

vel,

Mea

n pr

oduc

tion

requ

ired,

Feed

back

on

test

s so

* .2

1 Fe

edba

ck o

n qu

izze

s .I

0 .5

6*

Feed

back

on

hom

ewor

k ite

ms

.I9

.23

Sam

ple t

est q

uest

ions

for r

evie

w

-.57*

.2

1 .6

2**

Extra

tim

e fo

r que

stio

ns (r

evie

w)

.42

.04

Teac

her c

alls

for s

tude

nt

ques

tions

- .0

4 .2

4

item

s .7

4**

.55*

prac

tice

item

s -.3

9

prac

tice

item

s .6

8**

.57*

Num

ber o

f qui

zzes

-

.53*

-.3

4 .3

6 -.3

0 T

otal

num

ber o

f pra

ctic

e ite

ms

- .5

4 -.

51*

Com

pens

atio

ns

Perc

enta

ge o

f ide

ntity

item

s - .2

9 -.

I8

- .4

0 -.2

1 - .4

9*

- .6

0**

- .7

2**

Expl

icit

stud

y ad

vice

.0

2 -.6

1**

.08

.13

-.45*

O

pen

book

hote

test

s -.

I0

.49

.34

.27

*p<.

15; *

*p<.

OS.

Extra

cre

dit o

ptio

n - .0

2 -.5

7*

-.04

cn

0

+

Study Activities, Self-concept of Ability, and Achievement 51 1

The more content categories that were covered on the test, the more students reported engaging in generative memory augmentation activities. The second course demand found to be related to changes on the SAQ scales was the mean number of text pages assigned. There was a negative relationship between this demand variable and changes on the initiative ( r= - .48), effort management ( r= -.54), and autonomous management ( r= - .52) scales. These findings suggest that high information load, defined in terms of number of text pages assigned, is associated with decrements in students’ reported level of engagement in self-initiated, proactive, and diligent study practices.

In contrast to the pattern for demands, support features were found generally to be associated with positive changes in students’ study activities. The processing level of practice items, which is a summary measure of the intellectual challenge associated with items that appeared on homework and in-class worksheets, was found to be positively correlated with students’ reported level of processing during studying (r=.74) and time spent studying (r=.55). Another support variable related to the cognitive challenge of practice items was the mean production requirement of practice items. This feature was positively related to changes on both the memory augmentation (r=.68) and time spent studying scales ( r = .57). Exposure to more extensive production requirements in practice contexts (e.g. recall versus recognition) was associated with the use of more generative study strategies and more time spent studying during test preparation.

The provision of feedback on tests was also found to be associated with positive changes on study activity scales, specifically, the initiative ( r = .70) and effort management ( r = .50) scales. Similarly, feedback on quizzes was associated with increases on the initiative (r=.53) and the autonomous management (r=.56) scales. Additio- nally, the practice of giving students sample test questions for review was positively related to initiative ( r = .53) and autonomous management ( r= .62) change scores.

Although most of the correlations between support features and SAQ change scores were in the hypothesized positive direction, a number of support features showed significant, negative correlations with changes on SAQ scales. One unexpected finding was that the provision of sample test questions for review was negatively associated with memory augmentation change scores ( r = - .57). Other anomalous find ings were the negative relationship observed between the total number of practice items and scores on the memory augmentation and time scales ( r= - S4) and r = - .5 1, respectively), and that between the number of quizzes administered and scores on the memory augmentation scale ( r= -.53). A possible explanation for these findings is that the more extensive the practice students receive in their coursework, the less disposed they are to see the need for extensive memory augmentation activities and study time investment prior to the test.

Course compensations were expected to be negatively related to changes in SAQ scores, and with one exception, this prediction was confirmed. Giving students test items in advance of the test (percentage of identify items) was associated with decrements in effort management scores ( r = - .49), autonomous management scores (r=-.60) and the total amount of time spent studying (r=-.72). Similar results were found for the provision of explicit study advice (i.e. telling students exactly how and what to study). This provision was found to be negatively correlated with changes in memory augmentation scores ( r = - .67) and with time (-.45). Finally,


the provision of extra credit options was found to be negatively correlated with changes in autonomous management scores ( r = - .57).

The observed pattern of results seems to point to the detrimental effects of compensatory practices on student engagement in extensive, generative, and autonomous study activities. The only exception to this pattern of results was the positive correlation between allowing open bookhote tests and change scores on the effort management scale. It is plausible that this practice acted more like a support for students’ engagement in effort management activities rather than as a compensation for study activities to the extent that students were challenged to prepare study notes or become familiar with the book while preparing for the test.

Results from the analyses concerning the interrelationships between the principal factors in the investigation include several worthy of note. First, with respect to the relationship between course features and achievement, the results of this investigation failed to support the popular notion that student achievement can be increased by means of increasing the academic rigour of course requirements (e.g. National Commission on Excellence in Education, 1983). In fact, two indices of rigorous courses, the presence of highly weighted tests and the administration of tests that focus on higher-level knowledge products (e.g. principles vs. facts), were found to be inversely related to student achievement, as defined by the experimenter-developed achievement test. In addition, the provision of a fairly common compensatory practice, the use of extra credit assignments to make up for failing test grades, was also found to be negatively related to student achievement. In contrast, the provision of extensive feedback on quizzes, homework, and tests was found to be positively associated with student achievement, a result consistent with recent laboratory and classroom research (Crooks, 1988; Kulhavy, 1977).

Second, with regard to the relationship between students’ study activities and achievement, a positive relationship was found between achievement and students’ engagement in study activities described in terms of (a) higher-level cognitive processing (level of processing), (b) a focus on high-level knowledge products (representational level), and (c) self-directed practices (initiative). These results are consistent with those of other studies in which proficient performance in a subject matter area is linked to engagement in deep, elaborative processing of learning material (Ferguson- Hessler and de Jong, 1990; Mayer, 1987), the disposition to recognize and focus on higher-level propositions during learning (Einstein, Morris, and Smith, 1985; Meyer, Brandt, and Bluth, 1980), and engagement in self-directed, self-regulated learning activities (Zimmerman, 1990).

Third, results from the present investigation seem to confirm the importance of students’ self-concept of academic ability in the study-achievement relationship. In line with the results from other investigations, self-concept of academic ability was found to significantly predict students’ achievement (Brookover, 1987; Gadzella and Williamson, 1984; Mboya, 1989; Thomas et al., 1987; Wilhite, 1990). Further, self- concept of academic ability was found to be positively associated with student engagement in the specific study activity dimensions found to be related to academic achievement (Thomas et al., 1987; Wilhite, 1990). Thus, indirect evidence was provided for one of the overall hypotheses of the present study that the causal mechanism behind the relationship between self-concept of academic ability and achievement

Study Activities, SelJlconcept of Ability, and Achievement 5 13

involves the tendency for self-efficacious students to be more disposed to engage in selective, generative, diligent, and self-directed study activities than their peers.

Finally, the results obtained from analyses linking specific course features to particular study activity dimensions suggest that students’ study practices are conducted in response to the demands, supports, and compensatory practices that define a course of instruction. With respect to demands, relationships between students’ study activities and information load, test difficulty, and other demand characteristics were either negligible or negative. This failure to observe a positive relationship between academic rigour and the quality of students’ study activities is consistent with results from other studies in which students enrolled in courses that are perceived to have difficult requirements or tests reported engaging in low-level, desultory study activities (Entwistle and Tait, 1990; Natriello, 1987).

In contrast to the pattern for demands, several course supports were found to be associated with student engagement in active, generative, or diligent study activities. This finding for instructional supports, which included requiring students to engage in high-level cognitive processing and to produce extended responses when completing practice material, giving students extended feedback on coursework, and offering sample test questions for review, is in line with extant studies describing the importance of feedback, homework, guided learning practices, and other environ- mental supports for fostering proficient learning practices on the part of children and adolescents (Bereiter and Scardamalia, 1987; Brown and Reeve, 1987; Pressley and Ghatala, 1990).

Compensatory practices were expected to impede student engagement in generative, diligent study practices. This prediction was largely confirmed, at least for the practices of giving students test items in advance of the test, providing explicit study advice, and allowing extra credit alternatives. An unexpected finding with respect to compensatory practices was the positive relationship found between open book/ note tests and changes in effort management scores. In retrospect we had assumed that the effect of open book tests would be to reduce the demands on students’ study activities. However, the provision of open book or open note tests may have merely changed the nature of the demand. Students preparing for open book or open note exams may have been challenged to manage their study time and effort in deliberate ways in order to try to meet the anticipated demands of preparing study notes or using the book most efficiently during the test period.

According to the present hypotheses, the incidence and effectiveness of students’ study practices is expected to vary between courses due to differences in the characteristics of these courses. The results of the present study indicated substantial variability, across the courses surveyed, in the relationship between study activity dimensions and achievement (see Warkentin, 1991). Thus, it might be speculated that the relationship between study activity engagement and achievement, or that between self-concept of academic ability and study activity engagement, might vary with characteristics of courses. For example, the relationship between achievement and students’ level of processing during test preparation might tend to increase under conditions of highly demanding tests and diminish under conditions of highly supportive practice provisions. A second set of analyses examines the moderating role of course characteristics on relationships between student level factors.

5 14 J. W. Thomas et al.

Multi-level relationships

According to our conception, the incidence and effectiveness of engaging in various study activities may vary across courses depending on characteristics of students and on the pattern of demands, supports, and compensations that make up these courses. Note that predictions of this kind pertain to influences on the criterion performance of individuals that stem from two distinct levels: an individual differences level (study activities, student characteristics, achievement) and a course level (course characteristics). Such multi-level data can be appropriately analysed using methods based on hierarchically linear models.

Suppose that we represent the performance of student i in course j on the criterion variable by XJ (we follow the notation of Raudenbush and Bryk, 1988). Then if X u is that student’s measure on a selected SAQ variable, we use a within-course equation to model dependence of criterion performance on study activity:

where R, is a random error associated with student i in course j . Note that this can be done within each course, so we end up with as many intercepts CS,) and linear regression coefficients (Bj,) as there are courses. A selection of these regressions for a particular study skills variable (effort management) is shown in Figure 2. These regressions have been arranged down the page in a particular way. At the top is a course where the teacher provides written feedback on quizzes. In the next two courses, the teacher provides a score for each item in the quiz. In the next, the teacher provides only an overall score on the quiz. Finally, in the bottom course, the teacher provides no feedback at all. What is interesting about the regressions that are estimated within each of these courses is that the regression slope becomes more positive (i.e. steeper) as the quality of feedback increases.

We can display this in a more compact form by plotting the regression slopes as a function of the course characteristics variable, feedback on quizzes. Figure 3 shows where each of the five within-course regressions from Figure 2 occur on this plot. Figure 4 gives all of the within-course regressions, with each one represented merely as a dot. Thus, we can interpret this to imply that the effect of effort management on student performance is enhanced by higher-quality teacher feedback on quizzes. This analysis occurs at the between-course level, and can be represented by the following equation:

where yo and Y l k are the between-course intercept and linear regression coefficient for within-course effect k, respectively (in our case k is either 0 or 1, and we shall concentrate on k =l), y is the measure on the course characteristic for course j , and u,k is a random error associated with course j and within-course regression weight b,k . Thus, when we noted in Figures 2 4 that there was a positive correlation between within-course regression slope and quality of feedback, this corresponds to noting that Y ] k is estimated to be positive. This procedure could be done step-wise with many regressions at the within-course level and one at the between-course level. The HLM approach uses an iterative procedure that effectively estimates both levels simultaneously. This can be shown to be more appropriate where the data conform to the error assumptions in equations (1) and (2) (Raudenbush, 1988). Moreover,


Writ ten feedback

1

i

1 Score

for

each

itern

r 1 r 1

Overall score

No feed bsc k

'C 4

: z 0 6E 0

-4 -3 -2 - 1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4

b . , , _ . * _ . - . . , , -4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4

-4 -3 -2 -1 0 1 2 3 4 Pre-Port difference scores on the

Effort Hanqment Scrk

Figure 2. Fitted linear regressions relating changes in the relationship between effort management study activities (post-test-pre-test change scores) and GAT scores for each level of

feedback on quizzes for five classrooms

5 16 J. W. Thomas et al.

the HLM approach provides an overall test of whether the observed relationship is statistically significant (Raudenbush, 1988).

-1

l ' C 1 2

0 4 3 - 2 - 1 0 1 2 3 4

'1 2

o l 4 3 - 2 - 1 0 1 2 3 4

10

4 2

0 . mL 4 3 - 2 - 1 0 1 2 3 + 0 4 4 - 2 - 1 0 1 2 3 4

4 2 . 0 44-2-10 1 2 3 *

-2 -23 5 0 1 2 3

No Overall Score for Written feedback score each item feedback

feedback on Quizzes

Figure 3. Fitted linear regressions relating changes in the relationship between effort management study activities and GAT scores for each level of feedback on quizzes for the five class-

rooms shown in Figure 2

The present data were analysed using the computer program, HLM2 (Bryk, Rau- denbush, Seltzer, and Congdon, 1986). In an attempt to control for pre-existing differences in the naturally occurring courses, and in order to isolate potential effects of the course characteristics for the particular unit under study, we use as within- course predictors (X,,s above) diflerences in student study variables. Thus, we are modelling the effect of changes in students' study activities on students' performance (YUs) above) at the individual level. According to the teachers in the sample, this unit is the first time that students have encountered genetics in their school curriculum. Thus, we assume for interpretive purposes that all students taking a pre-test using the researcher-developed achievement test would receive a score of zero. Scores on this achievement test were analysed by the HLM program, with each of the separate SAQ scale scores used as the within-level predictor variable. Thus, for a given SAQ scale and course characteristic, all other things being equal, an estimate of a statistically significant y l k , indicates that, assuming the estimate is positive, changes in study behaviour (as measured by the SAQ scale) become more effective (i.e. result in greater increases in achievement per unit increase in SAQ scale) with increases in the course characteristic variable. Note that this interpretation relates only to the rate of increase, y l k . Interpretation may be somewhat more complicated when the intercept, yak, is also found to be different from zero. In the analyses reported below, yOk was found to be non-significant except where reported otherwise.


2.

1 .5.

1.

.5. Regression Coefficient 0

- .J

-1.

-1 5.

-2.

-2.5 +

:;; 8

Feedback on Quizzes Figure 4. Within-course coefficients of regression of changes in effort management study

activities on GAT scores as a function of feedback on quizzes for all 12 classrooms

Relationships between study activities and achievement as mediated by course features The between-level, course characteristic variables used in these HLM analyses were selected as those hypothesized to affect student engagement in the different levels and dimensions of the study activity hierarchies. That is, the course characteristic framework was used as a guide for the selection of a limited number of course variables that were expected to be relevant for each SAQ scale. Of these analyses, only those significant at the 5 per cent level are reported in this paper. These should not be considered strict statistical significance tests, as there was no proper sampling design for either level, and as the analyses share a great number of variables. Rather, the analyses should be considered to be exploratory.

There were five statistically significant course characteristic regression coefficients at the 5 per cent significance level. The summary of results that follows is organized by the pairs composed of the study activity and course characteristic variables from each analysis.

Eflort management change scores on the SAQ: feedback on quizzes. A higher level of feedback on quizzes is associated with increases in the relationship between effort management change scores and achievement (Figure 4). The more extensive the feedback provided on quizzes the stronger is the relationship between effort management study activities and achievement.

The provision of feedback to students on their coursework has been acknowledged as an effective teaching practice for enhancing performance (Crooks, 1988; Kulhavy, 1977). There is also some evidence suggesting that feedback is associated with more productive types of study behaviours (Duckworth et aZ., 1986). The present finding extends these results, suggesting that extensive feedback practices prompt students


Repression Coefficient ,

-.5,

to engage in diligent effort management activities which, in turn, is associated with high achievement.

i t

Efort management change scores on the SAQ: feedback on homework. High levels of feedback on homework are associated with decreases in the relationship between effort management change scores and achievement (Figure 5). The provision of extensive feedback provided on homework is associated with a diminishment in the relationship between effort management activities and achievement. Stated in the opposite way, this finding suggests that, in courses in which little or no feedback is given on homework assignments, the relationship between effort management activities and achievement is enhanced.

-2.5 1 0 1 2 3 No Overall Score for Written

feedback score swh item feedback

Feedback on homvork Figure 5. Within-course coefficients of regression of GAT scores on changes in effort management study activities (post-test-pre-test difference scores) as a function of feedback

on homework

Ostensibly, this result contradicts the hypothesis that high levels of support provided by teachers on students’ coursework, including homework, would be associated with more effective change in reported effort management activities. However, closer examination of the results shows that this apparent contradiction can be resolved. Note that Figure 5 shows the relationship between feedback on homework and the regression coefficients relating change in effort management to achievement (i.e., y l k in (2)), but this is not the only coefficient in (2): we must not forget the intercept yok. In this case the intercept is also significantly different from zero, indicating that interpretation must include the effect of this coefficient. Figure 6 shows the fitted linear regressions relating change in effort management to achievement for each of the values for feedback on homework. Examination of the figure shows


N6 Fcedbac k I 4 1 k 4 1 :v. f.,’, , ~ iv:.., , , . , , O L O I . . . . ,

4 3 - 2 - 1 0 1 2 3 4 4 4 - 2 - 1 0 1 2 3 ~ 4 3 - 2 - 1 0 1 2 3 4 4 ~ - 2 - 1 0 1 2 3 4

r

aachitern Score for rb?:, 4 l/l?jt<y 0 0 4 4 - 2 - 1 0 1 2 3 4 4 - 3 - 2 - 1 0 1 2 3 4 4 4 - 2 - 1 0 1 2 3 4

Written m a feed bhc k

Figure 6. Fitted linear regressions relating changes in the relationship between effort management study activities and GAT scores for each level of feedback on homework

that the ‘effect’ of giving differing levels of feedback is indeed in line with our prediction (that is higher levels of feedback are associated with higher achievement for a given level of change in effort management), except at the highest levels of change. But, it is also true that the effectiveness of changes in effort management diminishes with increasing feedback at a value on the change in effort management scale of approximately 2. This diminishment results in a reversal of the ‘effect’ of feedback, so that beyond this point, giving higher levels of feedback is counterproductive. This point on the change in effort management scale is one that few students in our sample attained-a typical student at this point would have had to give the most extreme possible responses (negative on pre-test, positive on post-test) to all, or all but one, of the questions on the scsle. Thus, we can say that for most students, giving higher levels of feedback on homework is associated with higher achievement at a given level of change in effort management. However, when the student makes very dramatic changes in effort management, the relationship is reversed.

It should be noted that in the Duckworth et al. (1986) study, the student ratings pertained to feedback provided on tests, not on homework or other coursework. It may be the case that students more frequently attend to and modify their study activities based on feedback received on tests or quizzes rather than on homework, which is a more routine course activity. Another possibility is that the type of feedback (i.e. the content and extent of written comments) provided to students may differ systematically for homework versus quizzes and tests. In fact, if teachers tend to give high-quality feedback on either quizzes or homework, but not both (which


2.

1.5.

1 .

was the trend in our small sample), then this might explain the apparently contradic- tory relationship between this result and the previous one.

Autonomous management change scores on the SA Qfeedback on homework. The provision of more extensive feedback on homework is associated with decreases in the relationship between change scores on the autonomous management scale and student performance on the achievement measure (see Figure 7). These findings are similar

0

-1.5 - I 1

-2SJ 3

Written feedback score each item feedback

0 1 2 No Overall Score for

Feedback on homework

Figure 7. Within-course coefficients of regression of GAT scores on changes in autonomous management study activities (post-test-pre-test difference scores) as a function of feedback

on homework

to those reported for the effect of homework feedback on the relationship between effort management scores and achievement. In fact the situation is almost identical, with a significant intercept giving a relationship that is very similar to that shown in Figure 6 , except that the point at which all the regressions intercept is somewhat further to the right, at a change score where no student actually scored on the autonomous management scale. Thus, the ‘effect’ of giving differing levels of feedback is in line with our prediction (that is, higher levels of feedback are associated with higher achievement for a given level of change in autonomous management), except at the highest levels of change. But it is also true that the effectiveness of changes in autonomous management diminishes with increasing feedback. These results are difficult to explain, and may require further research on the types of feedback provided on different course assignment. Perhaps the provision of extra homework feedback tends to act as a ceiling on the possible effects of increases in autonomous management activities.


2.

1.9.

Autonomous management change scores on the SA Q: extra time provided for student questions during teacher-led reviews. The teaching practice of allotting extra time for students to ask questions during the test review session is associated with increases in the relationship between autonomous management change scores and achievement (Figure 8). This finding suggests that providing students with an opportunity to ask questions about the up-coming test enhanced the effectiveness of changes in autonomous management activities.

-1

Regression Coefficient

0

NO

1 Yes

Extra Time for Student Questions

Figure 8. Within-course coefficients of regression of GAT scores on changes in autonomous management study activities (post-test-pre-test difference scores) as a function of extra time

for student questions

This result confirms our prediction that providing students with the opportunity to ask questions about the test can be a supportive teaching practice that informs students about the kinds of test items to expect and how to best study for the test. The research literature suggests that student expectations about task demands or learning goals influences their study activities and achievement (e.g. Hanmaker, 1986; Marton and Saljo, 1976; Rothkopf, 1973).

Memory augmentaiion change scores on the SAQ: number of content categories covered on the unit test. As the number of content categories on the test increases (see Figure 9), there is a corresponding decrease in the relationship between changes in memory augmentation scores and achievement test performance. The greater the number of categories on the test, the less effective are changes in memory augmentation strategies.

This result suggests that increasing the difficulty of the test by increasing the number of content categories covered may not lead to more productive memory augmentation strategies. A similar result was reported by other researchers (Entwistle and Ramsden,


2 -

1 -

-3i 1

9 to 1 1 12 13

Number o f Content Categories on the Test

Figure 9. Within-course coefficients or regression of GAT scores on changes in memory augmentation study activities (post-test-pre-test difference scores) as a function of number

of content categories on the test

1983; Entwistle and Tait, 1990) who found that students who described their courses as demanding in terms of workload (e.g. large amount of assigned reading) tended to engage in more superficial, unproductive types of study strategies. In fact, Natriello (1 987) suggested that the relationship between workload and study practices might be curvilinear. When the workload or requirements of the course become too demanding, students resort to superficial types of study practices.

Ri2lationslzips between self-concept of academic ability and achievement us mediated by course characteristics In this section we report a set of HLM analyses in which three course-level variables are used to predict the individual-level regression coefficients relating students’ self- concept of academic ability and achievement. In light of the small sample of courses and the exploratory nature of this investigation, a liberal level of significance (p<. 10) was employed for selecting the following results.

Demand characteristic: weight of tests. The outcome of the first HLM analysis in this series involves the course characteristic, weight of tests, in determining students’ final grade in the course. As can be seen in Figure 10, upper panel, in courses in which tests are given relatively high weight in determining students’ overall grade, there is an increase in the magnitude of the within-level regression coefficients relating SCAAT scores and achievement performance. That is, in accord with our hypotheses, the relationship between SCAAT and achievement is intensified under conditions of high demand on students’ autonomous test preparation study activities. Stated differently, in courses in which overall grades are determined to a great extent by test performance, students’ level of achievement is predicted better by their feelings

Study Activities, Seif-concept of Ability, and Achievement 523

2.5.' 2 -

1.5

.5 - 0.

-.5-

1 :


8 a - 8 - 8 8 8

8 8

3.5 8

8 8

.10 .20 .30 .40 .SO .60 .70 .SO .90 Weight o f Tests in Final Grade

3.5- 3.8

8 .5 - 0.

- .5- . 8

0 1 2 3 No Overall Score for Written

feedback score eachitern feedback

Feedback on Homework

3.5 31 a


No Yes Extra Credit

Figure 10. Within-course coefficients of regression of GAT scores on self-concept of academic ability (SCAAT) scores as a function of demands (weight of tests, upper panel), supports (feedback on homework, middle panel), and compensations (extra credit provisions, lower

panel)

524 J. W. Thomas et a1

of self-efficacy than is true in courses in which grades are determined by non-test criteria.

Support characteristic: feedback on homework. The middle panel of Figure 10 displays the result for a second course characteristic, the extent of feedback on homework. The provision of feedback on homework tends to diminish the relationship between SCAAT and achievement. Under conditions of minimal or no feedback, high achievement is more strongly predicted by SCAAT scores than in courses in which more extensive feedback is provided.

Compensatory practice: extra credit. The final HLM analysis reported in this set involves a course characteristic we refer to as extra credit. This is a dichotomous variable that refers to the instructor’s policy of allowing students to make up for low test grades by doing an extra assignment. As shown in the bottom panel of Figure 10, the provision of extra credit options diminishes the relationship between SCAAT and achievement; in the absence of options for making up a failing grade, the relationship between self-concept of ability and achievement is enhanced.

Relationships between self-concept of academic ability and study activities as mediated by course characteristics We regressed selected course-level variables on selected combinations of individual difference variables (the regression coefficients relating students’ self-reported engagement in study activities to their scores on the SCAAT). Our overall hypothesis was that the relationship between self-concept of academic ability and students’ engagement in diligent, generative study activities would be enhanced under conditions of rigorous course demands and reduced under conditions of supports and compensatory practices. Twenty-four HLM analyses were performed but only those that were significant (p < .lo) are presented here.

Demands. Figure 11 shows the results of an HLM analysis involving the demand characteristic, number of content categories on the test. Engagement in memory augmentation was found to be positively related to SCAAT scores in courses in which the number of content categories on the test was relatively few, but negatively related to SCAAT scores in courses where the number of content categories was relatively high. Thus, in contrast to our general hypothesis, high demand, as defined by the breadth of test coverage, did not result in an enhancement of the relationship between self-concept of academic ability and engagement in generative memory- augmentation.

Support provisions. Figure 12 displays the results of three HLM analyses. As can be seen in the figure, increases in the support provision, feedback on homework, was associated with decreases in the magnitude of the relationship between SCAAT scores and students’ engagement in autonomous management activities (upper panel) and that between SCAAT scores and engagement in effort management (middle panel). Thus, although we found the relationship between SCAAT and these two study activity dimensions to be negligible when we examined the relationships across courses, the magnitude of these relationships in any particular course seems to depend on the presence of support features.



9 10 1 1 12 Number of Content Categories

on Test

9 10 1 1 12 Number of Content Categories

on Test

Figure 11. Within-course coefficients of regression of memory augmentation study activities on self-concept of academic ability (SCAAT) as a function of the demand condition, number

of content categories on the test

Finally, an additional support feature, feedback on quizzes, was found to enhance the relationship between SCAAT and engagement in effort management activities. In contrast to our overall hypothesis, with increases in the course characteristic, feedback on quizzes, an increase was observed in the magnitude of the relationship between engagement in effort management activities and scores on the SCAAT. This pattern can be seen in the lower panel of Figure 12. This finding suggests that the disposition to engage in diligent time and effort management activity is more dependent on students’ self-perceptions of ability when teachers give extensive feedback on routine tests than when they do not. A possible explanation for this finding that might be pursued in future research is that test feedback, as opposed to coursework feedback, may be particularly oriented toward grades and class stand- ing. Performance feedback of the kind that follows test events may be debilitating for low self-concept of ability students in ways that routine feedback on coursework is not. Thus the presence of extensive feedback following tests may enhance the importance of self-worth factors in subsequent test preparation situations. An alternative explanation is that feedback on homework may provide more detailed corrective information or remedial advice than does feedback on quizzes. Thus, feedback on homework, but not feedback on tests, serves to enhance the success expectations of all students, thus reducing the relationship between self-concept of academic ability perceptions and the disposition to engage in diligent, test-preparation study activities.

Compensatorypractices. Figure I3 presents the results for two compensatory features, the provision of extra credit options and the practice of developing unit tests charac- terized by verbatim overlap (‘identity’) between a teacher’s unit test items and items appearing in handout material. Looking first at the upper panel, it can be seen that in courses in which teachers gave students the option to make up for poor test grades with extra credit assignments, the relationship between SCAAT and students’ engagement in autonomous management activities is diminished. In contrast with this finding, and with overall hypotheses, the compensatory practice of developing tests with a relatively high number of items that are identical to items students have seen before was found to enhance the relationship between SCAAT and initiative (middle panel) and that between SCAAT and effort management (lower panel).

526 J . W. Thomas et al.

Regression Coeffrcient - 'z ,5 p-$ - 1

I ." No O v i r a l l S c o A for WAtten

feedback score each item feedback

Feedback on H o m v w k

Regression Coefficient .:: 0

0 -.5 . b

No Overal l Score for Wri t ten feedback score each i t e m feedback

Feedback on Homework

2 I . 5 ]


L

No Overal l Score for Wr i t ten feedback score each item feedback

Feedback on Quizzes

Figure 12. Within-course coefficients of regression of self-reported engagement in study activities on self-concept of academic ability (SCAAT) as a function of support conditions. Upper panel: SCAAT and engagement in autonomous management activities as a function of feedback on homework. Middle panel: SCAAT and engagement in effort management activities as a function of feedback on homework. Lower panel: SCAAT and engagement

in effort management activities as a function of feedback on quizzes


Regression Coefficient . i r-w - .5 - 1 1

0 - 1 S j

No Yes

Extra Credit



. . . . . I . - -

0 .10 .20 .30 .40 .50 .60 .70 Percent of Identity Items on Test

1.51 0

1.5.

1 -

0

0 .10 .20 .30 .40 .SO .60 .70

0

0

- .5 0 .10 .20 .30 .40 .SO .60 .70 Percent o f Identity Items on Test

Figure 13. Within-course coefficients of regression of self-reported engagement in study activities on self-concept of academic ability (SCAAT) as a function of compensatory practices. Upper panel: SCAAT and engagement in autonomous management activities as a function of the provision of extra credit assignments. Middle panel: SCAAT and engagement in initiative activities as a function of the percentage of identity items on test. Lower panel: SCAAT and engagement in effort management activities as a function of percentage of identity items

on test.


Finally, as displayed in Figure 14, students’ engagement in memory augmentation activities is decreased by the provision of explicit study advice. Engagement in memory augmentation is positively related to SCAAT scores in courses that provide no study advice, but negatively related to SCAAT scores in courses where the teacher provides explicit information about what will be covered on the unit tests. The disposition to engage on one’s own in memory augmentation activities depended, in those courses, largely on students’ self-perceptions of ability. In courses in which the instructor was more forthcoming about what the test would cover, students’ willingness to engage autonomously in memory augmentation activities was not as dependent on their perceptions of academic ability.


No Yes

Study Advice

Figure 14. Within-course coefficients of regression of memory augmentation activities on self-concept of academic ability (SCAAT) as a function of the compensatory practice, provision

of explicit study advice

CONCLUSIONS

The results presented in this paper provide a basis for both further research and theory construction and for the improvement of educational practice. With respect to implications for research and theory, five tentative conclusions can be offered.

First, an attempt was made in this investigation to describe the dynamics of the relationship among study activities, context factors, and achievement. In keeping with our hypotheses that specific classes of study activities are influenced by specific features of the instructional context, and further, that the quality of learning outcomes following studying is specific to the activities engaged in during studying, the attempt was made to assess specific and hierarchical attributes of students’ study activities, course features, and achievement. The extent to which we were successful is limited by the small size of the participant sample. Nevertheless, the systemic nature of the design introduced in this study is regarded as important for the successful development of a psychology of studying (Rohwer, 1984). Future investigations might be expanded to include a larger number of courses, additional subject-matter areas, attention to students’ beliefs about learning and studying (Schoenfeld, 1983), and the conditions under which students’ improve their study strategies spontaneously (Brown, Bransford, Ferrara, and Campione, 1983).


Second, the procedure used to gather data on students’ study activities has several features to recommend it for future investigations. In contrast to most extant investigations of students’ study activities, even investigations designed to assess the role of context factors in students’ study practices (Entwistle and Ramsden, 1983; Keeves and Larkin, 1986; Thomas and Bain, 1984), the present investigation (a) surveyed students’ activities rather than attitudes or beliefs, (b) surveyed these activities across multiple study contexts, (c) asked students to report their activities with reference to a particular course of instruction, (d) used repeated measures in order to assess students’ adaptation to these courses in their study activities, and (e) obtained data on students’ reported engagement in different activities using a computer-administered, branching response program rather than more cumbersome paper-and-pencil methods.

A third result that should inspire future research is the extent to which the often strong positive relationship between students’ feelings of self-efficacy and their engagement in autonomous learning activities and subsequent achievement can be reduced by the administration of instructional provisions designed to help students cope with the demands of a course. Apparently, in tough or unsupportive courses, only confident students are willing to engage in diligent, generative, study activities. Under these conditions their peers may be unwilling to risk the threat to self-worth that would accompany failure after having given their best effort (Covington, 1984). More research is required in order to identify systems of instructional supports that could be provided in order to maintain both academic rigour and student confidence.

Fourth, the results demonstrate, at least preliminarily, the situation-specific nature of students’ study activities. That is not to say that there is no validity to the popular notion that students have pervasive styles of studying. However, it suggests that taking contextual factors into account, especially demand, support, and compensatory features of courses, might improve the validity of future investigations of students’ study practices. Acknowledging the situated nature of cognition (cf. Greeno, 1989; Lave, 1988) in this case means acknowledging the extent to which students’ study activities are influenced by the features of courses they take.

Overall, the results of this study point to the important role that teacher provisions play in prompting and impeding student engagement in productive, demand-responsive study activities. Whereas increasing the demands of a course (more rigorous tests, stricter grading standards, greater workload) may not prompt students to be more diligent and productive in their autonomous learning activities, nor is it apt to be associated with gains in achievement, giving students guidance, feedback, and rigorous practice exercises that are congruent with criterion demands do seem to prompt and maintain students’ engagement in demand-responsive study activities of the kind associated with high achievement. On the other hand, there are teacher provisions that appear to sabotage student effort and achievement. Teachers who provide students with handouts containing test questions or answers that will appear on a subsequent unit test, for example, may be acting with the intention of helping students and minimizing failure. However, provisions of this kind seem to discourage students from engaging in active, diligent, generative study activities on their own, and may even impede the development of competence at autonomous learning associated with academic success at subsequent grade levels (Thomas et al., 1991). That the presence of compensatory practices is widespread at the secondary school level (Sanford, 1987; Strage et al., 1987), and that differences between supports and com-

530 J. W. Thomas et a1

pensatory practices are quite subtle, make it a prime area for future research and teacher training.

ACKNOWLEDGEMENTS

Requests for reprints should be sent to John W. Thomas, Beryl Buck Institute for Education, 18 Commercial Boulevard, Novato, CA 94949, USA. The research reported in this article was supported by a grant from the US Office of Educational Research and Improvement (R 1 17E80189).

REFERENCES

Bereiter, C. and Scardamalia, M. (1987). Intentional learning as a goal of instruction. In Resnick (Ed.) Knowing, learning and instruction. Essays in honor of Robert Glaser. (pp.361- 392). Hillsdale, NJ: Erlbaum.

Brookover, W. B., Erickson, E. L. and Joiner, L. M. (1967). Self-concept of ability andschool achievement, ZZI. US Office of Education Cooperative Research Report, Project No. 2831). East Lansing, MI: US Office of Research and Publications, Michigan State University.

Brookover, W. B. (1987) The self-concept of ability scale-a review andfurther analysis. East Lansing, MI: Michigan State University.

Brown, A. L., Bransford, J. D., Ferrara, R. A. and Campione, J . C. (1983). Learning, remem- bering, and understanding. In J . H. Flavell and E. H. Markman (eds), Eandbook of child psychology: cognitive development, Vol. 3 (pp. 77-1 76), New York: Wiley.

Brown, A. L. and Reeve, R. A. (1987). Bandwidths of competence: the role of supportive contexts in learning and development. In L. S. Liben (ed.), Development and learning: conjlict or congruence (pp. 237-252). Hillsdale, NJ: Erlbaum.

Bryk, A. S., Raudenbush, S. W., Seltzer, M. and Congdon, R. T. (1986) An introduction to HLM: computer program and user’s guide. University of Chicago, Department of Edu- cation.

Connor, J . M. (1977). Effects of organization and expectancy on recall and recognition. Memory and Cognition, 5 ,3 15-3 18.

Covington, M. V. (1984). The motive for self-worth. In R. Ames and C. Ames (eds), Research on motivation in education: student motivafion, Vol 1 (pp. 78-113). New York: Academic Press.

Covington, M. V. and Omelich, C. L. (1985). Ability and effort valuations among failure- avoiding and failure-accepting students. Journal of Educational Psychology, 77,446459.

Covington, M. V. and Omelich, C. L. and Schwarzer, R. (1986). Anxiety, aspiration, and self-concept in the achievement process: a longitudinal model with latent variables. Motiva- tion and Emotion, 10, 71-88.

Crooks, T. J. (1988). The impact of classroom evaluation practices on students Review of Educational Research, 58,43848 1.

Duckworth, K, Fielding, G. and Shaugnessey, J. (1986). The relationship ofhigh school teachws’ class testingpractices to students’feelings qf eficacy and efforts to study. Eugene, OR: Center for Educational Policy and Management, Oregon University.

Einstein, G. O., Morris, J. and Smith, S. (1985). Note-taking, individual differences, and memory for lecture information. Journal of Educational Psychology, 77,522-532.

Entwistle, N. J . and Ramsden, P. (1983) Understanding student learning. London: Croom Helm.

Entwistle, N. J. and Tait, H. (1990). Approaches to learning, evaluations of teaching, and preferences for contrasting academic environments. Higher Education, 19, 168-194.

Ferguson-Hessler, M. G. M. and de Jong, T. (1990). Studying physics texts: differences in study processes between good and poor performers. Cognition and Instruction, 7,4 1-54.

Study Activities, Self-concept of Abili ty, and Achievement 53 1

Gadzella, B. M. and Williamson, J. D. (1984). Study skills, self-concept, and academic achieve-

Greeno, J. G. (1989). A perspective on thinking. American Psychologist, 44, 134-141. Haertel, E. (1986, April). Choosing and using classroom tests: Teachers perspective on assess-

ment. Paper presented at the annual Meeting of the American Educational Research Associa- tion, San Francisco.

Hamaker, C. (1986) The effects of adjunct questions on prose learning. Review of Educational Research, 56,212-242.

Keeves, J . P. and Larkin, A. L. (1986). The context of academic motivation. International Journal of Educational Research, 10,205-2 13.

Kulhavy, R. W. (1977). Feedback in written instruction, Review of Educational Research, 47,211-232.

Lave, J . (1988). Cognition in practice, Cambridge: Cambridge University Press. Mayer, R . E. (1987). Instructional variables that influence cognitive processes during reading.

In B. K. Britton and S. M. Glynn (eds), Executive controlprocesses in reading (pp. 201-216). Hillsdale, NJ: Erlbaum.

Marton, F., and Saljo, R. (1976). On qualitative differences in learning: 11. Outcome as a function of the learner’s conception of the task. British Journal of Educational Psychology,

Mboya, M. M. (1989). The relative importance of global self-concept and self-concept of academic ability in predicting academic achievement. Adolescence, 24,3946.

Meyer, B. J. F., Brandt, D. N. and Bluth, G. J . (1980). Use of the top level structure in text: Key for reading comprehension of ninth-grade students. Reading Research Quarterly,

National Commission on Excellence in Education (1983). A nation at risk: The imperutive for educational reform. Washington, DC: U.S. Department of Education.

Natriello, G. (1987). The impact of evaluation processes on students. Educational Psychologist, 22, 15-175.

Pressley, M. and Ghatala, E. (1990). Self-regulated learning: Monitoring learning from text. Educational Psychologist, 25, 19-33.

Raudenbush, S. W. (1988) Educational applications of hierarchical linear models: a review. Journal of Educational Statistics, 13,85-116.

Raudenbush, S. W. and Bryk, A. S. (1988). Methodological advances in analyzing the effects of schools and classrooms on student learning. Review of Educational Research, 15,423--474.

Rickards, J. P. and Friedman, F. (1978). The encoding versus the external storage hypothesis, Journal of Educational Psychology, 3, 136-143.

Rohwer, W. D. Jr. (1984). An invitation to a developmental psychology of studying. In F. J. Morrison, C . A. Lord, and D. P. Keating (eds), Advances in applied developmental psychology, Vol. 1 . (pp. 1-57). New York: Academic Press.

Rohwer, W. D. Jr. and Thomas, J. W. (1989). The role of autonomous problem-solving activities in learning to program. Journal of Educational Psychology, 81,584-593.

Rothkopf, E. Z . (1 973). Course content and supportive environments for learning. Educational Psychologist, 10, 123-128.

Sanford, J. P. (1987). Management of science tasks and effects on students’ learning opportunities. Journal of Research in Science Teaching, 24,249-265.

Schoenfeld, A. H. (1983). Beyond the purely cognitive: belief systems, social cognitions, and metacognitions as driving forces in intellectual performance. Cognitive Science, 7,329-363.

Strage, A., Tyler, A. B., Rohwer, W. D., Jr. and Thomas, J. W. (1987). An analytic framework for assessing distinctive course features within and across grade levels. Contemporary Edu- cational Psychology, 12,280-302.

Thomas, J. W. and Rohwer, W. D., Jr. (1987). Grade-level and course-specific differences in academic studying: summary. Contemporary Educational Psychology, 12,38 1-385.

Thomas, J. W. and Rohwer, W. D. Jr. (1993). Proficient autonomous learning: problems and prospects. In M. Rabinowitz (ed.), Cognitive science foundations of instruction (pp. 1-32). Hillsdale, NJ: Earlbaum.

Thomas, J . W., Bol, L. and Warkentin, R . W. (1991). Antecedents of college students study deficiencies: the relationship between course features and students’ study activities. Higher Education, 22,275-296.

ment. Psychological Reports, 54,923-929.

46,115-127.

16,72-103.

532 J. W. Thomaset a].

Thomas, J . W., Iventosch, L. and Rohwer, W. D. Jr. (1987). Relationships among student characteristics, study activities, and achievement as a function of course characteristics. Contemporary Educational Psychology, 12, 344-364.

Thomas, J. W., Strage, A., Bol, L. and Warkentin, R. W. (1990). Features of high school science courses expected to prompt students' study activities. Paper presented at the Annual Meeting of the American Educational Research Association, Boston, April.

Thomas, P. R. and Bain, J . D. (1984). Contextual dependence of human learning approaches: the effects of assessments. Human Learning, 3,227-240.

Warkentin, R. W. (1991). Evaluating the character and effectiveness of students' study actieities and study performances using hierarchical models of studying. Doctoral dissertation. Univer- sity of California, Berkeley. Graduate of School of Education.

Weinstein, C . E., Zimmerman, S. A. and Palmer, D. R. (1988). Assessing learning strategies: the design and development of the LASSI. In C. E. Weinstein, E. T. Goetz, and P. A. Alexander (eds), Learning and study strategies: Issues in assessment. instruction, and evaluation (pp. 2540). New York: Academic Press.

Wilhite, S. C. (1990). Self-efficacy, locus of control, self-assessment of memory ability, and study activities as predictors of college course achievement. Journal of Educational Psy- chology, 82, 696-700.

Zimmerman, B. J. ( I 990). Student differences in self-regulated learning: reiating grade, sex, and giftedness to self-efficacy and strategy use. Journal of Educational Psychology, 82,s 1-59,

interrelationships among students' study activities, self-concept of academic ability, and...

Documents