Motivation, Equity, and Measurement in EducationKAMDEN K. STRUNK, PH.D.

Motivation ResearchEXPECTANCY-VALUE, ACHIEVEMENT GOALS, AND TIME -RELATED ACADEMIC BEHAVIOR

Motivation ResearchExpectancy-Value and Achievement Goals◦ Strunk, K. K. (2014). A factor analytic examination of the achievement goal questionnaire,

revised: A three-factor model. Psychological Reports, 115(2), 400-414.

◦ Gravina, V., *Besick, C., & Strunk, K. (2013). Teaching strategies and value for STEM courses. Academic Exchange Quarterly, 17(1), 139-144.

◦ Strunk, K. K., *Beswick, C. S., *Fowlkes, C., Cho, Y., & Mwarumba, M. (2014, October). Gendered dynamics in student expectancy-value motivation for gateway STEM courses.Manuscript submitted for publication. [Revise and resubmit to Journal of Research in Science Teaching]

Procrastination & Motivation ResearchGeneralized Procrastination◦ *Bobo, J. L., *Whitaker, K. C., & Strunk, K. K. (2013). Personality and student self-

handicapping: A cross-validated regression approach. Personality and Individual Differences, 55(5), 619-621.

◦ Strunk, K. K., & Spencer, J. M. (2012). A brief intervention for reducing procrastination. Academic Exchange Quarterly, 16(1), 91-96.

◦ Strunk, K. K., & Steele, M. R. (2011). Relative contributions of self-regulation, self-efficacy, and self-handicapping in predicting student procrastination. Psychological Reports, 109(3), 983-989.

2x2 Theory of Time-Related Academic Behavior◦ Strunk, K. K., Cho, Y., Steele, M. R., & Bridges, S. L. (2013). Development and validation of a

2x2 model of time-related academic behavior: Procrastination and timely engagement. Learning and Individual Differences, 25(1), 35-44.

2x2 Model of Time-Related Academic Behavior

The Need for a New ModelWhat is procrastination?◦ Traditionally, it has been defined as something that happens to the individual,

rather than something the individual chooses to perform.

◦ In the traditional model, all people procrastinate to one degree or another.

◦ Procrastination can be understood in this conceptualization as a deficit of the individual, inherent in the person, which becomes explicit in their behavior in the form of procrastination.

The Traditional ModelPersonality:

◦ Neuroticism (Hess, Sherman, & Goodman, 2000; Johnson & Bloom, 1995; van Eerde, 2003); Perfectionism (Flett, Blanksten, Hewitt, & Koledin, 1992; Onwuegbuzie, 2000; Saddler & Buley, 1999)

◦ It is who the person is, not what the person does.

Self-protective mechanism:◦ Self-handicapping (Beck, Koons & Milgram, 2000; van Eerde, 2003); Avoidant coping/cognitive style (Alexander & Onwuegbuzie, 2007;

Burns, Dittmann, Nguyen, & Mitchelson, 2000; Carden, Bryant, & Moss, 2004; Collins, Onwuegbuzie, & Jiao, 2008; Deniz, Tras, & Aydogan, 2009; Fritsche, Young & Hickson, 2003; Owens & Newbegin, 1997)

◦ It is unintentional as a result of a deficient thinking style.

Inability to Self-Regulate

◦ Low self-regulation (Brownlow & Reasinger, 2000; Senecal, Koestner, & Vallerand, 1995); Low self-efficacy for self-regulation (Klassen,

Ang, Chong, Krawchuck, Huan, Wong, & Yeo, 2009; Klassen, Krawchuck, Lynch, & Rajani, 2008; Klassen, Krawchuch, & Rajani, 2008)

◦ The behavior happens to the individual due to a constitutional inability to self-regulate.

The ProblemEach of these approaches treats the individual as self-unaware and deficient in the act of procrastination.

However, there is evidence to suggest that procrastination is a motivated behavior, that individuals engage in it for specific reasons to satisfy their goals.

For example, Schraw, Wadkins, and Olafson (2007) found in qualitative work that students procrastinate to avoid failure at times, and at other times to increase their performance under more time pressure.

The ProblemOthers have suggested procrastination may be motivated.◦ Active versus Passive Procrastination (Choi & Moran, 2009; Chu & Choi, 2005)

◦ Different Goals Relate Differently to Procrastination (Howell & Buro, 2009; Seo, 2009).

Any model of procrastination that does not consider the motivation of the individual is incomplete and theoretically problematic.

Timely EngagementSo too is a model that does not consider timely engagement, as current models do not.

ExtremeProcrastination

Little Procrastination

Started Work On The Way Out of Class Today

Waited until the Day After the Due Date to

Start

Extreme Timely Engagement

Not just What, but WhyIt is necessary to consider not only the behavior, but the motivation.

In order to understand students’ time-related academic behavior, it is necessary to understand the underlying motivation.

Previous FindingsAmong a sample of 1,496 undergraduate students, the relationship of achievement goals to the 2x2 model of time-related academic behavior was examined.

In the same study, the validity and suitability of the 2x2 measure was assessed:◦ The model was a good fit for the observed data (CFI = .92, TLI = .91, RMSEA = .07, SRMR =

.05).

◦ It was superior to alternative models, using Δχ2 and ΔCFI.

◦ It correlated in the expected pattern with an existing measure of generalized procrastination.◦ Both timely engagement-approach (r = −.61) and timely engagement-avoidance (r = −.60) correlated negatively

with the existing scale (Lay, 1986), while both procrastination-approach (r =.46) and procrastination-avoidance (r = .46) correlated positively with the scale

◦ Coefficient alpha ranged from .76 to .87 for the four subscales of the measure.

Discussion of FindingsConsistent with prior research, the present study found that performance-avoidance goals are positively related to procrastination-avoidance.

◦ However, mastery-avoidance goals showed the opposite pattern demonstrating a negative relationship with procrastination-avoidance.

◦ Given that students adopting performance-avoidance goals tend to strive to not perform worse than others is viable that they are likely to use procrastination-avoidance as a strategy to avoid negative experiences such as fear of failure and mask their lack of ability.

◦ Conversely, students with mastery-avoidance goals are not likely to delay starting and completing tasks because those with this achievement goal are not afraid of poor performance, but rather are concerned about not being able to learn and improve as much as they could.

The 2×2 model of procrastination and timely engagement revealed more distinct and differentiated relationships with achievement goals, providing empirical evidence that considering the underlying motivation of procrastination helped illuminated the unique nature and function of different types of procrastination and timely engagement.

Taken together, while previous research found that approach versus avoidance goals play a determining role in predicting procrastination, these findings demonstrate that mastery versus performance goals, as well as approach versus avoidance goals, play an important role in predicting procrastination with distinct motivations.

Ongoing WorkStill in question: Is time-related academic behavior a function of the person only, or are these behaviors, at least in part, contextually driven?

To find out, data were collected from 2,146 participants in face-to-face undergraduate classes in a Fall semester, and then online from those same participants asking them to describe a different class.

◦ In the follow-up survey, 453 participated. Some attrition was due to individuals graduating or leaving the institution, but the rest was due to simply not completing the follow-up.

In these data:◦ Cluster invariance analyses were conducted to determine if individuals change their “type” of time-

related academic behavior over time. (50.6% do, which is statistically significant [χ2 = 16.31, p < .001])

◦ Canonical correlations are used to understand which factors contribute to the changes in time-related academic behavior that are observed.

Future WorkBased on identifying factors that are related to changes in time-related academic behavior, future research will involve:◦ Determining if mastery-supportive contexts are more likely to produce

adaptive time-related behavior.

◦ Determining if small changes in context (i.e., experimental or quasi-experimental work) can result in changes to time-related academic behavior.

◦ Determining if larger contexts (beyond the classroom, such as department, university) also play a role in determining behavior “type”. If so, how can orientation to institution be formed in adaptive ways?

Equity ResearchEQUITY, SOCIAL JUSTICE, AND EDUCATION

Race, Ethnicity, and EducationAmerican Indian STEM Education

◦ Thomas, J., Orona, C.*, Hulings, M.*, & Strunk, K. (2013). What impacts 3rd-5th graders? Presentation at the Oklahoma State Department of Education Vision 2020 Conference, Oklahoma City, OK.

◦ *Gravina, V., *Besick, C., & Strunk, K. (2013). Teaching strategies and value for STEM courses. Academic Exchange Quarterly, 17(1), 139-144.

◦ Thomas, J. T., & Strunk, K. K. (in progress). Parents matter: Problem versus key.

◦ Strunk, K. K., & Thomas, J. T. (in progress). Girls can’t get it: Perceptions of mathematics as a male-dominated field predict lowered achievement.

Race and Ethnicity in Understanding Southern U.S. Education◦ Strunk, K. K., Locke, L. A., & *McGee, M. K. (in press). Neoliberalism and contemporary reform efforts in

Mississippi’s public education system. In M. Abendroth, & B. J. Porfilio (Eds.), School against neoliberal rule: Educational fronts for local and global justice: A reader. Charlotte, NC: Information Age Publishing.

◦ Strunk, K. K., Locke, L. A., & Martin, G. L. (under review). Thank God for Mississippi: Education, equity, and democracy. New York, NY: Palgrave.

LGBTQ Students: Access, Equity, and ClimateQuantitative work on climate for LGBTQ students in education

◦ Strunk, K. K., & Bailey, L. E. (2015, February). The difference one word makes: Imagining sexual orientation in graduate school application essays. Manuscript submitted for publication. [Revise and resubmit to Psychology of Sexual Orientation and Gender Diversity]

◦ Strunk, K. K., & *Suggs, J. R. (2014). Research update on findings from the USM campus climate survey: Results related to LGBTQ students. Research Initiative on Social Justice and Equity Report.

LGBTQ Issues in Southern U.S. Education◦ Strunk, K. K., Bristol, D., & *Takewell, W. C. (in press). Queering South Mississippi: Going too far and the zero sum game. In S. J. Miller & N. M.

Rodriguez. Educators queering academia. New York, NY: Palgrave Macmillan.

◦ Strunk, K. K., Locke, L. A., & Martin, G. L. (under review). Thank God for Mississippi: Education, equity, and democracy. New York, NY: Palgrave.

LGBTQ Students in Conservative Christian Colleges◦ Strunk, K. K., Bailey, L. E., & *Takewell, W. C. (2014). “The enemy in the midst”: Gay identified men in Christian college spaces. In W. M.

Reynolds (Ed.), Critical studies of southern place: A reader (pp. 369-378). New York, NY: Peter Lang.

◦ *Takewell, W. C., & Strunk, K. K. (2014). Gay students at Christian colleges: Implications for student affairs practice. NASPA Gay, Bisexual, Lesbian, and Transgender Knowledge Community Research Summary and Compilation Whitepaper.

◦ Strunk, K. K., & Bailey, L. E. (2015, February). “A question everybody danced around”: Negotiating sexual identity in Christian colleges. Manuscript submitted for publication [Submitted to The Educational Forum].

Measurement ResearchCLASSICAL TEST THEORY AND META -ANALYTIC CONFIRMATORY FACTOR ANALYSIS

Measurement ResearchTraditional Measurement Work

◦ Strunk, K. K. (2014). A factor analytic examination of the achievement goal questionnaire, revised: A three-factor model. Psychological Reports, 115(2), 400-414.

◦ Strunk, K. K., Cho, Y., Steele, M. R., & Bridges, S. L. (2013). Development and validation of a 2x2 model of time-related academic behavior: Procrastination and timely engagement. Learning and Individual Differences, 25(1), 35-44.

◦ Strunk, K. K., Sutton, G. W., & Skadeland, D. R. (2010). The repeatable battery for the assessment of neuropsychological status may be valid in men ages 18 to 20. Psychological Reports, 107(2), 493-499.

◦ Strunk, K. K., Lane, F. C., & Mwavita, M. (in progress). Situated motivational factors predict changes in time-related academic behavior type over time.

Meta-Analytic Confirmatory Factor Analysis◦ Strunk, K. K., & Lane, F. C. (2014, December). The Beck Depression Inventory 2nd Edition (BDI-II): A Cross

Sample Structural Analysis. Manuscript submitted for publication. [Submitted to Measurement and Evaluation in Counseling and Development]

Meta-Analytic Confirmatory Factor AnalysisMETHOD FOR ANALYZING TEST STRUCTURE AT THE POPULATION LEVEL

Need for a New MethodMethods exist for synthesizing matrices in meta-analytic structural equation modeling (MA-SEM) applications.

One such approach when conducting MA-CFA is presented in Beretvas and Furlow (2009). This approach requires separate synthesis estimates for standard deviations and correlations, which are then converted to covariance matrices. This separate synthesis accounts for the fact that all information may not be present in all analyses.

However, in the case of confirmatory factor analysis (CFA), this should not be necessary given all elements should be present in published studies testing the instrument’s factor structure. This may also be unnecessary given that correlations and standard deviations can be converted into covariance matrices, and the covariance estimates synthesized.

Need for a New MethodThe second issue concerning MA-SEM techniques that they rely on generalized least squares estimation and often involve very complex multi-step matrix algebra procedures for estimation, some of which require the download of specialized software (for example, software that creates EQS or LISREL code).

A method that produces exact combined covariance matrices and which is procedurally simpler would benefit the field by making it easier for researchers and practitioners interested in producing an assessment of structure across samples to do so.

Also, by simplifying the analysis, such techniques may better encourage structural analyses and lead to a better understanding of psychological assessment instruments, in general.

This understanding is particularly important in cases where instrument structure has been controversial (e.g., BDI-II), and where the assessment instrument has an impact on quality of care, diagnoses, and treatment.

The MethodFirst, the inter-item correlation matrices are converted to variance (𝑠𝑥

2 = (𝑠𝑥)2)/covariance

(𝑠𝑥𝑦 = 𝑟𝑥𝑦(𝑠𝑥𝑠𝑦)) matrices.

Next, the variance/covariance matrices are converted to SSCP matrices ( (𝑋 − 𝑋)(𝑌 − 𝑌) =𝑠𝑥𝑦(𝑁 − 1))

Then, the SSCP matrices are added together.

Finally, the combined SSCP matrix is divided by the total sample size for all combined samples minus one.

This results in a combined variance/covariance matrix for all of the sampled studies.

This procedure is sufficiently simple to be done by hand, in Excel, or other readily available software.

Data SimulationA publicly available structure (Hoelzle, 2010) was used to generate an empirical population covariance matrix with one million cases.

Population data was specified to be multivariate normal using the MASS package (Genz, Bretz, Miwa, Mi, Leisch, Scheipl, Bornkamp, & Hothorn, 2013) in R version 3.0.2 (R Core Team, 2013).

All conditions were fully crossed and replicated 1,000 times.

Previously published MA-CFA studies were used to guide the identification of conditions likely to influence the replication of population covariance matrices (Beretvas & Furlow, 2006; Cheung & Chan, 2009).

◦ The first variable contained two (2) levels of population variance. One level contained data using the specified variances from the original covariance matrix. The second level was a new population with an identical covariance structure but with item variances twice the original level.

◦ In addition, five (5) levels of single study samples size were considered in the analysis: 50, 100, 200, 500, and 1000. ◦ Lastly, four (4) levels representing the number of studies used in the pooling of covariance matrices were also considered

within each population: 5, 10, 20, and 30 studies. ◦ Cases were randomly drawn with replacement and then used to estimate a covariance matrix among items for each

simulated draw. ◦ In total, forty (40) conditions were simulated within the data.

Simulation ResultsThe mean one-sample covariance test statistic for all simulated data was 15.407 (SD = 5.67) with an associated p value of .480 (SD = .290).

This suggested no significant difference between the pooled estimate of the population matrix and the actual population matrix used in the testing of this approach (Hoelzle, 2010).

Omnibus ANOVA results indicated no statistically significant interaction effects but the main effects for population variance (F1, 39960 = 6.247, p = .012, 𝜂2 = .001), number of pooled covariance matrices (F3, 39960 = 23.476, p < .001, 𝜂2 = .001), and single study sample size (F4, 39960= 40.590, p < .001, 𝜂2 = .004) were all statistically significant.

◦ M was significantly larger when either 15 or 30 matrices were pooled.

◦ Each increasing level of single study sample size also resulted in statistical decreases to M.

◦ However, none of the conditions simulated resulted in a p values less than α = .05 for any of the one-sample tests of covariance matrices.

◦ All conditions sufficiently approximated of the population covariance matrix.

An Illustrative CaseOne example is the Beck Depression Inventory, 2nd edition (BDI-II; Beck, Steer, & Brown, 1996).

This scale is widely used by clinicians in the measurement of depression, thus making it all the more important to understand its psychometric properties.

Among those who have explored the structure of the BDI-II, a number of differing solutions have emerged.

The controversy with the structure of the BDI-II is regarding both the number of factors, as well as their nature.

Data SourcesData were collected from published factor analyses of the BDI-II that included an inter-item correlation matrix.

Additionally, authors of other factor analytic work with the BDI-II were contacted and asked for copies of the inter-item correlation matrix with standard deviations.

In total, 10 studies were included in the final data set.

Although these studies include samples with varied characteristics, they are combined in this case in an attempt to approach the population as a whole, rather than any subset thereof.

As a result, for this study, the “population” is thought of as all individuals who may be assessed with the BDI-II, both depressed and not depressed, of all age groups, and all ethnicities.

ResultsModel χ2 df χ2/df CFI TLI RMSEA SRMR

Tw

o-F

actors

Arnau, et al. 2674.71 169 15.83 .92 .91 .05 .04

Palmer, et al. 3048.51 169 18.04 .91 .90 .06 .04

Patterson, et al. 540.84 34 15.91 .97 .96 .06 .04

Siegert, et al. 2643.12 151 17.50 .92 .91 .06 .04

Storch, et al. 3500.65 188 18.62 .90 .89 .06 .04

Viljoen, et al. 2547.39 151 16.87 .92 .91 .06 .04

Whisman, et al. 3056.69 169 18.09 .91 .90 .06 .04

Wilson VanVoorhis, et al. 3833.61 169 22.68 .88 .87 .07 .05

Vanhuele, et al. 1216.50 103 11.81 .95 .94 .05 .03

Model χ2 df χ2/df CFI TLI RMSEA SRMR

Three-F

actors

Brouwer, et al. 2664.48 186 14.33 .93 .92 .05 .04

Byrne, et al. 2168.77 167 12.99 .94 .93 .05 .04

Johnson, et al. 2475.15 186 13.31 .93 .93 .05 .04

Lindsay, et al. 2238.11 132 16.96 .93 .91 .06 .04

Lopez, et al. 1911.06 116 16.48 .94 .92 .06 .04

Vanhuele, et al. 770.56 87 8.86 .97 .96 .04 .03

H &

G F

actors

Byrne, et al. (H) 2168.77 167 12.99 .94 .93 .05 .04

Chilcot, et al. (G) 2441.18 179 13.64 .93 .92 .05 .04

Grothe, et al. (H) 2859.01 188 15.21 .92 .91 .05 .04

Quilty, et al. (G) 1760.25 176 10.00 .95 .95 .04 .03

Steer, et al. (H) 1855.40 188 15.19 .92 .91 .05 .04

Thombs, et al. (G) 1605.88 173 9.28 .96 .95 .04 .03

DiscussionBDI-II

◦ No one structure emerged as clearly superior to other structures at the population level.

◦ This highlights the need to consider other factors when deciding on a structure, such as diagnostic utility, predictive validity, and other practical factors. That is – factor analysis alone may be unable to resolve such controversies when there are multiple structures that seem to fit the data.

◦ Hierarchical models were not superior to others, as has been claimed in the literature, but general factor models offered some advantage.

DiscussionMethod:

◦ In the simulation study, the method adequately reproduced the population covariance matrix across all conditions, while performing slightly, but significantly, better with larger samples and larger numbers of conditions.

◦ The method was also useful when applied to an illustrative case example, and is simple enough to be used without specialized software or knowledge of syntax, making it more easily applicable for individuals interested in test structure or factor analysis.

Future Directions:◦ Applying the method to motivation measures, such as AGQ-R.

◦ Applying the method to measures used to understand marginalized populations, like the BSRI, which has controversial population structure.

◦ Further work to understand conditions under which the method is most appropriate.

Motivation, Equity, and Measurement ResearchOVERLAPPING AND INTERTWINED AREAS OF RESEARCH