ufdcimages.uflib.ufl.edu · 4 acknowledgments i thank my supervisor and chair dr. albert ritzhaupt...

3D PRINTING INTEGRATION IN K-12 SCIENCE CLASSROOMS: THE RELATIONSHIP WITH STUDENTS’ STEM MOTIVATION, 21ST CENTURY SKILLS,

AND INTEREST IN STEM CAREERS

By

LI CHENG

A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

UNIVERSITY OF FLORIDA

2019

To my loved ones

4

ACKNOWLEDGMENTS

I thank my supervisor and chair Dr. Albert Ritzhaupt for his tremendous support

on my dissertation and my entire Ph.D. journey. Dr. Ritzhaupt not only guided my

academic and professional development but also cared about me as an individual. I

also thank my co-chair Dr. Pavlo Antonenko for giving me the opportunity to conduct my

dissertation research through his funded project and I appreciate Dr. Antonenko’s

guidance and support throughout the dissertation process. I am grateful to my

committee members Dr. Kara Dawson and Dr. David Miller for their invaluable feedback

and suggestions on my dissertation. I also want to take this opportunity to thank Dr.

Carole Beal for all her guidance and support during my study in the program.

I am grateful to my husband, Peng Xu, for his love and care. No matter what I go

through, ups and downs, he is my rock to rely on. I would not have completed this

journey without his love and support. I thank Dr. Susan Herrick for all her prayers and

her mother-like love to me. I am grateful to Dr. Ann Gaudino, who always cares about

me. I would also like to thank all the professors who have taught me, the colleagues

who have worked with me, and my friends who have helped me. I especially thank

Wenru Zhou for all her help with SAS programming and multilevel modeling analysis.

She saved me from so many frustrations during my dissertation data analysis.

Lastly, I would like to thank my parents for their unspeakable love. They went

through numerous hardships to raise me and provide me access to education, and they

engraved perseverance in my heart, so I can grow up from a little girl in a poor rural

village in China to a Ph.D. in a prestigious university in the United States.

5

TABLE OF CONTENTS

page

ACKNOWLEDGMENTS .................................................................................................. 4

LIST OF TABLES ............................................................................................................ 9

LIST OF FIGURES ........................................................................................................ 12

ABSTRACT ................................................................................................................... 13

CHAPTER

1 INTRODUCTION .................................................................................................... 15

Research Context ................................................................................................... 15

Problem Statement ................................................................................................. 17 Research Purpose .................................................................................................. 18 Research Questions and Hypotheses..................................................................... 19

Significance of This Study ....................................................................................... 20 Definition of Key Terms ........................................................................................... 20

Organization of This Dissertation ............................................................................ 22

2 LITERATURE REVIEW .......................................................................................... 23

Technology Integration Background and Definition ................................................ 24 Nature of Technology ....................................................................................... 24

History of Technology in Education .................................................................. 25 Defining Technology Integration ....................................................................... 27

3D Printing Integration in K-12 Education ............................................................... 29

How 3D Printing Technology Has Been Integrated in K-12 Education ............. 30 Teaching students how to use 3D printing technology ............................... 31

Integrating 3D printing technology into disciplines ..................................... 33 The Influence of 3D Printing Integration ........................................................... 39 Benefits and Challenges of 3D Printing Integration .......................................... 40

Integrated STEM Education via 3D Printing Integration .......................................... 48 Models and Frameworks for Analyzing 3D Printing Integration .............................. 50

Technology Integration Matrix (TIM) ................................................................. 51

Technological Pedagogical Content Knowledge (TPACK) ............................... 53

Teacher Beliefs and Technology Integration ........................................................... 57 Pedagogical Beliefs .......................................................................................... 60 Self-Efficacy in Technology Integration ............................................................ 62 Technology Value Beliefs ................................................................................. 63

Teacher Beliefs and Student Learning Outcomes .................................................. 66 Students’ STEM Motivation, Interest in STEM Careers, and 21st Century Skills .... 68

Social Cognitive Career Theory ........................................................................ 69

6

Expectancy-Value Theory of Motivation and STEM ......................................... 71 Self-efficacy and STEM ............................................................................. 72

Value beliefs and STEM ............................................................................ 74 Students’ Interest in STEM Careers ................................................................. 75 21st Century Skills ............................................................................................ 77

Conceptual Framework ........................................................................................... 78

3 METHODOLOGY ................................................................................................... 82

Context and Participants ......................................................................................... 82 Instrumentation ....................................................................................................... 85

S-STEM Survey ................................................................................................ 85 Teacher Beliefs on 3D Printing Integration Survey ........................................... 86

Teacher pedagogical beliefs ...................................................................... 87 Teacher self-efficacy in 3D printing technology integration ........................ 88 Teacher 3D printing value beliefs ............................................................... 89

Lesson Plan Codebook .................................................................................... 90 3D printing integration levels ...................................................................... 90

STEM integration levels ............................................................................. 93 Data Sources and Data Collection .......................................................................... 94 Data Analysis .......................................................................................................... 94

Data Analysis for RQ1 ...................................................................................... 96 Descriptive statistical analysis .................................................................... 96

Lesson plan analysis .................................................................................. 96 Correlational analysis ................................................................................. 97 Thematic analysis ...................................................................................... 97

Data Analysis for RQ2 ...................................................................................... 98

Descriptive statistical analysis .................................................................... 98 Multilevel modeling analysis ....................................................................... 99 Multiple regression analysis ..................................................................... 106

4 RESULTS ............................................................................................................. 107

Descriptive Statistics of Variables ......................................................................... 107 Dependent Variables ...................................................................................... 109

Student-Level Independent Variables ............................................................ 109 Teacher-Level Independent Variables ............................................................ 110

Internal Consistency ............................................................................................. 111 Correlations between Variables ............................................................................ 113

Results for RQ1 .................................................................................................... 117 Correlations between Teacher Beliefs and 3D Printing Integration ................ 117 Open Responses in Teacher Beliefs Survey .................................................. 117

Results for RQ2 .................................................................................................... 121 Missing Data Evaluation ................................................................................. 121 Assumptions Testing ...................................................................................... 122 Results for Science Motivation ....................................................................... 123

Baseline model ........................................................................................ 124

7

Student-level models ............................................................................... 125 Adding teacher-level variables ................................................................. 129

Effect size calculation .............................................................................. 135 Results for Technology/Engineering Motivation ............................................. 136

Baseline model ........................................................................................ 137 Student-level models ............................................................................... 138 Adding teacher-level variables ................................................................. 141

Effect size calculation .............................................................................. 148 Results for Math Motivation ............................................................................ 149

Baseline model ........................................................................................ 149 Student-level models ............................................................................... 151 Adding teacher-level variables ................................................................. 154

Effect size calculation .............................................................................. 161

Results for 21st Century Skills ........................................................................ 162 Baseline model ........................................................................................ 162

Student-level models ............................................................................... 164

Adding teacher-level variables ................................................................. 168 Effect size calculation .............................................................................. 174

Results for Interest in STEM Careers ............................................................. 175

Baseline model ........................................................................................ 175 Student-level models ............................................................................... 176

Multiple regression ................................................................................... 178 Summary of Results ....................................................................................... 182

5 DISCUSSIONS ..................................................................................................... 190

Limitations and Delimitations ................................................................................ 190

Limitations ...................................................................................................... 190 Delimitations ................................................................................................... 192

Relationships between Teacher Beliefs and 3D Printing Integration .................... 193

Relationships between Teacher Variables and Student Outcomes ...................... 194 Relationships with Science Motivation ........................................................... 195 Relationships with Technology/Engineering Motivation .................................. 198

Relationships with Math Motivation ................................................................ 200 Relationships with 21st Century Skills ............................................................ 205 Relationships with Interest in STEM Careers ................................................. 207

Implications ........................................................................................................... 209 Implications for Practice ................................................................................. 210

Implications for Research ............................................................................... 213

Conclusions .......................................................................................................... 216

APPENDIX

A S-STEM SURVEY (UNFRIED ET AL., 2015) ....................................................... 218

B TEACHER BELIEFS ON 3D PRINTING INTEGRATION SURVEY ...................... 224

8

LIST OF REFERENCES ............................................................................................. 230

BIOGRAPHICAL SKETCH .......................................................................................... 249

9

LIST OF TABLES

Table page

2-1 How 3D printing has been integrated in K-12 education ..................................... 36

2-2 3D printing integration in K-12 education and the impacts on student ................ 43

3-1 Student demographics ........................................................................................ 84

3-2 Teacher demographics ....................................................................................... 84

3-3 Codebook for 3D printing integration levels ........................................................ 92

3-4 Codebook STEM integration levels .................................................................... 93

3-5 Overview of data analysis ................................................................................... 95

3-6 Phases of thematic analysis (from Braun & Clarke, 2006) ................................. 97

3-7 The meanings of symbols in equations (3-1), (3-2), and (3-3) .......................... 102

4-1 Variable names and their meanings ................................................................. 108

4-2 Descriptive statistics for dependent variables ................................................... 109

4-3 Descriptive statistics for student-level independent variables ........................... 110

4-4 Descriptive statistics for teacher-level independent variables .......................... 111

4-5 Cronbach’s alpha of rating scales ..................................................................... 112

4-6 Correlations between dependent variables ...................................................... 114

4-7 Correlations between value beliefs subscales .................................................. 114

4-8 Correlations between self-efficacy beliefs subscales ........................................ 115

4-9 Correlations between teacher beliefs ............................................................... 116

4-10 Correlations between Printing_Level, STEM_Level, and teacher beliefs ......... 117

4-11 Thematic analysis results of teachers’ open responses ................................... 120

4-12 Proportion of missing values for student-level variables ................................... 122

4-13 Skewness and kurtosis of dependent variables ................................................ 123

4-14 The meaning of symbols in equations (4-1), (4-2), (4-3) ................................... 125

10

4-15 Baseline model summary ................................................................................. 125

4-16 Random-intercept model summary ................................................................... 127

4-17 Random-slope model summary ........................................................................ 129

4-18 Multicollinearity of variables for science motivation posttest score ................... 130

4-19 Random-intercept model with teacher variables model summary .................... 134

4-20 Statistics for effect size calculation ................................................................... 136

4-21 The meaning of symbols in equations (4-18), (4-19), (4-20) ............................. 137



4-24 Multicollinearity of variables for technology/engineering motivation posttest score ................................................................................................................. 142







4-31 Multicollinearity of variables for math motivation posttest score ....................... 155







4-38 Multicollinearity of variables for 21st century skills posttest score .................... 168


11


4-41 The meanings of symbols in equations (4-68), (4-69), (4-70) ........................... 175


4-43 Multicollinearity of variables for interest in STEM careers posttest score ......... 178

4-44 Multiple regression model summary ................................................................. 181

4-45 Summary of results for student outcomes ........................................................ 186

4-46 Interactions between student pretest scores and teacher variables ................. 188

4-47 Interactions between student gender and teacher variables ............................ 189

12

LIST OF FIGURES

Figure page

2-1 Organization of the literature review ................................................................... 23

2-2 TIM with progression across levels of integration ............................................... 53

2-3 The TPACK framework and its knowledge components.. ................................... 54

2-4 Theoretical framework of relationships between teacher beliefs and technology integration ........................................................................................ 60

2-5 Theoretical framework for STEM motivation, interest in STEM careers, and 21st century skills ............................................................................................... 69

2-6 Conceptual framework of this study .................................................................... 81

13

Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

3D PRINTING INTEGRATION IN K-12 SCIENCE CLASSROOMS: THE RELATIONSHIP WITH STUDENTS’ STEM MOTIVATION, 21ST CENTURY SKILLS,

AND INTEREST IN STEM CAREERS

By

Li Cheng

August 2019

Chair: Albert D. Ritzhaupt

Cochair: Pavlo Antonenko

Major: Curriculum and Instruction

As an emerging technology in K-12 education, 3D printing has gained much

attention from educators and researchers. However, meaningful 3D printing integration

into K-12 STEM curricula is still scarce, and little is known about how teacher beliefs

influence 3D printing integration and how the integration may influence students’

learning outcomes. This study examined the relationship between teachers’ beliefs, 3D

printing integration, and students’ STEM motivation, 21st century skills, and interest in

STEM careers, which are essential for students to participate in STEM disciplines and

future STEM careers.

This study included 26 teachers across 6 states in the U.S. and their 1,501

students, who participated in the iDigFossils project. Teachers’ lesson plans were

analyzed to examine the 3D printing and STEM integration levels. Data on teachers’

beliefs and students’ STEM motivation, 21st century skills, and interest in STEM careers

were collected using scales adapted from previously validated surveys. This study

14

conducted correlational and multilevel modeling analyses to examine the relationships

between these variables.

Results indicated that teacher beliefs and 3D printing integration were generally

not correlated except for a negative relationship between teachers’ self-efficacy in

pedagogical content knowledge and STEM integration level. Teachers perceived 3D

printing integration as beneficial for students, but they encountered a few challenges

including logistic and technical issues, lack of time and resources, insufficient ability to

use 3D printers and connect 3D printing with curricula, and difficulty in teaching

students with individual differences. Furthermore, teachers’ STEM integration levels

were a positive predictor of students’ math motivation. Teachers’ 3D printing integration

levels were not significant for any student outcome variables. Teachers’ value beliefs

including interest in and perceived importance of 3D printing integration were not

significant, however, teachers’ perceived usefulness of 3D printing was a negative

predictor of students’ 21st century skills. Finally, interesting interaction effects were

observed between student variables (student gender and pretest scores) and teacher

variables (teacher beliefs and 3D printing integration). Future research may employ

experimental design to examine the effects of different 3D printing and STEM

integration levels on students’ learning outcomes, and how different levels may

influence students with individual differences.

15

CHAPTER 1 INTRODUCTION

Research Context

Science, technology, engineering, and mathematics (STEM) are critical for a

nation’s economic development and are fundamental aspects of our lives (NRC, 2011).

STEM knowledge and skills are not only important for professionals in STEM areas, but

many other jobs require a certain level of STEM knowledge and skills (NRC, 2011). The

employment projections 2016-26 released by the Bureau of Labor Statistics (BLS)

indicated there will be significant employment increases in STEM-related occupations

(BLS, 2017). There has been increasing need for an adequately prepared STEM

workforce and increasing demand for workers with STEM skills and competencies

(Honey, Pearson, & Schweingruber, 2014). Moreover, the United States found its STEM

workforce lagged behind some Asian countries (Friedman, 2005) and realized the

necessity to invest in STEM education.

Research suggests that students’ STEM motivation is influential to their STEM

learning and future career choices. Specifically, students’ motivation in STEM leads to

continuous engagement in STEM learning (Maltese, Melki, & Wiebke, 2014) and many

studies indicate that students’ interest and motivation in STEM are closely related to

their future career choices (e.g., Christensen & Knezek, 2017; Maltese & Tai, 2011;

Sadler, Sonnert, Hazari, & Tai, 2012; Tai, Liu, Maltese, & Fan, 2006). Furthermore,

students’ interest in STEM careers can potentially increase the possibility of selecting

STEM careers. Nonetheless, students need 21st century skills to succeed in STEM

learning and future careers (Unfried, Faber, Stanhope, & Wiebe, 2015). Therefore,

16

students’ STEM motivation, interest in STEM careers, and 21st century skills are all

essential for students to participate in STEM disciplines and STEM careers.

As one of the subject areas of STEM education, science has been facing the lack

of student motivation and engagement, and it is challenging for teachers to keep

students engaged in science learning (Schmidt, Rosenberg, & Beymer, 2018). Since

science is a core subject of STEM education and the subjects in STEM are

intercorrelated, integrated STEM education in science classrooms has great potential to

enhance students’ STEM motivation and interest in STEM careers, and also to increase

student participation and persistence in STEM learning and strengthen the future STEM

and STEM-related workforce.

With its rapid development and greater accessibility, technology has been

increasingly integrated into teaching and learning, and there have been tremendous

efforts of technology integration to promote STEM learning (Honey et al., 2014; Urban &

Favlo, 2016). As an emerging technology in K-12 education, 3D printing has gained

attention from teachers, administrators, and researchers, and many schools have

invested in 3D printing technologies. The iDigFossils project, namely the National

Science Foundation (NSF) funded project “iDigFossils: Engaging K-12 Students in

Integrated STEM via 3D Digitization, Printing and Exploration of Fossils” (PI: Dr. Pavlo

Antonenko. Award No. 1510410), was an integrated STEM education initiative aimed to

engage students in STEM learning through the integration of 3D printing technology in

K-12 science classrooms within the context of paleontology.

17

Problem Statement

The need for an adequately prepared STEM workforce has been increasing

(Honey et al., 2014). Meaningful integration of science, technology, engineering and

mathematics (STEM) in K-12 education is an important way to address the need, but it

is challenging (NRC, 2014). With the increasing access to 3D printing technology,

educators gradually discovered the power of integrating these technologies into

teaching, and many schools have 3D printers now (D. Thornburg, N. Thornburg, &

Armstrong, 2014). Integrating 3D printing technology into science classrooms is a

promising approach for meaningful STEM integration (Bull, Chiu, Berry, Lipson, & Xie,

2014). However, 3D printing is largely perceived as a recreational tool (Gonzalez &

Bennett, 2016). Furthermore, 3D printing technology has been mostly used in

undergraduate education, but the integration of 3D printing technology in K-12 science

classrooms is scarce. Although many schools have 3D printing technology, there is a

lack of meaningful integration of this technology into K-12 science classrooms.

Many external and internal factors may impact teachers’ intention and the

integration of 3D printing technology in their classrooms. Teachers’ beliefs about

technology integration have been found influential to teachers’ integration of

technologies in the classrooms (Ertmer, 2005; Ertmer et al., 2012). Especially when the

access to technologies, professional development, and technical and instructional

support are provided, teacher beliefs about technology integration become salient

barriers for teachers to meaningfully integrate technologies in their classrooms (Ertmer

& Ottenbreit-Leftwich, 2010). Moreover, relationships between teacher beliefs and

18

students’ cognitive and affective learning outcomes have been identified by previous

research (Zee & Koomen, 2016).

Regarding 3D printing technology, a newly emerged technology in education,

there is little evidence on how teachers’ beliefs about 3D printing technology integration

influence their integration of 3D printing technologies in the classrooms and how

teachers’ beliefs about 3D printing technology integration may potentially influence

students. Moreover, the influence of 3D printing technology integration in science

classrooms on students’ STEM motivation, 21st century skills, and interest in STEM

careers remains understudied.

Research Purpose

Guided by the social cognitive career theory, expectancy-value theory of

motivation, the Technology Integration Matrix (TIM), and the Technology Pedagogical

Content Knowledge (TPACK) framework, this study intended to examine the

relationships between teachers’ beliefs, their 3D printing technology integration in

science classrooms, and students’ STEM motivation, 21st century skills, and interest in

STEM careers within the context of the iDigFossils project. Specifically, this study

investigated: 1) how teachers’ beliefs are related to 3D printing technology integration;

and 2) how teachers’ beliefs and 3D printing technology integration predict students’

STEM motivation, 21st century skills, and interest in STEM careers.

The data sources for this study included: 1) teachers’ lesson plans for analyzing

teachers’ 3D printing integration in their science classrooms, including 3D printing

integration levels and STEM integration levels; 2) survey data on teachers’ beliefs,

including pedagogical beliefs, and value beliefs and self-efficacy beliefs about 3D

19

printing integration; and 3) pre-post survey data on students’ STEM motivation, 21st

century skills, and interest in STEM careers. The survey of teacher beliefs also included

open-ended questions to provide some qualitative data for understanding and

explaining the relationship between teachers’ beliefs and their 3D printing integration.

Correlational analysis was conducted to investigate the relationship between

teachers’ beliefs and their 3D printing integration levels and STEM integration levels.

Multilevel modeling analysis and multiple regression analysis were conducted to

examine how teacher beliefs, 3D printing integration levels, and STEM integration levels

predicted students’ STEM motivation, 21st century skills, and interest in STEM careers.

Research Questions and Hypotheses

This study was guided by the following overarching research question:

What are the relationships between teachers’ beliefs, their 3D printing integration

in science classrooms, and students’ STEM motivation, 21st century skills, and interest

in STEM careers? Specifically,

1. How are teachers’ beliefs correlated with their 3D printing integration in the science classrooms?

2. How do teachers’ beliefs and their 3D printing integration in the science classrooms predict students’ STEM motivation, 21st century skills, and interest in STEM careers?

It was hypothesized that teachers’ beliefs may be significantly and positively

correlated with their 3D printing integration including the 3D printing integration levels

and STEM integration levels. The higher teachers’ beliefs were, the higher the 3D

printing integration levels and STEM integration levels were. It was also hypothesized

that teachers’ beliefs, 3D printing integration levels, and STEM integration levels may

significantly and positively predict students’ STEM motivation, 21st century skills, and

20

interest in STEM careers. The higher teachers’ beliefs, 3D printing integration levels,

and STEM integration levels are, the higher students’ STEM motivation, 21st century

skills, and interest in STEM careers are.

Significance of This Study

The iDigFossils project was the first initiative and endeavor to systematically

integrate 3D printing in K-12 science classrooms across several states in the U.S. on a

large scale. The dissertation aimed to investigate how teachers’ beliefs are related to

their 3D printing integration in their science classrooms, and how teachers’ beliefs and

3D printing integration predict students’ STEM motivation, 21st century skills, and

interest in STEM careers within the context of the iDigFossils project. This dissertation

seeks to make the following contributions to educational practice and research: 1)

Investigate and analyze the practices of different teachers’ integration of 3D printing

technology in their science classrooms and contribute to the body of knowledge on 3D

printing integration in science classrooms; 2) Examine the relationship between

teachers’ beliefs and their 3D printing technology integration in the science classrooms

to inform educational practice on how to assist teachers with integrating 3D printing

technology in their science classrooms; 3) Shed light on the relationship between

teachers’ beliefs, their 3D printing integration, and students’ STEM motivation, 21st

century skills, and interest in STEM career, which may lay a foundation for future

research to examine the influence of teachers’ beliefs and their 3D printing integration

on students’ learning outcomes.

Definition of Key Terms

21st century skills – are important knowledge and skills students need to succeed in academic and career development, including critical thinking,

21

communication, collaboration, creativity, problem solving, and digital literacy (NRC, 2010; P21; PCAST, 2010).

3D printing technology: the “process of building a physical object, layer by layer, from a three-dimensional digital model” (Gonzalez & Bennett, 2016, p. 11).

First-order barriers – are external to teachers and are related to educational resources (including hardware and software), teacher training, and instructional support.

Integrated STEM education – is “an effort to combine some or all of the four disciplines of science, technology, engineering, and mathematics into one class, unit, or lesson that is based on connections between the subjects and real-world problems” (Moore et al., 2014, p. 38).

Pedagogy – is generally perceived as the science of teaching and it is “any conscious activity by one person designed to enhance learning in another” (Mortimore, 1999, p. 3).

Pedagogical beliefs – are commonly classified as beliefs on teacher-centered learning (behaviorist beliefs) and student-centered learning (constructivist beliefs) (Ertmer, Ottenbreit-Leftwich, Sadik, E. Sendurur, & P. Sendurur, 2012; Kim et al., 2013; Park & Ertmer, 2007; Tondeur, van Braak, Ertmer, & Ottenbreit-Leftwich, 2017).

Second-order barriers – are internal to the teacher and rooted in teachers’ underlying beliefs about teaching and learning (Ertmer, 1999), including teachers’ self-efficacy on technology integration, beliefs about how students learn, and perceived value of technology for teaching and learning (Ertmer et al., 2012).

Self-efficacy – refers to individuals’ beliefs in their abilities to complete tasks or achieve goals (Bandura, 1986, 1997).

Self-efficacy in technology integration – is teachers’ beliefs on their competence to integrate technology into teaching in order to facilitate student learning and achieve the teaching goals.

STEM – is primarily defined as composed of science, technology, engineering, and mathematics, but can also include other related areas.

Technology – 1) technology as tangible tools; 2) technology as techniques, strategies, and processes.

Technology integration – “the process of determining which digital tools and which methods for implementing them are the most appropriate responses to given educational needs and problems” (Roblyer & Doering, 2013, p.16).

22

Technology Integration Matrix (TIM) – a framework for effective technology integration with a focus on student-centered learning.

Technology Pedagogical Content Knowledge (TPACK) – a framework on teachers’ knowledge of technology integration, including content knowledge, pedagogy knowledge, and technological knowledge, and also the interactions between and among the three components, which are PCK (pedagogical content knowledge), TCK (technological content knowledge), TPK (technological pedagogical knowledge), and TPACK (Koehler & Mishra, 2009).

Technology value beliefs – are teachers’ beliefs about the value of integrating technology to facilitate teaching and learning and to achieve the instructional goals (Watson, 2006; Ottenbreit-Leftwich, Glazewski, Newby, & Ertmer, 2010).

Value beliefs – consist of intrinsic interest value, attainment value/importance, utility value/usefulness, and cost (Eccles et al., 1983). Intrinsic interest value is the enjoyment an individual gains from doing the task; attainment value is the importance of doing well on the task; utility value is how a task contributes to future plans; and cost refers to what the individual has to give up on other things in order to do the task as well as anticipated efforts the individual has to put into the task (Eccles et al., 1983; Wigfield, 1994).

Organization of This Dissertation

This dissertation consists of five chapters. Chapter 1 introduces the context of

the research, the problem statement, research purpose, research questions and

hypothesis, significance of this study, and definition of key terms that were used in this

dissertation. Chapter 2 provides a literature review on related research, theories, and

the conceptual framework. Chapter 3 expounds the methodology of this study. Chapter

4 presents the research findings. Chapter 5 discusses the research findings, provided

implications for educational practice and research, and generated conclusions of this

study.

23

CHAPTER 2 LITERATURE REVIEW

This literature review is structured based on the research purpose of this study:

how teachers’ beliefs correlated with their 3D printing integration and how teachers’

beliefs and their 3D printing integration predicted students’ STEM motivation, 21st

century skills, and interest in STEM careers in the context of the iDigFossils project. The

organization of this literature review can be viewed in Figure 2-1.

Figure 2-1. Organization of the literature review

This literature review begins with the general technology integration background

and definition to lay a foundation for the whole literature review. Following the

technology integration background and definition, this review analyzes and describes

Technology Integration Background and Definition

3D Printing Integration in K-12 Education

Integrated STEM Education via 3D Printing Integration

Models and Frameworks for Analyzing 3D Printing Integration

Teacher Beliefs and Technology Integration

Teacher Beliefs and Student Learning Outcomes

Students' STEM Motivation, Interest in STEM Careers, and 21st Century Skills

Conceptual Framework of This Study

24

3D printing integration in K-12 education, including how 3D printing technology has

been integrated in K-12 education, the influence of 3D printing technology integration,

and the benefits and challenges of 3D printing technology integration. After reviewing

3D printing integration in K-12 education, this review illustrates the iDigFossils project

which was an integrated STEM education project via 3D printing technology integration

in K-12 science classrooms and the benefits of integrated STEM education.

After illustrating the STEM integration of the iDigFossils project, this review

continues with the models and frameworks that can be used to analyze 3D printing

technology integration since a major component of this study was to analyze how

teachers have integrated 3D printing technology in their science classrooms. The

following two sections focus on teacher beliefs including pedagogical beliefs, self-

efficacy in technology integration, and technology value beliefs, and the relationship

with teachers’ technology integration and students’ learning outcomes. Finally, this

review focuses on students’ STEM motivation, interest in STEM careers, and 21st

century skills. This review concludes with the conceptual framework of the study, which

is informed by the literature and theories reviewed in this chapter.

Technology Integration Background and Definition

Nature of Technology

Before going further on the application of technology in education, it is necessary

to discuss the nature of technology. As we are immersed in all kinds of fancy digital

technologies nowadays, it is very easy to perceive technology as digital tools or

machines. However, technology is not merely a tool or machine that is tangible.

Technology should be seen as “a system of practical knowledge not necessarily

reflected in things or hardware” (Saettler, 1990, p. 3). Specifically, as Gendron (1977)

25

defined, technology is “any systematized practical knowledge, based on

experimentation and/or scientific theory, which enhances the capacity of society to

produce goods and services, and which is embodied in productive skills, organization,

or machinery” (as cited in Saettler, 1990, p.4). Therefore, we can understand

technology as a two-fold system, i.e., 1) technology as tangible tools; 2) technology as

techniques, strategies, and processes. In this vein, educational technologies can be

understood as educational tools, design, or instructional strategies that are used to

facilitate teaching and learning. In the context of 3D printing, the technology could be 3D

printers, 3D modeling software, 3D designing techniques, and also the strategies to

integrate the 3D printers, 3D modeling software, 3D designing techniques into teaching

and learning.

History of Technology in Education

Historically, many kinds of technological tools have been used for education,

starting from the very early murals, to paper, to radios, films, TVs, to computers,

projectors, smartboards, mobile phones, etc. The development of technology in

education gave rise to continuous debate on the influence of technology, specifically

media, on learning. The media-method debate originated from Clark and Kozma who

maintained different stances on the impact of media on learning and refuted each

other’s opinions back and forth in the 1980s and 1990s. Clark (1983) insisted that media

do not influence learning since no media comparison studies found effects of media on

learning. Furthermore, he suggested a moratorium on exploring the relationship

between media and learning and that future research should focus on the instructional

method instead of the media. Kozma (1991) disputed Clark’s separation of media and

method and thought of it as “an unnecessary schism between medium and method”

26

(Kozma, 1991, p. 205). Learning with media is “a complementary process within which

representations are constructed and procedures performed, sometimes by the learner

and sometimes by the medium” (Kozma, 1991, p. 179). Kozma (1991) reviewed

research on learning with a variety of media, including books, television, computers, and

multimedia and suggested that the technologies, symbol systems, and processing

capabilities of media can influence the learning process. He believed that media and

method are integral parts of instructional design. Responding to Kozma’s (1991)

criticism, Clark (1994a) published an article and concluded that media will never

influence learning, after examining media research of the past 70 years which showed

largely negative evidence.

It may be untenable to assert that media will never influence learning. Studies

reviewed by Clark (1983, 1994a) were overwhelmingly focused on the technology and

few authors indicated what the teachers did with the technology for their instruction at

the time when media were relatively new in education. Additionally, many researchers

were interested in studying which medium was more effective, neglecting what

instructional strategies were used to implement the media. Our attention should not be

on the technologies per se but how technologies are used to facilitate teaching and

learning. As Kozma (1991) stated, students will benefit most from the use of media with

certain capabilities if the capabilities are employed by the instructional method. Media

need to correspond to the particular learning situation to be effective for learning

(Kozma, 1991). Appropriate integration of media could enhance learning by taking the

advantages of media (Kozma, 1994). When designing and implementing instruction with

technology, it is important to consider the affordances of technology and under what

27

conditions the affordances of technology can influence learning. Teachers need to

understand the affordability of different technologies and integrate appropriate

technologies for instruction. The selection of media is critical in design because of

“learner preferences, available media, and the available time and funds” (Clark, 1994b,

p. 8). Technologies may have a novelty effect on learning as students are probably

most interested in the technologies when they first begin to use them. Teachers need to

carefully design instructions and use effective instructional strategies. Only by

integrating proper technologies and using effective instructional strategy will students’

learning be improved.

Nowadays, many emerging technologies such as virtual reality and augmented

reality, robotics, and 3D printing technology have gained much attention in education.

Researchers and educators have been studying how these technologies can be

integrated to make a real impact on students. While these technologies, especially 3D

printing in the context of this study, may seem cool and fascinating, we need to be

careful not to fall into the media-method debate of whether 3D printing can make an

impact on students. More importantly, it is how teachers integrate 3D printing into

teaching and learning that may facilitate students’ learning.

Defining Technology Integration

To define technology integration, it is first necessary to be clear about the

broader concept of educational technology. The Definition and Terminology Committee

of Association for Educational Communications and Technology (2018) issued a new

and soon to be published definition of educational technology,

Educational technology is the study and ethical application of theory, research, and best practices to advance knowledge as well as mediate and improve learning and performance through the strategic design,

28

management and implementation of learning and instructional processes and resources.

This omnibus definition suggests that educational technology is not confined to

educational tools or devices but the application of theory, research, and practices to

advance knowledge and improve learning. Therefore, technology integration is not just

about educational tools or devices per se but how they can be applied to theory,

research, and practices.

Researchers define technology integration in many different ways. Some are

very narrow and focus on the tools, but others not only addressed the tools, but also

methods for implementation, and the purpose of integration. According to Roblyer and

Doering (2013), technology integration refers to “the process of determining which

digital tools and which methods for implementing them are the most appropriate

responses to given educational needs and problems” (p.16). This definition indicates

that when integrating technology, teachers need to consider the educational needs and

problems and determine which tools and implementation methods can meet the

educational needs or solve the problems. Technology integration is not simply

incorporating digital technologies into teaching and learning. As Koehler, Mishra, & Cain

(2013) maintain, there are three core components for good integration of technology:

“content, pedagogy, and technology, plus the relationships among and between them”

(p. 14). A broad range of factors should be considered for effective technology

integration in the classrooms, including technology selection, instructional content,

learner characteristics, instructional strategies, and teaching and learning environment.

In order to select appropriate technology, it is important to match technology

affordances to teaching and learning need. Instructional content is critical for technology

29

selection in terms of what content will be taught and what technology has the capability

to meet with the need for delivering specific instructional content. We also consider the

interaction required during instruction for effective technology selection. We also have to

consider learner characteristics, including who the learners are, their age, their attention

span, motivation, and confidence to use technologies. Instructional strategy/pedagogy is

another important factor to be considered. We have to consider whether it is for

individualized instruction or for grouped instruction (Reiser & Gagné, 1983). Specific

learning tasks also impacts what functions of instructional strategy to select:

presentation, practice, or feedback (Richey, Tracey, & Klein, 2011). Gagné’s Nine

Events of Instruction can also guide instructional strategy selection. In addition,

teachers need to consider the learning context such as group size (Leshin, Pollack, &

Reigeluth, 1992), availability of resources (Romiszowski, 1981), and types of interaction

needed (Huddlestone & Pike, 2008).

3D Printing Integration in K-12 Education

Three-dimensional (3D) printing technology is the “process of building a physical

object, layer by layer, from a three-dimensional digital model” (Gonzalez & Bennett,

2016, p. 11). 3D printing technology uses materials such as plastics, ceramic, and

metals. Technically referred to as additive manufacturing or rapid prototyping, 3D

printing technology has revolutionized small-scale fabrication by allowing users to

create specialized objects with reasonable costs (Gonzalez & Bennett, 2016). People

have been excited about the possibilities of 3D printing technologies and regard it as

potentially revolutionary and disruptive. 3D printing is actually not a new technology. It

has been used in industry since the 1980s, but it was not until recently that 3D printing

technology has been adopted in education.

30

A search of literature related to 3D printing technology revealed that the majority

of the articles either focused on the functionality of 3D printing as a high-tech tool (such

as the use of 3D printing technology to design and/or develop tools for industry or

medical field, etc.), or integrating 3D printing technology in STEM classes in higher

education. There were very limited studies on 3D printing technology integration in K-12

education. During the past few years, the development in 3D printing technology and

decreasing cost makes it affordable to be introduced and explored in K-12 education.

Although 3D printing technology was invented about three decades ago, like many new

technologies, the availability of affordable 3D printing technology (for less than $5,000)

“suddenly changed the game from fantasy to reality” (Gonzalez & Bennett, 2016, p. 1)

until recently.

Nowadays, many schools have 3D printers and we can see all kinds of 3D

printed objects. However, many people think 3D printers are only for generating trinkets,

toys, and generally useless objects (Gonzalez & Bennett, 2016). There remains

deficiency in how 3D printing can be integrated for K-12 education, the evidence of

effectiveness of 3D printing integration in K-12 education, and the influence on teaching

and learning. In the following section, a review of the empirical literature related to 3D

printing integration in K-12 education is provided. The review is guided by three

questions: 1) How has 3D printing been integrated in K-12 education? 2) What is the

influence of 3D printing integration in K-12 education? 3) What are the benefits and

challenges of integrating 3D printing in K-12 education?

How 3D Printing Technology Has Been Integrated in K-12 Education

3D printing has been integrated in K-12 education in two major ways: 1) teaching

students about using 3D printing to create models or objects; 2) integrating it into

31

disciplines to teach other content areas. There are also studies that integrated 3D

printing to facilitate the needs of special education students, for example, students with

visual impairments (e.g., Al-Rajhi, Al-Abdulkarim, Al-Khalifa, & Al-Otaibi, 2015; Grice,

Christian, Nota, & Greenfield, 2015; Jo, 2016), and students with various disabilities

(e.g., Mcloughlin et al., 2016). This review focuses exclusively on the two major ways of

3D printing integration in K-12 education. Table 2-1 provides an overview of how 3D

printing technology was integrated in each study.

Teaching students how to use 3D printing technology

In several studies, the focus of 3D printing integration was to teach 3D printing to

students and have students design and create 3D objects (e.g., Bicer et al., 2017;

Chao, Po, Chang, & Yao, 2016; Chen, Zhang, & Zhang, 2014; Kwon, 2017; Nemorin &

Selwyn, 2017).

In Chen et al. (2014), 23 elementary students learned about how to design 3D

models and they constructed with 3D printers. Compared to a traditional teaching group

which did not use 3D printing, Chen et al. (2014) found significant improvement in the

spatial ability of boys. In a study that was conducted with 24 high school students who

used 3D printing software to design and print the models with 3D printers to make

cultural and creative products, Chao et al. (2016) found the students’ creativity was

enhanced after designing and creating the 3D printed products to show culture. In

another study, Bicer et al. (2017) taught 95 high school students in a summer camp how

to design and print 3D models. The students designed and printed 3D models and

explained the mathematics and science underlying the object, the strengths and

weaknesses of their designs, and the educational purpose of their 3D printed objects.

This study found that the students had significant and positive changes in their

32

perceptions about the need and importance for creativity and problem-solving skills in

STEM fields. Similarly, Kwon (2017) had 47 middle and high school students in a

summer camp use Google SketchUp and XYZware software to design and print their

own unique 3D objects. The students demonstrated a significant increase in

mathematical skills, learning motivation, and technical skills using the software.

Most of these reviewed studies that taught students how to use 3D printing and

had students design and print 3D objects found positive results either in students’

learning performance, learning motivation, spatial ability, technical skills, creativity, or

perceptions on the necessity of creativity and problem-solving skills. However, an

ethnographic study conducted by Nemorin and Selwyn (2017) with a high school

teacher and his students found negative results. This ethnographic study focused on the

experiences and perceptions of the teacher and individual students and provided in-

depth details. In this study, the teacher and students had an eight-week 3D printing

course in which students learned and designed race cars with a 3D modeling software

SketchUp and printed out the cars. However, the teacher and students were

disappointed that many cars looked great but were not working due to many reasons. A

student perceived SketchUp as boring because the student spent too much time on it

and was frustrated with the problems he had when designing the model. Another

student preferred making objects with hands instead of using computers and would not

do a 3D printing project again because the student felt 3D printing was nothing special.

Although teaching 3D printing to students and having students design and create

3D objects had positive influence on students in most of these reviewed studies, one

study that was reviewed found negative influence on students. Engaging students

33

deeply in the designing and printing process might be beneficial for some students in

some situations, but we cannot generalize that it would always be beneficial to all

students. All these reviewed studies focused on teaching students designing and

creating 3D objects regardless of disciplines. There were also studies that focused on

the integration of 3D printing technology to facilitate students’ learning in different

disciplines.

Integrating 3D printing technology into disciplines

Most of the studies integrated 3D printing into disciplines such as history/social

studies (Maloy, Trust, Kommers, Malinowski, & LaRoche, 2017), mathematics (Ng,

2017), science (Koehler, 2017), engineering-related disciplines (Chien, 2017; Chien &

Chu, 2018; Hsiao, Chen, Lin, Zhuo, & Lin, 2019; Weber, Kotsopoulos, & Senger, 2017),

and integrated STEM in the context of Paleontology (Grant, MacFadden, Antonenko, &

Perez, 2017).

In Maloy et al.’s (2017) yearlong study with 13 middle school in-service students,

10 pre-service teachers, and 4 classes of students, 3D printing was integrated into

history/social studies classes. The students participated in carefully designed 3D

printing projects which were connected to the curriculum standards. The students

designed 3D printing models and created objects for different history/social studies

topics. Maloy et al. (2017) found 3D printing allowed students to express their thinking

and ideas in a new way, helped students remember information, visualize causes and

consequences of events, and be creative in school projects. Integrating 3D printing into

middle school mathematics classes, Ng (2017) had students learned how to use

Tinkercad and design 3D models of keychains to facilitate their learning about the

volume of solids, a math topic. The students’ math learning was enhanced, and

34

students had positive attitudes about their experience that they could design, create,

and make personalized keychains for themselves. In another study which integrated 3D

printed objects in science classes with middle school students who had visual

impairments, Koehler (2017) compared the use of 3D printed objects with using

traditional tactile graphics for learning a science topic of plate tectonics. This study

found that although students in each group had increased conceptual understanding

and decreased misconceptions, there were no significant differences between the two

groups, indicating no absolute benefit of using 3D printed objects.

In the previously reviewed studies, 3D printing was integrated into

historical/social studies, mathematics, and science disciplines. However, 3D printing

was mostly integrated into engineering-related disciplines. In Chien’s (2017) study, 3D

printing was integrated into pre-engineering classes with 108 high school students.

Chien (2017) developed an eight-week course to teach the design of CO2 dragsters

using 3D printers and free 3D digital modeling software. The students learned 3D

printing and used 3D digital modeling software to design and create CO2 dragsters.

Chien (2017) compared the 3D printing group (n = 108) to a traditional handmade group

(n = 74) and found that the 3D printing group had significantly better engineering

concepts learning and creativity in terms of novelty and sophistication of their design

than the handmade group, but there was no significant difference in the overall learning

performance. Chien and Chu (2018) integrated 3D printing into a high school STEAM

engineering design curriculum. The students designed and created CO2 racing cars with

3D printing and participated in racing activities. Chien and Chu (2018) compared the 3D

printing group (n = 108) to a handmade group (n = 36) and found the 3D printing group

35

had significantly better creativity in terms of sophistication and appearance of the design

and accuracy of predictions regarding which car was going to win the race, but there

were no significant differences in overall learning performance and creativity in terms of

novelty.

Hsiao et al. (2019) integrated 3D printing into an 11-week pre-engineering course

with high school students. The students learned how to use 3D printing technology and

3D modeling software, and they used 3D printing technology to print out their 3D

designs of windmills. Hsiao et al. (2019) found that students who participated in the 3D

printing project outperformed the control group that used traditional hands-on tools with

lectures in abstract scientific concepts and hands-on ability. Weber et al. (2017)

integrated 3D printing into a 6-week design engineering course with middle school

students. The students designed and constructed cube puzzles and then printed the

models using a 3D printer. From the teacher and students’ reflection, Weber et al.

(2017) reported that the task provided opportunities for students to develop visual-

spatial reasoning and it inspired significant student engagement and further learning.

In the study of Grant et al. (2017), 3D scanning, printing, and analysis of teeth of

the Neogene shark Carcharocles megalodon were integrated into two middle and two

high school science classes in the context of paleontology. As an endeavor to integrate

all four STEM disciplines through 3D printing integration in science classes, Grant et al.

(2017) found increased students’ engagement and enthusiasm in the Megalodon topic.

36

Table 2-1. How 3D printing has been integrated in K-12 education

Study Subject/Content Area

how 3D printing was integrated Integration Category Student interaction with 3D printing

Chen et al. (2014)

NA Students learned about how to design 3D models and operated on 3D printers.

3D printing as the learning content (Students learn 3D printing to improve spatial ability)

design and print

Maloy et al. (2017)

History/Social studies

Teachers and preservice teachers partnered and participated in a 3D printing workshop and developed curriculum projects to connect 3D printing to their school curriculum with curriculum standards.

Teachers implemented 3D printing lessons.

Students designed 3D models and created 3D objects.

To integrate into the curriculum to teach other content areas

design, print, and use for learning

Koehler (2017)

Science Students used 3D printed objects to study plate tectonics.


use printed objects

Hsiao et al. (2019)

pre-engineering Students learned how to use 3D printing technology, 3D modeling software, and used 3D printing technology to print out their 3D designs of windmills.


design and print

Weber et al. (2017)

design engineering

Students designed and constructed cube puzzles and then printed these models using a 3D printer.


design and print

37

Table 2-1. Continued



Chao et al. (2016)

NA Students used 3D software to design models and then printed the models with the 3D printer to make cultural and creative products.

3D printing as the learning content (To create 3D products to show culture and creativity)

design and print

Ng (2017)

Mathematics Students learned how to use Tinkercad and designed 3D models of keychains to facilitate learning volumes of solids.


design

Bicer et al. (2017)

Informal STEM learning program (summer camp)

Students designed 3D models with CAD software and printed out the 3D models with 3D printers. Students explained the mathematics and science underlying the object, strengths and weakness of their designs, and explained the educational purpose for the object.

3D printing as the learning content (To improve creativity and problem solving skills)

design, print, and use

Grant et al. (2017)

Paleontology The activity used 3-D scanning, printing, and analysis of teeth of the Neogene shark Carcharocles megalodon as an example of how paleontology has the potential to integrate all four STEM disciplines and to help develop student interest and motivation in STEM.


print and use

Chien (2017)

Pre-engineering The researcher developed a course to teach the design of CO2 dragsters using 3D printers and free 3D digital modeling software. Students learned 3D printing and used 3D digital modeling software to design and create a CO2 dragster.



38




Kwon (2017)

summer camp Students used Google SketchUp and XYZware software to design and print 3D objects. The learning objective and outcome were to print students’ unique 3D object.

3D printing as the learning content

design and print

Nemorin and Selwyn (2017)

3D printing course

Students designed race cars with SketchUp, a 3D modeling software, and printed the cars.

3D printing as the learning content

design and print

Chien and Chu (2018)

STEAM engineering design curriculum

Students designed and created CO2 racing cars with 3D printing and participated in racing activities.



39

The Influence of 3D Printing Integration

The 3D printing technology has been integrated in different grade levels with

varied implementation durations and has had diverse impacts on students (view Table

2-2 for details). From the studies that have been reviewed on how 3D printing has been

integrated in K-12 education, it was found that the 3D printing has influence in three

main aspects: 1) cognitive learning performance outcomes such as conceptual

understanding and overall learning performance; 2) abilities and skills such as spatial

ability, creativity, hands-on ability, technical skills; and 3) affect learning outcomes such

as perceptions, attitude, engagement, and motivation.

Especially, in the study of Maloy et al. (2017) which integrated 3D printing to

middle school History/Social Studies classrooms, the teachers reported that many of the

students expressed interest in future career paths in STEM fields, in part because the

students felt comfortable with the 3D modeling software. In addition to express

themselves using spoken and written words, 3D printing provided the students an

opportunity to generate a tangible object to communicate their thinking and ideas

(Maloy et al, 2017). As 3D printing was a new technology not just for students but also

for teachers, the teachers and students were working together as partners instead of

adult experts and novice learners, which altered the teacher-as-expert/student-as-

novice relationship (Maloy et al., 2017). Maloy et al. (2017) also indicated that 3D

printing design activities changed the traditional teacher-centered presentations in

which students passively receive and remember information to student-centered active

learning, problem solving, design-based thinking, and collaborative group work.

Students noted that creating designs and objects helped them remember the historical

facts and events and printing objects allowed them to visualize the causes and

40

consequences of historical events. 3D printing not only helps students remember

specific information but also provides students the opportunities to be creative in their

school projects and many of the students took initiatives to use 3D printing in their other

school projects (Maloy et al., 2017).

Trust and Maloy (2017) surveyed 51 teachers about the impact of their 3D

projects on student learning and the teachers reported that their students developed a

number of skills, including 3D modeling, creativity, technology literacy, problem-solving,

self-directed learning, critical thinking, and perseverance, which are the essential 21st

century skills as the authors summarized.

Benefits and Challenges of 3D Printing Integration

3D printing has great potential to engage students in hands-on learning and

experiential learning which brings something impossible or unreachable in real life to

students and allows students to hold and touch 3D objects. 3D printing can make

abstract knowledge in visualizable ways. Holding objects by hand enables students to

learn by doing and can potentially motivate students to learn. 3D printing can be a great

teaching resource to enhance students’ learning performance, abilities of problem-

solving, critical thinking, communication and collaboration, and affective learning

outcomes such as positive attitudes, engagement, and motivation. In addition, 3D

printing can be a powerful resource for integrated STEM education, and it can not only

serve as a tool to create models for students but also a tool for students to design and

create their own models.

Although 3D printing integration has many potential benefits, it has some

challenges that have to be considered and addressed when integrating it into teaching

and learning. 3D printing integration requires a large amount of time and preparation

41

because it takes many hours to print a relatively small object, which may make it not

feasible to have students print out 3D objects during regular class time. 3D printing

integration has high requirements on the temperature for the 3D model layers to build

layer by layer and the model may not be successfully printed out as designed. The 3D

modeling software can also be difficult for teachers and students. The technical

difficulties can pose challenges to teachers and students. In their educator workshops

(Maloy et al., 2017), the teachers struggled to use 3D printing, which was an unfamiliar

technology for them, and in class students were openly frustrated and needed

assistance to use the 3D modeling software. 3D printing can also cause “frustration,

physical fatigue, mental exhaustion, tedium and occasional panic” (p. 592) if the activity

is not appropriately integrated. Maloy et al. (2017) reported that teachers in their study

recommended providing students with time to explore Tinkercad (a 3D modeling

software) to get more comfortable with the tool before starting the projects.

In addition to the technical challenges, it can also be difficult for teachers to

integrate 3D printing to connect to mandated course standards. Maloy et al. (2017)

indicated that both teachers and students in the study found the 3D printing idea

intriguing, but they were unsure how to connect 3D printing to the school district-

mandated and standards-driven curriculum topics. In most of the studies, whether and

how the teachers were trained to integrate 3D printing in their classes were not

mentioned or not provided. Very few studies provided professional development for

teachers to learn how to integrate 3D printing. In the study of Maloy et al. (2017),

teachers and preservice teachers partnered and participated in a 3D printing workshop

42

and developed curriculum projects to connect 3D printing to their school curriculum with

curriculum standards, which was helpful for them to implement the 3D printing lessons.

Professional development for teachers is essential for teachers to effectively

integrate 3D printing in their classes but it does not guarantee positive impacts on

students. As Schelly, Anzalone, Wijnen, and Pearce (2015) pointed out:

Simply putting 3-D printers in schools does not automatically provide a better learning environment or make students into maker and creator. Providing teachers with the training necessary to transform them into makers may make this more likely, but again does not guarantee that students will be transformed into creators rather than passive consumers of knowledge. (p. 235)

Effective 3D printing integration can also be influenced by teachers’ knowledge and

beliefs on technology integration. It is a challenging but important issue of how 3D

printing can be effectively integrated and make positive impacts on students.

43

Table 2-2. 3D printing integration in K-12 education and the impacts on student

Study Participants Duration Research Design Outcome Variables

Results

Chen et al. (2014)

46 primary students (mean age = 10)

7 months

Pre-post experimental design with two groups: 3D printing group (n = 23) vs. traditional teaching group (n = 23)

Spatial ability Significant improvement in spatial ability

Maloy et al. (2017)

13 middle school (eighth grade) in-service teachers and 10 preservice teachers; 4 classes of students

1 year Pre-survey on teachers’ 3D printing knowledge, skills, attitudes, and ideas, and post-survey on how teachers learned to incorporate 3D printing into their lessons; observation of 3D printing lesson implementation; focus groups with students

Teacher and student experiences and perceptions

It was challenging to connect 3D printing to standards-driven curriculum topics.

Teachers and students experienced technical challenges.

3D printing allows students to express thinking and ideas in a new way; helps students remember information; helps students visualize causes and consequences of events; help students be creative in school projects

Koehler (2017)

5 middle school science classroom students with visual impairments

3 weeks Qualitative study with two groups: 3D printed objects group (n = 3 students) vs. traditional tactile graphics group (n = 2).

Pre-post student interviews, student journals, audiotaped instructional sessions.

Conceptual understanding of plate tectonics; Misconceptions related to plate tectonics and associated geoscience concepts.

Increased conceptual understanding and decreased misconceptions for each group;

No significant differences in student conceptual understanding between the two groups.

44


Study Participants

Duration

Research Design Outcome Variables

Results

Hsiao et al. (2019)

184 10th-grade students from five classes

11 weeks (for a total duration of 960 min)

Quasi-experimental design: Experimental Group 1 (EG 1: n = 74) which used 3D printing with ELS (experiential learning strategies), Experimental Group 2 (EG2: n = 36) which used 3D printing with lecture, Control Group (n = 74) which used traditional hands-on tools with lecture.

EG1 and EG2 both learned how to use 3D printing technology, 3D modeling software, and used 3D printing technology to do the actual practices.

Comprehension of abstract scientific concepts; Hands-on ability

All three groups improved their comprehension of abstract scientific concepts.

Abstract scientific concepts understanding: EG 1 > CG, EG2 > CG, no significant difference between EG1 and EG2.

Hands-on ability: EG1 > EG2 > CG.

Weber et al. (2017)

one classroom teacher and 22 students in 7th grade

6 weeks

Teacher and student reflections Visual-spatial reasoning; student engagement

The authors reported the task provided opportunities for students to develop visual-spatial reasoning, the 3D printer inspired significant student engagement and further learning.

Chao et al. (2016)

24 high school students

18 hours in 9 classes

Assessment on students’ design

Visual design abilities of cultural and creative goods

Enhanced creativity

45



Results

Ng (2017)

Junior secondary students (aged 13 - 15) (sample size not available)

4 days Video recordings of the students’ communications, students’ calculations, and written

reflections

NA Enhanced math learning and positive attitude about their experience that they could “design”, “create”, and “make” personalized keychains for themselves.

Bicer et al. (2017)


2 weeks Pre-post surveys Students’ perceptions about creativity and problem solving skills in STEM disciplines

Students’ perceptions about the need for creativity in STEM fields were statistically significantly changed. Students indicated creativity was important for STEM fields and for engineering in particular and problem solving skills were essential for being successful in a STEM career. Cohen’s d effect sizes for students’ perceptions about creativity and problem solving skills in STEM disciplines at d = 0.61 and d = 0.66

Grant et al. (2017)

2 middle and 2 high schools

NA Case study, survey and observation

Student engagement

Student engagement increased. Increased enthusiasm in the Megalodon topic.

46



Results

Chien (2017)

182 Grade 10 students

8 weeks Quasi-experimental design: a 3D Printing group with 108 students and a Handmade group with 74 students.

Engineering concepts learning; Overall learning performance of STEM knowledge, engineering drawing exercises, etc.; Creativity.

Engineering concepts learning: 3D Printing group > Handmade group

Overall learning performance: No significant difference;

Creativity: 3D Printing group significantly outperformed the Handmade group in terms of novelty and sophistication of their dragsters.

Trust and Maloy (2017)

51 teachers NA Survey of asking teachers the impact of 3D projects on student learning.

The kinds of 3D projects teachers were doing with students;

Skills or knowledge students were developing by participating in the projects

Teachers reported their students developed several skills, including 3D modeling, creativity, technology literacy, problem-solving, self-directed learning, critical thinking, and perseverance.

Kwon (2017)

47 secondary school students

2 weeks Pre-post surveys Mathematical skills, motivation, technical skills

Statistically significant increase in mathematical skills, learning motivation, and technical skills.

47


Study Participants Duration Research Design

Outcome

Variables

Results

Nemorin and Selwyn (2017)

High school teacher and students (sample size not available)

Eight weeks

Ethnographic research

NA Only six cars were functioning. Many cars looked great but were not working due to many reasons. Teachers and students were disappointed. Students were excited and having fun at first and then felt bored. A student said SketchUp was boring because he spent too much time on it and he felt like giving up sometimes because he kept having to go back and change things because the models he designed were the wrong size or they didn’t fit together, which made him frustrated a lot. A student preferred making things with hands instead of using computers and would not do a 3D printing project again because he felt 3D printing was nothing special.

Chien and Chu (2018)


Eight weeks (two 50-min classes per week for 800 min)

Experimental design: 3D Printing group (n = 108) and handmade group (n = 36)

Novelty and sophistication of design;

Consistency of design (whether the appearance of the car was consistent);

Accuracy of predictions of the winning car;

Overall learning outcomes.

Novelty: no significant difference;

Sophistication: 3D printing group > handmade group;

Consistency: 3D printing group > handmade group;

Accuracy of predictions of the winning car: 3D printing group > handmade group;

Overall learning outcomes: no significant difference

48

Integrated STEM Education via 3D Printing Integration

Sanders (2009) defined integrated STEM education as “approaches that explore

teaching and learning between/among any two or more of the STEM subject areas,

and/or between a STEM subject and one or more other school subjects” (p. 21) More

specifically, Moore et al. (2014) defined integrated STEM education as “an effort to

combine some or all of the four disciplines of science, technology, engineering, and

mathematics into one class, unit, or lesson that is based on connections between the

subjects and real-world problems” (p. 38) and indicated that the STEM learning content

objectives primarily focused on one subject and learning contexts can be from other

STEM subjects. Kelley and Knowles (2016) also emphasized the importance of focusing

on real-world problems by stating that “an integrated approach seeks to locate

connections between STEM subjects and provide a relevant context for learning the

content. Educators should remain true to the nature in which science, technology,

engineering, and mathematics are applied to real-world situations” (p.3).

Integrated STEM learning experiences can enhance students’ interest and

motivation in STEM which leads to continuous engagement in STEM learning (Maltese

et al., 2014). Tai et al. (2006) surveyed 12,000 middle school students and found

students’ interest in STEM was a significant predictor of their career choices. Sadler et

al. (2012) found that students’ interest in STEM at the start of high school was a key

predictor of STEM career interest when they graduated. Maltese and Tai (2011) also

found that students who had interests in STEM were more likely to select and complete

a bachelor’s degree in STEM disciplines.

The National Science Foundation (NSF) funded project “iDigFossils: Engaging K-

12 Students in Integrated STEM via 3D Digitization, Printing and Exploration of Fossils”

49

(PI: Dr. Pavlo Antonenko. Award No. 1510410) was established with the spirit of STEM

integration by integrating 3D printing technology in K-12 science classrooms in the

context of paleontology, a rich and interdisciplinary science that examines life in deep

time. Paleontology integrates a variety of disciplines such as geology, biology,

anthropology, environmental science, and oceanography, providing great opportunities

for integrated STEM learning (Maltese et al., 2014) and to enhance students’ interest in

STEM. According to Grant et al. (2016), learning activities that integrated 3D scanning

and printing technologies with paleontology are a pathway for students to not only

enhance STEM learning through gathering data by analyzing 3-D printed fossils,

developing 3D modeling knowledge by using software, using mathematical estimations,

and building connections among STEM disciplines, but also to develop 21st Century

skills, such as collaboration, communication, critical thinking, problem solving, and

creativity (Partnership for 21st Century Skills, 2011).

In the following, an example iDigFossils lesson plan was illustrated in terms of its

integrated STEM approach. In the lesson plan, the iDigFossils activities were designed

with the instructional approach of integrated STEM by integrating science

(paleontology), technology, and math. Students built math skills by measuring and

calculating the 3D printed Hominins teeth. By comparing and contrasting the tooth sizes

students learned about the hominins’ diet. In the inquiry-based learning activities,

students played the role of paleoanthropologists and performed hands-on activities to

investigate a real-world problem of how hominins adapt to a variety of environments.

Students also worked collaboratively in teams to measure and estimate the teeth size

and discuss the similarities and differences of the teeth and how the teeth would

50

indicate different diet of the hominins. The iDigFossils activities were consistent with

how the authors defined integrated STEM education and specifically what Johnson

(2013) stated about integrated STEM, “integrated STEM education is an instructional

approach, which integrates the teaching of science and mathematics disciplines through

the infusion of the practices of scientific inquiry, technological and engineering design,

mathematical analysis, and 21st century interdisciplinary themes and skills” (p. 367).

Integrated STEM activities make learning more connected and relevant for

students (Stohlmann, Moore, & Roehrig, 2012), encourage students’ imagination and

curiosity and increase their motivation to learn (Laboy-Rush, 2011), and support

students’ interest development (Honey et al., 2014). The iDigFossils activities have

great potential to promote students’ motivation and career interest in STEM through

integrated STEM activities. These activities were also very likely to enhance students’

21st century skills. For instance, students develop critical thinking when they compare

and contrast the teeth size and analyze the diet of different hominins; problem solving

skills when they work on the problem of how different hominins adapt to a variety of

environments; communication and collaboration skills when they work in teams to

measure and calculate the teeth, and analyze the hominins’ teeth size and their diet;

and increase digital literacy by using 3D printers to print out fossil models.

Models and Frameworks for Analyzing 3D Printing Integration

An important part of this study was to analyze teachers’ 3D printing integration in

the classrooms, i.e., in what ways the teachers integrated 3D printing technology and

the levels of integration. This study used the Technology Integration Matrix (TIM) and

Technological Pedagogical Content Knowledge (TPACK) as the frameworks for 3D

printing integration analysis.

51

Technology Integration Matrix (TIM)

There are many technology integration frameworks. One of the most influential

frameworks is the Technology Integration Matrix (TIM). Technology Integration Matrix

(TIM) was created by the Florida Center for Instructional Technology (FCIT) from 2003

to 2005 and updated in 2011 (Welsh, Harmes, & Winkelman, 2011) to support students

in learning skills that are necessary for success in the 21st century (Harmes, Welsh, &

Winkelman, 2016). TIM provides a framework for effective technology integration with a

focus on student-centered learning. TIM is a two-dimensional matrix with five

characteristics of meaningful learning environments as the row and five levels of

technology integration as the column, resulting in 25 cells with each cell representing a

level of technology integration for each of the characteristics of the learning

environment. The 25-cell matrix is available at the FCIT website

https://fcit.usf.edu/matrix/matrix/.

The two dimensions of TIM are pedagogy and technology, with the unit of focus

being a lesson (Harmes et al., 2016). The technology aspect is composed of five levels

of technology integration: Entry, Adoption, Adaptation, Infusion, and Transformation, a

continuum from teacher-centered passive instruction to student innovative use of

technology in higher-order learning activities (Harmes et al., 2016). At the entry level,

teachers use technologies to deliver instruction and students do not have direct access

to the technologies. Even if students have some access to the technologies, the

purpose is to learn facts or basic skills through rote practice. At the adoption level,

teachers still dominate the use of the technologies and students have some but very

limited access to the technologies and use them for discrete tasks, which only require

procedural understanding. At the adaptation level, technologies are more integrated in

52

the lesson and students have more access to the technologies. Teachers still determine

when to use the technologies and students may begin exploring how to best use the

technologies. At the infusion level, students have full access to the technologies through

their study and the technologies are for learning rather than the technology tools per se.

Teachers guide students’ decision-making when using the technologies for learning.

Transformation is the highest level of technology integration. Teachers guide students’

use of the technologies and students can self-direct their use of the technologies.

Different from infusion level, transformation level of technology integration facilitates

higher-order learning activities that would otherwise be impossible or difficult without

using the technologies.

The pedagogical aspect focuses on five characteristics of meaningful learning

environments: Active, Collaborative, Constructive, Authentic, and Goal-Directed, which

were based on the work by Howland, Jonassen, and Marra (2012). The five

characteristics of meaningful learning enable students to “engage in higher-order

thinking and focus on real-world skills” (Harmes et al., 2016, p. 144). The TIM levels

describe a continuum of pedagogical approaches along with the levels of technology

integration from Entry to Transformation, reflecting four underlying differences:

ownership of learning, characterization of knowledge, use of technology tools, and

instructional focus (Harmes et al., 2016). The progression across levels of technology

integration of TIM can be viewed in Figure 2-2.

53

Figure 2-2. TIM with progression across levels of integration

Technological Pedagogical Content Knowledge (TPACK)

A comprehensive and prevalent technology integration framework on teachers’

knowledge of technology integration is the Technological Pedagogical Content

Knowledge (TPACK) framework, which has been adopted nationally and even

internationally. Teachers’ knowledge of technology integration is essential for effectively

integrating technologies into their teaching. Technological Pedagogical Content

Knowledge (TPACK) framework (previously named as TPCK), was conceptualized by

Mishra and Koehler (2006) based on Shulman’s (1986, 1987) description of

pedagogical content knowledge (PCK). Mishra and Koehler (2006) extended the PCK

framework by adding the component of technological knowledge and the interaction of

technological knowledge with other domains of knowledge in PCK. Different from

Technology Integration Matrix (TIM) which only has two dimensions: pedagogy and

technology, the TPACK framework incorporates the dimension of content. In TPACK

framework, there are three basic components, namely content knowledge, pedagogy

54

knowledge, and technological knowledge, and also the interactions between and among

the three components, which are PCK, TCK (technological content knowledge), TPK

(technological pedagogical knowledge), and TPACK (Koehler & Mishra, 2009). In a

nutshell, TPACK framework addresses the integration of technology, pedagogy, and

content knowledge in teaching and the intersections between these three domains of

knowledge. Figure 2-3 is the TPACK framework developed by Mishra and Koehler

(2009).

Figure 2-3. The TPACK framework and its knowledge components. Reprinted from http://tpack.org.

Content knowledge is teachers’ knowledge about the content in the subject

matter that will be learned or taught, including “knowledge of concepts, theories, ideas,

organizational frameworks, knowledge of evidence and proof, as well as established

practices and approaches toward developing such knowledge” (Koehler & Mishra, 2009,

p. 64; Shulman, 1986). Accurate, comprehensive, and deep knowledge of the content is

55

a fundamental requirement for teachers to teach students with appropriate and

adequate knowledge.

Pedagogical Knowledge is teachers’ knowledge about the pedagogy, namely,

how to teach, related to the processes and methods of teaching and educational

purposes and values. Pedagogical knowledge includes knowledge about “techniques or

methods used in the classroom; the nature of the target audience; and strategies for

evaluating student understanding” (Koehler & Mishra, 2009, p. 64). Pedagogical

knowledge requires teachers to understand how students construct knowledge and

acquire skills and how they develop positive attitudes toward learning (Koehler &

Mishra, 2009).

Technological Knowledge is teachers’ knowledge about technologies and the

fluency of using information communication technologies (ICT). Teachers need to

understand ICT to apply technologies in their teaching.

Pedagogical Content Knowledge is the knowledge of pedagogy for teaching

specific content, consistent with Shulman’s (1986, 1987) idea of pedagogical knowledge

that is applicable to the teaching of specific content. This knowledge has requirements

on both pedagogical knowledge and content knowledge and also appropriate pedagogy

for the specific content.

Technological Content Knowledge is teachers’ knowledge about what

technologies are appropriate for teaching the specific content. Technologies can afford

or constrain the type of content that can be taught, and the content can limit the types of

technologies that can be used (Koehler & Mishra, 2009). Teachers need to understand

the content and select technologies that can afford to teach the content.

56

Technological Pedagogical Knowledge is teachers’ knowledge about the

functions of specific technologies when using technologies to fulfill their pedagogical

design and instructional strategies. Teachers need to consider the affordances and

constraints of the technologies and how the technologies can facilitate the way they

would like to teach.

Technological Pedagogical Content Knowledge is the highest and most

comprehensive level of knowledge for effective technology integration. It is a

combination of the basic components of content knowledge, pedagogical knowledge,

and technological knowledge, and all the interactions among content, pedagogy, and

technological knowledge. Specifically, as Koehler and Mishra (2009) explained:

TPACK is the basis of effective teaching with technology, requiring an understanding of the representation of concepts using technologies; pedagogical techniques that use technologies in constructive ways to teach content; knowledge of what makes concepts difficult or easy to learn and how technology can help redress some of the problems that students face; knowledge of students’ prior knowledge and theories of epistemology; and knowledge of how technologies can be used to build on existing knowledge to develop new epistemologies or strengthen old ones. (p. 66)

Teachers need to develop not just knowledge in the key domains (Content, Pedagogy,

and Technology) but also knowledge of how these domains interrelate so as to integrate

technology effectively into their teaching.

Teaching with technology is challenging, especially when integrating new

technologies. The TPACK framework provides a comprehensive view of knowledge of

effective technology integration, but successful teaching with technology requires

teachers to continually create, maintain, and re-establish a dynamic equilibrium among

all components of TPACK framework (Koehler & Mishra, 2009). In addition to teachers’

technology integration knowledge, whether teachers can continually create, maintain,

57

and re-establish the equilibrium of all components of TPACK also depends on a number

of factors such as teachers’ beliefs of technology integration.

Teacher Beliefs and Technology Integration

Teachers may have designed and implemented 3D printing integration in a

variety of ways. What factors influence their 3D printing integration? To lay a foundation

for understanding the potential factors that may have influenced teachers’ 3D printing

integration, it is necessary to review the factors that can influence teachers’ technology

integration in the classrooms.

With the rapid development of information and communications technology (ICT),

teachers have been integrating technology into classrooms to facilitate teaching and

learning, and a large body of research has been conducted to examine the

effectiveness of technology integration and what factors or barriers impact the practice

of technology integration in K-12 classrooms. According to Ertmer (1999), there are two

types of barriers that can impact teachers’ integration of technology into classrooms,

external (first-order) barriers and internal (second-order) barriers. First-order barriers

are external to teachers and are related to educational resources (including hardware

and software), teacher training, and instructional support. Second-order barriers are

internal to the teacher and rooted in teachers’ underlying beliefs about teaching and

learning (Ertmer, 1999). Second-order barriers include teachers’ self-efficacy on

technology integration, beliefs about how students learn, and perceived value of

technology for teaching and learning (Ertmer et al., 2012).

Two decades ago, teachers had limited access to technologies, professional

development, and instructional support. First-order barriers were significant obstacles

for technology integration in the classrooms (O’Mahony, 2003; Pelgrum, 2001, as cited

58

in Ertmer et al., 2012), however, these barriers have been reduced with the substantial

funding dedicated to increasing technology access in K-12 education (Culp, Honey, &

Mandinach, 2005). These funding support has ensured teachers’ access to computers

and internet (Gray, Thomas, & Lewis, 2010). Moreover, professional development and

instructional support for teachers have also been improving (Ertmer et al., 2012). In the

current K-12 education context, the second-order barriers have a more remarkable

influence on teachers’ technology integration (Ertmer et al., 2012). Moreover, in the

iDigFossils project, the teachers were provided with 3D printers and related software,

professional development to learn how to use 3D printers and how to integrate 3D

printing into science classes, and technical and instructional support (e.g., Teachers’

lesson plans were reviewed by experts and suggestions were provided to the teachers)

for their 3D printing integration in their science classrooms. The external barriers in the

context of the iDigFossils project are trivial. Therefore, the current study focuses on

teachers’ beliefs which may have influenced their 3D printing technology integration.

Teacher beliefs is an ill-defined concept and can be broadly defined as “teachers’

implicit assumptions about students, learning, classrooms, and the subject matter to be

taught” (Kagan, 1992, p. 66). As individuals’ beliefs strongly influence their behavior

(Pajares, 1992), the influence of teacher beliefs on teaching practice has been widely

studied since the 1990s. There is a strong association between teacher beliefs and their

teaching practice (Ertmer, 2005). As stated by Pajares (1992), there is a "strong

relationship between teachers' educational beliefs and their planning, instructional

decisions, and classroom practices" (p. 326). Researchers perceived teacher beliefs as

the most valuable psychological construct to teacher education (Pintrich, 1990) and

59

teacher beliefs are even more influential on teaching practice than teachers’ knowledge

(Ertmer & Ottenbreit-Leftwich, 2010; Kagan, 1992; Pajares, 1992). Teacher beliefs “tend

to be associated with a congruent style of teaching that is often evident across different

classes and grade levels” (Kagan, 1992, p. 66). Teacher beliefs have been widely

studied in more than fourteen countries around the world such as U.K., Spain, Portugal,

Israel, the Netherlands, Turkey, China, South Korea, Singapore, the U.S., and Canada,

etc., as Kim, Kim, Lee, Spector, and DeMeester (2013) reviewed.

To understand the role of teacher beliefs in teaching with technology,

researchers have focused on a myriad of aspects of teacher beliefs, including teachers’

pedagogical beliefs, self-efficacy in teaching with technology, and technology value

beliefs (e.g., Ertmer & Ottenbreit-Leftwich, 2010; Park & Ertmer, 2007). Teachers’ self-

efficacy and technology value beliefs in teaching with technology are directly associated

with their technology integration in the classrooms and teachers’ pedagogical beliefs are

underlying beliefs that can impact how teachers integrate technology into teaching and

learning. As Kim et al. (2013) stated, pedagogical beliefs are teachers’ fundamental

beliefs that are regardless of the technology involved but can impact how teachers

integrate technology for teaching. In the following, research on teachers’ pedagogical

beliefs, self-efficacy beliefs in technology integration, and technology value beliefs will

be reviewed. A theoretical framework of teacher beliefs on technology integration can

be viewed in Figure 2-4.

60

Figure 2-4. Theoretical framework of relationships between teacher beliefs and technology integration

Pedagogical Beliefs

Pedagogy is generally perceived as the science of teaching and it is “any

conscious activity by one person designed to enhance learning in another” (Mortimore,

1999, p. 3). Therefore, pedagogy can be perceived as an effective instructional strategy

or method to enhance learning. Teachers’ pedagogical beliefs are assumptions about

effective instructional strategies that teachers can use to enhance students’ learning.

Access to technologies does not guarantee teachers’ effective integration of technology

into classrooms. Teachers’ pedagogical beliefs play a key role in how teachers integrate

technology in their classrooms (Deng, Chai, Chin-Chung, & Min-Hsien, 2014; Ertmer et

Teacher Beliefs on Technology Integration

Teacher beliefs related to technology

(Expectancy-Value Theory of Motivation)

Self-efficacy in

technology integration

(competency beliefs)

Technology value

beliefs

Teacher fundamental beliefs

Pedagogical beliefs

Teacher technology

integration in the

classrooms

61

al., 2012, 2015; Hermans, Tondeur, van Braak, & Valcke, 2008; Inan & Lowther, 2010;

Niederhauser & Stoddart, 2001; Tondeur et al., 2017).

Teachers’ pedagogical beliefs are commonly classified as beliefs on teacher-

centered learning (behaviorist beliefs) and student-centered learning (constructivist

beliefs) (Ertmer et al., 2012; Kim et al., 2013; Park & Ertmer, 2007; Tondeur et al.,

2017). Teachers with teacher-centered beliefs tend to act as an authority and expert in

highly structured learning environments and supervise the learning process (Tondeur et

al., 2017). Teacher-centered technology integration often involves low-level technology

uses such as using technology just to deliver information (Harmes et al., 2016).

However, teachers with constructivist beliefs tend to integrate technology in more

meaningful ways to facilitate student active learning and enhance students’ higher-order

thinking and problem-solving skills (Tondeur et al., 2017), such as problem-based

learning activities which involve students in authentic disciplinary problems and

collaborative learning activities (Ertmer & Glazewski, 2015; Park & Ertmer, 2007). A

systematic review on teachers’ pedagogical beliefs and technology use in education

indicated that there is a bi-directional relationship between teachers’ pedagogical beliefs

and their technology use in the classrooms: on one hand, teachers’ technology-rich

learning experiences can potentially change teachers’ beliefs towards more

constructivist beliefs; on the other hand, teachers with constructivist beliefs tend to

integrate technology to facilitate student-centered learning (Tondeur et al., 2017).

However, teachers’ technology integration may not always be consistent with their

pedagogical beliefs especially when external barriers such as lack of resources and

62

time prevent them from integrating technology in the ways they would like to (Ertmer et

al., 2012).

Self-Efficacy in Technology Integration

Teachers’ pedagogical beliefs are essential for teachers to decide how they will

integrate technology in their classrooms. However, teachers may not integrate

technology in the way they would like to if they do not feel they have the competence.

Rooted in Bandura’s (1986, 1997) social cognitive theory, self-efficacy is an individual’s

beliefs in his or her competence to accomplish a task or reach a goal. Teachers’ self-

efficacy in technology integration is teachers’ beliefs on their competence to integrate

technology into teaching in order to facilitate student learning and achieve the teaching

goals. Teachers may have abundant knowledge and skills in technologies but may not

be confident to integrate the technologies for teaching. Research suggests that

teachers’ self-efficacy in technology integration may be more important than their

knowledge and skills in technologies for integrating technologies in the classrooms

(e.g., Bauer & Kenton, 2005; Wozney, Venkatesh, & Abrami, 2006).

Teachers’ self-efficacy in technology integration is a key predictor of their

implementation of technology in the classrooms (Albion, 1999; Ertmer & Ottenbreit-

Leftwich, 2010; Gonzales, 2013; Haight, 2011; Heineman, 2018; Li, Garza, Keicher, &

Popov, 2018; Manglicmot, 2015; Marcinkiewicz, 1994; Tweed, 2013). Increasing

teachers’ self-efficacy in technology integration can encourage teachers to more

effectively integrate technology into their classrooms (Heineman, 2018). However, even

if teachers have strong self-efficacy in technology integration, they may not integrate the

technology in their classrooms if they do not perceive the technology as valuable to

63

achieve the teaching goals. Therefore, it is also important to take teachers’ technology

value beliefs into consideration.

Technology Value Beliefs

According to expectancy-value theory (Wigfield, 1994), self-efficacy in technology

integration, namely teachers’ beliefs in their competence to integrate technology to

achieve instructional goals, is necessary but not sufficient for teachers to implement

technology in teaching; teachers’ beliefs on the value of the technology are critical for

them to take actions to integrate technology into their classrooms. Teachers’ technology

value beliefs are teachers’ beliefs about the value of integrating technology to facilitate

teaching and learning and to achieve the instructional goals (Watson, 2006; Ottenbreit-

Leftwich et al., 2010). Teachers’ technology value beliefs are critical for teachers to

integrate technology in their classrooms (Ertmer et al., 1999; Ertmer & Ottenbreit-

Leftwich, 2010; Ertmer et al., 2012; Mueller, Wood, Willoughby, Ross, & Specht, 2008;

Vongkulluksn, Xie, & Bowman, 2018; Wozney et al., 2006). When teachers have access

to technology, they make judgments on whether the technology can be used to enhance

teaching and learning. Teachers who have more positive beliefs on the affordability of

technology for instruction tend to integrate technology more frequently in their

classrooms (Anderson & Maginger, 2007; Becker, 1999; Ottenbreit-Leftwich et al.,

2010; Park & Ertmer, 2007; Zhao & Frank, 2003). Integrating a new technology into

current teaching practice requires teachers to spend time and effort learning how to use

the technology and how to meaningfully integrate it into the curriculum. If teachers

perceive the technology as not having enough value to address teaching and learning

needs, they are not likely to use it. When teachers believe the values of using

64

technology to achieve instructional goals, they tend to use it even when barriers exist

(Ertmer et al., 2012; Ottenbreit-Leftwich et al., 2010; Snoeyink & Ertmer, 2001).

In light of the subjective task value in the expectancy-value theory of motivation

(Eccles et al., 1983; Eccles & Wigfield, 1995), teachers’ technology value beliefs can be

composed of intrinsic value (interest in integrating the technology), perceived attainment

value (importance of integrating the technology for teaching and learning), and

perceived utility value (usefulness of integrating the technology for teaching and

learning). When teachers believe that technology integration is interesting and important

and useful to enhance teaching and learning, they are more motivated to learn how to

integrate technology in their classrooms (Cheng & Xie, 2018). Studies have focused on

the general teachers’ technology value beliefs and it is unclear which specific value

beliefs contribute to teachers’ technology integration in the classrooms (Ottenbreit-

Leftwich et al., 2010; Smarkola, 2008).

There is limited research on the relationship between teachers’ technology

integration and their specific technology value beliefs such as teachers’ interest in

technology integration and perceived importance and usefulness of integrating

technology for teaching and learning. In the study of Inan and Lowther (2010) with 1382

in-service teachers, it was found that teachers’ beliefs on the importance of technology

for teaching and learning predicted teachers’ technology integration in the classrooms.

Cheng and Xie (2018) assessed teachers’ technology value beliefs in terms of teachers’

perceived interest, importance, and usefulness about learning digital content evaluation,

examined the relationship between teachers’ technology values beliefs and

Technological Pedagogical Content Knowledge (TPACK), and found teachers’

65

technology value beliefs significantly predicted TPACK. However, this study aggregated

the three aspects of technology value beliefs as a whole and did not examine how each

aspect of teachers’ technology value beliefs predicted TPACK. This study did not

examine the relationship between teachers’ technology value beliefs and the actual

technology integration either. Teachers’ interest in technology integration and perceived

importance and usefulness of technology integration may not necessarily positively

relate to each other. For instance, a teacher may perceive technology integration as

interesting but may not believe it is important and useful for teaching and learning, while

another teacher may perceive technology integration as not interesting but believe it is

important and useful for teaching and learning. It would be necessary to investigate how

the specific aspects of teachers’ technology value beliefs are correlated with teachers’

technology integration in the classrooms.

Teachers’ pedagogical beliefs, self-efficacy in technology integration, and

technology value beliefs are all essential for teachers to meaningfully integrate

technology into their classrooms. These teacher beliefs on technology integration may

also interrelate with each other. In a study with 152 teachers in the midwestern United

States, Hsu (2016) found that teachers who held constructivist pedagogical beliefs had

high self-efficacy in using technology for teaching and had positive value beliefs on the

use of technology. Teachers who have higher technology value beliefs are also more

likely to embrace constructivist pedagogical beliefs and integrate technology for student-

centered learning such as designing higher-order thinking and critical thinking activities

(Ertmer et al., 2012; Hixon & Buckenmeyer, 2009; Hsu, 2016). In this current study with

K-12 teachers who integrated 3D printing technologies into their science classrooms, it

66

is important to investigate how teachers’ pedagogical beliefs, self-efficacy in technology

integration, and technology value beliefs were related to their 3D printing technology

integration, and how these teacher beliefs correlated with each other, to provide

implications on improving teachers’ 3D printing technology integration in science

classrooms by initially investigating the relationship between teachers’ beliefs related to

technology integration and their teaching practice.

Teacher Beliefs and Student Learning Outcomes

In addition to the relationships between teacher beliefs and technology

integration, research also found teacher beliefs is associated with student learning

outcomes. Teacher beliefs can influence students’ learning performance and motivation

both “directly through observable teacher behaviors and indirectly through more subtle

forms of communication” (Midgley, Feldlaufer, & Eccles, 1989, p. 247). Therefore, even

if teachers’ technology integration practice is not aligned with their beliefs, students may

still be influenced by teacher beliefs.

Although there was little research on the direct relationship between teachers’

pedagogical beliefs and student learning outcomes, research indicated that teachers

with different pedagogical beliefs may tend to facilitate either teacher-centered or

student-centered learning (Tondeur et al., 2017), and student-centered learning fosters

students’ affective learning outcomes such as student engagement and motivation

(Cornelius-White & Harbaugh, 2009). As a meta-analysis study (Cornelius-White, 2007)

on student-centered teacher-student relationship demonstrated, learner-centered

teacher variables consisting of empathy, warmth, genuineness, nondirectivity (student-

initiated and student-regulated activities), higher-order thinking, encouraging

learning/challenge, and adapting to individual and social differences, had positive

67

associations with student cognitive and affective or behavioral outcomes including

student motivation. Therefore, there might be associations between teachers’

pedagogical beliefs and students’ cognitive learning outcomes such as knowledge and

skills, and affective learning outcomes such as motivation and interest.

There was also a lack of studies on the relationship between teachers’

technology value beliefs and student learning outcomes. However, research indicated

that teachers are more likely to integrate technology to promote student learning when

they believe the technology is valuable to be integrated (Ottenbreit-Leftwich et al.,

2010), and teachers’ technology integration may influence student learning.

Furthermore, even if teachers’ actual technology integration is not aligned with their

technology value beliefs due to some external barriers, teachers’ technology value

beliefs may still have influences on students because teacher beliefs can impact

students in some subtle forms of communication (Brophy & Good, 1974; Good, 1981;

Heller & Parsons, 1981, as cited in Midgley et al., 1989). Teachers’ technology value

beliefs may be communicated to the students and influence students even if teachers

cannot integrate the technology at a higher level as they would like to. For instance, a

teacher who has high value beliefs on the technology may show great enthusiasm when

just talking about the technology, which may make the technology appealing to students

and increase students’ motivation to learn or even enhance students’ interest in

pursuing a related career.

Most studies about the relationship between teacher beliefs and student learning

outcomes focused on teachers’ self-efficacy. The meta-analysis study conducted by

Zee and Koomen (2016) indicated that teachers’ self-efficacy in teaching was positively

68

related to students’ academic achievement and motivation. Specifically, researchers

found that teachers’ higher self-efficacy was associated with students’ learning

performance (e.g., Ashton & Webb, 1986; Brookover, Beady, Flood, Schweitzer, &

Wisenbaker, 1979; Brophy & Evertson, 1977; Hoy & Davis, 2005; Shahzad & Naureen,

2017) and students’ affective learning outcomes such as enhanced motivation (e.g.,

Eccles & Wigfield, 1985; Lazarides, Buchholz, & Rubach, 2018; Mojavezi & Tamiz,

2012; Pan, 2014; Schiefele & Schaffner, 2015).

Teacher beliefs may impact how teachers integrate the technologies into their

classrooms and teachers’ different levels of 3D printing technology integration may also

have different influences on students. Teacher beliefs may also have influences on

students even if their 3D printing integration levels are not aligned with their beliefs. It is

important to investigate both the relationships between teachers’ levels of 3D printing

integration and students’ learning outcomes, and the relationship between teacher

beliefs and students’ learning outcomes if teacher beliefs and teachers’ 3D printing

integration levels are not highly correlated. As the teachers were from different states

and different school levels, and they used different learning content and assessments, it

was not feasible to compare students’ learning performance in a consistent and

systematic way. Therefore, the current study did not include students’ learning

performance but focused exclusively on students’ STEM motivation, interest in STEM

careers, and 21st century skills.

Students’ STEM Motivation, Interest in STEM Careers, and 21st Century Skills

The review on students’ STEM motivation, interest in STEM careers, and 21st

century skills is guided by the underlying theoretical foundation of social cognitive

69

career theory and the expectancy-value theory of motivation. The theoretical framework

can be viewed in Figure 2-5.

Figure 2-5. Theoretical framework for STEM motivation, interest in STEM careers, and 21st century skills

Social Cognitive Career Theory

Social cognitive career theory (SCCT) is derived from Bandura’s (1986) social

cognitive theory and it provides a framework to understand people’s career interests,

career choice, and performance (Lent & Brown, 1996). SCCT focuses on a few

constructs including self-efficacy, outcome expectations, and personal goals, and how

these constructs interrelate with personal factors and environments in the process of

career development (Lent & Brown, 1996). Students’ academic goals and career choice

Social Cognitive Career Theory

Expectancy-Value Theory of Motivation

STEM

Motivation

STEM self-efficacy

(competency beliefs)

STEM value beliefs

Interest in STEM

careers

21st century

skills

STEM academic

and career pursuit

70

are influenced by their academic and career interests, self-efficacy, outcome

expectations, and environmental support and barriers.

SCCT has been applied by researchers to study undergraduate students’ STEM

career pathways (e.g., Byars-Winston, Estrada, Howard, Davis, & Zalapa, 2010; Lent et

al., 2005; Lent, Lopez, Lopez, & Sheu, 2008; Lent, Lopez, Sheu, & Lopez, 2011). Lent

et al. (2005) used an SCCT-based model to predict engineering interests and major

choice goals with 487 students at three universities, who enrolled in introductory

engineering courses. Their study found that self-efficacy and outcome expectations

were predictive of student interest in engineering, interests and perceived barriers were

predictive of their engineering academic goals, and the supports and barriers were

correlated with engineering self-efficacy. The SCCT-based model was also used in

computing disciplines (e.g., Lent et al., 2008; Lent et al., 2011) and science discipline

(Byars-Winston et al., 2010) to predict college students’ academic interests and major

choice in STEM.

SCCT has also been applied to investigate high school students’ choices to

pursue STEM majors (e.g., Nauta & Epperson, 2003; Wang, 2013). Nauta and

Epperson (2003) investigated 204 high school female students who attended science,

math, and engineering (SME) career conferences in a 4-year longitudinal study to

predict the girls’ choice of college major in SME, and their SME self-efficacy and

outcome expectations in college. The results showed there was a positive relationship

between students’ self-efficacy and interest in science; interest in science, self-efficacy,

and outcome expectations were positively correlated with students’ college major choice

in SME. Wang (2013) applied SCCT theory to understand recent high school graduates’

71

entrance into STEM majors in 4-year institutions. The results suggested that students’

intent to major in STEM was directly affected by their 12th-grade math achievement,

exposure to math and science courses, and math self-efficacy beliefs (Wang, 2013).

Suggested by Unfried et al. (2015), in terms of STEM career development,

student STEM self-efficacy and expectancy-value beliefs and interest in STEM careers

are key components of social cognitive career theory (SCCT) (Lent & Brown, 2006;

Lent, Brown, & Hackett, 2000; Lent, Sheu, Gloster, & Wilkins, 2010), and students also

need 21st century skills for academic and career development.

Expectancy-Value Theory of Motivation and STEM

Expectancy-value theory of motivation was originated by Atkinson (1957) who

developed the theory to understand individuals’ achievement motivation. Atkinson

(1957) defined expectancies as individuals’ anticipations of success or failure of their

performance and defined value as attractiveness of success on a task. Other

researchers continued the research on expectancy-value and further investigated the

relationship between students’ expectancies for success, subjective task values, and

their task choice, performance, and persistence (e.g., Crandall, 1969; Crandall,

Katkovsky, & Preston, 1962; Eccles et al., 1983; Feather, 1982, 1988, 1992; Wigfield &

Eccles, 1992). Eccles et al. (1983) expanded the expectancy-value theory by creating

an expectancy-value model of achievement performance and choice with more internal

and external variables, including achievement behaviors (e.g., students’ achievement

performance and choice) and belief and value constructs such as subjective task

values, expectancies for success, achievement goals, and competence or ability beliefs.

The gist of expectancy-value theory is that “individuals’ expectancies for success and

72

the value they have for succeeding are important determinants of their motivation to

perform different achievement tasks” (Wigfield, 1994, p. 50).

As Eccles et al. (1983) suggested, students’ choice of achievement tasks,

achievement performance, and persistence are most directly influenced by their

expectancies for success on those tasks and their subjective value of success on those

tasks. Expectancies for success and subjective task values are two key components of

expectancy-value theory (Eccles et al., 1983; Eccles & Wigfield, 2002; Wigfield, 1994;

Wigfield & Eccles, 2000). Expectancies for success refer to individuals’ beliefs about

how well they will perform on a task (Eccles et al., 1983; Wigfield, 1994), namely a

person’s competency or ability beliefs to complete a task or achieve a goal, which can

be represented as a person’s self-efficacy. Therefore, according to the expectancy-

value theory of motivation, students’ STEM motivation can be influenced by their self-

efficacy and values beliefs in STEM.

Self-efficacy and STEM

Self-efficacy is grounded in social cognitive theory which suggests that students’

achievement depends on interactions between students’ behaviors, beliefs, and

environmental conditions (Bandura, 1986, 1997). Self-efficacy refers to individuals’

beliefs in their abilities to complete tasks or achieve goals (Bandura, 1986, 1997). Self-

efficacy concerns with the “judgements of how well one can execute courses of action

required to deal with prospective situations” (Bandura, 1982). Students’ judgment of

their self-efficacy can be obtained from their actual performance, vicarious experiences,

verbal persuasions received from others, and their physiological and affective states

(Bandura, 1997; Schunk & Pajares, 2002).

73

Self-efficacy influences students’ thought patterns and emotional reactions when

faced with challenges. Those who have low self-efficacy dwell on their personal

deficiencies and imagine the potential challenges as more formidable than they really

are (Beck, 1976; Lazarus & Launier, 1978; Meichenbaum, 1977; Sarason, 1975; as

cited in Bandura, 1982). While individuals with low self-efficacy focus their attention on

concerns of failings, individuals with high self-efficacy divert their attention to the

demands of the task challenges and take efforts to overcome obstacles (Bandura,

1982). Students’ self-efficacy can influence their task choices, levels of efforts,

persistence, and achievement (Bandura, 1997; Schunk, 1995; Schunk & Pajares,

2002). Self-efficacy is important for a student to tackle learning challenges and be

persistent in learning to achieve learning goals. Self-efficacy is positively related to

student interests and engagement in learning (Schunk & Pajares, 2002) and student

engagement and motivation are highly related (Martin, Ginns, & Papworth, 2017;

Reeve, 2012; Schunk & Mullen, 2012). Students with high self-efficacy spend greater

effort and are more persistent to complete a task (Pajares, 2005; Zimmerman, 2000).

Research indicates that student self-efficacy in STEM is positively related to their

STEM performance and their persistence in STEM disciplines (Britner & Pajares, 2006;

Miller, 2015; Pajares, 2005; Schunk & Pajares, 2002). As Britner and Pajares (2006)

found, students’ performance in science class is positively related to their science self-

efficacy. Grigg, Perera, McIlveen, and Svetleff (2018) investigated math self-efficacy of

students from grade 6 to grade 10 and found that the students’ math self-efficacy

positively predicted their improvement in math performance. Brown, Concannon, Marx,

Donaldson, and Black (2016) examined middle school students’ STEM self-efficacy and

74

their intentions to persist in STEM and found students’ STEM self-efficacy was a

significant positive predictor for students’ intentions to persist in STEM.

Research also indicates that students’ self-efficacy in STEM is associated with

their pursuit in STEM fields when choosing majors. Scott and Mallinckrodt (2005)

surveyed students who previously participated in a high school enrichment program and

found that students who later majored in science had significantly higher science self-

efficacy than students who did not choose science major. Wang (2013) investigated

factors that impacted postsecondary school students’ choice of majors and found that

students’ math self-efficacy was positively correlated with the intent to major in STEM

disciplines. Sahin, Ekmekci, and Waxman (2017) examined the relationship between

2,246 high school graduates’ math and science self-efficacy and their likelihood of

majoring in STEM in college and found that male students who had higher math self-

efficacy and female students who had higher science self-efficacy are more likely to

choose a STEM major than their counterparts who had lower math and science self-

efficacy.

Value beliefs and STEM

In the expectancy-value theory, value beliefs consist of intrinsic interest value,

attainment value/importance, utility value/usefulness, and cost (Eccles et al., 1983).

Intrinsic interest value is the enjoyment an individual gains from doing the task;

attainment value is the importance of doing well on the task; utility value is how a task

contributes to future plans; and cost refers to what the individual has to give up on other

things in order to do the task as well as anticipated efforts the individual has to put into

the task (Eccles et al., 1983; Wigfield, 1994).

75

Expectancy-value theory provides a theoretical framework to explain the

relationship between students’ psychological factors and their interest in STEM and

career choices (Wang & Degol, 2013). Self-efficacy and value beliefs may influence

students’ motivation and task selection in STEM. Students’ beliefs about how well they

can perform in an activity and how much they value the activity can influence their

choices, performance, and persistence in STEM learning. When students are confident

that they can perform well and be successful in subject areas such as STEM, they are

more likely to engage and persist in learning which may lead to better academic

performance and course enrollment (Wigfield & Eccles, 2002).

Students’ value beliefs in STEM are predictive of their choice behaviors and

beliefs such as persistence in STEM learning and intentions to select STEM courses

and choose STEM careers (Eccles, 2009; Wang & Degol, 2013; Wang & Eccles, 2013).

Research shows that high value beliefs are associated with students’ persistence in

learning advanced science and math (Fan, 2011; Simpkins, Davis-Kean, & Eccles,

2006). Tai et al. (2006) surveyed 12,000 middle school students and found students’

interest in STEM was a significant predictor of their future career choices. Simpkins et

al. (2006) found that students’ interest in math and science in elementary school was

predictive of their course selection in high school. Expectancy-value theory is a strong

theoretical framework that can investigate the factors that influence students STEM

learning persistence and career aspirations.

Students’ Interest in STEM Careers

The construct of interest began with Herbart’s view that education should foster

unspecialized and multi-faceted interests which would facilitate learning (Wigfield &

Cambria, 2010). Interest is an essential motivational and driving force for learning

76

(Dewey, 1913). Interest is a psychological state which is “a particular relation of that

individual in engagement with that play object/task, relative to the other activities with

which he or she engages” (Renninger, 1992, p. 362). Interest is a relational concept

consisting of the relationship between an individual and an object or activity (Krapp,

2002; Schiefele, 2009). Interest can be mediated by the interaction between the

individual and the object or activity and both personal factors and environmental factors

can influence interest (Mitchell, 1993; Renninger & Hidi, 2002). Researchers

categorized interest as individual interest and situational interest (e.g, Hidi & Renninger,

2006; Krapp, 2002; Schiefele, 2009). Situational interest is “a temporary state aroused

by specific features of a situation, task, or object” and individual interest is “a relatively

stable affective-evaluative orientation toward certain subject areas or objects”

(Schiefele, 2009, p. 198).

Research shows that student interest has a powerful influence on learning in

terms of students’ attention, goals, and levels of learning (Hidi & Renninger, 2006).

Interest is also related to students’ self-efficacy and students with stronger interest

usually have higher self-efficacy and can persist on challenging tasks (Hidi & Ainley,

2008; Sansone, 2009, as cited in Honey et al., 2014) In the context of STEM learning,

there are strong associations among students’ STEM interest, interest in STEM careers,

and intention to pursue a STEM major or career. Sadler et al. (2012) found that

students’ interest in STEM at the start of high school was a key predictor of their STEM

career interests when they graduated. Christensen and Knezek (2017) collected data

from over 800 middle school students who participated in a hands-on and real-world

application curriculum and examined the relationship between students’ STEM interests

77

and their intentions to pursue STEM careers. Results showed that students’ interest in

STEM positively aligned with their intent to pursue STEM careers. Maltese and Tai

(2011) found that eighth-grade students who had interest in a science career and

believed science would be useful in their future were more likely to earn a bachelor’s

degree in a STEM discipline.

Since students’ interest in STEM and STEM-related careers are strong predictors

of their participation in STEM disciplines and future STEM career, it is important to

foster students’ interest in STEM disciplines and STEM careers. Individual interest is

relatively more difficult to change but teachers could engage students in classroom

activities to foster students’ situational interest which may subsequently develop into

personal interest.

21st Century Skills

The construct of 21st century skills has been defined in many different ways.

Twenty-first century skills are important knowledge and skills that students need to

succeed in academic and career development (P21). The core components of 21st

century skills include critical thinking, communication, collaboration, creativity, problem

solving, and digital literacy (NRC, 2010; P21; PCAST, 2010). Twenty-first century skills

are necessary for academic and career development in the 21st century information

era. Twenty-first century skills are essential for students to participate and succeed in

STEM disciplines and STEM careers and students also develop 21st century skills

through STEM education. As Jones (2014) stated, 21st century skills and STEM

education are coalesced to educate students to become productive and technologically

literate citizens of tomorrow. Twenty-first century skills have been integrated into K-12

STEM curricula as instructional goals and course standards. There is a close alignment

78

between K-12 STEM standards and 21st century skills. National policymakers,

researchers, and educators have been calling for attention and taking efforts to improve

students’ 21st century skills (NRC, 2010).

As students’ STEM motivation and interest in STEM careers are associated with

students’ persistence in STEM learning and future career choice, and students need the

21st century skills for STEM learning and STEM career development, enhancing

students’ STEM motivation, interest in STEM careers, and 21st century skills altogether

is imperative to increase and strengthen students’ participation in STEM disciplines the

future STEM workforce.

Conceptual Framework

A conceptual framework is an argument based on theory and driven by evidence

with the purpose to “justify the research problem, define relevant concepts, establish

theoretical and empirical rationale, select appropriate methods, and interpret results

relative to theory” (Antonenko, 2015, p. 58). Based on the theories and literature

reviewed above, the conceptual framework (Figure 2-6) guiding this study posits that: 1)

Teacher beliefs (pedagogical beliefs, self-efficacy beliefs in 3D printing integration, and

3D printing value beliefs) may be correlated with teachers’ 3D printing technology

integration in K-12 science classrooms (teacher proximal outcome); and 2) Teacher

beliefs and teachers’ 3D printing technology integration in K-12 science classrooms may

predict students’ STEM motivation, 21st century skills, and interest in STEM careers

(student proximal outcome). Correlational and regressional analyses were conducted to

examine these relationships and multiple data sources were used for the analyses. As

indicated in the conceptual framework, teacher beliefs data were collected with a

teacher beliefs survey, teachers’ lesson plans were used to analyze teachers’ 3D

79

printing integration, and students’ STEM motivation, 21st century skills, and interest in

STEM careers were collected with the S-STEM survey (Unfried et al., 2015).

Although teacher beliefs on 3D printing technology integration may impact how

teachers integrate the technologies into their science classrooms, this study cannot

determine if there was a causal relationship between teacher beliefs and teachers’

technology integration practice. Therefore, this study focused on the correlation

between teacher beliefs and teachers’ 3D printing technology integration in the

classrooms. As extant literature indicated that teachers’ 3D printing integration can

influence students’ learning outcomes and teacher beliefs may also influence students’

learning outcomes, this study explored how both teacher beliefs and teachers’ 3D

printing integration might predict students’ learning outcomes including students’ STEM

motivation, 21st century skills, and interest in STEM careers.

Teachers’ different 3D printing technology integration involved students in varied

learning activities which may have different impacts on students. Some teachers may

have just printed out the 3D objects by themselves and showed the objects to students,

and students did not have much interaction with the 3D printing technology or the

printed objects; while some other teachers may engage students in the 3D printing

process for students to learn how 3D printers work or even have students design and

print out 3D models, which may have enhanced students’ engagement in learning.

Some teachers may just introduce what 3D printing technology is without adequate

connection to the curriculum while other teachers may involve students in problem-

based learning activities with 3D printing process or 3D printed objects to engage

students in learning. Additionally, some teachers may just integrate 3D printing

80

technology in science learning, while others may also integrate math learning or

engineering learning. Teachers may have integrated 3D printing technologies in a

myriad of ways, which could have different influences on students’ STEM motivation,

21st century skills, and interest in STEM careers.

Abundant research suggests that students’ STEM motivation and interest in

STEM careers are associated with students’ participation in STEM disciplines and

STEM careers. The 21st century skills have been integrated as curriculum standards

and curriculum frameworks and improving student motivation in STEM disciplines and

career skills have gained increasing attention from national policymakers (Unfried et al.,

2015). Students’ STEM motivation, 21st century skills, and interest in STEM careers are

essential for students to participate in STEM disciplines and STEM careers. Therefore,

it is important to improve students’ STEM motivation, 21st century skills, and interest in

STEM careers, in order to increase students’ likelihood of choosing majors in STEM

disciplines and participate in STEM careers and also enhance students’ persistence and

success in STEM disciplines and careers. The current study explained the importance

of enhancing students’ STEM motivation, 21st century skills, and interest in STEM

careers to increase students’ participation in STEM disciplines and STEM careers

(student distal outcome), but this potential influence was not examined empirically due

to time constraints and feasibility of this study.

81

Figure 2-6. Conceptual framework of this study

Teacher

Proximal

Outcome

Student

Proximal

Outcome

Student

Distal

Outcome

Teacher

Beliefs

Self-Efficacy

Beliefs in 3D

Printing Integration

STEM

Motivation

Interest in

STEM careers

Participation

in STEM

disciplines

and STEM

careers

21st Century

Skills

3D Printing

Integration

in K-12

Science

Classrooms

3D Printing Value

Beliefs

Pedagogical Beliefs

Data Source:

Lesson Plan

Date Source:

S-STEM Survey

Not

measured in

this study

Data Source:

Teacher Beliefs

Survey

82

CHAPTER 3 METHODOLOGY

This study used quantitative methodology by conducting correlational analysis to

investigate how teacher beliefs are related to their 3D printing integration in the science

classrooms and multilevel modeling analysis to examine how teacher beliefs and

teachers’ 3D printing integration predict students’ STEM motivation, 21st century skills,

and interest in STEM careers. This chapter describes the study context and

participants, instrumentation, data sources and data collection, and data analysis.

Context and Participants

This study used a convenient sample of teachers and students who participated

in the iDigFossil project, a 3-year National Science Foundation (NSF) funded project

“iDigFossils: Engaging K-12 Students in Integrated STEM via 3D Digitization, Printing

and Exploration of Fossils” (PI: Dr. Pavlo Antonenko. Award No. 1510410). The

iDigFossils project was an integrated STEM initiative aimed to engage students in

STEM learning through the integration of 3D printing technology in K-12 science

classrooms within the context of paleontology, a rich, interdisciplinary science that

examines life in deep time. A group of teachers and their students in different states

across the United States participated in this project. Each teacher was provided a 3D

scanner, a 3D printer, a laptop, and related resources. To facilitate teachers’ 3D printing

integration in their science classrooms, a week-long workshop during the summer of

each year of the project was provided for the teachers to learn how to use 3D printing

technologies and how to integrate 3D printing technologies in science classes within the

context of paleontology. All the teachers participated in the workshop. The teachers also

had assistance from experts in educational technology and paleontology for their lesson

83

plan design. After the teachers designed the lesson plans, they sent them to the experts

for review and they received feedback on how to improve the lesson plans. Throughout

teachers’ participation in the project, the professionals in the iDigFossils project were

available through face-to-face or online meeting for any assistance the teachers

needed.

The participants in this study were a portion of the teachers and students who

participated in the iDigFossils project. The selection of participants for this study

depended on whether there were adequate data from the teachers and students. If

teachers' lesson plans on how they integrated 3D printing into their science classrooms

were available and they had students who completed both the S-STEM survey (Unfried

et al., 2015) pretest and posttest, their data were included for analysis. In this study, 26

teachers met with these criteria. Among the 26 teachers, 24 teachers completed the

teacher beliefs survey. The 2 teachers who did not take the teacher beliefs’ survey were

treated as missing data when doing statistical analyses that involved data on teachers’

beliefs. After data cleaning and screening of students who completed both the pretest

and posttest with the S-STEM survey, a total number of N = 1,501 students were

included for data analysis.

The students (see demographics in Table 3-1) consisted of 712 males and 789

females, with 192 elementary students, 847 middle school students, and 462 high

school students. Students’ race/ethnicity included White (N = 719), Hispanic (N = 197),

Mixed race or multi-race (N = 146), Asian (N = 100), Black or African American (N = 96),

American Indian or Alaska Native (N = 66), Native Hawaiian or other Pacific Islander (N

= 19), other race/ethnicity (N = 122), and students who did not wish to answer (N = 36).

84

Table 3-1. Student demographics

Demographics category Student N (%)

Gender Male: N = 712 (47.44%); Female: N = 789 (52.56%).

School Level Elementary school: N = 192 (12.79%); Middle school: N = 847

(56.43%); High school: N = 462 (30.78%).

Race/Ethnicity White: N = 719 (47.90%); Hispanic: N = 197 (13.12%); Asian: N

= 100 (6.66%); Mixed race or multi-race: N = 146 (9.77%);

Black or African American: N = 96 (6.4%); American Indian or

Alaska Native: N = 66 (4.4%); Native Hawaiian or other Pacific

Islander: N = 19 (1.27%); Other: N = 122 (8.13%); Do not wish

to answer: N = 36 (2.40%).

Among the 26 teachers (view Table 3-2 for demographics), there were 4 males

and 22 females. Their ages ranged from 21 to above 60 with 3 teachers aged between

21 and 30, 9 teachers aged between 31 and 40, 7 teachers aged between 41 and 50, 5

teachers aged between 51 and 60, and 2 teachers aged 61 or above. There were 4

elementary teachers, 13 middle school teachers, and 9 high school teachers. The

majority of the teachers were White (N = 20) and there were 2 teachers being African

American, 2 teachers being Hispanic, and 2 teachers being mixed-race. The teachers

were across 6 states including Oklahoma (N = 4), California (N = 9), Georgia (N = 3),

Texas (N = 1), Louisiana (N = 1), and Florida (N = 8).

Table 3-2. Teacher demographics

Demographics Category Teacher N

Gender Male: N = 4; Female: N = 22.

Age 21-30: N = 3; 31-40: N = 9; 41-50: N = 7; 51-60: N = 5; 61+: N = 2.

School Level Elementary school: N = 4; Middle school: N = 13; High school: N = 9.

Race/Ethnicity White: N = 20; African American: N = 2; Hispanic: N = 2; Mixed race: N = 2.

State Oklahoma: N = 4; California: N = 9; Georgia: N = 3; Texas: N = 1; Louisiana: N = 1; Florida: N = 8.

85

Instrumentation

S-STEM Survey

The S-STEM survey developed and validated by Unfried et al. (2015) from the

Friday Institute for Educational Innovation at North Carolina State University was used

as the instrument to measure students’ attitude towards STEM, 21st century skills, and

interest in STEM careers. Although the authors used the construct of attitude, the items

in the survey either reflected students’ self-efficacy and expectancy value. For instance,

in the science attitude subscale, there were 9 items, and 4 items were about students’

self-efficacy and 5 items were about students’ expectancy value. The 4 self-efficacy

items consisted of “I am sure of myself when I do science”, “I know I can do well in

science”, “I can handle most subjects well, but I cannot do a good job with science”, and

“I am sure I could do advanced work in science”. The 5 items regarding expectancy

value were “I would consider a career in science”, “I expect to use science when I get

out of school”, “Knowing science will help me earn a living”, “I will need science for my

future work”, and “Science will be important to me in my life’s work”. Students’ self-

efficacy and expectancy value can be conceptualized as student motivation according

to Eccles and Wigfield’s (2002) motivational theory. Therefore, this study utilized the

construct of motivation instead of attitude. The latent variables of STEM motivation and

21st century skills consist of four constructs: math motivation, science motivation,

technology and engineering motivation, and 21st century skills, and the reliability of the

four constructs was .90, .89, .90, and .92 respectively, according to the validation study

by Unfried et al. (2015).

There were 8 items for math motivation, 9 items for science motivation, 9 items

for technology and engineering motivation, and 11 items for 21st century skills

86

(Appendix A). All the items were rated using a five-point Likert scale, from “Strongly

Disagree”, “Disagree”, “Neither Agree nor Disagree”, “Agree”, to “Strongly Agree”, with 1

indicating “Strongly Disagree” and 5 indicating “Strongly Agree”. The items No. 1, 3, 5 in

math motivation and item No. 8 in science motivation were expressed in reversed ways,

so these items were reverse coded. The S-STEM survey also included 12 items to

assess students’ interest in STEM careers. These items were descriptions of STEM-

related subject areas that involve math, science, engineering and/or technology, and

corresponding jobs connected to each subject area. The 12 STEM-related career

pathways included physics, environmental work, biology and zoology, veterinary work,

mathematics, medicine, earth science, computer science, medical science, chemistry,

energy, and engineering. The STEM career interest items used a 4-point Likert scale,

including “Not at all Interested”, “Not so Interested”, “Interested”, and “Very Interested”,

with 1 indicating “Not at all Interested”, and 4 indicating “Very Interested”. The S-STEM

survey (Friday Institute for Educational Innovation, 2012) is attached in Appendix A. A

pretest and a posttest of the online version of the S-STEM survey were administered

before the first activity and upon completion of the last activity by each teacher and in

each classroom.

Teacher Beliefs on 3D Printing Integration Survey

The teacher beliefs on 3D printing integration survey (Appendix B) consisted of a

demographic section including teachers’ names (for matching with their lesson plan and

reflection), gender, age range, and years of teaching in K-12 education; rating scales for

pedagogical beliefs, self-efficacy in 3D printing integration, 3D printing value beliefs; and

a few open-ended questions regarding teacher beliefs on 3D printing integration and the

challenges they encountered when designing and implementing the lessons.

87

Specifically, the open-ended questions included: 1. How do you feel about integrating

3D printing technology into your science teaching? 2. Do you think you have sufficient

knowledge and skills to integrate 3D printing technology in your science classes?

Please explain. 3. What are the biggest advantages in integrating 3D printing

technology in science teaching? 4. What are the biggest challenges in integrating 3D

printing technology in science teaching?

The following section focused on the scales for measuring: teacher pedagogical

beliefs, teacher self-efficacy in 3D printing integration, and teacher 3D printing value

beliefs.

Teacher pedagogical beliefs

Teacher pedagogical beliefs were measured with 15 items that assess teachers’

constructivist (student-centered learning) beliefs from three sub-scales (4 items in

subscale J1, 7 items in J2, and 4 items in J3) of the Teaching, Learning, and Computing

(TLC) survey (Becker, 2001; Ravitz, Becker, & Wong, 2000), which was developed by a

project funded by the National Science Foundation and the U.S. Department of

Education.

The three sub-scales (J1, J2, J3) assess teachers’ pedagogical beliefs with a

teacher-centered and student-centered learning continuum. Scale J1 asks teachers to

rate on how open-ended they think the class discussions should be. Scale J2 asks

teachers to indicate how much they disagree or agree with a few statements about

teaching and learning. The original scale of J2 used a 6-point Likert scale without the

neutral option (strongly disagree, moderately disagree, slightly disagree, slightly agree,

moderately agree, strongly agree). The current study modified it with a 5-point Likert

scale (strongly disagree, disagree, neither disagree nor agree, agree, strongly agree) to

88

make it consistent with the 5-point Likert scale in scales J1 and J3, and scales in the

other sections of the teacher beliefs on 3D printing technology integration survey. Scale

J3 provides two opposite teaching philosophy statements on student-centered learning

and teacher-centered learning with five radio buttons in the middle for teachers to select

which statement they are more inclined to. In J2, items No. 5, 6, 7, 10, and 11, and all

the items in J3 were expressed in reversed ways, so these items were all reversed

before further analysis.

The validity of the entire TLC survey was verified by classroom observation and

teacher interview with 72 teachers in 24 schools in the United States and the reliability

(Cronbach’s alpha) of 13 out of the 15 items (2 items in J1, 7 items in J2, and 4 items in

J3) of TLC was 0.83 (Ravitz et al., 2000). The reliability verification did not include two

other items in J1. However, in the study of Kim et al. (2013), the reliability (Cronbach’s

alpha) of all the four items in J1 was 0.92. Therefore, the current study included all the 4

items in J1, with 15 items in total. An overview of the scale with the 15 items is attached

in Appendix B.

Teacher self-efficacy in 3D printing technology integration

Teachers’ self-efficacy in technology integration was measured using an adapted

version of the TPACK survey that was developed and validated by Schmidt et al. (2009)

to measure pre-service teachers’ technology integration knowledge. The content validity

of the original TPACK survey was evaluated and verified by experts in TPACK. The

construct validity was verified using principal components factor analysis for each

knowledge domain which yielded acceptable factor loadings. The reliability (Cronbach’s

alpha) for each knowledge domain ranged from 0.80 to 0.92.

89

The adapted scale for this study did not add the sections designed specifically for

pre-service teachers. Since all teachers in this study were in-service science teachers

and the classes were science classes, the survey excluded the items about how pre-

service teachers’ professors model combining content, technologies, and teaching

approaches in their teaching of different subjects, and items focusing on content other

than science. Items about general technologies were also adapted as 3D printing

technology if they were possible to be modified. The other sections about teachers’ self-

efficacy in different types of TPACK knowledge also apply to in-service teachers.

Specifically, the survey for this study included teachers’ self-efficacy in Technology

Knowledge (TK), Content Knowledge (CK), Pedagogical Knowledge (PK), Pedagogical

Content Knowledge (PCK), Technological Content Knowledge (TCK), Technological

Pedagogical Knowledge (TPK), and Technological Pedagogical Content Knowledge

(TPACK). The scale on teacher self-efficacy in 3D printing technology integration is

attached in Appendix B.

Teacher 3D printing value beliefs

The teacher 3D printing value beliefs scale was adapted from Eccles and

Wigfield’s (1995) survey to measure teachers’ intrinsic interest value, attainment value

(importance), and extrinsic utility value (usefulness) regarding integrating 3D printing

technology into their science classrooms.

The items of the original scale were developed with the guidance of the

expectancy-value theory of motivation and were validated (Eccles & Wigfield, 1995).

The reliability (Cronbach’s alpha) of the original scale was acceptable with 0.76, 0.70,

and 0.62 for the intrinsic interest value, attainment value, and extrinsic utility value

respectively given there were only 2 to 3 items in each subscale. Cheng and Xie (2018)

90

adapted the original scale to measure teachers’ value beliefs about learning digital

content evaluation and treated the value beliefs as a unidimensional construct, which

showed good reliability with Cronbach’s alpha of 0.88 and 0.91 of the scale

administered at the beginning and end of their professional development program.

In the current scale on teachers’ 3D printing value beliefs, there were 2 items to

measure the intrinsic interest value in integrating 3D printing technology, 3 items to

measure the perceived importance of doing well in integrating 3D printing technology,

and 2 items to measure the perceived usefulness of integrating 3D printing technology

to enhance engage students and enhance student learning. All the 7 items in the scale

used a 5-point Likert scale. The scale is attached in Appendix B.

Lesson Plan Codebook

Lesson plan codebooks were developed to code the 3D printing integration

levels, STEM integration levels, and school levels including elementary, middle, and

high school levels. The original coding plan also included teachers’ 3D printing

implementation duration. However, the duration information provided in the lesson plans

did not allow consistent coding. For instance, some teachers just provided the number

of days but did not provide specific class periods information. Although some teachers

provided the class periods information, the time for one class period varied from teacher

to teacher. Therefore, the final coding only included teachers’ 3D printing integration

levels and STEM integration levels without the implementation duration.

3D printing integration levels

Researchers have developed a few technology integration assessment

instruments based on the TPACK framework to evaluate teachers’ technology

integration in the classrooms. The general technology integration assessment

91

instrument developed by Britten and Cassady (2005) and the TPACK-based technology

integration assessment instrument by Harris, Grandgenett, and Hofer (2010) did not

focus on any specific technology or content area. Pringle, Dawson, and Ritzhaupt

(2015) developed a technology integration assessment instrument based on the TPACK

framework to evaluate teachers’ technology integration in science classes. All the

technology integration assessment instruments mentioned here focus on the general

technology integration which encompasses different technologies teachers integrate in

their classes.

Because the current study focused on the integration of a specific technology –

3D printing technology in science classes, the assessment instrument for this study was

adapted from the general TPACK-based technology integration assessment instruments

to specifically focus on the integration of 3D printing technology in science classes. As

teachers’ technological pedagogical content knowledge is the core of the TPACK

framework, this instrument just focused on teachers’ technological pedagogical content

knowledge (TPACK) related to the integration of 3D printing technology. In addition to

referencing the technology integration assessment instruments developed by other

researchers, the instrument for this study was also guided by the Technology

Integration Matrix (TIM) which illustrated the levels of technology integration and

meaningful learning environments.

The levels of 3D printing integration included entry, adoption, adaptation,

infusion, and transformation (Harmes et al., 2016). Specifically, the levels were

determined by how 3D printing was used and how the integration engaged students in a

meaningful learning environment and facilitated higher-order learning activities such as

92

apply, analyze, evaluate, and create (Krathwohl, 2002). The levels were scored from 1

to 5. Table 3-3 provides the definition and the coding and scoring criteria.

Table 3-3. Codebook for 3D printing integration levels

Category Criteria Score

3D printing integration levels

Entry: The teacher just introduced 3D printing technology or brought in 3D printed objects, students had no access to 3D printing technology and had minimal access to 3D printed objects, 3D printing technology did not contribute to the learning environment or student learning.

1

Adoption: Students had access to 3D printing technology or 3D printed objects, but the technologies were not used during the learning activity. The use of 3D printing technology or 3D printed objects minimally contributed to a meaningful learning environment and student learning.

2

Adaptation: Students used 3D printed objects for some part of the learning activity but there was no deep interaction. The use of 3D printing technology or 3D printed objects contributed to a meaningful learning environment and student learning but not strong.

3

Infusion: Students used 3D printed objects throughout or for the most part of the learning activity. The use of 3D printing technology or 3D printed objects strongly contributed to a meaningful learning environment, but the 3D printing technology was not used to facilitate higher-order learning activities such as apply, analyze, evaluate, and create.

4

Transformation: Students participated in the 3D printing process, printed 3D models, and used 3D printed objects throughout or for the most part of the learning activity. The use of 3D printing technology and 3D printed objects strongly contributed to a meaningful learning environment and the 3D printing technology was used to facilitate higher-order learning activities such as apply, analyze, evaluate, and create.

5

93

STEM integration levels

A codebook with scoring criteria was created to evaluate teachers’ STEM

integration levels (Table 3-4). The codebook was created by focusing on the integration

of STEM content. As required by the iDigFossils project, all teachers indicated how they

designed learning activities to facilitate student collaboration in the classrooms and all

the teachers designed inquiry-based learning activities. Some teachers might have used

more instructional strategies than other teachers and some may have also used other

instructional strategies, however, there were no consistent ways to compare which

instructional strategy would be more effective. According to the systematic review and

meta-analysis articles on STEM integration (e.g., Mustafa, Ismail, Tasir, Said, &

Haruzuan 2016; Thibaut et al., 2018), there are a myriad of instructional strategies that

have been used in STEM integration, but there is no evidence on which instructional

strategy would be more effective than another one or whether STEM integration with

more instructional strategies would be necessarily more effective than STEM integration

with fewer strategies. Therefore, this study focused exclusively on the integration of

STEM content without including the instructional strategies used by teachers. The initial

coding found there were no integration of science, 3D printing technology, and

engineering, so the codebook included four criteria as shown in Table 3-4. The STEM

integration levels were scored from 1 to 4.

Table 3-4. Codebook STEM integration levels

Criteria Score

Science 1

Integration of STEM content

Science + 3D Printing Technology 2

Science + 3D Printing Technology + Math 3

Science + 3D Printing Technology + Math + Engineering 4

94

Data Sources and Data Collection

The data sources included: S-STEM survey, which assessed students’ science

motivation, technology/engineering motivation, math motivation, 21st century skills, and

interest in STEM careers before and after the 3D printing integration; teacher beliefs

survey, which investigated teachers’ pedagogical beliefs, 3D printing value beliefs, and

self-efficacy beliefs in 3D printing integration; and teachers’ lesson plans, which

provided data on teachers’ 3D printing integration levels and STEM integration levels. A

pretest of the S-STEM survey was administered by teachers in their science classroom

before the first activity, and a posttest with the same S-STEM survey was administered

upon completion of the last activity. Teachers’ lesson plans and teacher beliefs data

were collected after the teachers completed their 3D printing integrated science classes.

Data Analysis

This study used descriptive statistical analysis, lesson plan analysis, Pearson’s

Correlation analysis, multilevel modeling analysis, and multiple regression analysis to

answer the two research questions. To explain the relationship between teacher beliefs

and teachers’ 3D printing integration, this study also conducted thematic analysis for the

open responses in the teacher beliefs survey. Table 3-5 displays an overview of the

data analysis for each research question. This study intended to include school levels in

the data analysis, however, the sample size of different school levels was unbalanced.

Most of the classes were middle or high school levels and few were elementary levels

(Elementary school: N = 4 teachers with 192 students; Middle school: N = 13 teachers

with 847 students; High school: N = 9 teachers with 462 students.). Therefore, school

levels were only reported in the descriptive statistics without further analysis.

95

Table 3-5. Overview of data analysis

Research Questions Data Sources Data Analysis

RQ1: How are teachers’ beliefs correlated with their 3D printing integration in the science classrooms?

1. Lesson plan

2. Teacher beliefs on technology integration survey.

1. Descriptive statistical analysis for teacher beliefs survey;

2. Lesson plan analysis: 3D printing technology integration levels and STEM integration levels;

3.Pearson’s Correlation analysis with teacher beliefs, 3D printing integration levels, and STEM integration levels;

4. Thematic analysis for open responses in the teacher beliefs survey.

RQ2: How do teachers’ beliefs and their 3D printing integration in the science classrooms predict students’ STEM motivation, 21st century skills, and interest in STEM careers?

1. S-STEM pre- and post-survey;

2. Lesson plan.

1. Descriptive statistical analysis for S-STEM survey;

2. Lesson plan analysis: 3D printing technology integration levels and STEM integration levels;

3. Multilevel modeling analysis with student gender and pretest scores as student-level independent variables, 3D printing integration levels, STEM integration levels, and each component of teacher beliefs variables as teacher-level independent variables, and student posttest scores in science motivation, technology/engineering motivation, math motivation, and 21st century skills as the dependent variable respectively.

4. Multiple regression analysis with student posttest scores in their interest in STEM careers as the dependent variable, and all the student variables (student gender and pretest scores) and teacher variables (3D printing integration levels, STEM integration levels, and each component of teacher beliefs variables) as independent variables at the same level.

96

Data Analysis for RQ1

To address research question 1, teacher beliefs data and teachers’ lesson plans

were analyzed. First, descriptive statistical analysis was conducted; second, teachers’

lesson plans were analyzed; third, correlational analysis was conducted using

Pearson’s correlation; and lastly, thematic analysis was conducted for teachers’ open

responses in the teacher beliefs survey.

Descriptive statistical analysis

To conduct descriptive statistical analysis, an aggregated average score,

standard deviation, and the minimum and maximum were calculated for each

component of the teacher beliefs: pedagogical beliefs, self-efficacy beliefs, and 3D

printing value beliefs.

Lesson plan analysis

To investigate teachers’ 3D printing integration features, teachers’ lesson plans

were analyzed. The lesson plans were coded for 3D printing integration levels and

STEM integration levels. A few professors in educational technology reviewed the

codebooks and provided suggestions, and revisions were made accordingly, which

ensured the validity of the codebooks. To ensure scoring reliability, I scored all the

lesson plans using the codebooks and a week later I scored all the lesson plans again

to check the percentage of the consistency. There was about 92.3% consistency

between the scoring. The only 2 lesson plans that I scored differently in the previous

and later scoring were due to that I did not find some specific information when I scored

them the first time. The discrepancies were reviewed, and corrections were made

accordingly.

97

Correlational analysis

The reliability of the survey was evaluated with Cronbach’s alpha. Pearson’s

correlation was conducted to examine the directionality and degree of association

between teacher beliefs, 3D printing integration levels, and STEM integration levels.

Thematic analysis

Thematic analysis was conducted to analyze teachers’ responses to the open-

ended questions in the teacher beliefs survey. According to Braun and Clarke (2006),

thematic analysis is “a method for identifying, analysing and reporting patterns (themes)

within data” (p. 79). The analysis of the open responses followed the six phases of

thematic analysis (Table 3-6) developed by Braun and Clarke (2006).

To ensure reliability of the coding and themes searching and reviewing, after one

week of the original coding, I selected a random sample of one-third of the scripts to

code and search themes again. The coding and themes were compared to the original

coding and themes, and there were few discrepancies except using some different

expressions which had the same meanings, for instance, lack of time and insufficient of

time. The responses for each question were analyzed separately, then the themes for

each question were synthesized across the questions.

Table 3-6. Phases of thematic analysis (from Braun & Clarke, 2006)

Phase Description of the process

1. Familiarizing yourself with your data:

Transcribing data (if necessary), reading and re-reading the data, noting down initial ideas.

2. Generating initial codes:

Coding interesting features of the data in a systematic fashion across the entire data set, collating data relevant to each code.

3. Searching for themes: Collating codes into potential themes, gathering all data relevant to each potential theme.

98


Phase Description of the process

4. Reviewing themes: Checking if the themes work in relation to the coded extracts (Level 1) and the entire data set (Level 2), generating a thematic ‘map’ of the analysis.

5. Defining and naming themes:

Ongoing analysis to refine the specifics of each theme, and the overall story the analysis tells, generating clear definitions and names for each theme.

6. Producing the report: The final opportunity for analysis. Selection of vivid, compelling extract examples, final analysis of selected extracts, relating back of the analysis to the research question and literature, producing a scholarly report of the analysis.

Data Analysis for RQ2

To address research question 2, the S-STEM survey data were analyzed, and

teacher beliefs survey data and lesson plans were already analyzed when addressing

research question 1. Descriptive statistical analyses were conducted using the S-STEM

survey data, then multilevel modeling analyses were conducted using the S-STEM

survey data including students’ science motivation, technology/engineering motivation,

math motivation, and 21st century skills, teacher beliefs survey data, and lesson plan

data. Multiple regression analysis was conducted for students’ interest in STEM careers

because it was found that multilevel modeling analysis did not work for it.

Descriptive statistical analysis

Descriptive statistical analyses were conducted with the pretest and posttest data

of STEM motivation, STEM career interests, and 21st century skills to provide the mean

and standard deviation for each construct. As STEM motivation and 21st century skills

were rated with 5-point Likert scales, an aggregated average score was calculated for

99

science motivation, math motivation, engineering/technology motivation, and 21st

century skills for each student.

As suggested by Wiebe, Unfried, and Faber (2018), students’ understanding of

specific occupations is not very likely to have matured in the younger grades and

students study STEM subject areas in school but not necessarily study specific STEM

careers in school. Therefore, an aggregated average score of students’ ratings on the

12 STEM career interest items was calculated for the pre-survey and post-survey and

this average score represented a student’s interest in STEM careers.

Multilevel modeling analysis

This study involved two levels of factors: level-1 factors, i.e., student-level

factors, and level-2 factors, i.e., teacher-level factors. Student-level factors included

student gender and the pretest scores of STEM motivation, interest in STEM careers,

and 21st century skills. Teacher-level factors included teachers’ 3D printing integration

levels and STEM integration levels. The response variables were students’ posttest

scores on STEM motivation, 21st century skills, and interest in STEM careers. Multiple

regression analysis with just the student-level factors or the teacher-level factors would

omit the variances of the other level of factors that were not included in analysis. Since

there were two levels of factors that were of interest in this study, a more advanced

regression analysis, i.e. multilevel modeling analysis, would be a necessary approach to

address the potential influence of the two levels of factors (Peugh, 2010). As there were

a few response variables in this study, a multilevel modeling analysis was independently

conducted with each response variable and its corresponding student-level factors, for

instance, students’ science motivation posttest score as the response variable, and

student gender and science motivation pretest score as the student-level factor. All the

100

teacher-level factors were included in the multilevel modeling analysis. The software

program SAS 9.4 was used for all the multilevel modeling analyses.

The multilevel modeling analyses of this study were guided by the major steps

proposed by Peugh (2010): clarifying the research question under investigation,

choosing the correct parameter estimation method (i.e., full information or restricted

maximum likelihood), assessing whether multilevel modeling is needed, building the

level-1 model, building the level-2 model, comparing nested models using likelihood

ratio test, and reporting multilevel effect sizes).

Clarify the research question. Research question 2 is: How do teacher beliefs

and their 3D printing integration in the science classrooms predict students’ STEM

motivation, 21st century skills, and interest in STEM careers? The primary interest was

to examine the relationship between teacher-level factors and student outcome

variables. As students were nested within teachers, the student-level factors (student

gender and pretest scores) were treated as covariates to be controlled when analyzing

the relationship between teacher-level factors and student outcome variables.

Choose parameter estimation method. After clarifying the research question,

the next step is to choose the correct parameter estimation method. Due to the small

sample size at the teacher level (N < 50), the restricted maximum likelihood (REML)

estimation was adopted because REML can estimate variances more accurately than

full information maximum likelihood (FIML) estimation when the higher-level sample size

is small (Peugh, 2010).

Assess whether multilevel modeling is needed. The next step is to assess

whether multilevel modeling was needed by building the baseline model since nested

101

datasets may not require multilevel modeling if there is no response variable variation at

level-2 (Peugh, 2010), which basically meant that the level-2 factors have no impact on

the response variable and the level-1 units can be treated as independent observations

using OLS multiple regression (Peugh, 2010). Therefore, whether multilevel modeling is

needed depended on how much response variable variation was in level 2 (namely the

teacher level), which involved calculating the intraclass correlation (ICC) and the design

effect statistics (Peugh, 2010). If there is variation in the mean response variable scores

(e.g., student science motivation posttest scores) across teachers, multilevel modeling

is needed to separately estimate the variance of response variable scores that occurs at

both across students and across teachers. To test the variance in the mean response

variable scores, the next step is to build an unconditional model, namely a baseline

model that examines the variation in mean response variable scores without including

the student-level and teacher-level factors yet.

The baseline model is shown by the following equations (Hox, 2002;

Raudenbush & Bryk, 2002, as cited in Peugh, 2010):

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝑖𝑗 (3-1)

Teacher Level: 𝛽0𝑗 = 𝛾00 + 𝑢0𝑗 (3-2)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝑢0𝑗 + 𝑖𝑗 (3-3)

To make it easier to illustrate the multilevel modeling analysis procedure, the

following analysis focuses on a specific response variable – students’ science

motivation posttest scores. The analysis procedures apply to the other response

variables. The meanings of the symbols in equations (3-1), (3-2), and (3-3) are provided

in Table 3-7.

102

Table 3-7. The meanings of symbols in equations (3-1), (3-2), and (3-3)

Symbol Meaning

𝑌𝑖𝑗 The science motivation posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean science motivation posttest score for teacher j.

𝑖𝑗 A residual term – individual student differences around the mean of teacher j.

𝛾00 Grand-mean science motivation posttest score across all teachers.

𝑢0𝑗 Deviation/difference between the mean score for each teacher and the grand mean.

ICC is the proportion of student science motivation posttest score variance that

can be explained by the mean student science motivation posttest score differences

across teachers.

𝐼𝐶𝐶 = 𝜎𝑢02 / (𝜎𝑢0

2 + 𝜎𝜀2) (3-4)

where 𝜎𝑢02 is the variance of 𝑢0𝑗 and 𝜎𝜀

2 is the variance of 𝑖𝑗.

ICC values between 0.05 and 0.20 are common in multilevel modeling analysis

applications in social research studies (Peugh, 2010).

The design effect is “an estimate of the multiplier that needs to be applied to

standard errors to correct for the negative bias that results from nested data” (Peugh,

2010, p. 91) and the equation is:

Design Effect = 1 + (𝑛𝑐 - 1) ICC (3-5)

where 𝑛𝑐 is the average number of students per teacher.

103

A design effect estimates greater than 2.0 indicates a need for multilevel

modeling analysis (Muthén, 1991, 1994; Muthén & Satorra, 1989, 1995, as cited in

Peugh, 2010).

If the ICC values are lower than 0.05 and the design effect estimate is less than

2.0, multiple regression analysis will be conducted to examine research question 2. If

the ICC value for the response variable is greater than 0.05 and/or the design effect

estimate is greater than 2.0, the multilevel modeling analysis will be continued with

further procedures and the next step is to build the level-1 model.

Build the student-level model. The student-level model adds students’ science

motivation pretest scores into the model. As predictor variables were measured with

scales that did not contain zero, a score of zero would have no substantive meaning,

therefore, the method of centering which involves rescaling a predictor variable is

necessary to make a value of zero can be interpreted meaningfully (Peugh, 2010). A

grand mean centering is adopted because students’ pretest scores were independent of

each other’s scores. The student-level model starts with a random-intercept model

which estimates the impact of students’ pretest scores on the posttest scores as a fixed

effect, indicating the impact of students’ pretest scores on posttest scores did not vary

across teachers. The equations for the random-intercept model with grand-mean

centering are as follows:

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑅𝐸𝑖𝑗 − 𝑃𝑅𝐸) + 𝛽2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑖𝑗 (3-6)


𝛽1𝑗 = 𝛾10 (3-8)

𝛽2𝑗 = 𝛾20 (3-9)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑅𝐸𝑖𝑗 − 𝑃𝑅𝐸) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑢0𝑗

+ 𝑖𝑗

(3-10)

104

where 𝛽1𝑗 is the regression coefficient that shows the impact of pretest scores on

posttest scores across all students of teacher j , 𝛽2𝑗 is the regression coefficient that

shows the impact of student gender on posttest scores across all students of teacher j

𝑃𝑅𝐸𝑖𝑗 is pretest score of student i of teacher j, 𝑃𝑅𝐸 is the grand mean of science

motivation pretest scores across teachers, 𝛾10 is the average effect of pretest scores on

posttest scores across all teachers, and 𝛾20 is the average effect of student gender on

posttest scores across all teachers.

However, if the impacts of student gender and students’ pretest scores on

posttest scores vary significantly across teachers, variance components (i.e., random

effects) would need to be added to the teacher-level slope equation to model this

variation (Peugh, 2010). The teacher level equations for the random-slope model are as

follows:


𝛽1𝑗 = 𝛾10 + 𝑢1𝑗 (3-12)

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (3-13)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑅𝐸𝑖𝑗 − 𝑃𝑅𝐸) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 +

𝑢1𝑗 (𝑃𝑅𝐸𝑖𝑗 − 𝑃𝑅𝐸) + 𝑢2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑢0𝑗 + 𝑖𝑗

(3-14)

where 𝑢1𝑗 is a residual term to show a random effect, indicating the impact of

pretest score on posttest score can vary randomly across teachers, and 𝑢2𝑗 is a residual

term to show a random effect, indicating the impact of student gender on posttest score

can vary randomly across teachers. Multilevel modeling analysis does not estimate the

residuals, but the variance of the residuals (Peugh, 2010).

Before proceeding to the next model during each step, a likelihood ratio test

needs to be conducted to examine whether the more complicated model fits better than

105

the simpler model. If the more complicated model fits better, this model is continually

built on with more parameters. Otherwise, the simpler model will be kept.

Build the teacher-level model. The teacher-level model adds the teacher-level

factors into the student-level model. Teacher level factors include 3D printing integration

level, STEM integration level, and teacher beliefs, which consisted of pedagogical

beliefs, technology value beliefs, and self-efficacy beliefs in 3D printing integration, and

the value beliefs and self-efficacy beliefs each had a few subcomponents. Before

adding these variables into the teacher-level model, the multicollinearity between the

variables had to be assessed. Variables that have high multicollinearity would not be

added into the teacher-level model. The equations for each of the student outcome

variables might be different depending on which teacher-level variables would be

added, therefore, the specific equations for the teacher-level model are presented in the

results section along with the specific analysis steps. This teacher-level model is also

considered as a full model with both student-level and teacher-level variables.

Report multilevel effect sizes. In addition to the regression coefficients of the

student-level and teacher-level factors, the effect sizes at both levels will be calculated

and reported. The effect size at student-level can be obtained by comparing the

random-intercept model and the baseline model.

𝑅𝐿12 =

𝜎𝜀 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒2 − 𝜎𝜀 𝑟𝑎𝑛𝑑𝑜𝑚−𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡

2

𝜎𝜀 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒2

(3-15)

𝑅𝐿12 measures how much student-level variance is explained by the random-

intercept model compared to the baseline model.

The effect size at teacher-level can be obtained by comparing the full model and

the random-slope model if the random-slope model fits better than the random-intercept

106

model. Otherwise, the effect size would be obtained by comparing the full model and

the random-intercept model.

𝑅𝐿22 =

𝜎𝜀 𝑟𝑎𝑛𝑑𝑜𝑚−𝑠𝑙𝑜𝑝𝑒2 − 𝜎𝜀 𝑓𝑢𝑙𝑙

2

𝜎𝜀 𝑟𝑎𝑛𝑑𝑜𝑚−𝑠𝑙𝑜𝑝𝑒2

(3-16)

𝑅𝐿22 measures how much teacher-level variance is explained by the full model

compared to the random-slope model, namely, how much variance is explained by the

teacher-level variables.

Multiple regression analysis

After a few initial steps with multilevel modeling analysis, it was found that the

nested structure of students within teachers did not contribute to explaining the

variances in teacher-level variables for students’ interest in STEM careers. Therefore,

multiple regression analysis was conducted by treating students as independent of each

other and all the student and teacher variables were at the same level without nesting.

The dependent variable was the posttest scores of students’ interest in STEM

careers. The independent variables were student gender, pretest scores of students’

interest in STEM careers, teachers’ 3D printing integration levels, STEM integration

levels, each component of the teacher beliefs variables, and interactions between the

student variables and teacher variables. Except for student gender, all the other

variables were centered with their grand mean. The assumptions of multiple regression

analysis were examined, including normality assumption of the dependent variable, no

multicollinearity, normal distribution of residuals, and no autocorrelation in the residuals.

SAS 9.4 was used for the multiple regression analysis. The specific procedures of the

data analysis with the results have been reported in Chapter 4.

107

CHAPTER 4 RESULTS

This chapter is organized by presenting the descriptive statistics of the variables

included in the correlational analysis, multilevel modeling analysis, and multiple

regression analysis; the internal consistency of the rating scales; correlations between

the variables; missing data evaluation; assumptions testing; and then the multilevel

modeling analysis results for each of the dependent variables: students’ science

motivation, technology/engineering motivation, math motivation, and 21st century skills;

and the multiple regression analysis results for students’ interest in STEM careers.

Descriptive Statistics of Variables

The variables included in the analyses consisted of dependent variables,

student-level independent variables, and teacher-level independent variables. The

dependent variables included students’ posttest scores in science motivation,

technology/engineering motivation, math motivation, 21st century skills, and interest in

STEM careers. The student-level independent variables included student gender and

students’ pretest scores in math, science, and technology/engineering motivation, 21st

century skills, and interest in STEM careers. The teacher-level independent variables

included teachers’ 3D printing integration levels, STEM integration levels, pedagogical

beliefs, 3D printing value beliefs (interest in 3D printing integration, perceived

importance of 3D printing integration, perceived usefulness of 3D printing integration),

and self-efficacy beliefs in 3D printing integration in science classrooms (self-efficacy in

TK, PK, CK, TPK, TCK, PCK, and TPACK). The variable names and their meanings are

presented in Table 4-1.

108

Table 4-1. Variable names and their meanings Variable Categories

Variable Names Variable Meanings

Dependent variables

Post_Science Science motivation posttest scores

Post_TechEngi Technology/Engineering motivation posttest scores

Post_Math Math motivation posttest scores

Post_21st 21st century skills posttest scores

Post_Career Interests in STEM careers posttest scores

Student-level independent variables

Pre_Math Math motivation pretest scores

Pre _Science Science motivation pretest scores

Pre _TechEngi Technology/Engineering motivation pretest scores

Pre _21st 21st century skills pretest scores

Pre_Career Interests in STEM careers pretest scores

Gender_Student Student gender

Teacher-level independent variables

Printing_Level 3D printing integration levels

STEM_Level STEM integration levels

Pedagogical_Beliefs Teachers’ pedagogical beliefs

Interest_Teacher Teachers’ intrinsic interest value in 3D printing integration

Importance_Teacher Teachers’ attainment value (perceived importance) of 3D printing integration

Usefulness_Teacher Teachers’ extrinsic utility value (perceived usefulness) of 3D printing integration

Self_Efficacy_TK Teachers’ self-efficacy in Technological Knowledge

Self_Efficacy_PK Teachers’ self-efficacy in Pedagogical Knowledge

Self_Efficacy_CK Teachers’ self-efficacy in Content Knowledge

Self_Efficacy_TPK Teachers’ self-efficacy in Technological Pedagogical Knowledge

Self_Efficacy_TCK Teachers’ self-efficacy in Technological Content Knowledge

Self_Efficacy_PCK Teachers’ self-efficacy in Pedagogical Content Knowledge

Self_Efficacy_TPACK Teachers’ self-efficacy in Technological Pedagogical Content Knowledge

109

Dependent Variables

Students’ posttest scores in science motivation, technology/engineering

motivation, math motivation, 21st century skills, and interest in STEM careers were the

average scores of the corresponding items in the subscales of the S-STEM survey. The

number of students who completed the subscales, the mean, standard deviation, the

minimum score, and the maximum score for all the students’ posttest scores can be

viewed in Table 4-2. Science motivation, technology/engineering motivation, math

motivation, and 21st century skills were measured with 5-point Likert scales and interest

in STEM careers were measured with a 4-point Likert scale. For the posttest scores of

all the students, the posttest scores in science motivation, technology/engineering

motivation, math motivation, and 21st century skills ranged from 1 to 5 and the posttest

scores of students’ interest in STEM careers ranged from 1 to 4.

Table 4-2. Descriptive statistics for dependent variables

Variable N M SD Min Max

Post_Science 1492 3.5747 0.7126 1.0000 5.0000

Post_TechEngi 1485 3.4941 0.7364 1.0000 5.0000

Post_Math 1501 3.5601 0.8760 1.0000 5.0000

Post_21st 1482 4.0748 0.5522 1.0000 5.0000

Post_Career 1476 2.4230 0.5124 1.0000 4.0000

Student-Level Independent Variables

Student gender was included in the multilevel modeling analysis. There were 712

males and 789 females. Consistent with how students’ posttest scores were calculated,

students’ pretest scores in science motivation, technology/engineering motivation, math

motivation, 21st century skills, and interest in STEM careers were the average scores of

110

the corresponding items in the subscales of the S-STEM survey. The number of

students who completed the subscales, the mean, standard deviation, the minimum

score, and the maximum score for all the students’ pretest scores are in Table 4-3.

Table 4-3. Descriptive statistics for student-level independent variables

Variable N M SD Min Max

Gender_Student 0 = Male: N = 712; 1 = Female: N = 789.

/ / / /

Pre _Science 1497 3.5761 0.6877 1.2222 5.0000

Pre _TechEngi 1490 3.5356 0.6911 1.0000 5.0000

Pre_Math 1501 3.5396 0.8448 1.0000 5.0000

Pre _21st 1482 4.0926 0.5009 1.4545 5.0000

Pre_Career 1475 2.4582 0.5139 1.0000 4.0000

Teacher-Level Independent Variables

Teachers’ 3D printing integration levels and STEM integration levels were rated

with their lesson plans. Teachers’ 3D printing integration levels ranged from 2 to 5, with

the mean of 3.6136 and standard deviation of 1.0411. Teachers’ STEM integration level

ranged from 2 to 4, with the mean of 3.2352 and standard deviation of 0.6831.

Among the 26 teachers whose lesson plans were analyzed, 24 teachers

responded to the survey on their pedagogical beliefs and value beliefs and self-efficacy

beliefs in 3D printing integration in science classrooms. Teachers’ pedagogical beliefs

ranged from 2.8000 to 4.4667, with the mean of 3.5577 and standard deviation of

0.4654. Teachers’ value beliefs consisted of interest, perceived importance, and

perceived usefulness of 3D printing integration in science classrooms. On average,

teachers scored high on the three subscales of the value beliefs. The average scores of

111

all the teachers’ self-efficacy beliefs varied from 3.5123 in self-efficacy in TK to 4.4966

in self-efficacy in PK. The mean, standard deviation, minimum score, and maximum

score of all the teacher-level independent variables can be viewed in Table 4-4.

Table 4-4. Descriptive statistics for teacher-level independent variables

Variable Teacher N M SD Min Max

Printing_Level 26 3.6136 1.0411 2.0000 5.0000

STEM_Level 26 3.2352 0.6831 2.0000 4.0000

Pedagogical_Beliefs 24 3.5577 0.4654 2.8000 4.4667

Interest_Teacher 24 4.3805 0.5389 3.5000 5.0000

Importance_Teacher 24 3.9764 0.5720 3.0000 5.0000

Usefulness_Teacher 24 4.4395 0.4678 3.5000 5.0000

Self_Efficacy_TK 24 3.5123 0.6985 2.2000 5.0000

Self_Efficacy_PK 24 4.4966 0.4378 3.4286 5.0000

Self_Efficacy_CK 24 4.2085 0.5139 2.7500 5.0000

Self_Efficacy_TPK 24 4.0540 0.4945 3.0000 5.0000

Self_Efficacy_TCK 24 4.0068 0.5182 3.0000 5.0000

Self_Efficacy_PCK 24 4.3256 0.5845 3.0000 5.0000

Self_Efficacy_TPACK 24 3.9263 0.4754 3.0000 5.0000

Internal Consistency

The internal consistency of the value beliefs scales, pedagogical beliefs scales,

self-efficacy beliefs scales, and S-STEM survey were assessed with Cronbach’s alpha

(see Table 4-5). According to commonly accepted rules, a Cronbach’s alpha of above .8

is good, between .7 and .8 is acceptable, and between .6 and .7 is questionable. The

intrinsic interest value and attainment value/importance had acceptable internal

112

consistency with Cronbach’s alpha of .776 and .803 respectively. The internal

consistency of extrinsic utility value/usefulness approached the acceptable level with

Cronbach’s alpha of .667, and there was a small number of items in this scale, so it was

considered as acceptable. The Cronbach’s alpha of the overall value beliefs scale was

.869. The Cronbach’s alpha of pedagogical beliefs scale was .727, which was

acceptable. In terms of the internal consistency of self-efficacy beliefs scales, the

Cronbach’s alpha of self-efficacy in TK, PK, CK, TPK, and TPACK were .844, .878,

.652, .841, and .607 respectively. The Cronbach’s alpha of the overall self-efficacy

beliefs scale was .909. As TCK and PCK only had one item in the scale, the Cronbach’s

alpha was not available. The Cronbach’s alpha of self-efficacy in CK and TPACK were

questionable, but they were considered as acceptable in this study due to the small

number of items in the scale. The Cronbach’s alpha of the S-STEM survey including the

pretest and posttest of students’ math motivation, science motivation,

technology/engineering motivation, 21st century skills, and interest in STEM careers

ranged from .801 to .916. All these Cronbach’s alphas were above .800, indicating good

internal consistency of the scales.

Table 4-5. Cronbach’s alpha of rating scales

Scale Cronbach’s Alpha

Intrinsic interest value (2 items) .776

Attainment value/importance (3 items) .803

Extrinsic utility value/usefulness (2 items) .667

Overall value beliefs (7 items) .869

Pedagogical beliefs (15 items) .727

113


Scale Cronbach’s Alpha

Self-efficacy in TK (5 items) .844

Self-efficacy in PK (7 items) .878

Self-efficacy in CK (4 items) .652

Self-efficacy in TPK (5 items) .841

Self-efficacy in TCK (1item) /

Self-efficacy in PCK (1 item) /

Self-efficacy in TPACK (3 items) .607

Overall self-efficacy beliefs (26 items) .909

Math motivation (8 items) .908 (pre); .916 (post)

Science motivation (9 items) .881 (pre); .892 (post)

Technology/Engineering motivation (9 items) .866 (pre); .890 (post)

21st century skills (11 items) .866 (pre); .902 (post)

Interest in STEM careers (12 items) .801 (pre); .804 (post)

Correlations between Variables

The correlations between the dependent variables were significant and ranged

from .190 to .542, and only one correlation was slightly above .500. Because the

correlations between the outcome variables were low to moderate and there were no

high correlations, the multilevel models were fit separately for each outcome variable,

and = .05 was used as the significance level. The correlation coefficients can be

viewed in Table 4-6.

114

Table 4-6. Correlations between dependent variables

Post_Science Post_TechEngi Post_Math Post_21st Post_Career

Post_Science 1

Post_TechEngi .366** 1

Post_Math .291** .270** 1

Post_21st .360** .269** .321** 1

Post_Career .478** .542** .258** .190** 1

**. Correlation is significant at the 0.01 level (2-tailed).

The correlations between the subscales of the value beliefs survey were all

significant (see Table 4-7). The correlations between intrinsic interest value and

attainment value/importance and extrinsic utility value/usefulness were .546 and .460

respectively. The correlation between attainment value/importance and extrinsic utility

value/usefulness was .784. Since there was a high correlation between the attainment

value/importance and extrinsic utility value/usefulness, the collinearity between the two

variables was assessed to decide whether they both should be kept in the model when

conducting multilevel modeling analysis.

Table 4-7. Correlations between value beliefs subscales

Interest_Teacher Importance_Teacher Usefulness_Teacher

Interest_Teacher 1

Importance_Teacher .546** 1

Usefulness_Teacher .460* .784** 1

**. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed).

115

The correlations between self-efficacy beliefs subscales (see Table 4-8) were all

positive and ranged from .031 to .706. The correlations between TK and TPK, TK and

TPACK, CK and PK, PK and PCK, PK and TCK, PK and TPACK, PCK and TCK, TCK

and TPK, TCK and TPACK, and TPK and TPACK were all significant with low to

moderate correlation coefficients. The correlations between TK and TPACK, PK and

PCK, and TPK and TPACK were relatively high compared to other correlations between

the subscales. When fitting the multilevel models with the variables of the subscales,

multicollinearity was assessed to determine whether some of the variables should be

removed from the models.

Table 4-8. Correlations between self-efficacy beliefs subscales

TK CK PK PCK TCK TPK TPACK

TK 1

CK .298 1

PK .393 .454* 1

PCK .282 .326 .687** 1

TCK .351 .031 .519** .468* 1

TPK .540** .382 .366 .165 .408* 1

TPACK .617** .242 .494* .395 .478* .706** 1


There were no significant correlations between pedagogical beliefs and any of

the teacher value beliefs or self-efficacy beliefs (see Table 4-9). Teachers’ intrinsic

interest value significantly and positively correlated with teachers’ self-efficacy beliefs in

TPK, TCK, and TPACK with correlation coefficients of .438, .487, and .687 respectively.

116

Teachers’ perceived importance of 3D printing integration significantly and positively

correlated with teachers’ self-efficacy in TPACK with r = .565. Teachers’ perceived

usefulness of 3D printing integration had no significant correlations with teachers’

pedagogical beliefs or self-efficacy beliefs.

Table 4-9. Correlations between teacher beliefs

Pedagogical_Beliefs

Interest _Teacher

Importance _Teacher

Usefulness _Teacher

Pedagogical _Beliefs 1

Interest _Teacher .049 1

Importance _Teacher -.039 .546** 1

Usefulness _Teacher -.066 .460* .784** 1

Self_Efficacy_PK .287 .338 .004 -.103

Self_Efficacy_CK -.105 -.141 -.205 -.402

Self_Efficacy_TPK -.086 .438* .330 .194

Self_Efficacy_TCK .308 .487* .115 .088

Self_Efficacy_PCK .339 .210 .020 -.152

Self_Efficacy_TPACK .013 .687** .565** .373


After reporting the fundamental statistics, the next step is to report the results for

the two research questions:

How are teachers’ beliefs correlated with their 3D printing integration in the science classrooms?

How do teachers’ beliefs and their 3D printing integration in the science classrooms predict students’ STEM motivation, 21st century skills, and interest in STEM careers?

117

Results for RQ1

Correlations between Teacher Beliefs and 3D Printing Integration

The correlations between 3D printing integration levels, STEM integration levels,

and teachers’ pedagogical beliefs, value beliefs, and self-efficacy beliefs are presented

in Table 4-10. The correlation between 3D printing integration levels and STEM

integration levels was significant with r = .673, a moderate correlation. The correlation

between teachers’ self-efficacy beliefs in PCK was negative and significant with r = -

.457, a low-to-moderate correlation. All the other correlations were nonsignificant.

Table 4-10. Correlations between Printing_Level, STEM_Level, and teacher beliefs

Printing_Level STEM_Level

Printing_Level 1

STEM_Level .673** 1

Pedagogical_Beliefs -.011 -.085

Interest_Teacher -.259 -.107

Importance_Teacher -.274 .031

Usefulness_Teacher -.040 .292

Self_efficacy_TK .006 -.122

Self_efficacy_PK -.170 -.353

Self_efficacy_CK .136 -.033

Self_efficacy_TPK -.352 -.344

Self_efficacy_TCK -.156 -.265

Self_efficacy_PCK -.135 -.457*

Self_efficacy_TPACK -.152 -.127


Open Responses in Teacher Beliefs Survey

Four open-ended questions were included in the teacher beliefs survey to obtain

some detailed data to provide specific information on teachers’ experience and

118

perceptions on their 3D printing integration, which may potentially explain the

relationship between teacher beliefs and their 3D printing integration. The four

questions were: 1. How do you feel about integrating 3D printing technology into your

science teaching? 2. Do you think you have sufficient knowledge and skills to use 3D

printing technology in your science classes? Please explain. 3. What are the biggest

advantages of integrating 3D printing technology in science teaching? 4. What are the

biggest challenges in integrating 3D printing technology in science teaching?

The responses of each question were coded and then the themes were

generated with the codes. Afterwards, related themes were synthesized across the

questions. The analysis of the first question yielded three themes: teachers’ and

students’ positive attitude towards 3D printing, benefits for students, and challenges of

3D printing integration. The teachers felt integrating 3D printing into their science

classrooms was great, beneficial, fun, exciting, and amazing, and they also indicated

that students loved the 3D printing integrated activities and felt amazed at the 3D

printed objects. The teachers felt 3D printing integration was beneficial for students,

including engaging students by allowing students to hold and visualize objects,

promoting hands-on learning, and enhancing students’ cognitive learning. However, the

teachers mentioned that they encountered a few challenges, including the lack of 3D

printers, technical issues, the lack of time to print 3D objects, the lack of time to

implement lessons, difficulty in using 3D printers, difficult in connecting 3D printing to

curriculum standards, and difficulty in adapting to students’ different abilities.

For the second question, most teachers thought they had sufficient knowledge

and skills. A few teachers thought they had sufficient knowledge and skills but had

119

some challenges including the lack of 3D printers, technical support, ability to print 3D

objects, time to learn how to print 3D objects, and the difficulty in connecting 3D printing

to curriculum standards and creating lessons. Moreover, several teachers thought they

did not have sufficient knowledge and skills in integrating 3D printing in their science

classrooms.

For the third question, the teachers indicated a few benefits of 3D printing

integration for teaching and learning: convenience for learning with access to objects;

enabling hands-on learning and engage students; stimulating student interest and

motivation; enhancing students’ cognitive learning including science knowledge,

scientific investigation, shifting science thinking and learning, understanding of science

learning content, and creativity; and making connections with other disciplines such as

science and math.

The last question provided information on the challenges that teachers

encountered when integrating 3D printing into their science classrooms, including

logistical and technical issues, insufficient 3D printers and related resources, and the

lack of time, including the time to print 3D objects, the time to plan, develop, and

integrate 3D printing into curriculum, and the time to teach students how to use 3D

printing. According to Ertmer et al. (2012), these challenges were external barriers for

teachers. Additionally, the teachers had some internal barriers, including the lack of

ability to print 3D objects and connect 3D printing to curriculum standards, difficulty in

teaching and making sure all students were able to use the 3D printing software, and

also difficulty in teaching students who were not enthusiastic, motivated, or having

limited ability.

120

After analyzing the themes of the teacher responses for each question, it was

found that there were some common themes across the questions. The responses of

question 1 and question 3 both had themes regarding the benefits of 3D printing

integration in science classrooms. The responses of question 1, 2, and 4 all had themes

regarding the challenges of 3D printing integration in science classrooms. The common

themes were synthesized across the questions and finally there were five themes with

some sub-themes for all the teacher responses: teachers’ and students’ attitude

towards 3D printing integration; teachers’ self-efficacy in their knowledge and skills in

3D printing integration in science classrooms; teachers’ value beliefs in 3D printing

integration, i.e., 3D printing integration was beneficial for teaching and learning;

teachers’ external barriers to integrating 3D printing technology in their science

classrooms; and teachers’ internal barriers to integrating 3D printing technology in their

science classrooms. All the sub-themes have been illustrated earlier on. The themes,

sub-themes, and the frequency of the sub-themes can be viewed in Table 4-11.

Table 4-11. Thematic analysis results of teachers’ open responses

Themes Sub-themes Frequency

Teachers’ and students’ attitude towards 3D printing integration

Students’ positive attitude perceived by teachers: Teachers felt that students loved 3D printing and were amazed at the 3D printed objects.

2

Teachers’ positive attitude: Teachers felt 3D printing integration in the science classrooms were great, beneficial, fun, exciting, and amazing.

14

Teachers’ self-efficacy in their knowledge and skills in 3D printing integration in science classrooms

High self-efficacy: Teachers thought they had sufficient knowledge and skills.

13

Moderate self-efficacy: Teachers thought they had some knowledge and skills but had some challenges.

7

Low self-efficacy: Teachers thought they did not have sufficient knowledge and skills.

4

121


Themes Sub-themes Frequency

Teachers’ value beliefs in 3D printing integration: 3D printing integration was beneficial for teaching and learning

Convenience for learning with access to objects.

15

Enable hands-on learning and engage students by allowing students to hold and visualize the 3D printed objects.

25

Stimulate student interest and motivation. 4

Enhance students’ cognitive learning including science knowledge, scientific investigation, shifting science thinking and learning, understanding of science learning content, and creativity.

11

Make connections with other disciplines such as science and math.

2

Teachers’ external barriers to integrating 3D printing technology

Logistical and technical issues. 8

Insufficient 3D printers and related resources. 10

Lack of time to print 3D objects. 9

Lack of time to plan, develop, and integrate 3D printing into curriculum.

9

Teachers’ internal barriers to integrating 3D printing technology

Lack of ability to print 3D objects and connect 3D printing to curriculum standards.

10

Difficulty in teaching students who were not enthusiastic, motivated, or having limited ability.

3

Results for RQ2

Missing Data Evaluation

In this study, there was a very small portion of missing data for student-level

variables and teacher-level variables. The proportions of missing data of student-level

variables were all less than 2 percent (see Table 4-12). There were 26 teachers in total

and 24 teachers completed the survey for their pedagogical beliefs, value beliefs, and

self-efficacy beliefs. The proportion of missing data of all the teacher-level variables was

7.6923%. The missing data were assumed as missing completely at random (MCAR)

122

since the proportions of missing data were small and the probabilities of missing data of

the variables were unrelated to the observed variables and the variable with missing

data per se. Therefore, full information maximum likelihood (FIML) was used to deal

with the missing data by using all the available data. The software program SAS 9.4

automatically uses FIML to deal with missing data.

Table 4-12. Proportion of missing values for student-level variables

Variable Total N of Students

N of Responses N of Missingness

Proportion of Missingness

Gender_Student 1501 1501 0 0

Pre _Science 1501 1497 4 0.2664%

Pre _TechEngi 1501 1490 11 0.7328%

Pre_Math 1501 1501 0 0

Pre _21st 1501 1482 19 1.2658%

Pre_Career 1501 1475 26 1.7322%

Post_Science 1501 1492 9 0.5996%

Post_TechEngi 1501 1485 16 1.0660%

Post_Math 1501 1501 0 0

Post_21st 1501 1482 19 1.2658%

Post_Career 1501 1476 25 1.6656%

Assumptions Testing

There are several assumptions for conducting multilevel modeling analysis. First,

the dependent variable has to be normally distributed. Second, there is no

multicollinearity between the independent variables. Third, the residuals of the models

have to be normally distributed. In this study, the assumptions of the multilevel models

were checked along with the analysis process. The normality of the dependent variable

123

was examined before the models were built. All the dependent variables in this study

including science motivation posttest scores, technology/engineering posttest scores,

math motivation posttest scores, 21st century skills, and interest in STEM careers met

with the normality assumption. The skewness of each outcome variable was well

between -1 and 1, and the kurtosis of each outcome variable was well between -3 and

3. The skewness and kurtosis statistics can be viewed in Table 4-13. The

multicollinearity was examined when teacher-level variables were added into the model

for each dependent variable. The residual normality was examined after the best model

was identified for each dependent variable. The results of the multicollinearity and

residual normality assumptions testing will be reported later.

Table 4-13. Skewness and kurtosis of dependent variables

Variables Skewness Kurtosis

POST_Science -0.1149 0.1147

POST_TechEngi -0.2659 0.2747

POST_Math -0.4892 -0.2634

POST_21st -0.5046 1.0369

POST_Career -0.3245 0.7069

Results for Science Motivation

Multilevel modeling analyses were conducted to examine how teachers’ 3D

printing integration levels, STEM integration levels, pedagogical beliefs, value beliefs,

and self-efficacy beliefs in 3D printing integration predicted students’ STEM motivation,

21st century skills, and interest in STEM careers respectively while controlling for

student gender and students’ pretest scores.

124

The total number of students was 1,501. The student sample size for all the 26

teachers was unbalanced and varied from 20 to 103, with an average of 𝑛𝑐 = 57.7308,

SD = 25.6976, Median = 54. Because the student sample size for each teacher was

unbalanced, and the teacher-level sample size was small (N = 26), Kenward-Roger

adjustment (abbreviated as kr) (Kenward & Roger, 2009) was used in the SAS program

to adjust for degrees of freedom.

A series of multilevel models were built to examine how teachers’ 3D printing

integration levels, STEM integration levels, pedagogical beliefs, value beliefs, and self-

efficacy beliefs in 3D printing integration predict students’ science motivation while

controlling for student gender and science motivation pretest scores. Student gender

and science motivation pretest scores were included in the model as covariates to be

controlled while interpreting the intercept and the coefficients of other variables.

Baseline model

The first step was to build the baseline model, which predicted a student’ science

motivation posttest score from the grand mean science motivation posttest score of all

the teachers’ students. There were no student-level or teacher-level predictors in this

model. The purpose of this model was to examine the ICC and design effect to

determine whether multilevel models were necessary for the analysis. The meanings of

symbols in the model are provided in Table 4-14.

The baseline model is shown by the following equations:




125

Table 4-14. The meaning of symbols in equations (4-1), (4-2), (4-3)

Symbol Meaning

𝑌𝑖𝑗 The science motivation posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean science motivation posttest score for teacher j.


𝛾00 Grand-mean science motivation posttest score across all teachers.


The baseline model statistics summary can be viewed in Table 4-15. The ICC =

𝜎𝑢02 / (𝜎𝑢0

2 + 𝜎𝜀2) = 0.06968 / (0.06968 + 0.4559) = 13.2577%. The variance in teacher

means accounted for 13.2577% of the total variance in science motivation posttest

scores. The Design Effect = 1 + (𝑛𝑐 - 1) ICC. The average number of students per

teacher 𝑛𝑐 = 57.7308. The Design Effect was 7.5212. Both the ICC and the Design

Effect indicated it was necessary to conduct multilevel modeling analysis for students’

science motivation posttest scores. The next step was to build the student-level models.

Table 4-15. Baseline model summary

Parameter Estimate Standard Error Z Value DF t Value p

𝜎𝑢02 0.06968 0.02297 3.03 / / .0012

𝜎𝜀2 0.4559 0.01685 27.06 / / <.0001

𝛾00 3.5923 0.05526 / 23.7 65.01 <.0001

-2 Log Likelihood𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒= 3122.2

Student-level models

Random-intercept model. First, a random intercept model was built to estimate

the impact of student-level variables including students’ science motivation pretest

scores and gender on students’ science motivation posttest scores as fixed effects,

126

indicating the impact (coefficient) of pretest scores and gender did not vary across

teachers. As science motivation pretest was measured with a scale that did not contain

zero, a score of zero would have no substantive meaning. Students’ pretest scores

were also independent of each other’s scores. Therefore, grand mean centering was

used to enable a value of zero to be interpreted meaningfully. The equations for the

random-intercept model with grand-mean centering are as follows:

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒)

+ 𝛽2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑖𝑗

(4-4)


𝛽1𝑗 = 𝛾10 (4-6)

𝛽2𝑗 = 𝛾20 (4-7)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒)

+ 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑢0𝑗 + 𝑖𝑗

(4-8)

Besides with the symbols that have been explained previously, 𝛽1𝑗 is the

regression coefficient that shows the impact of science motivation pretest scores on

posttest scores across all students of teacher j , 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 is science motivation

pretest score of student i of teacher j, 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒 is the grand mean of science

motivation pretest scores across teachers, 𝛾10 is the average effect (coefficient) of

pretest scores on posttest scores across all teachers, and 𝛾20 is the average effect

(coefficient) of student gender on science motivation posttest scores across all

teachers. In the model, student gender was treated as a categorical variable and male =

0 was the reference. The random-intercept model statistics summary can be viewed in

Table 4-16.

To determine whether the random-intercept model fit better than the baseline

model, a likelihood ratio (LR) test was used to evaluate the difference between the log

127

likelihood values for the nested models, i.e., the baseline model and the random-

intercept model.

LR = -2LogLikelihoodbaseline

Likelihoodfull

= (-2Log Likelihoodbaseline) - (-2Log Likelihoodfull)

In the baseline model, -2Log Likelihoodbaseline = 3122.2.

In the student-level random-intercept model, -2Log Likelihood𝑓𝑢𝑙𝑙1 = 2102.1.

Therefore, LR = 3122.2 - 2102.1= 1020.1. LR follows a 2 distribution with df = 2.

The df = 2 because the degree of freedom of the baseline model and the random-

intercept model differed by 2, which was the difference between the number of

parameters in the two models. The p value of LR was calculated with the CHIDIST (2

value, df) function in SPSS. The likelihood ratio test was significant with 2 (2) = 1020.1,

p < .0001, indicating the random-intercept model was significantly better than the

baseline model. At least one of the student-level variables, i.e. science motivation

pretest score and student gender, can significantly predict teacher mean science

motivation posttest score.

Table 4-16. Random-intercept model summary


𝜎𝑢02 0.007481 0.003474 2.15 / / .0156

𝜎𝜀2 0.2340 0.008663 27.02 / / <.0001

𝛾00 3.5672 0.02545 / 43.6 139.89 <.0001

𝛾10 0.7372 0.01897 / 1395 38.87 <.0001

𝛾20 0.009815 0.02531 / 1479 0.39 .6982

-2 Log Likelihoodfull1 = 2102.1

128

Random-slope model. As the random-intercept model fit significantly better than

the baseline model, a random-slope model was built to evaluate whether the impact of

students’ science motivation pretest scores and student gender on the posttest scores

varied significantly across teachers. Variance components (i.e., random effects) of

pretest score and student gender were added to the teacher-level slope equation to

model the variation. The equations for the teacher-level model are as follows:


𝛽1𝑗 = 𝛾10 + 𝑢1𝑗 (4-10)

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-11)

After combining with the student-level model (4-4), the combined model is:


+ 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑢1𝑗 (𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒)

+ 𝑢2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 + 𝑢0𝑗 + 𝑖𝑗

(4-12)

In the random-slope model, 𝑢1𝑗 and 𝑢2𝑗 are residual terms to show random

effects, indicating the impact of pretest score and student gender respectively on

posttest score can vary randomly across teachers.

However, the SAS Log noted, “Convergence criteria met but final Hessian is not

positive definite”, “Estimated G matrix is not positive definite”, and “Asymptotic variance

matrix of covariance parameter estimates has been found to be singular and a

generalized inverse was used”, indicating the random-slope model did not fit well after

adding the random effects. After removing Pre_Science from the random effects, SAS

Log provided the same notes. After removing Gender_Student from the random effects

while keeping Pre_Science in the random effects, the notes disappeared, and the model

fit well. Therefore, the final random-slope model just contained the intercept and

129

PRE_Science in the random effects. The random term 𝑢2𝑗 was removed from Formula

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-11) and the final combined model for the random-slope model is:



+ 𝑢1𝑗 (𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒) + 𝑢0𝑗 + 𝑖𝑗

(4-13)

The random-slope model statistics summary can be viewed in Table 4-17. For

the random-slope model, -2 Log Likelihoodfull2= 2096.2.

LR = (-2Log Likelihoodfull1) - (-2Log Likelihoodfull2) = 2102.1 - 2096.2 = 5.9

The likelihood ratio test was nonsignificant with 2 (2) = 5.9, p = .0523, indicating

the random-slope model was not significantly better than the random-intercept model.

Therefore, it was not necessary to build the random-slope model. The next step was to

add teacher-level variables to the random-intercept model.

Table 4-17. Random-slope model summary


𝜎𝑢02 0.006467 0.003194 2.02 / / .0214

𝜎𝑢0𝑢1 0.000998 0.002535 0.39 / / .6938

𝜎𝑢12 0.006525 0.004246 1.54 / / .0622

𝜎𝜀2 0.2314 0.008611 26.87 / / <.0001

𝛾00 3.5649 0.02493 / 45.3 142.98 <.0001

𝛾10 0.7391 0.02544 / 25.9 29.05 <.0001

𝛾20 0.009479 0.02529 / 1477 0.37 .7079


Adding teacher-level variables

Before building the model with teacher variables, multicollinearity was evaluated

to determine whether all the teacher-variables could be included in the model. As shown

130

in Table 4-18, all the teacher-level variables had a Tolerance of higher than 0.1 and a

Variance Inflation Factor (VIF) of less than 10. Therefore, all the teacher-level variables

were included in the model for further analysis.

Table 4-18. Multicollinearity of variables for science motivation posttest score

Variable Tolerance Variance Inflation Factor

Intercept . 0

Pre_Science 0.93130 1.07377

Gender_Student 0.98416 1.01609

Printing_Level 0.38082 2.62590

STEM_Level 0.18647 5.36284

Pedagogical_Beliefs 0.33510 2.98423

Interest_Teacher 0.23051 4.33827

Importance_Teacher 0.16425 6.08843

Usefulness_Teacher 0.13881 7.20423

Self_Efficacy_TK 0.23528 4.25019

Self_Efficacy_PK 0.31878 3.13695

Self_Efficacy_CK 0.28754 3.47772

Self_Efficacy_TPK 0.14508 6.89290

Self_Efficacy_TCK 0.28801 3.47212

Self_Efficacy_PCK 0.22842 4.37798

Self_Efficacy_TPACK 0.10609 9.42587

All the teacher-level variables were centered with the grand mean of the variable.

The equations for the teacher-level model are:

Teacher Level: 𝛽0𝑗 = 𝛾00 + 𝛾01(𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 − 𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 ) +

𝛾02(𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 − 𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙) + 𝛾03(𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠𝑖𝑗 −

𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) + 𝛾04(𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −

𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾05(𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −

(4-14)

131

𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾06(𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾07(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾) + 𝛾08(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) + 𝛾09(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾) + 𝛾010(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾) + 𝛾011(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾) + 𝛾012(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾) + 𝛾013(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾) + 𝑢0𝑗

𝛽1𝑗 = 𝛾10 + 𝛾11(𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 − 𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 ) +



𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾15(𝐼𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −



𝑆𝑒𝑙𝑓_𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾) + 𝛾18(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾)

+ 𝛾19(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾) +

𝛾110(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾) +

𝛾111(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾) +

𝛾112(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾) +

𝛾113(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾)

(4-15)













𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾)

(4-16)

After evaluating the cross-level interactions between the student-level variables

and teacher-level variables, it was found the cross-level interactions

132

Pre_Science*Pedagogical_Beliefs and Gender_Student* STEM_Level were significant.

Therefore, after combining the student-level model (4-4) and teacher-level models

including the significant cross-level interaction terms, the equation for the final model is:

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝛾01(𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 − 𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 ) + 𝛾02(𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 −

𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙) + 𝛾03(𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠𝑖𝑗 − 𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) +

𝛾04(𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 − 𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) +

𝛾05(𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 − 𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) +

𝛾06(𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 − 𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) +

𝛾07(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾) +

𝛾08(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) +

𝛾09(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾) +

𝛾010(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑐𝑦_𝑃𝐾) +



𝛾013(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾) +

𝛾13(𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒𝑖𝑗 − 𝑃𝑟𝑒_𝑆𝑐𝑖𝑒𝑛𝑐𝑒) ∗ (𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠𝑖𝑗 −

𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) + 𝛾22𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 ∗ (𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 −

𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙) + 𝑢0𝑗 + 𝑖𝑗

(4-17)

For this model with teacher-level variables, -2 Log Likelihoodfull3 = 1921.2.

LR = -2 Log Likelihoodfull1 - 2 Log Likelihoodfull3 = 2102.1 – 1921.2 = 180.9.

The likelihood ratio test was significant with 2 (15) = 180.9, p < .0001. The

model with teacher-level variables was significantly better than the random-intercept

model. Therefore, the random-intercept model with teacher variables was the best

model.

The residual normality for student-level and teacher-level residuals were

evaluated. The student-level residual had a skewness of -0.4500 and a kurtosis of

2.3855. The teacher-level residual was just the intercept variance. The skewness of

teacher-level residual was 0.8258 and the kurtosis of it was 0.1792. The skewness and

133

kurtosis for the residuals were between -1 and 1, and -3 and 3 respectively. Therefore,

the residual normality assumptions were met.

The statistics summary for the model with teacher variables can be viewed in

Table 4-19. After grand mean centering, the average intercept 𝛾00 = 3.5802 and it was

statistically significant with t (14.1) = 126.82, p < .0001, indicating on average the

science motivation posttest score was 3.5802 when student gender was male

(Gender_Student = 0) and all other variables were equal to their grand mean, and the

average intercept was significantly different from zero. After accounting for all the

student-level and teacher-level variables, the teacher-level residual variance in the

teacher means, i.e., the intercept variance 𝜎𝑢02 = 0.008594 and it approached

significance with Z = 1.35, p = .0890, indicating the teacher means of student science

motivation posttest scores varied across teachers with approaching significance. On

average, the teacher means of student science motivation posttest scores varied from

the grand mean by √0.008594 = 0.0927. The student-level residual variance 𝜎𝜀2 =

0.2365 and it was statistically significant with Z = 25.38, p < .0001. Student science

motivation posttest scores significantly varied within each teacher. On average,

individual student’s science motivation posttest score varied from their teacher mean by

√0.2365 = 0.4863.

As the best fit model was a random-intercept model with teacher variables, each

teacher had a different intercept but the same slope for each variable. The slope of

science motivation pretest scores 𝛾10 = 0.7107, statistically significant with t (1235) =

34.33, p < .0001. The slope 𝛾10 indicated on average one score increase in science

motivation pretest score increased science motivation posttest score by 0.7107 when

134

controlling for other variables. After controlling for student variables, none of the teacher

variables were significant predictors of student science motivation posttest scores. The

slope of teachers’ perceived usefulness of 3D printing integration 𝛾06 = -0.2507 and it

approached significance with t (9.88) = -2.02, p = .0708. The slope 𝛾06 indicated one

score increase in teachers’ perceived usefulness of 3D printing integration decreased

student science motivation posttest score by 0.2507 with approaching significance when

controlling for other variables.

The interaction between student science motivation pretest score and teachers’

pedagogical beliefs was significant with slope 𝛾13 = -0.1291, t (1296) = -2.97, p = .0030.

One score increase in student science motivation pretest score decreased the slope of

teachers’ pedagogical beliefs by 0.1291. The interaction between student gender and

teachers’ STEM integration level was significant with slope 𝛾22 = 0.09404, t (1299) =

2.48, p = .0133. As male students were coded as 0 (reference level) and female

students were coded as 1, the slope of teachers’ STEM integration level for female

students (one level increase in student gender) was 0.09404 higher than the slope for

male students.

Table 4-19. Random-intercept model with teacher variables model summary

Parameter Estimate Standard Error

Z Value

DF t Value

p

Teacher-level intercept variance (𝜎𝑢02 ) 0.008594 0.006379 1.35 / / .0890

Student-level residual variance (𝜎𝜀2) 0.2365 0.009320 25.38 / / <.0001

Intercept (𝛾00) 3.5802 0.02823 / 14.1 126.82

<.0001

Pre_Science (𝛾10) 0.7107 0.02070 / 1235 34.33 <.0001

Gender_Student (𝛾20) -0.00002 0.02719 / 1296 -0.00 .9993

Printing_Level (𝛾01) -0.00819 0.03618 / 8.63 -0.23 .8262

135



Z Value

DF t Value

p

STEM_Level (𝛾02) 0.01457 0.07678 / 9.77 0.19 .8533

Pedagogical_Beliefs (𝛾03) 0.1076 0.08125 / 9.98 1.32 .2148

Interest_Teacher (𝛾04) -0.07845 0.08367 / 9.97 -0.94 .3706

Importance_Teacher (𝛾05) 0.1069 0.09401 / 10 1.14 .2822

Usefulness_Teacher (𝛾06) -0.2507 0.1238 / 9.88 -2.02 .0708

Self_Efficacy_TK (𝛾07) 0.02327 0.06138 / 11.1 0.38 .7117

Self_Efficacy_PK (𝛾08) 0.1317 0.09701 / 7.94 1.36 .2118

Self_Efficacy_CK (𝛾09) -0.06820 0.08284 / 8.78 -0.82 .4322

Self_Efficacy_TPK (𝛾010) 0.07921 0.1215 / 9 0.65 .5308

Self_Efficacy_TCK (𝛾011) -0.1127 0.07729 / 10.6 -1.46 .1736

Self_Efficacy_PCK (𝛾012) -0.06159 0.07818 / 9.53 -0.79 .4500

Self_Efficacy_TPACK (𝛾013) 0.05178 0.1450 / 9.13 0.36 .7290

Pre_Science*Pedagogical_Beliefs (𝛾13)

-0.1291 0.04343 / 1296 -2.97 .0030

Gender_Student*STEM_Level (𝛾22) 0.09404 0.03793 / 1299 2.48 .0133


Effect size calculation

The effect size was calculated by assessing the R2 at the student level and R2 at

the teacher level. R2 at the student level measured how student-level variance was

explained by the final model compared to the baseline model. R2 at the teacher level

measured how teacher-level variance was explained by the final model compared to the

student-level random-intercept model. The final model was the random-intercept model

with teacher variables. The student-level variance of the baseline model and final model

and the teacher-level variance of the student-level random-intercept model and the final

model for effect size calculation are presented in Table 4-20.

136

Table 4-20. Statistics for effect size calculation

Parameter Estimate

baseline model 𝜎𝜀2 0.4559

student-level random-intercept model 𝜎𝑢02 0.007481

final model 𝜎𝜀2 0.2365

final model 𝜎𝑢02 0.008594

R2 at the student level = (baseline model 𝜎𝜀2 - final model 𝜎𝜀

2 ) / baseline model 𝜎𝜀2

= (0.4559 – 0.2365) / 0.4559 = 48.1246 %.

R2 at the teacher level = (student-level random-intercept model 𝜎𝑢02 - final model

𝜎𝑢02 ) / student-level random-intercept model 𝜎𝑢0

2 = (0.007481 - 0.008594) / 0.007481 = -

14.8777%.

The final model (random-intercept model with teacher variables) explained about

48.1246 % of the student-level variance but the teacher-level variables did not

contribute to explaining teacher-level variance.

Results for Technology/Engineering Motivation

A series of multilevel models were built to examine how student gender,

teachers’ 3D printing integration levels, STEM integration levels, pedagogical beliefs,

value beliefs, and self-efficacy beliefs in 3D printing integration predict students’

technology/engineering motivation while controlling for student gender and

technology/engineering motivation pretest scores. Student gender and

technology/engineering motivation pretest scores were included in the model as

covariates to be controlled while interpreting the intercept and the coefficients of other

variables.

137

Baseline model

The first step was to build the baseline model, which predicted a student’

technology/engineering motivation posttest score from the grand mean

technology/engineering motivation posttest score of all the teachers’ students. There

were no student-level or teacher-level predictors in this model. The purpose of this

model was to examine the ICC and design effect to determine whether multilevel

models were necessary for the analysis. The meanings of symbols in the model are

provided in Table 4-21.






Symbol Meaning

𝑌𝑖𝑗 The technology/engineering motivation posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean technology/engineering motivation posttest score for teacher j.


𝛾00 Grand-mean technology/engineering motivation posttest score across all teachers.



𝜎𝑢02 / (𝜎𝑢0


means accounted for 2.9993% of the total variance in technology/engineering

motivation posttest scores. The Design Effect = 1 + (𝑛𝑐 - 1) ICC. The average number of

138

students per teacher 𝑛𝑐 = 57.7308. The Design Effect was 2.7015. The ICC was

smaller than 0.5 but the Design Effect was larger than 2, so multilevel modeling analysis

was used. The next step was to build the student-level models.



Z Value

DF t Value p

Teacher-level intercept variance (𝜎𝑢02 ) 0.01636 0.008364 1.96 / / 0.0253


Intercept (𝛾00) 3.4843 0.03226 / 19.4 108.00 <.0001




the impact of student-level variables including students’ technology/engineering

motivation pretest scores and gender on students’ technology/engineering motivation

posttest scores as fixed effects, indicating the impact (coefficient) of pretest scores and

gender did not vary across teachers. As technology/engineering motivation pretest was

measured with a scale that did not contain zero, a score of zero would have no

substantive meaning. Students’ pretest scores were also independent of each other’s

scores. Therefore, grand mean centering was used to enable a value of zero to be

interpreted meaningfully. The equations for the random-intercept model with grand-

mean centering are as follows:

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖𝑖𝑗 − 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖)


(4-21)


𝛽1𝑗 = 𝛾10 (4-23)

𝛽2𝑗 = 𝛾20 (4-24)

139

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖𝑖𝑗 − 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖)


(4-25)


regression coefficient that shows the impact of technology/engineering motivation

pretest scores on posttest scores across all students of teacher j , 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖𝑖𝑗 is

technology/engineering motivation pretest score of student i of teacher j, 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖

is the grand mean of technology/engineering motivation pretest scores across teachers,

𝛾10 is the average effect (coefficient) of pretest scores on posttest scores across all

teachers, and 𝛾20 is the average effect (coefficient) of student gender on

technology/engineering motivation posttest scores across all teachers. In the model,

student gender was treated as a categorical variable and male = 0 was the reference.

The random-intercept model statistics summary can be viewed in Table 4-23.




intercept model.


Likelihoodfull



In the student-level random-intercept model, -2Log Likelihood𝑓𝑢𝑙𝑙1 =2180.9.

Therefore, LR = 3298.4- 2180.9 = 1117.5. LR follows a 2 distribution with df = 2.



140




baseline model. At least one of the student-level variables, i.e. technology/engineering

motivation pretest score and student gender, can significantly predict teacher mean

technology/engineering motivation posttest score.



𝜎𝑢02 0.001456 0.001784 0.82 / / .2073

𝜎𝜀2 0.2525 0.009384 26.91 / / <.0001

𝛾00 3.5493 0.02123 / 64.6 167.22 <.0001

𝛾10 0.7507 0.01988 / 1427 37.76 <.0001

𝛾20 -0.1060 0.02731 / 1474 -3.88 .0001




students’ technology/engineering motivation pretest scores and student gender on the

posttest scores varied significantly across teachers. Variance components (i.e., random

effects) of pretest score and student gender were added to the teacher-level slope

equation to model the variation. The equations for the teacher-level model are as

follows:


𝛽1𝑗 = 𝛾10 + 𝑢1𝑗 (4-27)

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-28)


141

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖𝑖𝑗 − 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖)


+ 𝑢1𝑗 (𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖𝑖𝑗 − 𝑃𝑟𝑒_𝑇𝑒𝑐ℎ𝐸𝑛𝑔𝑖)


(4-29)








adding the random effects. When deleting Pre_TechEngi from the random effects, the

SAS Log showed the same warning. When deleting Gender_Student from the random

effects, SAS Log indicated “Estimated G matrix is not positive definite”. These warnings

indicated the model did not fit well with either Pre_TechEngi or Gender_Student or both

of them in the random effects. Therefore, both Gender_Student and Pre_TechEngi were

deleted from the random effects and only the random-intercept model fit well.

Continuing with the random-intercept model, teacher-level variables were added to the

model for examining the influence of the teacher-level variables on student

technology/engineering motivation posttest scores.





142



Table 4-24. Multicollinearity of variables for technology/engineering motivation posttest score


Intercept . 0

Pre_TechEngi 0.90113 1.10972



STEM_Level 0.18995 5.26460



















(4-30)

143





















(4-31)














(4-32)



Gender_Student*Importance_Teacher and Gender_Student*Self_Efficacy_PK were

144

significant. Therefore, after combining the student-level model (4-21) and teacher-level

models including the significant cross-level interaction terms, the equation for the final

model is:










𝛾010(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑦_𝑇𝑃𝐾) +




𝛾25𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 ∗ (𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −

𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾28𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 ∗

(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) + 𝑢0𝑗 + 𝑖𝑗

(4-33)



The likelihood ratio test was significant with 2 (15) = 180.9, p < 0.0001. The



model.




teacher-level residual was -0.2086 and the kurtosis of it was -0.6442. The skewness

145

and kurtosis for the residuals were between -1 and 1, and -3 and 3 respectively.

Therefore, the residual normality assumptions were met.



statistically significant with t (18.9) = 153.33, p < .0001, indicating on average the

technology/engineering motivation posttest score was 3.5477 when student gender was

male (Gender_Student = 0) and all other variables were equal to their grand mean, and

the average intercept was significantly different from zero. After accounting for all the


teacher means, i.e., the intercept variance 𝜎𝑢02 = 0.001550 and it was not significant,

indicating the teacher means of student technology/engineering motivation posttest

scores did not vary significantly across teachers. On average, the teacher means of

student technology/engineering motivation posttest scores varied from the grand mean

by √0.001550 = 0.0394. The student-level residual variance 𝜎𝜀2 = 0.2644 and it was

statistically significant with Z = 25.27, p < .0001. Student technology/engineering

motivation posttest scores significantly varied within each teacher. On average,

individual student’s technology/engineering motivation posttest score varied from their

teacher mean by √0.2644 = 0.5142.


teacher had a different intercept but the same slope for each teacher-level variable. The

slope of technology/engineering motivation pretest scores 𝛾10 = 0.7310, statistically

significant with t (1264) = 33.82, p < .0001. The slope 𝛾10 indicated on average one

score increase in technology/engineering motivation pretest score increased

146

technology/engineering motivation posttest score by 0.7107 when controlling for other

variables.

The slope of teachers’ self-efficacy in TK 𝛾07 = -0.09374 and it approached

significance with t (11.8) = -1.98, p = .0709. The slope 𝛾07 indicated one score increase

in teachers’ self-efficacy in TK decreased student technology/engineering motivation

posttest score by 0.09374 with approaching significance when controlling for other

variables. The slope of teachers’ self-efficacy in PK 𝛾08 = 0.03467 and it was significant

with t (9.44) = 2.68, p = .0241. However, student gender interacted with teachers’ self-

efficacy in PK and the slope of the interaction 𝛾28 = -0.1452, statistically significant with t

(1287) = -2.18, p = .0292. Male students were coded as 0 (reference level) and female

students were coded as 1. Therefore, the slope of teachers’ self-efficacy in PK for male

students was 0.1452 higher than the slope for female students. When student gender

was 0 (male students), the slope for male students was 0.03467, indicating one score

increase in teachers’ self-efficacy in PK increased male students’

technology/engineering motivation posttest score by 0.03467. When student gender

was 1 (female students), the slope for female students was -0.11053, indicating one

score increase in teachers’ self-efficacy in PK decreased female students’

technology/engineering motivation by 0.11053.

The slope of student gender 𝛾20 = -0.1029, statistically significant with t (1289) = -

3.45, p = .0006. Student gender interacted with teachers’ perceived importance of 3D

printing integration and the slope of the interaction 𝛾25 = 0.1189, statistically significant

with t (1288) = 2.35, p = .0191. The technology/engineering motivation posttest scores

of male students were 0.1029 higher than female students when teachers’ perceived

147

importance of 3D printing integration and self-efficacy in PK were at their grand mean

and also controlling for other variables. When controlling for other variables, the slope of

teachers’ perceived importance of 3D printing integration for female students was

0.1189 higher than the slope for male students. Therefore, although teachers’ perceived

importance of 3D printing integration was not a significant predictor, it had a more

positive effect on female students.



Z Value

DF t Value p



Intercept (𝛾00) 3.5477 0.02314 / 18.9 153.33 <.0001

Pre_Tech/Engi (𝛾10) 0.7310 0.02162 / 1264 33.82 <.0001

Gender_Student (𝛾20) -0.1029 0.02980 / 1289 -3.45 .0006

Printing_Level (𝛾01) 0.03631 0.02586 / 6.84 1.40 .2040

STEM_Level (𝛾02) -0.00529 0.05288 / 6.63 -0.10 .9233



Importance_Teacher (𝛾05) -0.07549 0.07510 / 11.6 -1.01 .3353

Usefulness_Teacher (𝛾06) 0.08973 0.09140 / 9.08 0.98 .3517

Self_Efficacy_TK (𝛾07) -0.09374 0.04723 / 11.8 -1.98 .0709


Self_Efficacy_CK (𝛾09) 0.2052 0.07648 / 7.3 0.58 .5777

Self_Efficacy_TPK (𝛾010) 0.05224 0.08817 / 7.04 0.59 .5720

Self_Efficacy_TCK (𝛾011) 0.001128 0.05869 / 9.43 0.02 .9851

148



Z Value

DF t Value p

Self_Efficacy_PCK (𝛾012) -0.04992 0.05767 / 9.04 -0.87 .4091


Gender_Student*Importance_Teacher (𝛾25)

0.1189 0.05068 / 1288 2.35 .0191

Gender_Student*Self_Efficacy_PK (𝛾28)

-0.1452 0.06647 / 1287 -2.18 .0292












Parameter Estimate





149



= (0.5291 – 0.2644) / 0.5291 = 50.0284%.



2 = (0.001456 - 0.001550) / 0.001456 = -

6.4560%.


50.0284% of the student-level variance but the teacher-level variables did not contribute

to explaining teacher-level variance.

Results for Math Motivation



efficacy beliefs in 3D printing integration predict students’ math motivation while

controlling for student gender and math motivation pretest scores. Student gender and

math motivation pretest scores were included in the model as covariates to be


Baseline model

The first step was to build the baseline model, which predicted a student’ math

motivation posttest score from the grand mean math motivation posttest score of all the

teachers’ students. There were no student-level or teacher-level predictors in this





150





Symbol Meaning

𝑌𝑖𝑗 The math motivation posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean math motivation posttest score for teacher j.


𝛾00 Grand-mean math motivation posttest score across all teachers.



𝜎𝑢02 / (𝜎𝑢0


means accounted for 12.1061% of the total variance in math motivation posttest scores.

The Design Effect = 1 + (𝑛𝑐 - 1) ICC. The average number of students per teacher 𝑛𝑐 =

57.7308. The Design Effect was 7.8679. Both the ICC and the Design Effect indicated it

was necessary to conduct multilevel modeling analysis for students’ math motivation

posttest scores. The next step was to build the student-level models.



𝜎𝑢02 0.09501 0.03150 3.02 / / 0.0013

𝜎𝜀2 0.6898 0.02541 27.15 / / <.0001

𝛾00 3.5589 0.06490 / 24 54.82 <.0001


151



the impact of student-level variables including students’ math motivation pretest scores

and gender on students’ math motivation posttest scores as fixed effects, indicating the

impact (coefficient) of pretest scores and gender did not vary across teachers. As math

motivation pretest was measured with a scale that did not contain zero, a score of zero

would have no substantive meaning. Students’ pretest scores were also independent of

each other’s scores. Therefore, grand mean centering was used to enable a value of

zero to be interpreted meaningfully. The equations for the random-intercept model with

grand-mean centering are as follows:

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) + 𝛽2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑖𝑗

(4-37)


𝛽1𝑗 = 𝛾10 (4-39)

𝛽2𝑗 = 𝛾20 (4-40)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑢0𝑗 + 𝑖𝑗

(4-41)


regression coefficient that shows the impact of math motivation pretest scores on

posttest scores across all students of teacher j , 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 is math motivation pretest

score of student i of teacher j, 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ is the grand mean of science motivation pretest

scores across teachers, 𝛾10 is the average effect (coefficient) of pretest scores on

posttest scores across all teachers, and 𝛾20 is the average effect (coefficient) of student

gender on math motivation posttest scores across all teachers. In the model, student

gender was treated as a categorical variable and male = 0 was the reference. The

random-intercept model statistics summary can be viewed in Table 4-29.

152




intercept model.


Likelihoodfull



In the student-level random-intercept model, -2Log Likelihood𝑓𝑢𝑙𝑙1 = 2051.6.

Therefore, LR = 3759.7 - 2051.6 = 1708.1. LR follows a 2 distribution with df = 2.






baseline model. At least one of the student-level variables, i.e. math motivation pretest

score and student gender, can significantly predict teacher mean math motivation

posttest score.



𝜎𝑢02 0.008004 0.003481 2.30 / / 0.0108

𝜎𝜀2 0.2232 0.008223 27.15 / / <.0001

𝛾00 3.5487 0.02552 / 44.1 139.06 <.0001

𝛾10 0.8603 0.01513 / 1420 56.85 <.0001

𝛾20 0.004700 0.02467 / 1491 0.19 .8490


153



students’ math motivation pretest scores and student gender on the posttest scores





𝛽1𝑗 = 𝛾10 +𝑢1𝑗 (4-43)

𝛽2𝑗 = 𝛾20 +𝑢2𝑗 (4-44)



+ 𝑢1𝑗 (𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ)


(4-45)





positive definite” and “Asymptotic variance matrix of covariance parameter estimates

has been found to be singular and a generalized inverse was used”, indicating the

random-slope model did not fit well after adding the random effects. After removing

Pre_Math from the random effects, SAS Log provided the same notes. After removing

Gender_Student from the random effects while keeping Pre_Math in the random

effects, the notes disappeared, and the model fit well. Therefore, the final random-slope

model just contained the intercept and Pre_Math in the random effects. The random

154

term 𝑢2𝑗 was removed from Formula 𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-44) and the final combined

model for the random-slope model is:


+ 𝑢1𝑗 (𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) + 𝑢0𝑗 + 𝑖𝑗

(4-46)



LR = (-2Log Likelihoodfull1) - (-2Log Likelihoodfull2) = 2051.6 - 2050.2 = 1.4







𝜎𝑢02 0.008191 0.003580 2.29 / / .0111

𝜎𝑢0𝑢1 -0.00186 0.002319 -0.80 / / .4228

𝜎𝑢12 0.002292 0.002938 0.78 / / .2176

𝜎𝜀2 0.2219 0.008264 26.85 / / <.0001

𝛾00 3.5523 0.02595 / 43.3 136.90 <.0001

𝛾10 0.8570 0.01865 / 13.4 45.94 <.0001

𝛾20 0.004782 0.02472 / 1489 0.19 0.8466






155



Table 4-31. Multicollinearity of variables for math motivation posttest score


Intercept . 0

Pre_Math 0.91060 1.09817



STEM_Level 0.18755 5.33194




















(4-47)

156





















(4-48)














(4-49)


and teacher-level variables, it was found the cross-level interactions PRE_MATH *

Usefulness_Teacher, PRE_MATH*Self_Efficacy_TPK, PRE_MATH*Self_Efficacy_PK,

157

and Gender_Student*Self_Efficacy_PK were significant. Therefore, after combining the

student-level model (4-37) and teacher-level models and including the significant cross-

level interaction terms, the equation for the final model is:

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 +

𝛾01(𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 − 𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 ) + 𝛾02(𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙𝑖𝑗 −












𝛾16(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) ∗ (𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟𝑖𝑗 −

𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛾18(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 − 𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) ∗

(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) + 𝛾110(𝑃𝑟𝑒_𝑀𝑎𝑡ℎ𝑖𝑗 −

𝑃𝑟𝑒_𝑀𝑎𝑡ℎ) ∗ (𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾) +

𝛾28𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 ∗ (𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) +

𝑢0𝑗 + 𝑖𝑗

(4-50)



The likelihood ratio test was significant with 2 (17) = 148.8, p < 0.0001. The



model.




158

teacher-level residual was 0.2018 and the kurtosis of it was -0.9543. The skewness and





statistically significant with t (19.1) = 137.99, p < .0001, indicating on average the math

motivation posttest score was 3.5446 when student gender was male (Gender_Student

= 0) and all other variables were equal to their grand mean, and the average intercept

was significantly different from zero. After accounting for all the student-level and

teacher-level variables, the teacher-level residual variance in the teacher means, i.e.,

the intercept variance 𝜎𝑢02 = 0.005908 and it approached significance with Z = 1.31, p =

.0944, indicating the teacher means of student math motivation posttest scores varied

across teachers with approaching significance. On average, the teacher means of

student math motivation posttest scores varied from the grand mean by √0.005908 =

0.0769. The student-level residual variance 𝜎𝜀2 = 0.2291 and it was statistically

significant with Z = 25.51, p < .0001. Student math motivation posttest scores

significantly varied within each teacher. On average, individual student’s math

motivation posttest score varied from their teacher mean by √0.2291 = 0.4786.


teacher had a different intercept but the same slope for each variable. The slope of

math motivation pretest scores 𝛾10 = 0.8593, statistically significant with t (1279) =

51.66, p < .0001. The slope 𝛾10 indicated on average one score increase in math

motivation pretest score increased math motivation posttest score by 0.8593 when

159

controlling for other variables. The slope of STEM integration level 𝛾02 = 0.1674,

statistically significant with t (10.5) = 2.55, p = .0280. The slope 𝛾02 indicated one score

increase in STEM integration level increased student math motivation posttest score by

0.1674 when controlling for other variables. The slope of teachers’ perceived usefulness

of 3D printing integration 𝛾06 = -0.2413, statistically significant with t (12.3) = -2.19, p =

.0482. However, the interaction between student math motivation pretest score and

teachers’ perceived usefulness of 3D printing integration was significant with slope 𝛾16 =

-0.1199, t (1286) = -3.16, p = .0016. The slope 𝛾06 indicated one score increase in

teachers’ perceived usefulness of 3D printing integration decreased student math

motivation posttest score by 0.2413 when student math motivation pretest score was

equal to the grand mean and controlling for other variables. The interaction slope 𝛾16

indicated one score increase in student math motivation pretest score decreased the

slope of teachers’ perceived usefulness of 3D printing integration by 0.1199 when

controlling for other variables. Teachers’ self-efficacy in PCK 𝛾012 = 0.1354 and

approached significance with t (12.1) = 1.94, p = .0758. The slope 𝛾012 indicated one

score increase in teachers’ self-efficacy in PCK increased student math motivation

posttest score by 0.1354 with approaching significance when controlling for other

variables.

There were also some other cross-level interactions between the student-level

and teacher-level variables. The interaction between student math motivation pretest

score and teachers’ self-efficacy in PK was significant with slope 𝛾18 = -0.08292, t

(1193) = -2.18, p = .0298. One score increase in student math motivation pretest score

decreased the slope of teachers’ self-efficacy in PK by 0.08292. The interaction

160

between student math motivation pretest score and teachers’ self-efficacy in TPK was

significant with slope 𝛾110 = 0.1096, t (1303) = 2.89, p = .0039. One score increase in

student math motivation pretest score increased the slope of teachers’ self-efficacy in

PK by 0.1096. The interaction between student gender and teachers’ self-efficacy in PK

was significant with slope 𝛾28 = -0.2276, t (1308) = -3.71, p = .0002. As male students

were coded as 0 (reference level) and female students were coded as 1, the slope of

teachers’ self-efficacy in PK for female students (one level increase in student gender)

was 0.2276 lower than the slope for male students.



Z Value

DF t Value p



Intercept (𝛾00) 3.5446 0.02569 / 19.1 137.99 <.0001

Pre_Math (𝛾10) 0.8593 0.01663 / 1279 51.66 <.0001

Gender_Student (𝛾20) 0.008217 0.02666 / 1309 0.31 .7580

Printing_Level (𝛾01) 0.001953 0.03210 / 10.7 0.06 .9526

STEM_Level (𝛾02) 0.1674 0.06570 / 10.5 2.55 .0280



Importance_Teacher (𝛾05) 0.1357 0.08381 / 12.5 1.62 .1304




Self_Efficacy_CK (𝛾09) -0.03538 0.07340 / 10.8 -0.48 .6394

Self_Efficacy_TPK (𝛾010) 0.1412 0.1079 / 11 1.31 .2173

Self_Efficacy_TCK (𝛾011) 0.02345 0.06911 / 13.4 0.34 .7397

161



Z Value

DF t Value p

Self_Efficacy_PCK (𝛾012) 0.1354 0.06977 / 12.1 1.94 .0758

Self_Efficacy_TPACK (𝛾013) -0.1192 0.1305 / 11.9 -0.91 .3791

Pre_Math*Usefulness_Teacher (𝛾16) -0.1199 0.03797 / 1286 -3.16 .0016

Pre_Math*Self_Efficacy_PK (𝛾18) -0.08292 0.03812 / 1193 -2.18 .0298

Pre_Math*Self_Efficacy_TPK (𝛾110) 0.1096 0.03788 / 1303 2.89 .0039

Gender_Student*Self_Efficacy_PK (𝛾28)

-0.2276 0.06135 / 1308 -3.71 .0002












Parameter Estimate





162



= (0.6898 – 0.2291) / 0.6898 = 66.7875%.



2 = (0.008004 - 0.005908) / 0.008004 =

26.1869%.

Therefore, the final model (random-intercept model with teacher variables)

explained about 66.7875% of the student-level variance and the teacher-level variables

explained about 26.1869% of the teacher-level variance.

Results for 21st Century Skills



efficacy beliefs in 3D printing integration predict students’ 21st century skills while

controlling for student gender and 21st century skills pretest scores. Student gender and

21st century skills pretest scores were included in the model as covariates to be


Baseline model

The first step was to build the baseline model, which predicted a student’ 21st

century skills posttest score from the grand mean 21st century skills posttest score of all

the teachers’ students. There were no student-level or teacher-level predictors in this




163






Symbol Meaning

𝑌𝑖𝑗 The 21st century skills posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean 21st century skills posttest score for teacher j.


𝛾00 Grand-mean 21st century skills posttest score across all teachers.



𝜎𝑢02 / (𝜎𝑢0




57.7308. The Design Effect was 3.3776. The ICC was slightly less than 0.05 but the

Design Effect was greater than 2, therefore multilevel modeling analysis was conducted

to examine the influence of student-level and teacher level variables on students’ 21st

century skills. The next step was to build the student-level models.



𝜎𝑢02 0.01283 0.005412 2.37 / / 0.0089

𝜎𝜀2 0.2933 0.01087 26.97 / / <.0001

𝛾00 4.0683 0.02693 / 23.1 151.06 <.0001


164



the impact of student-level variables including students’ 21st century skills pretest

scores and gender on students’ 21st century skills posttest scores as fixed effects,

indicating the impact (coefficient) of pretest scores and gender did not vary across

teachers. As 21st century skills pretest was measured with a scale that did not contain

zero, a score of zero would have no substantive meaning. Students’ pretest scores

were also independent of each other’s scores. Therefore, grand mean centering was

used to enable a value of zero to be interpreted meaningfully. The equations for the

random-intercept model with grand-mean centering are as follows:

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡) + 𝛽2𝑗𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑖𝑗

(4-54)


𝛽1𝑗 = 𝛾10 (4-56)

𝛽2𝑗 = 𝛾20 (4-57)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑢0𝑗 + 𝑖𝑗

(4-58)


regression coefficient that shows the impact of 21st century skills pretest scores on

posttest scores across all students of teacher j , 𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 is 21st century skills pretest

score of student i of teacher j, 𝑃𝑟𝑒_21𝑠𝑡 is the grand mean of 21st century skills pretest

scores across teachers, 𝛾10 is the average effect (coefficient) of pretest scores on

posttest scores across all teachers, and 𝛾20 is the average effect (coefficient) of student

gender on 21st century skills posttest scores across all teachers. In the model, student

gender was treated as a categorical variable and male = 0 was the reference. The

random-intercept model statistics summary can be viewed in Table 4-36.

165




intercept model.


Likelihoodfull = (-2Log Likelihoodbaseline) - (-2Log Likelihoodfull)


In the student-level random-intercept model, -2Log Likelihood𝑓𝑢𝑙𝑙1 =1581.0.

Therefore, LR = 2423.6 - 1581.0 = 842.6. LR follows a 2 distribution with df = 2.






baseline model. At least one of the student-level variables, i.e. 21st century skills pretest

score and student gender, can significantly predict teacher mean 21st century skills

posttest score.



𝜎𝑢02 0.004259 0.002101 2.03 / / 0.0213

𝜎𝜀2 0.1677 0.006249 26.84 / / <.0001

𝛾00 4.0338 0.02076 / 50.7 194.33 <.0001

𝛾10 0.6941 0.02182 / 1463 31.80 <.0001

𝛾20 0.07722 0.02174 / 1458 3.55 0.0004


166



students’ 21st century skills pretest scores and student gender on the posttest scores





𝛽1𝑗 = 𝛾10 + 𝑢1𝑗 (4-60)

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-61)


Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡)


+ 𝑢1𝑗 (𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡)


(4-62)








adding the random effects. When deleting Pre_21st from the random effects, the SAS

Log showed the same warning. After removing Gender_Student from the random

effects while keeping Pre_21st in the random effects, the notes disappeared, and the

model fit well. Therefore, the final random-slope model just contained the intercept and

167

PRE_21st in the random effects. The random term 𝑢2𝑗 was removed from Formula

𝛽2𝑗 = 𝛾20 + 𝑢2𝑗 (4-61) and the final combined model for the random-slope model is:

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡) + 𝛾20𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗

+ 𝑢1𝑗 (𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡) + 𝑢0𝑗 + 𝑖𝑗

(4-63)



LR = (-2Log Likelihoodfull1) - (-2Log Likelihoodfull2) = 1581.0 -1577.7 = 3.3







𝜎𝑢02 0.004591 0.002230 2.06 / / .0198

𝜎𝑢0𝑢1 -0.00098 0.002671 -0.37 / / .7140

𝜎𝑢12 0.007788 0.005999 1.30 / / .0971

𝜎𝜀2 0.1658 0.006242 26.57 / / <.0001

𝛾00 4.0361 0.02110 / 48.1 191.30 <.0001

𝛾10 0.6943 0.02862 / 21.3 24.26 <.0001

𝛾20 0.07687 0.02168 / 1445 3.54 .0004


168







Table 4-38. Multicollinearity of variables for 21st century skills posttest score


Intercept . 0

Pre_21st 0.97464 1.02602



STEM_Level 0.19046 5.25047














169














(4-64)













(4-65)














(4-66)

170



Pre_21st*Self_Efficacy_CK and Gender_Student*Pedagogical_Beliefs were significant.

Therefore, after combining the student-level model (4-54) and teacher-level models

including the significant cross-level interaction terms, the equation for the final model is:










𝛾010(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾𝑖𝑗 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) +




𝛾19(𝑃𝑟𝑒_21𝑠𝑡𝑖𝑗 − 𝑃𝑟𝑒_21𝑠𝑡) ∗ (𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾𝑖𝑗 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾) + 𝛾23𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡𝑖𝑗 ∗

(𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠𝑖𝑗 − 𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) + 𝑢0𝑗 + 𝑖𝑗

(4-67)



The likelihood ratio test was significant with 2 (15) = 92.4, p < .0001. The model

with teacher-level variables was significantly better than the random-intercept model.

Therefore, the random-intercept model with teacher variables was the best model.




171

teacher-level residual was -0.5489 and the kurtosis of it was 0.3091. The skewness and





statistically significant with t (20.9) = 207.22, p < .0001, indicating on average the 21st

century skills posttest score was 4.0246 when student gender was male

(Gender_Student = 0) and all other variables were equal to their grand mean, and the

average intercept was significantly different from zero. After accounting for all the


teacher means, i.e., the intercept variance 𝜎𝑢02 = 0.001683 and it was not significant,

indicating the teacher means of student 21st century skills posttest scores did not vary

significantly across teachers. On average, the teacher means of student 21st century

skills posttest scores varied from the grand mean by √0.001683 = 0.0410. The student-

level residual variance 𝜎𝜀2 = 0.1734 and it was statistically significant with Z = 25.20, p <

.0001. Student 21st century skills posttest scores significantly varied within each

teacher. On average, individual student’s 21st century skills posttest score varied from

their teacher mean by √0.1734 = 0.4164.


teacher had a different intercept but the same slope for each teacher-level variable. The

slope of 21st century skills pretest scores 𝛾10 = 0.6943, statistically significant with t

(1278) = 29.53, p < .0001. Students’ pretest scores interacted with teachers’ self-

efficacy beliefs in CK. To interpret the slope of the pretest scores 𝛾10, the interaction

172

had to be ruled out, namely teachers’ self-efficacy beliefs in CK had to be at its grand

mean. Thus, the slope 𝛾10 indicated on average one score increase in 21st century

skills pretest score increased 21st century skills posttest score by 0.6943 when

teachers’ self-efficacy beliefs in CK was at its grand mean and also controlling for other

variables. The slope of the interaction between 21st century skills pretest scores and

teachers’ self-efficacy beliefs in CK 𝛾19 = 0.08887 and it was statistically significant with

t (1225) = 2.08, p = .0380. The interaction slope 𝛾19 indicated one score increase in

students’ 21st century skills pretest scores increased the slope of teachers’ self-efficacy

beliefs in CK by 0.08887 when controlling for other variables.

The slope of student gender 𝛾20 = 0.08505, statistically significant with t (1278) =

3.61, p = .0003. Student gender interacted with teachers’ pedagogical beliefs. Male

students were coded as 0 (reference level) and female students were coded as 1. To

interpret the slope of gender 𝛾20, the interaction had to be ruled out, namely teachers’

pedagogical beliefs had to be at its grand mean. Thus, the 21st century skills posttest

scores of female students were 0.08505 higher than male students when teachers’

pedagogical beliefs were at the grand mean and also controlling for other variables. The

slope of the interaction between student gender and teachers’ pedagogical beliefs 𝛾23 =

0.1186 and it was statistically significant with t (1280) = 2.36, p = .0186. The slope of

teachers’ pedagogical beliefs for female students was 0.1186 higher than the slope for

male students. Lastly, the slope of teachers’ perceived usefulness of 3D printing

integration 𝛾06 = -0.2280 and it was statistically significant with t (11.1) = -2.89, p =

.0147. The slope 𝛾06 indicated one score increase in teachers’ perceived usefulness of

173

3D printing integration decreased students’ 21st century skills posttest scores by 0.2280

when controlling for other variables.



Z Value

DF t Value p



Intercept (𝛾00) 4.0246 0.01942 / 20.9 207.22 <.0001

Pre_21st (𝛾10) 0.6943 0.02351 / 1278 29.53 <.0001

Gender_Student (𝛾20) 0.08505 0.02357 / 1278 3.61 .0003

Printing_Level (𝛾01) 0.006339 0.02248 / 8.61 0.28 .7846

STEM_Level (𝛾02) 0.05526 0.04611 / 8.44 1.20 .2633


Interest_Teacher (𝛾04) 0.005602 0.05383 / 11.3 0.10 .9189

Importance_Teacher (𝛾05) 0.05545 0.06078 / 11 0.91 .3811




Self_Efficacy_CK (𝛾09) -0.07383 0.05160 / 9 -1.43 .1863

Self_Efficacy_TPK (𝛾010) 0.03989 0.07648 / 8.94 0.52 .6146

Self_Efficacy_TCK (𝛾011) -0.02865 0.05062 / 11.8 -0.57 .5820

Self_Efficacy_PCK (𝛾012) 0.03742 0.04979 / 10.9 0.75 .4683


Pre_21st*Self_Efficacy_CK (𝛾19) 0.08887 0.04278 / 1225 2.08 .0380

Gender_Student*Pedagogical_Beliefs

(𝛾23) 0.1186 0.05033 / 1280 2.36 .0186


174











Parameter Estimate







= (0.2933 – 0.1734) / 0.2933 = 40.8796%.



2 = (0.004259 - 0.001683) / 0.004259 =

60.4837%.


40.8796% and the teacher-level variables explained about 60.4837% of the teacher-

level variance.

175

Results for Interest in STEM Careers

Multilevel models were built to examine how teachers’ 3D printing integration

levels, STEM integration levels, pedagogical beliefs, value beliefs, and self-efficacy

beliefs in 3D printing integration predict students’ interest in STEM careers while

controlling for student gender and interest in STEM careers pretest scores, which were

included in the model as covariates to be controlled while interpreting the intercept and

the coefficients of other variables.

Baseline model

The first step was to build the baseline model, which predicted a student’ interest

in STEM careers posttest score from the grand mean interest in STEM careers posttest

score of all the teachers’ students. There were no student-level or teacher-level

predictors in this model. The purpose of this model was to examine the ICC and design

effect to determine whether multilevel models were necessary for the analysis. The

meanings of symbols in the model are provided in Table 4-41.





Table 4-41. The meanings of symbols in equations (4-68), (4-69), (4-70)

Symbol Meaning

𝑌𝑖𝑗 The interest in STEM careers posttest score of student i of teacher j.

𝛽0𝑗 The students’ mean interest in STEM careers posttest score for teacher j.


𝛾00 Grand-mean interest in STEM careers posttest score across all teachers.


176


𝜎𝑢02 / (𝜎𝑢0




57.7308. The Design Effect was 3.4289. The ICC was slightly less than 0.05 but the

Design Effect was greater than 2, therefore multilevel modeling analysis was conducted

to examine the influence of student-level and teacher level variables on students’

interest in STEM careers. The next step was to build the student-level models.



𝜎𝑢02 0.01133 0.005009 2.26 / / .0118

𝜎𝜀2 0.2533 0.009419 26.89 / / <.0001

𝛾00 2.4277 0.02524 / 20.8 96.17 <.0001




the impact of student-level variables including students’ interest in STEM careers

pretest scores and gender on students’ interest in STEM careers posttest scores as

fixed effects, indicating the impact (coefficient) of pretest scores and gender did not vary

across teachers. As interest in STEM careers pretest was measured with a scale that

did not contain zero, a score of zero would have no substantive meaning. Students’

pretest scores were also independent of each other’s scores. Therefore, grand mean

centering was used to enable a value of zero to be interpreted meaningfully. The

equations for the random-intercept model with grand-mean centering are as follows:

177

Student Level: 𝑌𝑖𝑗 = 𝛽0𝑗 + 𝛽1𝑗(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟𝑖𝑗 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟)


(4-71)


𝛽1𝑗 = 𝛾10 (4-73)

𝛽2𝑗 = 𝛾20 (4-74)

Combined: 𝑌𝑖𝑗 = 𝛾00 + 𝛾10(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟𝑖𝑗 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟)


(4-75)


regression coefficient that shows the impact of interest in STEM careers pretest scores

on posttest scores across all students of teacher j , 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟𝑖𝑗 is interest in STEM

careers pretest score of student i of teacher j, 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 is the grand mean of interest

in STEM careers pretest scores across teachers, 𝛾10 is the average effect (coefficient)

of pretest scores on posttest scores across all teachers, and 𝛾20 is the average effect

(coefficient) of student gender on interest in STEM careers posttest scores across all

teachers. In the model, student gender was treated as a categorical variable and male =

0 was the reference.

However, after running the random-intercept model in the SAS program, the SAS

log indicated that “Estimated G matrix is not positive definite” and “Asymptotic variance


generalized inverse was used” and the teacher-level intercept variance was zero,

indicating the clustering of students within teachers did not help explain the variance

and students can be considered as independent of each other no matter they had the

same teacher or not. Therefore, a multiple regression model was built to evaluate the

influence of the student and teacher variables on students’ interest in STEM careers

posttest scores.

178

Multiple regression

Before building the multiple regression, multicollinearity was evaluated to

determine whether all the teacher-variables could be included in the model. As shown in

Table 4-43, all the teacher-level variables had a Tolerance of higher than 0.1 and a



Table 4-43. Multicollinearity of variables for interest in STEM careers posttest score


Intercept . 0

Pre_Career 0.94111 1.06257



STEM_Level 0.19071 5.24354













The interactions Pre_Career*Printing_Level, Pre_Career*STEM_Level,

179

Pre_Career*Pedagogical_Beliefs, Pre_Career*Usefulness_Teacher,

Pre_Career*Self_Efficacy_PCK, and Gender_Student* Self_Efficacy_CK were

significant. Therefore, after including the interaction terms and centering the variables at

their grand means, the equation for the final multiple regression model is:

𝑌 = 𝛽0 + 𝛽1(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) + 𝛽2𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡 +

𝛽3(𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 − 𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 ) + 𝛽4 (𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙 − 𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙) + 𝛽5

(𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠 − 𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) + 𝛽6(𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟 −

𝐼𝑛𝑡𝑒𝑟𝑒𝑠𝑡_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛽7(𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟 − 𝐼𝑚𝑝𝑜𝑟𝑡𝑎𝑛𝑐𝑒_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) +

𝛽8(𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟 − 𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) + 𝛽9(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐾) + 𝛽10(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐾) +

𝛽11(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾) + 𝛽12(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐾) + 𝛽13(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝐶𝐾) +

𝛽14(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾) + 𝛽15(𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾 −

𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑇𝑃𝐴𝐶𝐾) + 𝛽16(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) ∗ (𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙 −

𝑃𝑟𝑖𝑛𝑡𝑖𝑛𝑔_𝐿𝑒𝑣𝑒𝑙) + 𝛽17(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) ∗ (𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙 − 𝑆𝑇𝐸𝑀_𝐿𝑒𝑣𝑒𝑙) +

𝛽18(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) ∗ (𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠 − 𝑃𝑒𝑑𝑎𝑔𝑜𝑔𝑖𝑐𝑎𝑙_𝐵𝑒𝑙𝑖𝑒𝑓𝑠) +

𝛽19(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) ∗ (𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟 − 𝑈𝑠𝑒𝑓𝑢𝑙𝑛𝑒𝑠𝑠_𝑇𝑒𝑎𝑐ℎ𝑒𝑟) +

𝛽20(𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟 − 𝑃𝑟𝑒_𝐶𝑎𝑟𝑒𝑒𝑟) ∗ (𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝑃𝐶𝐾)

𝛽21𝐺𝑒𝑛𝑑𝑒𝑟_𝑆𝑡𝑢𝑑𝑒𝑛𝑡 ∗ (𝑆𝑒𝑙_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾 − 𝑆𝑒𝑙𝑓_𝐸𝑓𝑓𝑖𝑐𝑎𝑐𝑦_𝐶𝐾)

(4-76)

In addition to the normality assumption of the student interest in STEM careers

posttest scores and the multicollinearity assumption, multiple regression models also

assume the residuals are normally distributed and there is no autocorrelation in the

residuals. The residuals had a skewness of -0.2879 and a kurtosis of 2.2548. The

skewness and kurtosis of the residuals were between -1 and 1, and -3 and 3

respectively. Therefore, the residual normality assumption was met. The autocorrelation

in the residuals was evaluated with the Durbin-Watson test. A value of 2 of the Durbin-

Watson test indicates no autocorrelation. For this multiple regression model, the Durbin-

180

Watson statistic was 2.02, very close to 2. Therefore, there was no autocorrelation in

the residuals.

The main regression model was significant at F (21, 1264) = 45.67, p < .0001,

with an R2 of 0.4314, indicating 43.14% of the total variance in student interest in STEM

careers posttest scores was associated with the student and teacher variables. The

statistics summary for the multiple regression model can be viewed in Table 4-44. After

grand mean centering, the intercept 𝛽0 = 2.44112, indicating the posttest score was

2.44112 when student gender was male (Gender_Student = 0) and all other variables

were equal to their grand mean.

Except for student STEM career interest pretest scores, none of the other

variables were significant. The slope of student pretest score 𝛽1 = 0.62555 and it was

significant with t (1) = 28.56, p < .0001, when controlling for other variables. Student

STEM career interest pretest score had significant interactions with a few teacher

variables. The slope of the interaction between the pretest score and teachers’ 3D

printing integration level 𝛽16 = -0.06581 and it was significant with t (1) = -2.26, p =

.0243. The slope of the interaction between the pretest score and teachers’ STEM

integration level 𝛽17 = 0.10874 and it was significant with t (1) = 2.07, p = .0386. The

slope of the interaction between the pretest score and teachers’ STEM integration level

𝛽17 = 0.10874 and it was significant with t (1) = 2.07, p = .0386. The slope of the

interaction between the pretest score and teachers’ pedagogical beliefs 𝛽18 = 0.14733

and it was significant with t (1) = 2.82, p = .0048. The slope of the interaction between

the pretest score and teachers’ perceived usefulness of 3D printing integration 𝛽19 = -

0.12311 and it was significant with t (1) = -2.43, p = .0150. The slope of the interaction

181

between the pretest score and teachers’ self-efficacy beliefs in PCK 𝛽20 = 0.09823 and

it was significant with t (1) = 2.30, p = .0217. One score increase in students’ STEM

career interest pretest score decreased the slope of teachers’ 3D printing integration

level by 0.06581, increased the slope of teachers’ STEM integration level by 0.10874,

increased the slope of teachers’ pedagogical beliefs by 0.14733, decreased the slope of

teachers’ perceived usefulness of 3D printing integration by 0.12311, and increased the

slope of teachers’ self-efficacy beliefs in PCK by 0.09823, when controlling for other

variables. Lastly, there was an interaction between student gender and teachers’ self-

efficacy beliefs in CK. The slope of the interaction between student gender and

teachers’ self-efficacy beliefs in CK 𝛽21 = 0.11747 and it was significant with t (1) = 2.72,

p = .0066. As male students were coded as 0 (reference level) and female students

were coded as 1, the slope of teachers’ self-efficacy in CK for female students (one

level increase in student gender) was 0.11747 higher than the slope for male students.

Table 4-44. Multiple regression model summary

Variable DF Parameter Estimate

Standard Error

t Value p

Intercept (𝛽0) 1 2.44112 0.01592 153.37 <.0001

Pre_Career (𝛽1) 1 0.62555 0.02190 28.56 <.0001

Gender_Student (𝛽2) 1 -0.02974 0.02197 -1.35 0.1761

Printing_Level (𝛽3) 1 -0.01301 0.01719 -0.76 0.4493

STEM_Level (𝛽4) 1 -0.00483 0.03511 -0.14 0.8905

Pedagogical_Beliefs (𝛽5) 1 -0.03610 0.04150 -0.87 0.3845

Interest_Teacher (𝛽6) 1 -0.00008138 0.04270 -0.00 0.9985

Importance_Teacher (𝛽7) 1 0.01843 0.04788 0.39 0.7003

Usefulness_Teacher (𝛽8) 1 0.04856 0.06208 0.78 0.4342

Self_Efficacy_TK (𝛽9) 1 -0.01088 0.03313 -0.33 0.7425

182


Variable DF Parameter Estimate

Standard Error

t Value p

Self_Efficacy_PK (𝛽10) 1 -0.04413 0.04467 -0.99 0.3234

Self_Efficacy_CK (𝛽11) 1 -0.00624 0.04671 -0.13 0.8937

Self_Efficacy_TPK (𝛽12) 1 -0.02654 0.05755 -0.46 0.6447

Self_Efficacy_TCK (𝛽13) 1 -0.00334 0.03920 -0.09 0.9322

Self_Efficacy_PCK (𝛽14) 1 -0.01669 0.03931 -0.42 0.6713

Self_Efficacy_TPACK (𝛽15) 1 0.05855 0.07112 0.82 0.4106

Pre_Career*Printing_Level (𝛽16) 1 -0.06581 0.02918 -2.26 0.0243

Pre_Career*STEM_Level (𝛽17) 1 0.10874 0.05252 2.07 0.0386

Pre_Career*Pedagogical_Beliefs (𝛽18) 1 0.14733 0.05220 2.82 0.0048

Pre_Career*Usefulness_Teacher (𝛽19) 1 -0.12311 0.05057 -2.43 0.0150

Pre_Career*Self_Efficacy_PCK (𝛽20) 1 0.09823 0.04275 2.30 0.0217

Gender_Student* Self_Efficacy_CK (𝛽21) 1 0.11747 0.04315 2.72 0.0066

Summary of Results

To provide an overview of the results, the main effects and interactions between

student and teacher variables for each student outcome are provided in Table 4-45. All

the interactions between student variables and teacher variables are organized in Table

4-46 and Table 4-47. As student gender and pretest scores were treated as covariates

and the emphasis was on the relationship between teacher variables and student

outcome variables, the summary of results and the interpretations focused on the main

effects of teacher variables and the interactions between student variables and teacher

variables. The specific results and interpretations with statistics can be referred to in the

results sections for each student outcome variable.

183

Teachers’ 3D printing integration level was not a significant predictor for any of

the student outcomes, however, it had a negative interaction with students’ prior interest

in STEM careers. Therefore, 3D printing integration level had a more positive effect on

students with lower prior interest in STEM careers. Teachers’ STEM integration level

was a positive predictor for students’ math motivation, indicating that when teachers’

STEM integration level increased, their students’ math motivation also increased.

Teachers’ STEM integration level also had a positive interaction with students’ prior

interest in STEM careers and a positive interaction with student gender in terms of

students’ science motivation. Therefore, teachers’ STEM integration level had a more

positive effect on students with higher prior interest in STEM careers and STEM

integration level had a more positive effect on female students’ science motivation.

Although teachers’ pedagogical beliefs did not significantly predict any of the

student outcomes, it negatively interacted with students’ prior science motivation,

positively interacted with students’ prior interest in STEM careers, and positively

interacted with student gender in terms of students’ 21st century skills. Teachers’

pedagogical beliefs had a more positive effect for students with lower prior science

motivation, students with higher prior interest in STEM careers, and female students in

terms of 21st century skills.

Teachers’ interest in and perceived importance of 3D printing integration were

not significant predictors for any of the student outcomes. However, teachers’ perceived

usefulness of 3D printing integration negatively predicted students’ 21st century skills

and it negatively predicted students’ science motivation with approaching significance.

Therefore, when teachers’ perceived usefulness of 3D printing integration increased,

184

students’ 21st century skills decreased, and students’ science motivation decreased

(approaching significance). Teachers’ perceived usefulness of 3D printing integration

was a negative predictor for students’ math motivation, and it had a negative interaction

with students’ prior math motivation, indicating that when teachers’ perceived

usefulness of 3D printing integration increased, students’ math motivation increased if

students’ prior math motivation was at the grand mean, and teachers’ perceived

usefulness of 3D printing integration had a more positive effect on students with lower

prior math motivation. Teachers’ perceived usefulness of 3D printing integration also

had a negative interaction with students’ prior interest in STEM careers, indicating that

teachers’ perceived usefulness of 3D printing integration had a more positive effect on

students with lower prior interest in STEM careers. Lastly, teachers’ perceived

importance of 3D printing integration had a positive interaction with student gender in

terms of students’ technology/engineering motivation. Therefore, teachers’ perceived

importance of 3D printing integration had a more positive effect on female students in

terms of their technology/engineering motivation.

Teachers’ self-efficacy in PCK positively predicted students’ math motivation with

approaching significance while teachers’ self-efficacy in TK negatively predicted

students’ technology/engineering motivation with approaching significance, indicating

when teachers’ self-efficacy in PCK increased, students’ math motivation increased

(approaching significance), however, when teachers’ self-efficacy in TK increased,

students’ technology/engineering motivation decreased (approaching significance). In

addition to these main effects, there were varied interactions between student variables

(pretest scores and student gender) and teachers’ self-efficacy beliefs.

185

With regard to the interactions with students’ pretest scores, students’ prior 21st

century skills had a positive interaction with teachers’ self-efficacy in CK, students’ prior

math motivation had a positive interaction with teachers’ self-efficacy in TPK, and

students’ prior interest in STEM careers had a positive interaction with teachers’ self-

efficacy in PCK. Therefore, teachers’ self-efficacy in CK had a more positive effect on

students with higher prior 21st century skills, teachers’ self-efficacy in TPK had a more

positive effect on students with higher prior math motivation, and teachers’ self-efficacy

in PCK had a more positive effect on students with higher prior interest in STEM

careers. However, students’ prior math motivation had a negative interaction with

teachers’ self-efficacy in PK, indicating teachers’ self-efficacy in PK had a more positive

effect on students with lower prior math motivation.

Student gender and teachers’ self-efficacy beliefs also had varied interactions.

Teachers’ self-efficacy in PK positively predicted students’ technology/engineering

motivation (coefficient = 0.03467), however, student gender and teachers’ self-efficacy

in PK had a negative interaction (coefficient = -0.1452), indicating teachers’ self-efficacy

in PK had a more positive effect on male students. After accounting for the negative

interaction, the coefficient for male students was 0.03467 and the coefficient for female

students was -0.11053. Therefore, teachers’ self-efficacy in PK positively predicted

male students’ technology/engineering motivation but negatively predicted female

students’ technology/engineering motivation, indicating when teachers’ self-efficacy in

PK increased, male students’ technology/engineering motivation increased, however,

female students’ technology/engineering motivation decreased. The interaction between

student gender and teachers’ self-efficacy in PK was also negative for students’ math

186

motivation, indicating teachers’ self-efficacy in PK had a more positive effect on male

students. Although teachers’ self-efficacy in PK was a positive predictor for male

students’ technology/engineering motivation and had a more positive effect on male

students’ math motivation, teachers’ self-efficacy in CK had a more positive effect on

female students’ interest in STEM careers since there was a positive interaction

between teachers’ self-efficacy in CK and student gender.

Table 4-45. Summary of results for student outcomes Student outcome

Main effects Interactions

Science motivation

Teachers’ perceived usefulness of 3D printing integration was a negative predictor with approaching significance. When teachers’ perceived usefulness of 3D printing integration increased, students’ science motivation decreased (approaching significance).

Students’ science motivation pretest scores and teachers’ pedagogical beliefs had a negative interaction. Teachers’ pedagogical beliefs had a more positive effect on students with lower prior science motivation.

Other predictors were non-significant. Student gender and teachers’ STEM integration level had a positive interaction. Teachers’ STEM integration level had a more positive effect on female students.

Technology/ Engineering motivation

Teachers’ self-efficacy in TK was a negative predictor with approaching significance. When teachers’ self-efficacy in TK increased, students’ technology/engineering motivation increased.

Student gender and teachers’ perceived importance of 3D printing integration had a positive interaction. Teachers’ perceived importance of 3D printing integration had a more positive effect on female students.

Teachers’ self-efficacy in PK was a positive predictor when student gender was male. When teachers’ self-efficacy in PK increased, male students’ technology/engineering motivation increased.

Student gender and teachers’ self-efficacy in PK had a negative interaction. Teachers’ self-efficacy in PK had a more positive effect on male students.

Other predictors were non-significant.

187

Table 4-45. Continued Student outcome


Math motivation

STEM integration level was a positive predictor. When teachers’ STEM integration level increased, students’ math motivation increased.

Students’ math motivation pretest scores and teachers’ perceived usefulness of 3D printing integration had a negative interaction. Teachers’ perceived usefulness of 3D printing integration had a more positive effect on students with lower prior math motivation.

Teachers’ perceived usefulness of 3D printing integration was a significant and negative predictor when students’ math motivation pretest scores were at the grand mean. When teachers’ perceived usefulness of 3D printing integration increased, students’ math motivation increased when students’ math motivation pretest scores were at the grand mean.

Students’ math motivation pretest scores and teachers’ self-efficacy in PK had a negative interaction. Teachers’ self-efficacy in PK had a more positive effect on students with lower prior math motivation.

Teachers’ self-efficacy in PCK was a positive predictor with approaching significance. When teachers’ self-efficacy in PCK increased, students’ math motivation also increased (approaching significance).

Students’ math motivation pretest scores and teachers’ self-efficacy in TPK had a positive interaction. Teachers’ self-efficacy in TPK had a more positive effect on students with higher prior math motivation.

Other predictors were non-significant.

Student gender and teachers’ self-efficacy in PK had a negative interaction. Teachers’ self-efficacy in PK had a more positive effect on male students.

21st century skills

Teachers’ perceived usefulness of 3D printing integration was a negative predictor. When teachers’ perceived usefulness of 3D printing integration increased, students’ 21st century skills decreased.

Students’ 21st century skills pretest scores and teachers’ self-efficacy in CK had a positive interaction. Teachers’ self-efficacy in CK had a more positive effect on students with higher prior 21st century skills.

Other predictors were non-significant. Student gender and teachers’ pedagogical beliefs had a positive interaction. Teachers’ pedagogical beliefs had a more positive effect on female students.

188

Table 4-45. Continued Student outcome


Interest in STEM careers

None of the predictors were significant.

Students’ interest in STEM careers pretest scores had negative interactions with teachers’ 3D printing integration levels and perceived usefulness of 3D printing integration. Teachers’ 3D printing integration levels and perceived usefulness of 3D printing integration had a more positive effect on students with lower prior interest in STEM careers.

Students’ interest in STEM careers pretest scores had positive interactions with teachers’ STEM integration levels, pedagogical beliefs, and self-efficacy in PCK. Teachers’ STEM integration levels, pedagogical beliefs, and self-efficacy in PCK had a more positive effect on students with higher prior interest in STEM careers.

Student gender and teachers’ self-efficacy in CK had a positive interaction. Teachers’ self-efficacy in CK had a positive effect on female students.

Note: Male students were coded as 0 and female students were coded as 1.

Table 4-46. Interactions between student pretest scores and teacher variables

Pre_Science Pre_TechEngi Pre_Math Pre_21st Pre_Career

Printing_Level -0.06581 (.0243)

STEM_Level 0.10874 (.0386)

Pedagogical_Beliefs -0.1291 (.0030)

0.14733 (.0048)

Importance_Teacher

Usefulness_Teacher -0.1199

(.0016)

-0.12311

(.0150)

Self_Efficacy_PK -0.08292

(.0298)

189


Pre_Science Pre_TechEngi Pre_Math Pre_21st Pre_Career

Self_Efficacy_CK 0.08887

(.0380)

Self_Efficacy_TPK 0.1096

(.0039)

Self_Efficacy_PCK 0.09823

(.0217)

Note: The number in each cell is the interaction coefficient with the p value in the bracket.

Table 4-47. Interactions between student gender and teacher variables

Gender

(Science)

Gender

(TechEngi)

Gender

(Math)

Gender

(21st)

Gender

(Career)

Printing_Level

STEM_Level 0.09404

(.0133)

Pedagogical_Beliefs 0.1186

(.0186)

Importance_Teacher 0.1189

(.0191)

Usefulness_Teacher

Self_Efficacy_PK -0.1452

(.0292)

-0.2276

(.0002)

Self_Efficacy_CK 0.11747

(.0066)

Self_Efficacy_TPK

Self_Efficacy_PCK

Note: Gender (Science), Gender (TechEngi), Gender (Math), Gender (21st), and Gender (Career) indicate interactions between student gender and teacher variables for science motivation, technology/engineering motivation, math motivation, 21st century skills, and interest in STEM careers. The number in each cell is the interaction coefficient with the p value in the bracket.

190

CHAPTER 5 DISCUSSIONS

The purpose of this study was to examine the relationship between teachers’

beliefs, their 3D printing integration in science classrooms, and students’ STEM

motivation, 21st century skills, and interest in STEM careers. The research questions

were:

How are teachers’ beliefs correlated with their 3D printing technology integration in the science classrooms?

How do teachers’ beliefs and their 3D printing technology integration in the science classrooms predict students’ STEM motivation, 21st century skills, and interest in STEM careers?

This chapter begins with reviewing the limitations and delimitations of this study

and then the results are interpreted. First, the relationships between teacher beliefs and

3D printing integration are discussed, and then the relationships between teacher

beliefs, 3D printing integration, and student outcome variables are discussed. After

discussing the results for each of the student outcome variables, the themes and

patterns are discussed. Finally, implications for practice and future research are

proposed.

Limitations and Delimitations

Like many other studies, this study had a few limitations and delimitations that

have to be clarified. The limitations were potential issues that I was not able to address

in this study and the delimitations elucidated the boundaries of this study and the

situations or areas this study may not apply.

Limitations

First, this study analyzed teachers’ 3D printing technology integration through

their lesson plans without classroom observation. Lesson plan analysis may not exactly

191

reflect teachers’ actual implementation. There might be some discrepancy between the

lesson plans and the actual classes. Moreover, teachers may make adjustments as they

progressed with different classes and students may have had similar classes but with

some differences. Therefore, the lesson plans may not have completely captured these

nuances.

Second, most of the lessons were short; usually a few classes with the

implementation of 3D printing technology. Students’ learning outcomes including their

STEM motivation, 21st century skills, and interest in STEM careers may not have

significant change with short interventions. Moreover, the S-STEM survey with 5-point

or 4-point Likert scales may not be sensitive enough to detect the changes in students’

learning outcomes associated with the 3D printing technology activities even if there

were trivial changes.

Third, the 3D printing integration levels defined and coded in this study may not

have been sensitive enough to capture all possible differences among various

implementations of 3D printing technology in the science classrooms. The 3D printing

integration level was an important predictor but may not reflect all aspects that can

potentially influence students.

Finally, the teacher beliefs survey was administered after teachers’ 3D printing

technology integration in their classrooms. Therefore, the correlation between teacher

beliefs and their 3D printing technology integration practice can only account for the

relationship with teacher beliefs after the 3D printing integration. Although teacher

beliefs may not change significantly in a short time, the beliefs may have evolved to

192

some extent during their 3D printing integration. It was unknown how teacher beliefs

prior to or during the integration would have influenced their 3D printing integration.

Delimitations

First, this study focused on 3D printing integration in K-12 science classrooms

within the context of paleontology. It may not account for 3D printing technology

integration in other disciplines or in contexts other than paleontology.

Second, teacher beliefs only included pedagogical beliefs, self-efficacy in

technology integration, and technology value beliefs, which are salient factors that may

influence teachers’ technology integration practice. Some more fundamental beliefs

such as epistemological beliefs were not included in this study because teachers’

epistemological beliefs are beliefs about the nature of knowledge and learning, which

are correlated with teachers’ pedagogical beliefs but are not directly correlated with their

technology integration practice (Kim et al., 2013). Therefore, this study cannot account

for the relationship between teachers’ 3D printing integration and their epistemological

beliefs, which may potentially have relationships with their technology integration

practice.

Third, this study was conducted with teachers who voluntarily participated in the

iDigFossils project. It was assumed that these teachers were interested in integrating

3D printing technology integration in their science classrooms to some extent, as part of

the reason they participated in the project. Thus, the findings may not apply to teachers

who are not interested at all in 3D printing technology integration.

Lastly, this study utilized correlational and multilevel modeling analysis to

examine the relationship between teacher beliefs, 3D printing technology integration,

and students’ STEM motivation, interest in STEM careers, and 21st century skills.

193

These types of analyses could not determine causal relationships. Therefore, this study

accounted for the correlational or regressional relationship between the variables but

could not demonstrate the influence of teacher beliefs on 3D printing integration, and

the influence of 3D printing integration on student learning outcomes.

Relationships between Teacher Beliefs and 3D Printing Integration

Surprisingly, there were no significant correlations between teacher beliefs and

their 3D printing integration level and STEM integration level except that teachers’ self-

efficacy in PCK significantly and moderately correlated with STEM integration level in a

negative direction. This result countered previous research which indicated important

relationships between teacher beliefs and their technology integration. Tondeur et al.

(2017) suggested that teachers with constructivist beliefs are more likely to integrate

technology to facilitate student-centered learning. In addition, teachers’ self-efficacy is a

key predictor of their technology integration in the classrooms (Albion, 1999; Ertmer &

Ottenbreit-Leftwich, 2010; Gonzales, 2013; Haight, 2011; Heineman, 2018; Li et al.,

2018; Manglicmot, 2015; Marcinkiewicz, 1994; Tweed, 2013). Moreover, teachers’

technology value beliefs are critical for teachers to determine whether and how they will

integrate technology in the classrooms (Ertmer et al., 1999; Ertmer et al., 2012; Ertmer

& Ottenbreit-Leftwich, 2010; Mueller et al., 2008; Vongkulluksn et al., 2018; Wozney et

al., 2006).

The discrepancy between this result and previous research findings was

probably due to the barriers that teachers encountered when integrating 3D printing into

their science classrooms. As Ertmer et al. (2012) indicated, external barriers such as

lack of resources can influence how teachers actually integrate technology, thus

teachers’ technology integration may not be consistent with their beliefs. As reported in

194

the open responses of the teacher beliefs survey in this study, the teachers

encountered a few barriers including logistical and technical issues, an insufficient

number of printers and related resources, the lack of time, and constraints in the

curriculum which made it difficult to fit 3D printing integrated lessons into the current

curriculum. These barriers may have deviated teachers from how they would have

actually wanted to integrate 3D printing in their science classrooms.

It is also necessary to note that the duration of the teachers’ 3D printing

integration was relatively short (mostly a few classes) compared to the durations of

many of the previous studies on teacher beliefs and technology integration in which the

duration of integration was from a semester to a few years (e.g., Ertmer et al., 2012;

Ertmer & Ottenbreit-Leftwich, 2010; Tondeur et al., 2017; Vongkulluksn et al., 2018).

Teacher beliefs and technology integration may be more aligned with each other after a

long time of teaching practice, whereas the correlation between teacher beliefs and

technology integration may not emerge within a short period of time integrating a new

technology (3D printing) in a new context such as paleontology.

Relationships between Teacher Variables and Student Outcomes

In this section, the results of the relationship between teacher variables and

student outcome variables including math motivation, technology/engineering

motivation, 21st century skills, and interest in STEM careers respectively were

discussed. Teacher variables included teachers’ 3D printing integration levels, STEM

integration levels, and teacher beliefs. Teacher beliefs consisted of pedagogical beliefs,

value beliefs which consisted of teachers’ interest in, perceived importance, and

perceived usefulness of 3D printing integration, and self-efficacy beliefs in 3D printing

195

integration, which was operationalized as self-efficacy in TK, PK, CK, TPC, TCK, PCK,

and TPACK.

Relationships with Science Motivation

It was surprising that none of the 3D printing integration level and STEM

integration level had significant relationships with students’ science motivation. This was

similar to Swayne’s (2017) finding that technology integration levels did not have a

statistically significant relationship with student engagement in middle-school English

Language/Arts and mathematics classrooms that used 1:1 tablet technology. Previous

research indicated that technology integration in science classes can increase students’

science motivation (e.g., Xie & Reider, 2014) and STEM integration can also enhance

student motivation (Honey, 2014), but there was little empirical evidence about how

different 3D printing integration levels or STEM integration levels may influence student

motivation. Since 3D printing was integrated into science classes, it was assumed that

different levels of 3D printing integration and/or STEM integration may have different

influences on students’ science motivation. This may be due to the short duration of 3D

printing integration as most of the teachers only integrated 3D printing for a few classes.

The effect of 3D printing integration may need a longer time to emerge.

It was also possible that a higher level of 3D printing integration and/or STEM

integration may benefit students but may not always be good for students. A higher

level of STEM integration requires students’ sufficient knowledge in the individual

subjects in order to successfully connect the concepts and ideas across STEM

disciplines (Pearson, 2017). With regards to the non-significant relationship between 3D

printing integration levels and students’ science motivation, similar to what Nemorin and

Selwyn (2017) found in their study, students may encounter difficulties when using 3D

196

printing software to create models, which may make them feel frustrated. As the

interaction between teachers and students, the problems encountered, and students’

responses could not be identified through the lesson plans, future studies are highly

recommended to conduct classroom observations to obtain more comprehensive data.

Although teachers’ STEM integration level was not a significant predictor, it had a

positive interaction with student gender. For both male students (student gender = 0)

and female students (student gender = 1), STEM integration level had a positive

relationship with student gender and STEM integration level had a stronger effect on

female students’ science motivation. This finding is interesting as research suggested

that male students are typically more interested and motivated to learn STEM than

female students (Wang & Degol, 2013). Although student gender was not a significant

predictor of students’ science motivation in this study, the interaction suggested that as

STEM integration level increases, female students’ science motivation may have higher

increase than male students.

In addition, it was found that teachers’ perceived usefulness of 3D printing

integration had a negative relationship with students’ science motivation with

approaching significance. When teachers’ perceived usefulness of 3D printing

integration increased, students’ science motivation decreased. This result was

unexpected, and it countered previous research that teachers’ technology value beliefs

may positively correlate with their technology integration (Ottenbreit-Leftwich et al.,

2010), which may further have positive influences on student motivation. It is possible

that even if a teacher perceived 3D printing integration as useful, he or she may not be

able to integrate 3D printing in effective ways, and external barriers can impact how

197

they integrate, thus students’ science motivation may not increase or even decrease.

This finding suggested that although teachers’ perceived usefulness of 3D printing may

potentially have positive impacts on students’ science motivation, the ways teachers

actually integrate 3D printing to facilitate science learning is probably more important.

None of teachers’ pedagogical beliefs and self-efficacy beliefs were significant

predictors of students’ science motivation. Although teacher beliefs may be associated

with students’ learning motivation, many external barriers can impact the actual

technology integration, which eventually impact student motivation. The barriers include

but not limited to external barriers such as the lack of resources or insufficient of time

and technical support (Ertmer et al., 2012), which were also demonstrated in the

teachers’ open responses on the challenges they encountered when integrating 3D

printing into their science classrooms.

Although no significant relationship between teachers’ pedagogical beliefs and

students’ science motivation was identified, there was a negative interaction between

students’ prior science motivation and teachers’ pedagogical beliefs. For students with

higher prior science motivation, teachers’ pedagogical beliefs had less effect, but for

students with lower prior science motivation, teachers’ pedagogical beliefs had a

stronger effect. Similar to what Ross (1994) suggested that teachers with higher self-

efficacy attend to the lower ability students more closely, and lower ability students

typically have lower motivation, teachers with higher pedagogical beliefs may be more

student-centered and pay more attention to students with lower motivation when they

teach. This finding suggested that in order to increase students’ science motivation

198

especially students with lower prior science motivation, teachers probably need a higher

level of beliefs in student-centered learning.

Relationships with Technology/Engineering Motivation

It was surprising that 3D printing integration level was not a significant predictor

for students’ technology/engineering motivation, which countered the general findings of

previous research. Previous research indicated that technology integration has positive

impacts on student motivation (e.g., Francis, 2017; Heafner, 2004; Xie & Reider, 2014).

However, there was little research on how different levels of technology integration

influence student motivation. The non-significant effect may due to that the 3D printing

integration levels may not be sensitive enough to account for all the differences in

teachers’ implementation of 3D printing in the classrooms. Higher levels of 3D printing

integration might be beneficial for students, but the higher levels have more demand on

students’ knowledge and skills, which might make some students feel frustrated. In

addition, teachers’ 3D printing integration levels negatively interacted with students’

prior interest in STEM careers, indicating higher 3D printing integration levels may have

more influence on students with lower prior interest in STEM careers. This may due to

that higher 3D printing integration levels engage students in deeper learning with 3D

printing technology, which may increase students’ interest in STEM careers especially

for students with lower prior interest.

For students’ technology/engineering motivation, teachers’ self-efficacy in PK

was a significant predictor and it had a negative interaction with student gender.

Teachers’ self-efficacy in PK positively predicted both male students’ and female

students’ technology/engineering motivation, which was consistent with previous

research that teachers’ self-efficacy beliefs have a positive relationship with student

199

motivation (Zee & Koomen, 2016). Furthermore, teachers’ self-efficacy in PK had a

stronger effect on male students’ technology/engineering motivation. One explanation

was that teachers with higher self-efficacy in PK may implement more student-centered

learning activities and male students may be easier to be engaged in the student-

centered learning activities as male students are typically more motivated to learn

technology/engineering (Wang & Degol, 2013).

Although teachers’ self-efficacy in PK was a positive predictor, teachers’ self-

efficacy in TK was a negative predictor with approaching significance for students’

technology/engineering motivation. These findings suggested that teachers’ self-efficacy

in PK instead of TK was of more importance to potentially increase students’

technology/engineering motivation. As previous research on the relationship between

teachers’ self-efficacy and student learning outcomes focused on teachers’ general

teaching self-efficacy and there was little research on how teachers’ self-efficacy in PK

and TK may have different correlations on student motivation, the findings can only be

conjectured without an adequate empirical or theoretical base. One explanation was

that teachers’ self-efficacy in TK may not be consistent with their actual integration of

3D printing in the classrooms due to some external barriers, thus may not necessarily

have positive relationships with students’ technology/engineering motivation.

In addition to teachers’ self-efficacy, it was also found that student gender was a

significant predictor and male students had higher technology/engineering motivation

than female students when teachers’ perceived importance and self-efficacy in PK were

at the average scores of all the teachers. However, the direction of the significance of

gender was conditional and gender could be a negative or positive predictor when

200

teachers’ perceived importance of 3D printing integration and self-efficacy in PK

change. Since the direction of the significance of gender was conditional, it was

undetermined whether student gender was a positive or negative predictor in general,

thus it was unknown whether male or female students had higher

technology/engineering motivation. Although research indicated male students are more

motivated to learn STEM and may have higher STEM motivation (Wang & Degol, 2013),

the findings on the relationship between student gender and their

technology/engineering motivation cannot confirm the findings of previous research.

There might be other factors that were more influential on students’

technology/engineering motivation.

Relationships with Math Motivation

It was found that teachers’ STEM integration level significantly and positively

predicted students’ math motivation while controlling for other variables. In the science

classrooms, teachers’ integrated science, 3D printing technology, math, and

engineering at different levels. All of the teachers integrated 3D printing technology in

their science classrooms, but some teachers also integrated math or both math and

engineering with 3D printing integrated science classes. This result suggested that

when math was integrated with other subjects in STEM, higher STEM integration level

contributed to students’ math motivation. This finding was consistent with previous

research that in general integrated curriculum or instruction can increase students’

learning motivation (Bragow, Gragow & Smith, 1995; Gutherie, Wigfield, & VonSecker,

2000), and specifically, STEM integration can enhance students’ interest in STEM

(Mustafa et al., 2016; Riskowski, Todd, Wee, Dark, & Harbor, 2009) and STEM learning

motivation (Laboy-Rush, 2011; Wang, Moore, Roehrig, & Park, 2011).

201

Integrating math with other STEM subjects enabled students to learn math in

more interesting and connected ways. Students may not even realize they were doing

math while engaging in science learning activities supported by technologies. The

integration of math and other STEM subjects also allowed students to make

connections between different subjects which may potentially increase their knowledge

and skills in math and make them feel more confident and motivated to learn math. As

Furner and Kumar (2007) stated, STEM integration makes learning “more relevant, less

fragmented, and more stimulating experiences for learners” (p.186), which potentially

increases students’ STEM motivation. Although math has been a challenging subject for

many students and math anxiety has been a long-lasting educational issue (Ashcraft &

Ridley, 2005; Maloney & Beilock, 2012; Ramirez, Gunderson, Levine, & Beilock, 2013;

Wigfield & Meece, 1988) that can decrease students’ motivation to learn math, the

finding of this study was encouraging that integrating math with other STEM subjects

might be helpful to increase students’ motivation to learn math.

Although STEM integration level was a significant predictor of students’ math

motivation, this study did not find a significant relationship between teachers’ 3D printing

integration level and students’ math motivation. The coding of teachers’ 3D printing

integration level did not specifically include the component of math learning, so

teachers’ 3D printing integration levels may not have a significant relationship with

students’ math motivation. This finding suggested that students’ math motivation may

not have a direct relationship with the integration of just technology in science classes

and math has to be specifically designed and integrated with other STEM subjects in

order to increase students’ math motivation.

202

This study found that teachers’ pedagogical beliefs were not significant for

students’ math motivation. Teachers’ pedagogical beliefs may not necessarily determine

how teachers integrate technology and there might be some other factors that impact

how teachers actually taught such as limited resources and time (Ertmer et al., 2012).

As corroborated by the open responses in the teacher beliefs survey in this study, the

lack of 3D printers and related resources, the large amount of time required to print 3D

objects, and the time required to integrate 3D printing technologies into the current

curriculum were some of the major challenges encountered by the teachers. Although

teachers may have intended to integrate 3D printing technologies to facilitate student-

centered learning, the external barriers could have impacted how they actually

integrated 3D printing technologies into their classes. Thus, it was not surprising that

teachers’ pedagogical beliefs did not have a significant relationship with students’ math

motivation.

Among teachers’ value beliefs, teachers’ perceived usefulness of 3D printing

integration significantly but negatively predicted students’ math motivation when

students’ math motivation pretest scores were equal to or higher than the average score

of all students’ pretest scores. For students whose prior math motivation was higher

than the average, when teachers’ perceived usefulness was higher, students’ math

motivation was lower. However, because students’ math motivation pretest scores

negatively interacted with teachers’ perceived usefulness of 3D printing integration, for

students who had lower prior math motivation, the effect of teachers’ perceived

usefulness of 3D printing integration was stronger; for students who had higher prior

math motivation, the effect of teachers’ perceived usefulness of 3D printing integration

203

was weaker. It suggested that increasing teachers’ perceived usefulness of 3D printing

integration may be more beneficial for students who had relatively lower prior math

motivation. Although research indicated that teachers’ technology value beliefs are

critical factors for their technology integration, and teachers with high value beliefs even

tend to integrate the technology when faced with external barriers (Ertmer et al., 2012;

Ottenbreit-Leftwich et al., 2010; Snoeyink & Ertmer, 2001), there was little evidence on

the relationship between teachers’ technology value beliefs and student motivation, and

even little was research on how teachers’ technology value beliefs impact students with

individual differences such as prior math motivation. Future research may further

investigate the relationship between teachers’ technology value beliefs and how and

why teachers’ value beliefs may be more influential on students with lower prior

motivation.

Most aspects of teachers’ self-efficacy beliefs did not have significant

relationships with students’ math motivation, however, teachers’ self-efficacy beliefs in

PCK was a positive predictor approaching significance. The higher teachers’ self-

efficacy beliefs in PCK, the stronger students’ math motivation. Although there was little

literature regarding the relationship between teachers’ self-efficacy in PCK and

students’ learning motivation, a meta-analysis study conducted by Zee and Koomen

(2016) indicated that teachers’ self-efficacy has positive relationships with students’

academic achievement and motivation as shown by all the qualified studies identified by

the authors. Teachers’ self-efficacy not only influenced students' learning performance

(e.g., Ashton & Webb, 1986; Brookover et al., 1979; Brophy & Evertson, 1977; Hoy &

Davis, 2005; Shahzad & Naureen, 2017) but also students’ learning motivation (e.g.,

204

Eccles & Wigfield, 1985; Lazarides et al., 2018; Mojavezi & Tamiz, 2012; Pan, 2014;

Schiefele & Schaffner, 2015). The finding of this study suggested that increasing

teachers’ self-efficacy in PCK may potentially increase students’ math motivation.

This study also found that student gender negatively interacted with teachers’

self-efficacy beliefs in PK, indicating teachers’ self-efficacy beliefs in PK had a stronger

effect on male students and a lesser effect on female students. Additionally, students’

math motivation pretest scores had a negative interaction with teachers’ self-efficacy

beliefs in PK but a positive interaction with teachers’ self-efficacy beliefs in TPK, which

indicated that for students whose prior math motivation was higher, teachers’ self-

efficacy beliefs in PK had less effect but teachers’ self-efficacy beliefs in TPK had

stronger effect, comparing to students whose prior math motivation was lower, and vice

versa.

It suggested that teachers’ self-efficacy beliefs in PK may have more influence on

students with lower prior math motivation and teachers’ self-efficacy beliefs in TPK may

have more influence on students with higher prior math motivation. Although there was

no direct causal relationship between teachers’ self-efficacy beliefs and students’ math

motivation, teachers’ self-efficacy beliefs may influence how they teach and even the

learning environment in the classrooms. This finding suggested that for students with

relatively low prior math motivation, teachers probably need to focus more on the

pedagogy they use to teach math; for students with relatively high prior math motivation,

teachers probably need to focus more on effective ways that they use technology to

teach math.

205

Relationships with 21st Century Skills

In this study, 3D printing integration level and STEM integration level were not

significant predictors for students’ 21st century skills either, which was not consistent

with previous research. In Trust and Maloy’s (2017) study with teachers who integrated

3D printing into their classrooms, the teachers reported that 3D printing integration

enhanced their students’ creativity, technology literacy, problem-solving, self-directed

learning, critical thinking, and perseverance, essential 21st century skills as the authors

summarized. However, the findings in Trust and Maloy’s (2017) study were from

teachers’ report, which may not objectively reflect the real influence of 3D printing

integration. Nevertheless, research suggested that STEM integration can also

contribute to students’ 21st century skills (Honey et al., 2014). The non-significant

effects of 3D printing and STEM integration levels in this study were probably due to

that teachers’ 3D printing and STEM integration levels may not necessarily influence all

aspects included in the 21st century skills scale developed by Unfried et al. (2015), such

as items on students’ self-regulation in learning and leadership.

Consistent with the relationship between teachers’ perceived usefulness of 3D

printing integration and students’ science motivation, teachers’ perceived usefulness of

3D printing integration was also a negative predictor for students’ 21st century skills.

This finding was unexpected because previous research indicated that teachers’

technology value beliefs are critical for them to effectively integrate technology

(Ottenbreit-Leftwich et al., 2010), which may positively influence students. The negative

relationship between teachers’ perceived usefulness of 3D printing technology and

students’ 21st century skills may due to the limitations of the measurement. The rating

scale for 21st century skills developed by Unfried et al. (2015) included some

206

components that may not be influenced by teacher beliefs or teachers’ 3D printing

integration. For instance, the rating scale included items such as “I am confident I can

manage my time wisely when working on my own”, “When I have many assignments, I

can choose which ones need to be done first”, etc. These components may not be

influenced by teacher beliefs or how teachers integrate 3D printing technology.

Student gender was a significant predictor for students’ 21st century skills and

there was a positive interaction between student gender and teachers’ pedagogical

beliefs. When teachers’ pedagogical beliefs were average or above average, student

gender had a positive effect on students’ 21st century skills. It suggested that when

teachers’ pedagogical beliefs were relatively high, female students had higher 21st

century skills than male students. In the rating scale of students’ 21st century skills,

there were a few items regarding students’ self-regulation. For instance, “I am confident

I can set my own learning goals”, “I am confident I can manage my time wisely when

working on my own”, and “When I have many assignments, I can choose which ones

need to be done first”. Research indicated that female students have better self-

regulation than male students (Matthews, Ponitz, & Morrison, 2009). The items on self-

regulation might have contributed to female students’ higher ratings on the scale of the

21st century skills. However, there was little empirical evidence or theoretical

foundations to explain why teachers’ pedagogical beliefs had more impact on female

students’ 21st century skills. It is possible that female students were more motivated to

learn when teachers were student-centered.

Although teachers’ self-efficacy in CK was not a significant predictor, there was a

positive interaction between students’ prior 21st century skills and teachers’ self-efficacy

207

beliefs in CK. Teachers’ self-efficacy beliefs in CK had stronger effects for students who

had higher prior 21st century skills and vice versa. It suggested that teachers’ self-

efficacy in CK was more influential and may become a significant predictor for students

with higher prior 21st century skills. There was little empirical evidence or theoretical

foundations to explain why teachers’ self-efficacy in CK had more influence on students

with higher prior 21st century skills. It is possible that students with higher prior 21st

century skills were more motivated to learn and have better learning skills, which may

help them benefit more from teachers with higher self-efficacy in CK who probably have

more content knowledge as well, thus to have more improvement in their 21st century

skills than students with lower 21st century skills.

Relationships with Interest in STEM Careers

Surprisingly, 3D printing integration and STEM integration level were both non-

significant for students’ interest in STEM careers. Although there was little research

regarding the influence of 3D printing integration on students’ interest in STEM careers,

Xie and Reider (2014) found the integration of innovative technologies can enhance

students’ motivation for science career, which is related to students’ interest in science

career. Honey et al. (2014) suggested integrated STEM education can facilitate

students’ interest development. However, there was little research on how different

levels of 3D printing integration and STEM integration may influence students’ interest

in STEM careers.

The non-significant findings on 3D printing integration level and STEM integration

level indicated that higher level of 3D printing integration level or STEM integration level

may not necessarily be better for students. As discussed previously, a higher level of 3D

printing integration might cause higher challenges on both teachers and students

208

(Nemorin & Selwyn, 2017) and higher level of STEM integration requires students to

have sufficient knowledge in each subject (Pearson, 2017), thus higher levels of 3D

printing integration and STEM integration may not contribute to students’ interest in

STEM careers. Additionally, teachers may have not made specific connections between

the 3D printing integration and STEM career pathways, so students may not make

connections between what they learned in the 3D printing integrated classes and the

possible future STEM careers, thus 3D printing integration and STEM integration levels

were not significant predictors for students’ interest in STEM careers.

In this study, none of the teacher variables were significant predictors of

students’ interest in STEM careers. However, there were a few interactions between

students’ prior interest in STEM careers and some of the teacher variables and also

interaction between student gender and teachers’ self-efficacy beliefs in CK. There were

positive interactions between students’ prior interest in STEM careers and teachers’

STEM integration levels, pedagogical beliefs, and self-efficacy beliefs in PCK

respectively. There was a negative interaction between students’ prior interest in STEM

careers and teachers’ 3D printing integration levels and a negative interaction between

students’ prior interest in STEM careers and teachers’ perceived usefulness of 3D

printing integration. There was a positive interaction between student gender and

teachers’ self-efficacy in CK.

These interactions suggested that teachers’ STEM integration levels,

pedagogical beliefs, and self-efficacy beliefs in PCK may have stronger positive

influences on students with higher prior interest in STEM careers. However, teachers’

3D printing integration levels and perceived usefulness of 3D printing integration may

209

have stronger positive influences on students with lower prior interest in STEM careers.

Lastly, teachers’ self-efficacy in CK may have stronger positive influences on female

students. Although previous research found correlations between teacher beliefs and

students’ affective learning outcomes (Zee & Koomen, 2016) in general, there was little

research on how teachers’ pedagogical beliefs, technology value beliefs including

interest, perceived importance, and perceived usefulness, and self-efficacy beliefs in

TK, PK, CK, TPK, TCK, PCK, and TPACK might have relationships with students’

affective learning outcomes including interest in STEM careers, and also how these

teacher beliefs may interact with student gender or students’ prior interest. There were

no empirical evidence or theoretical foundations to explain these findings and future

research would be necessary to further explore and explain these relationships.

Implications

Although extant literature suggested teacher beliefs can impact teachers’

educational practice and teacher beliefs can be significantly associated with students’

cognitive and affective learning outcomes, this study did not find significant relationships

between teacher beliefs and teachers’ 3D printing integration or relationships between

many aspects of teacher beliefs and student learning outcomes. This was probably due

to the challenges that teachers encountered as they reported in the open-ended

questions in the teacher beliefs survey and also the short duration of 3D printing

integration. In order for teachers to integrate 3D printing effectively, first, the challenges

have to be addressed. It may also take a longer time and more practice for teachers’

beliefs and their 3D printing integration to be aligned with each other. In addition, the

findings on the relationship between teachers’ 3D printing integration levels, STEM

210

integration levels, teacher beliefs, and student outcomes shed some light on future

research.

Implications for Practice

In this study, the teachers perceived 3D printing integration as beneficial for

students, but it was challenging for teachers to integrate 3D printing technology in their

science classrooms due to the external barriers and internal barriers reported in chapter

4. To facilitate teachers’ technology integration, it is necessary to reduce external

barriers by providing adequate funding, equitable access to recourses, and technical

support (ISTE, 2019). The teachers encountered some external barriers, including

logistical and technical issues, insufficient 3D printers and related resources, lack of

time to print 3D objects, and lack of time to plan, develop, and integrate 3D printing into

curriculum. Schools need to provide necessary resources and technical support.

Specifically, schools may provide more 3D printers and relevant resources so teachers

can print 3D objects more efficiently and also engage students in activities in smaller

groups. Technical support can help teachers save time on troubleshooting and focus

more on integrating 3D printing with the curriculum. Moreover, as the teachers had

insufficient time to plan, develop, and integrate 3D printing into the curriculum, schools

may provide more support to help teachers balance the current workload and the

initiative of integrating a new piece of technology into their teaching.

Besides the external barriers, the teachers also had some internal barriers

including the lack of ability to print 3D objects and connect 3D printing to curriculum

standards, and also to teach students who were not enthusiastic, motivated, or having

limited ability. It is essential to provide professional development and instructional

support to equip teachers with the ability to print 3D objects, connect 3D printing with

211

curriculum standards, and also develop skills in teaching students with individual

differences, such as varied enthusiasm, motivation, and ability. Schools may consider

providing professional development opportunities and instructional support for teachers

to learn how to use 3D printing technology, how to meaningfully connect 3D printing

with curriculum standards, and how to teach students with individual differences.

Successful 3D printing integration not only requires teachers being proficient with how

to use the technology, but also how to connect it to the curriculum, especially how to

integrate their technological knowledge, pedagogical knowledge, and content

knowledge to enhance their teaching and student learning outcomes.

Furthermore, schools may need to invest more on training teachers to effectively

integrate 3D printing into STEM disciplines. Although teachers may have abilities to

teach some individual STEM disciplines, they may not have knowledge and skills in all

the STEM disciplines that are involved when they design and implement STEM

integrated lessons. STEM integration could also be challenging even if teachers have

the necessary knowledge and skills in different STEM disciplines. STEM integration is

not just simply combining the different content area. Effective STEM integration requires

teachers to 1) be explicit about the goals and design STEM integrated lessons to

purposefully achieve the goals, 2) make STEM connections explicit to students through

scaffolding and opportunities to engage in activities in order to address the connected

ideas, and 3) keep learning goals in mind and pay attention to students’ learning

outcomes in individual STEM subjects (Honey et al., 2014). Additionally, schools may

build networking between teachers by establishing a community of support to facilitate

communication and collaboration between teachers in the same school and/or across

212

different schools. Schools may also partner with a local university to improve teachers’

3D printing integration and STEM integration through research and practice.

In addition to actively participating in professional training, teachers need to apply

what they have learned in their daily teaching. It is necessary for schools to provide

professional development opportunities for teachers to learn how to teach students with

individual differences, and it is critical that teachers taking the challenges and practicing

what they have learned. In this study, STEM integration level positively predicted

students’ math motivation but didn’t predict other student outcomes and STEM

integration level had different effects for male and female students and also students

with different prior interest in STEM careers. Teachers may consider students’ individual

differences such as student gender and prior interest when designing STEM integrated

lessons. Teachers’ technology integration level was not a significant predictor for any of

the student outcomes, but it had interaction with students’ prior interest in STEM

careers. Teachers may also consider students’ prior interest when they consider how to

integrate 3D printing in the classrooms.

Although providing necessary resources, technical support, professional

development, instructional support, and building network between teachers and schools

could be necessary for teachers to integrate 3D printing technologies in their science

classrooms, these efforts do not guarantee the teachers can integrate the technologies

without difficulties. In this study, the teachers were provided with 3D printers and

relevant resources, professional development, technical support, instructional support,

and network with the iDigFossils project team and other teachers, however, the

teachers still encountered many external barriers and internal barriers, which may have

213

limited and impacted their 3D printing integration in the science classrooms. Schools

and teachers have to be aware that integrating an emerging technology such as 3D

printing is not easy. It can take innumerous efforts of the schools and teachers to make

3D printing integration successful.

Although some external and internal barriers can be addressed by providing

resources, technical support, professional development, instructional support, and

building networking between teachers and schools, the intrinsic technical issues within

3D printing technologies such as the necessity of a long time and appropriate

temperature to successfully print 3D objects may require schools and teachers to take

extra efforts and strategies to leverage these intrinsic technical challenges. Additionally,

the durations of the 3D printing integration in this study were relatively short, which may

have contributed to the inconsistency between teachers’ beliefs and their 3D printing

integration. Moreover, positive relationships between 3D printing integration and

students’ learning outcomes may not show immediately with a short time of intervention.

It can take a long time of investment and continuous support for the teachers to

successfully integrate 3D printing technologies in their science classrooms. Schools

have to be judicious when making the decisions on whether or not to make investments

on a large-scale 3D printing integration in their classrooms.

Implications for Research

In addition to the implications for practice, this study also had some implications

for future research. In this study, teachers’ 3D printing integration level was not a

significant predictor for any of the student learning outcomes, which was probably

because 3D printing integration level did not capture all the possible differences in

teachers’ 3D printing integration and also the short durations of the 3D printing

214

integration. Future studies may include more aspects of teachers’ 3D printing integration

to detect the potential influence on student learning outcomes. It would also be

necessary to test the effects of 3D printing integration with long-term interventions.

Although STEM integration level positively predicted students’ math motivation,

this study cannot assert a causal relationship between STEM integration level and

students’ math motivation. In addition, the interaction between student gender and

STEM integration level suggested that as STEM integration level increases, female

students’ science motivation may have a higher increase than male students. As the

results of this study could not depict causal relationship, future research may conduct

experimental studies to examine the effects of STEM integration level on students’ math

motivation, and also the influence of STEM integration levels on students with individual

differences such as student gender.

Furthermore, teachers’ STEM integration level was a significant predictor for

students’ math motivation, but it was not significant for other student learning outcomes.

As stated previously, STEM integration can make learning more connected and

meaningful for students, but it also requires sufficient knowledge and skills for individual

subjects, which may explain the varied relationship between STEM integration level and

students’ motivation in different STEM subjects, students’ 21st century skills, and

interest in STEM careers. However, this conjecture needs future research to verify.

Future studies may obtain more quantitative student data such as students’ prior

knowledge and skills in the different subjects involved in STEM integration and also

some qualitative data such as student interviews to further explore the relationship

between STEM integration levels and students’ learning outcomes.

215

In addition, STEM integration level had positive interactions with students’ prior

interest in STEM careers and with student gender for science motivation, which

suggested that STEM integration level also had different effects on students with

individual differences. All in all, the varied associations STEM integration levels and

students’ science, technology/engineering, and math motivation, 21st century skills, and

interest in STEM careers, and also the interactions between STEM integration level and

students with individual differences call for further investigation to determine how STEM

integration can positively influence students with individual differences.

In this study, there were no significant relationships between teacher beliefs and

their 3D printing integration, however, there were some significant relationships

between teacher beliefs and student outcomes. Future studies may further explore why

teacher beliefs may influence students even if teachers’ technology integration is not

influenced. This study was not able to include duration and school level as predictors.

Future studies may investigate the effects of these variables. It is also highly

recommended that future research use classroom observation to obtain more data to

ensure a full picture of how 3D printing was integrated and the interactions in the

classrooms.

The interactions between student variables (prior motivation and interest and

student gender) and teacher beliefs showed some consistency but also had

discrepancies on the effect of teachers’ 3D printing integration and teacher beliefs on

male and female students and on students with different prior motivation, interest, and

21st century skills. Previous studies that investigated the relationship between teacher

beliefs and student learning performance and motivation did not examine how teacher

216

beliefs may influence students with individual differences such as gender and prior

motivation and interest. Although previous studies found significant relationships

between teacher beliefs and student learning outcomes in general, the influence of

teacher beliefs on students with individual differences needs further investigation.

Future studies may consider students’ individual differences when examining the

relationships between teacher beliefs, 3D printing integration, and students’ learning

outcomes.

Conclusions

The purpose of this study was to investigate the relationship between teacher

beliefs, teachers’ 3D printing integration, and students’ STEM motivation, 21st century

skills, and interest in STEM careers. Specifically, the correlation between teacher beliefs

and their 3D printing integration, and how teachers’ 3D printing integration predict

students’ STEM motivation, 21st century skills, and interest in STEM careers.

This study produced several interesting results. First, teacher beliefs and 3D

printing integration were generally not correlated except that teachers’ self-efficacy in

pedagogical content knowledge and STEM integration level were significantly but

negatively correlated. Second, teachers perceived 3D printing integration as beneficial

for students, but they encountered several challenges including logistic and technical

issues, lack of resources, lack of time, and insufficient abilities to use 3D printers and

connect 3D printing with curriculum, and also difficulty in engaging and teaching

students with individual differences. Third, STEM integration level was a positive

predictor for students’ math motivation. Fourth, teachers’ 3D printing integration level

was not a significant predictor for any of the student learning outcomes. Fifth, teachers’

extrinsic utility value (perceived usefulness) of 3D printing was a negative predictor for

217

students’ 21st century skills. Finally, there were some interesting interaction effects

between student variables (student gender and pretest scores) and teacher variables

(teacher beliefs and 3D printing integration).

Although not able to claim causal relationships, this study laid a foundation for

future educational practice and research. Schools may provide adequate resources,

technical support, professional development, instructional support, and build networking

between teachers and schools to facilitate teachers’ 3D printing integration. Schools

also need to be judicious when making decisions on whether or not to invest in a large-

scale 3D printing integration. Teachers need to actively participate in professional

development and apply what they have learned in their classrooms. Moreover, teachers

need to consider students’ individual differences including but not limited to student

gender and students’ prior motivation, interest, and skills when integrating 3D printing

into their classrooms. Regarding future research, it would be necessary to employ other

types of research design such as experimental studies to examine the effects of

different 3D printing integration levels and STEM integration levels on students’ learning

outcomes, and also how the different levels may influence students with individual

differences. Last, future studies may further investigate the relationships between

different aspects of teacher beliefs and students’ cognitive and affective learning

outcomes, and how teacher beliefs influence students with individual differences such

as student gender and prior motivation, skills, and interest.

218

APPENDIX A S-STEM SURVEY (UNFRIED ET AL., 2015)

Math

Strongly Disagree

Disagree Neither Agree nor Disagree

Agree Strongly Agree

1. Math has been my worst subject. ○ ○ ○ ○ ○

2. I would consider choosing a career that uses math. ○ ○ ○ ○ ○

3. Math is hard for me. ○ ○ ○ ○ ○

4. I am the type of student to do well in math. ○ ○ ○ ○ ○ 5. I can handle most subjects well, but I cannot do a good job with math.

○ ○ ○ ○ ○

6. I am sure I could do advanced work in math. ○ ○ ○ ○ ○

7. I can get good grades in math. ○ ○ ○ ○ ○

8. I am good at math. ○ ○ ○ ○ ○

Science

Strongly Disagree



9. I am sure of myself when I do science. ○ ○ ○ ○ ○

10. I would consider a career in science. ○ ○ ○ ○ ○

11. I expect to use science when I get out of school. ○ ○ ○ ○ ○

219

12. Knowing science will help me earn a living. ○ ○ ○ ○ ○

13. I will need science for my future work. ○ ○ ○ ○ ○

14. I know I can do well in science. ○ ○ ○ ○ ○

15. Science will be important to me in my life’s work. ○ ○ ○ ○ ○ 16. I can handle most subjects well, but I cannot do a good job with science.

○ ○ ○ ○ ○

17. I am sure I could do advanced work in science. ○ ○ ○ ○ ○

Engineering and Technology

Please read this paragraph before you answer the questions.

Engineers use math, science, and creativity to research and solve problems that improve everyone’s life and to invent new products. There are many different types of engineering, such as chemical, electrical, computer, mechanical, civil, environmental, and biomedical. Engineers design and improve things like bridges, cars, fabrics, foods, and virtual reality amusement parks. Technologists implement the designs that engineers develop; they build, test, and maintain products and processes.

Strongly Disagree



18. I like to imagine creating new products. ○ ○ ○ ○ ○ 19. If I learn engineering, then I can improve things that people use every day.

○ ○ ○ ○ ○

20. I am good at building and fixing things. ○ ○ ○ ○ ○

21. I am interested in what makes machines work. ○ ○ ○ ○ ○ 22. Designing products or structures will be important for my future work.

○ ○ ○ ○ ○

220

23. I am curious about how electronics work. ○ ○ ○ ○ ○

24. I would like to use creativity and innovation in my future work. ○ ○ ○ ○ ○ 25. Knowing how to use math and science together will allow me to invent useful things.

○ ○ ○ ○ ○

26. I believe I can be successful in a career in engineering. ○ ○ ○ ○ ○

21st Century Skills

Strongly Disagree



27. I am confident I can lead others to accomplish a goal. ○ ○ ○ ○ ○

28. I am confident I can encourage others to do their best. ○ ○ ○ ○ ○

29. I am confident I can produce high quality work. ○ ○ ○ ○ ○

30. I am confident I can respect the differences of my peers. ○ ○ ○ ○ ○

31. I am confident I can help my peers. ○ ○ ○ ○ ○ 32. I am confident I can include others’ perspectives when making decisions.

○ ○ ○ ○ ○

33. I am confident I can make changes when things do not go as planned.

○ ○ ○ ○ ○

34. I am confident I can set my own learning goals. ○ ○ ○ ○ ○ 35. I am confident I can manage my time wisely when working on my own.

○ ○ ○ ○ ○

36. When I have many assignments, I can choose which ones need to be done first.

○ ○ ○ ○ ○

221

37. I am confident I can work well with students from different backgrounds.

○ ○ ○ ○ ○

Your Future

Here are descriptions of subject areas that involve math, science, engineering and/or technology, and lists of jobs connected to each subject area. As you read the list below, you will know how interested you are in the subject and the jobs. Fill in the circle that relates to how interested you are.

There are no “right” or “wrong” answers. The only correct responses are those that are true for you.

Not at all Interested

Not So Interested

Interested Very

Interested

1. Physics: is the study of basic laws governing the motion, energy, structure, and interactions of matter. This can include studying the nature of the universe. (aviation engineer, alternative energy technician, lab technician, physicist, astronomer)

○ ○ ○ ○

2. Environmental Work: involves learning about physical and biological processes that govern nature and working to improve the environment. This includes finding and designing solutions to problems like pollution, reusing waste and recycling. (pollution control analyst, environmental engineer or scientist, erosion control specialist, energy systems engineer and maintenance technician)

○ ○ ○ ○

3. Biology and Zoology: involve the study of living organisms (such as plants and animals) and the processes of life. This includes working with farm animals and in areas like nutrition and breeding. (biological technician, biological scientist, plant breeder, crop lab

○ ○ ○ ○

222

technician, animal scientist, geneticist, zoologist) 4. Veterinary Work: involves the science of preventing or treating disease in animals. (veterinary assistant, veterinarian, livestock producer, animal caretaker)

○ ○ ○ ○

5. Mathematics: is the science of numbers and their operations. It involves computation, algorithms and theory used to solve problems and summarize data. (accountant, applied mathematician, economist, financial analyst, mathematician, statistician, market researcher, stock market analyst)

○ ○ ○ ○

6. Medicine: involves maintaining health and preventing and treating disease. (physician’s assistant, nurse, doctor, nutritionist, emergency medical technician, physical therapist, dentist)

○ ○ ○ ○

7. Earth Science: is the study of earth, including the air, land, and ocean. (geologist, weather forecaster, archaeologist, geoscientist)

○ ○ ○ ○

8. Computer Science: consists of the development and testing of computer systems, designing new programs and helping others to use computers. (computer support specialist, computer programmer, computer and network technician, gaming designer, computer software engineer, information technology specialist)

○ ○ ○ ○

9. Medical Science: involves researching human disease and working to find new solutions to human health problems. (clinical laboratory technologist, medical scientist, biomedical engineer, epidemiologist, pharmacologist)

○ ○ ○ ○

10. Chemistry: uses math and experiments to search for new ○ ○ ○ ○

223

chemicals, and to study the structure of matter and how it behaves. (chemical technician, chemist, chemical engineer) 11. Energy: involves the study and generation of power, such as heat or electricity. (electrician, electrical engineer, heating, ventilation, and air conditioning (HVAC) technician, nuclear engineer, systems engineer, alternative energy systems installer or technician)

○ ○ ○ ○

12. Engineering: involves designing, testing, and manufacturing new products (like machines, bridges, buildings, and electronics) through the use of math, science, and computers. (civil, industrial, agricultural, or mechanical engineers, welder, auto-mechanic, engineering technician, construction manager)

○ ○ ○ ○

224

APPENDIX B TEACHER BELIEFS ON 3D PRINTING INTEGRATION SURVEY

Demographic Information

1. Your name: _______

2. Gender: a. Female b. Male c. Other

3. Age range: a. 21-30 b.31-40 c. 41-50 d. 51-60 e. 61+

4. Your race:

5. Your ethnicity:

Teacher Pedagogical Beliefs Survey (adapted from Ravitz, Becker, & Wong, 2000)

J1. The following paragraphs describe observations of two teachers’ classes, Ms. Hill’s and Mr. Jones’. Answer each question below by selecting the column that best answers that question for you.

Definitely

Ms. Hill

Tend

towards

Ms. Hill

Cannot

decide

Tend

towards

Mr. Jones

Definitely

Mr. Jones

1. Which type of class discussion are you more comfortable having in class?

○ ○ ○ ○ ○

2. Which type of discussion do you think most students prefer to have?

○ ○ ○ ○ ○

3. From which type of discussion do you think

○ ○ ○ ○ ○

Ms. Hill was leading her class in an

animated way, asking questions that the

students could answer quickly; based

on the reading they had done the day

before. After this review, Ms. Hill

taught the class new material, again

using simple questions to keep students

attentive and listening to what she said.

Mr. Jones’ class was also having a

discussion, but many of the questions

came from the students themselves.

Though Mr. Jones could clarify

students’ questions and suggest where

the students could find relevant

information, he couldn’t really answer

most of the questions himself.

225

students gain more knowledge?

4. From which type of discussion do you think students gain more useful skills?

○ ○ ○ ○ ○

J2. Indicate how much you disagree or agree with each of the following statements about teaching and learning.

Strongly Disagree

Disagree Neither Disagree nor Agree


5. Teachers know a lot more than students; they shouldn't let students muddle around when they can just explain the answers directly.

○ ○ ○ ○ ○

6. A quiet classroom is generally needed for effective learning. ○ ○ ○ ○ ○

7. It is better when the teacher – not the students - decides what activities are to be done.

○ ○ ○ ○ ○

8. Students will take more initiative to learn when they feel free to move around the room during class.

○ ○ ○ ○ ○

9. Students should help establish criteria on which their work will be assessed.

○ ○ ○ ○ ○

10. Instruction should be built around problems with clear, correct answers, and around ideas that most students can grasp quickly.

○ ○ ○ ○ ○

11. How much students learn depends on how much background knowledge they have – that is why teaching facts is so necessary.

○ ○ ○ ○ ○

226

J3. Different teachers have described very different teaching philosophies to researchers. For each of the following pairs of statements, check the button that best shows how closely your own beliefs are to each of the statements in a given pair. The closer your beliefs to a particular statement, the closer the button you check.

12. “I mainly see my role as a facilitator. I try to provide opportunities and resources for my students to discover or construct concepts for themselves.”

○ ○ ○ ○ ○

"That's all nice, but students really won't learn the subject unless you go over the material in a structured way. It's my job to explain, to show students how to do the work, and to assign specific practice."

13. "The most important part of instruction is that it encourages “sense-making” or thinking among students. Content is secondary."

○ ○ ○ ○ ○

"The most important part of instruction is the content of the curriculum. That content is the community’s judgment about what children need to be able to know and do."

14. "It is critical for students to become interested in doing academic work—interest and effort are more important than the particular subject matter they are working on."

○ ○ ○ ○ ○

"While student motivation is certainly useful, it should not drive what students study. It is more important that students learn the history, science, math and language skills in their textbooks."

15. "It is a good idea to have all sorts of activities going on in the classroom. Some students might produce a scene from a play they read. Others might create a miniature version of the set. It's hard to get the logistics right, but the successes are so much more important than the failures."

○ ○ ○ ○ ○

"It's more practical to give the whole class the same assignment, one that has clear directions, and one that can be done in short intervals that match students' attention spans and the daily class schedule."

227

Teacher Self-Efficacy in 3D Printing Integration Survey (adapted from Schmidt et al., 2009)

All the items used a 5-point Likert scale: Strongly Disagree = 1; Disagree = 2; Neither Agree/Disagree = 3; Agree = 4; Strong Agree = 5.

Notes: The subtitles here are just to show the components of the survey and they were not included in the survey administered.

Self-efficacy in Technology Knowledge (TK)

1. I am confident that I know how to solve technical problems of 3D printing technology.

2. I am confident that I can learn 3D printing technology easily.

3. I am confident that I can keep up with 3D printing technology.

4. I am confident that I have sufficient knowledge about 3D printing technology.

5. I am confident that I have the technical skills I need to use 3D printing technology.

Self-efficacy in Content Knowledge (CK)

6. I am confident that I have sufficient knowledge about science.

7. I am confident that I have sufficient knowledge about paleontology.

8. I am confident that I can use a scientific way of thinking.

9. I am confident that I have various ways and strategies of developing my understanding of science

Self-efficacy in Pedagogical Knowledge (PK)

10. I am confident that I know how to assess student performance in a classroom.

11. I am confident that I can adapt my teaching based upon what students currently understand or do not understand.

12. I am confident that I can adapt my teaching style to different learners.

13. I am confident that I can assess student learning in multiple ways.

14. I am confident that I can use a wide range of teaching approaches in a classroom setting.

15. I am confident that I am familiar with common student understandings and misconceptions.

16. I am confident that I know how to organize and maintain classroom management.

Self-efficacy in Pedagogical Content Knowledge (PCK)

17. I am confident that I can select effective teaching approaches to guide student thinking and learning in science.

228

Self-efficacy in Technological Content Knowledge (TCK)

18. I am confident that I know about 3D printing technology for understanding and doing science.

Self-efficacy in Technological Pedagogical Knowledge (TPK)

19. I am confident that I can use 3D printing technology to enhance the teaching approaches for a lesson.

20. I am confident that I can use 3D printing technology to enhance students’ learning for a lesson.

21. I am confident that I can think deeply about how 3D printing technology could influence the teaching approaches I use in my classroom.

22. I am confident that I can think critically about how to use 3D printing technology in my classroom.

23. I am confident that I can use 3D printing technology for different teaching activities.

Self-efficacy in Technological Pedagogical Content Knowledge (TPACK)

24. I am confident that I can design and teach lessons that appropriately combine science, 3D printing technology, and teaching approaches.

25. I am confident that I can use 3D printing technology to enhance what I teach, how I teach, and what students learn.

26. I am confident that I can provide leadership in helping others to coordinate the use of science content, 3D printing technologies, and teaching approaches at my school and/or district.

Teacher 3D Printing Value Beliefs (Adapted from Eccles & Wigfield, 1995)

Intrinsic Interest Value

1. In general, I find integrating 3D printing technology in my science classrooms (very boring, boring, neither boring nor interesting, interesting, very interesting)

2. How much do you like integrating 3D printing technology in your science classrooms? (strongly dislike, dislike, neither dislike nor like, like, strongly like)

Attainment Value/Importance

3. Is the amount of effort it took to do well in integrating 3D printing into your science classrooms worthwhile to you? (not at all worthwhile, not worthwhile, neutral, worthwhile, very worthwhile)

229

4. I feel that, to me, being good at integrating 3D printing technology into my science classrooms is (Not at all important, not important, neutral, important, very important)

5. How important is it for you to do well in integrating 3D printing technology into your science classrooms? (Not at all important, not important, neutral, important, very important)

Extrinsic Utility Value/Usefulness

6. How useful is integrating 3D printing technology in your science classrooms to enhance students’ learning? (not at all useful, not useful, neutral, useful, very useful)

7. How useful is integrating 3D printing technology in your science classrooms to engage students? (not at all useful, not useful, neutral, useful, very useful)

Open-Ended Questions on Teacher Beliefs on 3D Printing Integration

1. How do you feel about integrating 3D printing technology into your science teaching?

2. Do you think you have sufficient knowledge and skills to integrate 3D printing technology in your science classes? Please explain.

3. What are the biggest advantages in integrating 3D printing technology in science teaching?

4. What are the biggest challenges in integrating 3D printing technology in science teaching?

230

LIST OF REFERENCES

Al-Rajhi, N., Al-Abdulkarim, A., Al-Khalifa, H. S., & Al-Otaibi, H. M. (2015, July). Making Linear Equations Accessible for Visually Impaired Students Using 3D Printing. In Advanced Learning Technologies (ICALT), 2015 IEEE 15th International Conference on (pp. 432-433). IEEE.

Albion, P. (1999). Self-efficacy beliefs as an indicator of teachers' preparedness for teaching with technology. In Proceedings of the 10th International Conference of the Society for Information Technology & Teacher Education (SITE 1999) (pp. 1602-1608). Association for the Advancement of Computing in Education (AACE).

Anderson, S. E., & Maninger, R. M. (2007). Preservice teachers' abilities, beliefs, and intentions regarding technology integration. Journal of Educational Computing Research, 37(2), 151-172.

Antonenko, P. D. (2015). The instrumental value of conceptual frameworks in educational technology research. Educational Technology Research and Development, 63(1), 53-71.

Ashcraft, M. H., & Ridley, K. S. (2005). Math anxiety and its cognitive consequences: A tutorial review. In J. I. D. Campbell (Ed.), Handbook of mathematical cognition (pp. 315-327). New York, NY: Psychology Press.

Ashton, P. T., & Webb, R. B. (1986). Making a difference: Teachers' sense of efficacy and student achievement. New York, NY: Longman.

Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64(6), 359-372. http://dx.doi.org/10.1037/h0043445

Bandura, A. (1982). Self-efficacy mechanism in human agency. American psychologist, 37(2), 122-147.

Bandura, A. (1986). Social foundations of thoughts and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman and Company.

Bauer, J., & Kenton, J. (2005). Toward technology integration in schools: Why it is not happening. Journal of Technology and Teacher Education, 13, 519-546.

Beck, A. T. (1976). Cognitive therapy and the emotional disorders. New York, NY: International Universities Press.

http://dx.doi.org/10.1037/h0043445

231

Becker, H. J. (1999). Internet use by teachers: Conditions of professional use and teacher-directed student use. Irvine, CA: Center for Research on Information Technology and Organizations.

Becker, H. J. (2001, April). How are teachers using computers in instruction? In Paper presented at the annual meeting of the American Educational Research Association, Seattle, WA. Retrieved from https://msu.edu/course/cep/807/*cep240studyrefs/beckeraera2001howtchrsusing.pdf on December 17, 2018.

Bicer, A., Nite, S. B., Capraro, R. M., Barroso, L. R., Capraro, M. M., & Lee, Y. (2017, October). Moving from STEM to STEAM: The effects of informal STEM learning on students' creativity and problem solving skills with 3D printing. In Frontiers in Education Conference (FIE) (pp. 1-6). IEEE.

Bragow, D., Gragow, K. A., & Smith, E. (1995). Back to the future: Toward curriculum integration. Middle School Journal, 27, 39–46.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology, 3(2), 77-101.

Britten, J. S., & Cassady, J. C. (2005). The Technology Integration Assessment Instrument: Understanding planned use of technology by classroom teachers. Computers in the Schools, 22(3), 49-61.

Britner, S. L., & Pajares, F. (2006). Sources of science self-efficacy beliefs of middle school students. Journal of Research in Science Teaching, 43(5), 485–499.

Brookover, W., Beady, C, Flood, P., Schweitzer, J., & Wisenbaker, J. (1979). School social systems and student achievement: Schools can make a difference. New York, NY: Praeger.

Brophy, J. E., & Evertson, C. (1977). Teacher behaviors and student learning in second and third grades. In G. D. Borich (Ed.), The appraisal of teaching: Concepts and process (pp. 79-95). Reading, MA: Addison-Wesley.

Brophy, J. E., & Good, T. (1974). Teacher-student relationships: Causes and consequences. New York, NY: Holt, Rinehart, & Winston.

Brown, P. L., Concannon, J. P., Marx, D., Donaldson, C., & Black, A. (2016). An examination of middle school students’ STEM self-efficacy, interests and perceptions. Journal of STEM Education: Innovations and Research, 17(3), 27-38.

Bull, G., Chiu, J., Berry, R., Lipson, H., & Xie, C. (2014). Advancing children’s engineering through desktop manufacturing, in Mayer, R., and Alexander, P., eds., Handbook of Research on Educational Communications and Technology (pp. 675–688). New York, Springer.

232

Bureau of Labor Statistics (BLS). (2017). Employment projections – 2016-26. Retrieved from https://www.bls.gov/news.release/pdf/ecopro.pdf on Dec. 24, 2018.

Byars-Winston, A., Estrada, Y., Howard, C., Davis, D., & Zalapa, J. (2010). Influence of social cognitive and ethnic variables on academic goals of underrepresented students in science and engineering: A multiple-groups analysis. Journal of counseling psychology, 57(2), 205-218.

Chao, J. Y., Po, H. Y., Chang, Y. S., & Yao, L. Y. (2016, November). The study of 3D printing project course for indigenous senior high school students in Taiwan. In Advanced Materials for Science and Engineering (ICAMSE), International Conference on (pp. 68-70). IEEE.

Chen, M., Zhang, Y., & Zhang, Y. (2014). Effects of a 3D printing course on mental rotation ability among 10-year-old primary students. International Journal of Psychophysiology, 94(2), 240.

Cheng, S. L., & Xie, K. (2018). The relations among teacher value beliefs, personal characteristics, and TPACK in intervention and non-intervention settings. Teaching and Teacher Education, 74, 98-113.

Chien, Y. H. (2017). Developing a pre-engineering curriculum for 3D printing skills for high school technology education. Eurasia Journal of Mathematics, Science and Technology Education, 13(7), 2941-2958.

Chien, Y. H., & Chu, P. Y. (2018). The Different Learning Outcomes of High School and College Students on a 3D-Printing STEAM Engineering Design Curriculum. International Journal of Science and Mathematics Education, 16(6), 1047-1064.

Christensen, R., & Knezek, G. (2017). Relationship of middle school student STEM interest to career intent. Journal of Education in Science, Environment and Health, 3(1), 1-13.

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445-459. Doi:10.3102/00346543053004445

Clark, R. E. (1994a). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29. Doi:10.1007/BF02299088

Clark, R. E. (1994b). Media and method. Educational Technology Research and Development, 42(3), 7-10. Doi:10.1007/BF02298090

Cornelius-White, J. (2007). Learner-centered teacher-student relationships are effective: A meta-analysis. Review of educational research, 77(1), 113-143.

https://www/

233

Cornelius-White, J. H., & Harbaugh, A. P. (2009). Learner-centered instruction: Building relationships for student success. Thousand Oaks, CA: Sage.

Crandall, V. C. (1969). Sex differences in expectancy of intellectual and academic reinforcement. In C. P. Smith (ed.), Achievement-related motives in children. New York, NY: Russell Sage Foundation.

Crandall, V. J., Katkovsky, W., & Preston, A. (1962). Motivational and ability determinants of young children's intellectual achievement behavior. Child Development, 33: 643-661.

Culp, K. M., Honey, M., & Mandinach, E. (2005). A retrospective on twenty years of education technology policy. Journal of Educational Computing Research, 32(3), 279-307.

Definition and Terminology Committee of Association for Educational Communications and Technology. (2018). Definition of Educational Technology. Retrieved from https://www.aect.org/ on November 23, 2018.

Deng, F., Chai, C. S., Chin-Chung, T., & Min-Hsien, L. (2014). The relationships among Chinese practicing teachers' epistemic beliefs, pedagogical beliefs and their beliefs about the use of ICT. Journal of Educational Technology & Society, 17(2), 245.

Dewey, J. (1913). Interest and effort in education. Boston, MA: Houghton Mifflin Company. http://dx.doi.org/10.1037/14633-000

Eccles, J. S. (2009). Who am I and what am I going to do with my life? Personal and collective identities as motivators of action. Educational Psychologist, 44(2), 78-89.

Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J., and Midgley, C. (1983). Expectancies, values and academic behaviors. In J. T. Spence (ed.), Achievement and Achievement Motives. San Francisco, CA: W. H. Freeman.

Eccles, J. S., & Wigfield, A. (1985). Teacher expectations and student motivation. In J. Dusek (Ed.), Teacher expectancies (pp. 185-217). Hillsdale, NJ: Erlbaum.

Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents' achievement task values and expectancy-related beliefs. Personality and social psychology bulletin, 21(3), 215-225.

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual review of psychology, 53(1), 109-132.

http://psycnet.apa.org/doi/10.1037/14633-000

234

Ertmer, P. A. (1999). Addressing first-and second-order barriers to change: Strategies for technology integration. Educational technology research and development, 47(4), 47-61.

Ertmer, P. A. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration? Educational Technology Research and Development, 53(4) 25-39.

Ertmer, P. A., Addison, P., Lane, M., Ross, E., & Woods, D. (1999). Examining teachers’ beliefs about the role of technology in the elementary classroom. Journal of Research on Computing in Education, 32(1), 54–72.

Ertmer, P. A., Conklin, D., Lewandowski,]., Osika, E., Selo, M., & Wignall, E. (2003). Increasing preservice teachers' capacity for technology integration through the use of electronic models. Teacher Education Quarterly, 30(1), 95-112.

Ertmer, P. A., & Glazewski, K. D. (2015). Essentials for PBL implementation: Fostering collaboration, transforming roles, and scaffolding learning. In A. Walker, H. Leary, C. Hmelo-Silver, & P. A. Ertmer (Eds.), Essential readings in problem-based learning (pp. 89-106). West Lafayette, IN: Purdue University Press.

Ertmer, P. A., & Ottenbreit-Leftwich, A. T. (2010). Teacher technology change: How knowledge, confidence, beliefs, and culture intersect. Journal of research on Technology in Education, 42(3), 255-284.

Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: A critical relationship. Computers & Education, 59(2), 423-435.

Fan, W. (2011). Social influences, school motivation and gender differences: an application of the expectancy‐value theory. Educational Psychology, 31(2), 157-175.

Feather, N. T. (1982). Expectations and actions: Expectancy-value models in psychology. Hillsdale, NJ: Lawrence Erlbaum Association Incorporated.

Feather, N. T. (1988). Values, valences, and course enrollment: Testing the role of personal values within an expectancy-valence framework. Journal of educational psychology, 80(3), 381-391.

Feather, N. T. (1992). Values, valences, expectations, and actions. Journal of Social Issues, 48(2), 109-124.

Francis, J. (2017). The effects of technology on student motivation and engagement in classroom-based learning. Unpublished doctoral thesis. University of New England, USA. Retrieved from http://dune.une.edu/cgi/viewcontent.cgi?article=1120&context=theses

http://dune.une.edu/cgi/viewcontent.cgi?article=1120&context=theses

235

Friday Institute for Educational Innovation (2012). Upper Elementary School Student Attitudes toward STEM Survey. Raleigh, NC: Author.

Friedman, T. L. (2005). The world is flat: A brief history of the twenty-first century. New York, NY: Farrar, Straus and Giroux.

Furner, J., & Kumar, D. (2007). The mathematics and science integration argument: a stand for teacher education. Eurasia Journal of Mathematics, Science & Technology, 3(3), 185–189.

Gendron, B. (1977). Technology and the human condition. New York, NY: St. Martin’s.

Gonzales, S. (2013). Examining the relationship among high-school teachers' technology self-efficacy, attitudes towards technology integration, and quality of technology integration. Available from ProQuest Dissertations & Theses Global. UMI 3596649. Northern Illinois University.

Gonzalez, S. R., & Bennett, D. B. (2016). 3D printing: A practical guide for librarians (Vol. 22). Lanham, MD: Rowman & Littlefield.

Good, T. L. (1981). Teacher expectations and student perceptions: A decade of research. Educational Leadership, 38, 415-422.

Grant, C. A., MacFadden, B. J., Antonenko, P., & Perez, V. J. (2016). 3-D fossils for k–12 education: a case example using the giant extinct shark Carcharocles Megalodon. The Paleontological Society Papers, 22, 197-209.

Gray, L., Thomas, N., & Lewis, L. (2010). Teachers' Use of Educational Technology in U.S. Public Schools: 2009. Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Retrieved from https://files.eric.ed.gov/fulltext/ED509514.pdf on December 11, 2018.

Grice, N., Christian, C., Nota, A., & Greenfield, P. (2015). 3D printing technology: A unique way of making Hubble Space Telescope Images accessible to non-visual learners. Journal of Blindness Innovation and Research, 5(1). Retrieved from https://nfb.org/images/nfb/publications/jbir/jbir15/jbir050101.html. doi: http://dx.doi.org/10.5241/5-66

Grigg, S., Perera, H. N., McIlveen, P., & Svetleff, Z. (2018). Relations among math self efficacy, interest, intentions, and achievement: A social cognitive perspective. Contemporary Educational Psychology, 53, 73-86.

Gutherie, J. T., Wigfield, A., & VonSecker, C. (2000). Effects of integrated instruction on motivation and strategy use in reading. Journal of Educational Psychology, 92, 331–341, http://dx.doi.org/10.1037/0022-0663.92.2.331

https://nfb.org/images/nfb/publications/jbir/jbir15/jbir050101.html

https://psycnet.apa.org/doi/10.1037/0022-0663.92.2.331

236

Haight, K. W. (2011). Teacher self-efficacy in the adoption and integration of technology within the classroom. Dissertation published by ProQuest. UMI 3445014. Walden University.

Harmes, J. C., Welsh, J. L., & Winkelman, R. J. (2016). A framework for defining and evaluating technology integration in the instruction of real-world skills. In Handbook of Research on Technology Tools for Real-World Skill Development (pp. 137-162). Hershey, PA: IGI Global.

Harris, J., Grandgenett, N., & Hofer, M. (2010). Testing a TPACK-based technology integration assessment instrument. In C. D. Maddux, D. Gibson, & B. Dodge (Eds.). Research highlights in technology and teacher education 2010 (pp. 323-331). Chesapeake, VA: Society for Information Technology and Teacher Education (SITE).

Heafner, T. (2004). Using technology to motivate students to learn social studies. Contemporary Issues in Technology and Teacher Education, 4(1), 42-53.

Heller, K. A., & Parsons, J. E. (1981) Teacher influences on students' expectancies for success in mathematics. Child Development, 52, 1015-1019.

Heineman, A. J. (2018). Instructional Coaching, Educational Technology, and Teacher Self-Efficacy: A Case Study of Instructional Coaching Programs in a California Public K-12 School District. Dissertation published by ProQuest. ProQuest Number: 10801469. University of Southern California.

Hermans, R., Tondeur, J., van Braak, J., & Valcke, M. (2008). The impact of primary school teachers’ educational beliefs on the classroom use of computers. Computers & education, 51(4), 1499-1509.

Hidi, S., & Ainley, M. (2008). Interest and self-regulation: Relationships between two variables that influence learning. In D. H. Schunk and B. J. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and application (pp. 77-109). Mahwah, NJ: Erlbaum.

Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational psychologist, 41(2), 111-127.

Hixon, E., & Buckenmeyer, J. (2009). Revisiting technology integration in schools: Implications for professional development. Computers in the Schools, 26(2), 130-146.

Honey, M., Pearson, G., & Schweingruber, H. (Eds.). (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, DC: National Academies Press.

Howland, J. L., Jonassen, D. H., & Marra, R. M. (2012). Meaningful learning with technology. Upper Saddle River, NJ: Pearson.

237

Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum.

Hoy, A. W., & Davis, H. A. (2005). Teachers’ sense of efficacy and its influence on the achievement of adolescents. In T. Urdan & F. Pajares (Eds.), Adolescence and education: Volume V: Self-efficacy beliefs during adolescence (pp. 117-137). Greenwich, CT: Information Age.

Hsiao, H. S., Chen, J. C., Lin, C. Y., Zhuo, P. W., & Lin, K. Y. (2019). Using 3D printing technology with experiential learning strategies to improve preengineering students' comprehension of abstract scientific concepts and hands‐on ability. Journal of Computer Assisted Learning. 35(2), 178-187

Hsu, P. S. (2016). Examining current beliefs, practices and barriers about technology integration: A case study. TechTrends, 60(1), 30-40.

Huddlestone, J., & Pike, J. (2008). Seven key decision factors for selecting e-learning. Cognition, Technology & Work, 10(3), 237–247.

Inan, F. A., & Lowther, D. L. (2010). Factors affecting technology integration in K-12 classrooms: A path model. Educational Technology Research and Development, 58(2), 137-154.

Internatinoal Society for Technology in Education (ISTE). (2019). Essential Conditions. Retrieved June 10, 2019, from https://www.iste.org/standards/essential-conditions

Jo, W. (2016). Introduction of 3D printing technology in the classroom for visually impaired students. Journal of Visual Impairment & Blindness, 110(2), 115-121.

Johnson, C. C. (2013). Conceptualizing integrated STEM education. School Science and Mathematics, 113(8), 367-368.

Jones, V. R. (2014). Teaching STEM: 21st century skills. Children's Technology & Engineering, 18(4), 11-13.

Kagan, D. M. (1992). Implication of research on teacher belief. Educational psychologist, 27(1), 65-90.

Kelley, T. R., & Knowles, J. G. (2016). A conceptual framework for integrated STEM education. International Journal of STEM Education, 3(1), 1-11.

Kenward, M. G., & Roger, J. H. (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics & Data Analysis, 53(7), 2583-2595.

Kim, C., Kim, M. K., Lee, C., Spector, J. M., & DeMeester, K. (2013). Teacher beliefs and technology integration. Teaching and teacher education, 29, 76-85.

https://www.iste.org/standards/essential-conditions

https://www.iste.org/standards/essential-conditions

238

Koehler, K. E. (2017). Examining the conceptual understandings of geoscience concepts of students with visual impairments: Implications of 3-D printing (Order No. 10702675). Available from ProQuest Dissertations & Theses Global. (1999311016). Doctoral dissertation, The Ohio State University.

Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70.

Koehler, M. J., Mishra, P., & Cain, W. (2013). What is technological pedagogical content knowledge (TPACK)? Journal of Education, 193(3), 13-19.

Kozma, R. B. (1991). Learning with media. Review of Educational Research, 61(2), 179-211. doi:10.2307/1170534

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational Technology Research and Development, 42(2), 7-19. doi:10.1007/BF02299087

Krapp, A. (2002). Structural and dynamic aspects of interest development: Theoretical considerations from an ontogenetic perspective. Learning and instruction, 12(4), 383-409.

Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into practice, 41(4), 212-218.

Kwon, H. (2017). Effects of 3D printing and design software on students' overall performance. Journal of STEM Education: Innovations and Research, 18(4), 37-42.

Laboy-Rush, D. (2011). Integrated STEM education through project-based learning. Retrieved from https://www.rondout.k12.ny.us/common/pages/DisplayFile.aspx?itemId=16466975

Lazarides, R., Buchholz, J., & Rubach, C. (2018). Teacher enthusiasm and self-efficacy, student-perceived mastery goal orientation, and student motivation in mathematics classrooms. Teaching and Teacher Education, 69, 1-10.

Lazarus, R. S., & Launier, R. (1978). Stress-related transactions between person and environment. In L. A. Pervin & M. Lewis (Eds.), Perspectives in interactional psychology. New York, NY: Plenum Press.

Lent, R. W., & Brown, S. D. (1996). Social cognitive approach to career development: An overview. The Career Development Quarterly, 44(4), 310-321.

Lent, R. W., & Brown, S. D. (2006). Integrating person and situation perspectives on work satisfaction: A social-cognitive view. Journal of vocational behavior, 69(2), 236-247.

239

Lent, R. W., Brown, S. D., & Hackett, G. (2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of counseling psychology, 47(1), 36-49.

Lent, R. W., Brown, S. D., Sheu, H.-B., Schmidt, J., Brenner, B. R., Gloster, C. S., Wilkins, G., Schmidt, L. C., Lyons, H., & Treistman, D. (2005). Social cognitive predictors of academic interests and goals in engineering: Utility for women and students at historically black universities. Journal of Counseling Psychology, 52(1), 84-92.

Lent, R. W., Lopez Jr., A. M., Lopez, F. G., & Sheu, H. B. (2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73(1), 52-62.

Lent, R. W., Lopez, F. G., Sheu, H., & Lopez, A. M. (2011). Social cognitive predictors of the interests and choices of computing majors: Applicability to underrepresented students. Journal of Vocational Behavior, 78(2), 184-192. doi: 10.1016/j.jvb.2010.10.006

Lent, R. W., Sheu, H. B., Gloster, C. S., & Wilkins, G. (2010). Longitudinal test of the social cognitive model of choice in engineering students at historically Black universities. Journal of Vocational Behavior, 76(3), 387-394.

Leshin, C. B., Pollock, J., & Reigeluth, C. M. (1992). Instructional design strategies and tactics. Englewood Cliffs, NJ: Educational Technology Publications.

Li, Y., Garza, V., Keicher, A., & Popov, V. (2018). Predicting High School Teacher Use of Technology: Pedagogical Beliefs, Technological Beliefs and Attitudes, and Teacher Training. Technology, Knowledge and Learning, 1-18.

Maloney, E. A., & Beilock, S. L. (2012). Math anxiety: Who has it, why it develops, and how to guard against it. Trends in cognitive sciences, 16(8), 404-406.

Maloy, R., Trust, T., Kommers, S., Malinowski, A., & LaRoche, I. (2017). 3D modeling and printing in history/social studies classrooms: Initial lessons and insights. Contemporary Issues in Technology and Teacher Education, 17(2), 229-249.

Maltese, A. V., Melki, C. S., & Wiebke, H. L. (2014). The nature of experiences responsible for the generation and maintenance of interest in STEM. Science Education, 98(6), 937-962.

Maltese, A. V., & Tai, R. H. (2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among US students. Science Education, 95(5), 877-907.

240

Manglicmot, T. M. (2015). Relationship between teacher self-efficacy, outcome expectations and the level of technology integration in secondary classrooms. Dissertation published by ProQuest. UMI 3663020. The College of William and Mary.

Marcinkiewicz, H. R. (1994). Differences in computer use of practicing versus preservice teachers. Journal of Research on Computing in Education, 27(2), 184-197.

Martin, A. J., Ginns, P., & Papworth, B. (2017). Motivation and engagement: Same or different? Does it matter? Learning and Individual Differences, 55, 150-162.

Matthews, J. S., Ponitz, C. C., & Morrison, F. J. (2009). Early gender differences in self-regulation and academic achievement. Journal of educational psychology, 101(3), 689-704.

Mcloughlin, L., Fryazinov, O., Moseley, M., Sanchez, M., Adzhiev, V., Comninos, P., & Pasko, A. (2016). Virtual sculpting and 3D printing for young people with disabilities. IEEE computer graphics and applications, 36(1), 22-28.

Meichenbaum, D. H. (1977). Cognitive-behavior modification: An integrative approach. New York, NY: Plenum Press.

Midgley, C., Feldlaufer, H., & Eccles, J. S. (1989). Change in teacher efficacy and student self-and task-related beliefs in mathematics during the transition to junior high school. Journal of educational Psychology, 81(2), 247-258.

Miller, J. A. (2015). Predictors of student persistence in the STEM pipeline: Activities outside the classroom, parent aspirations, and student self-beliefs using NELS:88 data. Retrieved from ProQuest Dissertations & Theses Global.

Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teacher knowledge. Teachers College Record, 108 (6), 1017-1054.

Mitchell, M. (1993). Situational interest: Its multifaceted structure in the secondary school mathematics classroom. Journal of educational psychology, 85(3), 424-436.

Mojavezi, A., & Tamiz, M. P. (2012). The Impact of Teacher Self-efficacy on the Students' Motivation and Achievement. Theory & Practice in Language Studies, 2(3), 483-491.

Moore, T., Stohlmann, M., Wang, H., Tank, K., Glancy, A., & Roehrig, G. (2014). Implementation and integration of engineering in K-12 STEM education. In S. Purzer, J. Strobel, & M. Cardella (Eds.), Engineering in pre-college settings: Synthesizing research, policy, and practices (pp. 35-60). West Lafayette, IN: Purdue University Press.

241

Mustafa, N., Ismail, Z., Tasir, Z., Said, M., & Haruzuan, M. N. (2016). A meta-analysis on effective strategies for integrated STEM education. Advanced Science Letters, 22(12), 4225-4228.

Muthén, B. O. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28, 338-354.

Muthén, B. O. (1994). Multilevel covariance structure analysis. Sociological Methods and Research, 22, 376-398.

Muthén, B. O., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. In R. D. Bock (Ed.), Multilevel analysis of educational data (pp. 87−99). San Diego: Academic Press.

Muthén, B. O., & Satorra, A. (1995). Complex sample data in structural equation modeling. In P. V. Marsden (Ed.), Sociological methodology, 1995 (pp. 267−316). Oxford: Blackwell.

Mortimore, P. (Ed.). (1999). Understanding Pedagogy and its impact on learning. Thousand Oaks, CA: Sage Publications Inc.

Mueller, J., Wood, E., Willoughby, T., Ross, C., & Specht, J. (2008). Identifying discriminating variables between teachers who fully integrate computers and teachers with limited integration. Computers & education, 51(4), 1523-1537.

National Research Council (NRC). (2010). Exploring the Intersection of Science Education and 21st Century Skills: A Workshop Summary. Washington, DC: The National Academies Press. Retrieved from https://doi.org/10.17226/12771.

National Research Council (NRC). (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: National Academies Press.

National Research Council (NRC). (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington DC: National Academy of Science.

Nauta, M. M., & Epperson, D. L. (2003). A longitudinal examination of the social-cognitive model applied to high school girls' choices of nontraditional college majors and aspirations. Journal of Counseling Psychology, 50(4), 448-457.

Nemorin, S., & Selwyn, N. (2017). Making the best of it? Exploring the realities of 3D printing in school. Research Papers in Education, 32(5), 578-595.

Ng, O. L. (2017). Exploring the use of 3D Computer-Aided Design and 3D Printing for STEAM Learning in Mathematics. Digital Experiences in Mathematics Education, 3(3), 257-263.

242

Niederhauser, D. S., & Stoddart, T. (2001). Teachers’ instructional perspectives and use of educational software. Teaching and teacher education, 17(1), 15-31.

O'Mahony, C. (2003). Getting the information and communications technology formula right: access+ ability= confident use. Technology, Pedagogy and Education, 12(2), 295-311.

Ottenbreit-Leftwich, A. T., Glazewski, K. D., Newby, T. J., & Ertmer, P. A. (2010). Teacher value beliefs associated with using technology: Addressing professional and student needs. Computers & education, 55(3), 1321-1335.

Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of educational research, 62(3), 307-332.

Pajares, M. F. (2005). Gender differences in mathematics self-efficacy beliefs. In A. M. Gallagher & J. C. Kaufman (Eds.), Gender differences in mathematics: An integrative psychological approach (pp. 294–315). New York: Cambridge University Press.

Pan, Y. H. (2014). Relationships among teachers’ self-efficacy and students’ motivation, atmosphere, and satisfaction in physical education. Journal of Teaching in Physical Education, 33(1), 68-92.

Park, S. H., & Ertmer, P. A. (2007). Impact of problem-based learning (PBL) on teachers’ beliefs regarding technology use. Journal of research on technology in education, 40(2), 247-267.

Partnership for 21st Century Skills (P21). Framework for 21st century learning. Retrieved from http://www.p21.org/our-work/p21-framework.

Pearson, G. (2017). National academies piece on integrated STEM. The Journal of Educational Research, 110(3), 224-226. https://doi.org/10.1080/00220671.2017.1289781

Pelgrum, W. J. (2001). Obstacles to the integration of ICT in education: results from a worldwide educational assessment. Computers & education, 37(2), 163-178.

Peugh, J. L. (2010). A practical guide to multilevel modeling. Journal of school psychology, 48(1), 85-112.

Pintrich, P. R. (1990). Implications of psychological research on student learning and college teaching for teacher education. In W. R. Houston (Ed.), Handbook of research on teacher education (pp. 826-857). New York, NY: Macmillan.

http://www.p21.org/our-work/p21-framework

https://doi.org/10.1080/00220671.2017.1289781

243

President’s Committee of Advisors on Science and Technology (PCAST). (2010). Prepare and inspire: K-12 education in Science, Technology, Engineering, and Math (STEM) for America’s future. Washington, DC: Executive Office of the President. Retrieved from https://nsf.gov/attachments/117803/public/2a--Prepare_and_Inspire--PCAST.pdf.

Pringle, R. M., Dawson, K., & Ritzhaupt, A. D. (2015). Integrating science and technology: Using technological pedagogical content knowledge as a framework to study the practices of science teachers. Journal of Science Education and Technology, 24(5), 648-662.

Ramirez, G., Gunderson, E. A., Levine, S. C., & Beilock, S. L. (2013). Math anxiety, working memory, and math achievement in early elementary school. Journal of Cognition and Development, 14(2), 187-202.

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods, 2nd ed. Newbury Park, CA: Sage.

Ravitz, J. L., Becker, H. J., & Wong, Y. T. (2000, July). Constructivist-compatible beliefs and practices among U.S. teachers (Report # 4). Teaching, learning, and computing: 1998 National survey. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.307.315&rep=rep1&type=pdf on December 26, 2018.

Renninger, K. A. (1992). Individual interest and development: Implications for theory and practice. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 361-395). Hillsdale, NJ: Erlbaum.

Renninger, K. A., & Hidi, S. (2002). Student interest and achievement: Developmental issues raised by a case study. In A. Wigfield & J. S. Eccles (Eds.), Development of achievement motivation (pp. 173-195). San Diego, CA: Academic Press.

Reeve, J. (2012). A self-determination theory perspective on student engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 149-172). Boston, MA: Springer.

Reiser, R.A., & Gagné, R.M. (1983). Selecting media for instruction, Englewood Cliffs, NJ: Educational Technology.

Richey, R., Tracey, M. W., & Klein, J. D. (2011). The instructional design knowledge base: Theory, research, and practice. New York, NY: Routledge.

Riskowski, J. L., Todd, C. D., Wee, B., Dark, M., & Harbor, J. (2009). Exploring the effectiveness of an interdisciplinary water resources engineering module in an eighth grade science course. International Journal of Engineering Education, 25(1), 181–195.

244

Roblyer, M. D., & Doering, A. H. (2013). Integrating educational technology into teaching (6th ed.). Boston, MA: Pearson.

Romiszowski, A. J. (1981). Designing instructional systems: Decision making in course planning and curriculum design. London: Kogan Page, Ltd.

Ross, J. A. (1994). Beliefs that make a difference: The origins and impacts of teacher efficacy. Paper presented at the annual meeting of the Canadian Society for the Study of Education, Calgary. Retrieved from https://eric.ed.gov/?id=ED379216 on May 31, 2019.

Sadler, P. M., Sonnert, G., Hazari, Z., & Tai, R. (2012). Stability and volatility of STEM career interest in high school: A gender study. Science Education, 96(3), 411-427.

Saettler, P. (1990). The evolution of American educational technology. Englewood, CO: Libraries Unlimited.

Sahin, A., Ekmekci, A., & Waxman, H. C. (2017). The relationships among high school STEM learning experiences, expectations, and mathematics and science efficacy and the likelihood of majoring in STEM in college. International Journal of Science Education, 39(11), 1549-1572.

Sanders, M. (2009). STEM, STEM education, STEMmania. The Technology Teacher, 68(4), 20-26.

Sansone, C. (2009). What’s interest got to do with it? Potential trade-offs in the self-regulation of motivation. In J. P. Forgas, R. Baumeister, and D. Tice (Eds.), Psychology of self-regulation: Cognitive, affective, and motivational processes (pp. 35-51). New York, NY: Psychology Press.

Sarason, I. G. (1975). Anxiety and self-preoccupation. In I. G. Sarason & D. C. Spielberger (Eds.), Stress and anxiety (Vol. 2). Washington, D.C.: Hemisphere.

Schelly, C., Anzalone, G., Wijnen, B., & Pearce, J. M. (2015). Open-source 3-D printing technologies for education: Bringing additive manufacturing to the classroom. Journal of Visual Languages & Computing, 28, 226-237.

Schiefele, U. (2009). Situational and individual interest. In K. R. Wentzel & A. Wigfield (Eds.), Handbook of motivation in school (pp. 197-223). New York, NY: Taylor Francis.

Schiefele, U., & Schaffner, E. (2015). Teacher interests, mastery goals, and self-efficacy as predictors of instructional practices and student motivation. Contemporary Educational Psychology, 42, 159-171.

https://eric.ed.gov/?id=ED379216

245

Schmidt, D. A., Baran, E., Thompson, A. D., Mishra, P., Koehler, M. J., & Shin, T. S. (2009). Technological pedagogical content knowledge (TPACK) the development and validation of an assessment instrument for preservice teachers. Journal of research on Technology in Education, 42(2), 123-149.

Schmidt, J. A., Rosenberg, J. M., & Beymer, P. N. (2018). A person‐in‐context approach to student engagement in science: Examining learning activities and choice. Journal of Research in Science Teaching, 55(1), 19-43.

Schunk, D. H. (1995). Self-Efficacy and Education and Instruction. In Maddux J.E. (eds), Self-Efficacy, Adaptation, and Adjustment - The Plenum Series in Social/Clinical Psychology (pp. 281-303). Boston, MA: Springer.

Schunk, D. H., & Mullen, C. A. (2012). Self-efficacy as an engaged learner. In S. Christenson, A. Reschly, C. Wylie (Eds.), Handbook of research on student engagement (pp. 219-235). Boston, MA: Springer.

Schunk, D. H., & Pajares, F. (2002). The development of academic self-efficacy. In A. Wigfield & J. S. Eccles (Eds.), Development of achievement motivation: A volume in the educational psychology series (pp. 15-31). San Diego, CA: Academic Press. Retrieved from: http://www.uky.edu/~eushe2/Pajares/SchunkPajares2001.PDF

Scott, A. B., & Mallinckrodt, B. (2005). Parental emotional support, science self‐efficacy, and choice of science major in undergraduate women. The Career Development Quarterly, 53(3), 263-273.

Shahzad, K., & Naureen, S. (2017). Impact of Teacher Self-Efficacy on Secondary School Students' Academic Achievement. Journal of Education and Educational Development, 4(1), 48-72.

Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-22.

Simpkins, S. D., Davis-Kean, P. E., & Eccles, J. S. (2006). Math and science motivation: A longitudinal examination of the links between choices and beliefs. Developmental psychology, 42(1), 70-83.

Smarkola, C. (2008). Efficacy of a planned behavior model: Beliefs that contribute to computer usage intentions of student teachers and experienced teachers. Computers in Human Behavior, 24(3), 1196-1215.

Snoeyink, R., & Ertmer, P. A. (2001). Thrust into Technology: How Veteran Teachers Respond. Journal of Educational Technology Systems, 30(1), 85-111.

246

Stohlmann, M., Moore, T. J., & Roehrig, G. H. (2012). Considerations for teaching integrated STEM education. Journal of Pre-College Engineering Education Research, 2(1), 28-34.

Swayne, J. M. (2017). Staying Connected: Measuring the Impact of Technology Integration on Student Engagement and Achievement at the Middle Level. Unpublished doctoral dissertation. Concordia University, Portland. Retrieved from https://commons.cu-portland.edu/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1105&context=edudissertations

Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Life science, 1, 0-2.

Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A. ... & Hellinckx, L. (2018). Integrated STEM Education: A Systematic Review of Instructional Practices in Secondary Education. European Journal of STEM Education, 3(1), 02. https://doi.org/10.20897/ejsteme/85525

Thornburg, D., Thornburg, N., & Armstrong, S. (2014). The invent to learn guide to 3D printing in the classroom: Recipes for success. Constructing Modern Knowledge Press.

Tondeur, J., van Braak, J., Ertmer, P. A., & Ottenbreit-Leftwich, A. (2017). Understanding the relationship between teachers’ pedagogical beliefs and technology use in education: a systematic review of qualitative evidence. Educational Technology Research and Development, 65(3), 555-575.

Trust, T., & Maloy, R. W. (2017). Why 3D Print? The 21st-Century Skills Students Develop While Engaging in 3D Printing Projects. Computers in the Schools, 34(4), 253-266.

Tweed, S. R. (2013). Technology implementation: Teacher age, experience, self-efficacy, and professional development as related to classroom technology integration. Dissertation published by ProQuest. UMI 3570301. East Tennessee State University.

Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The development and validation of a measure of student attitudes toward science, technology, engineering, and math (S-STEM). Journal of Psychoeducational Assessment, 33(7), 622-639.

Urban, M. J., & Favlo, D. A. (Ed.). (2016). Improving K-12 STEM Education Outcomes through Technological Integration. Hershey, PA: IGI Global.

Vongkulluksn, V. W., Xie, K., & Bowman, M. A. (2018). The role of value on teachers' internalization of external barriers and externalization of personal beliefs for classroom technology integration. Computers & Education, 118, 70-81.

https://commons.cu-portland.edu/cgi/viewcontent.cgi?referer=https://scholar.google.com/&httpsredir=1&article=1105&context=edudissertations



247

Wang, H. H., Moore, T. J., Roehrig, G. H., & Park, M. S. (2011). STEM integration: Teacher perceptions and practice. Journal of Pre-College Engineering Education Research, 1(2), 1-13. https://doi.org/10.5703/1288284314636

Wang, M. T., & Degol, J. (2013) Motivational pathways to STEM career choices: Using expectancy-value perspective to understand individual and gender differences in STEM fields. Developmental Review, 33(4), 304–340

Wang, M. T., & Eccles, J. S. (2013). School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective. Learning and Instruction, 28, 12-23.

Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50(5), 1081-1121.

Watson, G. R. (2006). Technology professional development: Long-term effects on teacher self-efficacy. Journal of Technology and Teacher Education, 14(1), 151-165.

Weber, J., Kotsopoulos, D., & Senger, N. (2017). Visual-spatial reasoning, design engineering, and 3D printing. In Galindo, E., & Newton, J., (Eds.), Proceedings of the 39th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. Indianapolis, IN: Hoosier Association of Mathematics Teacher Educators.

Welsh, J., Harmes, J. C., & Winkelman, R. (2011). Florida’s Technology Integration Matrix. Principal Leadership, 12(2), 69-71.

Wiebe, E., Unfried, A., & Faber, M. (2018). The Relationship of STEM Attitudes and Career Interest. EURASIA Journal of Mathematics, Science and Technology Education, 14(10), em1580.

Wigfield, A. (1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational psychology review, 6(1), 49-78.

Wigfield, A., & Cambria, J. (2010). Students’ achievement values, goal orientations, and interest: Definitions, development, and relations to achievement outcomes. Developmental Review, 30(1), 1-35.

Wigfield, A., & Eccles, J. S. (1992). The development of achievement task values: A theoretical analysis. Developmental Review, 12(3), 265-310.

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary educational psychology, 25(1), 68-81.

248

Wigfield, A., & Meece, J. L. (1988). Math anxiety in elementary and secondary school students. Journal of Educational Psychology, 80(2), 210-216. http://dx.doi.org/10.1037/0022-0663.80.2.210

Wozney, L., Venkatesh, V., & Abrami, P. C. (2006). Implementing computer technologies: Teachers' perceptions and practices. Journal of Technology and Teacher Education, 14, 173-207.

Xie, Y., & Reider, D. (2014). Integration of innovative technologies for enhancing students’ motivation for science learning and career. Journal of Science Education and Technology, 23(3), 370-380.

Zee, M., & Koomen, H. M. (2016). Teacher self-efficacy and its effects on classroom processes, student academic adjustment, and teacher well-being: A synthesis of 40 years of research. Review of Educational research, 86(4), 981-1015.

Zhao, Y., & Frank, K. A., (2003). An ecological analysis of factors affecting technology use in schools. American Educational Research journal, 40, 807-840.

Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational Psychology, 25(1), 82-91.

https://psycnet.apa.org/doi/10.1037/0022-0663.80.2.210

249

BIOGRAPHICAL SKETCH

Li Cheng received her bachelor’s degree in public administration in China in

2010. She found her interest in technology integration when she taught 5th grade

English in a summer program in China. In 2011, she came to West Liberty University

(WLU) in West Virginia, the United States to pursue her master’s degree in education

with the emphasis of technology integration. During her study in the master’s program,

she initiated a Chinese club on campus and served as the president. She offered free

Chinese language and culture classes and cultural events on campus. Her initiation of

the Chinese club not only promoted the language and cultural communication between

American, Chinese, and other international students but also provided her with the

opportunity to integrate technology and teach a diverse group of students.

After graduation from WLU, Li Cheng worked as a Chinese language professor

at Marietta College in Ohio. She built three Mandarin Chinese courses in the Moodle

learning management system. Using her knowledge and skills in technology integration,

she designed, developed, and implemented a series of technological resources

including animations to engage students and enhance their learning. She was also the

language lab director to design and implement learning activities by using various

technology resources. She gained much experience in teaching with innovative use of

technologies. Her abilities in working with and mentoring a diverse group of students

have also been naturally nurtured through her teaching and leadership process.

Li Cheng’s teaching at Marietta College further increased her interest in

educational technology. In August 2015, she was admitted into the educational

technology Ph.D. program at the University of Florida. In addition to fulfilling the

coursework requirements, she worked as a Graduate Instructor and a Research

250

Assistant and she passionately involved in teaching and research. She taught EME2040

Introduction to Educational Technology, a blended course with online and face-to-face

classes, to undergraduate students for three semesters. She was the leading instructor

of EME2040 for Fall 2016 and she mentored three new instructors and coordinated this

course. She also co-taught EME6606 Advanced Instructional Design, an online course,

to graduate students including Ph.D. and Ed.D. students with diverse backgrounds. She

effectively facilitated the online discussions on the weekly instructional design case

analysis.

After her teaching as an independent instructor, she had the opportunity to work

as a Research Assistant. She worked for several NSF or IES funded projects and she

was a vital member of the project teams. In addition to assisting the funded projects,

she has also been an independent and active researcher. As an instructor and a

researcher, she has always been inspired to enhance teaching and learning through

innovative use of technologies. She has conducted several studies to investigate

effective instructional strategies and learning strategies to enhance teaching and

learning in technology-enhanced learning environments. During her teaching of the

Introduction to Educational Technology course to undergraduate students, she

conducted a study on engaging students through a technology-enhanced peer feedback

activity in presentation classes. The learning activity highly engaged students and

received very good feedback from students. Almost all the students would like to do this

activity again in their future learning. To promote students’ deep learning, she

conducted an experimental study to examine the effects of two generative learning

strategies (i.e., student generated-drawing and imagination) on science text reading in a

251

computer-based learning environment and her paper received the Distinguished Paper

Award with $500 at the 2018 Annual Conference of Florida Education and Research

Association (FERA). She recently published an article entitled “Effects of student-

generated drawing and imagination on science text reading in a computer-based

learning environment” in Educational Technology Research and Development, one of

the top journals in Educational Technology field. In addition to the effectiveness of

learning strategies, she is also devoted to research on effective instructional strategies

in technology-enhanced learning environments. She conducted a meta-analysis study

on the effects of Flipped Classroom instructional strategy. This research gained her the

Best Poster Award at the 2017 Annual Conference of FERA. Her paper entitled “Effects

of the flipped classroom instructional strategy on students’ learning outcomes: A

meta‐analysis” has been published in the journal Educational Technology Research and

Development. In addition to research on learning strategies and instructional strategies

for effective teaching and learning in technology-enhanced learning environments, her

research also aims to address pressing educational problems such as reducing the

digital divide in education. She collaborated with her professor Dr. Albert Ritzhaupt on a

book chapter entitled “The Digital Divide in Formal Educational Settings: The Past,

Present, and Future Relevance”, which is in press in the prestigious Handbook of

Research on Educational and Communications Technology. Another book chapter

entitled “Using Technology to Address Individual Differences in Cognitive Processing”,

on which she collaborated with her professors Dr. Pavlo Antonenko and Dr. Kara

Dawson, is also in press in that handbook.

252

Along with her dedication to teaching and research, she has regularly presented

at international academic conferences such as American Education and Research

Association (AERA), the Association for Educational Communications and Technology

(AECT), and the Florida Educational Research Association (FERA). She has also been

devoted to providing professional services. She worked as a volunteer classroom

assistant for a 3rd-grade math inclusion class in which some students had special

needs. She served as the Graduate Student Coordinator for Florida Educational and

Research Association (FERA) for the year of 2017 and 2018. Since 2014, she has

served as an Assistant to the Editor for The Excellence in Education Journal. She

reviewed articles for the journal Computers & Education, the TechTrends journal, and

the Journal of Educational Computing Research. She has also regularly reviewed

proposals for academic conferences including AERA, AECT, and the International

Conference of the Learning Sciences (ICLS).

The excellent education and the opportunities for teaching and research have

equipped her as an independent and thoughtful instructor and researcher. She is

determined to pursue her career goal in educational technology and make contributions

to education through vigorous research and teaching.

ufdcimages.uflib.ufl.edu · 4 acknowledgments i thank my supervisor and chair dr. albert ritzhaupt...

Documents