Research Proposal

Online Student Satisfaction

Susan Wilson

University of British Columbia

Introduction

In his 2006 research article, Herbert stated that the rate of discontinuance in

online, post-secondary courses is ten to twenty percent higher than that of traditional

classroom-based courses. As online education moves into secondary education

institutions, the problem of retention will compound. Post-secondary institutions work

with students who have already successfully completed one tier of education; students

who are motivated to learn and have a financial interest in successful course completion.

Public secondary schools educate students of varied abilities, attitudes, and backgrounds.

I believe that the rate of discontinuance in online, secondary courses is even greater than

that of online, post-secondary courses.

According to the 2008 Saskatchewan Education Indicators Report,

Saskatchewan’s population of 1 015 985 on July 1, 2008 was made up of approximately

40% of people aged 45 and over and only 27% of people aged 20 or younger. In addition

to the relatively small population, the report states that the number of people living in

rural communities has declined at a higher rate (11% decrease from 1971 to 2001) than

the Canadian average (4% decrease from 1971 to 2001). Saskatchewan’s large

geographical area and dispersed population make it necessary to operate small schools,

some of which have fewer than 50 students.

Online learning is a practical solution to challenges presented by rural

depopulation and a large geographical area. It is also a viable learning environment for

adult students, students who are required to work to support themselves financially, and

students who have significant family responsibilities. Saskatchewan’s 2005 rate of teen

pregnancy was the highest among the provinces and more than double that of the

Canadian rate (Saskatchewan Indicators, 2008).

The 2008 Saskatchewan Indicators report suggests that approximately 20% of

youth do not graduate. Retention of secondary courses is an important consideration in

Saskatchewan and everywhere. Having a Grade 12 certificate increases the chances that

a person will be employed (Saskatchewan Indicators, 2008) and increases the probability

that a student will continue their education. Having a post-secondary certificate, diploma

or degree increases the average income level to an average of 33% (statistic from 2005)

higher than that earned by those without the post-secondary training (Saskatchewan

Indicators).

As an online educator in rural Saskatchewan, I am motivated to learn how to best

implement online education opportunities into our high-schools. Rural depopulation and

economic factors translate to teacher cuts in our school division. If we are to remain

viable, we must offer and accept online education into our school. What factors must we

be cognizant of when designing online courses? How can we best support our students

before and during their involvement in online education?

Online education from K-Post Secondary reaches millions of students every day

(Corry, 2008). If online courses are being offered to students in situations like ours, then

students are not making a voluntary choice to take a class online, they are required to take

online education to complete graduation requirements. At times, this is because small

rural schools do not have the staing to offer all required courses, they do not have the

staing to offer various electives, or they do not have the staff to offer multiple forms of a

class to accommodate students who have missed credits or changed schools. If courses

are not otherwise be available, we, as educators, must ensure that students are satisfied

with their online experience and that they are receiving quality education.

Education is for everyone and online education should not be any different. It is

imperative that reliable instruments are developed to measure student success and

provide quality assurance to all stakeholders of public education. Educators who have

access to an instrument that can predict the probability of student success will be able to

design pro-active interventions and supports to facilitate student success thus widening

the doors to online education.

With this in mind, this research proposal is designed to answer the following

questions:

1. What factors do students perceive to aid in their satisfaction and success in

online courses at the secondary level?

2. What factors do online educators perceive to aid in student satisfaction

and success in online courses at the secondary level?

In answering these questions, I hope to collect data that will aid in the development of a

predictive instrument for use with Saskatchewan high-school students entering into

online education. The data will also be used to aid in the development of a student

support system that will be used to target areas of student weakness in an effort to

increase their probability of success in secondary, online courses. The data will not be

used to limit or restrict enrolment in online, secondary courses.

Quantitative and qualitative data will be collected by using student and instructor

survey responses. I hypothesize that student demographics (age, gender, location of

residence) will not be identified as factors correlating to student success. I believe that

institutional factors such as instructional design, hardware and human infrastructure,

communication, time, and course organisation will be identified by both students and

instructors as being significantly correlated to student satisfaction and success. I expect

that personal student factors such as ability, experience and motivation will have an effect

on satisfaction and success, but that they will not be as significant as institutional factors.

While circumstantial variables may present themselves and influence both satisfaction

and success, they are not controllable by student, instructor or institution and will

therefore not be considered in this study.

Student satisfaction will refer to the degree to which students enjoy the

educational experience and are engaged and interested in the course. Student success

refers to a student’s passing the course and retention refers to finishing the course.

Literature Review

Studies relating to factors influencing student success and satisfaction in online education

are becoming more prevalent as online education undergoes rapid and continuing growth

(Dailey-Hebert et al, 2005; Herbert, 2006; Leong et al, 2006; Roblyer & Davis, 2008;

Yukselturk & Bulut, 2007). Expressing a concern with growing rates of discontinuance,

researchers (Dailey-Hebert et al, 2005; Herbert, 2006; Leong et al, 2006; Roblyer &

Davis, 2008) have attempted to identify factors relating to student success and student

satisfaction in online courses.

Articles collected for this review study factors affecting online student satisfaction (by

perception) and online student success (passing the course), with emphasis on retention

(completing the course). Research surrounding factors affecting student satisfaction in

online education is extremely important as students who are satisfied with their online

experience are more likely to retain the course (Herbert, 2006; Swan et al, 2006).

Assessing student satisfaction can be valuable in terms of program and course

improvement (Herbert).

The literature review is organized according to guidelines by Gay, Mills & Airasian

(2009) using the headings Post-Secondary Student Satisfaction, Post-Secondary Student

Success and High-School Student Success.

Post-Secondary Student Satisfaction

1. Staying the course: A study in online students’ satisfaction and retention.

Herbert’s (2006) cross-sectional study was designed to identify institutional variables

affecting retention. He used the Noel-Levitz Priorities Survey for Online Learners™,

which was e-mailed to every student enrolled in an online course at a small, Midwestern

university during the 2005/06 school year. Herbert cited evidence that student variables,

among other factors, can be measured to predict the degree to which a student will

complete an online course. He also referenced research findings that institutional and

demographic variables are predictive as well.

Analysis of survey results was done by the Noel Levitz company. Of the 4100 students,

122 completed surveys were returned representing 40.1% of the students who did not

successfully complete their online course. When asked for the reason for discontinuance,

61.3% cited time constraints and 16.1%, personal problems.

The most significant variable identified was faculty response to student needs followed

by the quality of online instruction, timely feedback, quick institutional response to

requests, and frequency of student-instructor interaction. Student-student interactions

scored lowest. There was no statistical difference based on institutional variables

between course completers and non-completers. However, results of an independent

samples t-test showed a statistically significant difference between satisfaction levels of

completers and non-completers. Students who were satisfied with their online experience

were more likely to retain the course.

2. An empirical investigation of student satisfaction with web-based courses.

This quantitative study surveyed 128 students enrolled in 29 University of Hawaii, online

courses to examine the relationship between demographic variables, online course

experience and satisfaction. The researchers’ rationale was to identify factors to increase

student satisfaction.

Based on previous research, Leong et al (2006) produced a 47-item survey to address

instruction, instructor characteristics, management, technology, interaction, experience,

workload and fairness of grading. Two additional questions based on overall course

satisfaction and comparisons to face-to-face instruction were asked. A response rate of

25.2% (128 surveys) was achieved and results were calculated using factor and

regression analysis. Also, t-test and univariate analysis of variance were used. Results

showed that overall student satisfaction was influenced by four dimensions: instructor,

system-wide technology, workload/difficulty and interaction but not by demographic

factors or students’ prior experience.

Post-Secondary Student Success

3. Learner attribute research juxtaposed with online instructor experience: Predictors

of success in the accelerated online classroom.

Dailey-Hebert et al (2005) compared characteristics of learner attributes with online

instructor experience. The rationale was to develop practice-orientated understanding of

factors predicting student success. Literature exists on online student retention, their

attributive influences and their reasons for choosing online delivery but Dailey-Hebert et

al posited a need for more information about predictors of success as research based on

internal learner attributes is of limited use unless paired with external data elicited from

online educators.

The study surveyed a self-selected sample of 96 online educators with an average

teaching experience level of 3.5 years. All educators offered accelerated, 8-week courses

and no demographics were collected on the sample however future studies may compare

educator characteristics with factors influencing success. An e-mail solicited their

opinion on the five factors most likely to predict successful completion of an online

course and a survey identified skills, strategies or factors they perceived to lead to student

success. A content analysis of the responses identified 23 relevant factors which were

then grouped into six themes, student competence, student initiative, student personal

issues, time, technology, and instructional factors.

The four most predictive issues were time [timely, active involvement in the course

(67.71%), effective time-management (67.71%) and timely access to instructional resources

(19.79%)], initiative [personal initiative (52.08%), asking questions and seeking help

(37.50%), self-motivation (22.92%) and a positive attitude (4.17%)], technology [access

to efficient computer and internet literacy (40.63)], and competence [reading comprehension

(23.96%), writing skills (22.96%), communication (17.71%), awareness of expectations,

environment and workload (16.67%), and organizational skills (13.54%)].

4. Predictors for Student Success in an Online Course

Yukselturk and Bulut (2007) designed a correlational study to analyse student

characteristic variables (gender, age, education level, locus of control, learning style),

motivational beliefs (intrinsic and extrinsic goal orientation, task value, self-efficacy, and

test anxiety) and self-regulated learning components (cognitive strategy use and self-

regulation) that predict online student success. They also examined instructor views on

predicative factors. They hypothesized that the variables do not explain variance in

student success.

The researchers posited a need for the inclusion of instructor perception. Their rationale

was to fill a void in the research and to identify procedures for the design of high-quality

online learning environments. In a well-written, extensive literature review, Yukselturk &

Bulut provided history on online education and stated a need for maintaining quality.

The authors defined terms used in the study.

The sample of two course instructors and 80 voluntary participants enrolled in the Data

Structure and Algorithms with C course at the Middle East Technical University in

Ankara, Turkey in the 2005/06 school year. All students were computer literate with

intermediate English. There were more males (N = 56) than females (N = 24) and the

majority were between 19 and 29 years of age. In addition, most had an assimilator

learning style and a university degree.

Quantitative and qualitative data was collected through four online questionnaires:

Demographic Survey, Internal-External Locus of Control Scale, Learning Style

Inventory, and Motivated Strategies for Learning Questionnaire. Semi-structured

interviews were conducted; the ten questions were developed around central themes then

examined by two experts in the field of instructional technology.

Intrinsic goal orientation, task value, self-efficacy, cognitive-strategy use, and self-

regulation were significantly positively correlated with success. Educational level and

external locus of control were negatively correlated. Varied student characteristics and

general personal characteristics, did not directly affect success.

5. Online student success: Making a difference

Beyrer’s (2005) study compared online students enrolled in Online Student Success

(OSS), a course designed to prepare students for online learning, and online students who

have never taken this class. He also compared online performance of students before and

after completion of OSS. Beyrer hypothesized that students who take OSS will be more

successful than those who do not and they will be more successful after completion of

OSS than they were previous.

Beyrer conducted a well-written, comprehensive literature review stating that most

research focused on instructional design factors affecting student success, tips to aid

student success, and technology and interface characteristics. Beyrer defined the terms

used in the study.

The convenience sample included all students enrolled in fully online classes from

Cosumnes River College, Sacramento CA from 2003 to 2005. They were divided into 4

subgroups: those who had never enrolled, those who enrolled and passed, those who

enrolled and did not pass and those who enrolled but dropped. Historical data (academic

performance, enrollment, and demographic) was collected and analyzed and an online

survey of students was conducted. Twenty-one respondents completed the 18-question

(open and closed answers) follow-up survey.

Compared to students who did not take OSS, the ones that successfully completed OSS

had a higher success rate in their online classes. Those who were unsuccessful or dropped

OSS had a lower success rate. Though only 11 students took online classes before and

after enrolling in OSS, the improvement in their success rate was dramatic. Follow-up

survey responses from all three OSS groups were positive.

High-School Student Success

6. Predicting success for virtual school students: Putting research-

based models into practice.

Roblyver and Davis (2008) posited that prediction models should be developed support

school-age student success in online courses. They explained that the model should be

based on the combined factors that contribute to predicting success as identified by

research and should provide efficient measurement and implementation in virtual school

settings. The article contains an example of a prediction model, the Educational Success

Prediction Instrument (ESPRI) and a description of data collection and statistical

processes used to derive it. They outlined procedures for implementation to increase the

accuracy and utility of predictions.

Model testing was conducted using 4,110 students in the Virtual High School Global

Consortium (VHS) who were enrolled in 196 VHS courses in 2006. An electronic version

of the instrument was placed in course spaces and students were offered 10 points extra

credit on their first week's assignments to complete the survey. A completed ESPRI

survey, demographic data, and course scores and status were obtained for 2,162 students

or about 53% of the total school population. Reliability with the 25-item instrument

was .92. The model correctly predicted 93% of those who were successful, but only

30.4% of those who failed.

Common Themes

In general, research methods have consisted of survey instruments distributed online to

students enrolled in online courses at the post-secondary level with few conducted at the

high-school level. All studies in this literature review are cross-sectional in nature

indicating a need for longitudinal research which can identify trends from which

appropriate revisions can be made (Herbert, 2006).

Herbert’s literature review found demographic factors to be predictive of satisfaction and

retention but the studies compiled by the other researchers showed conflicting

conclusions or no evidence that demographics can predict success. Dailey-Hebert et al

and Yukselturk and Bulut identified a prevalence of reseach based on the effects of

student characteristics. Both research studies widened their examination of predictive

factors to include the perception of experienced online instructors. Linking student

responses with faculty perceptions can be powerful in creating a practical understanding

of the influences on student satisfaction and success.

Faculty response to student needs and student-instructor interaction were significant

factors identified in the research studies. Time constraints and management issues were

also significant predictors of satisfaction and success. Yukselturk and Bulut were the

only researchers to focus on student characteristics such as motivation and self-

regulation.

Summary

All researchers indicated a desire to inform educators of desirable factors to influence

student satisfaction and student success in an online environment. Most stated a desire to

counter-act course discontinuance at the post-secondary level. Studies of student and

institutional factors produce varied and somewhat contradicting results. Longitudinal

research ventures may aid in identifying trends in student success that will better inform

improvements. Predictive instruments of student success will be helpful if they can be

refined to identify student supports needed and not used to limit access.

This idea that research findings provide actionable information is supported by

Beyrer (2005) and Roblyer (2008). Both projects involved the development, testing or

implementation of an instrument, one predicative and one instructive, used to determine

student suitability to online education and support students in need. Beyrer’s work

focused on the benefit of enrolling students in an Online Student Success (OSS) course

designed to improve probability of student success in online courses. Combining this

idea with Roblyer’s focus on predicative models for online student success would allow

institutions to offer pre-assessment and targeted support to potential students. It is

important to note that the purpose of these instruments is to increase enrolment and

success in online courses, not to rank, stream or prohibit students.

Methodology

Elaine Strachota (2008) developed a 27-item survey instrument, The Student

Satisfaction Survey, based on student satisfaction with respect to online interaction

(learner-content, learner-instructor, learner-learner, and learner-technology), and general

satisfaction. Strachota defined each construct. Adult and distance education experts

edited the survey questions to establish content-validity. Construct validity was

established by conducting a pilot study of 249 online students at a Midwest Technical

College in the US. Data from the pilot underwent factor analysis to verify that items

loaded on the intended constructs. Those that did not load were eliminated.

This research project will use survey question ideas from Strachota’s Student

satisfaction Survey as well as student readiness survey question ideas from Slick (2004)

to develop an instrument suited to secondary students. Content validity will be

established through review by the Instructional Technology Department Consultant and

Coordinators of the SouthEast Cornerstone School Division, SK. This will take place at

our next scheduled meeting on April 29, 2009. Construct validity will be established

through a trial with online students in our division which will take place in June, 2009.

Statistical analyses on data received will inform decisions about question retention or

removal. Each survey item will require a Likert-response consisting of a 4-point scale:

strongly agree, agree, disagree, strongly disagree.

Once finalized, the survey will be distributed to a convenience sample of students

enrolled in online education courses through the SouthEast Cornerstone School Division

through their course management system. Course instructors will notify students of the

survey by e-mail and will send up to three reminders over the two week implementation

period. Completion of the survey will be expected as part of the assigned course work

and will be given to students in the final two weeks of their course. Parental permission

will not be required as the research will be done through the students’ attending school

division for the purpose of improving instruction.

Data for this longitudinal study will be collected and analysed in January and June

of each year for two complete years. Once complete, a trends analysis will be performed

to determine factors affecting student satisfaction and success. Results will be quantified

by converting frequency of responses to percentages. Results from open-ended survey

questions will be compiled and categorised. Data from respondents indicating course

satisfaction and feelings of success will be evaluated for factor correlation. Data from

respondents who report dissatisfaction and lack of success will be evaluated for factor

correlation.

This data, analysed and interpreted over a two year period, will be used to aid in

the development of a predicative instrument for student success in online environments.

It will also be used to develop a student support system that targets factors influencing

student success.

Significance

Even though online education reaches millions of students every day (Corry,

2008), the field of online learning is still in its developmental stage with respect to having

valid and reliable measurement instruments (Strachots, 2008). Creating a reliable survey

instrument, developed for and piloted on secondary school students will add to the field

of distance education at the secondary level. Further work with survey results to develop

a predictive instrument and student support component will directly benefit online

students.

Student feedback is an important component in the provision of quality education

when data received is used to improve student instruction, satisfaction and chance of

success. Many secondary teachers are new to online delivery. Survey results will be an

important component of professional development for online teachers. Data obtained

will help inform department or even provincial policies and standards created to ensure

quality in online course delivery. Data obtained from online instructors will provide

good fodder for conversations around instructional design, pedagogy and best practices.

Continuing the research over the course of two years will provide more reliable

and normally distributed data allowing for trend-analysis of student satisfaction, success,

and retention at the secondary level. This information will provide a significant

contribution to the field of distance education at the secondary level.

Conclusion

Canadian school divisions are developing internal capacity for online course

design, delivery and implementation. Current research around student satisfaction and

success concentrates on post-secondary students. Research needs to be done on factors

affecting the satisfaction and success of online, secondary students.

As evidenced in the research, surveys are effective tools for gathering both

quantitative and qualitative data from students and instructors. Embedding the survey

into the course management system ensures that all involved are presented with the

opportunity to respond. Managing the survey distribution and data collection through

such technology reduces errors and strengthens the reliability of results. Field testing and

content validation will also increase reliability.

Research results will be used to develop a more refined survey instrument that

will predict student success in an online learning environment. The instrument will

identify individual student strengths and weaknesses for the purpose of providing student

support. Student satisfaction and success factors will also be considered in the

development of a student support instrument that will enable students to improve their

weaknesses thus improving their chances of retention and success in online courses.

References

Beyrer, Gregory M. D. (2005) Online student success: Making a difference. Cosumnes

River College. @ONE Carnegie Scholar. 2005-06. Retrieved from

http://www.cccone.org/scholars/05-06/GregBeyrerMonograph.pdf

Corry, M. (2008). Online student success: Six categories of importance to the future. In

K. McFerrin et al. (Eds.), Proceedings of Society for Information Technology and

Teacher Education International Conference 2008 (pp. 312-317). Chesapeake,

VA: AACE. Retrieved from www.editlib.org/index.cfm/files/paper_27178.pdf?

fuseaction=Reader.DownloadFullText&paper_id=27178

Dailey-Hebert, Amber, Donnelli, Emily, and Mandernach, B. Jean. (2006). Learner

attribute research juxtaposed with online instructor experience: Predictors of

success in the accelerated online classroom. The Journal of Educators Online,

Volume 3, Number 2, July 2006. Retrieved from

http://www.thejeo.com/Volume3Number2/MandernachFinal.pdf

Gay, L.R., Mills, G.E., & Airasian, P.W. (2009). Educational research: Competencies for

analysis and application (9th ed.). Upper Saddle River, NJ: Merrill Prentice Hall.

Herbert, Michael PhD. (2006). Staying the course: A study in online students

satisfaction and retention. Online Journal of Distance Learning Administration,

Volume IX, Number IV, Winter 2006 University of West Georgia, Distance

Education Center. Retrieved from

http://www.westga.edu/~distance/ojdla/winter94/herbert94.htm

Leong, Peter, Ho, Curtis P., Saromines-Ganne, Barbara. (Winter, 2006). An empirical

investigation of student satisfaction with web-based courses. Online Journal of

Distance Learning Administration, Volume IX, Number IV, Winter 2006.

University of West Georgia, Distance Education Center. Retrieved from

www.editlib.org/index.cfm/files/paper_9442.pdf?

fuseaction=Reader.DownloadFullText&paper_id=9442

Roblyer, M.D., Davis, Lloyd. (2008). Predicting success for virtual

school students: Putting research-based models into practice.

Online Journal of Distance Learning Administration. Winter 2008.

Volume 11. Issue 4. University of West Georgia. Retrieved from

http://www.westga.edu/~distance/ojdla/winter114/roblyer114.html

Saskatchewan Education Indicators: Pre-Kindergarten to Grade 12. 2008. Available

from http://www.education.gov.sk.ca/adx/aspx/adxGetMedia.aspx?

DocID=609,135,107,81,1,Documents&MediaID=6109&Filename=Web+Version

+2008.pdf

Slick, J. (2004). Using Student Readiness as a Predictor of Satisfaction in the Online

Environment. In G. Richards (Ed.), Proceedings of World Conference on E-

Learning in Corporate, Government, Healthcare, and Higher Education 2004

(pp. 2130-2136). Chesapeake, VA: AACE. Retrieved from

www.editlib.org/index.cfm/files/paper_11492.pdf?

fuseaction=Reader.DownloadFullText&paper_id=11492

Strachota, Elaine. (2008). The use of survey research to measure student satisfaction in

online courses. Online Journal of Distance Learning Administration, Volume XI,

Number IV, Winter, 2008. University of West Georgia, Distance Education

Center. Retrieved from

http://www.umsl.edu/divisions/conted/education/mwr2p06/pdfs/D/Strachota_Use

_of_Survey_Research.pdf

Swan, Karen; Schenker, Jason; Lin, YiMei; Shea, Peter;; Aviv, Reuven (2006).

Student satisfaction with online learning: A concept analysis. Proc. Annual

Meeting of the American, Educational Research Association, San Francisco, April

2006. Retrieved April 13 from http://tinyurl.com/d2twn2

Yukselturk, E. & Bulut, S. (2007). Predictors for Student Success in an Online Course.

Educational Technology & Society, 10 (2), 71-83. 71 ISSN 1436-4522 (online)

and 1176-3647 (print). © International Forum of Educational Technology &

Society (IFETS). Retrieved from http://www.ifets.info/journals/10_2/7.pdf