Download - PhD Presentation (Doctorate)
11
PhD. – Information SystemThesis Viva
By: Sharif Omar SalemSupervisor: Dr. Khaironi Yatim SharifCo-Supervisor: Dr. Ilham Sentosa
The design and format is done by me, feel free to use the same format. But I am expecting appreciation notification.
2
Research Title
Developing a Hybrid Success Model for Different Content Management Systems in Higher
Education:
A Comparative Analysis of Students’ Perspective on Traditional and SNS systems.
Contents
Subtitle Slide No.
Introduction 4
Theoretical Framework and Proposed Model 12
Research Methodology 18
Findings 30
Hypothesis Discussion 36
Contributions, and Recommendation 41
Publications 46
3
Introduction
4
Research Brief
This academic research Investigated the learners’ outcome and its determinants via experiencing two different treatments.
First treatment by using traditional CMS and second treatment by using FB-based CMS.
A survey based on a developed hybrid eLearning success model is used to collect data.
Findings analyzed to assess the relations in the causal model and to compare the outcome constructs acquired via experiencing the two different systems.
5
Research Motivations
During the last five years; many researchers announced and recommend for further and future research to fill up two gaps:
6
Theoretical gap: The need for a
revised e-Learning system success model
Practical gap: The need for
more understanding of the FB effect
in learning outcome.
7
Theoretical gap: The need for a revised e-Learning system success model
• “A goal of continuing research would be an exploration of how the ISS model would be supplemented in order to more accurately reflect the E-learning environment”.
In 2010; Freeze, Alshare, Lane, & Wen state that
• Factors from the community of inquiry frameworks such as metacognitive, motivational, and behavioral traits of active online students may be a valuable add-on to the eLearning success model
In 2010; Shea & Bidjerano state that
• Considering the perspective of all players of the eLearning system and including additional different factors is important for a proper representation of the system success
In 2012; Bhuasiri et al. state that
• System success dimensions are not technology only, the revolution of web 2.0 and the uniqueness of eLeaning environment especially the different stakeholders guide the researchers to seek for new revised model
Cheng 2012; Lee et al. 2009; Chen 2010; Keramati et al. 2011; Sun et al. 2008; Hassanzadeh et al. 2012; Wang & Chiu 2011
8
Practical gap: The affect of using Facebook in learning outcome.
• Consolidating the Facebook in the learning and teaching process is very important for the students’ education lives and further research is needed to understand this phenomenon
In 2011; Bicen & Cavus
• Future research is needed to investigate more the usability of FB in education
In 2012; LaRue state that
• There is a need for understanding the relation between the learners’ interaction level in the Facebook and their academic success
In 2012; Junco stated that
• Considering the perspective of all players of the eLearning system and including additional different factors is important for a proper representation of the system success
In 2013; Ng & Wong stated that
Problem Statement:
Content Management System (CMS) is widely used in most of the universities worldwide to facilitate higher education stakeholders’ communications.
The challenge is whether Facebook environment “as a CMS system” is favorable and more effective than the traditional CMS system and what determinants/constructs affect the eLearning success.
Recently, few academic studies begin investigating this field of application. Further investigation is needed to fill up the illustrated gaps.
9
Research Objectives:To identify the dimensions of the information system success in the modern e-learning environment, and propose a modified model for e-learning system success.
To implement and test the proposed model when the implemented system is Moodle based.
To implement and test the proposed model when the implemented system is Facebook based.
To compare between the findings of the Moodle-based system analysis and the findings of the Facebook-based system analysis.
10
1
3
2
4
11
Scope of the Study
The research use Facebook as a social network system and Moodle as a traditional CMS system.
The research focus in the application of the system to facilitate higher education.
Respondents for the application of the system are from the LUCT University, Malaysia. Students are participants for the master degree level.
12
Theoretical Framework and Proposed Model
13
Learning System
OutcomeNet Benefits
Information Quality
Service Quality
Intention to Use/ System Use
User Satisfaction
System Quality
Theory 1: Delone and McLean IS success model - 2002
14
Learning System
Outcome
Learning System
Outcome
Teaching Presence
Social Presence
Cognitive Presence
Theory 2: The Community of Inquiry - 2000
15
Learning System Outcome
Community System Success
Sociability
Usability
(System Dimension)
Theory 3: Preece’s sociability and usability framework – 2001
16
Learner’s Outcome
System Sociability
Information Quality
Service Quality
Intention to (Use)
User Satisfaction
Teaching Presence
Learner’s Presence
System Quality
Proposed Hybrid Model for IS Success of Modern LMS
Learner’s Outcome
Learner Outcome
D&M IS Success ModelCoI Theory Preece’s S&U Framework
17
Learning System
Outcome
Learner Outcome
System Sociability
Information Quality
Service Quality
Intention to (Use)
User Satisfaction
Teaching Presence
Learner Presence
System Quality
Research Hypothesis
HA1; HB1HA2; HB2
HA4; HB4
HA3; HB3
HA5; HB5
HA6; HB6
HA7; HB7
HA8; HB8
HA9; HB9
HA12; HB12
HA10; HB10
HA11; HB11
HA15;HB15HA
14; H
B14
HA13
; HB1
3
HA17;HB17
HA18;HB18
HA16;HB16
HC1
HC2
HC3
HA## Hypothesis Set of System 1HB## Hypothesis Set of System 2HC## Hypothesis Set of Comparison
• IU of the FB-based system is different from and higher than the Moodle-based system.• US of the FB-based system is different from and higher than the Moodle-based system.• LO of the FB-based system is different from and higher than the Moodle-based system.
18
Research Methodology
19
Research Design• The study starts up with a theory and ends up with testing
the hypothesis. Deductive Approach
• Statistical analysis based on descriptive measures, variance, covariance techniques are used.Quantitative
Research
• Literature review and systematic reviews techniques are used to build the hybrid model.Qualitative
Research
• This study aims to assess a desired field subjects in two different treatments then compare the outcomes. Field Experiment
Design
• “Counterbalanced Measures Design” technique is performed by assigning participants in different groups and applying treatments to each group in a different order.
Counterbalanced Measure Design
20
Research Design
Scientific R
esearch
Approach
Instrument Tool (Survey)
A questionnaire in English was derived for the study with two major sections.
The first section is for the demographics data, such as age, sex, gender, country, major study, and semester level.
The second section consists of 43 questions covered the independent, mediating and dependent constructs.
The five-point Likert scale with pre-coded numerical scales is used in order to measure the extent of respondent’s view.
21
22
Population and Sample
The population is
all the Module-
based Master students of LUCT ~ 362.
The minimum sample size
for SmartPls = 70.
the effective sample size
based on the statistical
power value = 153.
In reality, the analysed
sample = 231.
Experimental Design (Counterbalanced Measures Design)
This research is Semi-Experimental design use
Field Experiment.
This approach is a mix between the “Between Subjects Design” and
“Repeated Measure Design”
23
Group Pretest TreatmentJan-Feb 2014 Test Treatment
Mar-Apr 2014 Posttest
Group A No Survey Moodle-Based Survey Facebook-Based Survey
Group B No Survey Facebook-Based Survey Moodle-Based Survey
24
Experimental Design (Treatment 1)
Top 5 Most Popular LMS software ranking
25
Experimental Design (Treatment 2)
Active Users of SNS Sites
Data Collection
Data collected for the two samples in
two different empirical
conditions.
Direct collected method is mainly
applied, but under certain conditions
online survey is applied.
Survey distribution and collection was
managed by the researcher and the module lecturer.
Student informed that the survey is
for academic purposes only and it is optional and
confidential.
26
Group TreatmentJan-Feb 2014 Test Treatment
Mar-Apr 2014 Posttest
Moodle Group A Survey Group B Survey
Facebook Group B Survey Group A Survey
Time
Analysis Tools
IBM SPSS v. 20 and SmartPLS 3 is used for statistical analysis.
Analysis Based in PLS-SEM (Variance based Sequential Equation Modeling)
If the research objective is prediction and theory development, then the appropriate
method is PLS SEM.‑27
Instrument Validity
Content Validity
• Experts Panel of five experts were asked to review and comment about the 1st draft survey items. “Measure what is intend to measure”; is the point of view for the feedback from the panel.
Face Validity
• A group of 10 students from different universities in Malaysia surveyed to figure out the goodness of the questionnaire (by observation and discussion)
28
29
Pilot Study - Reliability Test (Cronbach's Coefficient )
IQSQ
SrQTP
SPSS
SUUS
LO
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0.74
4
0.72
4
0.74
6
0.72
5
0.70
1
0.75
9
0.72
4
0.83
8
0.73
6
0.80
7
0.74
5
0.75
9
0.74
7
0.72
6
0.74 0.
806
0.88
6
0.69
7
68 bachelor students of LUCT
University - Malaysia for feasibility
of FB based system
40 bachelor students of Palestine
University-Palestine for Moodle
based system
0.7
0.6
30
Findings
31
Data Screening
Registered Students Distributed
Survey Collected Cases Valid Cases
0
50
100
150
200
250
300
350
400
362
310
26585%
22986%
45
36
Moodle-based SystemRetained Waived
Registered Students Distributed
Survey Collected Cases Valid Cases
0
50
100
150
200
250
300
350
400
363
308
26285%
23389%
46
29
Facebook-based SystemRetained Waived
32
Demographic Analysis
Total Gender Age Race Nationality Academic Eng. Prof. Internet Prof.0
20
40
60
80
100
120
100 %231
61.3 (M)51.5 (22-25)
33.1 (Arab)
14.7 (China)
69.5 (MBA)
16.5 (Excellent)
41 (Excellent)
38.7 (F)
35.9 (26-30)
18 (Chinese)
10 (Iran)
19.6 (MA)
44.5 (V. Good)
37.5 (V. Good)
12.1 (31-40)
14.5 (African)
9.1 (Yemen)
10.9 (MSc)
35.5 (Good)
20.6 (Good)
0.4 (>40)
13.2 (Persian)
9.1 (Syria)
3.5 (Poor) 0.9 (Poor)
6.1 (Indian)
8.5 (Malaysia)
5.9 (Malay)
6.7 (Nigeria)
33
Carryover Effect Analysis
Group Phase System
The variance of the learner outcome is explained only by System factor (different systems have a significant affect on LO)
0.05
0.10
P-Va
lue
Source F Sig.System 53.283 .000Phase .096 .757Group 1.997 .158
Dependent Variable: Learner Outcome.
34
Assessing PLS-SEM ResultsValidity
Relations
Internal consistency reliability
Convergent validity
Outer model loadings and significance
Outer Loading
Composite Reliability
“AVE” numbers and Latent Variable Correlations
Variance Inflation Factor (VIF)
• Predictive power (R2) and Predictive relevance (Q2)
• ƒ² effect size • P-Values, T Statistics, and Path Coefficient
Average Variance Extracted (AVE)
Discriminant validity
Collinearity Assessment
Indicator reliability
Hair (2014)
35
Significance of Construct Model Relations
IQ -> IU
IQ -> US
SQ -> IU
SQ -> US
SrQ -> IU
SrQ -> US
SS -> IU
SS -> LO
SS -> US
LP -> IU
LP -> LO
LP -> US
TP -> IU
TP -> LO
TP -> US
US -> IU
US -> LO
IU -> LO
FacebookCoefficient T Value P Values
0.057 0.811 0.209
0.264 3.77 0***
0.156 2.039 0.021*
0.227 2.778 0.003*
0.093 1.28 0.1
-0.04 0.506 0.306
-0.039 0.655 0.256
0.088 1.73 0.042*
-0.012 0.201 0.42
0.298 4.759 0***
0.222 3.552 0***
0.075 1.029 0.152
-0.005 0.074 0.47
0.196 2.886 0.002**
0.355 4.628 0***
0.326 3.839 0***
0.321 5.006 0***
0.145 2.174 0.015*
MoodleCoefficient T Value P Values
0.036 0.599 0.275
0.213 2.987 0.001***
0.142 2.059 0.02*
0.397 5.062 0.000***
0.041 0.653 0.257
-0.047 0.638 0.262
-0.07 1.689 0.046*
0.062 1.365 0.086
-0.01 0.19 0.425
0.209 3.85 0.000***
0.183 3.17 0.001***
0.015 0.234 0.407
0.125 2.036 0.021*
0.207 3.125 0.001***
0.334 4.721 0.000***
0.465 6.993 0.000***
0.263 3.351 0.000***
0.254 3.182 0.001***
36
Hypothesis Discussion
37
Hypothesis Regarding Relations: Moodle-based System
Learner's Outcome 0.704
SS LO Yes* + 0.09 1.37 0.008 0.06TP LO Yes + 0.00 3.13 0.051 0.21LP LO Yes + 0.00 3.17 0.048 0.18US LO Yes + 0.00 3.35 0.068 0.26
IU LO Yes + 0.00 3.18 0.060 0.25
User Satisfactio
n0.673
IQ US Yes + 0.00 2.99 0.049 0.21SQ US Yes + 0.00 5.06 0.152 0.40SrQ US No Null 0.26 0.64 0.002 -0.05SS US No Null 0.43 0.19 0.000 -0.01TP US Yes + 0.00 4.72 0.122 0.33
LP US No Null 0.41 0.23 0.000 0.02
Intention to Use 0.736
IQ IU No Null 0.28 0.60 0.002 0.04SQ IU Yes + 0.02 2.06 0.021 0.14SrQ IU No Null 0.26 0.65 0.002 0.04SS IU No* - 0.05 1.69 0.011 -0.07TP IU Yes + 0.02 2.04 0.020 0.13LP IU Yes + 0.00 3.85 0.070 0.21
US IU Yes + 0.00 6.99 0.268 0.47
Dependent Variable
Predictive Power R2 Hypothesis Sign P Value T Statistics f2 Values Path
Coefficient
38
Hypothesis Regarding Relations: FB-based System
Learner's Outcome 0.657
SS LO Yes + 0.04 1.73 0.013 0.09TP LO Yes + 0.00 2.89 0.046 0.20LP LO Yes + 0.00 3.55 0.060 0.22US LO Yes + 0.00 5.01 0.127 0.32
IU LO Yes + 0.02 2.17 0.027 0.15
User Satisfactio
n0.575
IQ US Yes + 0.00 3.77 0.073 0.26SQ US Yes + 0.00 2.78 0.048 0.23SrQ US No Null 0.31 0.51 0.001 -0.04SS US No Null 0.42 0.20 0.000 -0.01TP US Yes + 0.00 4.63 0.109 0.36
LP US Yes** + 0.15 1.03 0.006 0.08
Intention to Use 0.575
IQ IU No Null 0.21 0.81 0.003 0.06SQ IU Yes + 0.02 2.04 0.022 0.16SrQ IU Yes* + 0.10 1.28 0.007 0.09SS IU No Null 0.26 0.66 0.002 -0.04TP IU No Null 0.47 0.07 0.000 -0.01LP IU Yes + 0.00 4.76 0.092 0.30
US IU Yes + 0.00 3.84 0.106 0.33
Dependent Variable
Predictive Power R2 Hypothesis Sign P Value T Statistics f2 Values Path
Coefficient
39
Re-specified Hybrid Model
Learning System
Outcome
Learner Outcome
System Sociability
Information Quality
Service Quality
Intention to (Use)
User Satisfaction
Teaching Presence
Learner Presence
System Quality
Solid, Approved Relation
Mixed, Semi-Approved Relation
Rejected Relation
Moodle-based
Facebook-based
M
F
F
M
F
40
Hypothesis Regarding Comparison
Intention to Use User Satisfaction Learner Outcome3.3
3.4
3.5
3.6
3.7
3.8
3.9
4
4.1
4.2
4.3
3.6288
3.8035
3.6559
4.0129
4.2575
4.0918
Mean Value
The two-sample t-test shows a significant difference between the mean values of the two data sets for the two
systems with P-value > 0.5
41
Contributions, and Recommendation
CONTRIBUTIONS OF THE STUDY
A model is showing all the constructs for assessing the success of the modern eLearning system and showing interrelationships among these constructs learner outcome in higher education.
An instrument tool (survey) that include all the dimensions within the model. The tool is tested for validity and reliability and can be used by other researchers.
The experiment design process for comparing two information system in a quantitative approach.
Findings and it analysis is important to decision makers in many practitioners.
42
Academic Contributions
A new Hybrid eLearning Success model.
A new survey construct with new definitions and measures.
The experiment design and process of implementation and data collection
A Quantitative comparative analysis between the traditional system and SNS system.
The use of Smart-PLS in acquiring the results.43
RECOMMENDATIONS FOR FUTURE RESEARCH
Further research is needed to assess the model in different conditions.
Replicating the same assessment in other higher education institutes is recommended.
Replicating the same assessment in other levels of study such as Bachelor degree is recommended
44
RECOMMENDATIONS FOR FUTURE RESEARCH
Further research is needed to investigate additional eLearning system success determinants.
Further research is needed to perform another experiment approaches for comparing different systems.
Information technology researchers are recommended to produce an integration approaches for the traditional systems such as Moodle software and the SNS sites such as Facebook.
45
46
Publications
Publications
47
TitleThe role of System Sociability Factor in Modeling Learning Management System Success in University Education.
Status Published; April 2015
Journal/ConferenceInternational Conference on e-Commerce, e-Administration, e-Society, e-Education, and e-Technology (e-CASE & e-Tech 2015)
TitleDEVELOPING A SUCCESS MODEL FOR CONTENT MANAGEMENT SYSTEM IN HIGHER EDUCATION: ANALYSIS FROM STUDENTS’ PERSPECTIVE.
Status In Process. Initial Submission; Expected June 2016
Journal/Conference The Journal of the Association for Information Systems (JAIS)
TitleINVESTIGATION OF A MODIFIED INFORMATION SYSTEM SUCCESS IN UNIVERSITY LEARNING SUCCESS – STUDENTS’ PERSPICTIVE.
Status In Process. Accept Manuscript; Expected Dec. 2015
Journal/Conference Journal of Technology; UTM
Publications
48
TitleFactors Influencing the Learning Management System ( LMS ) Success Among Undergraduate Students in Limkokwing University of Creative Technology , Malaysia.
Status Published; June 2015
Journal/Conference International Journal of Multicultural and Multireligious Understanding
TitleLearning Management System (LMS) Success: An investigation among the university students.
Status Published; Aug. 2015
Journal/ConferenceThe IEEE Conference on e-Learning, e-Management and e-Services (IC3e 2015)
TitleThe effects of school management support on the use of interactive whiteboard (IWB) in high school.
Status Published; 2015
Journal/Conference International Journal of Multicultural and Multireligious Understanding
50
End of Formal VIVA Presentation
51
The next section are not presentedIt is for supporting during discussion
Population and Sampling
52
53
Population
Count of
Moodle-Based System FB-Based System
AllPhase1/Group A
Phase2/Group B
AllPhase1/Group B
Phase2/Group A
Registered Students 362 174 188 363 192 171
54
SmartPLS Minimum Sample Size
Sample size should be at least 10 times the largest number of formative measures of a particular construct, or 10 times the largest number of structural paths points to a single latent construct.
In this study, the minimum sample size is 70.
Hair (2014)
55
Effective Sample Size
55
Cohen (1988)
Dattalo (2008)
G*Power screenshot of the applied setting
56
Actual Respondents
Count of
Moodle-Based System FB-Based System
AllPhase1/Group A
Phase2/Group B
AllPhase1/Group B
Phase2/Group A
Registered Students 362 174 188 363 192 171
Distributed Survey 310 152 158 308 160 148
Collected Cases 265 128 137 262 130 132
Non-Fitted Cases -5 -2
Uncompleted Cases -11 -9
Initial Cases for Analysis 249 251
Unengaged Screening -7 -10
Univariate Screening -10 -6
Multivariate Screening -3 -2
Cleaned Cases for Analysis 229 112 117 233 118 115
Assessing PLS-SEM Results
57
Indicator reliability(Outer Loading)
Outer loading scale used in reflective models to assure the proper loading of measures in its construct.
The acceptable level of outer loading is 0.708 and above.
Levels between 0.4 and 0.7 can be deleted if other indicators reliability enhanced
(Hulland 1999; Hair et al. 2014).
58
59
Indicator reliability (Outer Loading) - MoodleIQ IU LO LP SQ SS SrQ TP US
IQ1 0.783IQ2 0.848IQ3 0.839IQ4 0.839IQ5 0.792IU1 0.773IU2 0.857IU3 0.860IU4 0.815LO1 0.874LO2 0.893LO3 0.861LO4 0.810LO5 0.840LP1 0.844LP2 0.740LP4 0.755LP5 0.685SQ1 0.814SQ2 0.839SQ3 0.800SQ4 0.847SQ5 0.871SS1 0.769SS2 0.811SS3 0.821SS5 0.816SrQ1 0.845SrQ2 0.842SrQ3 0.755SrQ5 0.776TP1 0.826TP2 0.792TP3 0.804TP5 0.860US1 0.893US2 0.882US3 0.905US4 0.859
60
Indicator reliability (Outer Loading) - FacebookIQ IU LO LP SQ SS SrQ TP US
IQ1 0.730IQ2 0.811IQ3 0.812IQ4 0.808IQ5 0.761IU1 0.832IU2 0.876IU3 0.834LO1 0.818LO2 0.821LO3 0.784LO4 0.758LO5 0.821LP1 0.790LP2 0.861LP4 0.763LP5 0.817SQ1 0.754SQ2 0.779SQ3 0.747SQ4 0.815SQ5 0.851SS1 0.711SS2 0.839SS3 0.808SS5 0.774SrQ1 0.820SrQ2 0.785SrQ3 0.796SrQ5 0.768TP1 0.727TP2 0.772TP3 0.801TP4 0.815TP5 0.696US1 0.866US2 0.832US3 0.809US4 0.833
Internal Consistency(Composite Reliability)
To show the consistency of items of the same construct.
Composite reliability should be 0.7 or higher. If it is an exploratory research, 0.6 or higher is
acceptable
(Bagozzi & Yi 1988; Hair et al. 2014)
62
Composite Reliability
IQSQ
SrQTP
LPSS
SUUS
LO
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.88
9
0.89
2
0.87
1
0.87
4
0.88
3
0.86
4
0.88
4
0.90
2
0.89
9
0.91
2
0.92
0.88
0.89
2
0.84
3
0.88
0.89
6
0.93
5
0.93
2
0.7
0.6
Convergent Validity(AVE value)
Average Variance Extracted (AVE) scale used to show that measure inside individual construct is related.
The acceptable level of AVE value is 0.5 and above
(Bagozzi & Yi 1988; Hair et al. 2014).
63
64
Convergent Validity: (AVE) Average Variance Extracted
IQSQ
SrQTP
LPSS
SUUS
LO
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
FBMoodle
0.61
6
0.62
4
0.62
8
0.58
3 0.65
4
0.61
6
0.71
8
0.69
8
0.64
2
0.67
4
0.69
6
0.64
9
0.67
3
0.57
5 0.64
7
0.68
4 0.78
3
0.73
3
0.5
AVE
Discriminant Validity (Fornell-Larcker Criterion Analysis)
“Correlation matrix of AVE values” scale used to show that measures outside a specific construct is not related to it.
Rule of thumb is that the square root of AVE value of a specific construct must be greater than all the other values in the same column or row of the correlation matrix
(Hair et al. 2014; Fornell & Larcker 1981).
65
66
Discriminant Validity (Fornell-Larcker Criterion Analysis)
IQ IU LO LP SQ SS SrQ TP US
IQ 0.821IU 0.679 0.827LO 0.633 0.769 0.856LP 0.664 0.672 0.683 0.758SQ 0.724 0.75 0.689 0.654 0.834SS 0.537 0.444 0.511 0.587 0.498 0.805
SrQ 0.726 0.662 0.618 0.635 0.752 0.532 0.805TP 0.678 0.73 0.739 0.656 0.738 0.522 0.72 0.821US 0.696 0.812 0.76 0.599 0.766 0.459 0.65 0.741 0.885
IQ IU LO LP SQ SS SrQ TP US
IQ 0.785IU 0.587 0.848LO 0.601 0.656 0.801LP 0.603 0.64 0.675 0.809SQ 0.652 0.628 0.635 0.595 0.79SS 0.508 0.429 0.528 0.607 0.428 0.785
SrQ 0.67 0.583 0.613 0.61 0.675 0.57 0.792TP 0.61 0.599 0.693 0.642 0.695 0.535 0.713 0.763US 0.641 0.668 0.715 0.565 0.658 0.432 0.582 0.686 0.835
Forn
ell-L
arck
er
Crite
rion
Anal
ysis
The
Diag
onal
Val
ue m
ust b
e gr
eate
r tha
n al
l th
e ot
her v
alue
s in
the
sam
e co
lum
n or
raw
.
Multicollinearity Analysis: (VIF) Variance Inflation Factor
Each set of exogenous latent variables in the inner model is checked for potential collinearity problem to see if any variables should be eliminated, merged into one, or simply have a higher-order latent variable developed.
The acceptable level of VIF value is above 0.20 and less than 5.00.
(Wong 2013)
67
68
Multicollinearity Analysis: (VIF) Variance Inflation Factor
5.0
0.2
VIF IU LO US
IQ 2.939 2.801
IU 3.647
LO
LP 2.372 2.365 2.372
SQ 3.644 3.164
SS 1.675 1.622 1.675
SrQ 3.052 3.045
TP 3.141 2.826 2.8
US 3.057 3.427
IU LO US
IQ 2.423 2.258
IU 2.23
LO
LP 2.282 2.404 2.269
SQ 2.621 2.5
SS 1.813 1.682 1.813
SrQ 2.784 2.78
TP 3 2.422 2.704
US 2.355 2.352
69
Predictive Power (R2) & Predictive Relevance (Q2)
0.2
0.5
R2W
eak
Mod
erat
eSt
rong
0.75
0.02
0.15
Q2
Smal
lM
ediu
mLa
rge
0.35R Square Q Square
Intention to Use (IU) 0.736 0.495
Learner Outcome (LO) 0.704 0.511
User Satisfaction (US) 0.673 0.516R Square Q Square
Intention to Use (IU) 0.575 0.399
Learner Outcome (LO) 0.657 0.417
User Satisfaction (US) 0.575 0.393
R2 is the “percent of variance explained” by the model in the dependent variables.
Q2 statistic measures the predictive relevance of the model by reproducing the
observed values by the model itself.
70
The Effect Size – ƒ2
The ƒ² effect size measures the change in the R² value when a specified exogenous construct is omitted from the model.
0.02
0.15
ƒ2Sm
all
Med
ium
Larg
e
0.35
IU LO US
IQ 0.002 0.049
IU 0.06
LO
LP 0.07 0.048 0
SQ 0.021 0.152
SS 0.011 0.008 0
SrQ 0.002 0.002
TP 0.019 0.051 0.122
US 0.268 0.0680.00
Extr
aSm
all
IU LO US
IQ 0.003 0.073
IU 0.027
LO
LP 0.092 0.06 0.006
SQ 0.022 0.048
SS 0.002 0.013 0
SrQ 0.007 0.001
TP 0 0.046 0.109
US 0.106 0.127
Significance of Construct Model Relations
Three values are used for the assessment that are significant level or probability estimate value (P value), the significance of path coefficient (T-statistics), and path coefficient.
As Hair (2014), the rule of thumbs for assessing the values is:
P-value could be on three levels 1%, 5% or 10%, but the popular level in psychological studies is 5% or (0.05).
With 5% significance level, T statistic > 1.96 is significant with a two-tailed test and T Statistics >.98 is significant for a one-tailed test.
71
72
Significance of Construct Model Relations
IQ -> IU
IQ -> US
SQ -> IU
SQ -> US
SrQ -> IU
SrQ -> US
SS -> IU
SS -> LO
SS -> US
LP -> IU
LP -> LO
LP -> US
TP -> IU
TP -> LO
TP -> US
US -> IU
US -> LO
IU -> LO
FacebookCoefficient T Value P Values
0.057 0.811 0.209
0.264 3.77 0***
0.156 2.039 0.021*
0.227 2.778 0.003*
0.093 1.28 0.1
-0.04 0.506 0.306
-0.039 0.655 0.256
0.088 1.73 0.042*
-0.012 0.201 0.42
0.298 4.759 0***
0.222 3.552 0***
0.075 1.029 0.152
-0.005 0.074 0.47
0.196 2.886 0.002**
0.355 4.628 0***
0.326 3.839 0***
0.321 5.006 0***
0.145 2.174 0.015*
MoodleCoefficient T Value P Values
0.036 0.599 0.275
0.213 2.987 0.001***
0.142 2.059 0.02*
0.397 5.062 0.000***
0.041 0.653 0.257
-0.047 0.638 0.262
-0.07 1.689 0.046*
0.062 1.365 0.086
-0.01 0.19 0.425
0.209 3.85 0.000***
0.183 3.17 0.001***
0.015 0.234 0.407
0.125 2.036 0.021*
0.207 3.125 0.001***
0.334 4.721 0.000***
0.465 6.993 0.000***
0.263 3.351 0.000***
0.254 3.182 0.001***
Survey
73
74
Questionnaire P1-2
75
Questionnaire P3-4
76
Research Variables and Literature Reference ModelsReferences IQ SQ SrQ SS TP LP IU US LO
IS success model by Delone and McLean (2003) x x x x x x
ELearning system model by Freeze et al. (2010) x x x x x
ELearning system model by Wang et al. (2007) x x x x x x
Hexagonal eLearning assessment model by Ozkan et al. (2009) x x x x x
Revised community of inquiry model by Shea & Bidjerano (2010) x x x
eLearning acceptance framework by Selim (2007) x x x
Hierarchical model for eLearning CSF.. by Bhuasiri et al. (2012) x x x x x
Model of online community attributes&benefit by Kim, Park and Jin (2008) x x
Sociability and Usability Framework by Lambropoulos (2005) x x
Online Community framework by de Souza & Preece (2004) x x
(Garrison et al. 2010) x x x
(Arbaugh 2008) x x x x
(Daspit & D’Souza 2012) x x x
(Lambert & Fisher 2013) x x x
(Lee-post 2009) x x x x x x
(Keramati et al. 2011) x x x
(Gao et al. 2010) x x x
(Lin et al. 2007) x x x x
(Phang et al. 2009) x x
(Paechter et al. 2010) x x x
Constructs’ Definitions
77
Information Quality (IQ)
This study defines information quality as the level of goodness of the information produced by the system and assessed by using different measures differ from system to another based on its nature and functions.
Those measures could include up to date, relevance, accuracy and much more.
78
System Quality (SQ)
This study defined system quality as the level of goodness of the information system features and tools excluding the output and assessed by using different measures differ from system to another based on its nature and functions.
Those measures could include flexibility, response time, system reliability and much more.
79
Service Quality (SrQ)
This study defines service quality as the level of goodness of the personnel support offered by the administrative affairs to the system users.
Those measures could include communication quality, technical competence, and empathy of the personnel staff and others.
80
System Sociability (SS)
This study defined system sociability as the system level of readiness and practice for the interaction services and activities including technology, policies and practice.
Those measures could include system interactive, members’ interaction, policies support, and others.
81
Teaching Presence (TP)
This study defines teaching presence as the level of instructor involvement and participation into the system
including content feeding quality and interaction with the system and members in a synchronous and asynchronous manner.
82
Learner Presence (LP)
This study defines learner presence as the level of learner readiness and participation in the system
including learner confident interaction, confident participation, ability to form an impression and others.
83
Intention to Use (IU)
This study defines intention to use as the level of willingness to use the information system.
Those measures could include frequency of use, dependency, reusability and others.
84
User Satisfaction (US)
This study defines user satisfaction as the level of goodwill achieved after experiencing the system.
Those measures could include system usefulness, adequacy, effectiveness and others.
85
Learner Outcome (LO)
This study defines learner outcome as the level of studying outcome achieved by using the information system.
Those measures could include productivity, performance, better thinking and others.
86
Community of Inquiry Framework
87
Brief of COI
The community of inquiry framework is an instructional design model for eLearning developed by Randy Garrison, Terry Anderson et al. (2000).
This framework is for educational context as it proposes a framework for the use of computer-mediated communication to support the education process.
The framework has three essential elements cognitive presence, social presence, and teaching presence
88
The Community of Inquiry (CoI) bundle is the result of a project ran from 1997 to 2001.
Definitions of COI
Social presence refers to the ability of learners to project their personality into the community of inquiry, means learners can introduce themselves as a real people within the online communication or interaction.
Teaching presence construct outlines task sets such as organization, design, discourse facilitation, and direct instruction and articulates the specific behaviours likely to result in a productive community of inquiry.
Cognitive presence refers to the extent to which the participants in any particular configuration of a community of inquiry are able to construct meaning through sustained communication.
89
Preece’s Sociability and Usability Framework
90
Brief of Preece’s S&U
Preece (2000, 2001) proposed system usability and system sociability as determinants of the online community success. Goals, purposes and functions of the community affect the needs of online communities.
This framework used in many studies where the finding mostly shows a relationship between sociability and social benefits and usability and functional benefit.
91
Definitions of Preece’s S&U
Sociability dimension involves the measures related to the purpose, people, and policies. Purpose factor refers to the interaction and involvement levels of community participants.
Usability dimension covers the measures related to dialog and social interaction support, information design, navigation, and access.
Success definition of the online community differs based on the perspective of whom.
92
Delone and Mclean Information System Success Model
93
Brief of D&M IS Model
D&M IS success model is a result of attempts to provide an integrated scene of IS success that enables comparisons between different studies. It propose a broad and acceptable meaningful of the information system success.
The founders of this famous theory are William H. Delone and Ephraim R. McLean in 1992. Later on, the same authors revised the original theory and proposed an updated model after ten years in response to comments announced by other different researchers.
The updated model proposed six different dimensions of the IS success and provided an identifying, describing, and explaining the relationships among the six dimensions of success.
94
Definitions of D&M IS Model
System quality construct comprises the desirable characteristics of the system itself and includes related measures of the IS itself.
Information quality construct comprises the desirable characteristics of the information system output.
Service quality construct characterizes the quality of the support offered to system users by the IS department and IT support workforce.
The (intention to) use construct characterizes the user utilization level of the desired information system.
User satisfaction construct comprises the user’s level of satisfaction when using the desired information system.
Net benefits construct comprises the extent to which desired information system are contributing to the success of the desired users.95
96