for review only - prince of songkla universityrdo.psu.ac.th/sjstweb/ar-press/60-sep/18.pdf ·...

For Review Only

Elective Course Recommendation Model for Higher

Education Program

Journal: Songklanakarin Journal of Science and Technology

Manuscript ID SJST-2016-0262.R1

Manuscript Type: Original Article

Date Submitted by the Author: 15-Mar-2017

Complete List of Authors: Praserttitipong, Dussadee; Chiang Mai University, Computer Science Srisujjalertwaja, Wijak; Chiang Mai University, Computer Science

Keyword: Collaborative Filtering Technique, Courses Recommendation System, Educational Data Mining, Knowledge Discovery

For Proof Read only

Songklanakarin Journal of Science and Technology SJST-2016-0262.R1 Srisujjalertwaja

For Review Only

Original Article

Elective Course Recommendation Model for Higher Education Program

Dussadee Praserttitipong and Wijak Srisujjalertwaja*

Department of Computer Science, Faculty of Science, Chiang Mai University,

Chiang Mai, THAILAND

Email address: [email protected], [email protected]*

Abstract

Educational data mining (EDM) is an application that applies data mining technique to

resolve educational managing problems. This paper proposes a general process of EDM for elective

courses recommendation based on student grades. Furthermore, the proper and inadequate reasons

for applying the conventional measurements to the course recommendation domain are studied and

fulfilled with the new proposed quality measurements. Several techniques were studied to establish

the model for estimating the grades of the elective courses. The experiments suggested that SVD-

based technique based on course information depicted the best results. The overall results indicate

that the proposed model is able to provide the personalized recommendation for individual students

based on the students’ abilities.

Keywords: Collaborative Filtering Technique, Educational Data Mining, Courses Recommendation

System, Knowledge Discovery.

Page 4 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

1. Introduction

The general objectives of EDM can be either improving the learning process and guiding

students’ learning or achieving a deeper understanding about educational situation (Romeron and

Ventura, 1992). One of the major problems is how to benefit the collected data for facilitating the

useful information to advise the students for planning their own enrollment courses which suitable

for their own potentials (Ray and Sharma, 2011).

Recommender system aims to provide its users with the relevance information. For modeling a

recommender system, the user personalized information is evaluated and the model for estimating

the rating scores for the items, which have not seen by its users, is developed. A successful type of

recommender system is based on collaborative filtering (CF) technique. The personalized guidance

is evaluated from the opinions of other users collected in the historical database. The similarities

between the elements are evaluated, in order to find the most suitable collaborative elements.

In the literatures, the best performance group of algorithm for CF technique are algorithms

based on the Singular Value Decomposition (SVD) technique (Cacheda, 2011; Kautkar, 2014).

Thus, SVD based algorithm was studied, its parameters were assigned their description for denoting

the terms with in the elective courses recommendation context. The general process of EDM for

elective courses recommendation based on student grades were proposed. The conventional

measurements for recommendation results quality were studied. Their inadequacies for using in

context of the elective courses recommendation were explained, besides, the new proposed quality

measurements were presented.

The remaining of this paper is organized as follows. Section 2 explains about related works

and Section 3 presents the proposed ideas. Section 4 exhibits the experimental results along with

discussions. Section 5 concludes the paper with some final thoughts.

Page 5 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

2. Related Works

Lee and Cho (2011) proposed an intelligent course recommendation system based on content

based CF technique. The system evaluated the relationship between each courses via the field of

knowledge their relevant. The recommendation process started from assessing a student’s weak

subjects and analyzing individual abilities to be able to advise individual students which fields of

study would be suitable and which courses they should take. This approach was suitable for

implementing in a small domain, where all fields of study were well specified.

Ray and Sharma (2011) presented a CF based approach for recommending elective courses.

The main idea was to provide students with the accuracy estimation of their grades. This

information about students’ performance was helpful when they decided to select elective courses.

The estimation grades of elective courses were provided to users based on the nearest neighbors of

both user-based and item-based CFs. Even though, CF recommender system based on nearest

neighbors techniques were simple, the accuracy of their results could be improved with other higher

performance CF techniques.

3. Proposed Model

This research aims to introduce the general process of EDM for elective courses

recommendation based on students’ grades. Furthermore, the conventional measurements for

recommendation results quality are studied. The usability and inadequacy of these measurements for

applying with the course recommendation domain are described. Besides, the inadequacies are

fulfilled with the new proposed quality measurements for course recommender system.

As depicted in Figure 1, the process of data mining typically consists of three main steps as:

data preparation, data modeling, and results evaluation. Data preprocessing is the process of

converting raw data collected from education systems into useful information. Subsequently, data

Page 6 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

modeling, the typical data mining techniques, such as classification, clustering, and association

rules, are applied to that information in order to find the model which represents an educational

situation issue. The interpretation is also taken into account to explain that meaning of that derived

information. Finally, the results acquired from data modeling process are evaluated along with the

results evaluation process.

3.1 Data Preparation

The input of this process is the historical data collected in the enrollment database. The

attributes of each record composes with students’ grades, students’ id, course id, course type and the

grades which they have been scored. The details of this process are as following.

1) Data extraction: the historical data collected in the enrollment database is query based on

the characteristics of the students. The studied records, which are assigned with grades which

contain grade point values (such as A, B+, B, or F), are retrieved. The values collected in

GRADEVALUE are assigned with grade point values which are numerical data.

2) Data cleansing: the data need to be cleansed. Because some students enroll in the same

courses in different semesters, these records must be grouped into one record. The grade point

values of re-enrollment courses are recalculated as the average values.

3) Data reconstruction: the values in STUDENTID and COURSEID are reconstructed to be

sequential number ordering from 1 to m and 1 to n, respectively. The value of m is the number of

students and n is the number of courses collected from historical database.

• In case of STUDENTID, the values in this field are selected with DISTINCT option

via SQL. Thus, the duplication values are removed. Then, STUDENTID is changed

to be STUDENTNO, where the values are reassigned with the sequential number.

Page 7 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

• In case of COURSEID, the value in COURSETYPE can be either ‘Compulsory

Course’ or ‘Elective Course’. Next, the data in this field is selected with DISTINCT

option and sorted ascending via SQL. The types of courses are specified in

COURSETYPE.

4) Data transformation: the historical studied records are reconstructed to the matrix

(hereafter referred to as matrix r, which is an � × � matrix). The index of each element in a matrix r

in term of (row, column) is corresponding to STUDENTNO and COURSENO, respectively.

3.2 Data Modeling

Several models can be applied for evaluating an estimated grade of a student s after attended

in courses c. The baseline estimation for an estimated grade of student s after attended in courses c

based on SVD-based technique, hereafter denoted as �̂�,�, is given by �̂�,� = �̅ + � + �� +∑ (��,� × ��,�)�� (1)

where �̅ is an average grade of all elements that have been assigned values in matrix r. �̅ is implemented in both �̅� (an average grade of a student s) and �̅�(an average grade of a course c). �, ��, ��,�, and ��,� denote model parameters, which are collected in � × 1, � × 1, m× �, and � × � matrices; respectively. m, n and k are number of students, number of courses and number of factors

for factoring matrix r. The optimal value of k for CF is 400 (Praserttitipong and Sophatsathit, 2012).

� and �� represent the observed deviations of student s from the average and the bias of observed

deviations of course c from the average, respectively. ��,� and ��,� are the extent of interest of student s on factor k-th and the extent to which the course possesses over factor k-th.

Page 8 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

The values of these parameters are initiated with the small randomized numbers. Then, these

values are recomputed via an iteration learning process with an objective for performing the

minimization of regularized square error between the students’ grades of the training elements and

the predicted values acquired from the baseline estimation equation. This iteration process performs

by utilizing a stochastic gradient descent optimization. The algorithm for SVD-based model learning

process described as following (Paterek, 2007).

o Assign the values to matrices b, d, p, and q with the small randomized numbers

o DO

1) Compute the value of the estimated grades of all studied records that have be

collected in the historical database by applying an Equation (1) as

�̂�,� = �̅ + � + �� +∑ (��,� × ��,�)��

2) Calculate the error of the estimated grade as ��,� = ��,� − �̂�,� 3) Compute the model parameters as

• � = � + �(��,� − � ∙ �) • �� = �� + �(��,� − � ∙ ��) • ��,� = ��,� + �(��,� ∙ ��,� − � ∙ ��,�) • ��,� = ��,� + �(��,� ∙ ��,� − � ∙ ��,�)

4) Calculate the value of a mean absolute error

o LOOP UNTIL meanabsoluteerror ≤ 0.5

The constants � and � are the stochastic gradient descent method constants. The values of � and � are set to 0.005 and 0.02, respectively (Paterek, 2007). This learning process repeats for

Page 9 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

setting the model parameters, which are collected in matrices b, d, p, and q, until the terminal

conditions are reached. The appropriate variable for establishing as the terminal condition is a mean

absolute error (MAE). The process is completed when the MAE is less than or equal to 0.50

(Praserttitipong and Sophatsathit, 2014), in which the process will not lead to overfitting.

The estimated grades evaluated based on classification concept to make their more

informative. The student grades are classified into 3 classes as Good recommended class (the

estimated grade is greater or equal to 3.0), Fair recommended class (the estimated grade is greater

or equal to 2.0), and Bad recommended class (the estimated grade is lower than 2.0).

3.3 Results Evaluation

The conventional measurements for recommendation results quality are studied. The usability

and inadequacy of these measurements for applying with the course recommendation domain are

described.

3.3.1 Estimation Accuracy Evaluation

The quality measurement for estimation results evaluated by the MAE that measures the

difference between the estimation results and the actual grade, which is defined as follows

./0 =∑ ∑ 123,452̂3,4167489:7389

;7×<7 (2)

where �be an � × � matrix collecting the historical grades of students. The variables ��,�and �̂�,�be an actual grade and an estimated grade of student s according to studying course c. Besides,

�= be a number of students that have been graded according to a course c and �= be a number of

courses that have been studied by a user c. Generally, lower MAE value reflects higher accuracy of

grades estimation.

Page 10 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

3.3.2 Conventional Classification Evaluation

The quality measurements for classification results assessed by means of basic information

retrieval metrics, i.e., accuracy, precision, recall, and F1-measure. These metrics are calculated from

the number of items that are either relevant or irrelevant and either contained in the recommendation

set of a user or not. These numbers clearly arranged in a contingency table that is called the

confusion matrix. The general form of confusion matrix for the course recommendation system

presented as Table 1.

The classification results categorized into 2 groups: correctly classified results and incorrectly

classified results. The results in group of correct classification results are named as TB, TF, and TG,

which are defined as the number of correctly classified results labeled as class Bad, class Fair and

class Good, respectively. The others are defined for the results in group of incorrect classification

classes. The evaluation measurements for course classification results described as following

(Cacheda et al., 2011).

1) Accuracy measures the correctness of the classification technique. The overall accuracy

measured as

/>>?�@>A = BCDBEDBFBGHIJIKL<M;NH2JOIH�IPIH;� (3)

2) Precision measures the exactness of the classification technique. The precision for

classification results is defined as

Q��>RSRT� = BGH<M;NH2JO�J22H�ILU�LK��POPHVPIH;�P<IGP��LK��BGH<M;NH2JOPIH;��LK��POPHVP<IGP��LK��

(4)

The low precision score means there is a small number of correct classified items compared with

other incorrect items classified in that class. The perfect precision score must be 1.0.

Page 11 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

3) Recall measures the completeness of the classification technique. A recall for classification

results is defined as

W�>@XX = YZ[\]^_[`abca``[cdefceghhibi[jid[^hi\dZihceghhYZ[\]^_[`abgcd]ge[e[^[\dhceghhibicgdia\`[h]edh (5)

The low recall score points out there is a small number of correct classified items compared with the

total number of items that actually belong to that class. The perfect recall score are also 1.0.

4) F1-measure is the combination of precision and recall. The quality in term of both precision

and recall are combined into a single score, which is calculated as the standard harmonic mean of

precision and recall. F1-measure for classification results is defined as

k� = l9

mno43pq6D 9no4rss

= l×t2H��PJ<×2H�KLLt2H�P�PJ<D2H�KLL (6)

However, some interested points in courses recommendation domain are not be identified by

the values of precision and recall. The inadequacies of precision values are depicted in Figure 2,

besides; the inadequacies of recall values are shown in Figure 3.

• The low precision value for class Bad indicates that there are a small number of correct

classified items in class Bad compared with other incorrect items classified in class Bad,

as shown in Figure 2(a). Because of this error, the students are informed with the

incomplete set of elective courses. This leads to a lost opportunity of enrollment with

some elective courses that probably make that students increase their accumulated grade

point average (GPA). This means there are some extra elective courses are classified in

class Bad, even though, they are fair or good courses. This situation is named as lost

opportunity recommendation case.

• The low precision value for class Good means there are some extra courses are classified

in class Good, even though, their actual classes are class Fair or class Bad, as illustrated

Page 12 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

in Figure 2(b). These classification results are serious cases. Because the students are

recommended with the wrong elective courses, these can cause many other crucial

problems, such as the problem of the students’ retirement causing from their GPA. This

situation is named as critical wrong recommendation case.

• The low precision value for class Fair means there are some extra courses are classified in

class Fair, as shown in Figure 2(c). This can be either a lost opportunity

recommendation case or critical wrong recommendation case.

• The low score is achieved from the recall measurement for class Bad implies that there

are many critical wrong recommendation results appear, as illustrated in Figure 3(a).

• The low recall score for class Good classification indicates that there are a small number

of correct classified items in class Good, as illustrated in Figure 3(b). This implies that

there are many lost opportunity recommendation situations exit.

• The low recall value for class Fair means there are some extra courses are classified in

class Fair, as shown in Figure 3(c). This can be either a lost opportunity

recommendation case or critical wrong recommendation case.

This can be seen that the values of both precision and recall values of different classes can infer

different meaning in course recommender system domain. For example, the values of both FBG and

FBF indicate there are some bad elective courses are classified in class Good and class Fair,

respectively. However, the most seriously situation in this case caused from the misclassified bad

courses into good courses. Thus, the significantly of incorrectly classified results must be defined in

different weights.

It can be concluded that the new measures for the course recommendation system domain are

called for. The confusion matrix for the course recommendation system is proposed as Table 2. The

Page 13 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

group of incorrectly classified results are designated into two subcategories; i.e., Wrong and LostOp,

in order to make the more distinguish for the degree of error values.

1) Lost metric: it uses for evaluating the degree of a lost opportunity recommendation

problem. The elective courses, which are classified in these classes, normally not be recommended

for the students. Thus, the students are informed with the incomplete set of elective courses. This

leads to a lost opportunity of enrollment with some elective courses that probably make that students

increment their accumulated GPA.

The LostOp variable is defined as the number of incorrectly classified results in case of the

misclassified items which classified into the class that lower than the actual abilities of the students.

The numerical values at the end of each category’s name; i.e., 1, 2, and 3, indicate the significance

of opportunity that the students lose. The variables LostOp3, LostOp2, and LostOp1 are sorted

according to the dramatic lost opportunity scores. The measurement for estimating the degree of a

lost opportunity recommendation problem is given as

uTSv = (w9×xJ�Iyt9)D(wz×xJ�Iytz)D(w{×xJ�Iyt{)BGHIJIKL<M;NH2JOIH�IPIH;� (7)

where the variables |�, |l, and |}, are the weighted of their impacts. The variables |�, |l, and |}

are assigned as 1, 2, and 3, respectively. The higher score of a lost opportunity (Lost) indicates the

higher numbers of students are informed with the incomplete set of elective courses.

2) Critical metric: it uses for evaluating the degree of a critical wrong recommendation

problem. These classification results are serious cases. Because the students are recommended with

the wrong elective courses, these may lead to many other crucial problems, such as the problem of

the students retirement causing from their GPA.

The Wrong variable is defined as the number of incorrectly classified results in case of the

misclassified items which classified into the class that higher than the actual abilities of the students.

Page 14 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

The degree of highest critical situation is denoted as variable Wrong3; beside, the less crucial critical

situation is denoted as Wrong1. The measurement for estimating the degree of a critical wrong

recommendation problem is given as

~�RvR>@X = (�9×�2J<�9)D(�z×�2J<�z)D(�{×�2J<�{)BGHIJIKL<M;NH2JOIH�IPIH;� (8)

where the variables ��, �l, variables �} are the weight of their impacts. The variables ��, �l,

variables �} are assigned as 2, 3, and 4, respectively. The values of ��, �l, and �} are more than

the values specified for |�, |l, and |}. The higher score of a critical wrong recommendation

(Critical) indicates a higher number of students are recommended with the wrong elective courses

which are not suitable with their abilities. These can lead to many other crucial problems.

3) Balanced accuracy metric: it uses for evaluating the overall accuracy of courses

recommender system based on the standard harmonic mean between an accuracy and an

ErrorClassified, where ErrorClassified is defined as the balance of the Critical and Lost. Because

the values of an accuracy and an ErrorClassified are not exactly equal to Accuracy=1-

ErrorClassified, these values need to be normalized. The ErrorClassified value evaluated based on

the concept of the standard harmonic mean (Cacheda et al., 2011) as

0��T�~X@SSR�R�� = K�H2K�H(w9,wz,w{)DK�H2K�H(�9,�z,�{)r�onr�o(�9,�z,�{)

�q3� Dr�onr�o(�9,�z,�{)�np�p4rs

= lD}z

�q3�D {�np�p4rs

= �×(xJ�I×�2PIP�KL)(l×�2PIP�KL)D(}×xJ�I) (9)

Thus, the balanced accuracy based on the concept of the standard harmonic mean (Cacheda et

al., 2011) is proposed as

�@X@�>��/>>?�@>A = l9

r44�nr4�D 9(9��rsr64o��nnqn)

= l×��M2K�U×(�5CKLK<�HV�22J2)��M2K�UD(�5CKLK<�HV�22J2) (10)

Page 15 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

The higher value returned from �@X@�>��/>>?�@>A measurement indicates that higher accuracy

achieved with a tolerable error classification in elective courses recommendation results.

4. Experimental Results and Evaluation

The data in this experiment were collected from enrollment records of 438 students who

studied in Department of Computer Science, Faculty of Science, Chiang Mai University during

academic year 2006-2010. The grades which they have been scored in 74 courses collected in a

historical enrollment database, which were 21 compulsory courses and 53 elective courses. This

database collected 11,158 studied records. The studied records which have been enrolled in different

academic years were treated to be equal priority.

These studied records were divided into the training sets and the testing sets via split testing

technique. Since the studied records represented data of two different categories (i.e., compulsory

courses and elective courses), the stratified sampling was performed to select the elements for the

training sets and the testing sets. The records in the testing set were randomly sampled from the

studied records that the students have enrolled in the elective courses; besides, the rest were used as

training set. Thus, the records in the training set were the historical enrollment information of both

compulsory courses and elective courses. The testing elements were randomly selected and others

were training set. According to the small amount of experimental data, the size of each testing set

was approximately 10% of studied records a collected in the historical database for reporting the

highest powerful of each techniques. Five training and testing sets based on split testing technique

were carried out along with the several testing approaches.

4.1 Testing Approaches

The grade estimation of a students received from enrollment in an elective course c (�̂�,�) has

Page 16 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

been evaluated according to several approaches. There were 12 experiments were conducted. The

details of the testing approaches were as following.

1) An average grade of each student: the grade estimation evaluated from an average grade

earned by students, which could be modeled as

�̂�,� = �� (11)

2) An average grade of each course: the grade estimation evaluated from an average grade

given in course c, which could be specified as

�̂�,� = �� (12)

3) Combination of (1) & (2) course: the grade estimation calculated from an average grade

earned by students acquired from an Equation (11) and an average grade given in course c

acquired from an Equation (12) , which could be evaluated as

�̂�,� = @��@��(��, ��) (13)

4) User-based CF: This approach applied a user-based CF with Pearson correlation technique

(Cacheda et al., 2011). The detail of this technique could be described as

o First, the similarities between a current student s and other students were calculated. A

similarity value between student s and student t based on Pearson correlation technique

could be evaluated as

SR��,I = ∑ �23,452̅3�(2�,452̅�)6489�∑ �23,452̅3�z6489 �∑ (2�,452̅�)z6489

(14)

o Then, a subset of k-students who were the most similar to students are selected. In this

experiment, k was set to 7. The grade estimation based on user-based CF could be

calculated as

Page 17 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

�̂�,� = �̅� + ∑ (2�,452̅�)��89� (15)

5) Item-based CF: This approach applied an item-based CF with Pearson-correlation technique

(Cacheda et al., 2011). The detail of this technique could be described as

o First, the similarities between a course c and other course s were calculated. A similar

between a course c and a course d based on Pearson correlation technique evaluated as

SR��,V = ∑ �23,452̅4�(23,�52̅�):389�∑ �23,452̅��z:389 �∑ (23,�52̅�)z:389

(16)

o Then, a subset of k-courses which were the most similar to a course c are selected. In this

experiment, k was set to 7. The grade estimation based on item-based CF could be

evaluated as

�̂�,� = ∑ 23,��89� (17)

6) Combination of (4) & (5): the grade estimation calculated from the grade estimation based

on user-based CF technique acquired from an Equation (15) and the grade estimation based

on item-based CF technique acquired from an Equation (17) , which could be evaluated as

�̂�,� = @��@��(�̂�,��T�@�0�?@vRT�15, �̂�,��T�@�0�?@vRT�17) (18)

7) Weighted user-based CF:

This approach also applied a user-based CF with Pearson correlation technique as an

Equation (14). The weights of the similarity values were taken into account. The grade

estimation could be calculated as

�̂�,� = �̅� + ∑ �P;3,�×(2�,452̅�)��89∑ 1�P;3,�1��89

(19)

8) Weighted item-based CF: This approach also applied an item-based CF with Pearson

correlation technique as an Equation (16). The grade estimation could be evaluated as

Page 18 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

�̂�,� = ∑ �P;4,�×23,��89∑ 1�P;4,�1��89

(20)

9) Combination of (7) & (8): the grade estimation could be calculated as an average of the

results from an Equation (19) and an Equation (20).

�̂�,� = @��@��̂�,��T�@�0�?@vRT�18, �̂�,��T�@�0�?@vRT�20� (21)

10) SVD based on student info: the grade estimation calculated from SVD technique based on an

average grade earned by student s, which could be modeled as

�̂�,� = �̅� + � + �� +∑ (��,� × ��,�)�� (22)

11) SVD based on course info: the grade estimation calculated from SVD technique based on an

average grade given in course c, which could be modeled as

�̂�,� = �̅� + � + �� +∑ (��,� × ��,�)�� (23)

12) Combination of (10) & (11): the grade estimation calculated from SVD technique based on

an average grade earned by student s acquired from an Equation (22) and an average grade

given in course c acquired from an Equation (23), which could be evaluated as

�̂�,� = @��@��(�̂�,��T�@�0�?@vRT�22, �̂�,��T�@�0�?@vRT�23) (24)

4.2 Experimental Results and Discussion

4.2.1 Estimation Accuracy

The estimation accuracy evaluated by comparing the numerical estimated grades against the

actual grades via MAE as described in an Equation (1). Figure 4 shows that the estimation accuracy

results achieved from several CF techniques.

As shown in Figure 4, the MAE values returned from the SVD technique based on the

combination of SVD based on student and course information was 0.5543, which was the lowest

MAE value. This inferred that the estimation accuracy of the SVD technique based on student and

Page 19 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

course information was highest.

4.2.2 Classification Evaluation

The estimated grades derived from CF techniques were further assessed based on

classification concept to make their more informative. The student grades of both training and the

testing data are classified into 3 classes as Good, Fair, and Bad recommended class along this

simple rules-based.

/*Classification rules-based for actual grades */

IF rh,c ≥ 3.00 then actual class = Good

ELSEIF rh,c ≥ 2.00 then actual class = Fair

ELSE THEN actual class = Bad

/*Classification algorithm for recommended classes */

IF r£h,c ≥ 3.00 then recommended class = Good

ELSEIF r£h,c ≥ 2.00 then recommended class = Fair

ELSE THEN recommended class = Bad

The evaluation measurements for recommended courses classification results compared with

their actual class were evaluated in this section. The results of classification according to the

conventional measurements are shown in Figure 5. The results in terms of Lost, Critical, and

Balanced accuracy are proposed in Figure 6.

In Figure 5, the classification accuracy from SVD-based techniques returned the high values,

where an average of accuracy values was above 60%. An Accuracy achieved from SVD technique

based on course information approach was the highest scores, which was 0.6314 or approximately

63%. For illustrating an advantage of our proposed SVD-based algorithm, the results were compared

to the results presented in Anuradha and Velmurugan (2015). In that work, several classification

Page 20 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

algorithms based on content-based filtering technique were applied on students’ information. The

contents of the information were about attendance, class test, seminar, lab work and assignment

marks; etc. The overall accuracy result also revealed about 60%, even though; they acquired much

information to perform their work than SVD-based algorithm. Additionally, the degree of Precision

and F1-Measure shown that the SVD technique based on course information approach was highest.

Even though the degree of Recall of user-based CF depicted the highest score, it is a bit higher than

the result returned from the SVD technique based on course information approach.

Moreover, the degree of Lost, Critical, and Balanced accuracy were taken into account for

making the more precise in evaluation the course recommendation classification results. As depicted

in Figure 6, the degree of lost opportunity recommendation returned from the SVD technique based

on course information approach was lowest. The Lost value was 0.1768. This inferred that there

was less lost opportunity recommendation situation returned from this approach compared with

others. The degree of critical wrong recommendation returned from the SVD technique based on

course information approach was also lowest value. The Critical value was 0.2830. This signified

that there was less critical wrong recommendation result appeared from the course recommendation

classification results based on the SVD technique based on course information approach compared

with other approaches. Furthermore, the Balanced accuracy values, which were computed from the

standard harmonic mean between an Accuracy and an ErrorClassified of different approaches of CF

recommender techniques, were observed. The results from Figure 6 illustrated that the SVD

technique based on course information approach was the highest value. This figured out that the

harmony scores over the accuracy score and the inaccuracy score derived from the SVD technique

based on course information approach was the best score.

This can be concluded that the SVD technique based on course information is the most proper

Page 21 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

approach for implementing in the elective courses recommendation system. It reports the acceptable

scores in almost all aspects of consideration. This is because the data matrix, especially in part of

elective course, tends to be sparse in which the SVD technique has higher ability to extract the

hidden information in case of low density of the data matrix than others techniques. The hidden

information are excerpted and the baseline estimation for that data set are modelled with coarsely

representing the overall patterns of data collected in the data matrix.

5. Conclusion

This paper proposed the general process of EDM for elective courses recommendation based

on student grades. These grades are collected in the historical database via routine work.

Furthermore, the new quality measurements for course recommender system are proposed in order

to fulfill an inadequate of the precision and recall. The results from the experiment show that SVD-

based technique based on course information presented the best recommendation results in many

aspects compared with others. However, the SVD technique has the limitation which depends on the

density of the data matrix. The accuracy of this technique will drop with relative to the higher

density of data matrix. Because, the high density data matrix contains variety data patterns, this

causes the difficulty to model the generic baseline estimation model which represents all assortment

in data patterns. Thus, some effort for seeking the solutions to fulfill this issues is called for. Further

investigation on combining some contents about courses and students to this proposed model is a

challenging research endeavor remained to be explored.

Acknowledgement

This research has been supported by Faculty of Science Research Fund, Chiang Mai University.

Page 22 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

References

Anuradha, C. and Velmurugan T. 2015. A comparative analysis on the evaluation of classification

algorithms in the prediction of students performance. Indian Journal Of Science and

Technology. 8(15), 1-12.

Cacheda, F., Carneiro, V., Fernández D., and Formoso, V. 2011. Comparison of collaborative

filtering algorithms: limitations of current techniques and proposals for scalable, high-

performance recommender systems. ACM Transactions on the Web. 5(1), 2:1-33.

Kautkar, R. A. 2014. A comprehensive survey on data mining. International Journal of Research in

Engineering and Technology. 3(8), 185-191.

Lee, Y. and Cho, J. 2011. An intelligent course recommendation system. Smart Computing Review.

1(1), 69-83.

Paterek, A. 2007. Improving regularized singular value decomposition for collaborative filtering.

Proceedings of the KDD Cup Workshop at SIGKDD'07, 13th ACM International Conference

on Knowledge Discovery and Data Mining, 39-42.

Praserttitipong, D. and Sophatsathit, P. 2012. A distributed recommender agent model based on

user’s perspective SVD technique. International Journal of Digital Content Technology and its

Applications. 6(10), 108 -117.

Praserttitipong, D. and Sophatsathit, P. 2014. An agent model for information filtering using

revolutionary RSVD technique. Chiang Mai Journal of Science. 41(5/2), 1429 - 1438.

Ray, S. and Sharma, A. 2011. A collaborative filtering based approach for recommending elective

courses. Proceedings of the 5th International Conference on Information Intelligence, Systems,

Technology and Management, March 2011, 141, 330-339.

Page 23 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

Romeron, C. and Ventura, S. 1992. Educational data mining: a review of the state of the art. IEEE

Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews. 40(6),

601-618.

Page 24 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

Figure file

Figure 1 Elective Courses Recommendation Model.

Figure 2 Impact errors of precision values.

Page 25 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

Figure 3 Impact errors of recall values.

Figure 4 Comparison of MAE values according to different approaches of

CF recommender techniques.

Figure 5 Comparison of classification results from different approaches of

CF recommender techniques based on conventional measurements.

Page 26 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

Figure 6 Comparison of classification results from different approaches of

CF recommender techniques based on proposed measurement.

Page 27 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

For Review Only

Table file

Recommended Classes

Bad Fair Good Recall

Actual

Classes

Bad TB FBF FBG

Fair FFB TF FFG

Good FGB FGF TG

Precision

Table 1 A general form confusion matrix for the recommendation system.

Recommended Classes

Bad Fair Good

Actual Bad TB Wrong2 Wrong3

Classes Fair LostOp2 TF Wrong1

Good LostOp3 LostOp1 TG

Table 2 A confusion matrix for the course recommendation system.

Page 28 of 28

For Proof Read only


123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

for review only - prince of songkla universityrdo.psu.ac.th/sjstweb/ar-press/60-sep/18.pdf ·...

Documents