Ontology literature review as guidelines for

improving Croatian Qualification Framework

K. Klarin*, I. Nazor* and S. Celar** * University of Split/University Department of Professional Studies, Split, Croatia

** FESB, Split, Croatia

karmen.klarin@oss.unist.hr, igor.nazor@oss.unist.hr, stipe.celar@fesb.hr

Abstract – Development of information systems dealing

with education and labour market using web and grid

service architecture enables their modularity, expandability

and interoperability. Application of ontologies to the web

helps with collecting and selecting the knowledge about a

certain field in a generic way, thus enabling different

applications to understand, use, reuse and share the

knowledge among them. A necessary step before publishing

computer-interpretable data on the public web is the

implementation of common standards that will ensure the

exchange of information. Croatian Qualification

Framework (CROQF) is a project of standardization of

occupations for the labour market, as well as

standardization of sets of qualifications, skills and

competences and their mutual relations. This paper analysis

a respectable amount of research dealing with application of

ontologies to information systems in education during the

last decade. The main goal is to compare achieved results

according to: 1) phases of development/classifications of

education-related ontologies; 2) areas of education and 3)

standards and structures of metadata for educational

systems. Collected information is used to provide insight

into building blocks of CROQF, both the ones well

supported by experience and best practices, and the ones

that are not, together with guidelines for development of

own standards using ontological structures.

Keywords – ontology, education, e-learning, Croatian

Qualification Framework

I. INTRODUCTION

Communication between two or more dislocated information systems via web implies their ability to automatically interpret exchanged information. Use of ontology enables clear and unambiguous access to structured knowledge, while allowing enough freedom in conceptualizing and creating the final software product. [1] analyses the role of ontologies in the semantic web and the corresponding data structures, such as Linked data and Big data. Authors investigate advantages and problems of development of ontologies, their sharing and reuse, and analyze tools and techniques that enable their development and use. In general, ontologies are still not widely implemented in practical applications. Although there are differing opinions on the need to further implement them, there is a common agreement about the need for unanimous semantic, understanding and formal representation with the aim of connecting heterogeneous systems with large amounts of data.

CROQF (Croatian Qualification Framework) is based on EQF (European Qualifications Framework) and on QF-EHEA (Qualifications Frameworks in the European Higher Education Area). Main task of CROQF is to classify qualifications based on the criteria set for certain levels of learning achievements. The purpose is to integrate and coordinate national qualification systems and to improve transparency, accessibility, mobility, and quality of qualifications in relation to the market labor and civil society. It should allow effective integration of education programs and labor market requirements. One of the ways of linking labor markets and education systems is to provide support by a common language called ESCO (European Skills, Competences, Qualifications and Occupations taxonomy) standard [2], shown on Fig. 1.

ESCO should improve the exchange of information and enable cooperation between education systems, public and private job search platforms, as well as among different countries. ESCO standards should enrich and upgrade tools and applications so that they provide better options for compatibility, up-skilling and information exchange. A detailed breakdown and description of ESCO taxonomies are shown in the Chapter II and on Fig. 3.

This paper analyses the research done in the last decade in the area of application of ontologies to e-learning and education systems. Analyzed papers were identified among publicly available resources by keywords: ontology, e-learning and education. A total of 33 publications and papers (30 in journals and 3 conference papers) were analyzed. From the overview of analyzed papers by year, shown on Fig. 2, one can conclude that the subject of ontologies in education has been continuously researched.

Chapter II describes the three dimensions used for classification of papers: purpose and coverage with basic

Figure 1. ESCO standards connect education and labour market

systems

1634 MIPRO 2019/DE-DS

functionality of education system, ontology content for education system and application scenarios of ontology implementation. An overview of all papers with a summarized description of ontology content with purpose and coverage of these ontology in education systems is shown in Chapter III. An overview of papers according to application scenarios of ontology implementation, as well as purpose and coverage of these ontologies in education is given in Chapter IV. Chapter V concludes our analysis and emphasizes the problem of lack of basic concepts, such as learning outcomes, which connect the labor market system with the education system.

II. CLASSFICATION

Our domain of interest is the education ontology cluster (shown on the left-side of Fig. 1). The analysis of papers was performed over the following dimensions (main topics): Purpose and coverage, Content and Application scenarios [3] [4] [5] [6].

A. Purpose and coverage

What is it for? The purpose of ontologies covering e-learning systems can be classified according to the following characteristics [3]:

Curriculum modelling and management covers institution’s processes and organization for development, design, and implementation of each degree program’s organization, structure, content, assessment of outcomes, and pedagogy.

Describing learning domains incorporate subject-domain ontologies (providing substance for learning process) and task ontologies (activities, exercises, case studies, projects, simulations, and assessment items).

Describing learner data is related to an individual learner (learning profile, completed content and

performance data).

Describing e-learning services - collaboration between heterogeneous e-learning systems. Ontologies are used for searchable, accessible, and sharable learning resources (services such as IEEE LOM, Dublin Core, and SCORM).

B. Content

What is in it? Fig. 3 shows detailed set of ontologies from Fig. 1, their connections and applied standards (taxonomies). According to ESCO interoperability map [4] and main relationships and classifications from [5] the following domains were identified:

Education Ontology includes information about levels and fields of education. Fields of education are based on FOET (Fields of Education and Training) taxonomy and level of education on standard ISCED (International Standard Classification of Education).

Competency Ontology defines the proven ability to use knowledge, skills and personal, social and methodological abilities. ESCO defines DigComp (Digital Competence Framework) which provides a vocabulary of digital competences at the European level.

Knowledge Ontology contains the body of facts, principles, theories and practices that is related to a field of work or study (e.g. CS2013 – Computer Science Curricula 2013, from https://dl.acm.org/ citation.cfm?id=2534860).

Skills Ontology defines the ability to apply knowledge and use know-how to complete tasks and solve problems. The task of CROQF is to define the National skills classification.

Qualification Ontology is described in terms of knowledge, skills, responsibility and autonomy and should be attainable though a variety of educational and career paths.

Occupation Ontology includes information about occupations and workplaces. An integral part of this ontology is ISCO (International Standard Classification of Occupations). Occupation ontology is outside of this paper’s scope.

Figure 2. Overview of analysed papers by year

Figure 3. Ontologies cluster for occupation and education systems

MIPRO 2019/DE-DS 1635

C. Application scenarios

How do ontologies help? Four main categories of ontology application scenarios are given in [6], showing a wide range of ontology representations, from semantically lightweight to semantically rich ones, such as:

Neutral ontology for authoring, where individuals develop their own ontologies for authoring operational data with different applications. Individual organizations or companies use ontologies as translators for various proprietary software systems.

Ontology as specification, domain-specific ontology created to be implemented manually or by partially-automated software tools. This level of ontology application automates documentation, maintenance, reliability, knowledge use and reuse.

Common access to information, where ontology information is used by multiple agents, as a mutually agreed basic standard for information conversion and mapping, thus facilitating interoperability and effective knowledge (re)use.

Ontology-based search, where an ontology is used by a structuring device (e.g. search engine) as information repository allowing better access to information. This ontology application scenario allows using the same ontology for multiple systems as a way of concept-based structuring of information.

D. Two types of analyses

The analysis is performed by grouping the available literature according to the topics covered, in relation to the topics (dimensions) of interest:

1. Purpose and coverage in relation to Content (Chapter III with Table 1), and

2. Purpose and coverage in relation to Application scenarios (Chapter IV with Table 2).

During the both analysis a special focus was placed on Curriculum modelling and management, and Describing e-learning services, which are parts of Purpose and coverage dimension, as well as the main functionalities of processes implemented within the CROQF.

Skills, Competences and Qualifications ontologies, which belong to dimension Content, are integral parts of both the ESCO and the CROQF standards, so the research results that include these ontologies are a valuable asset that can be used for development of CROQF.

Furthermore, the knowledge about Application scenarios of ontology implementation obtained from researched papers can be used to provide insight into the building blocks of CROQF.

III. PURPOSE AND COVERAGE VS CONTENT

Table 1 show frequency of articles according to the two main topics covered (dimensions), each shown as a hierarchy of characteristics. The hierarchies contain a relatively large number of characteristics in order of their

influence to be quantified as precisely as possible during the phase of analysis. One paper’s scope may cover more than one characteristic, therefore the same reference may be found in more than one place within a single table.

As shown in Table 1, ontologies Knowledge and Education are mentioned in a large number of papers (30 of 33 papers analyzed), as opposed to Skills, Competences and Qualifications, mentioned in only 11 papers, and consequently researched to a lesser extent.

Ontology-based system for educational and Curriculum modeling and management [22] [36] with semantic repository and predefined semantic queries. An approach to achieve optimal sequence of learning the contents of a course and taking tests is researched in [19]. In [25] a technique for developing an individualized learning path that meets user’s requirements is described. It uses revised Bloom’s taxonomy to specify concepts of a domain ontology: courses, lessons and their sequences. Besides development process of an educational ontology [32] emphasizes the importance of specifying competences in terms of learning outcomes. A semantic web-based system, Gescur [22] [24], is used for managing the education curriculum, data about institutions, knowledge areas and subjects.

A Java ontology integrated toolkit, used to create an ontology-based application that specifies the Describing learning domain is described in [26]. Development of an ontology-based e-learning support system [7] [30] [35] includes specification of curriculum, syllabus and subject ontologies, as well as discovery and sharing of knowledge based on subjects’ ontology (teachers and students). [24] demonstrates the use of semantic web system Gescur for improvement of education process quality. An intelligent approach to compose optimal learning groups with members having different domain backgrounds is described in [13]. [34] deals with interoperability problem and ontology mapping between educational domains elements (syllabus, courses, materials, exams) and pedagogical resources. Modeling of a particular knowledge area with concept maps was created by experts and with the guidance of teacher judgment in [39]. In [8] and [27] authors define the main knowledge domain concepts and relations in form of graphs, maps or

TABLE I. PURPOSE AND COVERAGE VS CONTENT

Content

Purpose and coverage

Curriculum

modeling and manag.

Describing

learning domains

Describing

learner data

Describing

e-learning services

Education Ontology

[19] [22] [25] [36]

[7] [8] [13]

[24] [26] [30] [34]

[35]

[7] [8] [12] [16] [28]

[9] [17]

[18] [20] [21] [23]

[33] [38]

Competency

Ontology [32] [11] [29]

[15] [19] [21] [25]

[37]

[9]

Knowledge Ontology

[22] [24]

[30]

[12] [14]

[27] [32] [39]

[8] [14] [15]

[16] [23] [27] [39]

[11] [31]

Skills Ontology

[10] [10] [13]

Qualification Ontology

[11]

1636 MIPRO 2019/DE-DS

diagrams. An ontological representation of cognitive skills is described in [10].

Educational domain modelling and Describing learner data in [8] are a basis for personalized e-learning. A method for tracking students’ active learning and self-leading learning is proposed in [7] and [12]. Ontology-based e-learning systems [16] [28] make adaptations and improvements to the learning process by analyzing students’ learning performance and courses’ ontology. Knowledge-based systems which assists learners in achieving desired knowledge and goals are described in [13] [15] and [19]. A domain ontology offers learners a personalized choice of registered courses and materials [21] [25] [37], in order to achieve personalized competences. To improve the content and structure of personalized course knowledge [14] [23] use ontology-based metadata for describing both the entire domain and specific learning objects.

Metadata standards of the Describing e-learning services domain, such as LOM, Dublin Core and SCORM, are used to develop different educational systems: PASER system [9] supporting educational process; GrEd [17] used for education in Greece; LO management model [18] used to find correlation between learning styles and LO characteristics; SMILE system [20] enabling semantic ontology mapping for interoperability of learning resource systems, and a framework for building an adaptive Learning Management System [21]. MOODLE system is used in [33] to classify learning objects and to develop modern semantic learning management system. Education objective metadata repository in [9] provides a system-wide consistent competency vocabulary. [11] and [23] describes e-learning at the workplace and link between learning and work performance, as well as the communication between individuals.

An overview of papers frequency across the Content dimension hierarchy (shown on the left side of Fig. 4) shows that a large number of papers cover the Knowledge and Education ontologies, specifically their following elements: syllabus, courses, materials, exams, and pedagogical resources. On the other hand, Competency, Skills and Qualification ontologies, relevant to CROQF, are covered in a small number of papers. The breakdown across functionalities Curriculum modeling and management and Describing e-learning services, which

are relevant to CROQF (right side of Fig. 4), shows the same trend.

The experiences gathered from the current research on the education systems are a valuable reference for the task of designing basic concepts of CROQF. The ontologies that not adequately researched should be developed from the beginning, from conceptualization to formal application.

IV. PURPOSE AND COVERAGE VS APPLICATION

SCENARIOS

Regarding different application scenarios, which can be divided according to the level of maturity of implementation of ontologies, lower levels of maturity have received a greater focus. As shown in Table 2, scenarios Neutral ontology for authoring and Ontology as specification, representing lower levels of maturity, have been researched in 22 papers, as opposed to Common access to information and Ontology-based search, which were mentioned in 14 papers.

Curriculum modeling and management is presented in [22] [25] [32] [36] as conceptualization of education ontology. In [24] ontology structure is used by Gescur web–based system to execute semantic queries. [33] presents the design of curriculum ontology and syllabus ontology in detail with implementation of mapping rules with the aim of connecting different entities that contain different formats of syllabi. Web applications in [17] and [19] use ontologies to: define vocabularies, share common

Figure 4. Frequency of articles for Purpose and coverage dimesion in relation to Content dimension

TABLE II. PURPOSE AND COVERAGE VS APPLICATION

SCENARIOS

Application

scenarios

Purpose and coverage

Curriculum

modeling and manag.

Describing

learning domains

Describing

learner data

Describing

e-learning services

Neutral ontology for

authoring

[22] [25]

[32] [36]

[8] [30] [32]

[35] [39]

[13] [23]

[25] [39] [23]

Ontology as specification

[24] [30]

[7] [10] [11]

[13] [14] [24]

[27] [34]

[7] [10]

[19] [27]

[28]

[11] [18] [31] [33]

Common access to

information

[19] [12] [26] [29]

[15] [16] [21] [37]

[17] [38]

Ontology-based

search

[8] [12]

[14]

[9] [20]

[21]

MIPRO 2019/DE-DS 1637

understanding of information structure among people or software agents, and enable reuse of domain knowledge.

Describing learning domains by conceptualizing educational knowledge structure, resulting in concept maps, is given in [32] [35] and [39]. Domain-specific ontology is created for adaptive learning path implementation [7] or for multidisciplinary learning group composition [13] [14] and [34]. Knowledge and content structuring, using collaborative ontology design methods, is presented in [27]. In [25] and [26] the ontology of student activities is designed to store information about their learning process and results. An agent platform, described in [12], supports various personalization levels by using machine-understandable representations of educational domains and learners’ characteristics.

An approach to learning where learning objects are based on students' specific needs results in them developing a unique learning style. Describing learner data [23] [25] [39] deals with determining, managing and monitoring such learning objects, as well as tracking students' time spent learning different content. In [13] authors propose a flexible e-learning self-evolving system, so that enrolled trainees can become trainers at the end of the process. [10] [19] and [28] describe different ontology-supported methods of processing contents of e-learning systems. Improved methods of student assessment during learning process used by educators are described in [12] [14] [16] and [37]. Results of assessment can be used to adjust syllabus concepts, such as body of knowledge and area/unit of knowledge taxonomies.

Describing e-learning services using [23] ontology and LOM standard to create a concept tree of learning objects and detect the student’s previous knowledge level. Application of semantic web techniques to services of e-learning system, such as searching context relevant learning resources by means of decision trees and ontology mapping, are described in [18] and [33]. An ontology-based learning management system was used to manage e-learning knowledge in [31]. [9] and [21] present intelligent systems with ontologies, metadata repositories and knowledge-based modules that use queries and reasoning on learning metadata.

An overview of papers frequency across the Application scenarios dimension hierarchy (shown on the left side of Fig. 5) shows that a large number of papers cover Neutral ontology for authoring and Ontology as specification, which shows that the current research is focused on specification and conceptualization of subject-domain ontology. From the perspective of CROQF-related

ontologies, such results are encouraging, because they constitute a solid basis for their development. A small number of papers that covered Common access to information and Ontology based search, points out the complex task of ontology development, implementation, use and reuse. The breakdown across functionalities Curriculum modeling and management and Describing e-learning services, which are relevant to CROQF (right side of Fig. 5), shows the same trend. Consequently, work on the CROQF will be a demanding and long-lasting job.

V. CONCLUSION

The main recommendation for the process of establishment of EQF and national qualifications, CROQF in our case, is for them to be related to learning outcomes. Development of a methodology for description, use and application of learning outcomes has the aim of increasing transparency, understanding and comparability of qualifications. Web technology and ontology-based systems can ensure semantic interoperability between characteristics used to describe learning outcomes, such as skills, competences and qualifications, which would help to reduce the communication gap between EU countries and between domains of employment, education and training.

The research described in this paper is focused on ontology-based systems in education. Literature is analyzed with respect to two aims: representation of ontologies relevant to CROQF (Skills, Competences and Qualifications), and maturity of ontology-based software implementation.

Analyzed papers, as expected, contain many references to the Education and Knowledge ontology which support the learning process. However, the most important aim of the education process, learning and learning outcomes, and the appurtenant ontologies Skills, Competences and Qualifications, are researched to a much lesser extent. They represent an upgrade of the e-learning process and a connection with learning outcomes and qualifications. Their further analysis is recommended, as well as research into the possibilities of their implementation in both the CROQF and EQF.

For maturity of ontology-based software implementation, majority research is focusing on low-level ontology implementations, such as modelling of interest domains, and less on a holistic approach aimed at integrating heterogeneous systems and achieving their interoperability. In addition to further research ontology-based systems and their application to CROQF, additional

Figure 5. Frequency of articles for Purpose and coverage dimesion in relation to Application scenarios dimension

1638 MIPRO 2019/DE-DS

work should be done developing web services and applications for facilitating interoperability and effective knowledge use and reuse.

REFERENCES

[1] M. Bennett and K. Baclawski, “The role of ontologies in Linked Data, Big Data and Semantic Web applications,” Applied Ontology, vol. 12, pp. 189-194, November 2017.

[2] ESCO - European classification of skills/competences, qualifications and occupations, Luxembourg: Publications Office of the European Union, 2013.

[3] M. Al-Yahya, R. Geore and A. Alfaries, “Ontologies in e-learning: review of the literature,” International Journal of software engineering and its applications, vol. 9, pp. 67-84, January 2015.

[4] European Commision, ESCO Handbook. European Union, 2017.

[5] A. Gómez-Pérez, J. Ramírez and B. Villazón-Terrazas, “An ontology for modelling human resources management based on standards,” In KES 2007, vol. 4692, B. Apolloni, R.J. Howlett and L. Jain, Eds. Springer: Berlin, 2007, pp. 534-541.

[6] M. Uschold and M. Gruninger, “Ontologies and semantics for seamless connectivity,” ACM SIGMOD Record, vol. 33(4), pp. 58-64, December 2004.

[7] H.-S. Chung and J.-M. Kim, “Ontology design for creating adaptive learning path in e-learning environment,” Proc. of IMECS, vol. I, July 2012.

[8] M. Gaeta, F. Orciuoli and P. Ritrovato, “Advanced ontology management system for personalised e-learning,” Knowledge-Based System, vol. 22, pp. 292-301, 2019.

[9] E. Kontopoulos, D. Vrakas, F. Kokkoras, N. Bassiliades and I. Vlahavas, “An ontology-based planning system for e-course generation,” Expert Systems with Applications, vol. 33, pp. 398-406, July-August 2009.

[10] P. Askar and A. Altun, “CogSkillnet: An ontology-based representation of cognitive skills,” Journal of Educational Technology & Society, vol. 12(2), pp. 240-253, 2009.

[11] H. Jia, M. Wang, W. Ran, S.J.H. Yang, J. Liao, and D.K.W. Chiu, “Design of a performance-oriented workplace e-learning system using ontology,” Expert Systems with Applications, vol. 38(4), pp. 3372-3382, 2010.

[12] G. Acampora, M. Gaeta and V. Loia, “Exploring e-learning knowledge through ontological memetic agents,” IEEE Computational Intelligence magazine, vol 5(2), pp. 66-77, 2010.

[13] M.-I. Dascalu, C.-N. Bodea, M. Lytras, P.O. de Pablos and A. Burlacu, “Improving e-learning communities through optimal composition of multidisciplinary learning groups,” Computers in human behavior, vol. 30, pp. 362-371, 2014.

[14] L. Zhuhadar, O. Nasraoui, R. Wyatt and E. Romero, “Multi-model ontology-based hybrid recommender system in e-learning domain,” IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, September 2009.

[15] A. Nitchot, W. Wettayaprasit and L. Gilbert, “Assistive tool for constructing knowledge structures and suggesting related study materials links,” Education and Information Technologies, vol. 24(1), pp. 219-230, January 2019.

[16] F. Colace, and M. De Santo, “Ontology for e-learning: A Bayesian approach,” IEEE Transactions on Education, vol. 53(2), pp. 223-233, June 2010.

[17] Y. Poulakakis, K. Vassilakis, M. Kalogiannakis and S. Panagiotakis, “Ontological modeling of educational resources: a proposed implementation for Greek schools,” Education and Information Technologies, vol. 22(4), pp. 1737-1755, July 2017.

[18] S. Wanapu, C. Che Fung, N. Kerdprasop, N. Chamnongsri and S. Niwattanakul, “An investigation on the correlation of learner styles and learning objects characteristics in a proposed Learning Objects Management Model (LOMM),” Education and Information Technologies, vol. 21(5), pp. 1113-1134, 2016.

[19] Y.-L. Chi, “Ontology-based curriculum content sequencing system with semantic rules,” Expert Systems with Applications, vol. 36(4), pp. 7838–7847, May 2009.

[20] N. Arch-int and S. Arch-int, “Semantic ontology mapping for reasoning approach,” Expert Systems with Applications, vol. 40(18), pp. 7428-7443, December 2013.

[21] M. Yaghmaieand A. Bahreininejad, “A context-aware adaptive learning system using agents,” Expert Systems with Applications, vol. 38(4), pp. 3280-3286, April 2011.

[22] J. T. Fernández-Breis et al., “A semantic platform for the management of the educative curriculum,” Expert Systems with Applications, vol. 39(5), pp. 6011-6019, April 2012.

[23] S. Cakula and M. Sedleniece, “Development of a personalized e-learning model using methods of ontology,” Procedia Computer Science, vol. 26, pp. 113-120, 2013.

[24] D. Castellanos-Nieves et al., “Semantic Web-based system for managing the educative curriculum,” Procedia Social and Behavioral Sciences, vol. 26(2), pp. 521-526, 2010.

[25] V. Shmelev, M. Karpova, and A. Dukhanov, “An Approach of Learning Path Sequencing based on Revised Bloom’s Taxonomy and Domain Ontologies with the use of Genetic Algorithms,” Procedia Computer Science, vol. 66, pp. 711-719, 2015.

[26] O. Holanda, S. Isotani, I.I. Bittencourt, E. Elias and T. Tenorio, “JOINT: Java ontology integrated toolkit,” Expert Systems with Applications, vol. 40, pp. 6469–6477, 2013.

[27] T. A. Gavrilova and I. A. Leshcheva, “Ontology design and individual cognitive peculiarities: A pilot study,” Expert Systems with Applications, vol. 42(8), pp. 3883-3892, May 2015.

[28] B. Saleena and S.K. Srivatsa, “Using concept similarity in cross ontology for adaptive e-Learning systems,” Journal of King Saud University - Computer and Information Sciences, vol 27(1), pp. 1-12, January 2015.

[29] D. Mouromtsev, F. Kozlov, Li. Kovriguina and O. Parkhimovich, “Combined method for e-learning ontology population based on NLP and user activity analysis,” LILE@ISWC, Riva del Garda, Trentino, Italy, 2014.

[30] H. Chung and J. Kim, “An ontological approach for semantic modeling of curriculum and syllabus in higher education,” International Journal of Information and Education Technology, vol. 6(5), pp. 365-369, May 2016.

[31] M. Rani, K. V. Srivastava and O. P. Vyas, “An ontological learning management system,” Computer Applications in Engineering Education, Wiley Online Library, 2016.

[32] E. Katis , H. Kondylakis, G. Agathangelos and K. Vassilakis, “Developing an ontology for curriculum and syllabus,” In The Semantic Web ESWC 2018 Satellite Events LNCS, vol. 11155, A. Gangemi et al., Eds. Springer: Cham, 2018, pp. 55-59.

[33] A. Ali, A. Ahmed and A. Yousif, “Applying semantic web technologies to services of e-learning system,” International Journal of Engineering Inventions, vol. 4(6), pp. 24-38, 2014.

[34] G. Moise and L. Netedu, “Ontologies for interoperability in the eLearning systems,” Bulletin of Petroleum-gas University of Ploiesti, series of mathematics -computer science-physics, vol. LXI(2), pp. 75-88, 2009.

[35] N. Khozooyi, N. Seyedi and R. Malekhoseini, “Ontology-based e-learning,” International Journal of Computer Science and Information Technology & Security, vol. 2(4), pp. 859-864, 2012.

[36] L. Tang and X. Chen, “Ontology-based semantic retrieval for education management systems,” Journal of computing and information technology, vol. 23(3), pp. 255-267, September 2015.

[37] B. Vesin, M. Ivanovic, A. Klasnja-Milicevic. and Z. Budimac, “Protus 2.0: Ontology-based semantic recommendation in programming tutoring system,” Expert Systems with Applications, vol. 39(15), pp. 12229–12246, November 2012.

[38] P. Wongthongtham et al., “State-of-the-Art ontology annotation for personalised teaching and learning and prospects for smart learning recommender based on multiple intelligence and fuzzy ontology,” International Journal of Fuzzy Systems, vol. 20(4), pp. 1357–1372, April 2018.

[39] K. Icoz, V. A. Sanalan, M. A. Cakar, E. B. Ozdemir and S. Kaya, “Using students’ performance to improve ontologies for intelligent e-Learning system,” Educational Sciences: Theory & Practice, vol. 15(4), pp. 1039-1049, August 2015.

MIPRO 2019/DE-DS 1639