Download - Recommender Systems in TEL
![Page 1: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/1.jpg)
Recommender Systems Recommender Systems in TELin TEL
Nikos ManouselisNikos ManouselisGreek Research & Technology Network (GRNET)Greek Research & Technology Network (GRNET)
![Page 2: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/2.jpg)
about meabout me•Computer EngineerComputer Engineer•MSc on Operational ResearchMSc on Operational Research•PhD from Informatics Lab of an Agricultural PhD from Informatics Lab of an Agricultural
UniversityUniversity•working on services for agricultural & rural working on services for agricultural & rural
communitiescommunities– learning repositorieslearning repositories– social information retrievalsocial information retrieval– Organic.Edunet Organic.Edunet eeContentContentplusplus
![Page 3: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/3.jpg)
(promised)aim of this (promised)aim of this lecturelecture
• introduce recommender introduce recommender systemssystems
•discuss how they relate to TELdiscuss how they relate to TEL• identify open research issuesidentify open research issues
![Page 4: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/4.jpg)
(actual)aim of this (actual)aim of this lecturelecture
•share some concerns about TEL share some concerns about TEL and recommender systemsand recommender systems
![Page 5: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/5.jpg)
structurestructure
•tale of 3 friendstale of 3 friends•taskstasks•modeling & techniquesmodeling & techniques•evaluationevaluation•wrap upwrap up
![Page 6: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/6.jpg)
intro: tale of 3 friendsintro: tale of 3 friends
![Page 7: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/7.jpg)
which movie?which movie?
![Page 8: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/8.jpg)
lets ask some friendlets ask some friend
““Guys, heard about the last Batman Guys, heard about the last Batman movie… should I watch it?”movie… should I watch it?”
“You will definitely
like it”
“Maybe not, the scenario is
too weak”
![Page 9: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/9.jpg)
lets ask some friendlets ask some friend
““Wait – did you like the previous one?”Wait – did you like the previous one?”
![Page 10: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/10.jpg)
……so, which movie?so, which movie?
• taking advantage of knowledge or experience taking advantage of knowledge or experience from people in the social circle or networkfrom people in the social circle or network– e.g. colleagues, friends, peerse.g. colleagues, friends, peers
• need to answer several questions need to answer several questions – how to identify like-minded people?how to identify like-minded people?– on which dimensions?on which dimensions?– for which types of items? for which types of items? – does context matter?does context matter?– ……
![Page 11: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/11.jpg)
recommender recommender systemssystems
![Page 12: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/12.jpg)
![Page 13: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/13.jpg)
![Page 14: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/14.jpg)
• using the opinions of a community of users– to help individuals in that community to
identify more effectively content of interest
– from a potentially overwhelming set of choices
Resnick P. & Varian H.R., “Recommender Systems”, Communications of the ACM, 40(3),1997
definition definition (1/2)(1/2)
![Page 15: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/15.jpg)
definition definition (2/2)(2/2)
• any system that – produces individualized
recommendations as output – or has the effect of guiding the user in a
personalized way to interesting or useful objects in a large space of possible options
Burke R. “Hybrid Recommender Systems: Survey and Experiments”, User Modeling & User-Adapted Interaction, 12, 331-370, 2002
![Page 16: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/16.jpg)
why do we need them?why do we need them?• A trip to a local supermarket [F. Ricci]:
– 85 different varieties and brands of crackers– 285 varieties of cookies– 165 varieties of “juice drinks”– 75 iced teas– 275 varieties of cereal– 120 different pasta sauces– 80 different pain relievers– 40 options for toothpaste– 95 varieties of snacks (chips, pretzels, etc.)– 61 varieties of sun tan oil and sunblock– 360 types of shampoo, conditioner, gel, and mousse.– 90 different cold remedies and decongestants.– 230 soups, including 29 different chicken soups– 175 different salad dressings
![Page 17: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/17.jpg)
wait a secondwait a second
is TEL like a super is TEL like a super market??market??
![Page 18: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/18.jpg)
large number of optionslarge number of options
![Page 19: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/19.jpg)
tasks for tasks for recommender recommender
systemssystems
![Page 20: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/20.jpg)
tasks usually supportedtasks usually supported
1.1. annotation in contextannotation in context2.2. find good itemsfind good items3.3. find find all all good itemsgood items4.4. receive sequence of itemsreceive sequence of items
(+some less important ones)(+some less important ones)
Herlocker et al., “Evaluating Collaborative Filtering Recommender Systems” ACM Transactions on Information Systems, 22(1), 5-53, 2004.
![Page 21: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/21.jpg)
1. annotation in context1. annotation in context
• integrated in existing working integrated in existing working environment to provide additional environment to provide additional support or information, e.g.support or information, e.g.– predicted usefulness of an item that predicted usefulness of an item that
the user is currently viewingthe user is currently viewing– links within a Web page that the user links within a Web page that the user
is recommended to followis recommended to follow
![Page 22: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/22.jpg)
annotation in contextannotation in context
• Screenshot/exampleScreenshot/example
![Page 23: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/23.jpg)
2. find good items2. find good items
• suggesting specific item(s) to a suggesting specific item(s) to a useruser– characterized as core characterized as core
recommendation task, since recommendation task, since occurring in most systemsoccurring in most systems
– e.g. presenting a ranked list of e.g. presenting a ranked list of recommended itemsrecommended items
![Page 24: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/24.jpg)
find good itemsfind good items
• Screenshot/exampleScreenshot/example
![Page 25: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/25.jpg)
3. find all good items3. find all good items
• user wants to identify user wants to identify all all items items that might be interestingthat might be interesting– when its important not to when its important not to
overlook any potentially relevant overlook any potentially relevant casecase
– e.g. medical or legal casese.g. medical or legal cases
![Page 26: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/26.jpg)
find all good itemsfind all good items
![Page 27: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/27.jpg)
4. sequence of items4. sequence of items
• sequence of related items is sequence of related items is recommended to the userrecommended to the user– e.g. entertainment applications e.g. entertainment applications
such as TV or radio programssuch as TV or radio programs
![Page 28: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/28.jpg)
sequence of itemssequence of items
![Page 29: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/29.jpg)
and what about TEL?and what about TEL?
• informal reminder:informal reminder: – technology enhanced learningtechnology enhanced learning
is generally dealing with the ways is generally dealing with the ways ICTICT can be used to support can be used to support learninglearning, , teachingteaching, and , and competence developmentcompetence development
[http://cordis.europa.eu/fp7/ict/telearn-digicult/telearn_en.html][http://cordis.europa.eu/fp7/ict/telearn-digicult/telearn_en.html]
![Page 30: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/30.jpg)
break2thinkbreak2think
• bring yourself in one typical bring yourself in one typical learning situationlearning situation that that occurs very often to occurs very often to YOUYOU
![Page 31: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/31.jpg)
break2thinkbreak2think
• imagine that some imagine that some magicmagic TEL TEL system is there to support yousystem is there to support you
– it could make some great it could make some great suggestionssuggestions about about somethingsomething to to youyou
• name name one learning task one learning task where a recommender system where a recommender system would be would be usefuluseful
![Page 32: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/32.jpg)
modeling & techniquesmodeling & techniques
![Page 33: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/33.jpg)
typical classification
• content-based: information needs of user and characteristics of items are represented in some (usually textual) form
• collaborative filtering: user is recommended items that people with similar tastes and preferences liked
• hybrid: methods that combine content-based and collaborative methods
…other categorizations also exist (Burke, 2002)
![Page 34: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/34.jpg)
example: content-based
![Page 35: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/35.jpg)
example: collaborative filtering
![Page 36: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/36.jpg)
generally speaking: some generally speaking: some useruser
• has a profile with some user has a profile with some user characteristics, e.g.characteristics, e.g.– past ratings past ratings [collaborative filtering][collaborative filtering]
– keywords describing past keywords describing past selections selections [content-based [content-based recommendation] recommendation]
![Page 37: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/37.jpg)
generally speaking: some generally speaking: some itemsitems
• are represented using some are represented using some dimensions, e.g.dimensions, e.g.– satisfaction over one (or satisfaction over one (or
more) criteria more) criteria [collaborative [collaborative filtering]filtering]
– item attributes/features item attributes/features [content-based recommendation][content-based recommendation]
![Page 38: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/38.jpg)
generally speaking: a generally speaking: a mechanismmechanism
• is taking advantage of the is taking advantage of the user user profileprofile and the and the item item representationsrepresentations
– it provides personalised it provides personalised recommendations of items to recommendations of items to usersusers
![Page 39: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/39.jpg)
rings some bell?
for TEL, this sounds so…for TEL, this sounds so…
adaptive educational adaptive educational hypermedia systemshypermedia systems ((AEHSAEHS))
![Page 40: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/40.jpg)
a generic architecturea generic architecture
[Karampiperis & Sampson, 2005][Karampiperis & Sampson, 2005]
![Page 41: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/41.jpg)
an examplean example
[Karampiperis & Sampson, 2005][Karampiperis & Sampson, 2005]
![Page 42: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/42.jpg)
enhanced version of [Hanani et al., "Information Filtering: Overview of Issues, Research and Systems", User Modeling and User-Adapted Interaction, 11, 2001]
classification/analysis
![Page 43: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/43.jpg)
recommend in TEL based on recommend in TEL based on what?what?
• on learner models/profileson learner models/profiles– e.g. learning styles, competence e.g. learning styles, competence
gapsgaps– ……other ideas?other ideas?
• on item characteristicson item characteristics– e.g. interactivity, granularity, e.g. interactivity, granularity,
accessibilityaccessibility– ……other ideas?other ideas?
![Page 44: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/44.jpg)
evaluationevaluation
![Page 45: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/45.jpg)
evaluating recommendation
• currently based on performance “how good are your algorithms?”
• e.g. – how accurate are they in predictions?– for how many unknown items can they
produce a prediction?
…mainly information retrieval evaluation approaches
[Herlocker et al., “Evaluating Collaborative Filtering Recommender Systems” ACM Transactions on Information Systems, 22(1), 5-53, 2004]
![Page 46: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/46.jpg)
typical results
MAE per # of neighbors
0.55000
0.60000
0.65000
0.70000
0.75000
0.80000
0.85000
0.90000
0.95000
1.00000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
# of neighbors
MA
E
means that a prediction could be 4,6 stars instead of 4 or 5 … does this really matter in TEL?
![Page 47: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/47.jpg)
other issues
•live experiments vs. offline analyses
•synthesized vs. natural data sets– properties of data sets– existing data sets
![Page 48: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/48.jpg)
metrics (popular)
• accuracy– predictive accuracy (MAE)– classification accuracy
• Precision and Recall– probability that a selected item is relevant– probability that a relevant item will be selected
• ad hoc– Rank Accuracy Metrics– Prediction-Rating Correlation
• coverage– percentage of items for which prediction is
possible
![Page 49: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/49.jpg)
metrics (not popular)
• novelty • serendipity• confidence• user evaluation
– explicit (ask) vs. implicit (observe)– laboratory studies vs. field studies– outcome vs. process– short-term vs. long-term
![Page 50: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/50.jpg)
evaluation in TEL recommenders
• few systems actually evaluated– even fewer actually tried with users
• recent analysis of 15 TEL recommender systems:– half of the systems (8/15) still at
design or prototyping stage– only 5 systems evaluated through
trials with human users
[N.Manouselis, H.Drachsler, R.Vuorikari, H.Hummel, R.Koper, “Recommender Systems in Technology Enhanced Learning”, Handbook of Recommender Systems (under review)]
![Page 51: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/51.jpg)
example: Altered Vista
• evaluate the effectiveness and usefulness– system usability and performance– predictive accuracy of recommender engine– extent to which reviewing Web resources within
a community of users supports and promotes collaborative and community-building activities
– extent to which critical review of Web resources leads to improvements in user’s information literacy skills
[Walker et al., “Collaborative Information Filtering: a review and an educational application”, International Journal of Artificial Intelligence in Education 14, 2004]
![Page 52: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/52.jpg)
another look at it
• e.g. using Kirckpatrick’s model on evaluating training programsa. reaction of student - what they thought
and felt about the trainingb. learning - the resulting increase in
knowledge or capabilityc. behaviour - extent of behaviour and
capability improvement and implementation/application
d. results - the effects on the business or environment resulting from the trainee's performance
![Page 53: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/53.jpg)
what else could be what else could be evaluated?evaluated?
• when deploying a when deploying a recommender system in a TEL recommender system in a TEL settingsetting
……what could we evaluate and what could we evaluate and how to measure it?how to measure it?
![Page 54: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/54.jpg)
wrap up & directionswrap up & directions
![Page 55: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/55.jpg)
basic conclusionbasic conclusion
• assuming an assuming an information information overload overload problem in TELproblem in TEL– recommender systems are recommender systems are goodgood– need to think need to think out of the boxout of the box– connect with connect with existing researchexisting research– focus on TEL focus on TEL particularitiesparticularities– exploreexplore alternative alternative uses uses – integrate with integrate with existing theoriesexisting theories
![Page 56: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/56.jpg)
interesting (?) issuesinteresting (?) issues
• recommendation of peersrecommendation of peers• criteria for expressing learner criteria for expressing learner
satisfaction satisfaction (no more 5-stars)(no more 5-stars)
• study actual usage/acceptancestudy actual usage/acceptance• assess performance/learning assess performance/learning
improvementimprovement
……implement, deploy, pilot!implement, deploy, pilot!
![Page 57: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/57.jpg)
but do they exist??but do they exist??
http://www.oerrecommender.orghttp://www.oerrecommender.org
![Page 58: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/58.jpg)
interested in more?interested in more?
• Journal of Digital Information (JoDI)Journal of Digital Information (JoDI)– Special Issue on Social Information Retrieval Special Issue on Social Information Retrieval
for Technology-Enhanced Learning, 10(2), 2009for Technology-Enhanced Learning, 10(2), 2009• Workshop on Social Information Workshop on Social Information
Retrieval for Technology Enhanced Retrieval for Technology Enhanced Learning (SIRTEL)Learning (SIRTEL)
– SIRTEL 2007 (http://ceur-ws.org/Vol-307) SIRTEL 2007 (http://ceur-ws.org/Vol-307) – SIRTEL 2008 (http://ceur-ws.org/Vol-382)SIRTEL 2008 (http://ceur-ws.org/Vol-382)– SIRTEL 2009 (http://celstec.org/sirtel)SIRTEL 2009 (http://celstec.org/sirtel)
• co-located with ICWL’09, Aachen, Germany, August co-located with ICWL’09, Aachen, Germany, August 2121stst - deadline: 12/6 - deadline: 12/6
![Page 59: Recommender Systems in TEL](https://reader033.vdocuments.mx/reader033/viewer/2022061218/54b6f4ab4a795915788b4580/html5/thumbnails/59.jpg)
thank you!thank you!
questions? ideas?questions? ideas?