Using Tags and Latent Factors in a Food RecommenderSystem

Mouzhi GeFree University ofBozen-BolzanoBolzano, Italy

mouzhi.ge@unibz.it

Mehdi ElahiFree University ofBozen-BolzanoBolzano, Italy

mehdi.elahi@unibz.it

IgnacioFernández-Tobías

Universidad Autónoma deMadrid

Madrid, Spainignacio.fernandezt@uam.es

Francesco RicciFree University ofBozen-BolzanoBolzano, Italy

francesco.ricci@unibz.it

David MassimoFree University ofBozen-BolzanoBolzano, Italy

david.massimo@stud-inf.unibz.it

ABSTRACTDue to the extensive growth of food varieties, making betterand healthier food choices becomes more and more complex.Most of the current food suggestion applications offer justgeneric advices that are not tailored to the user’s personaltaste. To tackle this issue, we propose in this paper a novelfood recommender system that provides high quality andpersonalized recipe suggestions. These recommendations aregenerated by leveraging a data set of users’ preferences ex-pressed in the form of users’ ratings and tags, which signalthe food’s ingredients or features that the users like. Ourempirical evaluation shows that the proposed recommenda-tion technique significantly outperforms state-of-the-art al-gorithms. We have found that using tags in food recom-mendation algorithms can significantly increase the predic-tion accuracy, i.e., the match of the predicted preferenceswith the true user’s preferred recipes. Furthermore, our userstudy shows that our system prototype is of high usability.

Categories and Subject DescriptorsH.4.2 [Information Systems Applications]: Types ofSystems—Decision support ; H.3.3 [Information Storageand Retrieval]: Information Search and Retrieval—infor-mation filtering

KeywordsRecommender systems, smart health, food, matrix factor-ization, tags

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for profit or commercial advantage and that copies bearthis notice and the full citation on the first page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from permissions@acm.org.DH’15, May 18–20, 2015, Florence, Italy.Copyright c© 2015 ACM 978-1-4503-3492-1/15/05 ...$15.00.http://dx.doi.org/10.1145/2750511.2750528.

1. INTRODUCTIONWith nowadays’ increasing changes in the food sector and

lifestyle, many people are facing the problem of making bet-ter and healthier food choices [10]. A typical daily activityis to prepare and cook a meal. In order to decide what tocook, people tend to select recipes that satisfy their per-sonal taste. However, because of the extensive growth offood varieties, making food choices by manually exploring afull recipe catalog can be time-consuming and frustrating.We believe that there is a need for decision-aid systems thatcan suggest and recommend personalized food options whiletaking into account the user’s preferences and eating history.

Recommender systems (RSs) are information search andfiltering tools that help users to make better choices whilesearching for products or services such as movies, vacations,or electronic products [21, 15]. The fundamental goal of aRS is to reduce the information overload and to provide per-sonalized suggestions that can assist users’ decision making.RSs are playing an important role throughout the Internet.They have been applied in a large number of Internet appli-cations such as Amazon, YouTube, Netflix, Yahoo, Tripad-visor, Last.fm, and IMDB [13]. Moreover, social networkssuch as LinkedIn and Facebook have also introduced recom-mendation technologies to suggest groups to join, people tofollow or posts to like [1].

Among the various recommender systems application do-mains that have been explored in last years, food is anemerging one and it is attracting a considerable researchfocus. The problem is that food, for many people, has be-come a black box and is associated with bad eating habitsand poor healthy conditions. For that reason, some foodrecommender systems have been proposed.

However, most of the existing applications provide justgeneric food advices that are not tailored to user’s specifictastes or poorly match them. One of the best recent foodrecommender system is described in [11]. This is a mealplanner system that provides personalized recommendationsusing a content-based recommendation technology. We haveconjectured that the accuracy of the system predicted user’spreferences (ratings) can be improved by better modelling

105

and acquiring user preferences, i.e., by letting the users, withtags, to signal what are in their opinion, the most importantingredients and features of the recipes. We also focussedon the overall system usability, and tried to improve it bydesigning an effective human-computer interaction.

This paper therefore aims at filling a research gap byproposing a mobile food recommender system that is easyto use and offers high-quality personalized food recommen-dations. To this end, we summarize our main contributionsas follows.

• We show how to incorporate user selected tags, whichdescribe important attributes of food, in a recommenderalgorithm based on matrix factorization, i.e., whichuses latent factors for modelling both users and recipes.

• We describe how to exploit this algorithm, which isable to predict the rating that a user will give to anot yet rated item, by designing a complete human-computer interaction in a tablet-based application.

• We show that the proposed solution offers personal-ized food recommendations with high prediction accu-racy, i.e., the recommended food recipes receive highevaluations from the users. In fact, our proposed al-gorithm significantly outperforms state-of-the-art al-gorithms, such as that described in [11], in terms ofrating prediction error (MAE and RMSE).

• We show that in a food recommender system, manag-ing user assigned tags can become an important suc-cess factor that can lead to a significant prediction ac-curacy improvement.

We have developed an Android-based food recommendersystem that is focused on supporting a user who wants tochoose what to cook at home. This system can be appliedto choose a recipe that the user is not necessarily cookingby herself. We believe that the developed system will havean important social and economic impact. In fact, moreand more users would like to interact with food applicationson mobile or embedded devices. For example, it is moreconvenient to read the cooking instructions from a tablet,which can be easily integrated into many cooking appliances,rather than using a traditional computer. Our applicationis implemented in a mobile setting, it supports a novel inter-action design, and can be further developed and extendedto cover a full range of user requirements related to foodmanagement and cooking. In our previous work [5], we haveintroduced the research goals and the general features of oursystem. In this paper, we describe the implemented ratingprediction algorithm in detail and we report the results ofan offline and online evaluation of our system.

The remaining of the paper is organized as follows. Thenext section reviews recent research works that are relatedto recommender system and existing food applications. Af-terwards, in Section 3 we describe our novel system HCIand system prototype. In section 4 we describe our pro-posed algorithm in detail. In section 5.2 we describe theoffline comparison of the recommendation algorithm withother competing algorithms and the analysis of the full sys-tem prototype in a live user study. Finally section 6 con-cludes the paper by discussing the evaluation results andoutlining future work.

2. RELATED WORKRecommender systems tackle the problem of generating

personalized suggestions for digital content, products or ser-vices, that suit the user’s needs and constraints better thanthe popular or the mainstream products [3]. In every daylife, getting recommendations from others by words, refer-ence letters, media reports, or travel guides, are commonpractices. Recommender systems may enhance this processby helping people to explore or search for available items,such as, books, articles, webpages, movies, music, or evenjokes. Recommender systems suggest to users items thatare judged to be desirable based on the users’ preferences.They turn out to be essential applications in e-commerceand information retrieval, helping users to find those itemsthat are most suitable for their needs and tastes by reduc-ing large space of options. Such a task can be very difficultwhen there are thousands of items available to people forwatching, reading, visiting, traveling, or buying.

The core computation of a recommender system is theprediction of the user evaluation, typically in the form ofratings, for the items that the user has not evaluated yet.These predictions are computed by using machine learningalgorithms that leverage a data set of already collected users’evaluations for a sample of the domain items. When a targetuser approaches a RS, for all the items in the cataloguethe system predicts the ratings that the user is expected toassign to the items and recommends the items that have thelargest predicted ratings [21].

The most popular recommendation techniques are Content-Based, Collaborative Filtering, and Hybrid [22]. Systems us-ing Content-Based techniques suggest items of interest basedon their associated features [18]. For instance, a content-based news recommender considers the terms appearing inthe news articles as distinctive features and recommend newsarticles that have the features possessed by the articles thatthe user preferred before. Systems adopting CollaborativeFiltering techniques predict a target user’s ratings by onlyusing the ratings that the various users of the system gave,and then recommends the items with the highest predictedratings [16, 4]. Hybrid RSs combine two or more techniques,e.g., collaborative and content-based, in order to cope withthe specific limitations of a single technique.

Focusing on collaborative filtering, nowadays, the mostpopular rating prediction technique is matrix factorization[16]. Fundamentally, in matrix factorization a given matrixof numerical ratings (e.g., between 1 to 5) of users for items{rui}m×n (m users and n items) is decomposed in two lowerdimensional matrices, so that a rating prediction for all theunknown entries in the original matrix {rui} can be com-puted [16]. In matrix factorization both users and items aremodelled with vectors of abstract factors that are learnedby mining the available ratings. For this reason matrix fac-torization algorithms are also called “factor models.” In afactor model, because users and items share a common rep-resentation, it is possible to establish the similarity of usersand items. These similarities and the relationships betweenusers and items are exploited for predicting missing ratingsand generating recommendations.

Matrix factorization originated in the context of the Net-flix prize1 around 2007, and have since then attracted a lot ofattention from the research community as well as the indus-

1http://www.netflixprize.com/

106

try. As a result, numerous variants have been developed.Some of them exploit additional data in order to improvethe prediction accuracy achieved by only relying on users’ratings. One of the most popular variant is SVD++ [16],which considers user provided implicit feedback such as thebrowsing history of the user, or the user’s purchased items.The use of implicit feedback proved to be valuable for theprediction algorithm, and was eventually part of the winningsolution to the Netflix competition.

Along this line, recent works have also successfully inte-grated explicit content metadata into matrix factorization.For instance, Enrich et al. [8] modified the SVD++ algo-rithm to introduce tagging information instead of implicitfeedback. They achieved higher accuracy in the cold-start,which is a situation that arises when there is not enoughuser preference data to provide the user with accurate sug-gestions. Manzato [19] proposed gSVD++, an extension ofSVD++ in which, besides the user’s implicit feedback, theitem model is also enhanced with attribute factors. Similarlyto [8], Fernandez-Tobıas and Cantador [9] adapted gSVD++to separately integrate user’s and item’s tags into the fac-torization model, further improving the performance of thesystem in the cross-domain recommendation setting. In thiswork, we build on the algorithm presented in [9], based onthe hypothesis that the tags assigned by the users to recipescan provide additional useful information about their pref-erences and therefore, if used in the rating prediction algo-rithm, can lead to a better system performance.

Food is an important and emerging application domain forRSs. Here the elements of interest are the food ingredients,which can be combined in recipes [10] and menus [17], or thenutritional concerns [12, 26]. In general, ingredient informa-tion are of primary importance and must strongly influencethe recommendation output. Food RSs can make person-alized recommendations for food items such as recipes, dietplans, meals or ingredients [25, 10]. In this paper, we focuson a food RS that suggests recipes that the user can cook.It exploits explicit food preferences, expressed in the formof ratings and tags regarding what user has eaten or cooked.to train the recommendation model [10].

In this domain there are some important elements to beconsidered when generating recommendation such as: thecombination or sequencing of food recommendations, theways of preparing [10], as well as the nutritional utility ofa particular food for a user or a group. This makes it al-most impossible to build a recommender that can considerall these important factors, hence a focus has to be defined,by only considering very few significant aspects of the overallproblem [11].

In this paper, we focus on a recommender system that canidentify recipes that the users will like. So far, food recom-mender systems have been developed using a variety of rec-ommendation techniques [23, 24, 10]. Freyne and Berkovsky[11] proposed a content-based recommender (CBFB), whichsuggests to the user recipes based on user’s preferred recipes.In this system first recipe preferences are converted into in-gredient preferences: as one recipe is typically constructedby a set of ingredients, a user’s rating for a recipe can betransferred to a rating for the ingredients in this recipe. Ifone ingredient appears in different recipes, an average of theuser’s assigned recipes’ ratings can be estimated to be therating of the user for that ingredient. Thus each ingredi-ent can be associated with a score that is obtained from the

user’s ratings for the recipes. In order to predict the rat-ing of a certain recipe, the scores of the ingredients in thisrecipe are averaged while considering all the ingredients tobe equally important. We considered this technique as abaseline solutions, and therefore we compared it with ourrecommender in the evaluation section.

3. SYSTEM INTERACTION AND PROTO-TYPE

In this section, we will first describe the main steps ofthe human-computer interaction that is supported by ourproposed recommender system and then will illustrate thedetails of the implemented system prototype.

The supported human-computer interaction is a funda-mental aspect of a recommender system. We have designed acomplete interaction process for collecting user preferences,in the form of ratings and tags, and presenting the recom-mendations to the user [5]. The user-system interaction in-cludes the following steps:

1. General Preference Elicitation: where the user ratesand tags recipes.

2. Session-Related Preference Elicitation: where the userdefines the important ingredients that she will like touse, and therefore should be included in the recom-mended recipe.

3. Recommendation: where the user is presented withrecommendations and can browse them.

Figure 1: Screen for locating the recipes that theuser cooks

We have then developed an Android-based food recom-mender system prototype that supports our interaction de-sign. The system is focused on supporting a user that wantto choose what to cook at home. We focus here on this sce-nario even though the system can also be applied to choose

107

a recipe that the user is not necessarily cooking by herself.The first step is general preference elicitation. The goal ofthis step is to collect the long term (stable) preferences ofthe users [20], i.e., what she generally likes to cook (or eat).This step includes two stages: (1) where the system asks theuser to specify the recipes she cooks at home and (2) wherethe user gives her ratings and tags to the recipes she cooked.

After the user logs in the system, the user can browse thefull catalog of recipes and mark those that she has experienceof. This step is required to quickly identify the recipes thatthe user can actually evaluate. Different categories of recipessuch as fish or salads are presented to the user (Figure 1).Users can easily navigate through these categories to finda detailed recipe, for example, Beef -> Roasted Beef ->Roasted Beef with Salad.

Inside each category, there is a list of recipes related to thiscategory. The system will then ask the user to specify whichrecipes she has experienced. The user can mark her familiarrecipes as “Eaten” or “Cooked” by clicking the option box(Figure 2). “Eaten” and “Cooked” options are currentlytreated by the system in the same way. In the future weplan to exploit the information that the user only eat somedish without cooking it.

Figure 2: Screen for marking the recipes that userexperienced

After the user has marked the recipes she has experienced,some recipes are presented to the user for rating and tagging.In fact, for the system to acquire some knowledge of theuser preferences the user is asked to rate and tag recipesselected among those that she has declared to have eaten orcooked before. But, the system needs also to explore a bitmore the user’s preferences and it presents additional recipesfor the user to rate. These additional recipes are found byguessing what the user may have experienced but has notyet marked in the first step. In order to find such recipesactive learning techniques, i.e., and in particular, the BinaryPrediction technique is used [6, 7].

In Figure 3 it is shown the interface where the user isasked to rate and tag the system identified recipes. Therating interface uses a classical 5-star Likert scale (Figure3). In addition, the users is requested to “explain” the coremotivations for an assigned rating by adding to the recipesome tags. In the supported interaction the system givestwo tagging possibilities to the user: (a) adding some of theingredients as tags, (b) adding any other tag that the userdeems as relevant to the recipe. Users can decide to tag therecipe with some of the suggested tags, or add their own tags.A list of tags (normally 5-12 tags) will be suggested to theusers when they are tagging the recipe. Tags are suggestedwhen they are evaluated by the system as relevant to therecipe. For example, the system can suggest ingredientsof the recipe, or key words from the recipe title, or somepopular tags like spicy or Asian style that were assigned byother users to this recipe.

When the user is tagging a recipe she must also indicatewhether there is a positive or negative attitude with theselected tags. For example, in Figure 3 the tag olive oilis coloured in green because the user believes that this is apositive aspect of the recipe, while garlic is coloured in redbecause she does not like garlic in this recipe. In principlethe rating and tagging only need to be done once when theuser uses the system for the first time. But the user canalways access this section to rate additional recipes. Webelieve that all these different types of tags should be used tobuild the rating prediction model because they bring preciseinformation about the user food preferences that may moredifficult to acquire exploiting only the ratings.

Figure 3: Rate and tag interface

In the session-related preference elicitation step, the useris supposed to be looking for a recipe recommendation andcan therefore enter the ingredient (e.g., tuna fish) or featureof the recipe that she wants to cook. This can be done byselecting a tag from the list of suggested tags (Figure 4).

Finally, in the recommendation step, by exploiting thecollected user preferences (long term and session specific),

108

Figure 4: Screen for choosing the distinguished fea-ture of the recipe that the user wants to cook

the system generates recommendations that are labeled bythe tag selected in the previous step. The system presents,one by one, a set of recipe recommendations to the userand asks the user to choose one. The user is supposed tobrowse the first recommendation and eventually to go tothe next one if she is not satisfied. When the user choosesone recommended recipe, the system presents its cookinginstructions as shown in Figure 5.

4. RECOMMENDATION ALGORITHMIn this section we describe the core algorithm that is used

to predict the individual users’ ratings for different recipes.In order to suggest useful recipes to the user, as we have

described above, the system first needs to collect informationabout her preferences. Either during the general preferenceelicitation step, or after a particular recipe is tried, the userhas the option to tell the system if the recipe fits her tastes.In our system, these user’s preferences are collected and rep-resented as ratings, i.e., numerical evaluations ranging from1 to 5, and the goal of the recommendation algorithm is topredict the ratings for recipes that have not been rated yetand the user may like to cook. The recipes with the high-est predicted ratings are more likely to be relevant and aretherefore suggested to the user. The recommended recipescan be those that have never been tried by the user or thosethat are worth to be cooked again.

In this work, we have implemented an algorithm that iscapable of exploiting tagging information in the rating pre-diction process [9]. This algorithm builds upon the popularmatrix factorization method [16], in which each user u isassociated with a parameters’ vector pu ∈ Rk that modelsher latent features. Likewise, each recipe m is modeled by avector qm ∈ Rk that contains its latent features. In matrixfactorization, ratings are estimated by computing the dotproduct of the vectors: rum = pT

uqm.

Figure 5: Recipe ingredients and cooking instruc-tions

The algorithm proposed in [9] extends matrix factoriza-tion by including additional parameters used for modellingthe dependencies between assigned tags and ratings. As wehave illustrated previously, in addition to ratings, in our sys-tem users can also specify their preferences by means of tags,and the tags’ valences (positive or negative), assigned to therecipes. For instance, a user may use the tag spicy to tellthe system she prefers that type of recipes. Similarly, anitem can be described with the tag tomato if this is one ofits ingredients.

We conjectured that tags assigned to an item are corre-lated to the ratings, and can be used to better predict user’sinterests. Thus, we have decided to adopt the rating predic-tion model described in [9] and have introduced additionalfeature vectors xt ∈ Rk and ys ∈ Rk for user’s and recipe’stags, respectively. Ratings are now estimated as follows:

rum =

(pu +

1

|Tu|∑t∈Tu

xt

)T (qm +

1

|Tm|∑

s∈Tm

ys

)(1)

where, Tu is the set of tags assigned by the user u to anyrecipe, and Tm is the set of tags assigned to the recipe mby any user. If Tu = ∅ or Tm = ∅ then we ignore the corre-sponding terms and the model reduces to standard matrixfactorization.

We note that by introducing latent feature vectors forusers and recipes separately, the algorithm is able to com-pute predictions even if the user did not assign any tag tothe recipe, hence the usage of tags does not impose that acertain number of tags are actually acquired.

Moreover, when incorporating the tags into the model, inthe current version of the system, we have used only thepositive tags from the user. This makes easier to isolate theeffect of the tags on the items’ ratings. In a future workwe will identify a viable technique for exploiting also thenegative tags.

109

We show in the following the minimisation problem thatmust to be solved in order to determine the model’s param-eters:

(2)

minp∗,q∗,x∗,y∗

∑(u,m) ∈R

(rum − rum)2

+ λ

(‖pu‖2 + ‖qm‖2 +

∑t∈Tu

‖xt‖2 +∑

s∈Tm

‖ys‖2)

where R is the training set, and the parameter λ ∈ R con-trols the complexity of the model; it is used to preventoverfitting. The parameters pu,qm,xt,ys are automaticallylearned during the training phase of the algorithm, by min-imizing the regularized squared prediction error. In our ex-periments, we found λ = 0.02 to be a good trade-off. Theprevious minimization problem can be efficiently solved us-ing stochastic gradient descent, as depicted in Algorithm 1.

Algorithm 1 Recommendation model training step

procedure TrainStepfor (u,m) ∈ R doeum ← rum − rum . Compute rum using eq. 1. Simultaneously update the parameter vectors:

pu ← pu − α[λpu − eum

(qm + |Tm|−1∑

s∈Tmys

)]qm ← qm − α

[λqm − eum

(pu + |Tu|−1∑

t∈Tuxt

)]for all t ∈ Tu do

xt ← xt−α[λxt − eum

|Tu|

(qm + |Tm|−1∑

s∈Tmys

)]end forfor all s ∈ Tm do

ys ← ys − α[λys − eum

|Tm|

(pu + |Tu|−1∑

t∈Tuxt

)]end for

end forend procedure

A single training step iterates over the whole training setupdating the model parameters each time. This procedureis typically repeated for a fixed number of iterations, or untilconvergence is reached. The parameter α controls to whatextent the parameters are updated on each step: a too smallvalue can make the learning process very slow, whereas withlarge values the algorithm may fail to converge. In our ex-periments, we found good results using α = 2 · 10−4, and asingle training iteration. We also found that a value of k = 5for the dimensionality of the latent vectors offered a goodtrade-off between prediction accuracy and training time.

Once the training phase is complete, we use the learnedparameters to compute rating predictions using the equa-tion 1. In order to provide the target user with recommenda-tions, the system first estimates her ratings for the unknownrecipes, sorts them from the highest to the lowest predictedvalue, and presents the top ranked items.

5. EVALUATIONWe have asked some users to test our system. The rat-

ings and tags collected from these users were then used toconduct an offline 5-fold cross validation. When using ourapplication, users also evaluated the usability of our devel-oped prototype. The results from the offline evaluation anduser study are reported in this section.

A total of 20 subjects have participated to the evaluationexperiment. The subjects are composed of 60% male and40% female in the range of 23-50 years old. The subjects’occupations vary from college students, researchers, com-pany employees as well as nutritionists. They are from avariety of countries such as France, Germany, China, USA,UK, Morocco, Persia and Italy.

Each subject used a tablet to test the system. The sys-tem is running in the Android v.4.4.3 operating system.The tablet that we used is a mobile device manufacturedby ASUS, model is “Fonepad 7 - ME372CG”. Hardwarespecification of the device are as follows: 7 inch 1280x800IPS display, Atom Z2560 Dual-Core 1.6 GHz, RAM 1Gb,network standard DC-HSPA, WCDMA, EDGE/GSM andWLAN802.11 a/b/g/n.

5.1 Offline EvaluationIn the offline evaluation, We have conducted a 5-fold cross

validation to compare the proposed rating prediction algo-rithm, which we call MF-T, and the content-based approach(CBFB) presented in [11], which we have briefly describedin the related work section. We have used a 5-fold crossvalidation, which is normally used in RSs research. In thisvalidation procedure, the full rating data set is split into fivedisjoint subsets of the same size. Then, the system predic-tion model is trained using four of these five subsets and thepredictions for the ratings in the fifth subset are comparedwith the true ratings found in that subset (test set). Thistraining and test procedure is repeated 5 times, each timewith a different test set. And finally the system error mea-sured on the five iterations is averaged in order to producea final measure.

In total we collected 113 recipes. All of them are real-world recipes listed in Total Wellbeing Diet [11]. We initiallycollected a total of 1888 tags derived from ingredients andInternet resources. Ingredient tags are simply the names ofthe ingredients. Other tags were crawled from the Instagramwebsite, where we found tags to images of the consideredrecipes. Furthermore, we manually discarded tags that bringalmost no information, such as food or hmmmm, asn othernormalization steps were applied, such as stemming. Assuch, we obtained 850 distinct tags associated to the recipes,on average 7.5 tags per recipe. The most popular tags areingredients such as chicken, salad and beef. From the userswho tried the system, we obtained 137 ratings and 176 tags,on average 6.85 ratings and 8.8 tags per user.

Mean Absolute Error (MAE) and Root Mean Squared Er-ror (RMSE) were used to evaluate the accuracy of the ratingprediction algorithms. They measures the average absolutedeviation between a predicted rating and the user’s true rat-ing. Lower MAE or RMSE means higher prediction accu-racy. Their definitions are as follows:

MAE =1

|T |∑

(u,i)∈T

|rui − rui| (3)

RMSE =

√√√√ 1

|T |∑

(u,i)∈T

(rui − rui)2 (4)

where T is a test set of user and item pairs. rui is the modelpredicted rating and rui is the true user ratings, which weobtained in the user study.

110

In Table 1, we can see that in each fold of the 5-fold crossvalidation procedure the MAE of MF-T, our matrix fac-torization algorithm extended with tags, is lower than thecontent based approach (CBFB) proposed in [11] (p<0.01).Similar results were obtained for RMSE (see Table 2).

Table 1: MAE of the considered rating predictionalgorithm

MAE MF-T CBFB Standard MFFold 1 0.702 1.559 1.122Fold 2 0.678 1.503 1.200Fold 3 0.733 1.393 1.052Fold 4 0.546 1.464 0.854Fold 5 0.772 1.025 0.860Mean 0.686 1.389 1.018

Table 2: RMSE of the considered rating predictionalgorithm

RMSE MF-T CBFB Standard MFFold 1 0.882 2.241 1.617Fold 2 0.855 2.052 1.566Fold 3 0.885 1.957 1.505Fold 4 0.717 2.001 1.272Fold 5 0.899 1.624 1.123Mean 0.848 1.975 1.416

In order to explain the observed good performance of MF-T, we compared it with standard Matrix Factorization, i.e.,when no tags information is used in the prediction algorithm.In Table 1 and 2 we can see that the standard Matrix Fac-torization performs slightly better than CBFB. But we alsoobserve that our extended matrix factorization algorithmwith tags outperforms standard Matrix Factorization. Thissuggests that the good performance of our algorithm is basedon the exploitation of the users’ preferences brought by thetags data.

In fact, the prediction model can learn that a user tends toprefer items that are tagged with certain positive tags andcan extrapolate that other items that are similarly taggedby the user will be also preferred. To illustrate this idea, werefer to one particular user’s tagging behavior, that we havefound in our data set. This user has added beef as a positivetag to one recipe. Then, in two other recipes that containthe beef ingredient the user again marked the recipes withthe positive tag beef. This clearly signals a tendency of theuser to like recipes containing beef.

We also note that our algorithm can be considered an hy-brid method, since it uses both ratings and recipe features,in the form of tags. We conjecture that our algorithm has abetter performance than the pure content-based model thatwe have considered because the last method uses indiscrim-inately all the recipe’s features when estimating its ratings.Conversely, our algorithm identifies the tags that are moreimportant for rating prediction, hence performing an im-plicit feature selection process. In the future work, we willconduct further experiments to determine the most infor-mative content features and we will double check if a CBalgorithm that only exploits these most informative featurescan compete with the tag-based solution proposed here.

5.2 User StudyIn order to evaluate the usability of the developed pro-

totype we adopted the System Usability Scale (SUS) from[2]. SUS is a widely accepted measurement for evaluatingsystem usability. It is composed of 10 standard questions.Each question is answered with values in a Likert scale rang-ing from 1 to 5 (1 means Strongly Disagree and 5 meansStrongly Agree). The replies are then converted from theLikert scale into scores ranging between 0 and 4, summedand normalised to 100. As [2] states, a SUS score of 68 canbe considered as a benchmark of (average) system usability.

The 20 users, whose demographical information have beendescribed in Section , have also evaluated the system usabil-ity by filling the SUS questionnaire. We found that for 70%of the users the system scored a SUS value larger than 68.This indicates that most of the users believe that the systemhas a large usability. The average SUS score among the 20subjects is 75.5 (margin of error is 6.01), which is larger thanthe SUS benchmark score of 68, indicating that the usabilityof our system is perceived as high.

In addition we measured the perceived recommendationquality of our system. The questionnaire that we used tomeasure the perceived recommendation quality is adoptedfrom [14], which has 7 questions and a highest score of 4 foreach of them. Hence, the maximum value is 28. Calculatingthe overall score of the perceived recommendation qualityfor the system, we obtained 19 out of 28. On average theuser assigned the“agree” level, which indicates that the usersagreed that the recommendations were well-chosen and alsosuit well their preferences.

6. CONCLUSIONIn this paper we have proposed a novel food recommender

system that generates recipe recommendations using an ex-tension of Matrix Factorization. The system in additionto ratings collects users’ tags. The system was developedfor a tablet-based platform, and implements a well designedhuman-computer interaction.

We have carried out an offline evaluation to compare theproposed recommendation algorithm with a state of the artcontent-based algorithm proposed in [11]. Our algorithmsignificantly outperforms the other algorithm. In order tobetter understand what causes the significant improvement,we further compared our solution with a standard MatrixFactorization algorithm and found that the usage of tags isthe important factor that leads to the observed rating pre-diction improvement. Furthermore, our user study showsthat our system prototype has a high utility, the recommen-dations are well-chosen and of high quality.

As future work, we plan to consider nutritional factorssuch as calories and proteins to really generate health-awarerecommendations. We want to generate recommendationsthat enable the user to correctly balance the intake of nutri-tional elements. Also, in order to improve our recommenderalgorithm, we will incorporate the negative tags into the rat-ing prediction model, we will consider the problem of gen-erating sequentially coherent recommendations, so to avoidthat the system offers the same recommendation two daysin a row. In addition to only offering our application toindividual user, we will develop a new recommendation al-gorithm and HCI design so that it can be used in the contextof a group of users, e.g., a family.

111

7. REFERENCES[1] N. Baghaei, S. Kimani, J. Freyne, E. Brindal,

S. Berkovsky, and G. Smith. Engaging families inlifestyle changes through social networking.International Journal of Human-ComputerInteraction, 27(10):971–990, 2011.

[2] J. Brooke. Sus: a quick and dirty usability scale. InJordan, editor, Usability Evaluation in Industry.London: Taylor and Francis, 1996.

[3] R. Burke. Knowledge-based recommender systems.Encyclopaedia of Library and Information Systems,69(32), 2000.

[4] C. Desrosiers and G. Karypis. A comprehensive surveyof neighborhood-based recommendation methods. InF. Ricci, L. Rokach, B. Shapira, and P. B. Kantor,editors, Recommender Systems Handbook, pages107–144. Springer, 2011.

[5] M. Elahi, M. Ge, F. Ricci, D. Massimo, andS. Berkovsky. Interactive food recommendation forgroups. In Poster Proceedings of the 8th ACMConference on Recommender Systems, RecSys 2014,Foster City, Silicon Valley, CA, USA, October 6-10,2014. 2014.

[6] M. Elahi, F. Ricci, and N. Rubens. Active learningstrategies for rating elicitation in collaborativefiltering: A system-wide perspective. ACMTransactions on Intelligent Systems and Technology,5(1):13, 2013.

[7] M. Elahi, F. Ricci, and N. Rubens. Active learning incollaborative filtering recommender systems. InProceedings of the 14th International Conference onE-Commerce and Web Technologies, pages 113–124.Springer, 2014.

[8] M. Enrich, M. Braunhofer, and F. Ricci. Cold-startmanagement with cross-domain collaborative filteringand tags. In Proceedings of the 13th InternationalConference on E-Commerce and Web Technologies,pages 101–112. Springer, 2013.

[9] I. Fernandez-Tobıas and I. Cantador. Exploiting socialtags in matrix factorization models for cross-domaincollaborative filtering. In Proceedings of the 1stWorkshop on New Trends in Content-basedRecommender Systems, Foster City, California, USA,pages 34–41, 2014.

[10] J. Freyne and S. Berkovsky. Intelligent food planning:personalized recipe recommendation. In Proceedings ofthe 15th International Conference on Intelligent UserInterfaces, pages 321–324. ACM, 2010.

[11] J. Freyne and S. Berkovsky. Evaluating recommendersystems for supportive technologies. In User Modelingand Adaptation for Daily Routines, pages 195–217.Springer, 2013.

[12] G. Geleijnse, P. Nachtigall, P. van Kaam, andL. Wijgergangs. A personalized recipe advice systemto promote healthful choices. In Proceedings of the16th International Conference on Intelligent UserInterfaces, pages 437–438. ACM, 2011.

[13] D. Jannach, M. Zanker, A. Felfernig, and G. Friedrich.Recommender Systems: An Introduction. CambridgeUniversity Press, 2010.

[14] B. P. Knijnenburg, M. C. Willemsen, Z. Gantner,H. Soncu, and C. Newell. Explaining the user

experience of recommender systems. User Modelingand User-Adapted Interaction, 22(4-5):441–504, 2012.

[15] J. A. Konstan and J. Riedl. Recommender systems:From algorithms to user experience. User Modelingand User-Adapted Interaction, 22(1-2):101–123, Apr.2012.

[16] Y. Koren and R. Bell. Advances in collaborativefiltering. In F. Ricci, L. Rokach, B. Shapira, andP. Kantor, editors, Recommender Systems Handbook,pages 145–186. Springer Verlag, 2011.

[17] F.-F. Kuo, C.-T. Li, M.-K. Shan, and S.-Y. Lee.Intelligent menu planning: Recommending set ofrecipes by ingredients. In Proceedings of the ACMMultimedia 2012 Workshop on Multimedia forCooking and Eating Activities, CEA ’12, pages 1–6,New York, NY, USA, 2012. ACM.

[18] P. Lops, M. de Gemmis, and G. Semeraro.Content-based recommender systems: State of the artand trends. In F. Ricci, L. Rokach, B. Shapira, andP. Kantor, editors, Recommender Systems Handbook,pages 73–105. Springer, 2011.

[19] M. G. Manzato. gsvd++: supporting implicitfeedback on recommender systems with metadataawareness. In Proceedings of the 28th Annual ACMSymposium on Applied Computing, SAC ’13,Coimbra, Portugal, pages 908–913, 2013.

[20] A. M. Rashid, I. Albert, D. Cosley, S. K. Lam, S. M.Mcnee, J. A. Konstan, and J. Riedl. Getting to knowyou: Learning new user preferences in recommendersystems. In Proceedings of the 2002 InternationalConference on Intelligent User Interfaces, IUI 2002,pages 127–134. ACM Press, 2002.

[21] F. Ricci. Recommender systems: Models andtechniques. In R. Alhajj and J. G. Rokne, editors,Encyclopedia of Social Network Analysis and Mining,pages 1511–1522. Springer, 2014.

[22] F. Ricci, L. Rokach, B. Shapira, and P. B. Kantor.Recommender Systems Handbook. Springer, 2011.

[23] J. Sobecki, E. Babiak, and M. S lanina. Application ofhybrid recommendation in web-based cookingassistant. In Knowledge-Based Intelligent Informationand Engineering Systems, pages 797–804, 2006.

[24] M. Svensson, J. Laaksolahti, K. Hook, and A. Waern.A recipe based on-line food store. In Proceedings ofthe 5th International Conference on Intelligent UserInterfaces, pages 260–263. ACM, 2000.

[25] C.-Y. Teng, Y.-R. Lin, and L. A. Adamic. Reciperecommendation using ingredient networks. InProceedings of the 3rd Annual ACM Web ScienceConference, pages 298–307. ACM, 2012.

[26] Y. van Pinxteren, G. Geleijnse, and P. Kamsteeg.Deriving a recipe similarity measure for recommendinghealthful meals. In Proceedings of the 16thInternational Conference on Intelligent UserInterfaces, pages 105–114. ACM, 2011.

112