<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yoav Navon on Medium]]></title>
        <description><![CDATA[Stories by Yoav Navon on Medium]]></description>
        <link>https://medium.com/@yoavnavon?source=rss-e187f164e9db------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*OqFZnjKv0GVijGP9.</url>
            <title>Stories by Yoav Navon on Medium</title>
            <link>https://medium.com/@yoavnavon?source=rss-e187f164e9db------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 09:00:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yoavnavon/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Effect of Explanations and Algorithmic Accuracy on Visual Recommender Systems of Artistic…]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/the-effect-of-explanations-and-algorithmic-accuracy-on-visual-recommender-systems-of-artistic-b8445e1f453c?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8445e1f453c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sun, 06 Oct 2019 20:38:21 GMT</pubDate>
            <atom:updated>2019-10-06T20:38:21.340Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wDx1YkejVjFteI07" /><figcaption>Photo by <a href="https://unsplash.com/@christianfregnan?utm_source=medium&amp;utm_medium=referral">Christian Fregnan</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>The Effect of Explanations and Algorithmic Accuracy on Visual Recommender Systems of Artistic Images</h3><h4>Vicente Dominguez, Pablo Messina, Ivania Donoso-Guzmán, Denis Parra</h4><p>This paper aims to evaluate the usefulness of explanations for a recommender system of pieces of art. The study was conducted with people from the Mechanical Turk platform, and different variations of the system were presented to them.</p><p>The first variation was the recommender system used, being all content-based. The authors used one model based on a Deep Neural Network to extract features, and also used Attractiveness Visual Features (AVF) as a second option. The other variation was the explanations presented to the users, having 3 different interfaces. The first interface consisted in recommendations for the user without any explanations, for the second one the user was given similar images as explanations, and the third one only applied for AVF, and consisted on a bar chart of the features.</p><p>As it’s expected, the perception of explicability was higher with the interfaces with explanations. So was the case for Trust, Relevance and Diverity ratings. The authors noted that the algorithm used for recommendation was also important, because the DNN model achieved significantly better ratings.</p><p>One question that I have about the research, is that for the DNN model, the interface 2 and 3 is exactly the same, giving to the user a top-3 similar view of the item recommended. Nevertheless, for some of the ratings there was a significant difference between interfaces (Diversity, Interface Satisfaction), and given that they are the same it shouldn’t be the case. I suspect that this is because of the sample size of the study, because only 41 and 39 people participated in interface 2 and 3 respectively.</p><p>Another question is about the differences between algorithms in interface 3. It is not clear if the differences were because of the model, or because of the bar chart visualization for AVF. Probably is a mix between the two, but measuring this would be interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8445e1f453c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/the-effect-of-explanations-and-algorithmic-accuracy-on-visual-recommender-systems-of-artistic-b8445e1f453c">The Effect of Explanations and Algorithmic Accuracy on Visual Recommender Systems of Artistic…</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[BPR: Bayesian Personalized Ranking from Implicit Feedback]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/bpr-bayesian-personalized-ranking-from-implicit-feedback-5eb2835113de?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/5eb2835113de</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sun, 08 Sep 2019 21:07:27 GMT</pubDate>
            <atom:updated>2019-09-08T21:07:27.896Z</atom:updated>
            <content:encoded><![CDATA[<h4>Steffen Rendle, Christoph Freudenthaler, Zeno Gantner and Lars Schmidt-Thieme</h4><p>The paper introduces an optimization criterion designed to provide good rankings: BPR-Opt, and a generic learning algorithm, based on SGD to optimize with respect to BPR-Opt. The framework is model agnostic, so diverse methods are implementable like Item/User-KNN and Matrix Factorization techniques.</p><p>The method consists in ranking pairs of items, in opposition to previous methods designed to predict a rating or score for individual items. It is assumed that watched items are more desirable than unwatched items, so a matrix of preferences is computed for every user, shown in figure 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X1LArB7u9v7c3jHhF7QuFQ.png" /><figcaption>Figure 1: Training data for BPR</figcaption></figure><p>Finally, the ranking task is solved using a Bayesian formulation of the problem. The idea is to maximize the posterior probability <em>p(</em>𝛩|&lt;), by computing the likelihood <em>p(&lt;|</em>𝛩) and prior probability <em>p(</em>𝛩).</p><p>I find it really good the argument presented in the paper against the usual methodologies. If you give the items with feedback a positive label (e.g. 1), and all others a negative label (e.g. 0), and you fit this data, there is a subjacent problem. A model with enough expressiveness would be able to fit the data perfectly, and thus would predict 0 for every unseen item. This way, regularization is the reason the method works. Nevertheless, regularization should be more of a way of improving a model and not the only reason for its success.</p><p>One concern that I have for the method is that given a training set of size I·U, it is necessary to create a new training set of I·I·U. How does this escalate? For datasets with millions of items, the memory usage is critical.</p><p>A possible problem of the approach presented in the paper is that we are assuming that all items are equally comparable. For example, if we consider the moment when the feedback is recorded, it is not the same to use a fresh data point or one that was recorded a few years ago, so the timeframe matters.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5eb2835113de" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/bpr-bayesian-personalized-ranking-from-implicit-feedback-5eb2835113de">BPR: Bayesian Personalized Ranking from Implicit Feedback</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deep Neural Networks for YouTube Recommendations]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/deep-neural-networks-for-youtube-recommendations-646209cfa7e9?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/646209cfa7e9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sat, 07 Sep 2019 23:46:52 GMT</pubDate>
            <atom:updated>2019-09-07T23:46:52.094Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MlFQey9MFPBTV0w5" /><figcaption>Photo by <a href="https://unsplash.com/@christianw?utm_source=medium&amp;utm_medium=referral">Christian Wiediger</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Paul Covington, Jay Adams, Emre Sargin</h4><p>This paper presents the recommendation system used in youtube, based on a deep learning network. The model consists of two parts: first, a candidate generation network reduces de possible videos from millions to a couple of hundreds. It uses watch and search history, as well as some context information. After the candidate generator returns the videos, a ranking network computes the predicted watch time of every video extract a subset of videos. The ranking network has access to more detailed information about each video. The full model is differentiable end-to-end and is trained used a held-out method, in which a fraction of a user is presented and the “future” is held out.</p><p>The model performs extreme multiclass classification, between millions of videos. I wonder if could be possible first to predict the topic of the video, and then perform the multiclass classification. This would follow a top-down approach. Although, this method would need a model to predict the topic or the use of tags.</p><p>The authors say that they feed the age of the training example as a feature during training, but what age is this? Because the training example consists of multiple videos that the user has watched, the age is the average? or some other way of computing it.</p><p>The ranking network is supposed to predict watch time, but I think that is necessary to normalize by some factor. This is because watching 1 minute of a 1.5-minute video should be a hit, but watching 1 minute from a 30-minute video should be categorized as a negative sample, because the user didn’t like the video.</p><p>The authors say that feature engineering is still necessary, and the main challenge is in representing a temporal sequence of user actions, and how these actions relate to the user. This problem could call for the use of an RNN, to encode the sequential watch history of users.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=646209cfa7e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/deep-neural-networks-for-youtube-recommendations-646209cfa7e9">Deep Neural Networks for YouTube Recommendations</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Content-based artwork recommendation: integrating painting metadata with neural and…]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/content-based-artwork-recommendation-integrating-painting-metadata-with-neural-and-1e6c7d9c627?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e6c7d9c627</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[convolutional-network]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sun, 01 Sep 2019 18:01:48 GMT</pubDate>
            <atom:updated>2019-09-01T18:04:37.776Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*h6HI11jaB0KwwirJ" /><figcaption>Photo by <a href="https://unsplash.com/@villxsmil?utm_source=medium&amp;utm_medium=referral">Luis Villasmil</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Content-based artwork recommendation: integrating painting metadata with neural and manually-engineered visual features</h3><h4>Pablo Messina, Vicente Dominguez, Denis Parra, Christoph Trattner, Alvaro Soto</h4><p>This paper discusses different ways to perform content-based recommendation, in the context of artwork pieces in an online website. These paintings are one-of-a-kind, so collaborative filtering it’s not an option. Different methods are explored to perform the recommendation, such as <em>Most Popular Curated Attribute Value, Favorite Artist, </em>and visual features: extracted from a CNN, and more traditional methods like LBP. The methods are compared with different metrics and are also combined to create hybrid systems.</p><p>Would have been possible to perform fine-tuning in the CNN used to extract features? They could have used the pre-trained network, and train it with the artwork dataset, trying to predict the attributes from the metadata. Maybe this way the net would be better suited to extract relevant features from the paintings.</p><p>I find it interesting the use of LBP to extract features. The traditional LBP method needs a grayscale image, and I think that any art buyer would tell you that the color is one of the most important attributes of an artwork. Interestingly, LBP was a helpful feature in the hybrid methods, with a 14.5% ponderation, so this indicates that there are other strong aspects of a painting besides the color.</p><p>I wonder why did they use expert curators to validate the recommendations of the system, when the final users are not experts, but normal people looking for art. Moreover, these experts were all part of UGallery, so at certain level, they root in favor of the success of the system. This produces a little bias, and the recommendations could be a bit better than what they are to a neutral person. The use of only 8 experts didn’t help either, as more samples from different people could have helped in reducing this little bias.</p><p>I find it great that in the paper, accuracy was not the only metric used, and diversity and coverage were also considered. Nevertheless, in the use of coverage they computed <em>user coverage </em>and <em>session coverage, </em>when probably for a store, the most important it’s <em>item coverage. </em>I think so, because a store doesn’t want some paintings to never be recommended, and you should try to recommend from every artist.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e6c7d9c627" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/content-based-artwork-recommendation-integrating-painting-metadata-with-neural-and-1e6c7d9c627">Content-based artwork recommendation: integrating painting metadata with neural and…</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Document Clustering Based On Non-negative Matrix Factorization]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/document-clustering-based-on-non-negative-matrix-factorization-83a9f4d19ce9?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/83a9f4d19ce9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Thu, 29 Aug 2019 19:04:51 GMT</pubDate>
            <atom:updated>2019-08-29T19:04:51.112Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*85Ut772l6e-DWYml" /><figcaption>Photo by <a href="https://unsplash.com/@rvignes?utm_source=medium&amp;utm_medium=referral">Romain Vignes</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Wei Xu, Xin Liu, Yihong Gong</h4><p>In the paper, it’s presented a document partitioning method based on the non-negative factorization of the term-document matrix of a given document corpus. This way they construct a topic model, where each axis of the space derived captures a certain topic. Each document is represented as an additive combination of the base topics, and the axes of the semantic space are not necessarily orthogonal.</p><p>I find that is ingenious to take advantage of the non-negative domain, to create this method of non-negative matrix factorization.</p><p>I have a question about the non-orthogonal basis. It is mention that each document takes only non-negative values in all the latent semantic directions. Then how would it be possible to represent the point marked in figure 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/346/1*mUQlWJdl6EXOpAEz2kXszQ.png" /><figcaption>Firgure 1</figcaption></figure><p>This picture is taken from the paper, where the arrows represent the basis vectors of the new space. We can see that there is no positive linear combination of the black arrows to create the blue dot, so it is not clear to me how is this document represented.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=83a9f4d19ce9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/document-clustering-based-on-non-negative-matrix-factorization-83a9f4d19ce9">Document Clustering Based On Non-negative Matrix Factorization</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Content-Based Recommendation Systems]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/content-based-recommendation-systems-62a1d89c9740?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/62a1d89c9740</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Thu, 29 Aug 2019 02:51:46 GMT</pubDate>
            <atom:updated>2019-08-29T02:51:46.471Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-y7YvC6jyTutngBW" /><figcaption>Photo by <a href="https://unsplash.com/@ugmonk?utm_source=medium&amp;utm_medium=referral">Jeff Sheldon</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Michael J. Pazzani and Daniel Billsus</h4><p>This paper discusses content-based recommendation systems in general. It explains how to represent items with structured data, and how to generate structured data from non-structured. For example, for natural language text, the tf-idf method it’s discussed. Afterward, it discusses two approaches to keep a user profile, with a model or a history of the user’s interaction. Finally, different popular models are presented as ways to create a content-based recommender system. Decision trees, KNN, Relevance Feedback, Linear Methods, and Naïve Bayes are proposed as classification learners.</p><p>The methodology for content-based recommender systems is explained as follows:</p><blockquote>… [the models] learn a function that models each user’s interests. Given a new item and the user model, the function predicts whether the user would be interested in the item…</blockquote><p>Isn’t that inefficient? To provide recommendations for a user, with this approach the system would pass all items through the model to see if they are a match or not for the user. Some pre-computation seems necessary, as the number of items can be really large.</p><p>For all the models presented, there is no scalable way of adding new items for the users, the only way would be to recompute the model all over again from scratch. You would like for example your Decision Tree to adapt to the new items that the user likes.</p><p>It’s surprising that the Naive Bayes method does well as the authors claim. That&#39;s because, for text, the algorithm assumes that all words are independent of each other, and that is of course not true, as some words are extremely more probable if they are present after another particular word. Maybe some way to address this would be to group all words that tend to appear together (with some clustering method maybe), and use only one as representant of the group.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62a1d89c9740" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/content-based-recommendation-systems-62a1d89c9740">Content-Based Recommendation Systems</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Performance of Recommender Algorithms on Top-N Recommendation Tasks]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/performance-of-recommender-algorithms-on-top-n-recommendation-tasks-c27aba5e391e?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/c27aba5e391e</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[evaluation-metric]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Mon, 26 Aug 2019 21:55:22 GMT</pubDate>
            <atom:updated>2019-08-26T21:55:22.403Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dfIKyXCMv_AUBWmv" /><figcaption>Photo by <a href="https://unsplash.com/@chrisliverani?utm_source=medium&amp;utm_medium=referral">Chris Liverani</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Paolo Cremonesi, Yehuda Koren, Roberto Turrin</h4><p>This paper explores the performance of diverse recommender systems in the top-N setting. It is shown that algorithms optimized for minimizing an error metric (e.g. RMSE) do not perform as well on this kind of tasks. The analysis is conducted of different kind of algorithms; not personalized (Most Popular, Average Rating), neighborhood-based (itemKNN) and latent factor models. The authors analyze two versions of the same datasets: using the full set of data, and using the “long tail”, which is the data without the top rating items. It is shown that the most-popular method does better than some algorithms, but in the “long tail” dataset, its performance drops significantly.</p><p>The methodology used consisted of using items rated 5-stars, combined with a random sample of additional 1000 items that were considered not relevant, and were not yet rated. Then the model outputs a set of N recommended items, and if the 5-star item is present, it is considered as a hit. Precision and Recall is computed from the average of repeating this methodology. It is worth questioning how good is the assumption that the 1000 items are not relevant. Maybe could be possible to sample these items with a more elaborate heuristic, maybe based on some neighborhood method to reduce the probability of good items slipping in the 1000 items set.</p><p>The analysis was conducted in two datasets from a similar domain. Would these results remain unchanged for datasets with other characteristics. For example, a scientific paper dataset has less most popular items, while being more sparse, so the skewness of the distribution would be less. This could result in different methods being the most effective.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c27aba5e391e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/performance-of-recommender-algorithms-on-top-n-recommendation-tasks-c27aba5e391e">Performance of Recommender Algorithms on Top-N Recommendation Tasks</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating Recommendation Systems]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/evaluation-recommendation-systems-6035271ddfd7?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/6035271ddfd7</guid>
            <category><![CDATA[evaluation-metric]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sun, 25 Aug 2019 23:24:36 GMT</pubDate>
            <atom:updated>2019-08-29T02:20:44.650Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dz9HHGE3KI2WDMtn" /><figcaption>Photo by <a href="https://unsplash.com/@jcoudriet?utm_source=medium&amp;utm_medium=referral">Jason Coudriet</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Guy Shani and Asela Gunawardana</h4><p>This paper discusses ways to compare different recommender systems. First, it talks about the different evaluation settings (<em>offline, user-studies, online), </em>and the pros and cons of using them to evaluate an algorithm. Then it discusses different properties of recommender systems that are important to consider at the time of selection of a particular method.</p><p><strong>User preference </strong>is proposed as a method for evaluation, conducting studies to assert which models the users report more satisfaction. <strong>Accuracy </strong>is discussed as the most usual way to measure a recommender system, and different metrics are presented, and the different types to measure accuracy (prediction, usage, ranking). <strong>Coverage</strong> is a measure that may go against accuracy, as a system may provide recommendations of high accuracy, but for a limited set of popular items. <strong>Confidence </strong>and <strong>Trust </strong>are discussed as the system trust in its ratings, and the users trust on the system ratings or rankings. <strong>Novelty </strong>and <strong>Serendipity</strong> are two similar concepts that measure the capacity of the system to provide items that the user wasn’t aware of; the main difference is that serendipity measures how surprising is this recommendation. <strong>Diversity </strong>is another measure that goes against accuracy, as a user may like items that are not similar between them. <strong>Utility </strong>is a measure that depends on the domain, as it records how the recommender system helps the business model. <strong>Risk </strong>is something that may be associated with a recommender system, as in certain domains we might like to reduce the risk to the minimum for the high cost of a false positive. <strong>Robustness </strong>measures the stability of the system, especially with respect to malicious users, and <strong>Privacy </strong>is important to retain the users trust in the system, as they provide personal information but don’t want third parties to have access to it. <strong>Adaptivity </strong>is explained as an important property to consider, as the relevant items may change with time, for example. Finally, <strong>Scalability </strong>is presented as the ability to remain functional with the addition of more and more data.</p><p>A question that surges on the part about diversity, is that it is explained that diversity may be desired, but as a global measure of the system. Diversity is a very subjective matter, as users may like a different quantity of diversity on their recommendations, and this aspect should be considered.</p><p>A usual problem of recommender systems with regard to their adaptivity capacity is that users may don’t like their current recommendations, but as they are recommended the same type of things, they keep consuming them. This enters the user in some kind of loop, where it can’t get off. Sometimes it is desirable for a user to report the explicit success of the system, rather than wait for the adaptability methods to kick off and change the content that is being presented.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6035271ddfd7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/evaluation-recommendation-systems-6035271ddfd7">Evaluating Recommendation Systems</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Evaluating Collaborative Filtering Recommender Systems]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/evaluation-collaborative-filtering-recommender-systems-7699cc180752?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/7699cc180752</guid>
            <category><![CDATA[evaluation-metric]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[recommendation-system]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Sat, 24 Aug 2019 22:23:18 GMT</pubDate>
            <atom:updated>2019-08-29T02:21:14.544Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mlvHOI50YrPJlqBO" /><figcaption>Photo by <a href="https://unsplash.com/@helloquence?utm_source=medium&amp;utm_medium=referral">Helloquence</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Evaluation Collaborative Filtering Recommender Systems</h3><h4>Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl.</h4><p>This is a long paper, so I’ll summarize only the most important aspects of it. The discussion revolves around different ways to evaluate a recommender system, and some of the relevant aspects to consider for a good evaluation.</p><p>Comparing two algorithms can be challenging because there is no consense about the most important attributes, and which metrics should be used to measure those attributes. Different algorithms may be better or worse on different data sets, and the goals for which evaluation is performed may differ.</p><p>Different systems may be built for different user tasks, like <em>Find Good Items, Annotation in Context, Recommend Sequence, </em>to name a few. Around this part of the paper there is a disturbing quote:</p><blockquote>Most evaluations of recommender systems focus on the recommendations; however if users don’t rate items, then collaborative filtering recommender systems can’t provide recommendations.</blockquote><p>This quote is clearly forgetting all the implicit feedback methods, where there is no rating whatsoever. However, the author was probably referring to the rating of items, as any kind of interaction with them.</p><p>Another aspect where there is variability in evaluation is with respect to the data sets. For a certain algorithm, can the evaluation be carried out offline, or does it require live user tests? Can evaluation be performed on simulated data? One topic that would have been interesting is if it’s possible to train a recommender system on datasets from more than one domain, for example, multiple movie rating datasets at the same time, with a posterior fine-tuning on the real movie dataset to be used.</p><p>Afterward, there is a long discussion about accuracy metrics, which are the most common kind of metric to evaluate recommender systems. One important aspect to consider is if a given metric measure the effectiveness of a system with respect to the user tasks for which it was designed; if it’s prediction, ranking or other. Different metrics are discussed, like <em>Mean Absolute Error, Precision &amp; Recall, F1, ROC Area, Correlation, Half-life Utility, </em>and <em>NDPM </em>are the most notable. After explaining each one, it compares the metrics on the performance to evaluate one algorithm on one dataset with different parameters. This yielded correlation charts between pairs of metrics, and the authors find that each metric belongs to a certain family of metrics, identifying 3 of those families.</p><p>The correlation between metrics is interesting, but none of the discussion talks about how this metric determines the success of a certain recommender system. Would it be really interesting to evaluate an algorithm with some metrics, and then testing it with real users and measuring the overall satisfaction.</p><p>Finally, the paper talks about how accuracy can’t be the only metric to evaluate a recommender system. <em>Coverage, Learning Rate, Novelty </em>and <em>Serendipity </em>are different aspects to keep an eye on during the design and implementation of a recommender system.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7699cc180752" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/evaluation-collaborative-filtering-recommender-systems-7699cc180752">Evaluating Collaborative Filtering Recommender Systems</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collaborative Filtering for Implicit Feedback Datasets]]></title>
            <link>https://medium.com/recommender-systems-iic-3633/collaborative-filtering-for-implicit-feedback-datasets-46bc64a4c62c?source=rss-e187f164e9db------2</link>
            <guid isPermaLink="false">https://medium.com/p/46bc64a4c62c</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[collaborative-filtering]]></category>
            <category><![CDATA[implicit-feedback]]></category>
            <category><![CDATA[matrix-factorization]]></category>
            <dc:creator><![CDATA[Yoav Navon]]></dc:creator>
            <pubDate>Wed, 21 Aug 2019 13:30:37 GMT</pubDate>
            <atom:updated>2019-08-21T13:30:37.293Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*H3a6zT1aeF1BWCeN" /><figcaption>Photo by <a href="https://unsplash.com/@andrewtneel?utm_source=medium&amp;utm_medium=referral">Andrew Neel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Yifan Hu, Yehuda Koren, Chris Volinsky</h4><p>This paper discusses a latent factor model that uses implicit feedback for training and evaluation. The model uses the implicit feedback to compute binary users preferences <em>p </em>and a certain confidence <em>c </em>in this preference. For example, if a user consumed a certain item, it will have <em>p=1,</em> and a confidence <em>c=1+a·r, </em>where <em>r </em>is the value given by the implicit feedback in the dataset and <em>a </em>is a constant. The goal is the same as in the explicit feedback setting, to find vectors <em>x, y</em> for each user and item respectively. The objective function is shown in Figure 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/745/1*aD0_ylOKo_6dYi2aROyGcg.png" /><figcaption>Figure 1: Objective Function</figcaption></figure><p>The function is similar to previous matrix factorization methods, with the exception that it needs to account for the confidence value and needs to compute all <em>u, i </em>pairs. Because of this last exception, the complexity explodes with SGD, so an alternating-least-squares optimization process is proposed.</p><p>One question that arises, is that it would be helpful to record negative feedback. For example, if a user exits a movie after a short period of time, it would probably mean that he disliked the movie. There are two options for the dataset, either this interaction wasn’t considered because of the short time so <em>r=0, </em>or was considered as a somewhat good thing, with r&gt;0. But the correct connotation would be to have something different that <em>r=0 </em>or <em>r&gt;0, </em>as something we haven’t watched is a better recommendation than something we disliked.</p><p>On the experiments section, it is shown that the model outperforms the Popularity and Neighborhood baselines, an keeps getting better with the number of factors. The authors recommend using the highest number of factors that is computationally feasible. Is this a characteristic of the model? Will always more factors help with the recommendation? There might be a factor number where some of them just apport ambiguity, decreasing the quality of the recommendation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=46bc64a4c62c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/recommender-systems-iic-3633/collaborative-filtering-for-implicit-feedback-datasets-46bc64a4c62c">Collaborative Filtering for Implicit Feedback Datasets</a> was originally published in <a href="https://medium.com/recommender-systems-iic-3633">Recommender Systems</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>