Are all rejected recommendations equally bad? Towards analysing rejected recommendations

Shir Frumerman, Guy Shani, Bracha Shapira, Oren Sar Shalom

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

When evaluating algorithms that recommend a list of relevant items to a user, it is common to use metrics such as precision to measure the system accuracy. When computing precision, one computes the number of items that were selected by the user among the recommended items. As such, recommended items that were not selected by the user, which we call rejected recommendations, are all considered to be bad recommendations, resulting in no increase to the system accuracy metric. Our ultimate goal is to develop a new recommendation accuracy evaluation metric, which may assign some value to the rejected recommendations. In this paper, as a first step, we claim that some rejected recommendations are better than others. Specifically, we consider items that are similar to the item that was finally selected, as better recommendations than items that bear little similarity. We conduct a user study, showing that rejected recommendations that have high content or collaborative similarity to the selected item are perceived by users as better recommendations than items with low similarity. In addition, we study the correlations between the recommended items shown to a user and the un-recommended items that the user has selected in a real-life job posting dataset. We show that when considering item similarity rather than simple precision, the correlations are much higher. This may be attributed to the influence of the recommended items on the decisions of the user.

Original languageEnglish
Title of host publicationACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization
PublisherAssociation for Computing Machinery, Inc
Pages157-165
Number of pages9
ISBN (Electronic)9781450360210
DOIs
StatePublished - 7 Jun 2019
Event27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019 - Larnaca, Cyprus
Duration: 9 Jun 201912 Jun 2019

Publication series

NameACM UMAP 2019 - Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization

Conference

Conference27th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2019
Country/TerritoryCyprus
CityLarnaca
Period9/06/1912/06/19

Fingerprint

Dive into the research topics of 'Are all rejected recommendations equally bad? Towards analysing rejected recommendations'. Together they form a unique fingerprint.

Cite this