Investigating confidence displays for top-N recommendations

Recommendation systems often compute fixed-length lists of recommended items to users. Forcing the system to predict a fixed-length list for each user may result in different confidence levels for the computed recommendations. Reporting the system's confidence in its predictions (the recommendation strength) can provide valuable information to users in making their decisions. In this article, we investigate several different displays of a system's confidence to users and conclude that some displays are easier to understand and are favored by most users. We continue to investigate the effect confidence has on users in terms of their perception of the recommendation quality and the user experience with the system. Our studies show that it is not easier for users to identify relevant items when confidence is displayed. Still, users appreciate the displays and trust them when the relevance of items is difficult to establish.