Serendipity-aware noise detection for recommender systems

Abstract

With the rise of the Internet, recommender systems are becoming a used solution to solve the information overload problem. These systems are like any other system, subject to noise: malicious noise that is caused by attacks and natural noise that is due to the human error. Many detection algorithms solve the noise problem. However, since natural noise and serendipity overlap in their definition, removing noise results in removing serendipity. Serendipity that is the happy surprise of finding something relevant unexpectedly is important for the issue of over personalization caused by the recommender system. Looking at it, this seems a cycle. The aim of this study is to solve the over personalization problem without damaging the user’s trust to the system. Multiple objectives are targeted to attain this aim. This work shows that existing datasets mirroring the user ratings and interactions contain noise and serendipity. Since the desired result is to protect user’s trust without leaving him/her in an echo room or filter bubble, the system needs to eliminate noise without affecting serendipity. A serendipity-aware noise detection algorithm is designed to differentiate between noise and serendipity. To measure the success of the proposed algorithm, a new metric is used: Top-N modified.