Evaluating Recommender System Stability with Influence-Guided Fuzzing
Authors:David Shriver, Sebastian Elbaum, Matthew B. Dwyer, David S. Rosenblum
Recommender systems help users to find products or services they may like when lacking personal experience or facing an overwhelming set of choices. Since unstable recommendations can lead to distrust, loss of profits, and a poor user experience, it is important to test recommender system stability. In this work, we present an approach based on inferred models of influence that underlie recommender systems to guide the generation of dataset modifications to assess a recommender's stability. We implement our approach and evaluate it on several recommender algorithms using the MovieLens dataset. We find that influence-guided fuzzing can effectively find small sets of modifications that cause significantly more instability than random approaches.
Recommended citation: David Shriver, Sebastian Elbaum, Matthew B. Dwyer, David S. Rosenblum. 2019. Evaluating Recommender System Stability with Influence-Guided Fuzzing. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019. 4934-4942. https://doi.org/10.1609/aaai.v33i01.33014934