Learning User Preferences in Non-Stationary Environments

Wasim Huleihel, Soumyabrata Pal, Ofer Shayevitz

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Recommendation systems often use online collaborative filtering (CF) algorithms to identify items a given user likes over time, based on ratings that this user and a large number of other users have provided in the past. This problem has been studied extensively when users’ preferences do not change over time (static case); an assumption that is often violated in practical settings. In this paper, we introduce a novel model for online non-stationary recommendation systems which allows for temporal uncertainties in the users’ preferences. For this model, we propose a user-based CF algorithm, and provide a theoretical analysis of its achievable reward. Compared to related non-stationary multi-armed bandit literature, the main fundamental difficulty in our model lies in the fact that variations in the preferences of a certain user may affect the recommendations for other users severely. We also test our algorithm over real-world datasets, showing its effectiveness in real-world applications. One of the main surprising observations in our experiments is the fact our algorithm outperforms other static algorithms even when preferences do not change over time. This hints toward the general conclusion that in practice, dynamic algorithms, such as the one we propose, might be beneficial even in stationary environments.
Original languageEnglish
Title of host publicationProceedings of The 24th International Conference on Artificial Intelligence and Statistics
EditorsArindam Banerjee, Kenji Fukumizu
Number of pages9
StatePublished - 1 Jan 2021
Event24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021 - Virtual
Duration: 13 Apr 202115 Apr 2021
Conference number: 24

Publication series

NameProceedings of Machine Learning Research


Conference24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021
Internet address


Dive into the research topics of 'Learning User Preferences in Non-Stationary Environments'. Together they form a unique fingerprint.

Cite this