Beyond Words: Can ChatGPT support state-of-the-art Recommender Systems?

Large Language Models (LLMs) have recently demonstrated impressive capabilities across a range of natural language processing tasks. Notably, ChatGPT has exhibited superior performance in numerous tasks, particularly under zero and few-shot prompting conditions. Motivated by these successes, the Recommender Systems (RS) and Information Retrieval research communities have started exploring the potential applications of ChatGPT in recommendation and information filtering scenarios. This study investigates the performance of ChatGPT-3.5 and ChatGPT-4 in recommendation tasks under zero-shot conditions, employing a role-playing prompt. We specifically analyze the models’ ability to re-rank recommendations in three domains: movies, music, and books. Our experiments indicate that ChatGPT excels in re-ranking tasks, providing high-quality recommendations. Furthermore, we measure the similarity between ChatGPT’s recommendations and those generated by other recommendation systems, offering insights into how ChatGPT can be positioned within the RS landscape.