QPP-RA: Aggregating Large Language Model Rankings

Recent advances in Large Language Models (LLMs) have significantly improved search and recommendation systems. In the field of information retrieval, LLMs are increasingly used as re-rankers to refine the relevance of documents initially retrieved by search algorithms. It is also a common strategy to apply ranking aggregation techniques to improve the performance of the search algorithm. Existing aggregation methods, such as Borda Count and Reciprocal Rank Fusion (RRF), rely exclusively on the positional information of the retrieved documents and do not take into account the different performance levels of the different models, potentially degrading the overall effectiveness. To overcome this limitation, we propose a new rank aggregation algorithm called Query Performance Prediction Rank Aggregation (QPP-RA). Our method uses historical model performance data in three different datasets to improve the aggregation process. Our results show that QPP-RA significantly improves the quality and efficiency of search results, significantly outperforming both the single best model and all other aggregation models.