Header

Search

Making Recommender Systems More Explainable: Two Papers at RecSys 2026

How can we evaluate whether explanations provided by recommender systems are actually useful? At RecSys 2026, Kathrin Wardatzky from the Department of Informatics presented two research papers exploring the challenges of evaluating these explanations – from the reliability of AI-based assessments to the representation of users in research studies.

Recommender systems help us discover everything from films and music to products and services. But understanding why a system makes a particular recommendation is not always straightforward. While researchers have developed numerous methods to explain recommendations, evaluating which explanations work best remains a challenge.

At the ACM Conference on Recommender Systems (RecSys 2026), Kathrin Wardatzky from the Department of Informatics presented two research papers addressing different aspects of this challenge.

Can large language models evaluate explanations?

The first paper, “The Utility of LLMs in Recommender Systems Explanation Evaluation”, investigates whether large language models (LLMs) can help evaluate explanations generated by recommender systems.

Evaluating explanations through user studies can be time-consuming, while existing automated methods have limitations. The authors therefore examined whether LLMs could serve as evaluators and compared their assessments with ratings from human participants.

The findings show that LLMs can exhibit rating patterns similar to those of human evaluators, but their agreement on individual ratings remains limited. The results also vary depending on the model size and the criteria being evaluated.

The study offers practical recommendations for using LLMs in explanation evaluation, including the importance of checking explanations for factual accuracy.

Whose perspectives are represented in user studies?

The second paper, “Whom do Explanations Serve?", examines how researchers evaluate explanations with users – and whether the participants in these studies adequately represent the people who use recommender systems.

Through a large-scale literature survey, the authors found that many studies focus on specific user groups whose characteristics may not reflect the broader target audience. Inconsistent reporting of participant information also makes it difficult to reproduce and compare findings.

The paper highlights the need for more inclusive and reproducible evaluation practices.

It was selected by the journal ACM Transactions on Recommender Systems as one of three papers to be presented in a special session at RecSys 2026.

Towards more reliable evaluation

Together, the two studies highlight important challenges in assessing explanations provided by recommender systems. They show why both the reliability of automated evaluation methods and the diversity of participants in user studies matter when developing systems that can explain their recommendations effectively.

Subpages