ImpReSS: Designing and Evaluating a Lightweight Implicit Recommender System in Conversational Support Agents

Large language model (LLM)-powered AI agents have transformed customer support, yet little research has addressed the integration of product recommendations into problem-solving dialogues. We introduce ImpReSS, a lightweight implicit recommender system for conversational support agents based on small language and embedding models, making it suitable for on-premise deployment where data privacy is critical. Unlike traditional conversational recommender systems (CRSs), ImpReSS does not assume purchasing intent. Instead, it identifies relevant solution product categories (SPCs) from the conversational context to assist in problem resolution. Our offline evaluation on three real-world datasets demonstrates strong performance, achieving an MRR@1 of up to 0.477 and outperforming five competing methods, including a state-of-the-art CRS. Algorithmic relevance alone is insufficient for effective adoption. A controlled user study with 144 participants shows that the perceived naturalness of recommendations depends strongly on their delivery. Conventional UI patterns such as pop-ups were rated as more appropriate than in-conversation insertions. Optimal timing varied by context, suggesting that recommendations should adapt dynamically to user needs. Thematic analysis of participant feedback further highlights a need for greater user agency, including the ability to interact with, question, and explore alternatives. We present the first comprehensive study of integrating implicitly-inferred recommendations in support dialogues. Our findings highlight the challenges of balancing accuracy with interaction design and yield empirically grounded implications for integrating recommender systems into conversational support agents.