Systematic evaluation of the operational effectiveness and user experience of generative chatbots in ITSM platforms

Abstract

The accelerated digitalization process has exposed the inadequacy of traditional tool-based IT support. In response, generative chatbots powered by large language models have emerged as a potential solution, although their actual effectiveness remains under debate. This study assessed their impact on ITSM platforms in terms of response time reduction, accuracy, and user satisfaction. A systematic review was conducted following the PRISMA 2020 guidelines, covering studies published between 2021 and 2025 in Scopus, SpringerLink, and Google Scholar. The search strategy incorporated the Boolean operator AND to combine keywords such as chatbot, conversational agent, customer support, customer service, service management systems, virtual assistants, intelligent chatbots, ITSM, operational efficiency, and productivity. Fourteen articles met the eligibility criteria. The analysis showed that chatbots reduced response times by 38 % to 68 %, achieved accuracy levels ranging from 85 % to 97 %, and increased user satisfaction by 12 to 27 percentage points. However, some studies warned that the lack of empathy limits their effectiveness when dealing with complex queries. It is concluded that, although these systems demonstrate promising performance, widespread adoption requires stronger empirical support through standardized metrics and longitudinal studies that reinforce the available evidence.