Language differences in online complaint responses between generative artificial intelligence and hotel managers

Abstract

Since November 2022, the use of generative artificial intelligence (GAI) technology has increased in many customer service industries. However, little is known about AI’s language choices and meaning-making resources compared to human responses from a systematic linguistic point of view. The present study is a discourse analysis that explores negative online guest complaints made to four luxury heritage hotels in Hong Kong that are classified as cultural heritage sites with rich interpersonal and historical values. We collected authentic guest complaints and responses from hotel managers from April 2012 to October 2022 in online travel forums, and then had GAI draft response letters on behalf of the hotel managers. Our total dataset was 65,539 words and consisted of three subcorpora: guest complaints (Text a of 115 complaints totaling 26,224 words), hotel manager responses (Text b of 115 response letters totaling 14,975 words), and AI-generated responses (Text c of 115 response letters totaling 24,340 words). This study used systemic functional linguistics to explore interpersonal meanings in texts; for example, appraisal resources, verb processes, and personal pronouns were compared between texts. First, we identified the most frequent words of the common themes across the three subcorpora and found significant differences in lexicogrammatical features between hotel managers and AI-generated responses using the log-likelihood ratio. The results suggest that AI-generated texts are able to provide a tailored and empathetic response to guests, but hotel managers may need to introduce some modifications, such as time indicators, sensory verbs used, and complimentary offers. This study explores the differences in word choices and communication strategies, which have implications and insights for the hospitality industry, especially luxury heritage hotels where caring and personalized customer service are considered important.