How Language Models Translate Responses in Mandarin Conversation

Abstract

A speaker who says “okay” and then asks a question may be checking how to carry out an agreement. An English translation can preserve both the acknowledgement and the question yet change what the speaker is asking about. This study examines how ChatGPT and DeepSeek translated responses from Mandarin conversation transcripts, with and without the preceding dialogue. Two bilingual judges consulted the Chinese conversations and found most translations consistent with the original response. The clearest changes concerned who was speaking and what a question was meant to confirm. In one exchange, a caller checked which number to dial; the English wording instead appeared to ask whether a phone call was correct. Other apparent changes were harder to establish because the Chinese and English each allowed more than one reading. For translation reviewers, the useful question is what specific wording warrants a correction. Tracing a disputed phrase to its speaker and to the action under discussion can identify a consequential change even when the English sounds plausible.