The Surprising Relationship Between ChatGPT and Machine Translation

Note: This blog post was originally written in Japanese for our Japanese website. We use our machine translation platforms to translate and make automatic corrections, and then partially edit to fit the content in English. The original Japanese post can be found here.

This past couple of years, generative AI (represented by ChatGPT) has become a hot topic. Generative AI refers to artificial intelligence capable of creating images, text, music, and more. Did you know that many of these generative AI models are based on technologies developed in the field of machine translation? In this post, we will look at generative AI from the perspective of a translation agency.

Two technologies that changed natural language processing

ChatGPT is based on a language model* called GPT (Generative Pre-trained Transformer). This "Transformer" is a technology now used in many generative AI systems, but it was actually developed through advancements in machine translation.

*Language model: A model that represents human speech or writing based on the probability of word occurrence. It can predict the next word when a partial sentence is provided.

Google's Neural Machine Translation (NMT), which was released in 2016 and sent shockwaves through the translation industry, utilized two innovative technologies: sub-word segmentation and attention.

Sub-word segmentation allows for the reduction of unknown words while keeping the vocabulary size manageable by breaking sentences into units (tokens) smaller than words. Additionally, the attention mechanism provides information regarding the relationships between tokens, contributing to improved translation accuracy for long sentences.

Furthermore, in 2017, Google announced the Transformer model, which relies entirely on the attention mechanism, and significantly enhanced the performance of machine translation. Shortly thereafter, major machine translation services, including our Kawamura NMT, shifted to the Transformer model across the board.

Following the success of Transformer and sub-word segmentation in NMT, language models using these technologies began to be developed for other natural language processing* tasks as well. GPT is one such example.

*Natural Language Processing: The process of making machines process the natural languages that humans use in their daily lives.

General-purpose large language models

NMT can be broadly divided into a first half (encoder) that processes the source text and a second half (decoder) that generates the translation. GPT is a language model that performs the same processing as an NMT decoder; when a partial sentence is provided, it can generate the continuation.

To train AI in translation, humans must prepare parallel corpora to serve as training data (supervised learning). On the other hand, for generating continuations, if a vast amount of human-written text is provided, the AI can learn by creating its own problems and answers (self-supervised learning).

A language model that has been trained on a vast amount of text in this way is called a Large Language Model (LLM). In a sense, an LLM "remembers" a large amount of text. It is, so to speak, a pre-trained general-purpose language model.

By using an LLM as a foundation model and fine-tuning* it with training data for specific tasks, high-performance language models can be created. ChatGPT is essentially GPT fine-tuned for chat purposes.

*Fine-tuning: The process of fine-tuning a pre-trained model by retraining it on data for a different task.

It should be noted that what sets ChatGPT apart from conventional chatbots is its use of reinforcement learning called RLHF (Reinforcement Learning from Human Feedback) to prevent the generation of inappropriate sentences, but we won’t go into details in this post.

What ChatGPT can and cannot do

No matter how high its performance, GPT is ultimately just a language model and does not possess knowledge in the same sense as humans. Therefore, even if you ask ChatGPT questions to test its knowledge, the answers are not reliable.

Here is an example. There is an animal called a binturong at the zoo in Fukuoka City, where one of our members at Kawamura International lives. When we asked ChatGPT, "What do binturongs eat?", it gave a different answer every time it generated a response — sometimes saying they eat grass because they are sheep, or that they eat small animals because they are owls — and most of the answers were incorrect.

Furthermore, language models are incapable of logical reasoning. We asked ChatGPT the following question several times, but it never once provided the correct answer.

A: "B is a liar."
B: "C is a liar."
C: "A is honest."
If there is only one liar among them, who is the liar?

(Note: These answers for binturongs and logical reasoning were generated in fall 2024 when this post was initially published. The later model versions have improved their answers. It’s amazing how fast technology is changing.)

As a result, the sentences generated by language models may appear plausible on the surface, but their content can sometimes be logically flawed.

These limitations of generative AI are very similar to those of NMT. Current machine translation is also on par with humans in terms of fluency and generates seemingly plausible translations, but mistranslations, omissions, and additions are often seen, and it does not match human translators in terms of accuracy. Conversely, ChatGPT is suitable for tasks where humans can guarantee the accuracy of the content. Specifically, these include summarizing long texts and correcting grammatical errors. That said, it does not perform these tasks perfectly either.

When the performance of machine translation improved, quite a few people thought that translators' jobs would be taken over by machines. In reality, however, the work of translators has changed, but it has not disappeared. This is because current machine translation has not reached a level where the translated text can be delivered to end users as is.

The same will likely apply to generative AI. Generative AI is sometimes referred to as a "co-pilot." We believe it is best to position humans as the pilot who produces the final output, with generative AI serving as an assistant to humans.

ChatGPT as a machine translation engine

The instructions that humans give to generative AI such as ChatGPT are called prompts. Depending on the prompt, it is also possible to have ChatGPT perform translations.

As a test, let's have ChatGPT translate an introductory text about ChatGPT.

When you ask this, it performs the following English-to-Japanese translation.

Although it mistranslated "challenge," it can be considered a passing grade for machine translation.

(Note: This translation was generated in fall 2024 when this post was initially published. The later model versions have improved their translation output. Again, we’re a witness of this fast-moving technology.)

Unlike general machine translation engines, ChatGPT handles multilingual translation with a single model. While such multilingual translation models are starting to be put into practical use in some machine translation services, conventional bilingual unidirectional translation models still seem to have an advantage for language pairs with a sufficient amount of parallel data.

Furthermore, since ChatGPT is not dedicated solely to translation, a prompt such as "Please translate the following sentence" is required. While this may seem like a disadvantage at first glance, if used to one's advantage, it enables usage not possible with conventional machine translation engines. By changing the prompt, you can change the translation.

For example, let's try specifying the translation for "challenge," which was mistranslated earlier.

From this prompt, the following response was obtained.

This has eliminated the mistranslation. It is impressive how it correctly conjugated "異議を唱える" (igi o tonaeru) as "異議を唱えたり" (igi o tonaetari).

This practice of refining prompts to achieve desired results is called prompt engineering. While additional training and terminology application are methods for customizing machine translation output, prompt engineering might just become the third customization technique.

Summary

We have explained the relationship between ChatGPT and machine translation. Although it was somewhat technical, we hope you enjoyed reading it. Our API service, LDX hub, also offers services utilizing generative AI including ChatGPT. Reach out to us if you are interested.

Kawamura International’s services

With XMAT or our other platforms, you can utilize machine translation and generative AI in a secure environment without worrying about the transparency of the cost. At Kawamura International, we do more than just provide machine translation or generative AI services. We also offer a range of suggestions to help make your translation operations run more smoothly. If you are facing translation challenges at the moment or have any questions about our language services, please feel free to contact us.