Note: This blog post was originally written in Japanese for our Japanese website. We use our machine translation platforms to translate and make automatic corrections, and then partially edit to fit the content in English. The original Japanese post can be found here.
As generative AI represented by ChatGPT has become more sophisticated, we are increasingly hearing discussions such as "How will generative AI change society?" and "Will generative AI take away our jobs?"
As mentioned in our previous post of this series "The Surprising Relationship Between ChatGPT and Machine Translation," the progress in generative AI was brought about by machine translation. In particular, the Transformer model, a core technology, began to be used for translation in 2017. This means that the translation industry experienced a shock on the same scale as ChatGPT several years ago.
Therefore, looking at the changes in the translation industry over the last few years may provide clues for considering the questions mentioned at the beginning.
Let's start by getting to know the "enemy." How do the capabilities of current machine translation and generative AI compare to those of humans?
There are several indicators used to measure translation quality, but we often evaluate it from three perspectives: fluency, accuracy, and consistency. These criteria can also be applied to generative AI.
Fluency refers to whether the output consists of natural-sounding sentences. In this regard, just as ChatGPT is often said to be "like talking to a human," we can say that both machine translation and generative AI have reached human-like levels.
Accuracy in translation refers to whether the content of the source text is correctly rendered. For generative AI, this means whether it correctly grasps the meaning of a prompt and provides a complete answer without omissions or additions. In this respect, AI still does not match human performance.
In machine translation based on the Transformer model, mistranslations (meaning of the source text is misunderstood) have decreased, but they have not yet disappeared. On the other hand, we frequently see omissions (information in the source text missing from the translation) and additions (information not in the source text appearing in the translation), which rarely occurred in rule-based machine translation.
Even with generative AI, it can misunderstand the meaning of complex questions and often answers things it wasn't asked. Furthermore, due to the operating principles of current generative AI mechanisms (subword tokenization with Transformer model), they cannot perform logical reasoning or numerical calculations. Therefore, there is no guarantee that they will correctly answer questions involving these.
(Note: This post was initially published in Japanese in summer 2025. The later model versions may have improved their output.)
Consistency includes consistency with surrounding sentences and consistency with extra-linguistic facts. The former evaluates whether the style and terminology are consistent throughout the entire text, while the latter evaluates whether the output contradicts background facts. In fact, AI is no match for humans at all in this regard.
Whether using machine translation or generative AI, there are constraints on the amount of information that can be processed at once, so maintaining consistency across a document of dozens of pages is a tall order. Additionally, the Transformer model has no means of referencing facts outside of the language itself. It is well known that generative AI, in particular, can produce content that contradicts facts, a phenomenon known as hallucination.
As we can see, the capabilities of machine translation are still no match for humans. Machine translation is highly fluent, so the translation may look correct if you only read the output, but you never know where a mistranslation might be hiding. Furthermore, even if a translation is technically correct, it might violate the laws of the target country or infringe upon religious or cultural taboos.
Therefore, for content intended for outbound communication purposes, machine translation is not a substitute for human translation. In particular, for texts intended to move people's hearts, such as literary translation, movie subtitling, or marketing content within industrial translation, there is almost no role for machine translation alone. Such craftsmanship will continue to require a human touch in the future.
Conversely, there is translation for inbound purposes, where you are reading for your own understanding. In this area, machines are displacing humans. Machine translation without any human edit is essentially still close to "junk translation" that cannot be used for outbound communication, but if that level of quality is sufficient, human translators cannot compete in terms of speed and cost.
However, in the case of junk translation, it might be more accurate to say that machines have created new demand rather than taking away human jobs. It is likely that the overall volume of translation is increasing because people are now choosing to translate texts that they previously hesitated to translate due to the time and cost of human translation.
In terms of expected quality, much of industrial translation falls somewhere between craftsmanship and junk translation. In other words, these are translations where mistranslations are unacceptable, but the text does not need to move people's hearts as long as the facts are conveyed. In this field, a form of translation called machine translation with post-editing (MTPE), where human translators revise MT output, is common.
For craftsmanship-level translation, MT output is often useless, so it is faster for a human to translate from scratch than to perform MTPE. However, for documents with many fixed phrases such as operation manuals, MTPE allows us to efficiently produce translations equivalent to human translation. In this context, machine translation is not competing with translators but has become a tool to improve translator productivity.
Like machine translation, generative AI has issues with accuracy and consistency. Therefore, we never recommend using text generated by AI as-is for business output. Such text may contain false information or offensive expressions, and if disseminated, the negative impact on the business could be immeasurable. Furthermore, it has been pointed out that Transformer models may copy learned expressions verbatim, and publishing generative AI output could potentially infringe on copyrights.
When publishing text generated by AI, human fact-checking is still essential. Therefore, generative AI is unlikely to eliminate human jobs for now. In the field of translation, we have seen repeated instances where machine translation was used as-is in situations that required human translation or MTPE, leading to errors that became public scandals in newspapers in Japan. The same will undoubtedly happen with generative AI if we simply replace people with machines.
On the other hand, we can improve work efficiency by using generative AI for your tasks. While many applications are possible, we will introduce two translation-related uses here.
First, by giving instructions such as "Please summarize the following English text concisely in Japanese," you can have translation and summarization performed simultaneously. Since generative AI translation is still a form of machine translation and thus "junk translation," translating the full text is often only useful for getting the gist. If that is the case, we can save human reading time by having the AI output only the main points from the start.
Next, if you provide an instruction like "Please provide a sentence-by-sentence side-by-side translation of the following German text into English and Japanese," it will translate the German into both languages. In many machine translation systems, the training data is overwhelmingly English, so English translations tend to have higher accuracy than Japanese translations. Therefore, if there is a discrepancy between the English and Japanese translations, you can notice that the Japanese translation might be incorrect.
Of course, it is possible that the English translation is also wrong, or even if there is no discrepancy, both translations might share the same error. Nevertheless, compared to simple machine translation from German to Japanese, we can increase the reliability of the translation.
We tried to review machine-translated or human-translated text using the prompt "Please point out any errors in the Japanese translation of the following English-to-Japanese translation." However, there were some cases where the AI identified correct translations as errors and made them worse, or conversely, missed mistranslations, omissions, and typos in the translated text. Just as machine translation falls short of human translators, generative AI falls short of human reviewers.
When using generative AI, please note that some services collect the prompts you enter as training data, so please check the terms of use and settings carefully. This is also true for machine translation services.
For example, in the case of ChatGPT's free web interface, at the time of this writing, the default setting is to store user conversations and use them to train the model. If you enter personal information in Japan, it may be considered illegal as use for purposes other than the original purpose under Japan's Act on the Protection of Personal Information. Furthermore, if you enter sensitive information, it may be used to answer another person's question, which could lead to serious consequences.
Neither machine translation nor generative AI possesses abilities equal to humans, and they are not meant to replace humans. Rather, they are tools for humans to use; while they are helpful if used correctly, they can cause trouble if used incorrectly.
Always keep in mind that the output of machine translation or generative AI, while superficially plausible, can be erroneous. Only humans can guarantee the accuracy of the information being disseminated. We believe that business efficiency can be improved by leaving what machines can do to the machines, while humans focus on what machines cannot do.
With XMAT or our other platforms, you can utilize machine translation and generative AI in a secure environment without worrying about the transparency of the cost. At Kawamura International, we do more than just provide machine translation or generative AI services. We offer a range of suggestions to help make your translation operations run more smoothly. If you are facing translation challenges at the moment or have any questions about our language services, please feel free to contact us.