Large language models (LLMs) have changed the game for machine translation (MT). LLMs vary in architecture, ranging from decoder-only designs to encoder-decoder frameworks.

    大语言模型（LLM）改变了机器翻译（MT）的面貌。大语言模型的结构各不相同，既有解码器结构（decoder-only），也有编码器-解码器结构（encoder-decoder）。

Encoder-decoder models, such as Google’s T5 and Meta’s BART, consist of two distinct components: an encoder and a decoder. The encoder processes the input (e.g., a sentence or document) and transforms it into numeral values that represent the meaning and the relationships between words.

编码器-解码器模型，如谷歌的T5模型和Meta的BART模型，由编码器和解码器两个部分组成。编码器处理输入的内容（如一个句子或文档），并将其转换为表示词义和词与词之间关系的数值。

This transformation is important because it allows the model to “understand” the input. Then, the decoder uses the information of the encoder and generates an output, such as a translation of the input sentence in another language or a summary of a document.

这一转换非常重要，因为它能让模型“理解”输入的内容。接着，解码器利用编码器的信息生成输出，比如输入句在另一种语言中的翻译或一份文件的总结。

As Sebastian Raschka, ML and AI researcher, explained, encoder-decoder models “are particularly good at tasks where there is a complex mapping between the input and output sequences and where it is crucial to capture the relationships between the elements in both sequences” — such as translating from one language to another or summarizing long texts.

正如机器学习与人工智能研究者塞巴斯蒂安·拉施卡（Sebastian Raschka）所说，编码器-解码器模型“尤其擅长完成输入和输出序列之间存在复杂映射的任务，以及非常需要捕捉住两个序列中元素之间的关系的任务”——比如，从一种语言翻译成另一种语言或总结长文本。

In contrast, decoder-only models, like OpenAI’s GPT family models, Google’s PaLM, or Meta’s Llama, consist solely of a decoder component. These models generate an output based on the input by predicting the next word or character in a sequence based on the previous words or characters, without the need for a separate encoding step.

相比之下，解码器模型如OpenAI的GPT系列模型、谷歌的PaLM 模型和Meta的Llama模型仅由一个解码器组成。这些模型根据前面的单词或字符预测序列中的下一个单词或字符，从而基于输入生成输出，无需单独编码。

While they may struggle with understanding complex input structures or relationships, as encoder-decoder models do, they are highly capable of generating fluent text. This makes them particularly good at text generation tasks — like completing a sentence or generating a story based on a prompt.

虽然解码器模型可能会像编码器-解码器模型一样，在理解所输入的复杂结构或关系方面遇到困难，但在生成流畅文本方面表现出色。这使其格外擅长生成文本的任务，如根据指令完成一个句子或生成一个故事。

Researchers have explored the strengths and weaknesses of these architectures. A study published on September 12, 2024, evaluated encoder-decoder and decoder-only models in multilingual MT tasks, focusing on Indian regional languages such as Telugu, Tamil, and Malayalam. In this study, mT5, known for its “robust multilingual capability”, was used as the encoder-decoder example, while Llama 2 served as the decoder-only counterpart.

研究人员对解码器架构和编码器-解码器架构的优缺点进行了探究。2024年9月12日发表的一项研究评估了多语种机器翻译任务中的编码器-解码器模型和解码器模型，重点关注泰卢固语、泰米尔语和马拉雅拉姆语等印度地方语言。在这项研究中，以“强大的多语言能力”著称的mT5模型被用作编码器-解码器模型的示例，而Llama 2模型则作为解码器模型的示例。

The results showed that encoder-decoder models generally outperformed their decoder-only counterparts in translation quality and contextual understanding. However, decoder-only models demonstrated significant advantages in computational efficiency and fluency.

研究结果表明，编码器-解码器模型在翻译质量和语境理解方面总体优于解码器模型。但是，解码器模型在计算效率和流畅性方面则具有显著优势。

This led the researchers to conclude that both architectures have distinct strengths, contributing insights into the effectiveness of different model types in the evolving landscape of MT.

研究人员由此得出结论，这两种架构各有所长，有助于深入了解不断发展的机器翻译领域中不同模型的有效性。

The study’s primary goal was “to advance the field of machine translation, contributing valuable insights into the effectiveness of different model architectures,” according to the researchers.

研究人员表示，这项研究的首要目标是“推动机器翻译领域的发展，为了解不同模型架构的有效性提供有价值的见解”。

Yet, other studies suggest that decoder-only models, when properly fine-tuned, can match or even surpass state-of-the-art encoder-decoder systems.

然而，其他研究表明，解码器模型在经过适当微调后，可以与最先进的编码器-解码器模型相媲美，甚至超越它。

Research from 2023 and 2024 highlighted the advantages of the decoder-only structure over the encoder-decoder one. Researchers pointed out that without a separate encoder, decoder-only models are easier to train since they can efficiently process large datasets by directly concatenating documents. Additionally, their unsupervised pre-training approach allows them to leverage readily available training data, unlike encoder-decoder models, which require paired text inputs.

2023年和2024年的研究强调了解码器结构相对于编码器-解码器结构的优势。研究人员指出，解码器模型没有单独的编码器，更容易训练，因为它可以通过直接串联文档来高效处理大型数据集。此外，与需要输入成对文本的编码器-解码器模型不同，无监督预训练方法使解码器模型能够充分利用现成的训练数据。

The researchers of the latter study, published on September 23, 2024, concluded that “the flexibility and the simpler training setup of decoders should make them both more suitable and efficient for most real world applications,” with the decoder-only architecture being “more appropriate to answer the ever-growing demand for iterative, interactive and machine assisted translation workflow.”

在发表于2024年9月23日的另一项研究中，研究人员总结道，“解码器的灵活性和更简单的解码器训练设置将会使其更适合、更高效地用于大多数实际应用”，解码器结构“更能满足对迭代的、交互的、有机器辅助的翻译工作流程日益增长的需求”。

如您喜欢我们的内容，欢迎您点赞、在看、转发，更多问题可后台留言小编哦

    后台留言，小编会尽快回复