Tralalam Project – Translating with Large Language Models
Trained on multimodal gigacorpora, language models (LLMs) can be used for a variety of purposes. One possible purpose is machine translation, a task for which the LLM-based approach provides a simple answer to two difficult points:
- support for an extended and enriched context (including translation examples or bilingual terminological entries) ;
- handling translation domains and directions for which learning data is sparese or even non-existent.
The Tralalam project, accepted under the ANR 2023 call on very large language models (LLM), is positioned at the crossroads of artificial intelligence, linguistics and machine translation.
Led by Systran, the Tralalam project also involves ISIR at Sorbonne University and the ALMAnaCH project-team at the Inria center in Paris. This synergy between the private sector and academic research reinforces the quality and diversity of the skills deployed to achieve the project’s objectives.
Project description by François Yvon, project member at ISIR.
Challenges and prospects of large language models for machine translation
The TraLaLaM project aims to explore the use of large language models (LLMs) for machine translation, by asking two main questions:
- in what scenarios can contextual information be used effectively via prompts?
- for low-resource scenarios (with a focus on dialects and regional languages), can LLMs be trained effectively without any parallel data?
In-depth analysis of these language models
The main objective of the project is to analyze in depth the relevance of LLMs .
On the one hand, we will focus on industrial use cases by studying scenarios for domain adaptation, terminology data or translation memories, which correspond to realistic situations. On the other hand, we will focus on the realization of a machine translation system from and into all the languages of France, based on a massively monolingual LLM trained with little (if any) parallel data.
Significant scientific challenges lie ahead, such as extending pre-trained models to new, poorly endowed languages, or handling highly idiomatic texts featuring numerous instances of code switching between a minority language and French.
Evaluation of models and solutions for the languages of France
From an industrial point of view, Tralalam aims to evaluate the computational costs and trade-offs VS performance induced by the use of LLMs in machine translation. These new architectures could profoundly transform the way translation systems are trained and operationally deployed. Current solutions however, are either to costly to deploy, or underperform dedicated machine translation engines that are optimized for this task.
With regard to the languages of France, in partnership with various players representing the linguistic communities concerned, we aim to develop operational solutions for certain well-targeted applications, such as the translation of Wikipedia pages, administrative or regulatory texts, etc.
The Tralalam project is a natural fit with ISIR, and more specifically with the MLIA team’s “Large Language Models” theme. This team is interested in the study and development of language models for applications in automatic language processing, information retrieval, image processing and robotics – with the possibility of combining input and output signals (text+audio, text+image, text+actions, etc.).
Scientific contact: François Yvon, CNRS Senior Director
