{"id":10169,"date":"2024-01-16T15:56:34","date_gmt":"2024-01-16T14:56:34","guid":{"rendered":"https:\/\/www.isir.upmc.fr\/?post_type=isir_news&#038;p=10169"},"modified":"2024-01-17T10:36:34","modified_gmt":"2024-01-17T09:36:34","slug":"tralalam-project-exploring-large-language-models-for-machine-translation","status":"publish","type":"isir_news","link":"https:\/\/www.isir.upmc.fr\/news\/tralalam-project-exploring-large-language-models-for-machine-translation\/?lang=en","title":{"rendered":"TRALALAM PROJECT: EXPLORING LARGE LANGUAGE MODELS FOR MACHINE TRANSLATION"},"content":{"rendered":"\n<h4 class=\"wp-block-heading\"><strong>Tralalam Project &#8211;&nbsp;<em>Translating with Large Language Models<\/em><\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Trained on multimodal gigacorpora, language models (LLMs) can be used for a variety of purposes. One possible purpose is machine translation, a task for which the LLM-based approach provides a simple answer to two difficult points:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>support for an extended and enriched context (including translation examples or bilingual terminological entries) ;<\/li>\n\n\n\n<li>&nbsp;handling translation domains and directions for which learning data is sparese or even non-existent.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The Tralalam project, accepted under the ANR 2023 call on very large language models (LLM), is positioned at the crossroads of artificial intelligence, linguistics and machine translation.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Led by <a href=\"https:\/\/www.systransoft.com\/fr\/\" target=\"_blank\" rel=\"noreferrer noopener\">Systran<\/a>, the Tralalam project also involves ISIR at <a href=\"https:\/\/www.sorbonne-universite.fr\" target=\"_blank\" rel=\"noreferrer noopener\">Sorbonne University<\/a> and the <a href=\"https:\/\/www.inria.fr\/fr\/almanach\" target=\"_blank\" rel=\"noreferrer noopener\">ALMAnaCH<\/a> project-team at <a href=\"https:\/\/www.inria.fr\/fr\/centre-inria-de-paris\" target=\"_blank\" rel=\"noreferrer noopener\">the Inria center in Paris<\/a>. This synergy between the private sector and academic research reinforces the quality and diversity of the skills deployed to achieve the project&rsquo;s objectives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><em>Project description by Fran\u00e7ois Yvon, project member at ISIR.<\/em><\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Challenges and prospects of large language models for machine translation<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">The TraLaLaM project aims to explore the use of large language models (LLMs) for machine translation, by asking two main questions:&nbsp;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>in what scenarios can contextual information be used effectively via prompts?<\/li>\n\n\n\n<li>for low-resource scenarios (with a focus on dialects and regional languages), can LLMs be trained effectively without any parallel data?<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>In-depth analysis of these language models<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">The main objective of the project is to analyze in depth the relevance of LLMs .<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">On the one hand, we will focus on industrial use cases by studying scenarios for domain adaptation, terminology data or translation memories, which correspond to realistic situations. On the other hand, we will focus on the realization of a machine translation system from and into all the languages of France, based on a massively monolingual LLM trained with little (if any) parallel data.&nbsp;<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Significant scientific challenges lie ahead, such as extending pre-trained models to new, poorly endowed languages, or handling highly idiomatic texts featuring numerous instances of code switching between a minority language and French.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Evaluation of models and solutions for the languages of France<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">From an industrial point of view, Tralalam aims to evaluate the computational costs and trade-offs VS performance induced by the use of LLMs in machine translation. These new architectures could profoundly transform the way translation systems are trained and operationally deployed. Current solutions however, are either to costly to deploy, or underperform dedicated machine translation engines that are optimized for this task.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">With regard to the languages of France, in partnership with various players representing the linguistic communities concerned, we aim to develop operational solutions for certain well-targeted applications, such as the translation of Wikipedia pages, administrative or regulatory texts, etc.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"681\" src=\"https:\/\/www.isir.upmc.fr\/wp-content\/uploads\/2024\/01\/projet-Tralalam-image2-1024x681.png\" alt=\"\" class=\"wp-image-10165\" style=\"width:558px;height:auto\" srcset=\"https:\/\/www.isir.upmc.fr\/wp-content\/uploads\/2024\/01\/projet-Tralalam-image2-1024x681.png 1024w, https:\/\/www.isir.upmc.fr\/wp-content\/uploads\/2024\/01\/projet-Tralalam-image2-300x200.png 300w, https:\/\/www.isir.upmc.fr\/wp-content\/uploads\/2024\/01\/projet-Tralalam-image2-768x511.png 768w, https:\/\/www.isir.upmc.fr\/wp-content\/uploads\/2024\/01\/projet-Tralalam-image2.png 1048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The Tralalam project is a natural fit with ISIR, and more specifically with the MLIA team&rsquo;s \u00ab\u00a0Large Language Models\u00a0\u00bb theme. This team is interested in the study and development of language models for applications in automatic language processing, information retrieval, image processing and robotics &#8211; with the possibility of combining input and output signals (text+audio, text+image, text+actions, etc.).&nbsp;<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Scientific contact<\/strong>: Fran\u00e7ois Yvon, CNRS Senior Director<\/p>\n","protected":false},"featured_media":10168,"template":"","isir_news_categories":[252],"class_list":["post-10169","isir_news","type-isir_news","status-publish","has-post-thumbnail","hentry","isir_news_categories-research"],"_links":{"self":[{"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/isir_news\/10169","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/isir_news"}],"about":[{"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/types\/isir_news"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/media\/10168"}],"wp:attachment":[{"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/media?parent=10169"}],"wp:term":[{"taxonomy":"isir_news_categories","embeddable":true,"href":"https:\/\/www.isir.upmc.fr\/wp-json\/wp\/v2\/isir_news_categories?post=10169"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}