Episode 14: Can AI treat all languages equally? Students put LLMs to the test
DGT podcast: Languages and TechnologyShow Notes
How can we make sure AI works fairly across Europe’s languages?
In this episode of DGT podcast: Languages & Technology, Anna Holmén speaks with Pilar Sánchez Gijón from the Autonomous University of Barcelona and Susana Valdez from Leiden University about the EU MMLU project.
In this project, European Master’s in Translation students are translating an AI evaluation dataset into the EU official languages. They are gaining practical experience while helping develop a multilingual benchmark for large language models.
We also hear from DG Translation's Anna Kokkinidou, on how DG Translation and the European Master’s in Translation network collaborate on this project and more.
The guests discuss the challenges of adapting AI evaluations to different languages and cultures, the importance of human expertise, and plans to expand the benchmark to all 24 EU official languages.
More on the EU MMLU and the EU institutional LLM: An EU Institutional LLM to prevent digital language extinction - Multilingualism, translation and language-based AI services
More from DGT podcast: Languages and Technology
In this episode, we explore translation, terminology and artificial intelligence (AI), with a particular focus on the Irish language. We discuss how new AI technologies are changing the work of language professionals, and what it means to work with a language that has fewer digital resources than many larger languages.
How does a European Commission document make its way from an initial draft to 24 equally valid language versions?
Transcreation sits where translation meets copywriting, creativity and cultural adaptation. In this episode, Anna Holmén talks to Italian transcreation expert Claudia Benetello about why human nuance, humour, voice and individual flair still matter in a world increasingly shaped by AI – especially in marketing and creative language work.
Multilingualism is a democratic right and a core part of the EU’s identity. Christos Ellinides, Director-General for Translation, talks about what this means in the era of AI – and what we do to safeguard EU language diversity. Why is DG Translation a good place to develop AI tools? What is the role for linguists in all this? And are we really building an LLM of our own? Hear the answers to these questions and many more.
Anja Filko and Erik Oldenburg join Anna Holmén to speak about their unforgettable experience as winners of the translation contest Juvenes Translatores, Anja for Croatia and Erik for Germany. We hear about their journey - from being signed up for the contest by their teachers, to translating a text at the same time as thousands of other young Europeans, and finally receiving their prizes in an award ceremony in Brussels for the 27 winners. The experience profoundly changed their view on what translation is - and on what Europe is.