WLSP: Benchmarking 101: The road to finding the best-fit AI model for you

Over the last three years, LLM models have hit the market at a record pace. From what we hear online and in the news, it sounds like each model is outperforming the last one. But what does that really mean?

Is performance being tested for your use case? Will each model be just as great at working with your content? How can you find out the answers to these questions—and what can you do about them?

The answer lies in benchmarking. Benchmarking lets you compare how different models handle your content, and how they perform against the outcomes that matter to you.

Benchmarking has been growing more and more common among teams using AI. Since LLMs are built on a wide range of data, they can perform a wide variety of tasks. While that breadth is useful, it can also mean that a given LLM isn’t tuned to perform at the highest level on your key tasks, like translation or post-editing. By benchmarking, you’ll be able to evaluate exactly how well different models complete each of your specific tasks—so you can find the best fit for you!

The process can sound intimidating, but it doesn’t have to be. In this workshop, we’ll show you how to set up benchmarking, including:
- What data to collect and how to prepare it
- How to run and automate evaluations across models
- How to interpret the results and make decisions
- What you can DIY versus when you’ll want a linguist’s expertise

You’ll walk away with a practical approach to choosing the right model(s) for your goals and the confidence to put it into action.

Meet the panelists:

Marina Sánchez Torrón | Senior Linguistic Engineer | Smartling

Marina Sánchez Torrón is a Senior Linguistic Engineer at Smartling with over 20 years of experience in the language industry, having previously worked as a translator, a computational linguist, and a language analyst. She holds a Ph.D. in translation studies from The University of Auckland. Her expertise and research interests revolve around translation quality, UX, and AI.

Olivia Norris | Junior Data Scientist | Smartling

Olivia Norris began her work in localization in 2024 at Smartling, where she is currently a Junior Data Scientist. She holds a Bachelor of Science in Physics and Statistics & Data Science from the University of California, Santa Barbara. Olivia’s research at Smartling has focused on benchmarking LLMs, experimenting with DEI in MT, and developing automated solutions for file format conversion and reassembly. Her areas of expertise include predictive machine learning, prompt engineering, neural machine translation, and applied AI for localization.