Here is the link to the notebook used in the workshop.

The first problem you’re likely to encounter when fine-tuning an LLM is the “host out of memory” error. It’s more difficult for fine-tuning the 7B parameter Llama-2 model which requires more memory. In this talk, we are having Piero Molino and Travis Addair from the open-source Ludwig project to show you how to tackle this problem.

The good news is that, with an optimized LLM training framework like Ludwig.ai, you can get the host memory overhead back down to a more reasonable host memory even when training on multiple GPUs.

In this hands-on workshop, we‘ll discuss the unique challenges in finetuning LLMs and show you how you can tackle these challenges with open-source tools through a demo.

By the end of this session, attendees will understand:

How to fine-tune LLMs like Llama-2-7b on a single GPU
Techniques like parameter efficient tuning and quantization, and how they can help
How to train a 7b param model on a single T4 GPU (QLoRA)
How to deploy tuned models like Llama-2 to production
Continued training with RLHF
How to use RAG to do question answering with trained LLMs

This session will equip ML engineers to unlock the capabilities of LLMs like Llama-2 on for their own projects.

Speakers

Piero Molino is the co-founder and CEO of Predibase. He was one of the founding members of Uber AI Labs. He worked on several deployed ML systems, including an NLP model for Customer Support, and the Uber Eats Recommender System with graph learning and collision detection. Later, he became a Staff Research Scientist at Stanford University working on Machine Learning systems. He is the author of Ludwig.ai with 8900 stars on GitHub, an open source declarative deep learning framework. In 2021 he co-founded Predibase, the low-code declarative machine learning platform built on top of Ludwig.

Travis Addair is co-founder and CTO of Predibase, a low-code platform for predictive and generative AI. Within the Linux Foundation, he serves as lead maintainer for the Horovod distributed deep learning framework and is a co-maintainer of the Ludwig declarative deep learning framework. In the past, he led Uber’s deep learning training team as part of the Michelangelo machine learning platform.

Efficient Fine-Tuning for Llama-7b on a Single GPU