ODSC AI East 2026 Conference | Open Data Science Conference

A Practical Guide to LLM Evaluation

This session cuts through the noise to provide a hands-on, practical guide for product, business, and technical practitioners tasked with validating LLM-powered systems. We will explore a comprehensive evaluation toolkit that balances cost, speed, and quality. We'll cover the pros and cons of automated metrics (e.g., ROUGE, BLEU), the strategic use of LLM-as-judge frameworks for nuanced qualitative assessment, and best practices for designing efficient and scalable human-in-the-loop evaluation pipelines.

5 Rules of Enterprise AI for 2026

In this 30 minute session, AI veteran Cameron Turner will guide the audience through 5 rules all AI practitioners and decision makers should adhere to when building scaled industrial AI systems. Spanning technical and organizational topics, he will share wins and pitfalls Fortune500/Global1k companies have made in early generative AI adoption and offer practical steps to reduce risk and ensure value creation. With an eye toward sustainability and ethics, Turner will describe how to include governance and human oversight into AI release processes to improve AI performance and ROI for any enterprise. Attendees will leave with a better understanding of the current trends in enterprise AI in 2025, and a practical toolset for building/deploying AI in their environments in 2026.

From Automated to Agentic to Scaled Agentic

Session will include design blueprints/demo, and lessons learned from real production rollouts—enough to start building your own ecosystem, but with plenty still under wraps for the hallway track. Join us to glimpse the agentic future and leave with the roadmap to reach it.