Using LLMs to Evaluate Code

The SEI is a not-for-profit federally funded research and development center (FFRDC) at Carnegie Mellon Universityspecifically established by the Department of Defense to focus on software andcybersecurity. We help government and industry organizations developsoftware-reliant systems that are more affordable and more agile, morereliable and more secure. We provide immediate and long-termintegrated solutions via the strength of our workforce and focusedresearch. The SEI is positioned to enhance impact, integration, and agility.

Software Engineering Institute at Carnegie Mellon University

In this webcast, Dr. Mark Sherman will summarize  results of experiments  conducted to see if LLMs could identify problems with source code.

Finding and fixing weaknesses and vulnerabilities in source code has been an ongoing challenge. There is a lot of excitement about the ability of large language models (LLMs, e.g., GenAI) to produce and evaluate programs. One question related to this ability is: Do these systems help in practice? We ran experiments with various LLMs to see if they could correctly identify problems with source code or determine that there were no problems. This webcast will provide background on our methods and a summary of our results.What Will Attendees Learn?<ul><li>how well LLMs can evaluate source code</li><li>evolution of capability as new LLMs are released</li><li>how to address potential gaps in capability</li></ul>Who Should Attend?<ul><li>development managers</li><li>product owners</li><li>software executives</li></ul>About the SpeakerDr. Mark Sherman is the Technical Director of the Cybersecurity Foundations directorate in the CERT Division of the Carnegie Mellon University Software Engineering Institute (CMU SEI). Sherman leads a diverse team of researchers and engineers on projects that focus on foundational research on the lifecycle for building secure software, data-driven analysis of cybersecurity, cybersecurity of quantum computers, cybersecurity for and enabled by machine learning applications, and detecting fake media. Prior to his tenure at the SEI, Sherman worked on mobile systems, integrated hardware-software appliances, transaction processing, languages and compilers, virtualization, network protocols, and databases at IBM and various startups.

Our {communityGuidelinesLink} describe the sort of content we prohibit on Eventbrite. If you suspect an event may be in violation, you can report it to us so we can investigate.

To report other categories of prohibited or illegal content, submit {link}

Using LLMs to Evaluate Code

Still looking for the right event?

Explore all online events to browse and filter by date, category, and more.

Using LLMs to Evaluate Code

More Science & Tech events

Browse more Science & Tech events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all online events to browse and filter by date, category, and more.