SESSION

Moving Faster and Reducing Risk: Using LLMs in Release Deployment

This talk discusses the challenge of determining what should be released in large-scale software development, such as at Meta’s scale. To address this, we developed models to determine the risk of a pull request (diff) causing an outage (aka SEV). We trained the models on historical data and used different types of gating to predict the riskiness of an outgoing diff. The models were able to capture a significant percentage of SEVs while gating a relatively small percentage of risky diffs. We also compared different models, including logistic regression, BERT-based models, and generative LLMs, and found that the generative LLMs performed the best.

View the slides Arrow Down
PRESENTED BY
  • Company Logo

    Rui Abreu

    Research Software Engineer @ Meta

Moving Faster and Reducing Risk: Using LLMs in Release Deployment Watch Video

Watch the video Play Button

View the slides Arrow Down
Develocity

Take it further with Develocity

You can’t improve what you don’t measure. Better toolchain observability empowers you to identify the right problems to work on the right solutions—saving time, money, and a whole lot of stress. Develocity Reporting & Visualization provides the most comprehensive and actionable reporting yet, so you can see how your toolchain is performing on every level and across every project. The sooner you learn more about it, the sooner you can stop wasting time solving the wrong problems…

Learn more Arrow Right

The slides

Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 1Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 2Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 3Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 4Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 5Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 6Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 7Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 8Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 9Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 10Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 11Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 12Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 13Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 14Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 15Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 16Moving Faster and Reducing Risk: Using LLMs in Release Deployment — slide 17

Meet the speaker

  • Company Logo

    Rui Abreu

    Rui Abreu holds a Ph.D. in Computer Science – Software Engineering from the Delft University of Technology, The Netherlands, and a M.Sc. in Computer and Systems Engineering from the University of Minho, Portugal. His research revolves around software quality, with emphasis in automating the testing and debugging phases of the software development life-cycle as well as self-adaptation. He has extensive expertise in both static and dynamic analysis algorithms for improving software quality. He is the recipient of 5 Best Paper Awards, and his work has attracted considerable attention. Before joining the Instituto Superior Técnico of University of Lisbon as an Associate Professor, he was a member of the Model-Based Reasoning group at PARC’s System and Sciences Laboratory. He is also with DashDash, a startup that aims to automate workflows using Excel skills only, as Head of Engineering.

    X (Twitter) LinkedIn