Navigating AND–OR Graph Modifications to Debug Failing Proof Search
Proof search powers our most advanced programming tools, from type systems, to search tactics for interactive theorem provers, to Datalog-backed program analyses. Although proof search tooling is powerful and now pervasive, \emph{debugging} it is hard, even for experts. When proof search cannot prove the goal, the programmer's best source of information is a massive AND–OR graph representing the tool's internal state during the proof search process. The difficulty of understanding and debugging this vast trace of internal state locks programmers out of exactly the high-assurance automated reasoning tools we want them to adopt.
We propose a new formulation of proof search debugging, which: (i)~views AND–OR graphs as a partial representations of the underlying proof system, (ii)~treats debugging as a process of applying modifications to this proof system, and (iii)~uses a debugging tool to solicit these modifications until the resulting proof system proves the original goal. This approach unifies decades of ad-hoc strategies in a single general-purpose framework and is applicable to the diverse range of programming tools that use proof search. Our framework can express existing ``why-not'' debugging strategies as well as new strategies, and we evaluate such strategies on 284 AND–OR graphs. We find that a strategy that enforces a property called \textsc{Strong Soundness} reduces the number of decisions by 1.4$\times$–3.2$\times$ compared to an unsound baseline, and a new property we call \textsc{Strong Completeness Modulo Observability} enables pruning to further reduce decisions by 1.0$\times$–2.8$\times$ for an overall reduction of 2.0$\times$–3.8$\times$.
Fri 19 JunDisplayed time zone: Mountain Time (US & Canada) change
14:00 - 15:40 | Verification and Proof ReasoningPLDI Research Papers at Flatirons 2 Chair(s): Qianchuan Ye University at Buffalo, SUNY | ||
14:00 20mTalk | Heterogeneous Dynamic Logic: Provability Modulo Program Theories PLDI Research Papers DOI | ||
14:20 20mTalk | Intrinsically Correct Algorithms and Recursive Coalgebras PLDI Research Papers Cass Alexandru RPTU Kaiserslautern-Landau & Radboud University Nijmegen, Henning Urbat Friedrich-Alexander University Erlangen-Nürnberg, Thorsten Wißmann Friedrich-Alexander University Erlangen-Nürnberg DOI Pre-print | ||
14:40 20mTalk | CRIS: The Power of Imagination in Hybrid Verification PLDI Research Papers Yonghee Kim Seoul National University, Taeyoung Yoon Seoul National University, Sanghyun Yi Seoul National University, Jaehyung Lee Seoul National University, Soonwon Moon Seoul National University, Yeji Han Seoul National University, Seonho Lee Seoul National University, Taeyoung Rhee Seoul National University, Yujin Im Seoul National University, Donghyun Nam Seoul National University, Jieung Kim Yonsei University, Chung-Kil Hur Seoul National University DOI | ||
15:00 20mTalk | Navigating AND–OR Graph Modifications to Debug Failing Proof Search PLDI Research Papers Justin Lubin University of California at Berkeley, Marlena Preigh University of California at Berkeley, Max Willsey University of California at Berkeley, Sarah E. Chasins University of California at Berkeley DOI Pre-print | ||
15:20 20mTalk | [TOPLAS] Project Everest: Perspectives from Developing Industrial-Grade High-Assurance Software PLDI Research Papers Danel Ahman University of Tartu, Karthikeyan Bhargavan Cryspen, France, Barry Bond , Jay Bosamiya Microsoft Research, Christopher Brzuska , Antoine Delignat-Lavaud Microsoft Research, n.n., Cédric Fournet Microsoft Research, Aymeric Fromherz Inria, Sydney Gibson , Chris Hawblitzel Microsoft Research, Cătălin Hriţcu MPI-SP, Markulf Kohlweiss , Guido Martínez Microsoft Research, Haobin Ni University of Washington, Bryan Parno Carnegie Mellon University, Jonathan Protzenko Microsoft Azure Research, Tahina Ramananandro Microsoft Research, Aseem Rastogi Microsoft Research, Exequiel Rivas Tallinn University of Technology; Ahrefs, Nikhil Swamy Microsoft Research, Santiago Zanella-Béguelin Microsoft Research, Cambridge Link to publication DOI | ||