11# Verified hand-written Lean solutions to HumanEval
22
33[ HumanEval] is a collection of small programming tasks originally intended as a benchmark for LLMs.
4- In this repo, we collect hand-written * formally verified * [ Lean 4] solutions to these problems.
4+ In this repo, we collect hand-written _ formally verified _ [ Lean 4] solutions to these problems.
55
66The idea is to build a small set of simple examples for verified software development using Lean.
77It is inspired by [ human-eval-verus] , which does a similar thing for the Verus Rust verification
@@ -24,10 +24,18 @@ as the resulting code can still be considered idiomatic.
2424We use the [ Lean 4 standard library style guide and naming convention] ( https://github.com/leanprover/lean4/tree/master/doc/std ) ,
2525but we won't be very strict about enforcing it.
2626
27+ ### Missing lemmas
28+
2729One of the goals of the goals of this project is to assess the out-of-the-box
2830experience of Lean 4 as a software verification tool, so this project will not
2931take on dependencies such as Batteries.
3032
33+ Solutions to most tasks will encounter lemmas missing from Lean core. Some of the missing
34+ lemmas might be available in a place like Batteries and mathlib. The procedure in this case is
35+ to put the missing lemmas in the file where they are needed, with a note if they are available
36+ somewhere else. In this way, the files in this repository clearly document what was missing
37+ from Lean core.
38+
3139[ HumanEval ] : https://github.com/openai/human-eval
3240[ Lean 4 ] : https://lean-lang.org/
3341[ human-eval-verus ] : https://github.com/secure-foundations/human-eval-verus
0 commit comments