Your first eval
"It looks better to me" is how most prompt changes ship, and how most regressions sneak in. An eval turns that feeling into a number you can compare: version 7 passes 41 of 50, version 8 passes 46.
In this lesson you'll learn to
- Read real outputs, note what's wrong, and group the notes into failure types you can count.
- Build a small golden set of real inputs with what counts as a pass for each.
- Grade outputs with code checks where possible and a carefully checked model judge where not.
- Compare pass rates between versions and know how much a small set can tell you.
Create a free account to open this lesson
It's free. You'll get every free lesson, quizzes that save your progress, daily flashcards and streaks.