Sampling: temperature, top-p, top-k
The model hands you a list of probabilities, not a word. Which word you get depends on a few settings most people never touch, and a few that matter a lot for your app.
In this lesson you'll learn to
- Explain greedy decoding and sampling, and why the same prompt can give different answers.
- Describe what temperature does to the probability list, and pick a value for a given job.
- Explain top-k and top-p (nucleus) sampling and how they cut off unlikely tokens.
- Choose sampling settings for real tasks, and know why fixed settings still don't guarantee identical output.
Create a free account to open this lesson
It's free. You'll get every free lesson, quizzes that save your progress, daily flashcards and streaks.