Prompt caching and batch APIs
Most of your bill is the same few thousand tokens sent again and again, or work nobody needs answered this second. Two features, caching and batches, can cut that cost by half or much more without changing a single answer.
In this lesson you'll learn to
- Explain how prompt caching reuses an exact prefix, and what breaks a cache hit.
- Lay out a prompt so the stable parts come first and stay cacheable.
- Work out the savings from caching, including cache writes and expiry.
- Decide when to use a batch API, and how to handle its results.
Create a free account to open this lesson
It's free. You'll get every free lesson, quizzes that save your progress, daily flashcards and streaks.