This article was originally published on my blog. For the latest version and future updates, please visit the original post: https://jaketao.com/language/en/kv-cache-vs-prompt-cache/.Every time a large language model generates a token, it draws on the content that came before it. If it had to compute everything from scratch at every step, responses would be much slower. When building an agent, the same set of system prompts, tool definitions, and conversation history is used over and over again. If these were reprocessed each time, latency and computational costs would continually increase.