Nilay Baranwal
  • WorkSection
  • WritingSection
  • LabSection
  • Open sourceSection
  • GallerySection
  • AboutSection
  • ResumeSection
  • ContactSection
  • Hive MindProject
  • Pre-trade dashboardProject
  • Social LineProject
  • PartymakerProject
  • Orvio.aiProject
  • Agentic property managementProject
  • AI agents now outwork humans at Anthropic. Junior engineers are feeling it first.Article
  • Anthropic replaced git. Here's why you probably shouldn't.Article
  • Recruiters can now interview your AI twin. Here's what it actually checks.Article
  • Everyone's building a Jev rival. The independent numbers tell a different story.Article
  • Jev vs OpenAI Decisions API: which decision model should your agent use?Article
  • Top 10 Jev use cases in agentic appsArticle
  • Jev vs Laya: decision modelsArticle
  • LLMs or Jev: where each one actually fitsArticle
  • Everyone's talking about Jev. Here's a closer look at what the data shows.Article
  • Live agent with an approval gateLab
  • AI engineer interview, beginner pathLab
  • Behavioural interview coachLab
  • Interview drill, experiencedLab
  • AI adoption plannerLab
  • Discovery call simulatorLab
  • Scope a vague problemLab
  • MCP analyzerLab
  • MCP inspectorLab
  • Tool gate playgroundLab
  • Prompt-injection spotterLab
  • RAG chunking visualizerLab
  • Eval benchLab
  • Agent trace debuggerLab
  • Prompt caching simulatorLab
  • Context window budgetLab
  • textagentOpen source
  • Help EngineerOpen source
  • Email nilaybaranwal@gmail.comContact
  • Connect your agent to my MCP serverLab
  • LinkedInContact
  • GitHubContact
Lab

For AI engineering

Prompt caching simulator

Reorder the parts of a request and watch cache hits and cost change over a conversation.

Order of the request, top to bottom

  1. 1Current date and timechanges every turn, 50 tokens
  2. 2System promptsame every turn, 2.0k tokens
  3. 3Tool definitionssame every turn, 6.0k tokens
  4. 4Reference documentssame every turn, 20.0k tokens
  5. 5Conversation so fargrows every turn, +800/turn tokens
  6. 6The user's new messagechanges every turn, 200 tokens

Cache hit rate

0%

Input cost, this chat

$0.745

Without caching

$0.745

  1. Turn 10 / 28.3k
  2. Turn 20 / 29.1k
  3. Turn 30 / 29.9k
  4. Turn 40 / 30.6k
  5. Turn 50 / 31.4k
  6. Turn 60 / 32.3k
  7. Turn 70 / 33.0k
  8. Turn 80 / 33.9k

read from cache (about 10% of the price) processed in full

“Current date and time” sits above system prompt, tool definitions, reference documents, so 28.0k stable tokens are re-processed on every turn. Move the parts that never change to the top.

Simplified: real APIs also have cache breakpoints, a minimum cacheable size, expiry after a few minutes of no use, and a small premium for writing to the cache.