Reuse a long prefix across requests and pay less for it.
When many requests share a long prefix — a system prompt, a tool schema, a document you
keep asking about — caching lets the provider reuse that work instead of reprocessing it.
Cached input is billed at a much lower rate than fresh input.PipeLLM passes caching through to the upstream provider and meters it separately, so the
savings land on your bill. How you turn it on depends on the protocol.
Anthropic: explicit
Anthropic caching is opt-in. Mark the end of the prefix you want cached with
cache_control, and everything before that point becomes the cache key.
The first request pays a write price; later requests that match the prefix pay the much
cheaper read price. Caches are short-lived — a five-minute window by default, with a
one-hour option priced separately.Responses report what happened:
cache_creation_input_tokens means you paid to write. cache_read_input_tokens means you
got a hit.
Bedrock and Vertex AI accept at most 4 cache_control blocks. When a request carries
more, PipeLLM keeps the last four — the ones closest to the end of the prompt, which are
usually the most valuable — and drops the earlier ones before forwarding. The request
still succeeds, so a request that works on Anthropic’s own API can quietly cache less
here. Keep to four breakpoints if you route across providers.
OpenAI and Gemini: automatic
No request field to set. The provider detects a repeated prefix on its own and applies a
discount to the cached portion. OpenAI reports it under
usage.prompt_tokens_details.cached_tokens.Because it is automatic, the only thing you control is prompt shape — see below.
Writing cacheable prompts
Caching keys on an exact prefix match, so the rule is simple: put what is stable first
and what varies last.
Static system prompt, tool definitions, and reference documents at the top.
The user’s current turn at the bottom.
Nothing variable early on. A timestamp, a request ID, or a shuffled list near the start
invalidates the whole prefix and you pay full price every time.
A cache only pays off when it is reused inside its lifetime. One-off requests cost slightly
more with an explicit cache write, so reserve cache_control for prefixes you will
actually hit again within a few minutes.
Checking the price
Cache rates are per model. Look them up with
GET /v1/models/pricing — the response carries them
alongside the regular token prices:
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"PipeLLM","colors":{"primary":"#12B76A","light":"#3DF58F","dark":"#0E9152"},"logo":{"light":"https://mintcdn.com/pipellm/c14KpYQEcMuO-i7Z/logo/light.svg?fit=max\u0026auto=format\u0026n=c14KpYQEcMuO-i7Z\u0026q=85\u0026s=e63d66e7731e79c17fab3ac12709e0b1","dark":"https://mintcdn.com/pipellm/c14KpYQEcMuO-i7Z/logo/dark.svg?fit=max\u0026auto=format\u0026n=c14KpYQEcMuO-i7Z\u0026q=85\u0026s=3205f7a8fd0d53df4ccbd10fc7fff796"},"favicon":"https://mintcdn.com/pipellm/c14KpYQEcMuO-i7Z/favicon.svg?fit=max\u0026auto=format\u0026n=c14KpYQEcMuO-i7Z\u0026q=85\u0026s=696ba46d25b2028f32c94b319736a36c","api":{"params":{"expanded":"all"},"playground":{"display":"interactive"},"examples":{"defaults":"required","languages":["curl","python","javascript","go"],"prefill":true},"mdx":{"auth":{"method":"bearer"},"server":"https://api.pipellm.ai"}},"appearance":{"default":"dark"},"background":{"color":{"light":"#FCFCFD","dark":"#05080A"}},"navbar":{"links":[{"href":"https://pipellm.ai/models","label":"Models"},{"href":"https://pipellm.ai/playground","label":"Playground"},{"href":"https://pipellm.ai/pricing","label":"Pricing"},{"href":"https://pipellm.ai/blog","label":"Blog"},{"href":"mailto:support@pipellm.ai","label":"Support"}],"primary":{"type":"button","label":"Console","href":"https://console.pipellm.ai"}},"footer":{"socials":{"x":"https://x.com/pipellm","github":"https://github.com/pipellm","linkedin":"https://linkedin.com/company/pipellm"}},"search":{},"fonts":{"heading":{"family":"Plus Jakarta Sans","weight":400},"body":{"family":"Plus Jakarta Sans","weight":400}},"icons":{"library":"lucide"},"styling":{"eyebrows":"breadcrumbs","codeblocks":"dark","latex":true},"contextual":{"options":["copy","view","chatgpt","claude","perplexity","mcp","cursor","vscode"],"display":"header"},"navigation":{"languages":[{"language":"en","href":""},{"language":"zh","href":"zh"}]}},"deployment":{"deploymentMetadataValue":{"subdomain":"pipellm","hasAudioNarration":false,"actualSubdomain":"pipellm","customDomains":["docs.pipellm.ai"],"gitSource":{"type":"github","owner":"pipellm-ai","repo":"docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true},"entitlements":{},"buildId":"6ab0bf26528446e729be3cf7","clientVersion":"0.0.4177","searchFilterCounts":[{"language":"en","tag":"API Reference","count":51},{"language":"zh","tag":"API 参考","count":51},{"language":"en","tag":"Relay","count":16},{"language":"zh","tag":"Relay","count":16},{"language":"en","tag":"Integrations","count":7},{"language":"zh","tag":"集成","count":7},{"language":"en","tag":"Guides","count":6},{"language":"en","tag":"WebSearch","count":6},{"language":"zh","tag":"指南","count":6},{"language":"zh","tag":"WebSearch","count":6},{"language":"en","tag":"API Guides","count":5},{"language":"en","tag":"Protocol Conversion","count":5},{"language":"zh","tag":"API 指南","count":5},{"language":"zh","tag":"协议转换","count":5},{"language":"en","tag":"Images","count":4},{"language":"en","tag":"Video","count":4},{"language":"zh","tag":"图像","count":4},{"language":"zh","tag":"视频","count":4},{"language":"en","tag":"V2 (Recommended)","count":3},{"language":"zh","tag":"V2(推荐)","count":3}]},"assistantConfig":{},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":true},"customCss":[{"filePath":"style.css","content":".dark img[src*=\"/logo/providers/\"]{filter:brightness(0.715)}:root:root{--gray-50: 246 248 249;--gray-100: 240 243 244;--gray-200: 224 228 230;--gray-300: 206 211 214;--gray-400: 158 165 169;--gray-500: 112 119 123;--gray-600: 80 87 91;--gray-700: 62 69 73;--gray-800: 36 42 46;--gray-900: 22 27 30;--gray-950: 10 14 16}.dark{--default-border-color: rgb(252 252 253 / 0.14)}#request-example,#response-example{border-radius:0.75rem}method-pill,.method-pill{letter-spacing:0.04em}#pipellm-playground-mask{display:none}[data-pipellm-playground]>[role=\"presentation\"].bg-black\\/40{background:rgb(3 8 12 / 0.55)!important;backdrop-filter:blur(14px) saturate(1.2);-webkit-backdrop-filter:blur(14px) saturate(1.2);transition-duration:320ms!important}:root:not(.dark) [data-pipellm-playground]>[role=\"presentation\"].bg-black\\/40{background:rgb(248 250 250 / 0.55)!important}[data-pipellm-playground] [role=\"dialog\"]{border-radius:16px!important;box-shadow:0 0 0 1px rgb(255 255 255 / 0.08),\n 0 32px 96px rgb(0 0 0 / 0.55),\n 0 8px 24px rgb(0 0 0 / 0.35)!important;transition-property:transform, scale, translate, opacity!important;transition-duration:320ms!important;transition-timing-function:cubic-bezier(0.16, 1, 0.3, 1)!important}:root:not(.dark) [data-pipellm-playground] [role=\"dialog\"]{box-shadow:0 0 0 1px rgb(0 0 0 / 0.06),\n 0 32px 96px rgb(0 0 0 / 0.18),\n 0 8px 24px rgb(0 0 0 / 0.10)!important}[data-pipellm-playground] [role=\"dialog\"][data-starting-style]{opacity:0!important;scale:0.97!important;translate:0 18px!important}[data-pipellm-playground] [role=\"dialog\"][data-ending-style]{opacity:0!important;scale:0.985!important;translate:0 8px!important;transition-duration:180ms!important}"}],"page":{"pagePath":"guides/prompt-caching","slug":"/guides/prompt-caching","pageMetadata":{"title":"Prompt Caching","description":"Reuse a long prefix across requests and pay less for it.","i18n":"en","href":"/guides/prompt-caching"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Reuse a long prefix across requests and pay less for it.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,em:`em`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsx(_components.p,{children:`When many requests share a long prefix — a system prompt, a tool schema, a document you\nkeep asking about — caching lets the provider reuse that work instead of reprocessing it.\nCached input is billed at a much lower rate than fresh input.`}),`\n`,_jsx(_components.p,{children:`PipeLLM passes caching through to the upstream provider and meters it separately, so the\nsavings land on your bill. How you turn it on depends on the protocol.`}),`\n`,_jsx(Heading,{level:`2`,id:`anthropic-explicit`,children:`Anthropic: explicit`}),`\n`,_jsxs(_components.p,{children:[`Anthropic caching is opt-in. Mark the end of the prefix you want cached with\n`,_jsx(_components.code,{children:`cache_control`}),`, and everything before that point becomes the cache key.`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`11`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`11`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"model\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"claude-sonnet-4-6\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"system\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"type\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"text\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"text\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"\u003ca long style guide>\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"cache_control\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: { `}),_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:`\"type\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"ephemeral\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` }`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` ],`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"messages\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [{ `}),_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"Rewrite this paragraph: ...\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` }]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`The first request pays a `,_jsx(_components.strong,{children:`write`}),` price; later requests that match the prefix pay the much\ncheaper `,_jsx(_components.strong,{children:`read`}),` price. Caches are short-lived — a five-minute window by default, with a\none-hour option priced separately.`]}),`\n`,_jsx(_components.p,{children:`Responses report what happened:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`4`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`4`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"usage\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"cache_creation_input_tokens\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`12043`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"cache_read_input_tokens\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`cache_creation_input_tokens`}),` means you paid to write. `,_jsx(_components.code,{children:`cache_read_input_tokens`}),` means you\ngot a hit.`]}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[_jsxs(_components.strong,{children:[`Bedrock and Vertex AI accept at most 4 `,_jsx(_components.code,{children:`cache_control`}),` blocks.`]}),` When a request carries\nmore, PipeLLM keeps the last four — the ones closest to the end of the prompt, which are\nusually the most valuable — and drops the earlier ones before forwarding. The request\nstill succeeds, so a request that works on Anthropic’s own API can quietly cache less\nhere. Keep to four breakpoints if you route across providers.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`openai-and-gemini-automatic`,children:`OpenAI and Gemini: automatic`}),`\n`,_jsxs(_components.p,{children:[`No request field to set. The provider detects a repeated prefix on its own and applies a\ndiscount to the cached portion. OpenAI reports it under\n`,_jsx(_components.code,{children:`usage.prompt_tokens_details.cached_tokens`}),`.`]}),`\n`,_jsx(_components.p,{children:`Because it is automatic, the only thing you control is prompt shape — see below.`}),`\n`,_jsx(Heading,{level:`2`,id:`writing-cacheable-prompts`,children:`Writing cacheable prompts`}),`\n`,_jsxs(_components.p,{children:[`Caching keys on an exact prefix match, so the rule is simple: `,_jsx(_components.strong,{children:`put what is stable first\nand what varies last.`})]}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsx(_components.li,{children:`Static system prompt, tool definitions, and reference documents at the top.`}),`\n`,_jsx(_components.li,{children:`The user’s current turn at the bottom.`}),`\n`,_jsx(_components.li,{children:`Nothing variable early on. A timestamp, a request ID, or a shuffled list near the start\ninvalidates the whole prefix and you pay full price every time.`}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`A cache only pays off when it is reused inside its lifetime. One-off requests cost slightly\n`,_jsx(_components.em,{children:`more`}),` with an explicit cache write, so reserve `,_jsx(_components.code,{children:`cache_control`}),` for prefixes you will\nactually hit again within a few minutes.`]}),`\n`,_jsx(Heading,{level:`2`,id:`checking-the-price`,children:`Checking the price`}),`\n`,_jsxs(_components.p,{children:[`Cache rates are per model. Look them up with\n`,_jsx(_components.a,{href:`/api-reference/model-pricing`,children:_jsx(_components.code,{children:`GET /v1/models/pricing`})}),` — the response carries them\nalongside the regular token prices:`]}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`9`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`9`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"pricing\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"kind\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"token\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"text\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: { `}),_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:`\"prompt\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"0.000003\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:`\"completion\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"0.000015\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` },`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"cache\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"read\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"0.0000003\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"write\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"0.00000375\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` \"write_1h\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"0.0000075\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[`A model with no `,_jsx(_components.code,{children:`cache`}),` object does not support caching on the mapping you are routed to.`]}),`\n`,_jsx(Heading,{level:`2`,id:`related`,children:`Related`}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/api-reference/model-pricing`,children:`Get model pricing`}),` — authoritative cache rates`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.a,{href:`/api-reference/anthropic/messages`,children:`Anthropic Messages`}),` — where `,_jsx(_components.code,{children:`cache_control`}),` goes`]}),`\n`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":"dark","config":{},"pageMetadata":{"title":"Prompt Caching","description":"Reuse a long prefix across requests and pay less for it.","i18n":"en","href":"/guides/prompt-caching"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Anthropic: explicit","slug":"anthropic-explicit","depth":2,"children":[]},{"title":"OpenAI and Gemini: automatic","slug":"openai-and-gemini-automatic","depth":2,"children":[]},{"title":"Writing cacheable prompts","slug":"writing-cacheable-prompts","depth":2,"children":[]},{"title":"Checking the price","slug":"checking-the-price","depth":2,"children":[]},{"title":"Related","slug":"related","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/_mintlify/custom/pipellm/df480d64db6c22d0/playground.js"],"subdomain":"pipellm","siteName":"PipeLLM","title":"Prompt Caching - PipeLLM","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{"en":{"href":"/guides/prompt-caching","matchLevel":0},"zh":{"href":"/zh/guides/prompt-caching","matchLevel":1}}},"lastModified":"2026-09-19T04:58:52.658Z","originalFileLocation":"guides/prompt-caching.mdx","locale":"en"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"dark\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Reuse a long prefix across requests and pay less for it.\">\u003cmeta data-mint-head name=\"application-name\" content=\"PipeLLM\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Prompt Caching - PipeLLM\">\u003cmeta data-mint-head property=\"og:description\" content=\"Reuse a long prefix across requests and pay less for it.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"PipeLLM\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://docs.pipellm.ai/guides/prompt-caching\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://pipellm.mintlify.site/_mintlify/api/og/v2?division=Guides\u0026amp;title=Prompt+Caching\u0026amp;description=Reuse+a+long+prefix+across+requests+and+pay+less+for+it.\u0026amp;theme=3c00a4b989b0438e6fc46fe4\u0026amp;signature=76c2c97710c356686af058c32795567c90dd9df818f5388c6f9cd5077d5547ac\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Prompt Caching - PipeLLM\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Reuse a long prefix across requests and pay less for it.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://pipellm.mintlify.site/_mintlify/api/og/v2?division=Guides\u0026amp;title=Prompt+Caching\u0026amp;description=Reuse+a+long+prefix+across+requests+and+pay+less+for+it.\u0026amp;theme=3c00a4b989b0438e6fc46fe4\u0026amp;signature=76c2c97710c356686af058c32795567c90dd9df818f5388c6f9cd5077d5547ac\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/pipellm/JKtU23MtMj4TtKyB/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"PipeLLM\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#12B76A\">\u003clink data-mint-head rel=\"canonical\" href=\"https://docs.pipellm.ai/guides/prompt-caching\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/guides/prompt-caching.md\">\u003clink data-mint-head rel=\"alternate\" hreflang=\"en\" href=\"https://docs.pipellm.ai/guides/prompt-caching\">\u003clink data-mint-head rel=\"alternate\" hreflang=\"zh\" href=\"https://docs.pipellm.ai/zh/guides/prompt-caching\">\u003clink data-mint-head rel=\"alternate\" hreflang=\"x-default\" href=\"https://docs.pipellm.ai/guides/prompt-caching\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://docs.pipellm.ai/#organization\",\"name\":\"PipeLLM\",\"url\":\"https://docs.pipellm.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/pipellm/c14KpYQEcMuO-i7Z/logo/light.svg?fit=max\u0026auto=format\u0026n=c14KpYQEcMuO-i7Z\u0026q=85\u0026s=e63d66e7731e79c17fab3ac12709e0b1\"}},{\"@type\":\"WebSite\",\"@id\":\"https://docs.pipellm.ai#website\",\"name\":\"PipeLLM\",\"url\":\"https://docs.pipellm.ai\",\"publisher\":{\"@id\":\"https://docs.pipellm.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://docs.pipellm.ai/guides/prompt-caching#webpage\",\"url\":\"https://docs.pipellm.ai/guides/prompt-caching\",\"name\":\"Prompt Caching\",\"description\":\"Reuse a long prefix across requests and pay less for it.\",\"dateModified\":\"2026-09-19T04:58:52.658Z\",\"isPartOf\":{\"@id\":\"https://docs.pipellm.ai#website\"},\"breadcrumb\":{\"@id\":\"https://docs.pipellm.ai/guides/prompt-caching#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://docs.pipellm.ai/guides/prompt-caching#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Guides\",\"item\":\"https://docs.pipellm.ai/guides/chat\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Prompt Caching\",\"item\":\"https://docs.pipellm.ai/guides/prompt-caching\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://docs.pipellm.ai/guides/prompt-caching#article\",\"headline\":\"Prompt Caching\",\"name\":\"Prompt Caching\",\"description\":\"Reuse a long prefix across requests and pay less for it.\",\"url\":\"https://docs.pipellm.ai/guides/prompt-caching\",\"mainEntityOfPage\":{\"@id\":\"https://docs.pipellm.ai/guides/prompt-caching#webpage\"},\"dateModified\":\"2026-09-19T04:58:52.658Z\",\"publisher\":{\"@id\":\"https://docs.pipellm.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://docs.pipellm.ai#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"PipeLLM\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"PipeLLM\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/guides/prompt-caching.md","siteName":"PipeLLM","nav":{"global":null,"languages":[{"language":"en","tabs":[{"tab":"Relay","icon":"route","groups":[{"group":"Get Started","icon":"rocket","pages":[{"title":"Overview","href":""},{"title":"Quickstart","href":"/quickstart"},{"title":"Models","href":"/models"}]},{"group":"Guides","icon":"book-open","pages":[{"title":"Chat Completions","href":"/guides/chat"},{"title":"Images","href":"/guides/images"},{"title":"Nano Banana Image Generation","href":"/use-cases/nano-banana"},{"title":"Video generation","href":"/guides/video"},{"title":"Routing \u0026 Protocols","description":"Choose the correct native route or converter route for each request format","href":"/guides/routing-protocols"},{"title":"Prompt Caching","href":"/guides/prompt-caching"}]},{"group":"WebSearch","icon":"globe","pages":[{"title":"Overview","description":"PipeLLM WebSearch is a standalone search and retrieval product surface for developers and agents","href":"/websearch/overview"},{"title":"pipellm-websearch (npm)","href":"/websearch/npm-package"}]},{"group":"Protocol Conversion","icon":"shuffle","pages":[{"title":"Protocol Conversion Overview","href":"/converter/overview"},{"title":"OpenAI Format Converter","href":"/converter/openai-format"},{"title":"Responses Format Converter","href":"/converter/responses-format"},{"title":"Anthropic Format Converter","href":"/converter/anthropic-format"},{"title":"Gemini Format Converter","href":"/converter/gemini-format"}]}]},{"tab":"API Reference","icon":"code","pages":[{"title":"API Reference","href":"/api-reference/introduction"}]},{"tab":"Integrations","icon":"plug","pages":[{"title":"Integrations","href":"/integrations/overview"}]}]},{"language":"zh","pages":[{"title":"zh","href":"/zh/guides/prompt-caching"}]}]},"basePath":"","title":"Prompt Caching - PipeLLM","deploymentId":"dpl_HhUetB7eHHDAmK9T1Teip38TgFx9","buildId":"dpl_HhUetB7eHHDAmK9T1Teip38TgFx9","dynamics":[],"chunks":[]}