cached_tokens is 0 because your system prompt isn't stable
I added a [cache] log line next to our usage tracker and read usage.prompt_tokens_details.cached_tokens off a warm conversation. It said 0 . Then 1…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
LLMOps for Compound AI Systems — Observability & Cost
Why most GenAI pilots crumble after launch Most GenAI pilots don't fail because the models are bad — they fail because the surrounding system wasn't…
Why a 99% Cache Hit Rate Is 10x Faster Than 90% (Not 9%)
I ran the same query 10,000 times: almost 2 seconds of database work for an answer that never changed. The database did not fail. I just kept asking…
The Cache Hit Ratio Was Fine, but the Burst Still Melted the Database
When I started profiling a small FastAPI service, the first chart suggested the cache was doing its job. Single requests were fast, the hit ratio sa…
You Don't Need More Free Model Quota. You Need a Content-Addressed Cache.
Three identical prompts. Three model calls. Same answer. I saw this in a GitLab CI pipeline that runs a free model over release notes. The prompt te…
Full-Page Edge Caching for Laravel Behind Cloudflare: A Practical Guide
Of every performance optimization we have shipped for Laravel clients — OPcache tuning, query fixes, Redis fragment caching, queue offloading — noth…
Caching: How Backend Systems Get Faster with Redis
Analogy Imagine you have built a backend API that works perfectly. You deploy it. Users start coming in. At first, everything feels fast. Then the t…
DeepSeek V4 Flash API Cost: Thinking Mode Corrupts Strict JSON
DeepSeek V4 Flash costs $0.14 per million input tokens and $0.28 per million output, with cache hits at $0.0028, and the retrained 0731 build that n…
A failed read must throw, not return 0 — how Next.js ISR bakes your fallback into the cache
I run a directory site: ~3,500 rows in Postgres, read over PostgREST, rendered by Next.js with revalidate on every route. The headline number in the…
Redis Complete Guide: Caching, Queues & Pub/Sub (2026)
Redis is a data structure server that lives in memory. It's fast because reads and writes never touch a disk during normal operation. It's versatile…
Qwen 3.8 Max API Pricing: 16 Thinking Tokens Beat the Off Switch
Qwen 3.8 Max bills $2 per million input tokens and $6 per million output, and its cheapest reliable configuration is not the one the API seems to of…
Kmemo 2.0 is out, and the two gaps I admitted to in the first post are closed
This is a follow-up to Kmemo: a semantic cache for LLM calls that refuses to serve you the wrong answer . If you have not read it, the one-line vers…
Next.js Sitemap Not Updating? Here's the Real Fix
Next.js Sitemap Not Updating? Here's the Real Fix If your Next.js sitemap is not updating after you publish new content, you're dealing with a cache…
Prompt Cache Write Cost: When Does the 1.25x Premium Pay?
A prompt-cache write pays for itself after a single re-read within the TTL: the write bills 1.25x the input rate, the read bills 0.1x, so one hit al…
Claude Opus 5 vs Opus 4.8, Measured: Same Price, 3x Apart
Claude Opus 5 and Claude Opus 4.8 bill the same $5 per million input tokens and $25 per million output, and on identical prompts the default Opus 5…
Understanding Next.js Page Caching (with a real example)
Caching can be a tricky topic, especially when pages don’t update the way you expect. I recently ran into an interesting issue while working on the…
My "prompt caching" change made the bill go up — here's what I got wrong
I turned on prompt caching for an agent I run all day, watched the next invoice, and the number went up . Not by a rounding error — meaningfully up.…
Beyond TTL: Explicit Cache Invalidation
Most engineers default to TTL for cache invalidation. But a simple Time-To-Live guarantees stale reads for a duration, which is unacceptable for cri…
HTTP Caching for APIs: Cache-Control, max-age, and stale-while-revalidate
Most API teams reach for a Redis cache or a CDN long before they've spent the one HTTP header that does the job for free. Cache-Control lets clients…
The Fastest Query Is the One You Never Run: The Four Layers of Rails Caching
Rails Performance: Lessons from Production — #4 The first three posts were about making queries cheaper — fewer N+1s, indexes, not dragging data bac…
Cloud TTS Chirp3-HD with Caching: Fixing Voice Readout for Accessibility
Cloud TTS Chirp3-HD with Caching: Fixing Voice Readout for Accessibility As a solo developer, keeping the product lean and accessible is paramount.…
The X-Inertia cache trap: Cloudflare, Vary, and edge-cached HTML
I clicked the logo on one of my Laravel + Inertia sites and the homepage rendered inside Inertia's error modal — in production, on a 200 response. T…
Want to Go Deeper?
Your LLM bill is exploding because 70% of user queries are semantically identical, yet your traditional cache ignores them completely. Even worse, i…
Caching Layers in 2026: CDN, App, DB, Query: What Goes Where
Book: System Design Pocket Guide: Fundamentals — Core Building Blocks for Scalable Systems Also by me: Thinking in Go (2-book series) — Complete Gui…
Web performance beyond Core Web Vitals
Originally published at thatdevpro.com . Part of ThatDevPro's open SEO + AI framework library. ThatDevPro is an SDVOSB-certified veteran-owned web +…
Service Worker Caching Strategies: Cache-First, Network-First, and SWR
Related: Network Optimization for SPAs and React Apps covers the broader network optimization picture including HTTP caching and API request optimiz…
Your "Cache Invalidation is Hard" Answer Misses the Real Horror
Your "Cache Invalidation is Hard" Answer Misses the Real Horror Most engineers parrot "cache invalidation is hard" as a standard interview response,…
Best practices for CDN caching and origin caching optimization
CDN and origin caching optimization: 12 strategies that actually work If you're watching your server costs climb while page load times disappoint us…
Making It Fast: Caching, Hot Paths, and Avoiding DB Calls
The Auth Gateway sits in front of every authenticated request in the platform. Its latency isn't just its own latency — it's the floor for every ser…