The Slow Lane: Latency Engineering When Your AI Endpoint Is Free
Free model access solves the cost problem and creates a latency problem, and most teams measure the wrong number. My position is direct: the p95 of…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
Your agent's p99 is a different animal
Originally published on Loop & Retry — field notes on building LLM agents that survive production. The demo felt instant. The agent answered in…
Best-of-N is prepaid retries: the cost math of racing parallel attempts
Originally published on Loop & Retry — field notes on building LLM agents that survive production. Here's the pitch for best-of-N: instead of tr…
Your token bill is the cheap part: dimensioning the real cost of an agent
Originally published on Loop & Retry — field notes on building LLM agents that survive production. The token bill is the cost you can see, becau…
How I tried to write an article about slow Chinese LLMs
Recently, I've added a bunch of hype-monsters to my AI Werewolf : Kimi K3 Qwen 3.8 Max, Qwen 3.7 Plus, Qwen 3.7 Flash MiniMax M3 Plus the ones I've…