Deduplicated Fallback Query Uses Wrong Relevance Threshold
Deduplicated fallback search queries can accidentally trigger the wrong relevance threshold and reject valid results when the requested query matche…
Testing & QA
⚑ Report a ProblemLatest Testing & QA news from Tech News
All topics
agents
ai
api
architecture
automation
backend
beginners
career
claude
cybersecurity
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
softwareengineering
testing
tutorial
typescript
webdev
Production failures: the things that page you · 3. Retry storms
Production failures: the things that page you · Chapter 3 of 24 · Reliability · new chapter every Friday morning By the end of this chapter: See how…
Kill the bug class, not just the bug
The dry run said the reminder text would go out to a customer with the window "12PM–2PM." The actual appointment was 8AM–10AM. Four hours off, on a…
How many jobs should a Node.js queue run at once?
Part 2 of 10: Reliable background jobs in Node.js, from zero . By Sandro Dezerio , senior full-stack engineer. The signup scenario is made up, the s…
Honoring Margaret Hamilton: Preserving Legacy of Pioneers Behind Modern AI and Autonomous Systems
Technical Reconstruction of Margaret Hamilton's Contributions to Software Engineering and AI Foundations Main Thesis: Margaret Hamilton's pioneering…
Compare Startup SMS Alerts Provider Templates — 4 US-EU Signature Tests
Operational constraint: a startup that needs to compare an SMS alerts provider cannot treat every failed message as proof that a recipient is invali…
Transactional Email Operations: Compliance, Custom Domains, and Bounce Reconciliation
A signup system has one unforgiving constraint: a verification link that arrives late, twice, or not at all is a reliability failure at the exact mo…
Approval timeouts: what your agent should do when nobody answers
§ 01 · The missing fifth field In the approval queue pattern I described four fields every approval item should carry: the action, the justification…
DNS Record Write Rejected Because Zone ID Is Not Domain Name — Validation Debug
Short answer: when a DNS record write is rejected because a zone ID is not a domain name, resolve the opaque reference first, canonicalize the retur…
We scheduled a question for an hour when nobody could answer it
The run was fully unattended. That was the point. Everything upstream of the final step had been automated for weeks, and the last remaining human t…
Feature flags are the loaded gun modern deploys pretend is safe
A trading firm experienced a loss of $440 million within 45 minutes. This was not due to a cyber attack or a sudden crash in the market. It was a fo…
Asynchronous Form Schema Discovery: Idempotent Retries and Secure Temporary Files
Use a durable job record as the unit of work for form schema discovery: validate the document at admission, submit one asynchronous extraction job u…
What Idempotent Actually Means: Why Retries Are Safe (and When They Aren't)
Originally published on Loop & Retry — field notes on building LLM agents that survive production. Search for "what does idempotent mean" and yo…
Production Hardening an AI Video Pipeline: Retries, Fallbacks, and Crash Guards
The video generation platform I work on orchestrates a long chain of AI calls — a video generation provider for clip rendering, Claude for scene pla…
Preventing Duplicate Password-Reset Notifications (Under SMS Timeout and Retry Pressure)
Treat an SMS timeout as an unknown outcome, not a failed send: accept each password-reset event once, persist its expiry and idempotency key before…
A Free Model Endpoint Replied Twice, Then Went Silent. The Fix Was a C++ Replay Envelope, Not Retries
Late on a Tuesday, a C++ tooling team noticed their warning classifier was duplicating classifications. The batch runner sent forty compiler diagnos…
Test OmniRoute Fallbacks for Semantic Consistency, Not Just Availability
diegosouzapw/OmniRoute is an MIT-licensed AI gateway that advertises one endpoint across many providers and models, with quota-aware fallback and to…
Test Saga Compensation When Payment Succeeds but Inventory Times Out
An order reserves inventory, charges payment, then times out waiting for reservation confirmation. Cancellation begins. A late inventory success arr…
If 30% of Coding Tasks May Be Broken, Your Leaderboard Needs an Uncertainty Budget
OpenAI published an audit of SWE-Bench Pro on July 8, 2026 and estimated that roughly 30% of its tasks are broken. The reported issues make a famili…
AI memory가 자기 출력을 다시 입력으로 세지 않게 하는 법
AI memory가 자기 출력을 다시 입력으로 세지 않게 하는 법 AI memory 시스템은 대화에서 기억을 만들고, 그 기억을 다시 요약해 더 높은 수준의 결론을 만들 수 있습니다. 여기서 원본에 가까운 입력과 시스템이 만든 파생 결과를 구분하지 않으면, 파생 결…
Canary Agentic Autofix With Failure Classes and Reliability Gates
GitHub announced agentic autofix for code scanning alerts in public preview on July 10, 2026. Primary source: GitHub Changelog, July 10, 2026 . The…
Merge Concurrent Agent Patches by Base Commit and Hunk Ownership
Two coding agents start from commit A. One finishes quickly and merges as B. The second finishes later with a patch that was valid against A. Applyi…
Treat Per-Task Model Switching as a Concurrency Protocol
Changing the model for a running AI task is not a settings update. It is a distributed operation: read current task -> prepare credentials/config…
Building a Culture of Reliability: Beyond the SRE Handbook
You Can't Hire Your Way to Reliability I've seen companies hire 5 SREs and expect reliability to magically improve. It doesn't. Reliability is a cul…
The 10 Most Expensive Software Failures in History — and the One Thing They Share
The biggest losses in software history were, with one deliberate exception, not attacks. They were silent, correlated, self-inflicted — and they tea…
Escalation Policies: How to Route Alerts Without Waking Everyone
A misconfigured escalation policy has two failure modes, and both are bad. Either every engineer on the team gets paged for a minor hiccup, or nobod…
On-Call Engineer Burnout: How to Build Sustainable Rotations
Most on-call rotations start with good intentions and end with someone updating their LinkedIn. The schedule ships as a shared Google Sheet, the tea…
The Pod That Lied: A Kubernetes Readiness Probe War Story
Last time I told you to bring coffee. I did. This one didn't start at 2 a.m. — it started at 10:40 on a perfectly ordinary Tuesday, which, if anythi…
SLOs That Product Managers Actually Understand
The SLO Translation Problem You define an SLO: 99.95% availability with p99 latency under 200ms. Engineering loves it. Product managers glaze over.…
Sliding-Window Spend Guard: the $47K Loop Per-Call Caps Miss
Sliding-Window Spend Guard for AI Agents: Catch the $47K Loop Per-Call Caps Miss A sliding-window spend guard sums what your agent has spent over th…