Testing & QA

A screenshot that fails on its own: pin the data, not the pixel

A “flaky” visual test is almost never the pixel comparator. It is the page changing by itself between two captures — and the diff, correctly, points…

The Test Wasn't Wrong. It Was Answering an Older Question.

The Test Wasn't Wrong. It Was Answering an Older Question. I shipped a change I had reviewed, simulated, and deployed. An hour later my acceptance s…

You Cannot Unit Test a Summary. You Can Still Test It.

Every developer I know has the same reflex when a feature is done: write a test that feeds it an input and asserts the output. That reflex is correc…

Evaluate RAG retrieval before changing your chatbot prompt

Before tuning a prompt, check if the retriever actually surfaced the evidence. A fluent answer often hides a missing document. And a retrieved docum…

Validate bounding boxes from a general vision model

A general model can take an image plus a question and return segmentation, bounding boxes, and object detection, work that used to require its own t…

Методология оптимизации игр, или как 3 дня работы над тестовым стендом дало больше FPS, чем месяц написания кода

Я работал месяц над переписыванием физического движка и оптимизацией отрисовки и получил ошибочный успех. Спустя несколько месяцев я вернулся с друг…

Build a 10-Case Eval Set for Your AI Agent and Run It on Every Change

Your agent worked in the demo. Then you tweaked the system prompt, swapped the model, or renamed a tool argument, and now you're not sure whether it…

Exit Criteria for a Free Model Tier

A free model tier is a lease with an unpublished end date. Signing it is not the mistake. Learning the terms during a production incident is. The us…

Interrupted Runs Belong in the Denominator

A benchmark score that quietly drops interrupted runs is not a conservative score; it is a score whose hardest cases were deleted before anyone coun…

The Portability Audit Take-Home: Grade Environment Assumptions, Not Features

A take-home that runs on the reviewer's laptop grades features; a take-home that runs anywhere grades engineering discipline. The task below asks a…

Privacy Budgets for Test Email in Web Apps

Email is often treated as harmless test plumbing. In a web application, it is closer to a small data store: messages can contain personal-looking ad…

Cold Starts, Lost Files, and Green Dots: Five Myths About Using a Free Server as Staging

My demo broke at 9:58 PM. The same URL answered at 9:40. My code did not change in between. That gap is the whole lesson. A free server is a shared,…

Properties Without Witnesses Are Decorations: A Falsifiability Gate for Agent Patches

A property that passes on the patched tree and on the pre-patch tree carries no information. That is the first thing to measure, before you trust an…

The build went green because it stopped checking

A teammate deleted a test file on a Friday. Not on purpose. The suite still passed. The pipeline still went green. Nobody noticed for three weeks. H…

A 200 response and a broken page: what screenshot monitoring taught me

An uptime check answers one question: did the server respond? Visitors ask a different one: can I use this page? After a few years of looking after…

Stop Using Random Card Numbers in Your Checkout Tests (Use These Instead)

When you build a checkout form, you need card numbers to test it. The usual shortcut is to type 4111 1111 1111 1111 or make up some digits. That wor…

A Green Test Suite Can Still Hide a Broken PR: 5 Questions to Ask Before Merging

AI coding assistants can write implementation code and tests in minutes. The problem is that both can look correct while the behavior that matters r…

DevOps Email Tests Need a Handoff Contract

DevOps Email Tests Need a Handoff Contract Email tests in CI often fail at the boundary between two jobs: one job creates a disposable inbox, anothe…

How would you approach end-to-end testing a browser extension's functionality and its interaction with a web page using

🔥 Playwright Pro Tip: How do you reliably perform end-to-end testing for browser extensions, especially when they interact with web page content? 📌…

The Uniface test that "hung" had died - finding a silent crash with SQLite

This is a side story from the series Building a customer app in Uniface 10 . It stands on its own: you only need to know that the app is a Uniface 1…

40 features, 872 tests, one SQLite file - how the Uniface test suite stayed honest

The series Building a customer app in Uniface 10 ended its last round with 688 tests in 13 test services. This round added 40 features in one go - t…

How to Test If Your DDoS Protection Actually Works (Before an Attacker Shows You)

Your CDN dashboard says "attack mitigated". Your origin server says otherwise. One of them is lying, and finding out which one — before a real attac…

Weekend Build Log: A Model-Swap Harness for a Side Project on Free Endpoints

Three commands decide whether a weekend side project survives a backend change. The app built this weekend is deliberately small: it turns a build l…

The Cold-Start Budget Packet: A Take-Home Where the Free Server Is Allowed to Vanish

A take-home that never simulates a host restart measures almost nothing about cost discipline. The packet below asks one question: after the machine…

A Green Pipeline That Ran Three of Five Shards Is Not Green

Your merge queue trusts a color. It should trust a coverage claim. A sharded CI run where shard 3 died during setup, shard 5 collected zero tests, a…

Some quick thoughts on Unit Testing ActivityPub

It Had Been Working Fine

Four people apologized. None of them was lying. The Screenshot It started with a screenshot. A user was testing the product. They called an endpoint…

Your pass-rate table under-budgets the verification job in 77% of cases

A planner is handed an obligation set and a deadline and asked one question: how much budget does this need? The field answers a different one — how…

Your retry logic has never seen a real 429. Here is how to test it with a local fault-injecting mock

Most LLM client code has a retry branch that has never actually run. The provider returns 200 in every test, so the 429 , 500 and timeout paths sit…

TypeScript Email Tests Need a Typed Event Boundary

Email tests often start with a simple promise: trigger a signup flow, wait for a message, click the link, and assert the account is active. It sound…