Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
This article provides a step by step guide to serving Gemma 4 E2B and E4B with Google's QAT assistant drafters through llama.cpp's speculative decod…
Testing & QA
⚑ Сообщить о проблемеПоследние новости Testing & QA от Tech News
Все темы
agents
ai
api
architecture
automation
backend
beginners
career
claude
cybersecurity
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
softwareengineering
testing
tutorial
typescript
webdev
csperf Diff: Comparing Two Runs Without Spreadsheet Chaos
Why this lesson exists In Slack threads you’ll often see two people paste a mean execution time and immediately start a debate. One says “my run is…
Noise Sources: Affinity, Frequency, and Background Load — How to isolate and quantify non‑compiler variance with csperf
Why this lesson exists Unexplained variance in a csperf run often gets blamed on the compiler, when in fact the machine is the culprit. This episode…
Six of seven models contradicted themselves on the same deprecation question
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Whether a model knows which parts of an API still work. Not "is this d…
Kaniko's Performance Gap Narrows: Outdated 2018 Benchmarks Mislead, Recent Updates Close BuildKit Speed Divide
Introduction: Reassessing the Kaniko-BuildKit Performance Gap A 2018 benchmark comparison labeled kaniko as 15 times slower than BuildKit , a result…
The Benchmark That's Half Traps — and Why That's Brilliant
There's a test suite for security scanners where 47% of the test cases are deliberately designed to make your tool look stupid. Not edge cases. Not…
Our AI agents' "verified success" claims: 10 out of 10 failed independent recompute — including ours
We run a five-agent organization that has been operating for 130+ days — agents claim tasks, produce fixes, and report success. Like everyone else,…
A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4
This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the sam…
Memory efficiency: bytes per key
Part 5 of 7 · ← Latency under load · Next: Operational simplicity → Part 5. The shortest post in the series, and the least arguable number in it — b…
Same Model, 13.3% to 38.3%
Two API settings. Same model. Same benchmark. Same task set. 13.3% to 38.3%, using one sixth the output tokens. OpenAI published that result about G…
The Compiler Got 5% Slower. The Benchmark Called It a 10% Regression a Quarter of the Time.
IBM's Benchpress pins the seeds that build its circuits and passes no seed to the code that compiles them. On one circuit, over 400 seeds per versio…
I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured
I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured The setup I was asked to benchmark CognoDB C…
I benchmarked CognoDB against four other graph databases. The most interesting result had nothing to do with CognoDB.
I was asked to benchmark CognoDB Cloud, a managed graph database, against four other graph platforms on the same hardware, the same dataset, and the…
Don't Trust a New Model's Benchmarks Until You Run Your Own 30-Minute Smoke Test
Last week my feed filled with screenshots of MiniMax H3 benchmark results, and every post seemed to reach a different conclusion about whether the r…
DeepSeek Harness: What "Everything is a Plugin" Actually Means for Agent Frameworks
DeepSeek released Harness into developer preview yesterday — MIT license, source on GitHub, and a claim that "everything is a plugin." I spent some…
Native HTTP Engine for Node: Performance Benchmarks Against uWS, Bun, Fastify, and Hono
Overview A native HTTP engine for Node.js has been developed with the goal of providing a faster, built-in alternative to the standard node:http mod…
GuardRate: как мы построили независимую арену для guardrail-моделей (часть 1)
Всем привет, на связи команда HiveTrace! Мы уделяем много времени разработке собственных моделей и часто задаемся вопросом: "какой из двух гар…
Correctness Has a Price: We Benchmarked Fair Leaderboards
Engineering posts often end with: The new design is correct, scalable, and fast. Fast compared with what? When we changed Podium so tied players ran…
.NET Matrix: взвешенный выбор библиотеки, а не по звёздам на GitHub
.NET Matrix — это новый открытый проект, который сравнивает .NET-библиотеки внутри одной категории по трём аспектам: возможности , скорость и исполь…
LiteSpeed vs Nginx for WordPress: 3 months of production benchmarks
TL;DR Ran LiteSpeed Enterprise (LSWS) and Nginx side-by-side on identical VPS instances for 3 months, hosting the same 12 WordPress sites on each. L…
If 30% of Coding Tasks May Be Broken, Your Leaderboard Needs an Uncertainty Budget
OpenAI published an audit of SWE-Bench Pro on July 8, 2026 and estimated that roughly 30% of its tasks are broken. The reported issues make a famili…
Benchmarking Apple's SpeechAnalyzer API vs. Whisper: Performance, Accuracy, and Use Cases
Originally published on tamiz.pro . Introduction With voice interfaces becoming ubiquitous in applications from virtual assistants to transcription…
How I Benchmarked an LLM Running Entirely on a Phone (No Cloud, No API)
"It works on my test input" is the most dangerous sentence in on-device AI development. I typed that sentence - or some version of it - a dozen time…
Your model speed benchmark is measuring the wrong thing
Model speed is not a property of the model. It is a property of the model plus your payload size plus your output format plus whether you're constra…
Как создать свой бенчмарк: 6 уроков с туториала NeurIPS
Посмотрела Туториал NeurIPS «The Art of Benchmarking» — панель с авторами SWE-bench, GPQA и ведущими исследователями из Google DeepMind, NYU и Berke…
Google Said It Had Native Function Calling. I Tested It.
Google released Gemma 4 E4B with a specific claim: native function calling. "Enhanced coding and agentic capabilities," the model card said. "Native…