Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
This article provides a step by step guide to serving Gemma 4 E2B and E4B with Google's QAT assistant drafters through llama.cpp's speculative decod…
Последние новости
⚑ Сообщить о проблемеTech news from the best sources
Все темы
- игры
AI
News
Tech
agents
ai
api
architecture
automation
backend
beginners
career
devchallenge
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
csperf Diff: Comparing Two Runs Without Spreadsheet Chaos
Why this lesson exists In Slack threads you’ll often see two people paste a mean execution time and immediately start a debate. One says “my run is…
Noise Sources: Affinity, Frequency, and Background Load — How to isolate and quantify non‑compiler variance with csperf
Why this lesson exists Unexplained variance in a csperf run often gets blamed on the compiler, when in fact the machine is the culprit. This episode…
Kev gives the same answer every time. Until you batch it.
Kev gives the same answer every time. Until you batch it. Part 2 of the jev-decision-models series. Part 1 put seven open-source Jev look-alikes on…
Six of seven models contradicted themselves on the same deprecation question
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Whether a model knows which parts of an API still work. Not "is this d…
Warmup Deep Dive: What You’re Throwing Away and Why
Why this lesson exists Warm‑up runs are the silent hero of performance measurement. When you skip them or treat the first run as the final result, y…
ReviewBench: An open benchmark for AI code review
We’re launching ReviewBench, a benchmark for code review agents built on representative GitHub pull requests, multi-source ground truth, calibrated…
Jev vs small LLMs: does a model that decides beat one that writes?
Key findings Jev tied two small LLMs on accuracy, beat them on probability quality and speed but failed completely at weekday arithmetic. Accuracy i…
MyanmarChemCalc-Bench: Do LLMs Do Chemistry Better in English Than in Burmese?
Benchmark: https://www.kaggle.com/benchmarks/winkoaung/myanmar-chem-calc-bench What tasks did I run? MyanmarChemCalc-Bench — 30 tasks built from 15…
Kaniko's Performance Gap Narrows: Outdated 2018 Benchmarks Mislead, Recent Updates Close BuildKit Speed Divide
Introduction: Reassessing the Kaniko-BuildKit Performance Gap A 2018 benchmark comparison labeled kaniko as 15 times slower than BuildKit , a result…
The Benchmark That's Half Traps — and Why That's Brilliant
There's a test suite for security scanners where 47% of the test cases are deliberately designed to make your tool look stupid. Not edge cases. Not…
Почему одна и та же модель может стоить в пять раз дороже
Исследователи из UC Berkeley и Arena сравнили семь моделей внутри Claude Code, Codex CLI и минималистичного Pi. На двух бенчмарках выбор харнесса по…
Our AI agents' "verified success" claims: 10 out of 10 failed independent recompute — including ours
We run a five-agent organization that has been operating for 130+ days — agents claim tasks, produce fixes, and report success. Like everyone else,…
A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4
This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the sam…
Counting bugs is the hard part of comparing AI review tools
Model benchmarks for code review usually report the same two numbers: how many bugs the tool found, and how many of its comments were actually right…
Operational simplicity: the cross-key tax, quantified
Part 6 of 7 · ← Memory efficiency · Next: When to use which → Part 6. Throughput and latency are what benchmarks measure. This is the thing they usu…
Memory efficiency: bytes per key
Part 5 of 7 · ← Latency under load · Next: Operational simplicity → Part 5. The shortest post in the series, and the least arguable number in it — b…
GuardRate: метрики, бенчмарки и инсайты из первого аудита (часть 2)
В первой части мы верхнеуровнево рассмотрели результаты оценки 37 моделей, обсудили проблему оценки и сравнения guardrail моделей, рассказали, как л…
GuardRate: метрики, бенчмарки и инсайты из первого аудита (часть 2)
В первой части мы верхнеуровнево рассмотрели результаты оценки 37 моделей, обсудили проблему оценки и сравнения guardrail моделей, рассказали, как л…
Same Model, 13.3% to 38.3%
Two API settings. Same model. Same benchmark. Same task set. 13.3% to 38.3%, using one sixth the output tokens. OpenAI published that result about G…
The Compiler Got 5% Slower. The Benchmark Called It a 10% Regression a Quarter of the Time.
IBM's Benchpress pins the seeds that build its circuits and passes no seed to the code that compiles them. On one circuit, over 400 seeds per versio…
Project HydraFusion: Frontier quality via multi-model orchestration
In controlled offline evaluations, HydraFusion’s selective coding workflows matched or exceeded the evaluated Opus 5 baseline while reducing estimat…
Fairness Under the Microscope: Why HY3 Beats Nemotron 3 Ultra on lforla's Bias Stereotypes Audit
Fairness Under the Microscope: Why HY3 Beats Nemotron 3 Ultra on lforla's Bias Stereotypes Audit TL;DR lforla's Bias Stereotypes (A/B Fairness) benc…
I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured
I benchmarked 5 managed graph databases — and the "obvious" winner changed depending on what I measured The setup I was asked to benchmark CognoDB C…
Last night we entered a memory benchmark against Tencent and Mem0. The score isn't back yet - and I'm publishing it either way.
Last night, my AI partner and I built a memory system from scratch, tested it about forty different ways, deployed it to the edge, and submitted it…
I benchmarked CognoDB against four other graph databases. The most interesting result had nothing to do with CognoDB.
I was asked to benchmark CognoDB Cloud, a managed graph database, against four other graph platforms on the same hardware, the same dataset, and the…
I benchmarked 5 graph databases. The first four hours measured the Indian Ocean.
A reproducible comparison of CognoDB Cloud, Neo4j AuraDB, Memgraph, FalkorDB and ArangoDB, and the two things that nearly made it lie. I was asked t…
Don't Trust a New Model's Benchmarks Until You Run Your Own 30-Minute Smoke Test
Last week my feed filled with screenshots of MiniMax H3 benchmark results, and every post seemed to reach a different conclusion about whether the r…
DeepSeek Harness: What "Everything is a Plugin" Actually Means for Agent Frameworks
DeepSeek released Harness into developer preview yesterday — MIT license, source on GitHub, and a claim that "everything is a plugin." I spent some…
Native HTTP Engine for Node: Performance Benchmarks Against uWS, Bun, Fastify, and Hono
Overview A native HTTP engine for Node.js has been developed with the goal of providing a faster, built-in alternative to the standard node:http mod…