The benchmark score is the number to trust least
Claude Opus 5.5 scores 58 on Artificial Analysis's Intelligence Index, more than double the median of 26 for comparable reasoning models. That is th…
DevOps
⚑ Report a ProblemLatest DevOps news from Tech News
Rust Coreutils 0.12 vs GNU 9.12: almost everything works the same, starting a process costs nearly 3x more
I measured Rust Coreutils (uutils) 0.12 against GNU coreutils 9.12 in containers with limits: the full GNU test suite, 157 commands with their outpu…
An AI Capture-the-Flag Tournament: What the Scoreboard Counted
Code: Megapixel99/capture-the-flag In April I ran five games of an AI capture-the-flag tournament between five small open-weight models (1.0B to 2.5…
AI News Roundup: Grok 4.5 Hits Tesla, Perplexity's Orchestrator Beats Opus, and Meta Undercuts Pricing
Five stories moved the AI-coding world today. None are about a single model winning forever — they are about the ground shifting under who runs the…
Turn the camera away, and the AI's world freezes
Video AI systems consistently fail to track what happens when the camera looks away: when a scene pans away from an object in motion and returns, cu…
Reliable, and still wrong
A large-scale audit of AI-as-judge evaluation — covering over half a million individual judgments — finds that AI judges are consistently reliable b…
Cross-Machine Memory Query: About 20 Milliseconds, Most Days
I wrote about hardware benchmarks twice this week. Different problem this time. Same machines. I have a Mac for daily work, a Linux box that runs a…
An AMD GPU Beat My Mac on Llama 8B. The Same GPU Lost on Phi-3.
I wrote a post yesterday about why GPUs barely help small text embeddings at batch=1. Different workload, same machines. This time I ran a local LLM…
Your GPU Probably Isn't Helping Your Retrieval System
Most "just use a GPU" advice is wrong for how anyone actually runs small models. I spent yesterday benchmarking a 33M parameter embedding model acro…
pypdf vs PdfPig: Text Extraction at Scale
Overview PDF text extraction is a common pre-processing step in data pipelines — ingesting research papers, legal documents, or reports before embed…
SurrealDB 3.x by the numbers
Author: Tobie Morgan Hitchcock One engine, multi-workloads, full durability. You can explore the full results, methodology, and per-database breakdo…
When Generic Benchmarks Fail: Building a Sales-Domain Evaluation Bench from Scratch
By Natnael Alemseged The gap that τ²-Bench retail cannot measure Tenacious is a B2B sales automation company. Its agent produces outreach emails for…