95% recall, 80% answers: what LongMemEval retrieval scores don't tell you
Memory layers for AI agents love to quote LongMemEval. You will see 94%, 95%, 96%. Those numbers are real, but they usually measure retrieval: was o…
DevOps
⚑ Report a ProblemLatest DevOps news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
cloud
database
devchallenge
devops
docker
javascript
kubernetes
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
Llmspeed Benchmark: Pick the Perfect On‑Device LLM Hardware Fast
Llmspeed Benchmark: Your Practical Guide to Choosing the Right Hardware for On‑Device LLMs Introduction If you’re tired of guessing whether a single…
Приниматоры решений: бенчмарк на русском для Jev-style моделей
Последние пару лет мы довольно странно используем большие языковые модели. Из-за их умения делать всё мы пытаемся решить с их помощью всё — роутинг…
Half the MCP servers that answer you don't actually work
A dated, reproducible health probe of 78 registry-listed MCP servers. Pennyforge Studio · 2026-10-05 · second dated point 2026-10-06 · repro + raw d…
Claude Opus 5.5: 40% cheaper, frontier-grade performance
This week was dominated by cost compression and gateway consolidation. Claude Opus 5.5 dropped with a meaningful price cut and no code changes requi…
How LLM Evaluation Actually Works: Inside the Satellite Geo QCM Leaderboard
How LLM Evaluation Actually Works: Inside the Satellite Geo QCM Leaderboard TL;DR: LLM evaluation only produces comparable numbers when every model…
JetBrains Ranked AI Agents on Real Kotlin Projects. The Token Column Is the Real Story.
JetBrains released the Kotlin Benchmark , its official benchmark for grading AI coding agents on real Kotlin engineering work. Claude Code with Opus…
Kaniko vs BuildKit: замеряем время, CPU и память сборки в кластере
В Kubernetes-кластере рано или поздно встаёт вопрос: где собирать Docker image приложений? Вариант «на своей машине разработчика» не мас…
ช่องว่าง 0.3% แต่ราคาต่าง 2 เท่า, อ่านตาราง Terminal-Bench 4.0 ให้เป็น
ช่องว่าง 0.3% แต่ราคาต่าง 2 เท่า, อ่านตาราง Terminal-Bench 4.0 ให้เป็น โดย Nokka (นก-กา), นักเขียนอิสระสายเทคโนโลยี ผู้เขียนบทความอธิบายเทคโนโลยีให้…
เมื่อ Benchmark โกหกคุณ, SWE-Bench ProMax กับคะแนนจริงที่โมเดลเก่งสุดทำได้แค่ 41.2%
เมื่อ Benchmark โกหกคุณ, SWE-Bench ProMax กับคะแนนจริงที่โมเดลเก่งสุดทำได้แค่ 41.2% โดย Nokka (นก-กา), นักเขียนอิสระสายเทคโนโลยี ผู้เขียนบทความอธิบา…
Tried `sponsors/wonderwhy-er`: A Practical MCP Server Review
Tried sponsors/wonderwhy-er : A Practical MCP Server Review sponsors/wonderwhy-er is an MCP server that gives Claude a controlled interface to devel…
AxonASP vs. Native IIS ASP: Performance Benchmarks and Engine Architecture
Legacy VBScript and Classic ASP are traditionally bound to the Windows native execution engine ( cscript.exe or w3wp.exe ). AxonASP replaces this de…
We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model.
We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model — and one task failed 20 times in a row. The LangChain…
A LongMemEval-S number you can reproduce
We held off on posting a benchmark for a long time. Not because we didn't have runs - because most memory benchmarks you read are a number with no w…
We tested "tokenize before you compress" against 452 configurations, and it mostly held up
A few weeks ago my friend @u84u and I ( @ronak-create ) had a simple, slightly annoying question: if LLMs get 30-45% smaller representations of text…
What should an MCP tool return? I ran 72 trials instead of arguing
There's an argument running about MCP right now. You've probably seen it: a 400-point thread called "MCP is dead?" with real token numbers in it, fo…
Benchmarking AI Coding Agents on Real Pull Requests
No synthetic puzzles, no contaminated suites: 25 tasks harvested from PRs merged in 2026 across five languages, graded by each project's own held-ou…
AI Daily Digest — August 1, 2026: ARC-AGI-3 Harness Discovery, EU AI Gigafactories, Devin SWE-1.7
🤖💻 AI Daily Digest — August 1, 2026 OpenAI Shows How Two Harness Settings Tripled ARC-AGI-3 Scores OpenAI published a rare technical deep-dive on Ju…
39,000 Torrents: The Bug My Green Benchmark Never Caught
[Torrent911] Inception.2010.TRUEFRENCH.1080p.BluRay.x264-YIFY That's a real filename. The goal: extract "Inception" and "2010" from it, query the TM…
OpenRouter vs Vercel vs LLMGateway Performance
Every AI gateway adds a hop between your app and the model. The question that matters is what that hop costs at the moment your user is staring at a…
Can You Beat an LLM? Building Humans vs. Humanity's Last Exam
Frontier models are crushing benchmark after benchmark...so I built a quiz to ask this simple yet humbling question: can a human still beat them? Hu…
Model Showdown Round 9: Qwen 3.6 27B vs Qwen 3.6 35B-A3B vs Qwythos-9B vs GLM-4.7-Flash vs Nemotron-3-Nano
Round 7 ended on a cliffhanger I couldn't stop thinking about. Qwen 3.6 35B-A3B built the entire feature — read the codebase, wrote the files, got a…
AdvancedMathBench: A New Benchmark for LLM Advanced Mathematical Reasoning
What Changed Large language models (LLMs) have demonstrated proficiency in high-school and olympiad-style mathematics. However, their performance in…
Which LLM should I actually code with? I built a small benchmark to find out
LLM code benchmark — Peculiar Engineer A small, self-run coding benchmark: 3 models on 14 problems across 3 languages, scored on pass@k, cost, and s…
I Benchmarked 42 Compression Formats Spanning Four Decades. Here's What to Actually Use.
I run ezyZip , a browser-based archive tool, so "which format should I use?" is a question I field constantly. The honest answer is usually "it depe…
AI Coding Tools Benchmark 2026: Cursor vs Copilot vs Windsurf vs Claude Code
I spent two weeks testing Cursor, GitHub Copilot, Windsurf, and Claude Code on the same set of tasks. Not vibes. Not feature lists. Actual work: bui…
GLM Is the New Hotness, So Let's Test It On the Homelab
GLM is the new hotness. I'm hearing it from both sides of the AI builder world. Software engineers are talking about it because the benchmark number…
Too cheap to be good? Think again.
For years, I ran my WordPress sites on OpenLiteSpeed. Fast server, LSCache is genuinely impressive, and the OLS/WordPress combo is hard to beat on r…
A UMAP With Arrows Is Not a Benchmark. This Is
How I built a three-task evaluation framework for RNA velocity trajectory inference -- measuring global ordering, pairwise rank preservation, and ro…
Engineering CellFateBench: A Reproducible Python Benchmark for Single-Cell Genomics Reasoning
CellFateBench is a scientific software and benchmark-engineering project for evaluating reasoning over single-cell genomics workflows. The project w…