95% recall, 80% answers: what LongMemEval retrieval scores don't tell you
Memory layers for AI agents love to quote LongMemEval. You will see 94%, 95%, 96%. Those numbers are real, but they usually measure retrieval: was o…
Open Source
⚑ Report a ProblemLatest Open Source news from Tech News
All topics
AI
agents
ai
api
architecture
automation
backend
beginners
career
database
devchallenge
devops
discuss
javascript
linux
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
Our agentic coding benchmark results are now an open dataset on Hugging Face
I've put our benchmark numbers on Hugging Face, as an open dataset with a small leaderboard Space on top, so anyone can pull the rows and argue with…
Llmspeed Benchmark: Pick the Perfect On‑Device LLM Hardware Fast
Llmspeed Benchmark: Your Practical Guide to Choosing the Right Hardware for On‑Device LLMs Introduction If you’re tired of guessing whether a single…
Kandinsky 6.0 on a 16 GB GPU: the official recipe runs out of memory, and only the 29B model passed my Spanish test
Kandinsky 6.0 generates five seconds of video with its audio , voices included, from a text prompt (MIT license). I wanted to know two things: does…
Half the MCP servers that answer you don't actually work
A dated, reproducible health probe of 78 registry-listed MCP servers. Pennyforge Studio · 2026-10-05 · second dated point 2026-10-06 · repro + raw d…
Больше чем WER: RESON — открытый инструмент и датасет для оценки ASR в телефонном канале
Привет, Хабр! Меня зовут Александр Шевченко, я техлид команды ASR ML (направление ASR/TTS-платформы Audiogram ) в MWS AI. Мы занимаемся совершенство…
Больше чем WER: RESON — открытый инструмент и датасет для оценки ASR в телефонном канале
Привет, Хабр! Меня зовут Александр Шевченко, я техлид команды ASR ML (направление ASR/TTS-платформы Audiogram ) в MWS AI. Мы занимаемся совершенство…
Claude Opus 5.5: 40% cheaper, frontier-grade performance
This week was dominated by cost compression and gateway consolidation. Claude Opus 5.5 dropped with a meaningful price cut and no code changes requi…
Умеет ли Jev быть guard-моделью для LLM?
За эти 2 года опенсорс Guard моделей не менялся принципиально: периодически выходили энкодеры, про которые быстро забывали, но SOTA всегда оставалис…
How LLM Evaluation Actually Works: Inside the Satellite Geo QCM Leaderboard
How LLM Evaluation Actually Works: Inside the Satellite Geo QCM Leaderboard TL;DR: LLM evaluation only produces comparable numbers when every model…
A code review benchmark that isn't the vendor ranking itself
Ask which AI code review tool is best and the answer you get depends on who is publishing it. The deepsource.com listicle ranks CodeRabbit first and…
JetBrains Ranked AI Agents on Real Kotlin Projects. The Token Column Is the Real Story.
JetBrains released the Kotlin Benchmark , its official benchmark for grading AI coding agents on real Kotlin engineering work. Claude Code with Opus…
I Sell Memory APIs. I'm Also Building the Benchmark. Here's How I'm Trying Not to Rig It.
Hey everyone. This time I'll go through what got me started on this benchmark, and the core of how it's actually built. It started from reading comp…
Kaniko vs BuildKit: замеряем время, CPU и память сборки в кластере
В Kubernetes-кластере рано или поздно встаёт вопрос: где собирать Docker image приложений? Вариант «на своей машине разработчика» не мас…
I Tested Q4_K_M vs MXFP4 on the Same Laptop — The Supposedly-Faster New Format Lost
I tested two local LLMs in two different quantization formats on the same laptop, on the same prompt, three trials each. The result is the opposite…
เมื่อ Benchmark โกหกคุณ, SWE-Bench ProMax กับคะแนนจริงที่โมเดลเก่งสุดทำได้แค่ 41.2%
เมื่อ Benchmark โกหกคุณ, SWE-Bench ProMax กับคะแนนจริงที่โมเดลเก่งสุดทำได้แค่ 41.2% โดย Nokka (นก-กา), นักเขียนอิสระสายเทคโนโลยี ผู้เขียนบทความอธิบา…
I Tested 4 LLM APIs for Coding Work — All 20 Runs Passed, But Speed Varies 10x
I tested 3 free LLM API providers and one paid Claude plan on the same 5 real coding tasks. All 20 runs passed on first attempt. The story isn't whe…
Tried `sponsors/wonderwhy-er`: A Practical MCP Server Review
Tried sponsors/wonderwhy-er : A Practical MCP Server Review sponsors/wonderwhy-er is an MCP server that gives Claude a controlled interface to devel…
AxonASP vs. Native IIS ASP: Performance Benchmarks and Engine Architecture
Legacy VBScript and Classic ASP are traditionally bound to the Windows native execution engine ( cscript.exe or w3wp.exe ). AxonASP replaces this de…
We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model.
We ran 160 agent tasks across two frameworks. The frameworks tied. Then we changed the model — and one task failed 20 times in a row. The LangChain…
A LongMemEval-S number you can reproduce
We held off on posting a benchmark for a long time. Not because we didn't have runs - because most memory benchmarks you read are a number with no w…
I Tested GLM-5.3-Flash and Qwen3.8-Flash on 24 Real Tasks
I test-ran both of this week's open-weight flash models against 24 small, real workloads from an actual product stack — structured extraction, SEO m…
We tested "tokenize before you compress" against 452 configurations, and it mostly held up
A few weeks ago my friend @u84u and I ( @ronak-create ) had a simple, slightly annoying question: if LLMs get 30-45% smaller representations of text…
What should an MCP tool return? I ran 72 trials instead of arguing
There's an argument running about MCP right now. You've probably seen it: a 400-point thread called "MCP is dead?" with real token numbers in it, fo…
Benchmarking AI Coding Agents on Real Pull Requests
No synthetic puzzles, no contaminated suites: 25 tasks harvested from PRs merged in 2026 across five languages, graded by each project's own held-ou…
AI Daily Digest — August 1, 2026: ARC-AGI-3 Harness Discovery, EU AI Gigafactories, Devin SWE-1.7
🤖💻 AI Daily Digest — August 1, 2026 OpenAI Shows How Two Harness Settings Tripled ARC-AGI-3 Scores OpenAI published a rare technical deep-dive on Ju…
.NET Matrix: взвешенный выбор библиотеки, а не по звёздам на GitHub
.NET Matrix — это новый открытый проект, который сравнивает .NET-библиотеки внутри одной категории по трём аспектам: возможности , скорость и исполь…
39,000 Torrents: The Bug My Green Benchmark Never Caught
[Torrent911] Inception.2010.TRUEFRENCH.1080p.BluRay.x264-YIFY That's a real filename. The goal: extract "Inception" and "2010" from it, query the TM…
MCPMark v2: InsForge on Sonnet 4.6
Originally published on the InsForge blog , written by Tony Chang (CTO & Co-Founder). Reposted here with permission. In December we published th…
OpenRouter vs Vercel vs LLMGateway Performance
Every AI gateway adds a hop between your app and the model. The question that matters is what that hop costs at the moment your user is staring at a…