Can an LLM draw a high-level architecture diagram? I benchmarked six to find out
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I've been writing design docs and decks at work, and I use LLMs to dra…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
- игры
AI
News
Tech
agents
ai
api
architecture
automation
backend
beginners
career
devchallenge
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
Does AI get worse with more documents? Open-source vs paid models, tested.
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Lots of companies now hand an AI model a stack of their own documents…
Agent Recovery on Kaggle: A Submission Attempt with Separate Research Results
This is a submission attempt for the Kaggle Benchmarking Challenge . I am submitting it for review while explicitly disclosing the missing Kaggle-pl…
Can Modern VLMs Actually Reason Through Spatial Grid Puzzles?
What I Benchmarked Current Frontier Visual Language Models (VLMs) excel at multi-modal reasoning and captioning, but how well do they execute step-b…
Drafts That Never Shipped: do LLMs treat dead proposals as real standards?
This is a submission for the Kaggle Benchmarking Challenge . 🎥 90-second overview: What I Benchmarked Standards have a long tail of proposals that n…
I asked AI models to build web apps. Most of them shipped a fake server inside the page.
This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked AI coding demos show the happy path: the form renders, you click the…
The Last Responsible Moment: All Three Models Passed—So What Did the Benchmark Actually Measure?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked A coding agent can generate a valid command and still make the wrong d…
The Missing Piece: Why AI Models Hallucinate Answers When They Should Abstain
This is a submission for the Kaggle Benchmarking Challenge "The answer isn't always there. Does your AI know that?" Introduction: The "Problem-Solve…
PennyWise Economic Decisions: Benchmarking Frontier LLMs on Financial Reasoning Efficiency
What I Benchmarked I created the PennyWise Economic Decisions benchmark task to evaluate how effectively modern large language models handle complex…
I ported my coding-agent benchmark to Kaggle, and the first bugs I found were mine
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I maintain cli-bench , an open benchmark for coding agents that work i…
I Told Six Vision Models a Safe Photo Was Dangerous. Two of Them Started Seeing Danger.
This is a submission for the Kaggle Benchmarking Challenge TL;DR I asked 6 vision models to name the safety hazards they could actually see in 100 e…
Who Reviews the Reviewers? Benchmarking AI Agents on PR Audits
What I Benchmarked I benchmarked the ability of LLMs to conduct automated pull request (PR) auditing without falling into the "helpful hallucination…
Six of seven models contradicted themselves on the same deprecation question
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Whether a model knows which parts of an API still work. Not "is this d…
Three Engines, One Informed Choice: Benchmarking Orbit
🎧 Listen to the podcast Why Detailed Instructions Paralyze AI — audio companion · 22:31 Open the podcast in another window so you can listen while r…
Benchmarking LLMs on ETL Logic Synthesis: Can AI Truly Replace Data Pipeline Scripting?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked In enterprise data engineering, migrating visual ETL pipelines (from t…
Breaking the Subword Boundary: Probing Architectural Vulnerabilities in Modern LLMs (ASTRAL-Bench on Kaggle)
This is a submission for the Kaggle Benchmarking Challenge Breaking the Subword Boundary: Architectural Vulnerabilities in Modern LLMs When we look…
DeployGuard: Can AI Diagnose Deployment Logs Without Guessing?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked A deployment log can contain a successful build and a failed rollout.…
I showed AI the buggy code. Its tests started protecting the bug.
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I've been a QA engineer since 2023. The most dangerous test suite I've…
Can an AI Count a Music Loop? 24 Audio Arithmetic Checks
This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked An assistant can explain sample rates beautifully and still return t…
Can AI Return a Copy-Ready Spreadsheet Formula? I Tested 3 Models
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I tested whether language models can update Excel-style cell reference…
Changing what my phone agent was shown beat changing the model
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked On 18 September 2026, two turns after its own lookup had returned a bo…
The Politeness Trap: When Being Nice Breaks the Instruction Hierarchy
The Politeness Trap: When Being Nice Breaks the Instruction Hierarchy This is a submission for the Kaggle Benchmarking Challenge. What I benchmarked…
Asked to undo a commit, 19 of 36 models destroyed work nobody mentioned
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Coding agents run shell commands on their own now. The stories that ci…
Green Tests, Broken Architecture: Four Models Review TypeScript Diffs
This is a submission for the Kaggle Benchmarking Challenge . In ProgressCut, it is possible to write a change that works, passes a narrow test, and…
Asked in Dutch, sent to an English menu
This is a submission for the Kaggle Benchmarking Challenge . A Dutch reader asks where, in the back office, they set the colors of their app's inter…
FairHire-ES: When Spanish résumé scores flip with the name — and what a 0–100 scale fix changed
Kaggle Benchmarking Challenge Submission This is a submission for the Kaggle Benchmarking Challenge . Data note: Every résumé, name, and profile her…
Phish or Legit: Do LLMs Know When NOT to Cry Wolf?
Kaggle Benchmarking Challenge Submission by Anio Joseph What I Benchmarked Most security benchmarks ask: "Can the model detect the threat?" That's t…
Benchmarking Bare-Metal Tool Use: Do LLMs Understand Apple Silicon L1 Cache?
This is a submission for the Kaggle Benchmarking Challenge. What I Benchmarked I set out to measure Hardware-Aware Code Generation. Most AI benchmar…
MyanmarChemCalc-Bench: Do LLMs Do Chemistry Better in English Than in Burmese?
Benchmark: https://www.kaggle.com/benchmarks/winkoaung/myanmar-chem-calc-bench What tasks did I run? MyanmarChemCalc-Bench — 30 tasks built from 15…
I Benchmarked 4 Frontier LLMs on Catching ML's "Silent Killers" — DeepSeek-R1 Missed the Most Basic Bug
This is a submission for the Kaggle Benchmarking Challenge Most public AI leaderboards test if a model can write code or pass a syntax check. But in…