Can an LLM draw a high-level architecture diagram? I benchmarked six to find out
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I've been writing design docs and decks at work, and I use LLMs to dra…
DevOps
⚑ Report a ProblemLatest DevOps news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
cloud
database
devchallenge
devops
docker
javascript
kubernetes
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
Agent Recovery on Kaggle: A Submission Attempt with Separate Research Results
This is a submission attempt for the Kaggle Benchmarking Challenge . I am submitting it for review while explicitly disclosing the missing Kaggle-pl…
The Missing Piece: Why AI Models Hallucinate Answers When They Should Abstain
This is a submission for the Kaggle Benchmarking Challenge "The answer isn't always there. Does your AI know that?" Introduction: The "Problem-Solve…
PennyWise Economic Decisions: Benchmarking Frontier LLMs on Financial Reasoning Efficiency
What I Benchmarked I created the PennyWise Economic Decisions benchmark task to evaluate how effectively modern large language models handle complex…
I ported my coding-agent benchmark to Kaggle, and the first bugs I found were mine
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I maintain cli-bench , an open benchmark for coding agents that work i…
I Told Six Vision Models a Safe Photo Was Dangerous. Two of Them Started Seeing Danger.
This is a submission for the Kaggle Benchmarking Challenge TL;DR I asked 6 vision models to name the safety hazards they could actually see in 100 e…
Who Reviews the Reviewers? Benchmarking AI Agents on PR Audits
What I Benchmarked I benchmarked the ability of LLMs to conduct automated pull request (PR) auditing without falling into the "helpful hallucination…
Six of seven models contradicted themselves on the same deprecation question
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Whether a model knows which parts of an API still work. Not "is this d…
Benchmarking LLMs on ETL Logic Synthesis: Can AI Truly Replace Data Pipeline Scripting?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked In enterprise data engineering, migrating visual ETL pipelines (from t…
Breaking the Subword Boundary: Probing Architectural Vulnerabilities in Modern LLMs (ASTRAL-Bench on Kaggle)
This is a submission for the Kaggle Benchmarking Challenge Breaking the Subword Boundary: Architectural Vulnerabilities in Modern LLMs When we look…
DeployGuard: Can AI Diagnose Deployment Logs Without Guessing?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked A deployment log can contain a successful build and a failed rollout.…
The Politeness Trap: When Being Nice Breaks the Instruction Hierarchy
The Politeness Trap: When Being Nice Breaks the Instruction Hierarchy This is a submission for the Kaggle Benchmarking Challenge. What I benchmarked…
Green Tests, Broken Architecture: Four Models Review TypeScript Diffs
This is a submission for the Kaggle Benchmarking Challenge . In ProgressCut, it is possible to write a change that works, passes a narrow test, and…
Phish or Legit: Do LLMs Know When NOT to Cry Wolf?
Kaggle Benchmarking Challenge Submission by Anio Joseph What I Benchmarked Most security benchmarks ask: "Can the model detect the threat?" That's t…
MyanmarChemCalc-Bench: Do LLMs Do Chemistry Better in English Than in Burmese?
Benchmark: https://www.kaggle.com/benchmarks/winkoaung/myanmar-chem-calc-bench What tasks did I run? MyanmarChemCalc-Bench — 30 tasks built from 15…
I Benchmarked 4 Frontier LLMs on Catching ML's "Silent Killers" — DeepSeek-R1 Missed the Most Basic Bug
This is a submission for the Kaggle Benchmarking Challenge Most public AI leaderboards test if a model can write code or pass a syntax check. But in…
Can GPT-5.4 mini handle least-privilege cloud incidents? A 16-case benchmark
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I built a small decision benchmark for cloud operations and incident r…
I showed 30 AI models 336 monitoring dashboards. Most of them can't tell time.
This is a submission for the Kaggle Benchmarking Challenge Most incidents start with a graph. Not a log line, but a panel: a latency line that jumpe…
Day 2: The model I want is the one that's boring everywhere
Kaggle Benchmarking Challenge. Previously: Day 0, the benchmark · Day 1, most of my bugs looked like model behaviour Where we are The benchmark asks…
It Quoted the Failure: Two Kinds of False 'Done'
This is a submission for the Kaggle Benchmarking Challenge Same log, one line changed. Sixteen pieces of work end in a final check. Each is written…
The explanation was right. The policy ID was wrong.
This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked A support model told me the correct fee, named the applicable policy…
I Found 60+ Live Supabase Keys in Public Repos, So I Benchmark Whether LLMs Can Spot Them
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked As part of my day-to-day security work, I scan public GitHub repos for…
I Spun a Wheel of Fortune at 13 AI Models. Here's Who Took the Bait.
This is a submission for the Kaggle Benchmarking Challenge How this was made: this benchmark and post were built end to end by Claude Code, an AI ag…
Promise Is Not Payment: Verification Errors That Amount Accuracy Misses
This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked “I have paid” is a claim. A payment marked PENDING is a state. A suc…
AI models catch bad code, then cry wolf on the good code
I wanted to know one thing. When an AI model reads a data-science tutorial, does it check the code or believe the caption? So I built Blog vs Byteco…
Does an AI Trust Itself More Than It Trusts You? A Benchmark for Belief Attribution
This article is a submission for the Kaggle Benchmarking Challenge . "I told an AI something was true. Later, I showed it an authoritative document…
UMKM-Bench: I tested 8 LLMs on how Indonesians really text online shops
This is a submission for the Kaggle Benchmarking Challenge . What I Benchmarked Here is a message a small coffee shop in Jogja might get on WhatsApp…
Can AI Diagnose a Linux Production Incident Without Making It Worse?
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked I wanted to test a practical question: Can an AI model diagnose a Linu…
100% vuln detection wasn't enough: measuring whether AI respects the patch
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked AI models are often like over-eager alarms. Show them a dangerous word…