Ollama Structured Outputs in Practice — Getting Type-Safe JSON from Local LLMs with Pydantic
json.loads(response) fails at a certain point. You told the model "return JSON only," but it added a ```json markdown code fence around everything. A …
Testing & QA
⚑ Report a ProblemLatest Testing & QA news from Tech News
All topics
agents
ai
api
architecture
automation
aws
beginners
career
claude
cybersecurity
database
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
security
showdev
softwareengineering
testing
tutorial
typescript
webdev
Block the Merge if the Model Isn't Ready": Shifting Local AI Evaluations Left with CI Gates
We’ve all heard "it works on my machine," but when it comes to AI-driven features, that phrase is a recipe for disaster. You can have a perfectly test…
Tokenization under the hood: BPE, WordPiece, SentencePiece, and Unigram compared
Tokenization under the hood: BPE, WordPiece, SentencePiece, and Unigram compared You deploy a chatbot. English queries average 42 tokens each. Then a …
AI Evals, Part 3: Golden Datasets That Dont Lie
Part 3 of a series on building production AI on .NET. Part 1 was the overview; Part 2 was error analysis. Now we turn the failure taxonomy you built i…
Stop letting the prompt be your state machine
Stop letting the prompt be your state machine You shipped an LLM feature six months ago. Now the same user input produces wildly different outputs dep…
The HTTP Code Your AI Agent Doesn't Handle Yet: 402
Your fetch agent knows two endings to a request. 200 : parse it. 403 : back off, rotate, or skip. That branch has been the whole game for years. There…
Why multi-agent orchestration is harder than it looks
One AI agent answering a question is useful. Five agents that divide a complex task, pass state to each other, and act on live enterprise systems is a…
Better Models Won't Fix AI Companions
I ran two small tests on AI companion behavior because I wanted to understand a question people keep circling around: Are AI companions bad because th…
RLHF vs DPO vs IPO vs KTO: which alignment method should you use
RLHF vs DPO vs IPO vs KTO: which alignment method should you use You have a base model, say Llama 3.2 8B, that can write poetry in any meter and pass …
How to Build an AI Coding Stack Without Going Broke in 2026
A solo developer with a $200/month budget can now access the same AI coding power that cost enterprises $50,000/month just two years ago. The secret i…
How I Tested 5 Small LLMs on a Weak PC (Intel i5, No GPU) – And Found a Winner
A practical guide to running LLMs on budget hardware: real speeds, real stories, and real conclusions 📌 Table of Contents My Setup (The "Weak" PC) Why…
Moving From Chatbots to Agents: Testing OpenAI Operator
For months, we’ve treated LLMs like fancy autocomplete engines. You prompt, you wait, you copy-paste the output into your terminal. OpenAI’s Operator …
NVIDIA Blackwell Leads AgentPerf, the First Agentic-AI Infra Benchmark: Trajectory-Replay Benchmarking
What: The AgentPerf benchmark from Artificial Analysis is the first test built for agentic-AI infrastructure : instead of timing one chat completion, …
LLM Cost Optimization: How We Cut Reply Generation from $0.011 to $0.0009
When we shipped the first version of AI-generated replies for HelperX , each reply cost us about $0.011 in API spend. That sounds tiny until you multi…
Why most AI apps fail in production (not in demos)
A demo is a story. Production is a stress test. I’ve seen AI apps that feel like magic on a laptop… then crash the moment 10 users show up. Why? Laten…
How to Fine-Tune LLMs on Your Own Data: Open-Source Models, RL Environments, and Evals
If you use LLMs long enough, you hit the same wall. The frontier model is impressive, but it is not always the best model for your job. It may be too …
Run GLM-5.2 Locally: The Open Model Nobody Can Ban
On June 9, Anthropic shipped Claude Fable 5 — the most capable coding model the industry had ever seen. Three days later, the U.S. government ordered …
The Model Context Protocol (MCP): what it is and how to build a server
The Model Context Protocol (MCP): what it is and how to build a server Your team's LLM-powered application talks to a search index through one custom …
We burned 136 million tokens running an autonomous agent studio. Here's how we cut the bill ~90%.
We run a studio where AI agents work mostly unattended — they write code, ship sites, produce content, and keep going without a human in the loop. Run…
The Rule Held. The Boundary Moved Up. AI Memory Judgment, CLAIM-31: Verified Carryover Across Closes
In my last claim, a sequence got allowed that probably should have made you nervous. Thirteen refunds, split across two windows, with a close in betwe…
Making a fleet of self-hosted LLM agents trustworthy
Originally published at llmkube.com/blog/making-self-hosted-llm-agents-trustworthy . Cross-posted here for the dev.to audience. Running a single local…
Show HN: I replaced database deadlock victim selection with an LLM — here's the data
Every mainstream database uses fixed rules for deadlock victim selection. MySQL kills the one with the fewest locks. CockroachDB kills the youngest. P…
AI Claim Verification Pipeline: Stop Hallucinations Before They Reach Customers
AI hallucinations rarely look broken at first glance. They look confident, polished, and ready to ship. That is the dangerous part. A generated report…
NVIDIA RTX Spark Superchip: Unified CPU–GPU Memory
What: NVIDIA's RTX Spark "superchip" (unveiled around Computex / Build 2026) pairs a 20-core Grace CPU with a Blackwell RTX GPU that together address …
Why Prompts Fail in Production (and the 4 Failure Vectors)
Originally published on AI School — free AI & ML courses, no signup. This is lesson 1 of the free course Prompt Patterns That Survive Production .…
We Built a 'Grovel Index' to Measure LLM Sycophancy —Here's What We Found
We Built a "Grovel Index" to Measure LLM Sycophancy —Here's What We Found TL;DR: We spent ~1.2M tokens measuring LLM sycophancy across DeepSeek and Cl…
The self-improving prompt engine that learns from your codebase history
Via v0.4.0: We Built a CLI That Gets Smarter Every Time You Use It We shipped Via v0.4.0 today another weekend project based on utilizing prompt devel…
Apple’s On-Device AI: The Quiet Revolution for Edge Computing and Local-First Apps
The story of AI for the last three years has been written in megawatts. Nvidia GPUs stacked in desert data centers . Models with trillion-parameter co…
The Direction of AI in 2026: Performance, Cost, and the End of One Model for Everything
Six months ago, I could tell you which model to use for almost any job, and I would have said it with confidence. Today I hedge, and so does almost ev…
Google Ships Gemma 4 QAT Checkpoints: Quantization-Aware Training
What: Google shipped quantization-aware-trained (QAT) checkpoints for the Gemma 4 family — open weights that were trained to survive being squeezed do…