Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
This article provides a step by step guide to serving Gemma 4 E2B and E4B with Google's QAT assistant drafters through llama.cpp's speculative decod…
Testing & QA
⚑ Report a ProblemLatest Testing & QA news from Tech News
All topics
agents
ai
api
architecture
automation
backend
beginners
career
claude
cybersecurity
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
softwareengineering
testing
tutorial
typescript
webdev
Gemma 4's MTP Drafter, Explained: Speculative Decoding and a 4-Bit Drafter at 60% of the Size
This article provides a step by step introduction to speculative decoding with Gemma 4's multi-token prediction (MTP) drafter, served by llama.cpp o…
Inside My llama.cpp Setup: Tuning Qwen 3.8 27B for 512K Context
Understanding My llama.cpp Qwen 3.8 Configuration I've been tuning llama.cpp for local AI development, and the command line can quickly become a col…
Ollama vs LM Studio 2026: Ollama Wins for Devs
Verdict: in the ollama vs lm studio decision, Ollama wins for developers and automation, and LM Studio wins for everyone else. If you are scripting…
llama.cpp vs Ollama in 2026: Which Runtime Should You Run?
Ollama and llama.cpp are often compared as if they were rival inference engines. The real choice is between a managed model service and a toolkit yo…
Two Rust Clients for Gemma 4: Calling the Endpoint vs. Calling the MCP Server 🦀
This article provides a step by step guide to two small Rust CLIs that ask a self-hosted Gemma 4 E2B the same question. The first calls the model's…
KV Cache on 16 GB GPUs: Making Long Context Actually Fit
A model can advertise a 128K context window and still fail at 40K tokens on a 16 GB GPU. The architecture ceiling never promised that weights, KV ca…
Can Qwen 3.8 running on your laptop really replace Claude Opus for Agentic coding?
Originally published at deepu.tech . When the whole AI coding frenzy started, I was still skeptical. Like most tech enthusiasts, I tried it but wasn…
Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6
This article provides a step by step deployment guide for Gemma 4 E2B's quantization-aware-trained (QAT) checkpoint to a local, laptop hosted GPU en…
Fix Local LLM Quality: Context Stacking & Rope Freq Tweaks
This article was originally published on BuildZn . Everyone's running local LLMs now, which is great. But then they hit the wall: "Why does my 7B mo…
"V cache quantization requires flash_attn" — the llama.cpp error that quietly halves your context window
I did not meet this error while debugging a crash. I met it while writing a calculator. llama_context: quantized V cache requires flash_attn to be e…
Nine ways to talk to a local model
A nine-tool survey of local-model interfaces on one GPU: what worked, what silently failed, and why what sits between you and the model matters more…
Running a 1.5B-Parameter LLM Entirely On-Device for Mental Health — The NilaMind Architecture
How I built a mental health companion that never connects to the internet, and why the most important safety decisions have nothing to do with the A…
llama-bench skipped FA on capable GPUs — b9437 corrects it
What flipped in b9437 Build b9437 , published on May 30, 2026 at 20:56 UTC , ships two targeted default-value corrections to llama-bench . Flash att…
How fast is LlamaStash? Overhead, throughput, and a fair comparison with Ollama and LM Studio
Originally published at deepu.tech . In my release post for LlamaStash I made a claim I need to back up. The wrapper adds zero overhead vs running l…
Benchmarking the Claude Agent SDK on a local LLM: Haiku and Sonnet tier performance
The Claude Agent SDK exposes three budget tiers ( haiku , sonnet , opus ) and reads its routing target from environment variables on every call. Tha…
Qwen 3.6 27B and 35B MTP vs Standard on 16GB GPU
I tested Speculative decoding (Multi-Token Prediction, MTP) performance in Qwen 3.6 27B and 35B on an RTX 4080 with 16 GB VRAM. For a broader view o…