Fix Local LLM Quality: Context Stacking & Rope Freq Tweaks
This article was originally published on BuildZn . Everyone's running local LLMs now, which is great. But then they hit the wall: "Why does my 7B mo…
Последние новости
⚑ Сообщить о проблемеTech news from the best sources
Все темы
- игры
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
"V cache quantization requires flash_attn" — the llama.cpp error that quietly halves your context window
I did not meet this error while debugging a crash. I met it while writing a calculator. llama_context: quantized V cache requires flash_attn to be e…
Nine ways to talk to a local model
A nine-tool survey of local-model interfaces on one GPU: what worked, what silently failed, and why what sits between you and the model matters more…
Running a 1.5B-Parameter LLM Entirely On-Device for Mental Health — The NilaMind Architecture
How I built a mental health companion that never connects to the internet, and why the most important safety decisions have nothing to do with the A…
llama-bench skipped FA on capable GPUs — b9437 corrects it
What flipped in b9437 Build b9437 , published on May 30, 2026 at 20:56 UTC , ships two targeted default-value corrections to llama-bench . Flash att…
How to Tune llama.cpp --n-gpu-layers: A Practical VRAM Guide (2026)
You already know what --n-gpu-layers does. It moves transformer layers onto your GPU. This post is the next step: how to actually pick the number. I…
How fast is LlamaStash? Overhead, throughput, and a fair comparison with Ollama and LM Studio
Originally published at deepu.tech . In my release post for LlamaStash I made a claim I need to back up. The wrapper adds zero overhead vs running l…
Benchmarking the Claude Agent SDK on a local LLM: Haiku and Sonnet tier performance
The Claude Agent SDK exposes three budget tiers ( haiku , sonnet , opus ) and reads its routing target from environment variables on every call. Tha…
Qwen 3.6 27B and 35B MTP vs Standard on 16GB GPU
I tested Speculative decoding (Multi-Token Prediction, MTP) performance in Qwen 3.6 27B and 35B on an RTX 4080 with 16 GB VRAM. For a broader view o…
Ollama vs llama.cpp vs vLLM: Which Should You Use in 2026?
From the Best GPU for LLM archive. The canonical version has interactive calculators, an up-to-date GPU comparison table, and live pricing. Three to…
Discontinued Optane Local LLM Powers a Kimi K2.5 Desktop Run
A user on r/LocalLLaMA reported on May 12 that an Optane local LLM desktop build ran Moonshot’s Kimi K2.5 at about 4 tokens per second using discont…