The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an RTX PRO 6000, or any workstation or consumer B…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
Qwen3-8B on workstation Blackwell: vLLM vs SGLang vs llama.cpp, plus an FP8 pass
Benchmarks of the same model on the same GPU across three serving stacks, then an FP8 pass on the winner. All numbers measured on our own hardware l…
Serving Gemma4 with Rust on vLLM 🦀
This tutorial walks through installing and setting up the Rust toolchain for vLLM on an AWS EC2 G5g instance — Graviton2 (aarch64) with an NVIDIA T4…
Installing Rust for vLLM on Graviton: a G5g walk-through 🦀
This tutorial walks through installing and setting up the Rust toolchain for vLLM on an AWS EC2 G5g instance — Graviton2 (aarch64) with an NVIDIA T4…
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
A field report on serving Google's Gemma 4 E2B on AWS EC2 **G5g * — a Graviton2 (aarch64) host with an NVIDIA T4G (Turing, SM 7.5) GPU. Three obstac…
GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension
Introduction During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai (2021–2023), I conducted thesis research on multivariate tim…
One RTX 5090 vs a 12-GPU Cluster — Benchmarking a Decade of GPUs on the Same Go Proof
You don't need to know anything about Go to read this. The game is just the fixed yardstick. The story is a hardware benchmark: the same program, th…
Adding GPU backends to a pure-C TTS engine: Metal, CUDA, and the rented-Mac trick
Part of qwen3-tts — a pure C inference engine for Qwen3-TTS. TL;DR The engine is pure C and CPU by default . We added two opt-in GPU backends that l…
Resurrecting Kepler: Getting Modern LLMs Running on a GTX 770 (Kernel 7.x)
⚠️ Experimental hack : Use on non-critical systems. Ensure you have backups. This patches a proprietary binary at the instruction level — no warrant…
TensorCircuit-NG vs cuQuantum on H200: JIT compilation beats the "magic GPU library" assumption
NVIDIA cuQuantum has a strong reputation as the natural high-performance baseline for GPU quantum simulation. That reputation is understandable: cuQ…
Where Tensor-Parallel Inference Hits the NVLink Wall
Where tensor-parallel inference hits the NVLink wall 2026-05-31 · GPU / distributed systems Tensor parallelism splits each layer across GPUs, so eve…