Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs
This article provides a step by step guide to serving Gemma 4 E2B and E4B with Google's QAT assistant drafters through llama.cpp's speculative decod…
Testing & QA
⚑ Report a ProblemLatest Testing & QA news from Tech News
All topics
agents
ai
api
architecture
automation
backend
beginners
career
claude
cybersecurity
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
softwareengineering
testing
tutorial
typescript
webdev
How Fast Can a 421M-Parameter Decision Model Run? I Benchmarked Laya Across NVIDIA GPUs
One H100 NVL. A 421M-parameter decision model. 15.1 million decisions per day while staying inside a p99 ≤ 130 ms latency budget. That number sounds…
Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It
This article provides a step by step deployment guide for **Gemma 4 E2B * to a Tesla T4 hosted GPU enabled system. A suite of Python MCP tools is bu…
Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16
This article provides a step by step deployment guide for **Gemma 4 E2B * to a Tesla T4 hosted GPU enabled system. A suite of Python MCP tools is bu…
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores , propose that modern NVIDIA GPUs express dependency constraints thro…
Gemma 4 on an 2021 4 GB Laptop GPU: QAT Takes It From 9.5 GiB to 1.6
This article provides a step by step deployment guide for Gemma 4 E2B's quantization-aware-trained (QAT) checkpoint to a local, laptop hosted GPU en…
g5g vs g6 for LLM Serving: the Same Code, and 3.7x the Throughput
This article compares two AWS GPU instance families for serving a small language model, using a payload that is byte-identical on both. The older fa…
Qwen3-8B on workstation Blackwell: vLLM vs SGLang vs llama.cpp, plus an FP8 pass
Benchmarks of the same model on the same GPU across three serving stacks, then an FP8 pass on the winner. All numbers measured on our own hardware l…
Serving Gemma4 with Rust on vLLM 🦀
This tutorial walks through installing and setting up the Rust toolchain for vLLM on an AWS EC2 G5g instance — Graviton2 (aarch64) with an NVIDIA T4…
Installing Rust for vLLM on Graviton: a G5g walk-through 🦀
This tutorial walks through installing and setting up the Rust toolchain for vLLM on an AWS EC2 G5g instance — Graviton2 (aarch64) with an NVIDIA T4…
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
A field report on serving Google's Gemma 4 E2B on AWS EC2 **G5g * — a Graviton2 (aarch64) host with an NVIDIA T4G (Turing, SM 7.5) GPU. Three obstac…
GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension
Introduction During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai (2021–2023), I conducted thesis research on multivariate tim…
Adding GPU backends to a pure-C TTS engine: Metal, CUDA, and the rented-Mac trick
Part of qwen3-tts — a pure C inference engine for Qwen3-TTS. TL;DR The engine is pure C and CPU by default . We added two opt-in GPU backends that l…
cuda-core 1.0 — пишем CUDA-ядра на Python без C++ (ну почти)
11 мая 2026 года NVIDIA выпустила cuda-core v1.0.0 — первый стабильный релиз библиотеки, которая даёт Python-разработчикам прямой доступ к CUDA Runt…