DevOps

Comparing INT4 and NVFP4 Palettes on Real Gradient Tensors

Four-bit training quantizes every number to one of 16 values. NVFP4's menu is {0, ±0.5, ±1, ±1.5, ±2, ±3, ±4, ±6} , with one scale factor per block…

Deploying a QAT Checkpoint Your Serving Stack Can't Load: Gemma 4 E2B in Pure JAX on One TPU

Cloud TPU v6e-1 ( ct6e-standard-1t , one v6e chip, 32 GB HBM), Compute Engine flex-start, europe-west4-a. All timings below measured 2026-08-19 unle…

Quantizing MedGemma to INT4 (GPTQ/W4A16): Everything That Broke Along the Way

Quantized Google's MedGemma-1.5-4B (a medical vision-language model) to INT4 (W4A16) via llm-compressor 's GPTQModifier, for self-hosted deployment.…

LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16 [2026]

Originally published at kunalganglani.com — read it there for inline code, hero image, and live links. LLM Quantization Levels Compared: Q4_K_M vs Q…

Quantization formats compared: GGUF vs GPTQ vs AWQ vs NF4

Quantization formats compared: GGUF vs GPTQ vs AWQ vs NF4 You just finished fine-tuning a 7B parameter model. The raw FP16 weights are 14 GB. Your t…