How much GPU energy does fp16 save? Measured on a Tesla T4
How much GPU energy does fp16 actually save over fp32? Spec sheets give TDP and peak FLOPS, not joules per job, so I measured it on a Tesla T4 using…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
Kandinsky 6.0 on a 16 GB GPU: the official recipe runs out of memory, and only the 29B model passed my Spanish test
Kandinsky 6.0 generates five seconds of video with its audio , voices included, from a text prompt (MIT license). I wanted to know two things: does…
Building a Distributed GPU Cluster for LLM Inference on Idle Apple Silicon Macs
A single M2 Max Mac Studio can serve 8‑token‑per‑second Llama‑2‑7B inference, yet most developers leave that compute idle overnight. ## Inventory an…
Nodus vs Modal: which is better for GPU training?
Pick Modal if your training jobs are short, sit next to web endpoints, or need one large RDMA cluster. Pick Nodus if your runs last hours or days an…
My Triton Kernel Ties bitsandbytes — and Both Lose to Math
A fused W4A16 GEMV for batch-1 decoding, measured against the RTX 3060 roofline. Scope note: one RTX 3060 12GB, batch-1 greedy decoding, Qwen2.5-1.5…
Spot vs on-demand GPUs: what breaks, and how do you make spot safe?
Short answer (October 10, 2026): spot GPUs are the same hardware as on-demand GPUs at a deep discount (AWS and Azure advertise up to 90% off, GCP up…
A $22,000 GPU bill with a very quiet night shift
The one I think about most is the account where the infrastructure was correct and the cost was still wrong ML startup. Eight engineers. Production…
"We let an AI agent pay for GPUs with USDC. Here is what happened (2 of 3 delivered, 1 auto-refunded)"
On October 2, 2026, we ran a live verification of the x402 pay-per-job service on Kilawatt Cloud. No simulation. An autonomous agent asked for GPU c…
Backends 101: Choosing the Right Measurement Surface
Why this lesson exists When a program runs on a GPU, the only way to know where the real cost lies is to measure that GPU. Defaulting to the CPU sur…
Vast.ai CLI 101: Finding a GPU Offer You Can Actually Use
TL;DR: Start with the workload’s GPU and memory needs, then search for offers with reliability at least 0.97, free inbound traffic, enough disk, and…
How to pick --n-cpu-moe in llama.cpp: Qwen3.6 35B-A3B on 12, 16 and 24 GB GPUs
Mixture-of-experts models like Qwen3.6 35B-A3B, gpt-oss or GLM Flash are too big for most consumer GPUs, but they only read a few experts per token.…
When GPU Resources Run Dry, It Still Looks Like Everything Is Working: A Tale of Two AI Competing for VRAM
📝 Originally published (in Japanese) at forge.workstyle.tech . On the same machine (1 GPU, 12GB VRAM), two Claude Code sessions were running separat…
How Much VRAM Do You Really Need to Run a 70B LLM?
How Much VRAM Do You Really Need for a 70B LLM? Running a large language model locally sounds simple until you start looking at GPU memory. A model…
Why GPU Availability Is Still the Biggest Bottleneck in ML Infra
If you've ever had a training job ready to go and then sat there refreshing your cloud console because there's no GPU capacity available in your reg…
Same nvJPEG2000, different numbers: timer boundaries and frames in flight
This is an excerpt from a longer write-up, Fastvideo JPEG2000 vs nvJPEG2000 on NVIDIA RTX 4090 , which compares two GPU JPEG2000 codecs at the same…
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores , propose that modern NVIDIA GPUs express dependency constraints thro…
A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4
This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the sam…
Inside SFPU Overflow Bugs: How a 40-Year-Old Rounding Trick Breaks on Modern AI Accelerators
Inside SFPU Overflow Bugs: How a 40-Year-Old Rounding Trick Breaks on Modern AI Accelerators The Bug ttnn.softplus(-1e7) returns +inf . Not approxim…
Name the Blackwell serving cell you are actually in
On sm_120 and sm_121, a model that will not serve often fails in one cell: one runtime version, one MoE or GEMM backend, one quantization, one topol…
Nvidia PAIR enables local AI cluster construction
Nvidia has introduced a free software tool that allows users to create AI inferencing clusters using separate computers on a single network. This ut…
From API to GPU, Week 6 (Part 2): Watching a Neural Network Learn
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 2 of 2. Part 1 built a one-neuron model that converts Celsius to Fahrenheit, mad…
From API to GPU, Week 6 (Part 1): A Model That Predicts, and How Wrong It Is
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 1 of 2. Every week so far, the model already existed. I ran Phi-4, read Qwen's f…
Deploying Inference Using NVIDIA Dynamo and vLLM
NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework for deploying large-scale generative AI and reasoning models acros…
Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04
K3s is a lightweight, fully compliant Kubernetes distribution designed for simplified deployment and operation in resource-constrained environments.…
How Many AI Avatars Can One GPU Handle? Real-World Test Reveals 4 Avatars at ¥7,600 Each per Month
📝 Originally published (in Japanese) at forge.workstyle.tech . Building an Unmanned System for 3D Avatar Live Streaming We're developing an unmanned…
I Tried Getting Closer to the GPU With Triton
I've been trying to understand what actually happens when a piece of ML code reaches the GPU. Not the usual model.cuda() and GPU go brrrr kind of un…
Pure JAX on G5g: Serving Gemma 4 on Graviton and a T4G
This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g instance using pure JAX. The code is here: gith…
The Matrix Wasn't A Battery Farm. It Was A GPU Cluster Made Of Human Brains.
Nvidia is worth more than most countries because we cannot figure out how to do cheap inference. There are eight billion units walking around that d…
Latest Trends in GPU Cloud Cost Reduction and Containerized Data Centers
📝 Originally published (in Japanese) at forge.workstyle.tech . Introduction In recent years, the demand for GPUs has surged with the widespread adop…
Deploying DeepSeek R1 Reasoning LLM Using SGLang
DeepSeek R1 is a first-generation reasoning model tuned for math, coding, and logical reasoning — reinforcement learning with a cold-start phase for…