How much GPU energy does fp16 save? Measured on a Tesla T4
How much GPU energy does fp16 actually save over fp32? Spec sheets give TDP and peak FLOPS, not joules per job, so I measured it on a Tesla T4 using…
Programming
⚑ Сообщить о проблемеПоследние новости Programming от Tech News
Все темы
AI
agents
ai
api
architecture
automation
backend
beginners
career
database
devchallenge
devops
discuss
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
rust
security
showdev
testing
tutorial
typescript
webdev
Kandinsky 6.0 on a 16 GB GPU: the official recipe runs out of memory, and only the 29B model passed my Spanish test
Kandinsky 6.0 generates five seconds of video with its audio , voices included, from a text prompt (MIT license). I wanted to know two things: does…
Building a Distributed GPU Cluster for LLM Inference on Idle Apple Silicon Macs
A single M2 Max Mac Studio can serve 8‑token‑per‑second Llama‑2‑7B inference, yet most developers leave that compute idle overnight. ## Inventory an…
Nodus vs Modal: which is better for GPU training?
Pick Modal if your training jobs are short, sit next to web endpoints, or need one large RDMA cluster. Pick Nodus if your runs last hours or days an…
My Triton Kernel Ties bitsandbytes — and Both Lose to Math
A fused W4A16 GEMV for batch-1 decoding, measured against the RTX 3060 roofline. Scope note: one RTX 3060 12GB, batch-1 greedy decoding, Qwen2.5-1.5…
Spot vs on-demand GPUs: what breaks, and how do you make spot safe?
Short answer (October 10, 2026): spot GPUs are the same hardware as on-demand GPUs at a deep discount (AWS and Azure advertise up to 90% off, GCP up…
Что делать, если доступ к Claude пропал, а на нем уже завязан рабочий процесс?
Привет, Хабр! На связи команда Рег.облака. С начала октября у российских пользователей Claude снова начали блокироваться аккаунты. Для тех, кто поль…
Как мы запустили Qwen3.8–27B целиком на RTX 5060 8 GB и получили ~30 токенов/с
Наш проект называется ExVRAM Lab. Это открытая исследовательская лаборатория, в которой мы проверяем, насколько большие локальные LLM можно зап…
A $22,000 GPU bill with a very quiet night shift
The one I think about most is the account where the infrastructure was correct and the cost was still wrong ML startup. Eight engineers. Production…
"We let an AI agent pay for GPUs with USDC. Here is what happened (2 of 3 delivered, 1 auto-refunded)"
On October 2, 2026, we ran a live verification of the x402 pay-per-job service on Kilawatt Cloud. No simulation. An autonomous agent asked for GPU c…
Backends 101: Choosing the Right Measurement Surface
Why this lesson exists When a program runs on a GPU, the only way to know where the real cost lies is to measure that GPU. Defaulting to the CPU sur…
Что и как меняется в ИТ-инфраструктуре при внедрении ИИ?
Привет, Хабр! Меня зовут Владислав Шершульский, я советник генерального директора Группы Rubytech по инновационным проектам и ИИ. Отвечая на вопрос,…
Vast.ai CLI 101: Finding a GPU Offer You Can Actually Use
TL;DR: Start with the workload’s GPU and memory needs, then search for offers with reliability at least 0.97, free inbound traffic, enough disk, and…
How to pick --n-cpu-moe in llama.cpp: Qwen3.6 35B-A3B on 12, 16 and 24 GB GPUs
Mixture-of-experts models like Qwen3.6 35B-A3B, gpt-oss or GLM Flash are too big for most consumer GPUs, but they only read a few experts per token.…
When GPU Resources Run Dry, It Still Looks Like Everything Is Working: A Tale of Two AI Competing for VRAM
📝 Originally published (in Japanese) at forge.workstyle.tech . On the same machine (1 GPU, 12GB VRAM), two Claude Code sessions were running separat…
Рынок вычислений в России, Казахстане и Беларуси
Сколько вычислительной мощности реально у России, Казахстана и Беларуси — и что из этого доступно CTO прямо сейчас. Прайсы: аренда H100 от 342 ₽/час…
How Much VRAM Do You Really Need to Run a 70B LLM?
How Much VRAM Do You Really Need for a 70B LLM? Running a large language model locally sounds simple until you start looking at GPU memory. A model…
Why GPU Availability Is Still the Biggest Bottleneck in ML Infra
If you've ever had a training job ready to go and then sat there refreshing your cloud console because there's no GPU capacity available in your reg…
Same nvJPEG2000, different numbers: timer boundaries and frames in flight
This is an excerpt from a longer write-up, Fastvideo JPEG2000 vs nvJPEG2000 on NVIDIA RTX 4090 , which compares two GPU JPEG2000 codecs at the same…
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores , propose that modern NVIDIA GPUs express dependency constraints thro…
A 4 GB Laptop GPU Beats a 12-Core CPU by 4.3x on Gemma 4
This article compares two ways of serving the same small language model on the same laptop: CPU-only, and on the 4 GB GTX 1650 Ti sitting in the sam…
Вместо конвертации ассетов перепишу с нуля рантайм игры
Допустим, вы хотите перенести модельку из одной игры в другую. Классический путь — извлечь, перегнать в промежуточный формат, ад…
Inside SFPU Overflow Bugs: How a 40-Year-Old Rounding Trick Breaks on Modern AI Accelerators
Inside SFPU Overflow Bugs: How a 40-Year-Old Rounding Trick Breaks on Modern AI Accelerators The Bug ttnn.softplus(-1e7) returns +inf . Not approxim…
Name the Blackwell serving cell you are actually in
On sm_120 and sm_121, a model that will not serve often fails in one cell: one runtime version, one MoE or GEMM backend, one quantization, one topol…
Я прогнал топ VRAM-калькуляторов для LLM — все ошибаются в одном и том же
Наивная формула «веса + KV < VRAM» промахивается по двум причинам, и обе стоят денег. Первая: движок (vLLM, SGLang, TensorRT-LLM) резервирует поч…
Nvidia PAIR enables local AI cluster construction
Nvidia has introduced a free software tool that allows users to create AI inferencing clusters using separate computers on a single network. This ut…
From API to GPU, Week 6 (Part 2): Watching a Neural Network Learn
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 2 of 2. Part 1 built a one-neuron model that converts Celsius to Fahrenheit, mad…
From API to GPU, Week 6 (Part 1): A Model That Predicts, and How Wrong It Is
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 1 of 2. Every week so far, the model already existed. I ran Phi-4, read Qwen's f…
Deploying Inference Using NVIDIA Dynamo and vLLM
NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework for deploying large-scale generative AI and reasoning models acros…
Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04
K3s is a lightweight, fully compliant Kubernetes distribution designed for simplified deployment and operation in resource-constrained environments.…