My Triton Kernel Ties bitsandbytes — and Both Lose to Math
A fused W4A16 GEMV for batch-1 decoding, measured against the RTX 3060 roofline. Scope note: one RTX 3060 12GB, batch-1 greedy decoding, Qwen2.5-1.5…
Architecture
⚑ Сообщить о проблемеПоследние новости Architecture от Tech News
Все темы
agents
ai
api
architecture
automation
aws
backend
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
news
opensource
performance
productivity
programming
python
react
security
showdev
softwareengineering
systemdesign
tutorial
typescript
webdev
A $22,000 GPU bill with a very quiet night shift
The one I think about most is the account where the infrastructure was correct and the cost was still wrong ML startup. Eight engineers. Production…
How Much VRAM Do You Really Need to Run a 70B LLM?
How Much VRAM Do You Really Need for a 70B LLM? Running a large language model locally sounds simple until you start looking at GPU memory. A model…
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores , propose that modern NVIDIA GPUs express dependency constraints thro…
From API to GPU, Week 6 (Part 2): Watching a Neural Network Learn
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 2 of 2. Part 1 built a one-neuron model that converts Celsius to Fahrenheit, mad…
Deploying Inference Using NVIDIA Dynamo and vLLM
NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework for deploying large-scale generative AI and reasoning models acros…
Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04
K3s is a lightweight, fully compliant Kubernetes distribution designed for simplified deployment and operation in resource-constrained environments.…
The Matrix Wasn't A Battery Farm. It Was A GPU Cluster Made Of Human Brains.
Nvidia is worth more than most countries because we cannot figure out how to do cheap inference. There are eight billion units walking around that d…
Распределённые вычисления: гарантия завершения на исполнителях, которые вам ничего не должны
Если вы ведете pet‑проекты, требующие большого количества вычислений, то наверняка сталкивались с проблемой их - вычислений - оркестрации. Содержать…
[Перевод] Масштабирование LLM: от одного чипа до ЦОДа. Глава 5. Инференс
Предыдущая глава Ну а теперь рассмотрим, как нам применить обученный трансформер. И применение уже обученной модели сильно отличается от тренировки,…
Stop Comparing GPU Clouds Only by $/hour
GPU Cloud Research · #002 — 11 min read The cheapest GPU instance is not always the cheapest way to finish a workload. GPU cloud comparison pages us…
Mastering Low-Precision AI: FP8 and FP4 Support Across Frameworks in Mid-2026
In mid-2026, FP8 and FP4 have become essential tools for making large-scale AI training and inference more efficient . FP8 uses two main formats-E4M…
Rust SIMD Just Came to the GPU — and It Changes How We Think About Parallel Programming
For decades, GPU programming has meant one of two things: writing CUDA kernels in C++ or wrestling with OpenCL. Both require you to think in a funda…
[Перевод] Масштабирование LLM: от одного чипа до ЦОДа. Глава 4. Тренировка трансформера
Наконец-то мы добрались непосредственно до того, как тренировать трансформер, не просто тренировать, а делать это эффективно и масштабируемо. Как мы…
The Same RTX 5090, but the GPU Sat Idle — a CPU-Bound Go Solver and the Case for L2 Cache
This is the second in a short series that benchmarks a single RTX 5090 by re-running published Go solvers — programs that don't just play Go but pro…
WebGPU Explained: The Browser’s New Graphics and Compute Engine
A practical introduction to WebGPU, WGSL, render pipelines, compute shaders, and the future of high-performance graphics on the web. Your browser ca…
GPUs for AI in 2026: NVIDIA, AMD, Intel Compared
The AI hardware landscape has shifted significantly in 2026, with NVIDIA, AMD, and Intel all competing for developers who need GPUs capable of runni…
Linux 7.2 Improves Multi-GPU Displays, M3 Support, Mesa Rusticl Defaults Arm Mali
Linux 7.2 Improves Multi-GPU Displays, M3 Support, Mesa Rusticl Defaults Arm Mali Today's Highlights This week's hardware and driver news highlights…
Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)
TL;DR Short version: no. I dropped a much older GPU ( Quadro P2000, 5GB, Pascal, 2016 ) next to an RTX 3090 (24GB, Ampere) on the same box, ran the…
От Triton Inference Server к NVIDIA Dynamo: как изменился inference для агентов в 2026
Привет, Хабр! Меня зовут Александра, я Data Scientist в компании Рафт. В этой статье я разберу NVIDIA Dynamo — новый open‑source фрей…
Resurrecting Kepler: Getting Modern LLMs Running on a GTX 770 (Kernel 7.x)
⚠️ Experimental hack : Use on non-critical systems. Ensure you have backups. This patches a proprietary binary at the instruction level — no warrant…
96% of cuBLAS, no `unsafe`: what cuTile Rust proves
GPU programming usually asks Rust developers to surrender the borrow checker at the launch boundary: references collapse into raw pointers, and alia…
Blackwell MLPerf Dominance, Intel Nova Lake Compute Runtime, & Weston 16 Vulkan HDR
Blackwell MLPerf Dominance, Intel Nova Lake Compute Runtime, & Weston 16 Vulkan HDR Today's Highlights NVIDIA's Blackwell architecture showcased…
Why You Need to Become a Neuro-Punk Right Now
A short essay on why the developer community should invest as much effort as possible into LLMs that are free from corporations and states. ML resea…
nvidia-smi Reports 97% Utilization While the GPU Sits Idle
TL;DR A GPU shows 97% utilization in nvidia-smi , but training throughput is a fraction of what benchmarks promise. The GPU is not computing; it is…
CUDA for AMD Lemonade, Intel Arc Pro Linux Gains, XPU Manager 2.0
CUDA for AMD Lemonade, Intel Arc Pro Linux Gains, XPU Manager 2.0 Today's Highlights Today's top GPU news highlights include AMD's Lemonade SDK gain…
Flash Attention: what it does and why it matters
Flash Attention: what it does and why it matters Your training job is paying for an A100 at $3/hour. The loss is going down, gradients are flowing,…
Vortex 3.0 RISC-V GPGPU, Pragtical SDL GPU Backend, NVIDIA RTX Spark Launch
Vortex 3.0 RISC-V GPGPU, Pragtical SDL GPU Backend, NVIDIA RTX Spark Launch Today's Highlights Today's top stories highlight significant advancement…
Linux 7.1 Boosts Intel Arc, Flatpak Integrates ROCm, Vintage AMD Driver Refined
Linux 7.1 Boosts Intel Arc, Flatpak Integrates ROCm, Vintage AMD Driver Refined Today's Highlights Recent developments enhance GPU performance and a…
I Tested 9 Serverless GPU Providers for AI Inference in 2026. Here's What I'd Actually Use
TL;DR If you're shipping AI inference and tired of babysitting GPUs, serverless is the way out. You deploy the model, the platform scales it from ze…