Собираем домашний сервер для локальных LLM на доступных GPU (NVIDIA CMP 50HX)
Привет! Меня зовут Александр Щербаков, я DevOps-инженер в X5 Tech. В одной из прошлых статей я рассказывал о домашних серверах на потребительском же…
DevOps
⚑ Сообщить о проблемеПоследние новости DevOps от Tech News
Building a Distributed GPU Cluster for LLM Inference on Idle Apple Silicon Macs
A single M2 Max Mac Studio can serve 8‑token‑per‑second Llama‑2‑7B inference, yet most developers leave that compute idle overnight. ## Inventory an…
Nodus vs Modal: which is better for GPU training?
Pick Modal if your training jobs are short, sit next to web endpoints, or need one large RDMA cluster. Pick Nodus if your runs last hours or days an…
Spot vs on-demand GPUs: what breaks, and how do you make spot safe?
Short answer (October 10, 2026): spot GPUs are the same hardware as on-demand GPUs at a deep discount (AWS and Azure advertise up to 90% off, GCP up…
Как делят GPU в докладах KubeCon Japan 2026 и у нас в Авито
Привет, Хабр! Меня зовут Антон Алексеев , я ML Ops-инженер в Авито . В конце июля мы с коллегами съездили в Иокогаму на KubeCon + CloudNativeC…
A $22,000 GPU bill with a very quiet night shift
The one I think about most is the account where the infrastructure was correct and the cost was still wrong ML startup. Eight engineers. Production…
"We let an AI agent pay for GPUs with USDC. Here is what happened (2 of 3 delivered, 1 auto-refunded)"
On October 2, 2026, we ran a live verification of the x402 pay-per-job service on Kilawatt Cloud. No simulation. An autonomous agent asked for GPU c…
PPU Zhenwu 810E от Alibaba: как китайский AI-ускоритель справляется с LLM
Привет, я Дмитрий, MLOps-инженер в Selectel . Современный ландшафт AI-ускорителей изобилует всевозможными устройствами: GPU, TPU, NPU, LPU и п…
Backends 101: Choosing the Right Measurement Surface
Why this lesson exists When a program runs on a GPU, the only way to know where the real cost lies is to measure that GPU. Defaulting to the CPU sur…
На чём стоят армянские ИИ-фабрики: энергия и три вектора
15 сентября Армения на десять дней осталась без российского газа — и пережила это на иранском бартере. Именно на этой энергосистеме работают обе ИИ-…
Vast.ai CLI 101: Finding a GPU Offer You Can Actually Use
TL;DR: Start with the workload’s GPU and memory needs, then search for offers with reliability at least 0.97, free inbound traffic, enough disk, and…
Две ИИ-фабрики Армении: что построили и на кого они работают
В августе в Раздане открылась AI-фабрика Firebird: 6144 ускорителя NVIDIA B200 на 15 МВт, стройка заняла чуть больше полугода. Вторая, Eleveight, ра…
Рынок вычислений в России, Казахстане и Беларуси
Сколько вычислительной мощности реально у России, Казахстана и Беларуси — и что из этого доступно CTO прямо сейчас. Прайсы: аренда H100 от 342 ₽/час…
От 6 часов до 20 минут: как мы ускорили коллаборативную модель в рекомендациях Авито
Привет, я Салават Динмухаметов, senior ML-engineer в команде рекомендаций Авито . В статье расскажу, как мы изменили систему рекомендаций на главной…
How Much VRAM Do You Really Need to Run a 70B LLM?
How Much VRAM Do You Really Need for a 70B LLM? Running a large language model locally sounds simple until you start looking at GPU memory. A model…
Why GPU Availability Is Still the Biggest Bottleneck in ML Infra
If you've ever had a training job ready to go and then sat there refreshing your cloud console because there's no GPU capacity available in your reg…
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
Huerta et al., in Dissecting and Modeling the Architecture of Modern GPU Cores , propose that modern NVIDIA GPUs express dependency constraints thro…
Два года, один человек, 66 контейнеров: как я построил AI‑платформу на железе под столом
Привет, Хабр. 26 августа 2024 года я получил у BotFather первый токен и написал кривой телеграм‑бот с одной моделью — обычн…
Nvidia PAIR enables local AI cluster construction
Nvidia has introduced a free software tool that allows users to create AI inferencing clusters using separate computers on a single network. This ut…
From API to GPU, Week 6 (Part 1): A Model That Predicts, and How Wrong It Is
Phase 2 of 8: Enough ML to understand inference. Week 6 of 32, part 1 of 2. Every week so far, the model already existed. I ran Phi-4, read Qwen's f…
Deploying Inference Using NVIDIA Dynamo and vLLM
NVIDIA Dynamo is an open-source, high-throughput, low-latency inference framework for deploying large-scale generative AI and reasoning models acros…
Installing K3s with NVIDIA GPU Operator on Ubuntu 22.04
K3s is a lightweight, fully compliant Kubernetes distribution designed for simplified deployment and operation in resource-constrained environments.…
How Many AI Avatars Can One GPU Handle? Real-World Test Reveals 4 Avatars at ¥7,600 Each per Month
📝 Originally published (in Japanese) at forge.workstyle.tech . Building an Unmanned System for 3D Avatar Live Streaming We're developing an unmanned…
I Tried Getting Closer to the GPU With Triton
I've been trying to understand what actually happens when a piece of ML code reaches the GPU. Not the usual model.cuda() and GPU go brrrr kind of un…
Pure JAX on G5g: Serving Gemma 4 on Graviton and a T4G
This article provides a step by step deployment guide for serving Google's Gemma 4 on an AWS EC2 G5g instance using pure JAX. The code is here: gith…
The Matrix Wasn't A Battery Farm. It Was A GPU Cluster Made Of Human Brains.
Nvidia is worth more than most countries because we cannot figure out how to do cheap inference. There are eight billion units walking around that d…
Latest Trends in GPU Cloud Cost Reduction and Containerized Data Centers
📝 Originally published (in Japanese) at forge.workstyle.tech . Introduction In recent years, the demand for GPUs has surged with the widespread adop…
Deploying DeepSeek R1 Reasoning LLM Using SGLang
DeepSeek R1 is a first-generation reasoning model tuned for math, coding, and logical reasoning — reinforcement learning with a cold-start phase for…
The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell
If you run vLLM with --kv-cache-dtype fp8 on a DeepSeek-family (MLA) model and your GPU is a GB10, an RTX PRO 6000, or any workstation or consumer B…
Stop Comparing GPU Clouds Only by $/hour
GPU Cloud Research · #002 — 11 min read The cheapest GPU instance is not always the cheapest way to finish a workload. GPU cloud comparison pages us…