Architecture — Tech News

All EN RU

Kimi K3 Open Weights Are Here: How to Self-Host the 2.8T-Parameter Model (Hardware, vLLM, and Data Sovereignty)

Kimi K3 Open Weights Are Here: How to Self-Host the 2.8T-Parameter Model On July 27, 2026, Moonshot AI released the open weights for Kimi K3 -- a 2.8-…

kimik3 openweights selfhostai vllm

What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage

In LLM inference clusters, the core bottleneck for KV Cache storage acceleration often lies not in the storage medium itself, but in network bandwidth…

kvcache lmcache vllm ai

Gemma 4 E2B on a Single TPU v6e Chip: A Serving Deep Dive

Measured 2026-07-21 on vllm/vllm-tpu:nightly (vLLM 0.23.1rc1.dev1076), a GCE flex-start ct6e-standard-1t (one TPU v6e chip, 32 GB HBM) in europe-west4…

tpu llm vllm googlecloud

Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)

TL;DR Short version: no. I dropped a much older GPU ( Quadro P2000, 5GB, Pascal, 2016 ) next to an RTX 3090 (24GB, Ampere) on the same box, ran the sa…

llm ollama vllm gpu

Qwen3.6-35B NVFP4 runs on one H100 — A100 owners are out

NVIDIA published nvidia/Qwen3.6-35B-A3B-NVFP4 on May 28, 2026 — a post-training FP4-quantized variant of Alibaba's 35B MoE model that fits on a single…

qwen3 nvfp4 vllm nvidia

I built an open-source alternative to Microsoft's KAITO that works on ANY Kubernetes cluster

Six months ago, my team needed to deploy DeepSeek-R1 for internal use. We have a Kubernetes cluster — like everyone does in 2026 — so I started lookin…

kubernetes vllm devops opensource

Prefix caching at scale: when it saves you 80% of prefill cost, and the eviction policies that quietly turn it into 5%

Prefix caching at scale: when it saves you 80% of prefill cost, and the eviction policies that quietly turn it into 5% Your chatbot deploys 70B Llama …

llm ai infrastructure vllm

KV cache quantization: what FP8/INT8 K and V actually buy you, and where they break

KV cache quantization: what FP8/INT8 K and V actually buy you, and where they break You just deployed a 70B Llama fine-tune on 8x H100s, and your serv…

llm ai vllm performance

Как мы построили корпоративную LLM-платформу: архитектура, грабли и выводы

Обычно внедрение AI в компаниях происходит по такому сценарию: собрали одного ассистента, показали руководству, получили аплодисменты. Потом второго, …

ai llm openwebui langflow langfuse litellm vllm openai