My First Paying Customer Failed 4 Times: Quality Is Not a Final Check
A 0.3-second disagreement between two sources of truth made my first paying customer fail four times. The browser preview stored the project duratio…
Latest News
⚑ Report a ProblemTech news from the best sources
All topics
- игры
AI
Gear
News
Tech
agents
ai
api
architecture
automation
beginners
career
database
devops
javascript
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
security
showdev
testing
tutorial
typescript
webdev
# От firing до postmortem: рабочее место дежурного поверх Grafana и Mattermost
Grafana показывала, что горит, Mattermost доставлял уведомления, но после смены дежурного приходилось заново выяснять, кто занимается проблемой и ко…
My Auto-Publish Pipeline Shipped a Two-Year-Old News Story. Here's the Fix — All Three Layers of It.
I run a content pipeline that finds industry news, has an LLM judge score it, and publishes what clears the bar. Fully automated, Cloudflare Workers…
Incident Retrospectives Without Blame
I've run over 100 post-mortems. The worst ones end with 'Alice will be more careful.' The best ones end with 'we fixed the system.' Here's how you g…
Post-Mortem Best Practices That Actually Drive Change
The Post-Mortem Nobody Learns From I've sat through hundreds of post-mortems. Most follow the same pattern: something breaks, someone writes a Googl…
Как reload ingress-nginx уронил прод. История про pids.max, threads-max и виртуалку, которая подросла
Схема входящего траффика в кластер Kubernetes простая: web → Envoy Gateway → Ingress Nginx → backend . За Ingress Nginx, помимо обычного HTTP, живут…
Переименовал две колонки и поймал два инцидента
Про безопасные миграции написано уже тысячу раз. Мы все наизусть знаем и про expand/contract, и про обратную совместимость, и про то, что схему нель…
The Day I Broke Production Because of an Icon
I'm a big fan of aviation, and one lesson from aviation safety has always stuck with me: accidents rarely happen because of a single mistake. Instea…
How To Write A Post Mortem Report Without Blame Language
I'll never forget my first post-mortem meeting. I was a junior engineer, and the server had crashed at 3 AM during a holiday sale. The report starte…
The Fire That Reached the Backups: The OVHcloud Strasbourg Data-Centre Fire, 2021
Tales from the Bare Metal — Episode 05 In the early hours of 10 March 2021, a fire began in a power room in Strasbourg. By morning an entire data ce…
I Vibe-Coded a Stock Screener Into Production. Then My 2GB Server OOMed and Google De-Indexed Me.
Series intro. I'm a non-CS solo dev who built and shipped a production stock screener almost entirely by "vibe coding" with an AI agent. The site wo…
I got tired of writing post-mortems — so I built RCAi for SREs
I'm an SRE at Sony Interactive Entertainment. After a week where my teammate had four incidents (and four RCAs), I built something for the blank-pag…
My agent could see the dropdown. It just couldn't pick anything.
The agent had a list. I asked it to pick an item. It refused. Element not found Refresh. Same. So I opened DevTools and pasted in: document . queryS…
Баги, которые нас воспитали: инженерные истории с Go Loto
Каждый разработчик помнит тот самый момент, когда система, которая только что работала идеально, вдруг начинает вести себя так, будто сошла с ума. К…
Как auto-update n8n нашёл мину которая лежала 8 месяцев в node_modules
20 мая в 06:01:55 МСК Watchtower по расписанию проверил 14 контейнеров на нашем VPS, нашёл 5 обновлений и пересоздал. Среди обновлённых - n8n, котор…
Культура инцидентов. Почему поиск виновных на постмортемах убивает надёжность системы
Когда прод падает, первый управленческий рефлекс — найти человека, после которого всё сломалось. Проблема в том, что такой разбор поч…
Postmortem: A Vercel Edge Function Timeout Caused Our Global API to Fail for 30 Minutes
Postmortem: A Vercel Edge Function Timeout Caused Our Global API to Fail for 30 Minutes On October 17, 2024, at 14:22 UTC, our global API experience…
Postmortem: How a Corrupted Node Modules Folder Caused 3-Hour Outage for Our CI Pipeline
Postmortem: How a Corrupted Node Modules Folder Caused 3-Hour Outage for Our CI Pipeline Published: October 26, 2024 | Author: DevOps Team | 5 min r…
Postmortem: AI Incident Classifier Failed Due to Biased Training Data and Scikit-Learn 1.5
In Q3 2024, a production AI incident classifier mislabeled 42% of critical security incidents as 'low priority' over 72 hours, causing $2.1M in SLA…
Postmortem: A Corrupted Loki 2.10 Log Store Caused 3 Days of Lost Debug Data
Postmortem: A Corrupted Loki 2.10 Log Store Caused 3 Days of Lost Debug Data Overview On October 12, 2024, at 09:15 UTC, our observability team dete…
Postmortem: How a LangGraph 0.1 Multi-Agent Bug Broke Our 2026 Customer Support Bot
Postmortem: How a LangGraph 0.1 Multi-Agent Bug Broke Our 2026 Customer Support Bot Executive Summary On October 12, 2026, our production customer s…
The Postmortem of a 20-Minute Kafka 3.8 Outage That Delayed 1M Order Messages
On October 12, 2024, a misconfigured Kafka 3.8 broker idempotent write setting caused a 20-minute total outage that delayed 1,047,892 order messages…