[Перевод] Компактация лога в Apache Kafka повреждала данные. Вот как мы это исправили
Компактация лога в Kafka может привести к ситуации, когда разные реплики одного топика начинают хранить разную версию данных — и расхождение сохраня…
Architecture
⚑ Report a ProblemLatest Architecture news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
news
opensource
performance
productivity
programming
python
react
security
showdev
softwareengineering
systemdesign
tutorial
typescript
webdev
Designing High-Throughput Event-Driven Microservices with Kafka and RabbitMQ
Introduction & Industry Context In the landscape of modern software architecture, event-driven microservices have transitioned from an advanced…
Microservices Event Pipelines: Outbox Pattern, CDC, and Idempotent Event Sourcing
In distributed microservice architectures, synchronizing database updates with downstream event streams is a notorious source of silent data drift.…
Kafka Partitions and Consumer Groups: How to Prevent Head-of-Line Blocking
Apache Kafka is celebrated for its incredible horizontal throughput, capable of streaming millions of messages per second. Yet, many engineering tea…
Integration Digest for September 2026
Articles 🔍 API Features as a First-Class Artifact with OpenAPI Overlays OpenAPI Overlay 1.1 documents model API features as separate, reviewable art…
Read Kafka Like a Database
Working with Kafka data on your own terms, using a local index. A search stack usually follows Kafka into a service. But the message you are looking…
2,404,811 answers on port 2181 and 1,381,208 on port 9092: the difference between a coordinator and a broker
2,404,811 answers on port 2181 and 1,381,208 on port 9092: the difference between a coordinator and a broker The counts Two ports associated with di…
Sidekiq to Kafka: A Mental-Model Map for Rails Developers
TL;DR: If you've spent years with Sidekiq, Kafka can look like "Sidekiq with extra steps". It isn't. Sidekiq moves jobs (commands to do something).…
Retry policy на практике: подводные камни внедрения
Что делать, если произошла ошибка? Житейская мудрость говорит, что можно попробовать еще раз. В контексте разработки ПО существует известный и…
Queues, Webhooks and Rate Limits: Retries, Backlogs and Backoff You Can Watch Happen
"At-least-once delivery" is four words that hide a dozen production incidents. So are "consumer lag", "dead letter queue", and "exponential backoff"…
Stop Chasing Exactly-Once. Your Kafka Consumer Just Needs to Not Care
Every few months a team I talk to decides they need exactly-once delivery in Kafka. They read the docs, discover processing.guarantee=exactly_once_v…
The Inbox Transaction Boundary: Getting Event Processing Right in Spring Boot
The Inbox Transaction Boundary: Getting Event Processing Right in Spring Boot One of the hardest parts of building an event-driven system isn't gett…
Did You Know? Kafka Stores Your Messages on Disk 🤔
Most people assume that a high-performance messaging system like Kafka must be keeping everything in memory to stay fast. But here's the truth — eve…
Очередь на Postgres: почему Kafka не нужна
Это история про архитектуру одного сервиса. Рассказанная до конца, а не только до того места, где обычно останавливаются статьи про архитектуру. Нам…
A question on Discord found two bugs in my RAG system. Fixing them found a third.
Every test passed. The tests had the same blind spots as the code. A developer saw Ossian's MCP server in a community showcase and asked two questio…
Resilient Fintech Microservices: High-Throughput with .NET 10 and Apache Kafka
In the fintech sector, data integrity and system reliability are paramount. When managing sensitive operations such as credit products, loan disburs…
Consuming AWS MSK from Azure Databricks over mTLS
Most guides for connecting Spark to Amazon MSK assume the two live in the same cloud and authenticate with IAM. That covers a lot of cases. It does…
Хватит рисовать интеграции в Miro: я сделал архитектуру, которую можно прокликать
Перевод по СБП — это не одна стрелка между двумя сервисами, а лимиты, антифрод, идемпотентность, НСПК, Kafka и уведомления. Я попробовал собрать это…
The compiler was never what you wanted
You have an orders topic on a Kafka cluster, its values encoded with Avro against a schema in the Schema Registry . You want the orders worth more t…
Reliable Event-Driven Architecture in Spring Boot: Outbox, Inbox, Retries, and Idempotency
Event-driven architecture looks simple at first. Your application performs a business operation, publishes an event to Kafka or SQS, and another ser…
Большая история крохотного BPMN
Что полезного может делать бизнес-процесс, в котором всего одна задача, и та пользовательская? Что можно рассказать про BPMN-схему, которая поместит…
How Kafka Saved a Failing System -The Story of Arjun & Meera
Modern systems run on data in the form of fast, real-time, continuous streams. But what happens when your system grows faster than your architecture…
Building Fault-Tolerant, Event-Driven Kafka Pipelines in Go: Reliable Reprocessing & Dead Letter Queues
A practical guide to building reliable event-driven systems in Go using Apache Kafka. Learn how to implement tiered retry strategies with delayed re…
The Outbox Pattern Is Not Enough
The textbook version of the transactional outbox is tight. You save the domain entity and an outbox row in one local transaction. A background sched…
2–3 тысячи сборок в день: как мы превратили Kafka из компонента в инфраструктурную платформу
Apache Kafka можно бесплатно скачать, развернуть и подключить к приложениям. При небольшом проекте этого может быть достаточно: создали несколько то…
The Redis & Kafka Interview Questions I Got Asked 23 Times (And How I Finally Answered Them)
Every senior backend interview I've done in the last 18 months asked at least one Redis question and one Kafka question. Below are the 10 that came…
SNS vs SQS vs Kinesis vs MSK vs EventBridge vs RabbitMQ: An Architect's Decision Matrix
By Swetha Golla · 8 min read · Senior Application Architect 🔗 This post has a live interactive version with a clickable per-service verdict and the…
The Kafka Incident That Was Just a Certificate (And the Operator That Saved MirrorMaker)
A Saturday afternoon alert: under-replicated partitions on the production Kafka cluster, well over the threshold. Status CRITICAL, replicas out of s…
Kafka for Engineers Who've Only Used REST: What Actually Changes
When I first read that Kafka was a "distributed event streaming platform," I nodded, closed the tab, and went back to writing REST endpoints. It too…
Event-Driven Architecture in Practice
Introduction In today's fast-paced digital landscape, businesses are constantly seeking innovative ways to stay ahead of the competition. One key st…