Spark's Modules, Seen Through Spark Connect
Spark is moving its centre of gravity out of the JVM. Spark Connect, introduced in 3.4, lets a client drive Spark over gRPC without running a JVM of…
DevOps
⚑ Report a ProblemLatest DevOps news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
cloud
database
devchallenge
devops
docker
javascript
kubernetes
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
Под капотом Kafka: путь сообщения от send() до commit offset. Часть 2
Привет, Хабр! Меня зовут Максим Шуматбаев, я инженер технической поддержки в Arenadata. Это вторая часть разбора, что на самом деле происходит при р…
Adding DRM to Native Video Apps and AWS Live Streaming: What to Validate
A protected video player has to work across several playback environments. Android, iOS, Apple TV and Samsung Smart TV do not share one player stack…
Understanding IPTV Streaming Protocols in 2026: HLS vs MPEG-TS & Hardware Acceleration
Understanding IPTV Streaming Protocols in 2026: HLS vs MPEG-TS & Hardware Acceleration When analyzing video streaming architecture, live Interne…
The Architecture of Live Video Streaming: How HLS, DASH, and CDNs Actually Work
Live video streaming has become the backbone of modern entertainment infrastructure. Whether a user is watching a football match, joining a video co…
Bridging Webhooks to Your Event Mesh: A Secure Edge Pattern for Kafka, Redpanda, and NATS
API event standardization API integration middleware API webhook payload mapping AsyncAPI specification AsyncAPI webhook design asyncAPI webhooks Cl…
The phone line was playing a web page into the call
nixamp streams have a phone number. Call +1 408-357-2326, key a six digit code, and if that code belongs to a live stream you hear the stream. That…
Live streaming technologies in crypto analytics
What Live Streaming Infrastructure Actually Means for Crypto Analytics Crypto data is not like most data. Prices move in milliseconds, on-chain even…
Xbox imposes harsh new time limits for Game Pass game streaming
Over 1 million Game Pass subscribers will have to pay more for their heavy streaming use.
WebSocket silent data gap in market feed
When Your WebSocket Lies to You by Saying Nothing The worst kind of bug is the one your system doesn't report. A connection is open. No errors. No r…
Под капотом Kafka: путь сообщения от send() до commit offset. Часть 1
Привет, Хабр! Меня зовут Максим Шуматбаев, я инженер технической поддержки в Arenadata. Эта статья — попытка собрать в одном месте внутренние механи…
Hidden complexity of building WebSocket servers from scratch
WebSockets Sound Simple Until You Have to Maintain One There's a moment every engineer hits when building a real-time feature: you look at the WebSo…
Почему ML-модель получает неправильные признаки: Event Time, Watermarks и Apache Flink
Представьте: вы разрабатываете антифрод-систему для банка. Пользователь совершает пять покупок подряд за пару минут. Все транзакции успешно попадают…
Building a Streaming Pipeline: From a System Design Perspective
Say you're on vacation and you've just recorded a fairly large video of yourself skydiving for the first time, naturally, you'd want to send it to a…
Flutter Streaming UI: How the Typewriter Experience of AI Replies Is Built
The typewriter effect looks simple: characters appear one by one. But behind "skip animation", "no truncation", and "no performance regression" lies…
Why Data Engineering Is the Oldest Profession in Infrastructure
Not that one. The other one: keeping water flowing to people who need it. The comparison between data engineers and plumbers gets made a lot, usuall…
Edge vs Cloud Inference for Live Sports Highlights: Where Should the Model Run?
When you build a system that detects key moments in a live sports feed, one architectural decision shapes everything downstream: where does the infe…
The Compression Layer Nobody Thinks About Until It's Too Late
WebSocket optimization tends to follow a predictable pattern. You build the connection layer, get events flowing, ship the feature, and move on. Com…
Kafka Topics, Partitions, and Offsets Explained: The Concepts Every Beginner Confuses
When I first started learning Apache Kafka, I quickly realized that three terms appeared everywhere: topics, partitions, and offsets. At first glanc…
Exactly-Once Semantics in Kafka: Promise vs. Reality
"We're using Kafka with exactly-once semantics, so we don't have to worry about duplicates." I've heard this in architecture reviews, design docs, a…
Capturing, Streaming, Storing, and Visualizing Crypto Market Data in Real Time with PostgreSQL, Debezium, Kafka, JDBC & Grafana
In the fast-moving world of cryptocurrency, market data changes every second — prices fluctuate, trades execute, and volumes shift continuously. Cap…
Build a Real-Time Dashboard with Python, FastAPI, and WebSockets
Your Dashboard Is Lying to You (And Polling Is Why) Every five seconds, your frontend fires a request. The server wakes up, queries the database, se…
The 1x1 Pixel That Broke My Streaming Extension
Saturday morning, 9 AM. I was at the kitchen island, making a pour-over coffee and testing StreamEnhancer on my MacBook while chopping onions for a…
WebSocket in Spring Boot, Explained Simply
Stop Polling. Your Server Already Knows the Answer. There's a pattern that shows up in codebases more often than it should: a frontend making repeat…
Streaming Responses in Next.js App Router — Server-Sent Events and ReadableStream
Your API Isn't Slow. Your Response Strategy Is. There's a class of bug that never shows up in your performance benchmarks but destroys user trust ev…
Building an Event-Driven User Signup Flow with Redpanda and Python
When a user signs up for a typical app, one function usually does it all: validate the input, write a row to the database, maybe queue a welcome ema…
Apache Paimon: steamhouse как логическое продолжение современных КХД
Apache Paimon: стриминговый lakehouse для дата-инженеров Сколько систем вы держите ради того, чтобы аналитики видели события через секунды, а годовы…
A Banking API Is Not Just CRUD: What Building a Money-Movement Ledger Taught Me
A Banking API Is Not Just CRUD: What Building a Money-Movement Ledger Taught Me I thought a banking API would be mostly CRUD. Make an account. Read…
Kafka is not a queue — and treating it like one will wreck your system
Stop Treating Kafka like a queue! A story about silent lag, vanishing messages, and the mental model shift nobody warns you about. You integrated Ka…
Kafka Partitioning Strategies: How to Get It Right Before It Costs You
Most engineers don't think seriously about Kafka partitioning until something breaks in production. A topic that worked fine at low volume starts fa…