DevOps

A Kafka Streams topology starts in a sandbox, not on your cluster

Somebody owns the Kafka cluster you need, and it is not you. You have a topology in mind: read orders , keep the big ones, write them somewhere. You…

Why Spark Couldn't Read from Kafka: A Real Debugging Journey Across PySpark, Hadoop, Docker, and Kafka

I thought this would be a simple task. I already had a Python Kafka producer running. Kafka was up in Docker. The topic existed, and I could send a…