Architecture — Tech News

Все темы agents ai api architecture automation aws backend beginners career database devchallenge devops javascript llm machinelearning mcp news opensource performance productivity programming python react security showdev softwareengineering systemdesign tutorial typescript webdev

EN

Slack Eliminates SSH in EMR Pipelines, Migrates 700+ Jobs to Rest-Based Architecture

Slack modernized its data platform by replacing SSH based execution in Amazon EMR pipelines with a REST driven orchestration layer called Quarry. Th…

Cloud Migration SSH Observability Slack REST CLI Amazon Apache Hadoop Data Pipelines Apache Spark JobRunr Downtime migration Apache Hive DevOps Development Architecture & Design news

EN

Article: Two Misconfigurations That Caused Spark OOM Failures on Kubernetes

After migrating Spark pipelines to Azure Kubernetes Service, two infrastructure settings interacted destructively: spark.kubernetes.local.dirs.tmpfs…

Apache Spark Cloud Kubernetes DevOps AI, ML & Data Engineering Development Architecture & Design article