Hosting an Entire Open Lakehouse on Port 8080
Most of my job, at least the part people see, is running Spark code in front of other people and promising it'll work. The catch is that I almost ne…
Architecture
⚑ Report a ProblemLatest Architecture news from Tech News
All topics
agents
ai
api
architecture
automation
aws
backend
beginners
career
database
devchallenge
devops
javascript
llm
machinelearning
mcp
news
opensource
performance
productivity
programming
python
react
security
showdev
softwareengineering
systemdesign
tutorial
typescript
webdev
The Snowflake Iceberg Pivot: Why Your Data Warehouse Should Be a Storage Engine
Ninety percent of the "performance" you pay for in Snowflake is actually just a tax on your inability to move your data. If you’re like most enginee…
Four Iceberg Tools, Three Agent Frameworks: What Ports, and What Doesn't
This article is a step by step build of one small Apache Iceberg data agent, written three times: in Google ADK, in AWS Strands and in Microsoft Age…
Four Iceberg Tools, Three Agent Frameworks: What Ports, and What Doesn't
This article is a step by step build of one small Apache Iceberg data agent, written three times: in Google ADK, in AWS Strands and in Microsoft Age…
Migrating Petabyte-Scale Parquet to Iceberg Without Dropping a Single Row
Roughly 70% of companies that migrate their data lakes to Iceberg end up with "zombie" datasets: half-migrated, out-of-sync, and consuming double th…
Всё что нужно знать про DuckLake
Когда вышел DuckDB 1.0, я написал статью Всё что нужно знать про DuckDB . Теперь вышел DuckLake v1.0 — LakeHouse-формат из той же экосистемы. Это от…
Why Athena/Iceberg Tends to Make Code the Spec
Every time this comes up, someone credits the same setup: Athena on Iceberg is where "the code is the spec" — where you open Git and read the whole…
AI Enrichment Pipeline: From Sample Classification to 100K-File Metadata Search with Bedrock and OpenSearch NextGen
Quick Recap: What We Built in Part 1 In Part 1 , we built a metadata catalog on Apache Iceberg (S3 Tables) that makes unstructured files on FSx for…
From Hours to Seconds: An AI-Powered Metadata Catalog for Unstructured Data on FSx for ONTAP
What Works Now vs What Requires Validation This article separates verified AWS-native capabilities from cross-platform paths that still require vali…
Rethinking Lakehouse Architecture Through Data Ownership: AWS vs. Snowflake
Original Japanese article : データの主導権から考えるAWSとSnowflakeのレイクハウスアーキテクチャ Introduction I'm Aki, an AWS Community Builder ( @jitepengin ). When designing a…
When does Iceberg beat Parquet+projection on AWS Glue, and when doesn't ?
Why this project I built this repo because I didn't have one of this kind yet and, having worked on data ingestion with Glue for a while, I wanted t…
Ursa — a new Diskless Lakestream engine for Kafka
The future of Kafka is diskless topics + native Apache Iceberg / Delta Lake integration Intro Ursa is a relatively new engine that's being fitted in…