Hosting an Entire Open Lakehouse on Port 8080
Most of my job, at least the part people see, is running Spark code in front of other people and promising it'll work. The catch is that I almost ne…
DevOps
⚑ Сообщить о проблемеПоследние новости DevOps от Tech News
Все темы
agents
ai
api
architecture
automation
aws
backend
beginners
cloud
database
devchallenge
devops
docker
javascript
kubernetes
llm
machinelearning
mcp
node
opensource
performance
productivity
programming
python
security
showdev
testing
tutorial
typescript
webdev
All About Iceberg's File Format API: simplifying the engine/format interface and Parquet's multimodal future
What is the file format API? Apache Iceberg traditionally exposed its data files through three libraries, one each for Parquet, Avro and ORC, and un…
Apache ICEBERG: Прикасаюсь к Iceberg… а он горячий
Про Iceberg сейчас много чего понаписано. Либо обзор «формат будущего, открытость, лейкхаус», после которого непонятно, где нажать, либо разбор одно…
What One Rust Client Can Reach Across Seven Iceberg Catalogs
This article provides a step by step guide to pointing the Apache Rust client for Iceberg REST catalogs at seven catalogs and recording what works.…
What One Rust Client Can Reach Across Seven Iceberg Catalogs
This article provides a step by step guide to pointing the Apache Rust client for Iceberg REST catalogs at seven catalogs and recording what works.…
Production RAG on the Lakehouse with BigQuery Vector Search and Apache Iceberg
As teams rush to build with Generative AI, they're creating a dangerous chasm between their data and AI stacks. This common architectural flaw intro…
Four Iceberg Tools, Three Agent Frameworks: What Ports, and What Doesn't
This article is a step by step build of one small Apache Iceberg data agent, written three times: in Google ADK, in AWS Strands and in Microsoft Age…
Four Iceberg Tools, Three Agent Frameworks: What Ports, and What Doesn't
This article is a step by step build of one small Apache Iceberg data agent, written three times: in Google ADK, in AWS Strands and in Microsoft Age…
Инфраструктура данных 2052: что, если у данных больше не будет «прода» и «истории»?
Если мысленно вернуться в 1980-е , аналитика во многом находилась буквально над production Данные появлялись в операционных системах, затем из них с…
Using Databricks FILE type against data on NAS: what works today with FSx for ONTAP S3 Access Points
A follow-up to Part 2: Databricks and FSx for ONTAP S3 Access Points in the "FSx for ONTAP S3 Access Points x Lakehouse Deep Dive" series. Some resu…
The query succeeded. Which table state did it read?
Why two successful queries can count different rows when they resolve different Delta or Iceberg table states. Two queries return different counts f…
The pipeline passed. The data still needs proof.
A pipeline job finishes without an error. The new table has 18% fewer rows. Should you rerun the pipeline, stop the release, or trust the new output…
Пока все хоронили пайплайны, ClickHouse достраивал слои
«Отдельные базы больше не нужны», «конец пайплайнов» - каждую неделю кто-то крупный со сцены хоронит то, что ты вчера поставил в прод. ClickHouse по…
[Databricks on AWS #0] The Target Architecture: Isolating Prod, Dev, and Sandbox with Unity Catalog
📚 Series: Databricks on AWS (Part 0, prologue) The Target Architecture ← you are here Building a Databricks AI Platform on AWS RBAC with Function-Ro…
TPC-DS в 07.2026. Lakehouse: Spark, Trino, StarRocks, Impala и Doris. GreenPlum & Cloudberry vs StarRocks как MPP
Привет, Хабр! На связи команда Data Sapience. С последней публикации результатов тестирования MPP-движков прошло уже несколько месяцев. За этот пери…
Apache Paimon: steamhouse как логическое продолжение современных КХД
Apache Paimon: стриминговый lakehouse для дата-инженеров Сколько систем вы держите ради того, чтобы аналитики видели события через секунды, а годовы…
Using Microsoft Fabric Shortcuts to Avoid Duplicate Data Copies
Enterprise data platforms are really good at one thing: creating copies of the same data everywhere. Different teams copy the same curated folders i…
Только Сигма выбирают Delta Lake
Привет, Хабр! Меня зовут Дмитрий Кравчук, я занимаюсь всем, что связано с данными в блоке AI&ML MAGNIT TECH. Расскажу про фундамент прибыльных п…
DatAasee 0.9 Released
DatAasee Tech Stack Canvas Setting: Many distributed data and metadata sources Goals: Centralize metadata Interlinked metadata catalog Super-index f…