FIFO на миллионах строк: как подружить бонусы, SQL и асимметричный N×M-граф
Всем привет! Меня зовут Иван Привалов , я разработчик в команде BI Авито Финтеха и в этой статье расскажу, как мы сделали FIFO-сопоставление между N н…
Programming
⚑ Report a ProblemLatest Programming news from Tech News
All topics
AI
agents
ai
api
architecture
automation
aws
beginners
career
claude
database
devchallenge
devops
javascript
learning
llm
machinelearning
mcp
opensource
performance
productivity
programming
python
react
rust
security
showdev
tutorial
typescript
webdev
Сквозная аналитика B2C на коленке: Google Sheets, Python и Claude Code за две недели
Сквозная аналитика для B2C: связать клик в рекламе с оплатой и посчитать ROMI. Ядро MVP (реклама + веб + биллинг в Google Sheets, дашборд в BI) собира…
From STTM to Snowflake SQL: Building a Metadata-Driven Data Engineering Copilot
Most data engineering teams do not struggle because they lack smart people. They struggle because too much of the delivery process is still repetitive…
Building My First End-to-End ETL Pipeline with Airflow, BigQuery, and Docker
Recently, I completed my first full Data Engineering project: building an end-to-end ETL pipeline using real-world Australian weather data spanning 10…
Why Metadata-Driven ETL Frameworks Scale Better Than Hardcoded Pipelines — and Where They Don't
Over the years, I've seen many data platforms start with good intentions. A few scripts are created to move data from one system to another, and every…
A torrent search engine,for actually active torrents!
The core paradox of modern BitTorrent: how do you search without a central index? Searching needs metadata, an address that you can query. BitTorrent …
Extract data from Databases into DuckLake
Introduction In the evolving landscape of data engineering, DuckLake is emerging as a powerful solution for building data lakes with ACID transactions…
ETL Pipeline: Fetching Real-Time News Data with Python and Postgres
The best way to actually understand data engineering is to build something that breaks, fix it, and watch it successfully run. In this article, we bui…
Replicate MySQL to ClickHouse with Sling
Introduction ClickHouse is a columnar OLAP database. It runs aggregate queries across billions of rows in seconds. MySQL is what most apps run on for …
Apache SeaTunnel Isn’t a Simple ETL Tool , Understanding Its DataFlow-Driven DAG Engine
In the field of data integration and synchronization, Apache SeaTunnel is undoubtedly one of the hottest tools today. This series will dive deep into …
Строим машину времени для данных (SCD-2) на движке Trino под управлением Airflow
Сегодня SCD-2-таблицы не только остаются актуальными для медленно меняющихся данных, но и, на мой взгляд, становятся гораздо проще в реализации благод…
Load PostgreSQL into Apache Iceberg with Sling
Introduction Apache Iceberg is the table format that turns a pile of Parquet files in object storage into something that behaves like a warehouse tabl…
Смотрим low-code коннектор к «1С: Шине» от «Денвик»
На связи Сергей Скирдин, технический директор компании «Белый код». Мы занимаемся проектами в сфере управления данными: интеграции, хранилища, BI. В п…