Trending

A sharded DuckDB on 63 nodes runs 1T row aggregation challenge in 5 sec

Optimizing writes to OLAP using buffers (ClickHouse, Redpanda, MooseStack)

A16Z-backed data firms Fivetran, dbt Labs to merge in all-stock deal

A whirlwind introduction to dataflow graphs (2018)

Pipelining in psql (PostgreSQL 18)

A SQL Heuristic: Ors Are Expensive

Haydex: From Zero to 178,600M rows a second in 30 days

Row-level transformations in Postgres CDC using Lua

Delete FROM users WHERE location = 'Iran';

[D] Name and describe a data processing technique you use that is not very well known.

Has anyone tried selling data on Opendatabay or similar platforms?

How Transaction Isolation Provides Data Integrity in Databases

What’s the biggest data governance challenge you face when building cross-agent pipelines?

Business Rules In Database Movement

Is anyone using Apache Seatunnel in production?

pg_csv: Flexible CSV processing for Postgres

SurrealDB is sacrificing data durability to make benchmarks look better

On creating spreadsheets/structured datasets from the web

pathwaycom

Save item

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

apache

Save item

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Unstructured-IO

Save item

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

dlt-hub

Save item

dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

dbt-labs

Save item

dbt-utils

Utility functions for dbt projects.

apache

Save item

arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

apache

Save item

dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

apache

Save item

seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.