Trending

A love letter to the CSV format (2024)

Data Modeling Guide for Real-Time Analytics with ClickHouse

6NF File Format

The two versions of Parquet

OS Yamato lets your data fade away

450× Faster Joins with Index Condition Pushdown

Ordered Insertion Optimization in OrioleDB

Why Semantic Layers Matter (and how to build one with DuckDB)

How to ingest 1B rows/s in ClickHouse

A Quick(ish) Introduction to Tuning Postgres

[D] looking for advice: end2end pipeline

[D] Build an in-house data labeling team vs. Outsource to a vendor?

I'm curating a list of every document parser out there and running tests on their features. Link in the comment.

How to avoid Bad Data before it breaks your Pipeline with Great Expectations in Python ETL…

finally found a digital marketing ETL tool that doesn’t make things harder

Properly handling missing values [D]

Tools recommendations for unstructured to structured database.

Unstructured-IO

Save item

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

pathwaycom

Save item

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

apache

Save item

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

apache

Save item

arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

apache

Save item

dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

apache

Save item

seatunnel

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.