Trending

Fast columnar JSON decoding with arrow-rs

Hann: A Fast Approximate Nearest Neighbor Search Library for Go

Palantir suggests 'common operating system' for UK govt data

It's Time to Stop Building KV Databases

"Data" Sucks

Global population datasets systematically underrepresent rural population

Piccolo: Large-Scale Graph Processing with Fine-Grained In-Memory Scatter-Gather

Ask HN: What is the simplest data orchestration tool you've worked with?

Sync Engines Are the Future

Lidar helps gas industry find methane leaks and avoid costly losses

We spent 33 months building a data grid, here's how we solved slow UIs.

4x 3090 local ai workstation

Spotify peeved after 10,000 users sold data to build AI tools

AI consultancy startup, how to deal with messy client data and unrealistic AI expectations?

Microsoft-backed research team builds hollow-core cable with lowest signal loss recorded in optical fiber

[P] Semlib: LLM-powered Data Processing

[D] Having trouble organising massive CSV files for your machine learning models?

Delta Flow | Generating buildable digital twins in minutes [D]

simdjson

Save item

simdjson

Parsing gigabytes of JSON per second : used by Facebook/Meta Velox, the Node.js runtime, ClickHouse, WatermelonDB, Apache Doris, Milvus, StarRocks

milvus-io

Save item

milvus

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

apache

Save item

doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

lancedb

Save item

lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

cvat-ai

Save item

cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.

Unstructured-IO

Save item

unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

StarRocks

Save item

starrocks

The world's fastest open query engine for sub-second analytics both on and off the data lakehouse. With the flexibility to support nearly any scenario, StarRocks provides best-in-class performance for multi-dimensional analytics, real-time analytics, and ad-hoc queries. A Linux Foundation project.

great-expectations

Save item

great_expectations

Always know what to expect from your data.

run-llama

Save item

llama_index

LlamaIndex is the leading framework for building LLM-powered agents over your data.