Trending

Content tagged with "data-engineering"

Show all

Hacker News

Top stories from the Hacker News community• Updated 3 minutes ago

HN

Show HN: I built a dashboard to compare mortgage rates across 120 credit unions

finfam.app

77

mhashemi

about 4 hours ago

webdev frontend fullstack api fintech data-engineering backend

HN

MinIO is now in maintenance-mode

github.com

392

hajtom

about 9 hours ago

cloud data-engineering self-hosted storage data-lakes

HN

Tidyverse: R Packages for Data Science

tidyverse.org

43

cl3misch

8 days ago

data-science r tidyverse data-engineering

HN

Super fast aggregations in PostgreSQL 19

cybertec-postgresql.com

214

jnord

8 days ago

data-lakes sql databases postgresql etl system-design data-engineering

HN

Mapping Every Dollar of America's $5T Healthcare System

healthisotherpeople.substack.com

122

brandonb

about 10 hours ago

data-science big-data health healthcare system-design data-engineering

HN

IBM CEO says there is 'no way' spending on AI data centers will pay off

businessinsider.com

759

nabla9

1 day ago

ai-research ai data-engineering cloud ai-ethics data-centers ibm

HN

Python Data Science Handbook

jakevdp.github.io

284

cl3misch

1 day ago

data-science python books data-engineering book

HN

Show HN: Marmot – Single-binary data catalog (no Kafka, no Elasticsearch)

github.com

84

charlie-haley

1 day ago

data-engineering data-lakes system-design databases open-source

HN

Genesis DB is a event sourcing database and there are the technical insights

docs.genesisdb.io

7

patriceckhart

8 days ago

databases event-sourcing system-design data-engineering

Reddit

Top posts from tech subreddits• Updated 15 minutes ago

Reddit

We reduced token usage by 60% using an agentic retrieval protocol. Here's how.

reddit.com

79

Future_AGI

8 months ago

r/AI_Agents ai rag llm optimization data-engineering

Reddit

U.S. Gov't eliminates tape data storage at the GSA to save $1M per year, but tape isn't dead yet

tomshardware.com

2

rtbot2

8 months ago

r/realtech cloud cost-saving big-data cost-savings data-engineering

Reddit

How to make the AI agent understand which question talks about code, which one talks about database, and which one talks about uploading file ?

reddit.com

1

CommunityOpposite645

8 months ago

r/AI_Agents ai data-engineering nlp

Reddit

Tools recommendations for unstructured to structured database.

reddit.com

3

rageagainistjg

8 months ago

r/AI_Agents databases etl data-engineering

Reddit

The Most Powerful Way to Build AI Agents: LangGraph + Pydantic AI (Detailed Example)

reddit.com

154

TheValueProvider

8 months ago

r/AI_Agents ai data-science llm python pydantic data-engineering generative-ai

Reddit

LLMs over torrent

i.redd.it

207

aospan

8 months ago

r/LocalLLaMA self-hosting llm cloud torrent data-engineering

Reddit

A love letter to the CSV format

github.com

6

namanyayg

8 months ago

r/coding data-science data-engineering csv

Reddit

The Data Rescue Project is a coordinated effort among a group of data organizations, including IASSIST, RDAP, and members of the Data Curation Network.

datarescueproject.org

4

rtbot2

8 months ago

r/realtech data-science data-lakes big-data data-engineering

Reddit

[P] Volga - Real-Time Data Processing Engine for AI/ML

reddit.com

13

saws_baws_228

8 months ago

r/MachineLearning ai big-data mlops data-engineering

Hugging Face Trending

Popular models from Hugging Face• Updated 16 minutes ago

No models found

Try removing the tag filter or searching for different content.

GitHub Trending

Popular repositories from GitHub• Updated 29 minutes ago

milvus

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

Go

41,347

3,680

vector-databases cloud go data-engineering vector-search milvus databases

perspective-dev

perspective

A data visualization and analytics component, especially well-suited for large and/or streaming datasets.

C++

9,985

1,262

data-visualization data-engineering big-data

facebookresearch

faiss

A library for efficient similarity search and clustering of dense vectors.

C++

38,274

4,142

vector-search embeddings data-engineering machine-learning data-science c++faiss ai

velero

Backup and migrate Kubernetes applications and their persistent volumes

Go

9,596

1,489

kubernetes backup vmware-tanzu data-engineering cloud

flatbuffers

FlatBuffers: Memory Efficient Serialization Library

C++

24,990

3,420

c++google performance cpp developer-tools data-engineering serialization flatbuffers memory-optimization memory-efficient data-serialization

dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

Python

4,494

376

python open-source data-engineering etl

mindsdb

Federated query engine for AI - The only MCP Server you'll ever need

Python

37,121

5,966

ai data-science python data-engineering mlops knowledge-graphs rag model-serving

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Python

43,110

15,905

data-engineering python workflow apache workflow-orchestration etl workflow-automation

arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

C++

16,122

3,898

data-engineering big-data columnar-databases databases apache apache-arrow data-lakes etl