DevTrends.fyi

Trending

Content tagged with "data-engineering"

data-engineering

Hacker News

Top stories from the Hacker News community• Updated 12 minutes ago

286
143
WithinReason
2 days ago
memory-cardhardwaredeep-learningstoragedata-engineeringself-hosted
HN

Is Postgres read heavy or write heavy?

crunchydata.com
150
38
soheilpro
1 day ago
databasespostgresdata-engineeringperformancesystem-design
HN

SQL Anti-Patterns You Should Avoid

datamethods.substack.com
234
154
zekrom
about 18 hours ago
sqldatabasessystem-designdatabase-tuningdata-engineeringbest-practices
35
2
mvdan
about 18 hours ago
iotdata-engineeringconfiguration-managementconfiguration
55
15
vbezhenar
about 20 hours ago
storagesystem-designperformancelinuxdistributed-systemsdata-engineering
HN

Your data model is your destiny

notes.mtb.xyz
368
92
hunglee2
4 days ago
data-engineeringsystem-designdata-sciencedata-modelingbig-datamodel-servingmlops
13
6
iaaan
7 days ago
system-designperformancedata-engineeringcompressiondata-transfersnetworkingdata-compression
21
12
rtk0
7 days ago
data-structuresperformancedata-compressionfile-archivingbig-datadata-engineeringcompression
14
20
mickeymounds
2 days ago
data-sciencebig-datadata-engineeringdata-visualizationopen-source
246

Reddit

Top posts from tech subreddits• Updated 12 minutes ago

7
0
zephyrzilla
26 days ago
r/MachineLearningdata-scienceknowledge-graphsgraphgraph-analyticsdata-engineeringgenerative-ai
3
1
rtbot2
26 days ago
r/realtechaiclouddata-engineering
3
0
aqjo
27 days ago
r/MachineLearningdata-managementdata-sciencedata-provenancedata-ethicsdata-engineering
1
1
rtbot2
26 days ago
r/realtechaidata-sciencemlopsdata-engineering
2
0
babaenki
27 days ago
r/MachineLearningaidata-sciencemlopsdata-engineeringgenerative-ai
74
66
YogurtclosetWise9803
28 days ago
r/webdevjsondatabasesdata-engineering
1
1
rtbot2
28 days ago
r/realtechweb3privacybig-datasecuritydata-engineering
1
1
Better-Department662
29 days ago
r/AI_Agentsaidata-integrationdata-engineering
2
1
Senior_Charity_2339
29 days ago
r/AI_Agentsdata-sciencedata-marketplacedata-lakesbig-datadata-setsetldata-engineering
15740

Hugging Face Trending

Popular models from Hugging Face• Updated about 1 hour ago

No models found

Try removing the tag filter or searching for different content.

GitHub Trending

Popular repositories from GitHub• Updated 8 minutes ago

pathwaycompathwaycom

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Python
48,848
1,420
pythonetlllmragdata-engineering
apacheapache

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Python
42,848
15,794
data-engineeringpythonworkflowapacheworkflow-orchestrationetlworkflow-automation
ClickHouseClickHouse

ClickHouse

ClickHouse® is a real-time analytics database management system

C++
43,434
7,727
databasessqlclickhousec++analyticscppperformancedata-engineeringdata-analyticsreal-timesystem-design
lancedblancedb

lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Rust
5,537
453
data-engineeringrustvector-databasesdatabasesvector-searchbig-data
datahub-projectdatahub-project

datahub

The Metadata Platform for your Data and AI Stack

Java
11,145
3,239
data-engineeringdata-lakesjavadata-scienceai
apacheapache

doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

Java
14,460
3,577
databasessqlapacheperformancejavaanalyticsdata-engineeringdata-warehousingbig-dataapache-dorisdata-analyticsdatabase-tuning
run-llamarun-llama

llama_index

LlamaIndex is the leading framework for building LLM-powered agents over your data.

Python
44,763
6,452
llmpythongenerative-aidata-engineeringaitransformersragvector-searchnlp
apacheapache

superset

Apache Superset is a Data Visualization and Data Exploration Platform

TypeScript
68,520
16,009
data-sciencedata-engineeringtypescriptdata-visualizationdatabasesvisualizationdata-lakesjupyterdata-explorationapachewebdev
OpenBB-financeOpenBB-finance

OpenBB

Financial data platform for analysts, quants and AI agents.

Python
53,491
5,132
pythondata-scienceopen-sourcefinancial-analyticsfinancial-datafinancial-analysisfintechfinancefinancialdata-engineeringai
26