DevTrends.fyi

Trending

Content tagged with "data-engineering"

data-engineering

Hacker News

Top stories from the Hacker News community• Updated 11 minutes ago

118
21
kirlev
about 19 hours ago
big-datahigh-availabilitystoragedata-engineeringdistributed-systemsdata-lakes
30
18
iamlintaoz
2 days ago
databasesnosqldata-engineeringmongodbmongo
HN

Designing EventQL, an Event Query Language

docs.eventsourcingdb.io
17
8
goloroden
1 day ago
system-designdistributed-systemsdatabasesevent-drivenapi-designdata-engineeringsql
398
195
WithinReason
4 days ago
securitymemory-cardunderwaterhardwaredeep-learningstoragedata-engineeringself-hosted
HN

Is Postgres read heavy or write heavy?

crunchydata.com
167
49
soheilpro
4 days ago
databasespostgresdata-engineeringperformancesystem-design
HN

SQL Anti-Patterns You Should Avoid

datamethods.substack.com
248
163
zekrom
3 days ago
sqldatabasessystem-designdatabase-tuningdata-engineeringbest-practices
35
2
mvdan
3 days ago
iotdata-engineeringconfiguration-managementconfiguration
55
15
vbezhenar
3 days ago
storagesystem-designperformancelinuxdistributed-systemsdata-engineering
HN

Your data model is your destiny

notes.mtb.xyz
368
92
hunglee2
7 days ago
data-engineeringsystem-designdata-sciencedata-modelingbig-datamodel-servingmlops
246

Reddit

Top posts from tech subreddits• Updated 2 minutes ago

Reddit

JSON

i.redd.it
234
9
Unlikely_Gap_5065
6 months ago
r/webdevapidata-sciencejsondatabasesdata-engineering
1
1
rtbot2
6 months ago
r/realtechtech-newseconomicsaitechnologystartupscloudnewshardwareotherbig-dataai-researchdata-engineering
27
3
Distinct-Gas-1049
6 months ago
r/MachineLearningself-hostingcloudopen-sourcedata-engineering
2
1
rtbot2
6 months ago
r/realtechcloudhardwaregpudata-engineering
1
5
SolidRemote8316
6 months ago
r/MachineLearningaidata-sciencepytorchdeep-learningdata-engineering
Reddit

Properly handling missing values [D]

reddit.com
0
0
QuadransMuralis
6 months ago
r/MachineLearningdata-scienceetldata-engineering
Reddit

Cut LLM Audio Transcription Costs

reddit.com
7
0
Kboss99
6 months ago
r/AI_Agentsaillmcost-optimizationdata-engineering
Reddit

Apache Airflow 3.0 released

airflow.apache.org
6
0
xtreak
6 months ago
r/programmingworkflow-automationopen-sourcedata-engineering
Reddit

Cut LLM Audio Transcription Costs

reddit.com
4
0
Kboss99
6 months ago
r/AI_Agentsaillmcost-optimizationdata-engineering
1343640

Hugging Face Trending

Popular models from Hugging Face• Updated 39 minutes ago

No models found

Try removing the tag filter or searching for different content.

GitHub Trending

Popular repositories from GitHub• Updated about 1 hour ago

redissonredisson

redisson

Redisson - Valkey & Redis Java client. Real-Time Data Platform. Sync/Async/RxJava/Reactive API. Over 50 Valkey and Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Bloom filter, Spring, Tomcat, Scheduler, JCache API, Hibernate, RPC, local cache..

Java
24,073
5,480
javaredisdata-engineeringdeveloper-toolsdistributed-systemsdatabases
pathwaycompathwaycom

pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Python
48,869
1,424
pythonetlllmragdata-engineering
apacheapache

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Python
42,851
15,796
data-engineeringpythonworkflowapacheworkflow-orchestrationetlworkflow-automation
ClickHouseClickHouse

ClickHouse

ClickHouse® is a real-time analytics database management system

C++
43,434
7,727
databasessqlclickhousec++analyticscppperformancedata-engineeringdata-analyticsreal-timesystem-design
lancedblancedb

lance

Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Rust
5,537
453
data-engineeringrustvector-databasesdatabasesvector-searchbig-data
datahub-projectdatahub-project

datahub

The Metadata Platform for your Data and AI Stack

Java
11,145
3,239
data-engineeringdata-lakesjavadata-scienceai
apacheapache

doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

Java
14,460
3,577
databasessqlapacheperformancejavaanalyticsdata-engineeringdata-warehousingbig-dataapache-dorisdata-analyticsdatabase-tuning
run-llamarun-llama

llama_index

LlamaIndex is the leading framework for building LLM-powered agents over your data.

Python
44,763
6,452
llmpythongenerative-aidata-engineeringaitransformersragvector-searchnlp
apacheapache

superset

Apache Superset is a Data Visualization and Data Exploration Platform

TypeScript
68,520
16,009
data-sciencedata-engineeringtypescriptdata-visualizationdatabasesvisualizationdata-lakesjupyterdata-explorationapachewebdev
26