Trending

Google Metrax Brings Predefined Model Evaluation Metrics to JAX

Recently open-sourced by Google, Metrax is a JAX library providing standardized, performant metrics implementations for classification, regression, NLP, vision, and audio models. By Sergio De Simone

[D] Published paper uses hardcoded seed and collapsed model to report fraudulent results

[D] Comparing Giskard and Rhesis for LLM evaluation — looking for experiences

Why I Hate Language Benchmarks | gingerBill

Made a website to track 348 benchmarks across 188 models.

[D] can we trust agents for time series forecasting?

I locally benchmarked 41 open-source LLMs across 19 tasks and ranked them

Evaluation frameworks and their trade-offs

"What, you don't like your new SOTA model?"

Any framework for Eval?