The video explains the ETH Evals framework, an open system for evaluating AI models and agents on real Ethereum tasks using local chains and mainnet forks. It is aimed at developers and researchers building or testing AI agents for blockchain applications. The hosts demonstrate how the system works, share initial results, and show how to contribute your own evaluations.
ما يغطيه الفيديو
Introduction to ETH Skills evaluation across 19 skills and 97 tasks
Findings that shorter rewritten skills performed better and pitfalls of misleading evals
Overview of ETH Evals architecture for testing models, web‑enabled agents, and skill‑augmented agents
How results are published to a public board and Hugging Face knowledge base
Instructions for adding your own evals or models to the platform
المناقشة 0