Hugging Face Community Evals

Local inspect-ai and lighteval

Hugging Face · Apache-2.0

When

Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.

The Catch

local inspect-ai/lighteval. Not HF Jobs.

Embed

[![Reviewed on Stack Gems](https://stackgems.com/badge/skill/huggingface-community-evals.svg)](https://stackgems.com/skills/huggingface-community-evals)