Argilla

A self-hosted AI data annotation tool that combines human feedback, AI suggestions and semantic search for training and evaluation datasets.

Argilla is a data annotation and feedback tool for AI engineers and domain experts building training and evaluation datasets. It gives reviewers a shared interface for checking examples and contributing judgments that inform model development.

Human annotation: Collect labels and expert feedback for text classification, named entity recognition and multimodal projects such as text-to-image models.
Focused review: Use filters, semantic search and AI feedback suggestions to find relevant records and inspect data that needs attention.
LLM datasets: Curate examples for retrieval-augmented generation, preference tuning and reinforcement learning from human feedback, alongside model evaluation work.
Programmatic workflows: Connect dataset review to ongoing evaluation and model improvement through the Python SDK, including workflows that combine human curation with AI-generated feedback from distilabel.

Argilla is open source under Apache 2.0. You can run your own server or deploy it on Hugging Face Spaces; the Spaces deployment runs on hosted infrastructure. The project is in maintenance mode: maintainers provide bug fixes and patches, but aren't developing new features.

Data Labeling and Annotation Datasets and Synthetic Data LLM Evaluation and Testing Web Multi-user access Semantic search