ModernBERT tutorial: fine-tuning self-hosted AI guardrails

Diego Carpintero explains how to fine-tune ModernBERT as a self-hosted safety classifier for LLM applications. He first examines prompt injection, malicious external context, adversarial suffixes, RAG poisoning, MCP tool descriptions and compromised agents. These examples frame his argument for checking inputs, responses and intermediate actions rather than relying entirely on model alignment or human approval.

The architecture discussion covers alternating local and global attention, an 8,192-token context, unpadding, sequence packing, rotary positional encoding and FlashAttention. Carpintero connects these choices to the cost of processing long inputs and running repeated safety checks. He also compares encoder classification with LLM-as-a-judge checks, which he says can add seconds across a pipeline.

The practical walkthrough uses the Inject Guard dataset, train/test splitting through Hugging Face Datasets, tokenization and a binary safe/unsafe classification head. He recommends starting with ModernBERT base before trying the large version. The inference example covers CPU use; for GPU inference, he directs viewers to enable FlashAttention.

Carpintero reports almost 85% accuracy on unseen benchmark data and 35 ms per classification. His demo tests several research attack prompts, but he explicitly describes the result as a baseline rather than a safety gold standard. The talk presents fine-tuning for under a dollar as his project result, not a guaranteed deployment cost.