rLLM and Qwen3: training a 4B finance agent versus 235B

Charles Dickens presents a Snorkel AI and UC Berkeley Sky Computing Lab experiment in training a small AI agent for financial questions. He reports that their Qwen3 4B model reached almost 60% pass@1, compared with 51% for its 235B variant. These results concern a simulated financial environment, rather than a general claim that smaller models outperform larger ones.

The team built FinQA from SEC 10-K filings, extracting roughly 6,900 SQL tables and generating question-answer pairs. Verification combined programmatic checks, independent agent reviews and human expert review. The held-out evaluation contained 290 examples, with company separation across dataset splits.

Training used the open source rLLM framework, GRPO and a binary correctness reward judged by GPT 5 nano against reference answers. Dickens puts the training cost below $500 on eight H100s, including roughly $420 for compute and $40 for the judge API. This is a research training account, not a local LLM installation guide.

The reported failures included invented table schemas, oversized query results and repeated unsuccessful strategies. Training on simpler single-table questions produced the strongest gains in the team's ablations. Dickens also reports transfer to questions involving two to five tables and a slight improvement on the BFCL tool-calling benchmark. A binary reward outperformed the more elaborate reward rubric they tested.