Codex, NeMo RL and NeMo Gym: autoresearch setup

This tutorial presents an AI agent workflow for reinforcement learning research with Codex, NVIDIA NeMo RL, NeMo Gym and Brev. Reusable skills describe where to store files and checkpoints, preserve session decisions, and guide experiments through baselines, metrics and stopping conditions.

The setup uses a Brev launchable configured with the research repository and skills on a single NVIDIA L40S GPU instance. The presenter describes it as internet connected and separate from the corporate network. This is a remote GPU workflow rather than an offline demonstration of how to run models locally.

In the paper-to-code example, the presenter reports that Codex implemented an algorithm and passed its tests, but training initially failed because the installed driver did not meet the workflow's CUDA 13 requirement. Further attempts resolved dependency conflicts and started training. Validation accuracy remained low, prompting the researcher to request larger batches.

A proposed run of 80 training steps was estimated to take about 100 hours. The researcher instead assigned a 20-hour budget and asked Codex to adjust the run. Separately, the introduction claims a visual star-counting experiment improved Qwen accuracy from about 25% to about 96%; the live walkthrough does not establish that result. Researchers still set goals, review results and decide how much compute to spend.