Agent Lightning v1.0 vs LEGO RL: agent training explained

This research overview examines Agent Lightning v1.0 and introduces LEGO RL as another approach to reinforcement learning for agents with a harness. The speaker explains how a harness controls tool execution and constructs each model prompt. When it summarizes context or starts a sub-agent, successive API calls may stop forming one continuous token history.

The Agent Lightning discussion focuses on preserving the exact tokens the model encountered. In the speaker's account, the trainer joins calls only when their token sequences continue exactly; otherwise, it keeps separate training fragments. A Python bug repair example shows how a conversation summary creates a new fragment within the same task attempt.

The speaker describes an attempt, or rollout, as a complete task run, including runs that fail or reach a limit. Rewards and total training weight belong to that rollout, so an attempt with 50 calls does not gain more influence merely by producing more fragments than one with two calls. The stated limitation is that this does not identify which fragment caused success.

The final section discusses collocated asynchronous training, where rollouts and weight updates share a GPU pool. It then introduces Huawei's LEGO RL work on coding agents and mixture-of-experts models. The detailed discussion of its additional problems is deferred to another video. This is a conceptual comparison, rather than an installation guide for a local AI agent.