Nemotron 3.5 Lightning: MTP vs DFlash and DSpark

NVIDIA researchers explain the training and inference choices behind Nemotron 3.5 Lightning. The discussion focuses on speculative decoding: a cheaper draft mechanism proposes tokens, and the target model accepts or corrects them. The speakers describe this as preserving the target distribution in theory, while noting that implementation differences can affect individual responses.

The comparison covers an integrated multi-token prediction (MTP) head, DFlash and DSpark. MTP drafts tokens sequentially; DFlash proposes a block in parallel. For people who run models locally, the practical choice depends on hardware, model and concurrent requests. The researchers favor parallel drafting for memory-bound workloads such as DGX Spark, while explaining why shorter MTP drafts can suit higher concurrency. They recommend measuring the workload rather than treating a batch-size estimate as a universal crossover point.

The team says Lightning builds on Nano, with distillation from Super to improve accuracy and an MTP head trained alongside it. Domain fine-tuning can reduce draft acceptance rates, so they advise adapting the head too. The released DFlash and DSpark drafts are specific to Lightning 3.5; the speakers point to ModelOpt training recipes for creating drafts for other models.