StarCoder2

StarCoder2 is a family of code generation models for developers who want code completion on their own hardware. These models predict continuations of code; they aren't instruction models, so conversational requests aren't their intended use.

• Code completion: Generate continuations using surrounding code as context, with training across programming languages from The Stack v2.
• Local inference: Run models through Hugging Face Transformers on a CPU, a GPU or multiple GPUs, with Text Generation Inference available for serving.
• Memory choices: Choose smaller or larger models to fit your hardware, and use bitsandbytes quantization to reduce the memory needed for model weights.
• Fine-tuning and evaluation: Adapt models to code or text datasets with LoRA through PEFT, then assess them with BigCode-Evaluation-Harness.

The fine-tuning approach combines LoRA with 4-bit quantization to reduce resource requirements. The Python repository uses the Apache 2.0 license.

StarCoder2 provides 3B, 7B and 15B coding checkpoints. The 3B model card identifies BigCode OpenRAIL-M v1 for model weights, including use restrictions. Apache 2.0 covers the associated repository code. Generated code can also carry attribution obligations from its source material.