StarCoder is a large language model (LLM) designed specifically for code generation, developed by the BigCode collaborative (Hugging Face & ServiceNow). It features a 15.5 B parameter decoder-only architecture with Multi‑Query Attention and Fill‑in‑the‑Middle training on a trillion tokens from “The Stack,” covering 80+ programming languages . A refined version fine-tuned on 35 B Python tokens achieves ~40 % pass@1 on HumanEval, outperforming many commercial models .The second iteration, StarCoder2, offers models of 3B, 7B, and 15B parameters, trained on 3–4 T tokens across 600+ programming languages using advanced attention (Group Query) and extended 16K token context . The flagship 15B model reaches or surpasses the capabilities of larger proprietary models despite being more lightweight .
StarCoder
Open-access code model family from the BigCode project (Hugging Face and ServiceNow Research). StarCoder2 ships in 3B, 7B, and 15B sizes, trained on The Stack v2 across 600+ languages, with the 15B variant passing 70% on HumanEval.
Reviews
- No reviews yet.
See something outdated? Suggest an update