Meta Platforms, formerly known as Facebook, has officially launched Code Llama, a generative AI large language model (LLM) specifically designed for programming. This open-source model is licensed for commercial use and aims to support software engineers in various sectors, including research, industry, open-source projects, NGOs, and businesses. With the release of Code Llama, Meta joins the competition against OpenAI’s Codex, Microsoft’s Codex-powered Github Copilot, and Stack Overflow’s OverflowAI.
Meta further elucidates on Code Llama’s structure, highlighting its “family” of LLMs for code. This family includes three main members: a 7-billion-parameter model, a 13-billion-parameter model, and a 34-billion-parameter model. All three models have been trained on a massive 500 billion tokens. The smaller models are designed to run efficiently on fewer GPUs, with the 7-billion-parameter model capable of operating on a single GPU. Meta claims that the smaller models outperform their larger 34-billion-parameter counterpart in terms of speed.
One notable feature of the LLaMA family of models is the support for up to 100,000 tokens for prompts. This extended context allows users to provide more information and context from their own codebase, ensuring that the generated code is more relevant and aligned with the intended outcome. This enhanced contextual understanding sets Code Llama apart from some of its competitors.
In addition to the main models, Code Llama also introduces two fine-tuned models: one for Python and one for Instruct. The Instruct model has been fine-tuned specifically to generate helpful and safe answers in natural language. According to Meta, this model should be used when generating new code from natural language prompts, as it prioritizes providing safer and more expected responses, even if they may be less creative.
Meta has made Code Llama available for download directly from their platform, and the source code can be found on GitHub. By offering this model to the programming community, Meta aims to foster collaboration and innovation among developers worldwide.
With the introduction of Code Llama, Meta Platforms has positioned itself as a key player in the field of AI language models tailored for programming. The specialized features, support for multiple programming languages, and the ability to provide extensive contextual understanding make Code Llama a valuable tool for software engineers across various domains. As the competition heats up in this space, it will be interesting to see how Code Llama stacks up against the existing offerings from OpenAI, Microsoft, and others.