Stability AI, a leading AI company, is renowned for its stable diffusion text-to-image generative AI models. However, their latest release, StableLM Zephyr 3B, proves that they are continuously expanding their scope. This 3 billion parameter large language model (LLM) is specifically designed for chat use cases, including text generation, summarization, and content personalization.
Unlike its predecessor, the 7 billion StableLM models, StableLM Zephyr 3B is smaller and optimized. This reduction in size presents several benefits for users. Firstly, it allows for deployment on a wider range of hardware and leads to a lower resource footprint. Despite its smaller size, StableLM Zephyr 3B still provides rapid responses. Moreover, the model has been optimized for Q&A and instruction following tasks, making it incredibly versatile. According to Emad Mostaque, CEO of Stability AI, StableLM Zephyr 3B matches the base performance of the larger models while being only 40% of their size.
StableLM Zephyr 3B is an extension of the pre-existing StableLM 3B-4e1t model rather than an entirely new model. Inspired by HuggingFace’s Zephyr 7B model, Stability AI’s Zephyr has been developed under the open-source MIT license and is designed to act as an assistant. This model utilizes the Direct Preference Optimization (DPO) training approach, which was previously used in larger 7 billion parameter models. StableLM Zephyr stands out as one of the first models to employ DPO with a smaller 3 billion parameter size. Stability AI employed DPO with the UltraFeedback dataset from the OpenBMB research group, which includes over 64,000 prompts and 256,000 responses.
With its combination of DPO, smaller size, and optimized training data set, StableLM Zephyr 3B delivers solid performance. In the MT Bench evaluation, for example, it outperforms larger models such as Meta’s Llama-2-70b-chat and Anthropric’s Claude-V1. This achievement highlights Stability AI’s commitment to providing cutting-edge solutions to its users.
StableLM Zephyr 3B is the latest addition to Stability AI’s ever-expanding range of new models. In the past few months, the generative AI startup has released StableCode, a generative AI model for application code development, as well as Stable Audio, a text-to-audio generation tool. Later in November, they entered the video generation space with a preview of Stable Video Diffusion. However, despite their foray into various domains, Stability AI remains dedicated to their text-to-image generation foundation. SDXL Turbo, a faster version of their flagship SDXL text-to-image stable diffusion model, was recently released. Emad Mostaque assures us that this is just the beginning of the innovative solutions that Stability AI will introduce. Their belief in the superiority of small, open, and performant models that are tailored to users’ data drives their constant pursuit of improvement.
Stability AI’s latest model, StableLM Zephyr 3B, sets a new standard for chat use cases. With its smaller size, optimized performance, and impressive metrics, it offers a range of benefits to users. Stability AI’s commitment to innovation is evident through their continuous release of new models in various domains. As they continue to push the boundaries of generative AI, Stability AI remains firmly rooted in their text-to-image generation foundation. The future holds even more exciting developments from Stability AI, as they strive to provide users with increasingly advanced and tailored AI solutions.