Snowflake has launched Arctic, its new open and efficient language model, which competes with or even surpasses Meta’s latest model, Llama 3, in business tasks such as SQL instruction and code creation, with a reduced training cost.
The cloud data storage specialist company has pointed out that, generally, developing AI tools based on LLMs for companies is expensive and resource-intensive. That is why they have created a system that significantly reduces the spending on training and inference of LLMs, making this technology more accessible and affordable.
The Snowflake AI Research team has introduced Arctic, a high-quality LLM designed for companies, which is “intelligently efficient and completely open.” It excels in functions such as code generation and SQL instruction, and is distributed under the Apache 2.0 license.
According to Snowflake, Arctic matches or improves the capabilities of Meta’s AI models, Llama 38B and Llama 270B, in terms of business metrics for training and inference, all with less than half the computational budget.
Despite having a computational budget 17 times lower than Llama 370B, Arctic achieves comparable results in SQL instruction (with a score of 79.0 compared to 80.2 of Llama 370B) and in instruction tracking, with a score of 57.4 that surpasses Meta’s model of 43.6.
The company has also highlighted that Arctic maintains competitive performance without compromising overall efficiency. Arctic’s “high training efficiency” allows users and organizations to train custom models more economically.
To achieve this training efficiency, the company uses a hybrid transformer architecture with dense Mixture of Experts (MoE), combining a dense transformer model of 10B with a residual 128 x 3.66B MoE MLP. This results in a total of 480B and 17B active parameters selected by a two-level gate system, ensuring higher efficiency.
Arctic is a “completely open” LLM, as Snowflake shares the model weights under the Apache 2.0 license, facilitating the dissemination of AI research and training methodology. Additionally, Arctic can be used for both commercial and research purposes.
Finally, Snowflake has announced that Arctic is already available on platforms such as Hugging Face, Replicate, and Nvidia’s API catalog, and will soon be available on Amazon AWS, Microsoft Azure, Lamini, or Perplexity.
