IBM has introduced the Granite 4.0 Tiny Preview, the initial release of its most compact model in the forthcoming Granite 4.0 language model family, targeting the open source community. Emphasizing both minimal memory requirements and high efficiency, Granite 4.0 Tiny Preview leverages FP8 precision to allow multiple, long-context (128,000 token) sessions to be executed concurrently on widely accessible consumer GPUs. Despite being only partially trained—processing about 2.5 trillion of an expected 15 trillion training tokens—the model already achieves performance competitive with IBM´s Granite 3.3 2B Instruct model, while utilizing significantly fewer parameters and offering approximately 72% less memory usage.
Set to become the smallest offering within the Granite 4.0 family, the Tiny Preview is just the start of a suite that will also include Small and Medium variants, all slated for official release later this summer. The current Granite 4.0 iteration reinforces IBM’s commitment to accessible and efficient enterprise large language models (LLMs), making high-performance Artificial Intelligence more attainable for a broad user base. The preview model, distributed on Hugging Face under the open-source Apache 2.0 license, is not yet recommended for enterprise deployment, but it is designed for experimentation by developers—even those with limited hardware resources.
The architecture introduced in Granite 4.0 Tiny is still awaiting integration with key platforms such as Hugging Face Transformers and vLLM, with support expected in the near future. Additionally, IBM plans to enable easy local deployment of Granite 4.0 models through partners like Ollama and LMStudio following the official release. As the training of Granite 4.0 Tiny progresses, IBM anticipates its final performance will match that of larger predecessors, such as the Granite 3.3 8B Instruct, further closing the gap between compactness and robust language capabilities. This approach underscores IBM´s strategic focus on practical, resource-conscious Artificial Intelligence innovation for both research and enterprise environments.
