The Technology Innovation Institute (TII), under the Advanced Technology Research Council (ATRC) in Abu Dhabi, has launched a new large language model within the “Falcon” series, named “Falcon Mamba (7B).”
According to the global platform “Hugging Face,” this model stands out as the most distinguished among open-source SSLM (Self-Supervised Learning Models) AI models.
This model represents the first release in the “Falcon” series of SSLMs and differs from previous models that relied on transformer technologies.
The new “Falcon Mamba (7B)” is a significant addition to the research conducted by the institute, offering advanced tools and products to the community as an open-source resource.
Faisal Al-Bannai, Secretary General of the ATRC and Strategic Advisor on Advanced Technology Research, stated that TII has previously launched four models consecutively, reinforcing Abu Dhabi’s position as a global hub for research and development in AI and underscoring the UAE’s commitment to innovation.
He added, “Compared to transformer-based models, ‘Falcon Mamba (7B)’ outperforms Meta’s ‘Llama 3.1 8B’ model and the ‘Mistral 7B’ model, according to benchmarks set by the ‘Hugging Face’ platform.
It also tops the new benchmark comparison on the platform, surpassing all other open-source SSLM models.”
Dr. Najwa Aaraj, CEO of TII, affirmed, “The institute continues to push the boundaries of technology with the ‘Falcon’ series of AI models.
The ‘Falcon Mamba (7B)’ is a pioneering achievement that paves the way for future innovations to enhance human capabilities and improve the quality of life.”
SSLM models excel in understanding complex contexts that evolve over time, such as long texts, as their data storage systems do not require additional memory to process large volumes of information.
While transformer-based models are highly effective at recalling and utilizing previous information, enabling them to perform tasks like content generation, they require substantial computational power due to the need to compare each word with others.
SSLM models offer practical solutions in various fields, such as estimation, prediction, and control, and excel in natural language processing tasks like machine translation, text summarization, and audio and image processing.
In this context, Dr. Hakim Haseed, Acting Head of Research at the AI Unit of TII, stated, “We are proud to launch the ‘Falcon Mamba (7B)’ model and the success of our collaborative approach at the institute, which enhances our ability to develop advanced models.
This achievement marks a significant step towards advancing intelligent systems research and fostering innovation in generative AI.”
It is worth noting that the large “Falcon” models have been downloaded over 45 million times, reflecting their widespread success.
The “Falcon Mamba (7B)” model was released under the TII “Falcon 2.0” license, based on the “Apache 2.0” license, which includes a responsible AI usage policy.
