Ahead of its annual Worldwide Developers Conference (WWDC), Apple has unveiled OpenELM, a suite of open-source language models designed for on-device tasks, signaling its ongoing advancements in artificial intelligence (AI).
Contrary to the perception of Apple lagging behind in the AI race, the introduction of OpenELM demonstrates the tech giant's longstanding efforts in AI development. Apple's traditionally secretive nature has obscured its AI initiatives, but the release of OpenELM sheds light on its progress in this field.
OpenELM comprises a family of small language models optimized for running efficiently on Apple devices like iPhones and Macs. Employing a layer-wise scaling strategy, OpenELM allocates parameters within each layer of the transformer model, resulting in enhanced accuracy. With four different parameter sizes—270M, 450M, 1.1B, and 3B—trained on public datasets, OpenELM offers performance comparable to other open language models while requiring less training data.
One of the key features of OpenELM is its suitability for on-device use, enabling AI-powered tasks without relying on cloud servers. Despite using half the training data compared to similar models like OLMo, OpenELM reportedly outperforms them. The models have been trained on CoreNet, an open-source library, facilitating efficient inference and fine-tuning on Apple devices.
In a departure from previous practices, Apple has provided a comprehensive release of OpenELM, including the framework for training and evaluation, training logs, multiple checkpoints, and pre-training configurations. Additionally, Apple has shared code to convert models to the MLX library for inference and fine-tuning on its devices. This move aims to empower the open research community and foster future research endeavors.
The timing of OpenELM's release, preceding the WWDC event in June, suggests potential AI-focused features in Apple's upcoming iOS 18. By making OpenELM open-source, Apple not only showcases its commitment to on-device AI but also adopts a more transparent approach, diverging from its previous secretive practices.
The emergence of OpenELM reflects a broader trend among tech giants, including Microsoft, towards developing small AI models for various applications. Apple's embrace of on-device AI and open-source release signifies a strategic shift, offering insights into its future direction in AI development.
