The Allen Institute for AI (AI2) made waves today with the introduction of OLMo, a groundbreaking open-source large language model (LLM) designed to revolutionize the landscape of AI development. Founded in 2014 by the late Microsoft co-founder Paul Allen, AI2 aims to drive a critical shift in AI development by offering an alternative to current restrictive and closed models.
Unlike its predecessors, OLMo sets itself apart by providing not only the model code and weights but also the training code, training data, associated toolkits, and evaluation toolkits. Furthermore, AI2 has released OLMo under the Apache 2.0 License, ensuring that all code, weights, and intermediate checkpoints are freely accessible to the public.
The unveiling of OLMo marks a significant moment for open-source AI, which has been striving to catch up with closed, proprietary LLMs like OpenAI's GPT-4 and Anthropic's Claude. Just yesterday, the CEO of Paris-based open-source AI startup Mistral confirmed the imminent release of a new open-source AI model nearing GPT-4 performance. Additionally, Meta recently launched an improved version of its code generation model, Code Llama 70B, while anticipation grows for the third iteration of its Llama LLM.
However, open-source AI has faced criticism from some researchers, regulators, and policymakers. Despite this, AI2 remains steadfast in its commitment to openness and transparency. The OLMo framework provides comprehensive AI development tools, including full pretraining data, training code, model weights, and evaluation metrics, available for public access. With over 500 checkpoints per model and an extensive evaluation suite, OLMo offers unprecedented transparency in LLM development.
Hanna Hajishirzi, OLMo project lead and senior director of NLP Research at AI2, emphasized the importance of transparency in AI development. She likened the lack of access to training data to conducting drug discovery without clinical trials, highlighting the necessity of understanding how AI models function scientifically.
Nathan Lambert, an ML scientist at AI2, expressed his excitement for OLMo, emphasizing its potential to drive new approaches to ML research and deployment. Lambert noted that OLMo's openness enables researchers to explore new research directions at every stage of development, a capability previously unavailable due to limited information and tools.
The announcement of OLMo has garnered praise from the open-source AI community, with Jonathan Frankle, chief scientist at MosaicML and Databricks, calling it "a giant leap for open science." Hugging Face CTO lauded OLMo for pushing the boundaries of open-source AI, while Meta chief scientist Yann LeCun highlighted the vibrant community fostered by open source as instrumental in driving innovation in generative AI.
Overall, AI2's introduction of OLMo marks a significant milestone in the advancement of open-source AI, promising to unlock new opportunities for innovation and collaboration in AI development.
