Microsoft has recently unveiled its latest small language model, Phi-2, boasting 2.7 billion parameters and positioned as an upgrade to Phi-1.5. Accessible through the Azure AI Studio model catalogue, Phi-2 has garnered attention by reportedly surpassing larger models like Llama-2, Mistral, and Gemini-2 in various generative AI benchmark tests.
Phi-2, announced by Satya Nadella at Ignite 2023, is a product of Microsoft's research team. The generative AI model is said to possess attributes such as "common sense," "language understanding," and "logical reasoning." Microsoft claims that Phi-2 can even outperform models 25 times its size on specific tasks.
Trained using "textbook-quality" data, including synthetic datasets, general knowledge, theory of mind, and daily activities, Phi-2 is a transformer-based model with a next-word prediction objective. Microsoft's training process involved 96 A100 GPUs over a 14-day period, showcasing efficiency compared to the extensive training requirements of larger models like GPT-4.
Phi-2's capabilities extend beyond language processing; it demonstrates proficiency in solving complex mathematical equations and physics problems. Additionally, it can identify errors in student calculations.
In benchmark tests covering commonsense reasoning, language understanding, math, and coding, Phi-2 has shown superiority over the 13B Llama-2 and 7B Mistral. Even against the formidable 70B Llama-2 LLM and the Google Gemini Nano 2, a 3.25B model, Phi-2 emerges as the more adept performer.
The significance of Phi-2's compact size lies in its cost-effectiveness, requiring less power and computing resources. Smaller models like Phi-2 can be trained for specific tasks, offering reduced latency when running natively on devices. Developers can now leverage Phi-2 via Azure AI Studio, marking a notable advancement in the landscape of small language models.
