Large language models (LLMs) have gained prominence for their ability to generate text based on prompts, facilitating tasks such as document summarization, brainstorming, and information retrieval. A recent study by researchers from the University of Georgia and Mayo Clinic delved into the biological knowledge and reasoning capabilities of various LLMs, with notable findings.
The study, pre-published on the arXiv server, highlighted the increasing impact of artificial intelligence (AI) on biological research. Zhengliang Liu, co-author of the paper, emphasized the evolution of LLMs, particularly citing the influence of OpenAI's ChatGPT introduced in November 2022. The study aimed to assess LLMs, including GPT-4, GPT-3.5, PaLM2, Claude2, and SenseNova, focusing on their aptitude for comprehending and reasoning through biology-related questions.
Researchers meticulously designed a 108-question multiple-choice exam covering molecular biology, biological techniques, metabolic engineering, and synthetic biology. Jason Holmes, a co-author, explained that the purpose was to evaluate the models' performance, consistency, and variation in answers across different phrasings of the same question.
GPT-4 emerged as a standout performer, achieving an average score of 90 across multiple trials and showcasing reliability and consistency. Xinyu Gong, another co-author, underscored GPT-4's capacity to assist biology research and education. The findings suggest that LLMs, particularly GPT-4, can effectively handle various biology-related questions and concepts.
The researchers envision broader applications for LLMs in biology, including supporting research, aiding in education, and generating testable hypotheses. Future studies will address computational demands and privacy issues associated with GPT-4, aiming to make advanced tools more accessible to the biology community. The team plans to explore GPT-4V's vision capabilities for multimodal analyses, focusing on drug discovery and development in synthetic biology.
In essence, this study signifies a pioneering effort in integrating advanced AI capabilities, especially LLMs, into the dynamic field of biology. It marks a shift where AI is not merely a supportive tool but a central element in navigating and deciphering the complex biological landscape, paving the way for potential scientific breakthroughs and advanced educational tools.
