Researchers from the Center for Cognition and Sociality and the Data Science Group at the Institute for Basic Science (IBS) have uncovered a noteworthy parallel between the memory processing of artificial intelligence (AI) models and the human brain's hippocampus. This revelation sheds light on the process of memory consolidation in AI systems, transforming short-term memories into enduring ones.
In the pursuit of Artificial General Intelligence (AGI), spearheaded by influential entities such as OpenAI and Google DeepMind, the study of human-like intelligence has become a focal point. At the heart of these endeavors lies the Transformer model, whose foundational principles are now under intense scrutiny.
Understanding how AI systems learn and retain information is pivotal to their efficacy. The researchers applied principles of human brain learning, specifically focusing on memory consolidation through the NMDA receptor in the hippocampus, to AI models.
The NMDA receptor, acting like a sophisticated door in the brain, facilitates learning and memory formation. The team made an intriguing discovery: the Transformer model appears to employ a gatekeeping process akin to the brain's NMDA receptor, prompting further exploration into whether the Transformer's memory consolidation can be controlled similarly.
In the human brain, a low magnesium level weakens memory function. The researchers found that mimicking the NMDA receptor could enhance long-term memory in the Transformer model. Just as changing magnesium levels impact memory strength in the brain, adjusting the Transformer's parameters to mirror the NMDA receptor's gating action resulted in improved memory in the AI model.
C. Justin Lee, a neuroscientist director at the institute, emphasized the significance of the research in advancing both AI and neuroscience. He stated, "This research makes a crucial step in advancing AI and neuroscience. It allows us to delve deeper into the brain's operating principles and develop more advanced AI systems based on these insights."
CHA Meeyoung, a data scientist in the team and at KAIST, highlighted the potential for creating low-cost, high-performance AI systems that learn and remember information like humans. She noted, "The human brain is remarkable in how it operates with minimal energy, unlike the large AI models that need immense resources. Our work opens up new possibilities for low-cost, high-performance AI systems."
This study stands out for its incorporation of brain-inspired nonlinearity into an AI construct, marking a significant leap in simulating human-like memory consolidation. The intersection of human cognitive mechanisms and AI design not only holds promise for creating advanced AI systems but also provides valuable insights into the intricacies of the brain through AI models.
