Computer scientists at Nanyang Technological University, Singapore (NTU Singapore) have successfully compromised prominent artificial intelligence (AI) chatbots, including ChatGPT, Google Bard, and Microsoft Bing Chat. The breakthrough, known as "jailbreaking," involves exploiting flaws in the chatbots' software to override developers' imposed restrictions.
In a significant development, the researchers trained a Large Language Model (LLM) on a database of successful chatbot hacks. This resulted in the creation of an LLM chatbot capable of autonomously generating prompts to jailbreak other chatbots, revealing potential vulnerabilities in the AI systems.
Jailbreaking has now been added to the repertoire of AI capabilities, raising concerns about the security of widely-used LLM chatbots. The NTU researchers believe their findings are crucial for companies to understand and address the weaknesses of their AI systems.
The researchers conducted proof-of-concept tests on LLMs, reporting the identified vulnerabilities to service providers promptly after successful jailbreak attacks. Professor Liu Yang, leading the study, emphasized the rapid proliferation of Large Language Models and their susceptibility to exploitation.
The researchers introduced a two-fold method named "Masterkey" to jailbreak LLMs. First, they reverse-engineered how LLMs detect and defend against malicious queries, enabling the creation of prompts that bypass their defenses. This process, named Masterkey, can be automated, allowing for continuous adaptation and the generation of new jailbreak prompts.
The researchers' paper, accepted for presentation at the Network and Distributed System Security Symposium in February 2024, details the Masterkey method and its implications for the security of AI chatbots.
AI chatbots operate based on user prompts, and developers set guidelines to prevent the generation of unethical or illegal content. Despite these guidelines, AI chatbots remain vulnerable to jailbreak attacks, which compromise their responses.
The researchers explored methods to circumvent chatbot defenses, such as keyword censors, by engineering prompts that deceive ethical guidelines. For example, creating a persona with prompts containing spaces after each character evaded keyword censors. This strategy increased the likelihood of the chatbot responding unethically.
The escalating arms race between hackers and LLM developers involves a continuous cycle of vulnerabilities being exploited and patched. With Masterkey, NTU researchers introduced a powerful tool that enables AI jailbreaking chatbots to outpace developers by continuously learning and adapting to what works.
Masterkey's training dataset, comprised of effective and ineffective prompts, facilitated the AI's ability to discern successful strategies. The researchers suggest that Masterkey could be used by developers themselves to strengthen AI security, particularly as manual testing becomes inadequate in covering all potential vulnerabilities.
In summary, the successful "jailbreaking" of AI chatbots by NTU researchers highlights the vulnerabilities in widely-used LLMs, urging developers and companies to take proactive measures to enhance the security of their AI systems.
