In a recent paper, researchers from the Oxford Internet Institute issued a warning about the potential risks posed by artificial intelligence (AI) chatbots, particularly large language models (LLMs) like ChatGPT and Google Bard, to the integrity of scientific research. These LLMs, known for their ability to generate human-like text, are under scrutiny due to their capacity to provide inaccurate and biased information.
The Oxford researchers emphasize that users often place unwarranted trust in LLMs, perceiving them as reliable human-like resources. Brent Mittelstadt, Director of Research at the Oxford Internet Institute, notes that the design of LLMs contributes to this trust, as they are crafted to sound helpful and confident in their responses. However, the researchers argue that these responses can lack a factual basis and may present a biased or partial version of the truth.
One significant concern raised by Mittelstadt is the potential for LLMs to provide outputs that are "slightly wrong or slightly biased," particularly in areas where specific expertise is required, such as references to scientific articles. The researchers point out that LLMs do not guarantee accurate responses, and false outputs can be attributed to the datasets used to train these models. If these datasets contain false statements, opinions, or creative writing from internet content, it can prompt the generation of incorrect outputs.
A notable issue highlighted in the investigation is the secrecy surrounding LLM datasets. For instance, Google Bard's dataset, which ranks second in popularity to ChatGPT, was found to include various internet forums, personal blogs, and entertainment websites like Screenrant. The lack of transparency in the dataset raises concerns about the reliability of the information generated by these models.
To address these concerns, the Oxford researchers recommend using LLMs as "zero-shot translators" rather than as knowledge bases. This approach involves providing the model with a set of inputs containing reliable information or data and a request to perform a specific task. The researchers argue that responsible use of LLMs is crucial, especially in the scientific community where confidence in factual information is paramount.
The skepticism towards LLMs in scientific research is not unique to the Oxford researchers. Nature, a leading scientific publication, has taken a stance on the issue by refusing to accept LLM tools as credited authors on research papers due to liability concerns. Nature also requires authors to disclose the use of large language models in a dedicated section of their papers to ensure transparency in the research process.
As AI chatbots continue to gain popularity and integration into various fields, including scientific research, it becomes imperative for the scientific community to establish clear guidelines and guardrails to ensure the responsible and accurate use of these powerful tools. The debate surrounding the role of LLMs in shaping scientific discourse is far from settled, and researchers and institutions must navigate these challenges to uphold the integrity of their work.
