The launch of ChatGPT marked a significant milestone in the era of large language models (LLMs). Alongside OpenAI's contributions, other notable LLMs include Google's LaMDA family, the BLOOM project by Microsoft, Nvidia, and others, Meta's LLaMA, and Anthropic's Claude.
According to an April 2023 Arize survey, 53% of respondents expressed intentions to deploy LLMs within the next year or sooner, indicating a growing trend in the adoption of these advanced language models. This includes the creation of "vertical" LLMs tailored to specific domains such as life sciences, pharmaceuticals, insurance, and finance.
However, deploying an LLM comes with challenges, as evidenced by the tendency of these models to produce inaccurate information, commonly referred to as "hallucinations." This issue raises concerns, diverting attention from essential considerations in the processes generating such outputs.
The Costly Journey of Training and Deploying LLMs
One of the primary challenges in utilizing LLMs is the substantial operating expenses involved due to the intense computational demands. The H100 GPU from Nvidia, a preferred choice for LLMs, commands a steep price, with each chip selling for around $40,000 on the secondary market. Estimates suggest it could cost approximately $240 million in GPUs alone to train an LLM comparable to ChatGPT-3.5, requiring about 6,000 chips.
Power consumption adds another layer of expense. Training a model is estimated to consume about 10 gigawatt-hours (GWh) of power, equivalent to the yearly electricity use of 1,000 U.S. homes. Once trained, the daily power consumption of running models like ChatGPT-3.5 reaches 1 GWh, matching the combined daily energy usage of 33,000 households.
This power consumption poses potential challenges for user experience, particularly when running LLMs on portable devices. Heavy usage on such devices could quickly deplete batteries, becoming a significant barrier to consumer adoption.
In navigating these challenges, organizations must carefully weigh the benefits of deploying LLMs against the substantial costs involved. As the adoption of large language models continues to rise, addressing these operational and financial hurdles will be essential for their successful integration into various industries.
