At Google I/O 2024, Google introduced the latest iteration of its custom hardware units, named Trillium, marking the sixth generation of its Tensor Processing Units (TPUs).
Trillium TPUs offer significant advancements over their predecessors, with 4.7 times increased peak compute performance per chip compared to the previous TPU v5es. They also feature double the High Bandwidth Memory (HBM) capacity, bandwidth, and Interchip Interconnect (ICI) bandwidth.
Trillium chips include the third generation of SparseCore, a specialized accelerator designed for processing ultra-large embeddings commonly used in ranking and recommendation workloads.
The new custom chips are not only faster but also 67% more energy-efficient compared to the TPU v5e. Trillium pods can contain up to 256 TPUs, providing cloud customers with scalable computing solutions.
Google CEO Sundar Pichai emphasized the importance of Trillium in meeting the growing demand for machine learning compute, stating that industry demand has grown exponentially in recent years. Trillium is designed to power AI training workloads faster, with reduced latency and lower cost.
Trillium will be available to Google Cloud customers by late 2024, offering enhanced AI services and affordability. Dr. Andrew Rogoyski from the University of Surrey's Institute for People-Centred AI praised Google's latest TPU as a critical enabler for affordable AI services in the future, highlighting the importance of AI hardware development in meeting growing demand.
Additionally, Google announced its support for Nvidia's new Blackwell hardware, scheduled to begin in early 2025. Google Cloud aims to be among the first cloud providers to offer support for Nvidia's latest GPUs, further enhancing its capabilities in AI and machine learning services.
