A few months ago, Nvidia’s CEO Jensen Huang outlined a growing datacentre problem. Talking to CNBC news, he revealed that not only will the company’s new next-generation chip architecture – the Blackwell GPU – cost $30,000 to $40,000, but Nvidia itself spent an incredible $10 billion developing the platform.
These figures reflect the considerable cost of trying to draw out more performance from current AI accelerator products. Why are costs this high?
Essentially, the performance demand needed to power the surge in AI development is increasing much faster than the abilities of the underlying technology used in today’s datacentre AI processors. The industry’s current solution is to add more silicon area, more power and, of course, more cost. But this is an approach pursuing diminishing returns.
Throw in the sizeable infrastructure bill that comes from activities such as cooling and power-delivery, not to mention the substantial environmental impact of datacentres, and the sector is facing a real necessity to create a new way of building its AI accelerators. This new way, as it turns out, is already being developed.
Optical processing techniques are an innovative and cost-efficient means to provide the necessary jump in AI performance. Not only will the technology accomplish this, however, but it will also simultaneously enhance the sector’s energy efficiency. This technique is 3D, or “free space”, optics.
Making the jump to 3D
3D optical compute is a perfect match for the maths that makes AI tick. If it can be harnessed effectively, it has the potential to generate immense performance and efficiency gains.
3D optics is one of two optics solutions available in the tech landscape – the other, is integrated photonics.
Integrated photonics is ideally suited to interconnect and switching where it holds huge potential. However, trials using integrated photonics for AI processing have shown that the technology can’t match the performance demand required for processing, like the fact it isn’t easily scalable and lacks compute precision.
3D optics, on the other hand, surpasses the restrictions of both integrated photonics and electronic-only AI solutions. Using just 10% of the power of a GPU, the technology easily provides the necessary leap in performance by using light rather than electrons to compute and performs highly parallel computing.
For datacentres, using a 3D optical AI accelerator will give them the many benefits seen in the optical communications we use daily, from rapid clock speeds to negligible energy use. These accelerators also offer far greater scalability than their ‘2D’ chip counterparts as they perform computations in all three spatial dimensions.
The process behind the processor
Copying, multiplying and adding. These are the three fundamental operations of matrix multiplication, the maths behind processing. The optical accelerator carries out these steps by manoeuvring millions of individual beams of light. In just one clock cycle, millions of parallel operations occur, with very little energy consumed. What’s amazing is that the platform becomes more power efficient as performance grows due to its quadratic scaling abilities.
Memory bandwidth can also impact an accelerator’s effectiveness. Optical processing enables a greater bandwidth without needing a costly memory chip, as it can disperse the memory across the vector width.
Certain components found in optical processors already have evidence of successful use in datacentres. Google’s Optical Circuit Switch has used such devices for years, proving that employing similar technology is effective and reliable.
Powering the AI revolution sustainably
Google’s news at the start of July illustrated the extent to which AI has triggered an increase in global emissions. It highlights just how much work the industry has to do to reverse this trend, and key to creating this shift will be a desire from companies to embrace new methods and tools.
It’s worth remembering that between 2015-2019, datacentre power demand remained relatively stable even as workloads almost trebled. For the sector, it illustrates what’s possible. We need to come together to introduce inventive strategies that can maintain AI development without consuming endless energy.
For every Watt of power consumed, more energy and cooling are needed and more emissions are generated. Therefore, if AI accelerators require less power, datacentres can also last longer and there is less need for new buildings.
A sustainable approach also aligns with a cost-efficient one. Rather than use expensive new silicon technology or memory, 3D optical processors can leverage optical and electronic hardware currently used in datacentres. If we join these cost savings with reduced power consumption and less cooling, the total cost of ownership is a tiny portion of a GPU.
An optical approach
Spiralling costs and rocketing AI performance demand mean current processors are running out of steam. Finding new tools and processes that can create the necessary leap in performance is crucial to the industry getting on top of these costs and improving its carbon footprint.
3D optics can be the answer to AI’s hardware and sustainability problems, significantly increasing performance while consuming a fraction of the energy of a GPU processor. While broader changes such as green energy and sustainable manufacturing have a crucial part to play in the sector’s development, 3D optics delivers an immediate hardware solution capable of powering AI’s growth.
- Data & AI
- Sustainability Technology