We are in the middle of a change in how we use computers. Artificial intelligence is not something that a few people use. It is changing how we plan our computer systems, how we make computer chips, and how we run our businesses. The size of this change is huge: big companies are using thousands and thousands of artificial intelligence computers. In each group they are training huge models that have trillions of parts and they are using a lot of power in each area.

People in the computer business are right to be excited about artificial intelligence. It can really change how we do things, helping us learn things faster and do our jobs better. Now we have to think about something that is not so exciting: the energy it takes to run artificial intelligence. This is becoming a big problem that is slowing down new ideas.

What people usually do about this problem is pretty obvious. They try to make the computers work better. They try to keep the computers cool. They buy power that is better for the earth. These things are not enough anymore. The idea that just making the computers work better will save energy is not true anymore. Actually, it might be hiding a good way to make a big difference. Artificial intelligence is a part of this problem and we need to think about how to make artificial intelligence work better with the energy we have.

The invisible

In AI infrastructure, memory and storage are often seen as supporting parts and not a top priority. In modern AI systems, the parts that help memory work like special memory called HBM, DRAM, SSDs, and the connections between them, can use up to 50% of the total power. This depends on how the system is set up and what it’s used for. As AI models get bigger and more data is moved around, the power used by memory and storage becomes more important.

Different ways of processing information, such as edge and distributed systems, have been adopted to make AI work better. Data has a kind of pull that brings processing power to it and a huge amount of data is created every day. It’s estimated that by 2025, over 402 exabytes of data will be generated daily. AI is moving to where the data is, and that means it’s moving to memory and storage. These new systems need memory, which means there are more chances to save power.

Moving data from memory to processing chips from storage to memory across different parts of the system now costs a lot of energy. A study by semi-analysis found that operations involving moving data like saving progress and communicating between parts are among the biggest causes of sudden power spikes in large AI systems. These spikes can be tens of megawatts, showing how important memory is for both energy use and keeping the power grid stable. Unlike processing power, which gets better with technology, memory systems have improved more slowly over time.

Moving forward with a plan

For people in charge of infrastructure this change is not something interesting to think about. It is something they really need to do. The amount of power available is now a problem for companies that want to get bigger. The total cost of owning things is getting too high. Companies are being told they must be sustainable. The demand for artificial intelligence is growing faster than traditional infrastructure can handle.

The amount of work that big artificial intelligence data centres can do is no longer limited by the computers they have. It’s by how much energy they can get from the power grid.

Memory-led efficiency is a way to solve the problem. It is available now and it can be used to make things better as the problem gets bigger. It lets big companies utilize capacity without using more power. It also reduces the cost of cooling and getting things ready. It helps infrastructure teams meet the demands of new artificial intelligence workloads without hurting the environment or spending too much money.

How the future looks

When we think about the future, the question is not if artificial intelligence will change the world: it will. The question is how we will make that happen. The answer is not about making computers faster or data centres colder. It is also about making smarter systems that use memory and storage in a better way to save energy.

Investing in technologies, partnerships, and systems is needed to make memory a key part of artificial intelligence at a big scale. Artificial intelligence is very powerful. The power to use artificial intelligence in a way that is efficient, sustainable, and can be used everywhere will define the next era of new ideas and innovations.