Managing the variable costs associated with graphics processing units (GPUs) is becoming increasingly important as they are the key drivers behind the AI revolution. GPUs are used to power large language models (LLMs) that are the backbone of chatbots and various other AI applications. The fluctuating prices of these chips will require businesses to learn how to effectively manage their costs, which is a challenge for industries that have little to no experience in dealing with volatility in costs.
Industries such as mining and logistics are already familiar with managing fluctuating costs, such as energy and shipping costs. They have developed strategies to balance different energy sources or shipping routes to achieve the right combination of availability and price. However, industries like financial services and pharmaceuticals, which are poised to benefit greatly from AI, will need to quickly learn how to manage variable costs associated with GPUs.
Nvidia is the leading provider of GPUs, which is why its valuation has soared in recent years. GPUs are highly sought after because they have the capability to process numerous calculations in parallel, making them ideal for training and deploying LLMs. The demand for GPUs is expected to continue to rise as businesses rush to implement AI applications. According to investment firm Mizuho, the total market for GPUs could grow tenfold over the next five years to reach over $400 billion.
However, the supply of GPUs is uncertain and hard to predict. Factors such as manufacturing capacity and geopolitical considerations can impact the availability of GPUs. Many GPUs are manufactured in Taiwan, whose independence is threatened by China. As a result, the supply of GPUs has been limited, with some companies waiting up to six months to receive Nvidia’s powerful H100 chips. This means that businesses relying on GPUs for AI applications need to be prepared to manage the variable costs associated with them.
There are several strategies that businesses can employ to manage GPU costs. One approach is for companies to manage their own GPU servers instead of relying on cloud providers. While this creates additional overhead, it provides greater control and potentially lower costs in the long run. Companies may also choose to buy up GPUs defensively, even if they don’t have immediate plans for their use. These defensive contracts ensure that they have access to GPUs for future needs and give them an advantage over competitors.
Optimizing costs also involves securing the right type of GPUs for specific purposes. Not all GPUs are the same, and different applications require different levels of processing power. For most companies, lower performance GPUs may be sufficient for running inference work, which involves running data against an existing model. This approach allows for greater cost efficiency.
Geographic location can also impact GPU costs. GPUs consume a significant amount of power, and the cost of electricity contributes to their unit economics. Locating GPU servers in regions with access to cheap and abundant power, like Norway, can significantly reduce costs compared to regions with higher electricity costs.
CIOs should also carefully consider the trade-offs between cost and quality when it comes to AI applications. They may be able to use less computing power for applications that require less accuracy or are less strategically important to their business. Switching between different cloud service providers and AI models can also help optimize costs by taking advantage of different pricing structures and performance capabilities.
Demand forecasting for GPUs is a challenge due to the rapid advancement of AI computing. Vendors are constantly developing new LLMs with more efficient architectures, and chip makers are working on techniques to make inference more efficient. Additionally, new applications and use cases can influence GPU demand, making it difficult for companies to accurately predict their future GPU needs.
With the continued growth of AI development, businesses need to start preparing for the volatile costs associated with GPUs. The global revenue associated with AI is projected to grow significantly in the coming years, presenting a great business opportunity for chip makers like Nvidia. However, for businesses that are new to managing variable costs, it is essential to start planning and implementing cost management strategies now.
In conclusion, managing variable costs for GPUs is becoming a critical discipline for businesses as they rely more on AI applications. Strategies such as managing GPU servers, securing the right type of GPUs, considering geographic location, and optimizing cost and quality trade-offs can help businesses effectively manage GPU costs. However, the challenge of accurately forecasting GPU demand requires businesses to stay updated on the rapid advancements in AI computing and be prepared to adapt their strategies accordingly.
Source link
