Compute is no longer a concern of the back office. According to McKinsey, data centers worldwide will require around $7 trillion in capital expenditures by 2030 to keep pace with demand. Of that, $5.2 trillion is earmarked for AI-specific infrastructure. Global data center capacity is set to triple, with AI workloads accounting for 70 percent of that appetite.
McKinsey also suggest that 156 GW of AI-related capacity will be needed by decade’s end, i.e.,125 incremental gigawatts between now and 2030. Yet these figures rest on assumptions. If enterprises fail to extract meaningful value from AI, demand could fall short. On the other hand, breakthrough applications could push requirements even higher. Efficiency gains are likely to be offset by increased experimentation across the broader market.
The result is a three-scenario planning nightmare. In an accelerated-growth world, capital needs could hit $7.9 trillion. In a constrained scenario, the bill drops to $3.7 trillion. None of these outcomes is inherently good or bad; each simply represents a different risk. Overinvest and you have assets that you won’t need. Underinvest and you fall behind. The only certainty is that standing still is not an option.
Meanwhile, IDC estimates that by 2027, the world’s largest enterprises will face up to a 30 percent rise in underestimated AI infrastructure costs. Traditional IT budgeting, built for ERP systems and warehouse management, collapses under AI’s economics. IDC says static budgets and quarterly forecasts cannot keep pace with workloads that self-scale overnight. Even as AI drives operational efficiency, its own operating costs can become one of the biggest drags on IT budgets.
In this environment, the GPU shortage becomes a serious problem, even for the biggest tech giant. OpenAI CEO Sam Altman told a Senate hearing that the deficit is so acute that “the less people use our products, the better.” Microsoft lists the GPU supply chain as a formal risk factor. AWS acknowledges that demand has outstripped supply.
For most CIOs, cloud providers currently provide a buffer. You do not buy chips directly from Nvidia; you procure compute through Dell, HP, or hyperscaler partnerships. That insulation, however, is thinning. Manufacturing capacity, dominated by TSMC, Samsung, and GlobalFoundries, remains the bottleneck.
Planning is further complicated by the efficiency paradox. Breakthroughs like DeepSeek’s V3 model, which reportedly slashed training costs by 18x and inference costs by 36x compared to predecessors, do not simply lower the bill. Instead, they often induce greater market-wide experimentation and demand for more complex applications, keeping pressure on capacity high. Efficiency gains are thus not a simple escape valve; they are a dynamic variable in the planning equation.
Smart CIOs are therefore initiating capacity forecast discussions with cloud providers now although forecasting is a difficult exercise. The key is to scenario-plan across multiple demand curves, secure reserved capacity where possible, and architect for flexibility. The question is not whether to use GPUs, but where and how. The emerging consensus favors hybrid architectures that treat compute location as a strategic variable.
On-premises infrastructure still makes sense when ultra-low latency, strict data sovereignty, or complete hardware control is mandatory. A manufacturing plant running predictive maintenance may deploy inference at the edge while using GPU-as-a-Service (GPUaaS) for periodic model retraining. A government agency processing sensitive legislative data may require region-bound, compliant infrastructure with rigorous audit controls.
For everything else, GPUaaS offers compelling advantages. It converts CapEx to OpEx, aligns cost with actual usage, and eliminates forklift upgrades. You can use thousands of GPUs for a two-week training sprint, then scale to zero. Managed stacks with pre-configured machine learning (ML) frameworks slash operational overhead. The practical architecture for most organizations is hybrid: Run predictable, latency-sensitive inference on-premises or at the edge; burst and train on GPUaaS. This model balances control with elasticity, cost with speed.
FinOps emerged a decade ago to tame cloud sprawl and AI has demanded a second evolution. The volatility of AI workloads requires continuous financial oversight that extends beyond cloud into every layer of the AI stack.
The new FinOps must incorporate AI-specific line items like model versioning costs, vector database overhead, prompt engineering experimentation, and the cascading storage expenses of ever-growing datasets. Teams must track GPU-hours per project, monitor inference token costs, and attribute spending to business outcomes. Without this alignment, AI stops being an innovation catalyst and becomes a financial liability.
Faced with these pressures, chief information officers (CIOs) must decide how to secure compute. The choice is not binary but a spectrum of three strategies:
- Buy (on-premises) when you have stable, predictable workloads with low latency requirements, possess the operational expertise to manage hardware, and can justify the capital outlay. This path retains full control but sacrifices flexibility and speed.
- Build (self-managed cloud) when you require custom infrastructure, specialized security postures, or want to avoid vendor lock-in. This approach demands deep platform engineering talent and a tolerance for operational complexity.
- Broker (GPU-as-a-Service) when speed-to-market matters, workloads are bursty, and you lack capital for upfront investment. Brokerage is the default for most enterprises because it converts fixed costs to variable, provides instant access to the latest hardware, and offloads operational burden.
For the majority, the answer is a blended strategy: Broker the bulk of capacity, build where differentiation demands it, and buy only when economics and control align irrevocably.
The GPU shortage will not resolve imminently. Nvidia expects improving availability, but manufacturing capacity remains constrained. Hyperscalers are hedging with proprietary chips, AWS Trainium and Inferentia, Google TPU upgrades, Microsoft’s AI processors, but these are long-term plays.
Meanwhile, enterprises are getting smarter about consumption. Smaller language models, such as Microsoft’s Phi suite or BloombergGPT, reduce GPU hunger. Many applications need only prompt engineering or fine-tuning rather than full model training. Inference can often run on high-powered CPUs. As CIOs align solutions to problem sets, GPU dependency will rationalize.
Yet the fundamental challenge remains: AI is a beast that must be fed and feeding it requires a new playbook. Tomorrow’s board presentation is not about whether you can provision GPUs. It is about how you will govern AI’s appetite while delivering revenue, cutting costs, and sustaining the planet. The CIOs who thrive will be those who treat compute not as a utility, but as a portfolio of strategic choices. Feed the beast, but feed it wisely.