AI Investment Costs vs Productivity Growth — Why the Gap Keeps Widening

The AI Investment Explosion Nobody Predicted

The artificial intelligence industry is experiencing an investment surge that has caught even its most ardent proponents off guard. According to a landmark Bruegel Working Paper by Bertin Martens, the tension between exponentially growing AI investment costs and stubbornly slow productivity growth represents one of the defining economic puzzles of our era. While companies race to pour hundreds of billions into frontier AI models, the measurable economic returns remain frustratingly modest.

This gap matters enormously. If AI investment costs continue on their current exponential trajectory while productivity gains remain linear at best, the global economy faces a reckoning. The paper argues that without roughly 3% annual productivity growth across advanced economies — double the current rate — the present investment trajectory becomes mathematically unsustainable within a decade. For business leaders, policymakers, and investors, understanding this tension is not merely academic; it is essential for strategic planning in an era of unprecedented technological spending.

The stakes extend well beyond Silicon Valley. As enterprise AI adoption accelerates globally, every major economy must grapple with how much to invest, what returns to expect, and when the bill comes due. Bruegel’s analysis provides the most rigorous framework yet for answering these questions.

Understanding the AI Scaling Law and Its Cost Implications

At the heart of the AI cost explosion lies what researchers call the “scaling law,” first formally described by Kaplan et al. in 2020. This empirical finding demonstrates that improving the performance of large language models requires simultaneously increasing three inputs: model parameters, training data volume, and computational power. The relationship exhibits what economists call constant returns to scale — doubling output requires doubling all inputs, with no efficiency shortcuts available.

The implications are staggering. Computing requirements for frontier AI models have increased by eight orders of magnitude between 2016 and 2023, expanding from fewer than 10,000 petaflops for the original transformer architecture to over 100 billion petaflops for models like GPT-4 and Gemini Ultra. Each generation of frontier models demands roughly an order of magnitude more compute than its predecessor, creating a relentless upward pressure on costs that no amount of hardware optimization has been able to offset.

While algorithmic efficiency gains have been substantial — reducing required computations by approximately 500% per year between 2012 and 2021 — and the cost per computation continues to halve every 2.1 to 2.5 years, these savings are overwhelmed by the sheer increase in computation demanded. As Martens puts it, the “quantity effect dominates the price effect,” meaning that the AI industry is running faster and faster just to keep costs from becoming even more extreme than they already are.

This dynamic creates what economists call a “Red Queen” situation: massive efficiency improvements are continuously absorbed by even more massive increases in computational demand, leaving net costs rising exponentially despite significant technological progress on the efficiency front.

From $1,000 to $200 Million — The Training Cost Trajectory

The raw numbers tell a remarkable story. Training the first transformer model in 2017 cost approximately $1,000. By 2023, Google’s Gemini Ultra training cost had reached $120 million. By late 2024, top-ranking generative AI models were costing around $200 million to train. The growth rate, documented by Cottier et al. (2024), works out to roughly 240% per year — a factor of 2.4 to 2.6 times annually from 2016 through 2023.

Extrapolating this trajectory forward produces numbers that challenge comprehension. A single frontier model could cost $60 billion to train by 2030 and approximately $6 trillion by 2035 — the latter figure approaching half of the entire European Union’s GDP. While such extrapolations assume the current scaling law remains unchanged, even modest deviations from the trend leave costs at levels that would strain the resources of even the largest technology companies.

The cost breakdown reveals important structural features. Staff costs often represent the largest single component, reflecting the scarcity of elite AI researchers. AI accelerator chips — dominated by Nvidia’s near-monopoly position — constitute the second-largest expense. Other server components, cluster interconnect costs, and energy round out the bill, with energy surprisingly accounting for only 4-6% of total training costs despite the enormous electricity consumption involved.

This cost structure has significant implications for competition and geographic distribution. The concentration of AI talent in a handful of companies and countries creates bottlenecks that market forces alone may not resolve. When the key input is not electricity or hardware but human expertise, the competitive dynamics look very different from traditional capital-intensive industries.

AI Infrastructure Costs Beyond Model Training

Training costs, dramatic as they are, represent only a fraction of the total AI investment burden. Cottier et al. (2024) estimate that computing infrastructure costs run at approximately 10 times the model training cost. This means that a $200 million training run requires roughly $2 billion in supporting infrastructure — data centers, networking equipment, cooling systems, and redundant power supplies.

By late 2023, GPT-4’s total infrastructure costs had reached an estimated $800 million. Hardware depreciation adds another layer of cost pressure, with AI accelerator hardware losing value at approximately 140% per year — meaning full depreciation in just 8.5 months. This extraordinarily rapid obsolescence cycle forces continuous capital replacement, further inflating the total cost of maintaining frontier AI capabilities.

When these infrastructure costs are projected forward and multiplied across the five to six hyperscale companies competing at the frontier, the aggregate numbers become macroeconomically significant. Total AI infrastructure investment could reach $500 billion by 2030 and move into the trillions by the mid-2030s. At $2.5 trillion in combined annual AI investment by 2030, this spending would represent roughly 7% of world GDP — a proportion historically associated with entire industrial sectors, not a single technology.

The geographic distribution of this infrastructure also raises strategic questions. With digital transformation strategies increasingly dependent on access to frontier AI, regions that cannot host significant computing infrastructure may find themselves structurally disadvantaged in the emerging AI economy.

Measuring AI Productivity Gains — The Evidence So Far

Against this backdrop of surging costs, the productivity evidence presents a sobering counterpoint. The range of estimates is remarkably wide, reflecting both the genuine difficulty of measuring AI’s economic impact and the different methodological approaches employed by researchers.

At the conservative end, MIT economist Daron Acemoglu estimates that AI will deliver productivity increases of no more than 0.5% over the coming decade. His analysis, grounded in a careful task-by-task assessment of which jobs AI can realistically augment or replace, suggests that the transformative potential of current AI systems is substantially more limited than popular narratives suggest.

More optimistic assessments come from Goldman Sachs, where analysts Nathan et al. (2024) project roughly 10% productivity gains — twenty times Acemoglu’s estimate. The difference largely reflects assumptions about the pace and breadth of AI adoption across the economy, with Goldman Sachs assuming faster diffusion into more sectors than Acemoglu considers realistic.

Micro-level studies provide more concrete but narrow evidence. Brynjolfsson et al. (2023) documented a 10% productivity increase for call-centre operators using generative AI tools, with the largest gains accruing to less experienced workers. Noy and Zhang (2023) found that ChatGPT raised writing task productivity by 0.8 standard deviations and quality by 0.4 standard deviations. These are meaningful gains, but they apply to specific tasks rather than entire economic sectors.

Perhaps most tellingly, the average professional subscription to generative AI tools costs less than €60 per month per employee — a trivial amount compared to the hundreds of billions being invested in the underlying infrastructure. This disconnect between the modest end-user price and the enormous infrastructure cost highlights the business model challenge facing AI companies: how to recover trillion-dollar investments through subscription fees measured in tens of euros.

The Productivity J-Curve and Why Returns Are Delayed

Defenders of AI investment often invoke the “productivity J-curve,” a concept developed by Brynjolfsson et al. (2020), to explain the gap between investment and measurable returns. The theory holds that transformative technologies require substantial complementary investments in organizational restructuring, worker retraining, and process redesign before their productivity benefits become visible in economic statistics.

Historical precedent supports this pattern. Electricity took decades to transform manufacturing productivity after its initial adoption, as factories needed to be completely redesigned around electric motors rather than the centralized steam engines they replaced. Similarly, the internet’s productivity impact was modest for years before the reorganization of business processes around digital workflows produced dramatic gains in the early 2000s.

Applied to AI, the J-curve argument suggests that we are currently in the trough — the period of heavy investment with limited visible returns — and that the upswing will come as organizations learn to restructure their operations around AI capabilities. Proponents argue that judging AI’s economic impact today is like judging the automobile’s impact in 1910, before the road network, suburbs, and logistics chains it would eventually enable had been built.

However, this argument faces a critical timing problem. The scaling law driving AI costs operates on an annual doubling cycle, while the productivity J-curve operates on a multi-decade timeline. If costs continue to grow at 240% per year while productivity benefits take 10-20 years to materialize fully, the financial pressure on AI-investing firms could become unsustainable long before the economic payoff arrives. The J-curve may be real, but the question is whether the AI industry can survive in the trough long enough to reach the upswing.

The Break-Even Math — Can the Economy Absorb AI Costs?

Martens’s paper provides a revealing back-of-the-envelope calculation that frames the challenge in concrete terms. If digital firms — representing roughly 10% of GDP — invest $1 trillion in AI, and this investment triggers 3% productivity growth across the remaining 90% of the economy, the resulting GDP increase would be 2.7%. This would roughly justify the investment, assuming all productivity gains flow back to the investing firms.

The 3% target is ambitious. Current productivity growth in advanced economies hovers around 1-1.5% annually. Achieving 3% would require a doubling that has not been seen since the post-World War II boom or the brief productivity surge of the late 1990s. While not impossible, it would represent a historically exceptional performance sustained over multiple years.

Moreover, the calculation assumes a favorable distribution of productivity gains. If only 50% of the productivity increase accrues to digital firms (with the rest flowing to consumers and non-digital businesses), the required GDP to break even roughly doubles to $72 trillion. In this scenario, AI investment only makes economic sense if the global economy is substantially larger than IMF projections suggest — or if AI firms can capture an unusually large share of the value they create.

The macroeconomic arithmetic becomes even more challenging when multiple firms are competing at the frontier. World GDP is forecast to reach $139 trillion by 2029 (IMF), with approximately 27% ($37 trillion) allocated to investment. If five firms each maintaining frontier AI capabilities require $500 billion each in annual AI investment by 2030, the total of $2.5 trillion would absorb nearly 7% of world GDP — a staggering proportion for a single technology category.

Non-Economic Drivers Pushing AI Investment Beyond Rationality

Understanding the tension between AI costs and productivity requires looking beyond pure economics. Martens identifies three non-economic factors that may sustain AI investment even when the financial returns do not justify it.

The first is what he calls the “altruism factor” — the genuine belief among many AI researchers and executives that artificial intelligence can solve pressing global challenges in healthcare, climate change, and scientific discovery. This motivation drives investment that is not primarily about financial returns but about the potential for transformative social impact, creating a willingness to accept lower or negative financial returns in pursuit of broader goals.

The second factor is the “chicken game” — a game theory dynamic where major technology companies feel compelled to continue investing in AI regardless of near-term returns because the cost of being left behind is perceived as catastrophic. In this dynamic, stopping AI investment is not a viable option even if the current economics are unfavorable, because falling behind competitors could lead to existential business risks. This creates a collective action problem where rational individual behavior (continuing to invest) produces collectively suboptimal outcomes (massive overinvestment).

The third and arguably most powerful driver is the “China factor” — the geopolitical competition between the United States and China for AI supremacy. This competition has military and national security dimensions that transcend economic calculus entirely. When governments view AI leadership as a matter of national security, the willingness to invest extends far beyond what commercial returns would justify, effectively creating an open-ended commitment to frontier AI development regardless of costs.

Three Scenarios for AI’s Future — Winter, Singularity, or Middle Path

Bruegel’s analysis concludes by mapping three possible trajectories for the AI investment-productivity dynamic, each with profoundly different implications for the global economy.

The pessimistic scenario envisions a new AI winter, but one driven by economics rather than technology. In this scenario, productivity gains fail to materialize at sufficient scale, investor patience runs out, and AI investment contracts sharply. Unlike previous AI winters triggered by technical dead ends, this one would occur despite functioning technology simply because the returns cannot justify the costs. The economic disruption could be substantial, given the enormous infrastructure already built and the workforce reoriented toward AI-related roles.

The optimistic scenario involves an economic singularity where AI achieves the capacity to automate the process of automation itself. Drawing on Nordhaus’s framework, if AI can direct its own research and development, the productivity growth rate could accelerate exponentially, potentially matching or exceeding the cost growth rate. This scenario, while theoretically possible, requires AI capabilities — particularly in scientific reasoning and autonomous research — that remain well beyond current systems, even as advances in agentic AI systems continue to push boundaries.

The intermediate scenario, which Martens considers most likely, involves technological changes that bend the scaling law toward increasing returns. New hardware architectures, more efficient algorithms, and novel model designs less dependent on human-generated training data could shift the cost curve, reducing the exponential pressure while still delivering meaningful capability improvements. This path avoids both the catastrophe of an AI winter and the implausibility of an immediate singularity, but it requires specific technological breakthroughs that are hoped for but not guaranteed.

The paper also raises the intriguing possibility that AI may evolve beyond current generative AI architectures entirely. As frontier models approach the limits of available human-generated text data — what researchers call the “data wall” — new approaches using synthetic data, adversarial training configurations, or entirely novel knowledge generation methods may emerge. These could fundamentally alter the cost-benefit equation in ways that current scaling law projections cannot capture.

Strategic Implications for European AI Policy and Investment

For European policymakers and business leaders, Bruegel’s analysis raises particularly urgent questions. The EU faces a structural cost disadvantage in AI development, with electricity prices several times higher than those in the United States. While energy represents only 4-6% of direct training costs, its impact on infrastructure operations and data center economics is more significant, potentially widening the competitive gap over time.

The question of strategic dependency looms large. If frontier AI development concentrates in a handful of US and Chinese hyperscale firms, European businesses and governments could find themselves dependent on foreign technology for critical economic and security functions. This dependency creates both economic vulnerability and geopolitical risk, suggesting that European AI strategy must balance cost efficiency with strategic autonomy.

Market concentration in the AI supply chain compounds these concerns. Nvidia’s near-monopoly position in AI accelerator chips and the dominance of a half-dozen cloud providers create supply bottlenecks and pricing power that particularly disadvantage smaller players and latecomers. European initiatives to develop sovereign computing capacity and alternative chip architectures may be economically justified even if they cannot match US hyperscale efficiency, simply as insurance against supply disruption.

Regulatory choices also interact with the cost-productivity dynamic. Stringent application of copyright law and data protection regulations may further reduce the volume of training data available in European jurisdictions, increasing costs and potentially limiting the quality of models trained primarily on European-available data. Finding the right balance between data protection obligations and AI innovation support remains one of the most consequential policy challenges the EU faces.

Business model innovation may matter as much as technical innovation. The enormous fixed costs of frontier models can only be amortized across very large user markets, requiring ecosystems of derived models, application stores, and compressed models optimized for specific use cases. European firms that cannot compete on raw model scale may find competitive advantage in specialized applications, domain-specific fine-tuning, and the development of efficient “last-mile” AI solutions that deliver value without frontier-scale infrastructure.