The Overlooked Carbon Footprint of AI Inference: A Methodological Pivot

AI inference services, think API subscriptions and SaaS products with embedded AI features, are officially in the crosshairs of the Corporate Sustainability Reporting Directive (CSRD). Starting January 2024, companies must disclose these services under Scope 3 Category 1 emissions. But there's a hitch: a standardized methodology to account for them is glaringly absent.

A Framework for Accurate Estimations

Current methods are either ignoring these emissions or grossly overestimating them. Many rely on broad economic input-output (EEIO) factors inaccurately applied to the entire ICT sector, skewing AI inference emission figures by as much as 10-40 times compared to alternatives based on physical data.

Enter a new four-tier framework that aims to align estimation precision with accessible data. From direct token-based physical estimation using GPU energy benchmarks and regional grid carbon intensities, to a fallback on EEIO for services lacking usage data, this framework offers a more nuanced approach. Peer-reviewed GPU energy benchmarks and confirmed grid carbon intensities serve as foundational elements, offering a level of accuracy previously unavailable.

The Real Numbers Behind Compliance

Applying this to a typical 200-person European firm yields a total below 1 tCO2e. This shows that the real issue is methodological inefficiency, not the magnitude of emissions. The compliance challenge is more about refining measurement techniques than facing a daunting carbon footprint.

But here's a twist: there's a water-carbon trade-off that hasn't been adequately addressed. Take Sweden, for example. Its hydro-dominated grid boasts the lowest carbon intensity but comes with the highest water footprint. This raises a critical point, how should companies choose data center locations when faced with such a dichotomy?

Why This Matters

As AI-driven services become more critical to business operations, understanding their hidden environmental costs becomes equally important. If AI inference services are falling through the cracks of sustainability reporting, how can businesses make informed decisions about their ESG strategies?

The strategic bet is clearer than the street thinks. Companies have a choice: stick with outdated estimation methods or adopt this refined framework to meet compliance and make smarter infrastructure decisions. The earnings call told a different story, but a closer look at the numbers might just change the narrative.