Work

A Deployment-Aware Framework for Carbon- and Water- Efficient LLM Serving

Year: 2025

Type: article

Abstract: Inference now dominates the lifecycle footprint of large language models, yet published estimates often use inconsistent boundaries and optimize carbon while ignoring water. We present a provider-agnostic framework that unifies scope-transparent measurement with time-resolved, SLO-aware orchestration and jointly optimizes carbon and consumptive water. Measurement reports daily medians at a comprehensive serving boundary that includes accelerators, host CPU/DRAM, provisioned idle, and PUE uplift, and provides accelerator-only whiskers for reconciliation. Optimization uses a mixed-integer linear program solved over five-minute windows; it selects region, batch size, and phase-aware hardware for prefill and decode while enforcing p95 TTFT and TPOT as well as capacity constraints. Applied to four representative models, a single SLO-aware policy reduces comprehensive-boundary medians by 57 to 59 percent for energy, 59 to 60 percent for water, and 78 to 80 percent for location-based CO2, with SLOs met in every window. For a day with 500 million queries on GPT-4o, totals fall from 0.344 to 0.145 GWh, 1.196 to 0.490 ML, and 121 to 25 t CO2 (location-based). The framework offers a deployable template for carbon- and water-aware LLM serving with auditable and scope-transparent reporting.

Source: Sustainability

DOI: 10.3390/su172310473

Authors: Julian Hoxha, Marsela Thanasi-Boçe, Tarek Khalifa

Institution: American University of the Middle East

Language: English

Topic: Green IT and Sustainability

Subfield: Electrical and Electronic Engineering

Field: Engineering

Domain: Physical Sciences

SDGs: Climate action, Clean water and sanitation

Open Access status: gold

Estimated APC paid: $2,896