LLM Energy Transparency with Scott Chamberlin

In this episode of Environment Variables, host Chris Adams welcomes Scott Chamberlin, co-founder of Neuralwatt and ex-Microsoft Software Engineer, to discuss energy transparency in large language models (LLMs). They explore the challenges of measuring AI emissions, the importance of data center transparency, and projects that work to enable flexible, carbon-aware use of AI. Scott shares insights into the current state of LLM energy reporting, the complexities of benchmarking across vendors, and how collaborative efforts can help create shared metrics to guide responsible AI development. Learn more about our people: Chris Adams: LinkedIn | GitHub | Website Scott Chamberlin: LinkedIn | Website

Find out more about the GSF: The Green Software Foundation Website Sign up to the Green Software Foundation Newsletter

News: Set a carbon fee in Sustainability Manager | Microsoft [26:45] Making an Impact with Microsoft's Carbon Fee | Microsoft Report [28:40] AI Training Load Fluctuations at Gigawatt-scale – Risk of Power Grid Blackout? – SemiAnalysis [49:12]

Resources: Chris’s question on LinkedIn about understanding the energy usage from personal use of Generative AI tools [01:56] Neuralwatt Demo on YouTube [02:04] Charting the path towards sustainable AI with Azure Machine Learning resource metrics | Will Alpine [24:53] NVApi - Nvidia GPU Monitoring API | smcleod.net [29:44] Azure Machine Learning monitoring data reference | Microsoft Environment Variables Episode 63 - Greening Serverless with Kate Goldenring [31:18] NVIDIA to Acquire GPU Orchestration Software Provider Run:ai [33:20] Run.AI NVIDIA Run:ai Documentation GitHub - huggingface/AIEnergyScore: AI Energy Score: Initiative to establish comparable energy efficiency ratings for AI models. [56:20] Carbon accounting in the Cloud: a methodology for allocating emissions across data center users

If you enjoyed this episode then please either: Follow, rate, and review on Apple Podcasts Follow and rate on Spotify Watch our videos on The Green Software Foundation YouTube Channel! Connect with us on Twitter, Github and LinkedIn!

TRANSCRIPT BELOW: Scott Chamberlin: Every AI factory is going to be power constrained in the future. And so what does compute look like if power is the number one limiting factor that you have to deal with? Chris Adams: Hello, and welcome to Environment Variables, brought to you by the Green Software Foundation. In each episode, we discuss the latest news and events surrounding green software. On our show, you can expect candid conversations with top experts in their field who have a passion for how to reduce the greenhouse gas emissions of software. I'm your host, Chris Adams. Hello and welcome to Environment Variables, where we bring you the latest news and updates from the world of sustainable software development. I'm your host, Chris Adams. We talk a lot about transparency on this podcast when talking about green software, because if you want to manage the environmental impact of software, it really helps if you can actually measure it. And as we've covered on this podcast before, measurement can very quickly become quite the rabbit hole to go down, particularly in new domains such as generative AI. So I'm glad to have our guest, Scott Chamberlain today here to help us navigate as we plum these depths. Why am I glad in particular? Well, in previous lives, Scott not only built the Microsoft Windows operating system power and carbon tracking tooling, getting deep into the weeds of measuring how devices consume electricity, but he was also key in helping Microsoft Azure work out their own internal carbon accounting standards. He then moved on to working at Intel to work on a few related projects, including work to expose these kinds of numbers in usable form to developers when people when making the chips that go in these servers. His new project Neuralwatt is bringing more transparency and control to AI language models. And a few weeks back when I