Best of LinkedIn: Sustainability & Green ICT CW 16/ 17

00:00:00: This episode is provided by Thomas Allgaier and Frennis, based on the most relevant link in post about sustainability in green ICT in CW-XVI and XVII.

00:00:10: Frennes supports ICT enterprises in the form of delivering precise ICT market and pricing intelligence that analyst subscriptions and existing databases cannot provide.

00:00:19: you can find more info in the description.

00:00:21: Welcome everyone!

00:00:22: i'm really looking forward to getting into details today because honestly The industry's going through a massive reality check right now.

00:00:30: Yeah, welcome to today's deep dive.

00:00:32: Today we are exploring the sustainability and green ICT trends seen across LinkedIn specifically looking at calendar week sixteen and seventeen And you can really feel this shift in the discourse.

00:00:42: Absolutely We're clearly past that era of you know vague climate ambitions and glossy corporate brochures.

00:00:49: Right those days are over.

00:00:50: The professionals driving this space now they are focused entirely on hard measurable execution.

00:00:55: It's really about the actual physics behind the software to, well engineering.

00:01:01: Exactly.

00:01:02: and let's just jump straight into the deepest end of the pool because you cannot talk about IT architecture today without confronting.

00:01:12: Yeah, the AI footprint is just THE biggest topic right now.

00:01:15: Everyone knows that compute demands are scaling but...the mechanics of where those emissions actually happening?

00:01:20: That's what's surprising people!

00:01:21: Right

00:01:22: I was looking at Ben Hillier recent analysis of LLM life cycles and it completely flipped conventional wisdom on its head.

00:01:30: He pointed out over ninety percent.

00:01:32: a large language models lifetime emissions come from inference Not the training phase

00:01:37: which is wild because when you read the media coverage The focus is almost exclusively on those massive multi-months training runs, right?

00:01:44: With tens of thousands of GPU's exactly.

00:01:47: but from an engineering standpoint.

00:01:49: The training run Is really just the factory floor Inference as the vehicle actually out on the road.

00:01:54: I mean think about chat GPT handling two point five billion daily queries.

00:01:58: Yeah, you're constantly spinning up silicon to process that data.

00:02:02: Every single token generated requires physical processors to flip states drawing current from the grid generating heat.

00:02:08: It's

00:02:09: like we focus so much on the energy to build a car but We just completely ignore the gas it burns driving around all day long.

00:02:15: That is a perfect analogy, and the sheer volume of that daily fuel consumption is staggering.

00:02:21: Boris Gomez-Echikov actually shared some calculations recently on this.

00:02:24: Oh

00:02:25: reacting to the NVIDIA CEO right?

00:02:26: Yeah

00:02:27: exactly!

00:02:28: The CEO was pushing for developers to use up to two hundred fifty thousand dollars annually in AI coding tools.

00:02:35: so Boris crunched those numbers And he estimated that level of compute translates somewhere between seven to eleven tons of CO² equivalent per individual engineer.

00:02:44: Wait

00:02:45: per engineer annually.

00:02:46: Annually, yes that is insane.

00:02:48: I mean the average global human carbon footprint across all activities housing transportation food it's roughly six point six tons a year

00:02:56: right?

00:02:57: so maximizing your AI usage at the workstation could raise an individual's footprint by two points seven times the global average.

00:03:04: you're basically adding the emissions of three extra human beings just to write code faster.

00:03:10: wow and The architecture itself actively works against efficiency doesn't?

00:03:14: yeah?

00:03:15: Aditya Manglik pointed out the compounding energy drain of the context window.

00:03:18: Ah,

00:03:20: yes!

00:03:21: The prompt history Yeah?

00:03:22: When you're twenty prompts deep into a conversation with an LLM... ...the system isn't just evaluating your newest sentence.

00:03:29: It is physically reprocessing entire history in chat to maintain context.

00:03:34: So the computational load just gets heavier and heavier with every back-and-forth.

00:03:38: The energy use compounds invisibly!

00:03:40: Exactly,

00:03:41: which kind of makes you wonder are we just doomed to burn the planet for smarter chatbots or other actual solutions out there?

00:03:47: Well...there ARE solutions but it requires a philosophical shift.

00:03:50: You see this in arguments from professionals like Syed Quasar Ahmed and Kanakalata Narayanan.

00:03:56: They're actively pushing back against the instant to throw an LLM at every single problem.

00:04:00: Right because sometimes you don't need AI

00:04:02: Exactly.

00:04:03: Kana Kalata makes the case that most efficient AI system is often one that recognizes when it shouldn't be using AI at all.

00:04:11: Using a sledgehammer for thumbtack Precisely,

00:04:13: but you do need heavy iron.

00:04:15: The hardware hosting those workloads actually adapts.

00:04:18: Robert Keith noted GreenTT recently launched Google's Gemma four thirty-one B model on hundred percent renewable energy.

00:04:26: Oh wow!

00:04:27: And what was impact of this?

00:04:29: It resulted in a forty percent lower emission rate compared to typical hyperscalar deployments.

00:04:34: That's

00:04:34: a massive drop!

00:04:35: Ruchali Goud also highlighted Google's latest Ironwood TPUs, right?

00:04:39: Yeah They deliver a three point.

00:04:40: seven X carbon efficiency improvement.

00:04:43: The silicon itself is becoming much more purposeful for these tensor operations

00:04:46: But the reality is whether you're using highly optimized TPU or standard GPU it all lives on cloud.

00:04:53: Which brings us to our second big theme how organizations are actually managing and measuring these cloud emissions.

00:04:59: Right,

00:04:59: Cloud Transparency & GreenOps!

00:05:01: And there is a genuinely counter-intuitive insight here from Frank Clausen.

00:05:04: Oh

00:05:05: about where to start?

00:05:06: Yeah He argues that if you want to start with sustainable IT the worst place to start Is with Carbon Reporting

00:05:12: which sounds completely backwards when first hear it.

00:05:15: It does because carbon reporting is what regulators want but CO-II data validates you.

00:05:21: It doesn't motivate engineering teams.

00:05:23: Frank says, You have to start with fine ops and observability Data.

00:05:27: money is the trigger.

00:05:29: I love that.

00:05:29: save money save the planet angle.

00:05:31: Yeah if you expose exactly where budget is bleeding out from over provisioned virtual machines Engineers will shut them down immediately.

00:05:39: Exactly Matthew Francois shared a brilliant use case about this involving the Antarctica platform.

00:05:45: A customer uploaded their AWS bill, and in a single session they found two million dollars in actionable savings.

00:05:51: Two million?

00:05:52: Yeah!

00:05:52: And because the physics of cloud computing are linked to financials cutting that waste inherently drops your carbon emissions.

00:05:58: The physical processor drops its power state...the cooling fans spin down.

00:06:03: Right, Gerald Lynn Tepwar reinforced this exact dynamic.

00:06:06: He pointed out that sustainable cloud architecture built for efficiency is inherently greener faster cheaper.

00:06:12: Waste Is Just Waste.

00:06:14: But historically, the bottleneck has been getting that transparency from the hyperscalers.

00:06:19: The tools just weren't there...

00:06:21: but they are catching up now right?

00:06:22: I saw Various Alice and Alexis Bateman discussing the new AWS Sustainability Console.

00:06:28: It finally offers deeper API access.

00:06:30: Yeah, Kevin Leslie highlighted that GreenPitzy now specifically covers AI in LLM metrics for CO-II water and electricity which is huge because standard web traffic calculators just cannot model AI compute burns.

00:06:44: Wait!

00:06:44: I want to pull on a thread.

00:06:45: you mentioned water Because talking about cloud makes it sound invisible But the cloud is comprised of massive resource-hungry physical buildings.

00:06:53: Data centers, The Physical Realities Of The Infrastructure.

00:06:56: Exactly!

00:06:57: Let's pivot to water consumption Because cooling these massive GPU clusters Is a huge bottleneck.

00:07:02: Sean Moran shared A fascinating post about Australia.

00:07:05: Oh...a desalination project.

00:07:07: Yes

00:07:07: They are using seawater desalitation To cool data centers Which is brilliant because it doesn't stress the local freshwater supplies In a really dry region.

00:07:16: That's a great approach, and you see massive corporate commitments happening too.

00:07:21: Faris Ahmad highlighted Oracle's project Jupiter which uses a closed loop non-evaporative cooling system... So

00:07:27: it just recirculates the same fluid continuously?

00:07:30: Exactly

00:07:30: instead of evaporating millions of gallons into the atmosphere.

00:07:33: And Ruben Ruzariazou noted that Violia & AWS have goal to cut their data center water footprints by seventy five percent.

00:07:42: By twenty thirty.

00:07:43: Okay

00:07:44: but I had point out the elephant in the room regarding energy here.

00:07:48: And we're just impartially reporting the findings, but Hannah Smith and Caitrin Frisch released their state-of-the-fossil free internet report.

00:07:55: Yeah it reveals that big tech is burning massive amounts of fossil fuels while kind And Pascal Jolie warned that new AI data centers are actually bypassing the grid entirely.

00:08:07: Right, to avoid multi-year wait times for grid upgrades.

00:08:11: Exactly!

00:08:11: They're using behind the meter gas turbines to generate their own electricity on site.

00:08:16: I mean it's like bragging about buying a state of the art electric vehicle but then charging with a dirty diesel generator in your backyard.

00:08:23: It completely breaks the chain of accountability for scope.

00:08:25: three emissions.

00:08:27: And this is exactly why out-of-the box infrastructure innovation, it's so critical right now like what Carl Ray brought up.

00:08:34: The wooden data centers.

00:08:35: Yeah!

00:08:35: The rise of mass timber datacenters as a credible low carbon alternative to steel and concrete Concrete manufacturing releases so much CO too.

00:08:45: but mass timber acts as a carbon sink.

00:08:47: It offsets the operational emissions with a negative carbon physical shell.

00:08:51: That is fascinating, yeah so we have this macro infrastructure mass timber desalination grid bypasses.

00:08:57: let's trace this issue all the way back down to the micro level.

00:09:00: The code developers right and the physical devices users hold

00:09:04: sustainable software in circular IT.

00:09:06: This is where individual decisions dictate physical waste.

00:09:09: Right, Colin Fry has shared a really frustrating real-world example of this.

00:09:13: Amazon Is ending support for older Kindles.

00:09:16: It's turning perfectly functional devices into an estimated six hundred and twenty four tons of eWaste

00:09:22: Just through software induced obsolescence.

00:09:24: The hardware is fine but the code bricks the silicon

00:09:27: Exactly.

00:09:28: But on the flip side Eric C talked about countering this waste with urban mining.

00:09:32: He visited N-twos where they are literally mining gold and rare earth metals directly out of old motherboards.

00:09:38: That is the circular economy in action, but you know we can't ignore this software itself.

00:09:43: Alayna Kholiznik made a great point.

00:09:44: that inefficient code Is A Massive Silent Emitter.

00:09:48: Oh for sure.

00:09:49: every unoptimized loop quietly forces a processor to work harder.

00:09:54: John Behan actually shared a massive win regarding This.

00:09:57: his team Simply Optimize The Database right size the architecture, improve the indexing and it cut carbon emissions by thirty three percent.

00:10:05: And it dropped costs by forty a percent.

00:10:07: that is the power of green software!

00:10:09: Honestly this no longer optional...the regulations are here.

00:10:12: Right

00:10:13: cake.

00:10:13: goldering in Namfundo and Coasey noted that CSRD making software emissions part mandatory.

00:10:18: reporting Europe now.

00:10:19: Yep

00:10:20: which means industry moving to standardized open source tools.

00:10:24: Aidan Mir-Mohamedi mentioned that the old cloud carbon footprint tool is being replaced by Karman, a much more robust measurement engine.

00:10:31: Standardization's really the only defense against greenwashing at this point?

00:10:35: Absolutely and you know looking all of us I want to leave with final thought to mull over.

00:10:40: we meticulously track financial cost every software decision today But we are rapidly approaching a reality where the carbon and water cost of single line-of code will be scrutinized just as heavily by auditors or regulators.

00:10:54: Are your IT systems prepared for

00:10:58: that level.