Best of LinkedIn: Green ICT & Sustainable AI CW 40/ 41

00:00:00: This episode is provided by Thomas Allgeier and Franis, based on the most relevant LinkedIn posts about green ICT in sustainable AI from CW-Forty and Forty One.

00:00:10: Franus supports ICT enterprises in the form of delivering precise ICT market and pricing intelligence that analyst subscriptions and existing databases cannot provide.

00:00:20: you can find more info in the description

00:00:23: Right and before we even jump into the actual agenda today.

00:00:26: We really want to set the stage for what were talking about.

00:00:28: Yeah, we're unpacking the top green ICT in sustainable AI trends that have basically been dominating The professional networks over last couple of

00:00:36: weeks exactly.

00:00:38: And to give you a mental image of the stakes here?

00:00:40: I want you to picture as standard You know five hundred milliliter plastic water bottle.

00:00:44: okay got it.

00:00:46: So the next time you sit down and have a normal back-and-forth conversation with chat GPT say like twenty to fifty queries To brainstorm or project, whatever.

00:00:54: You have essentially taken that bottle of fresh water And poured it directly onto a server rack to keep it from melting.

00:01:00: Oh

00:01:00: wow I mean That is really sobering visualization.

00:01:04: It really Is?

00:01:05: And it captures perfectly why we're doing this deep dive.

00:01:08: We have to uncover The hidden mechanisms that are driving these massive massive resource costs.

00:01:14: Yeah, and for you the digital transformation leaders.

00:01:16: The architects in the tech professionals tuning-in...the mission of your job has fundamentally shifted recently.

00:01:23: Oh absolutely

00:01:24: We're no longer just looking at a dashboard asking well how fast is this single AI model?

00:01:29: I mean we have to unpack how impact is measured governed it decided across the entire system

00:01:34: Right from the invisible code being written by autonomous agents all the way down to the physical water evaporating from your local utility grid

00:01:43: which, you know requires looking at layers of the technology stack that traditionally refuse to talk with each other.

00:01:48: Yeah they really do.

00:01:50: every single layer hides a completely different environmental cost and often solving a bottleneck in one area just well it pushes the emissions down the line somebody else.

00:01:59: so let's start right at this source of demand which is code.

00:02:02: specifically AI agents themselves

00:02:04: make sense

00:02:05: because before we even consider massive physical data centers We have look at software efficiency And Naveen Balani brought up a fascinating point recently about how the hidden cost of AI is often found in the way agents communicate with each other.

00:02:19: Yeah, The agent-to-agent boundary

00:02:21: Exactly!

00:02:22: He points out that it severely compounds AI energy costs

00:02:26: and the mechanics Of that compounding cost are really important to understand because when you input A single request into an egentic workflow.

00:02:33: It isn't just executing one simple operation.

00:02:36: Right.

00:02:37: The system often employs like a supervisor agent that receives the prompt, breaks it down and then escalates subtasks to specialist agents.

00:02:45: And they all start talking with each other?

00:02:47: Exactly!

00:02:48: Those specialists fire off clarification loops checking each others work for hallucinations or errors... ...and every single one of those handoffs, every internal ping requires a compute cycle.

00:02:59: So it adds up fast?

00:03:01: Incredibly fast.

00:03:02: Balani actually proposes a management ladder of trace, contract compact consolidate to turn this boundary from you know an unmeasured black hole of energy into A strictly governed cost center.

00:03:15: You don't the way you describe those clarification loops.

00:03:17: it sounds exactly like a bloated corporate bureaucracy.

00:03:20: That is very accurate analogy.

00:03:22: Actually

00:03:23: I mean these AI agents are basically having endless Energy intensive pre-meetings before they do any work.

00:03:30: we asked for

00:03:31: Yeah, and we are starting to see real world engineering solutions.

00:03:34: To cut through that bureaucracy.

00:03:36: Dennis Kraft recently shared an example from a green software development meetup in Karlsruhe That tackles this perfectly.

00:03:42: no right the ponytail skill?

00:03:44: Yes It's an open source agent scale called ponytail.

00:03:47: Okay I have to admit i'm A bit confused by The concept here because if AI is supposed to be you know infinitely smarter And faster than any human developer.

00:03:56: Why does it need a special skill?

00:03:57: just write efficient code?

00:03:59: Is the AI over-engineering things on purpose?

00:04:03: Well, it's not purposeful malice right.

00:04:05: It just a byproduct of its training.

00:04:07: I mean AI models are trained on massive repositories of human code much which relies on complex heavy frameworks.

00:04:13: Okay sure.

00:04:14: So if you ask an unchecked AI agent to build a simple date picker for our website... ...it might automatically download a heavy external library build custom wrapper component write hundreds lines new style sheets

00:04:25: Just for a simple calendar drop down

00:04:27: Exactly and it computes complex time zone logic.

00:04:30: for no reason.

00:04:32: Ponytail stops that behavior by forcing the agent through a strict seven step priority ladder.

00:04:37: Oh, interesting!

00:04:38: It essentially forces AI to ask hey can standard native HTML just accomplish this in one single line of code?

00:04:46: So the overarching philosophy is basically be lazy not careless Yes.

00:04:51: And that Lazy Philosophy's core principle of green coding.

00:04:54: The results Dennis Kraft shared are Actually, huge.

00:04:57: Implementing Ponytail resulted in fifty-four percent fewer lines of code and a twenty percent drop in overall costs.

00:05:03: Wow

00:05:04: I mean every line of code that is never written Is the line that never has to be compiled or executed by server

00:05:10: Or maintained

00:05:11: Exactly

00:05:11: exactly.

00:05:12: it is incredibly elegant And Pola Young highlighted another open source project approaching software efficiency from A slightly different angle.

00:05:21: It's called Code Weave and was built By Wilco Berger.

00:05:24: Oh yeah, CodeWeave is fascinating.

00:05:26: Right!

00:05:27: It uses AI to actually hunt for code efficiency gains and test them in real time.

00:05:32: like it looks for performance energy hotspots within massive architectures its first proposal for PyTorch cut top K latency by forty five point five percent.

00:05:42: We really should pause on that metric because cutting TopK Latency By Forty Five Percent Is a monumental shift.

00:05:48: Explain That A bit For Those Who Might Not Know Sure.

00:05:51: So In Machine Learning Top K refers to the model's process of fetching the most likely next pieces of data, like predicting the next word in a sentence.

00:05:59: Right And PyTorch is the foundational framework underpinning countless AI applications globally.

00:06:06: If you make the foundational process of fetching data forty-five percent faster The GPU finishes its workload sooner and powers down faster...

00:06:14: ...and that efficiency gain just ripples out across millions of servers worldwide.

00:06:18: Exactly

00:06:19: And developers are finally getting tools to track these gains without adding the problem too.

00:06:24: Nathan Bailey built a tool called Carbon Tracking MCP.

00:06:27: I saw that.

00:06:28: Yeah,

00:06:29: it estimates the energy and CO-to impact of AI coding sessions purely from local system logs Which is brilliant because it operates Without making external network

00:06:37: calls Right Because pinging a cloud server To ask how much carbon you're using Would itself consume server energy Exactly.

00:06:44: It just calculates The cost locally.

00:06:45: When we step back and look at all these local and agentic optimizations, Amy Leuers makes a crucial argument about the direction of industry.

00:06:53: She points out that our fundamental efficiency metrics are just broken!

00:06:56: Oh so?

00:06:57: Well right now tech world is obsessed with measuring tokens per watt basically how much text or data can be generated for given amount electricity.

00:07:04: but as she notes not all tokens provide same value.

00:07:07: That's true.

00:07:08: She argues We must transition to measuring outcomes delivered per resource

00:07:13: which makes perfect sense.

00:07:14: I mean, a million tokens generating a bloated useless custom wrapper for calendar app is massive waste of resources regardless how efficiently those specific tokens were processed by the hardware.

00:07:24: Exactly!

00:07:25: The ultimate efficiency question isn't volume output it's actual utility generated.

00:07:31: But do you want to interject a quick reality check here from Cal Ennis?

00:07:34: Because he points out that all theoretical hype surrounding green software right now AI energy savings remain largely unproven at scale.

00:07:42: Yeah, that is a fair point.

00:07:43: He knows the tangible efficiency gains we are seeing today Are mostly coming from narrow purpose-built machine learning models Not from massive generalized generative AI models dominating all headlines

00:07:55: And it's vital distinction to make.

00:07:57: Generative AI is intrinsically resource heavy Because its predicting patterns across vast generalized data sets.

00:08:05: And by the way, if you are finding these distinctions helpful and want to stay ahead of These rapid shifts in the tech landscape make sure You subscribe to catch future editions Of this deep dive absolutely because The standards for what constitutes sustainable Tech Are being rewritten on a weekly basis right now.

00:08:20: they really are.

00:08:22: And you know that software limitation directly forces A physical reality.

00:08:27: If we cannot perfectly optimize the code or rely entirely on narrow machine learning, The physical hardware simply has to absorb the blow.

00:08:35: Right And generative AI is pushing that hardware To literal boiling points which Is completely altering the architecture of massive data facilities

00:08:44: Which brings us right to the data center infrastructure layer.

00:08:47: K-Tan Deshpande shared an insight that puts the sheer scale of this heat problem into perspective.

00:08:53: The density numbers are wild, they

00:08:54: are.

00:08:55: Traditional cloud workloads ran on server racks utilizing about forty kilowatts power but AI workloads...they're pushing rack densities to one hundred kilowatt and beyond at that density.

00:09:06: traditional air cooling just hits a physical limit.

00:09:09: you literally cannot blow cold air fast enough.

00:09:13: That is simply the physics of thermodynamics, I mean air molecules are relatively spread out.

00:09:18: Air's actually a fairly good insulator not a conductor.

00:09:22: so liquid cooling is no longer an experimental luxury.

00:09:25: it is the baseline requirement now because water carries roughly thirty five hundred times more heat per unit volume than air.

00:09:32: moving thermal energy away from a GPU with liquid is fundamentally required at one-hundred kilowatts.

00:09:38: but Jonathan Medrano raised a really compelling counterpoint to this.

00:09:41: He argues that the industry's cooling design shouldn't just focus on making mechanical chillers more efficient.

00:09:47: Okay,

00:09:48: we need to step back in question if chilled water is even needed at all.

00:09:52: he advocates for a mindset shift.

00:09:53: basically move away from focusing on Chiller efficiency and start focusing entirely on heat rejection

00:10:00: which requires leveraging The natural environment rather than you know fighting it with mechanical force.

00:10:06: so wait Let me pushback per second.

00:10:09: We're talking about liquid cooling but Aren't we essentially just boiling water to generate tax?

00:10:15: I mean, aren't we trading an energy crisis for a water crisis.

00:10:19: Well yeah that is a huge concern.

00:10:21: but it's also why alternative siding models are suddenly getting so much traction To solve this exact issue.

00:10:27: Oh like what Carolyn Pistone shared

00:10:29: exactly she highlighted a fascinating deployment of this in Norway.

00:10:32: They have activated a floating data center

00:10:36: a floating data center.

00:10:37: Yeah, it is an offshore facility powered entirely by ocean waves and uses frictionless cooling by drawing in frigid seawater.

00:10:44: The term Frictionless Cooling is key there right?

00:10:46: Yes!

00:10:47: Instead of using massive energy-hungry mechanical pumps to force treated water through complex internal piping they are utilizing the natural temperature differentials and currents from the ocean pull heat away.

00:10:58: Wow

00:10:58: And best part?

00:11:00: It consumes zero fresh water.

00:11:02: It's almost surreal to think about, honestly an offshore ocean barge generating corporate chat responses just keep the servers from catching fire.

00:11:10: And Christopher Miestra shared another geographical approach with the Holyoke Massachusetts Green High Performance Computing Center.

00:11:17: it runs on carbon free power and uses free outdoor air cooling for over eight months of year Just by leaning on cold New England climate.

00:11:24: That makes a lot sense.

00:11:26: Beyond alternative geographical sightings there is engineering concept of circularity

00:11:32: finding a way to reuse the immense heat that these dense AI racks generate rather than just venting it into the atmosphere.

00:11:38: Exactly, Melissa Schaffer points out that Sweden is successfully executing this by pairing natural data center cooling with district heating.

00:11:46: they are capturing the waste-heat from cloud computing and pumping it directly in residential homes.

00:11:52: I love the idea of server farm heating neighborhood but Christopher Linder provided look at first EU energy efficiency directive report And the data is pretty bleak, honestly.

00:12:03: Yeah

00:12:03: it's not great

00:12:04: Despite pilot projects like the one in Sweden.

00:12:07: The report showed that across reporting sites a meager of one point eight percent of heat Is actually being reused

00:12:14: and the gap between Pilot promise an everyday practice just comes down to physical infrastructure constraints.

00:12:21: What do you mean?

00:12:22: Well, you cannot simply run a pipe of hot water from a data center on the outskirts of a city to a residential neighborhood twenty miles away.

00:12:30: Oh

00:12:30: right he dissipates

00:12:32: exactly.

00:12:33: building circular infrastructure requires a holistic integration of urban planning and server siding that Frankly just does not exist in most municipalities yet.

00:12:42: And that localized inability to dissipate heat brings us to the broader macro impact.

00:12:47: I mean all this power demand and liquid cooling doesn't happen at a vacuum.

00:12:51: no

00:12:51: it doesn't.

00:12:52: And the data center can't vent its heat into a district heating system.

00:12:55: It relies on immense resources that offload an invisible toll onto local communities and utility grids,

00:13:01: Right?

00:13:02: This is critical resource level of this conversation.

00:13:05: Muhammad al-Sahab shared the stark visualization you mentioned at very top of deep dive

00:13:10: The water bottle.

00:13:11: Yeah!

00:13:12: The cloud is incredibly thirsty.

00:13:14: That five hundred milliliter water bottle consumed by single A twenty to fifty question chat GPT conversation is compounding rapidly.

00:13:23: To give you a sense of corporate scale, Microsoft's overall water use rose thirty four percent in twenty-twenty two alone.

00:13:30: but Eric Soderstrom added massive hidden layer this water metric.

00:13:34: that just blew my

00:13:35: mind.

00:13:35: what was up?

00:13:36: he pointed out That in twenty three United States data center power generation used roughly twelve times more water at the power plants.

00:13:46: Then the data center is used on site.

00:13:48: Well, it's a classic shell game of resource accounting.

00:13:51: Exactly!

00:13:52: I mean... A massive tech company can build a state-of-the art facility use a closed loop liquid cooling system and issue a press release claiming they have zero onsite water consumption.

00:14:03: Right

00:14:03: But down the road The local power grid has to burn fossil fuels to boil millions gallons of municipal water To spin steam turbines just generate electricity that the datacenter demands.

00:14:16: They are quietly outsourcing their water evaporation to the local utility.

00:14:20: Yes, which brings us directly to the financial and emissions toll of that out source demand.

00:14:26: Sasha best lick broke down the staggering numbers on this.

00:14:29: US data centers are projected to need an estimated one hundred and ten billion dollars in new power plants by twenty thirty just To keep up with the demands of AI.

00:14:39: a hundred and Ten billion

00:14:40: yeah And most of that power, over thirty gigawatts will be generated by natural gas.

00:14:47: That infrastructure will emit eighty million metric tons of CO₂ annually.

00:14:51: I really struggle with the economics in this.

00:14:54: If the hundred and ten billion dollars an infrastructure costs The massive water evaporation at the power plants And the eighty-million tons of co₂ emissions are all being pushed onto local power and water grids Aren't ordinary citizens and local ratepayers essentially subsidizing big-tex AI arms race?

00:15:12: It is the defining economic friction point of the decade.

00:15:16: And Xiaolei Ren explains why fixing this misalignment is so exceptionally difficult.

00:15:20: Okay, it comes down to a vertically integrated stack where none of the layers actually share the same operational incentives.

00:15:28: How does that actually play out in real facility though?

00:15:30: Well, for computational power flexibility to actually work meaning a data center scales back its power consumption when the local grid is stressed four different layers have to agree.

00:15:40: Four layers?

00:15:41: Yeah You have The Grid Operator ,the physical Data Center owner .The company that owns the GPUs and the end user running the AI workload.

00:15:51: That sounds complicated.

00:15:52: It is say a hedge fund is running a massive training model.

00:15:56: They are paying the GPU owner for guaranteed uptime.

00:15:59: The data center operator has a strict service level agreement that penalizes them financially if they throttle power.

00:16:05: Oh, I see...

00:16:06: So even the local grid is begging to power down during heat wave?

00:16:10: The operator literally cannot unplug servers without breaching contract!

00:16:15: The burden remains entirely on the grid.

00:16:17: To be fair with tech companies though we are seeing some vendors proactively trying to mitigate this train at the local level.

00:16:22: That's

00:16:23: true.

00:16:24: I mean, the grados shared that AWS is currently returning about one point five liters of water for every leader used at its Aragon data centers by funding local agricultural irrigation projects.

00:16:34: Okay?

00:16:35: And Christopher Welles noted that Equinix ran nearly two thirds of it's sites without using any cooling water in twenty-twenty-five.

00:16:41: those

00:16:42: are positive steps definitely

00:16:43: but i have to push back here again.

00:16:45: Microsoft has funded local apprenticeships and aws his funding Irrigation which you know sounds great.

00:16:51: an annual report sure But if a utility is burning natural gas and using twelve times the water just to power the AI facility, Is an apprenticeship or localized irrigation project really inequitable trade for draining municipalities core resources?

00:17:05: And driving up their emissions.

00:17:07: Regulators are beginning to ask that exact same question.

00:17:10: The era of relying on voluntary corporate goodwill in offset projects closing?

00:17:15: Yeah.

00:17:16: With billions of dollars in civic stability at stake, strict governance is finally taking over the wild west of AI infrastructure.

00:17:23: Let's look at that policy and governance level

00:17:25: Right.

00:17:26: Nathaniel Barola highlighted a major regulatory shift.

00:17:29: The European Commission has released a data center food label

00:17:33: A Food Label?

00:17:34: yeah It grades facilities on a scale from A to G based On their power usage effectiveness or PUE And they're water usage effectiveness WUE.

00:17:44: It also legally tracks their energy sourcing and their readiness to reuse waste heat.

00:17:48: That PUE metric is critical for accountability, I mean a perfect PUE score of one point zero means every single watt of power pulled from the grid goes directly into computing data.

00:17:58: Got it?

00:17:59: A score of two point zero means the facility is so inefficient that for every watt used for computing, another full watt is wasted just running the cooling fans and chillers.

00:18:07: It's

00:18:07: terrible.

00:18:08: Yeah

00:18:09: Grading this publicly makes The Invisible Footprint highly visible to enterprise buyers And government regulators.

00:18:14: And we are seeing this governance get real teeth.

00:18:17: Matthias Hof posted about recent developments in the Scottish Parliament, they rejected an outright moratorium on data centers but The Institute of District rule that any project over fifty megawatts now requires a full environmental impact assessment before They can even get planning permission.

00:18:32: Oh wow

00:18:33: yeah.

00:18:33: as hoff brilliantly put it proof is becoming the price of admission.

00:18:37: You can no longer break ground and promise to buy green offsets later.

00:19:07: Microsoft is launching what they now call community-first AI commitments, explicitly cutting water intensity forty percent by twenty thirty.

00:19:15: And Metta recently signed a deal in Louisiana where they are paying their full cost of service for over fifty two hundred megawatts power specifically to protect local ratepayers from footing the bill for the grid upgrades.

00:19:27: it's textbook ESG action just you know wearing hard hat.

00:19:31: If we synthesize everything we've discussed today from agentic code loops down to power plant evaporation, there is one ultimate takeaway for the tech professionals planning AI rollouts.

00:19:42: And what's that?

00:19:43: It was captured in a World Economic Forum playbook discuss by Lubomila J., Dr Daniel Jeffrey Koiture and Benoit Begaux.

00:19:50: The WEF estimates cumulative AI infrastructure investment will hit a staggering seven trillion dollars by twenty thirty.

00:19:59: That number almost difficult to comprehend, seven trillion dollars.

00:20:03: It is.

00:20:04: but their most crucial finding is about the timeline of that spending.

00:20:07: carbon and sustainability outcomes are largely locked in at the siting design procurement and financing stages.

00:20:15: they are not determined during operations

00:20:18: because you can't magically move a data center next to it district heating grid once the concrete has poured precisely.

00:20:24: You cannot retrofit frictionless liquid cooling pipes into a ten-year old building designed for raised floor air conditioning without ripping the entire facility apart.

00:20:34: Decarbonization is fundamentally a design discipline.

00:20:37: you cannot engineer or offset your way out of a fundamentally unsustainable facility once the servers are powered on.

00:20:45: Okay, let's bring this all together.

00:20:48: We started this deep dive by talking about how measurement in the sustainable tech space really murky.

00:20:53: right now The traditional dashboard is broken.

00:20:58: We are getting very good at measuring top-k latency, calculating water usage effectiveness and tracking carbon on a local machine.

00:21:05: but as AI models become completely foundational to human society perhaps the ultimate metric we haven't invented yet is societal ROI.

00:21:16: A return on investment based on civic value?

00:21:18: Yes exactly!

00:21:21: tangible value of the intelligence generated against the exact community resources extracted to produce it.

00:21:29: Because right now, asking a generative AI model deeply complex query about meaning life consumes the same amount local reservoir water as asking you to lazily rewrite simple work email who ultimately gets?

00:21:43: decide which queries are actually worth.

00:21:46: That tension between technological capability and community extraction is the defining question of next decade of digital transformation.

00:21:55: If you enjoyed this episode, new episodes drop every two weeks!

00:21:58: Also check out our other editions on Cloud Insights & Sovereignty, Digital Products & Services, AI & Agenetic Systems, HealthTech, ICT & Tech Insights, DefenseTech & HealthTech.