Best of LinkedIn: Sustainability & Green ICT CW 22/ 23

00:00:00: This episode is provided by Thomas Allgaier and Frennis, based on the most relevant LinkedIn posts about sustainability in green ICT in CW-twenty two and twenty three.

00:00:09: Frennis supports ICT enterprises in the form of delivering precise ICT market and pricing intelligence that analysts subscriptions at existing databases cannot provide.

00:00:20: you can find more info.

00:00:22: So what if I told you that using artificial intelligence to just draft a single simple article essentially pours A third of a liter of fresh drinking water straight down the drain.

00:00:32: Yeah, it's.

00:00:32: uh It's pretty shocking visual right?

00:00:34: It really is.

00:00:34: i mean we usually think of the cloud as this.

00:00:36: You know Weightless friction less magic trick your type of prompt and they answer Just sort of materializes

00:00:43: like it comes from nowhere

00:00:44: exactly yeah.

00:00:45: But the physical machinery hiding backstage Is just grinding through land power in Water to terrifying scale.

00:00:51: So today we're diving into the sustainability and green ICT trends.

00:00:54: We've been seeing across LinkedIn, were looking at the invisible bill of our digital world.

00:00:59: Yeah for this deep dive where synthesizing all these critical insights from the latest Green ICT discussions Where you know deliberately stepping past the hype Of what digital tools can do right

00:01:12: getting passed The shiny object syndrome

00:01:13: exactly We want to focus squarely on the physical toll of running them.

00:01:18: So we're talking about hardware, thermodynamic realities for cooling and radical infrastructure innovations trying to manage what's rapidly becoming a huge bottleneck.

00:01:28: Okay let's unpack this entirely because the illusion of weightless digital action is incredibly pervasive.

00:01:34: but numbers coming out in our community right now are just...

00:01:37: well

00:01:38: they're massive wake up call.

00:01:39: They

00:01:39: really are.

00:01:40: Like I mentioned that water statistic at the top, Giorgio Nattili posted a breakdown recently noting that drafting a single article using generative AI cost roughly three hundred fifty milliliters of water and seventy-five watt hours of energy.

00:01:53: That is...I mean..that's a tangible physical costs for just a few paragraphs of

00:01:57: text.

00:01:57: Right it basically can have sodas worth of water on an email draft.

00:02:01: It totally shatters concept in cloud And Navin Bellani framed this dynamic brilliantly.

00:02:06: He calls it The Invisible Bill.

00:02:08: Well i like that phrasing the invisible bill.

00:02:11: Yeah, he points out that every single prompt we execute carries five simultaneous costs.

00:02:16: so that's energy water carbon hardware and money... Five

00:02:20: different cost.

00:02:20: yeah but the

00:02:21: problem is that right now for most engineers or corporate users four of those five costs are entirely hidden like hidden behind a clean user interface.

00:02:30: So they just see the monthly subscription fee and assume the rest is magic

00:02:33: Exactly!

00:02:34: And because that physical cost is invisible we get perilous.

00:02:38: Brian Lemus shared some data illustrating how this hidden bill leads to what we could call lazy AI.

00:02:43: Lazy AI, okay break that down for me!

00:02:45: Well he calculated a single Gemini query costs about five drops of water so around point two six milliliters.

00:02:53: Okay it sounds tiny in isolation right?

00:02:54: Five drops

00:02:55: Right but multiply by billions of queries across the globe.

00:02:59: The scale is what makes those default behaviors so dangerous.

00:03:03: Yeah because when impacts are people just start throwing large language models at absolutely everything, regardless of the task's complexity.

00:03:10: Which brings up a comparison Lemus made that I really want to drill into.

00:03:14: he pointed out that checking a date format or you know seeing if an email address has an at symbol using point zero zero zero one watt hours.

00:03:28: Basically nothing basically,

00:03:30: but people are increasingly using LLMs for those exact same deterministic

00:03:35: tasks.

00:03:35: Yeah which is?

00:03:36: I mean Using a massive AI model to check an email format Is like hiring A team of PhDs To tie your shoes.

00:03:43: It is vast overkill.

00:03:44: That's a perfect analogy, let's break down the mechanics of why that actually matters.

00:03:48: so when you use a simple line-of code to check an email address A single processor performs a basic logic gate operation.

00:03:53: Right just yes or no?

00:03:55: Yeah

00:03:55: it requires almost zero power.

00:03:57: But When You Ask An LLM To Do The Same Thing billions of parameters across multiple highly specialized GPUs.

00:04:04: Oh, it's processing the request through massive neural network layers just to output that same yes or no.

00:04:11: and every extra watt hour burned.

00:04:13: spinning up those GPUs generates heat

00:04:16: And heat means cooling costs.

00:04:18: So if you're listening to this and you manage IT budgets Or like digital transformation for your company This isn't just an abstract environmental problem for you.

00:04:26: Not at all!

00:04:27: Lazy AI is a massive drain on your compute infrastructure.

00:04:31: You're burning highly expensive GPU cycles On tasks that simple, lightweight script could solve For virtually free.

00:04:38: And that exponential growth and resource demand is exactly what Dr.

00:04:42: Ritraj Patil highlighted in his recent analysis of carbon emissions.

00:04:46: There were his numbers on there.

00:04:47: So

00:04:47: if we trace the trajectory, training GPT-III back in twenty-twenty emitted an estimated five hundred eighty eight tons of CO₂ equivalent.

00:04:54: Okay Five hundred eighty-eight tons.

00:04:56: Fast forward to today and training a model like Grock IV has estimated it hit over seventy two thousand tons.

00:05:01: Wait really?

00:05:02: Seventy two thousand Yeah

00:05:03: To contextualize that, seventy-two thousand tons is the equivalent of lifetime emissions over eleven hundred average cars.

00:05:10: That's just to get a model ready for day one... Just your training!

00:05:13: Not even running it yet….

00:05:15: So if every single query and training run requires this much physical energy Where exactly is all this infrastructure going?

00:05:24: I mean, it has to live somewhere in the real world.

00:05:26: Yeah and that brings us to the core of the infrastructure bottleneck.

00:05:30: It's a huge collision between thermodynamics and geography And specifically The reality of how data centers are actually cooled.

00:05:40: This is fascinating.

00:05:41: Brandy Frost shared an analogy about this.

00:05:43: That completely changed How iView server farms.

00:05:46: Right now.

00:05:47: you know the tech industry loves To praise closed-loop liquid cooling systems.

00:05:51: Oh yeah, the PR spin on that is everywhere!

00:05:54: It really is.

00:05:54: it makes it sound like because the liquid isn't a closed loop.

00:05:56: The environmental impact has just boom solved but frost points out That a closed Loop Just moves the heat away from the silicon chips.

00:06:03: right?

00:06:03: Because the laws of thermodynamics dictate that Heat doesn't just vanish.

00:06:06: you can't delete it.

00:06:07: it has to be rejected into the surrounding environment.

00:06:11: Exactly she compared it To A car radiator.

00:06:13: so the liquid coolant Absorbs the heat From the engine and Moves Into the Radiator.

00:06:18: But if the car is parked and idling, there's no natural air flow.

00:06:21: Right So a massive fan has to kick on to pull air across the radiator to cool liquid down.

00:06:28: Data centers operate the exact same way.

00:06:30: The heat is moved to edge of building And then facility either use massive industrial fans To blast that heat into the air

00:06:36: Which requires huge amounts electricity By the way

00:06:39: And generates deafening acoustic noise Yeah Or they used water.

00:06:43: That evaporative water cooling Is where we see severe geographic disconnect.

00:06:47: To reject that heat efficiently, data centers spray water over cooling towers.

00:06:52: The water evaporates carrying the heat

00:06:54: away.".

00:06:55: Okay so it literally just goes up into the air?

00:06:57: Exactly!

00:06:58: But Praful Care and Amir Alajuan both analyzed a very troubling trend in site selection.

00:07:04: since twenty-twenty two nearly two thirds of new U.S.

00:07:07: data centers are being built in high water stress areas.

00:07:10: Wait...high

00:07:11: water stress?

00:07:12: Yeah we're talking about deserts in California Arizona and Texas.

00:07:16: I don't understand logic there.

00:07:18: Why would any company build the most water-hungry industrial infrastructure imaginable in the middle of a drought stricken desert?

00:07:24: Well, it comes down to competing priorities.

00:07:26: Tech companies need massive tracks with cheap land access to high capacity fiber optic routes and Cheap subsidized power grids

00:07:35: And desserts.

00:07:36: check all those boxes

00:07:37: they do.

00:07:37: deserts provide those three things but The environmental oversight is glaring.

00:07:42: CARE pointed out that of the eight hundred nine data centers planned across

00:07:45: U.S.,

00:07:45: over five-hundred are slated for regions already facing severe water scarcity, and PSLE shared UN data that scales this impending crisis globally.

00:07:56: Listen to this... Global Data Center Electricity & Water Use Are Projected To Double By Twenty Thirty

00:08:02: double, wow.

00:08:03: Yeah we're looking at hitting nine point three trillion liters of water consumed annually alongside electricity consumption that matches the entire country of Japan and communities are noticing they're actively fighting back against this footprint.

00:08:16: like Curry noted a proposed data center in Alabama that was recently halted by the municipality because it demanded two million gallons of water a day.

00:08:25: Two

00:08:26: million gallons?

00:08:27: A day!

00:08:28: Yeah, That single facility would have consumed enough water to sustain two-thirds of the city's entire human population.

00:08:35: So freshwater is rapidly becoming as strategic bottleneck for tech growth As the microchips themselves

00:08:42: Which introduces are really fascinating and honestly highly sensitive PR dynamic for the hyperscalers.

00:08:48: Saskia Cullenbaughn observed this playing out.

00:08:50: in Belgium, a major tech giant built a massive highly water-intensive data center in the region and almost simultaneously they announced that were funding of precision agriculture project to help local farmers reduce their water use through AI optimization.

00:09:05: Okay

00:09:05: well what's fascinating here?

00:09:07: is it really tension point?

00:09:08: for corporate social license?

00:09:09: I mean objectively investing in local waters stewardship as positive action But as Kolamban frames the community's perspective, these initiatives are increasingly viewed as tech giants using agricultural projects to offset or just mask their own massive industrial

00:09:26: extraction.

00:09:26: Like a distraction tactic?

00:09:28: Right they're essentially telling farmers to optimize their drops so that data center can consume millions of gallons.

00:09:34: Wow!

00:09:35: So if traditional land is tapped out power grids are strained And local communities are literally halting construction to protect their drinking water.

00:09:44: Where is this industry supposed expand?

00:09:46: Well, when terrestrial real estate becomes unviable the engineering has forced into extreme environments.

00:09:52: We're seeing first signals of a radical shift in where data actually housed.

00:09:57: Yeah

00:09:57: I saw this Timothy Lawn shared report that sounds like pure science fiction.

00:10:00: China recently deployed world's first operational underwater commercial data center off the coast of Shanghai.

00:10:06: Underwater on ocean floor

00:10:08: Literally.

00:10:09: They sank the server racks to the seabed, surrounded them with an offshore wind farm for power and they're using the ocean's ambient temperature for cooling.

00:10:18: It consumes zero

00:10:19: freshwater.".

00:10:20: That is wild!

00:10:21: And we are seeing similar marine infrastructure in Europe too.

00:10:24: Caroline Pistone highlighted a project in Norway where a company launched a floating data center in a cold fjord.

00:10:30: Floating In A Fjord?

00:10:32: Yeah it essentially is specialized marine barge.

00:10:35: It's powered entirely by the kinetic motion of the ocean using wave energy converters, and it is cooled silently by freezing seawater right beneath the hull.

00:10:44: Okay hold on I have to play skeptic here for a second.

00:10:46: Go for it!

00:10:46: Putting highly sensitive multi-million dollar server racks in a barge in a freezing saltwater fjord sounds like an incredibly expensive PR stunt.

00:10:56: Does the engineering math on that actually scale, or is it just a tech giant trying to look innovative for a press release?

00:11:03: It's a fair question but physics make this highly pragmatic.

00:11:07: Water is roughly twenty-four times more efficient at conducting heat away than air.

00:11:11: Oh!

00:11:12: Twenty four times okay.

00:11:13: By resting steel hull in freezing fjord The entire ocean acts as massive passive heatsink.

00:11:20: So facility requires zero energy spent running industrial chillers.

00:11:24: The cooling is completely free and continuous.

00:11:27: So when you remove the electricity cost of cooling, the operating margins improve dramatically which makes up front marine engineering costs actually worthwhile.

00:11:37: That make a lot sense.

00:11:38: And for terrestrial solutions that can't move to ocean Ahmed El-Dimardash pointed towards a pivot toward radically different power sources.

00:11:47: He discussed how geothermal energy is emerging as kind of the holy grail for data centers.

00:11:52: Oh, geothermal right?

00:11:54: Yeah because unlike wind and solar which are intermittent and require massive battery storage Geothermal provides firm always-on base load power.

00:12:03: You tap the heat to earth to generate a constant stream of clean electricity Perfectly matching the two hundred four seven demand about hyperscale facility.

00:12:10: But developing deep geothermal plants, or you know marine infrastructure requires capital that even individual tech giants struggle to justify alone.

00:12:18: Which brings us a major ecosystem shift noted by Nina Benoit and Bruce Lee Goad.

00:12:23: Oh

00:12:23: the DCII?

00:12:24: Exactly!

00:12:25: They analyze data center innovation initiatives spearheaded by elemental impact.

00:12:30: This coalition completely blew my mind.

00:12:33: You have Amazon Google Meta And Microsoft.

00:12:37: All backing is shared initiative.

00:12:39: I mean, these are companies that fight tooth and nail for every fraction of market share.

00:12:43: Usually fierce competitors yeah.

00:12:45: And they're pooling their resources to jointly fund startups creating advanced cooling tech alternative energy storage in low-carbon construction materials.

00:12:55: It represents a total paradigm shift in corporate strategy.

00:12:59: the hyper scalars have realized They cannot optimize their way out of this physical crisis internally.

00:13:04: The supply chain constraints on power and water are so severe that their entire business models are at risk.

00:13:10: They literally have to work together.

00:13:12: Yeah, they've no choice but to collaborate upstream To fund an accelerated entirely new ecosystem of infrastructure.

00:13:18: So we had the hardware in physical buildings being completely rethought.

00:13:22: But replacing concrete and chillers is really only half a battle.

00:13:27: We also fundamentally change actual code running these machines.

00:13:30: That's

00:13:31: the invisible part

00:13:32: Right.

00:13:33: So how does the industry measure and manage that invisible software bill we talked about earlier?

00:13:38: Well,

00:13:39: The first step is dispelling the myths around green software engineering.

00:13:43: We have to look past the surface level metrics

00:13:46: right like the illusion of small AI.

00:13:48: Let's talk about that.

00:13:49: Adrienne Young break down some recent research by Sasha Lucioni.

00:13:53: There is this massive push in the industry right now towards smaller specialized AI models.

00:14:00: Yeah everyone wants a leaner model

00:14:02: Exactly.

00:14:03: And the baseline assumption developers have is that because a model is smaller and runs faster, it's automatically greener and more environmentally friendly.

00:14:11: But that assumption completely ignores the manufacturing process of the model itself.

00:14:15: Young highlights hitting catch in a process called distillation.

00:14:18: Wait

00:14:18: let's pause to define for the listener What exactly is distillation?

00:14:22: Why does this supposedly small model require so much energy to build?

00:14:26: Think of distillation like teacher or student To make a highly capable small AI model, so the student developers use a massive incredibly energy-hungry AI model.

00:14:36: The teacher

00:14:37: to train it!

00:14:38: Okay I'm following...

00:14:39: The Teacher Model is forced to generate millions of pieces of synthetic training data.

00:14:44: It also performs something called logit caching where it evaluates answers and calculates probability scores for every single possible

00:14:52: word choice.

00:14:53: That sounds incredibly heavy computationally it is,

00:14:56: It saves that massive amount of logic data.

00:14:58: so the small model can learn how to quote unquote reason.

00:15:03: So The final student model might look incredibly lean and fast when you know a user runs in on their laptop but required the massive teacher model to bring a mountain of energy, to write The textbook.

00:15:16: Exactly!

00:15:17: If we aren't measuring the entire lifecycle Of this software from training To deployment We are just lying to ourselves about our efficiency

00:15:24: Precisely.

00:15:25: and that demand for life cycle transparency is driving the adoption of standardized measurement.

00:15:29: like Nolan Gaddard pointed to a vital development, the Software Water Intensity Standard or SWI.

00:15:36: The industry is finally extending the logic of carbon-aware software to water aware software creating a standardized metric to measure exactly how much water a digital product consumes during its operation.

00:15:46: That's huge step and Sarah Sue shared that software.

00:15:50: carbon intensity or SCI, is officially being integrated into OpenTelemetry's semantic conventions.

00:15:56: Which is massive for developers!

00:15:58: Yeah – For anyone listening outside the DevOps world, OpenTelemetry is the open-source standard toolkit that developers use to monitor health of their software.

00:16:06: It's the dashboard tells them if an app running slow and throwing error codes

00:16:12: Right And by embed and this leads to a rather brilliant full circle moment, honestly.

00:16:17: The very technology causing this massive resource strain AI is actually becoming the primary enabler for managing and solving the ESG data problem.

00:16:27: it's true.

00:16:28: Mark Butcher introduced a tool called Report Zero, which uses AI to take massive amounts of siloed, messy facility data from datacenters.

00:16:37: So cooling logs power draws grid mix data and synthesizes it into accurate audit ready isometrics

00:16:44: Which is normally a nightmare to calculate manuals.

00:16:46: Oh

00:16:46: absolutely!

00:16:47: And Mason Corliss discussed Terralen A platform that collapses incredibly complex ESG reporting workflows Into single AI prompt generating full verifiable audit trail

00:16:57: And Kenneth Aang shared a really practical application of this.

00:17:00: Razer, the gaming hardware company is using green software principles in AI to compress their product lifecycle reporting.

00:17:06: What used to take them six months of manual data collection and analysis has been reduced just minutes

00:17:11: Six months down to minutes.

00:17:12: Yeah When AI is applied purposefully To untangle complex datasets Rather than lazily deployed for simple text generation It dramatically accelerates our capacity to execute on sustainability.

00:17:25: We have covered incredible ground today.

00:17:27: We started with the hidden water cost of drafting a single email, unpacked the thermodynamics of why data centers are draining desert municipalities looked at server farms floating in freezing fjords and explored how developers were finally getting real-time carbon dashboards embedded into their code.

00:17:44: we've dissected the mechanics of Carbon & Water at length Today but before we wrap up I want to leave you With one final broader perspective based on a post from Tim Christofferson.

00:17:53: Okay what did he share?

00:17:54: He highlighted the birds framework and it points to a blind spot.

00:17:57: The entire tech industry is currently missing, we're obsessed with carbon in water.

00:18:01: but the next frontier of impact is AI's biodiversity footprint.

00:18:05: Biodiversity?

00:18:07: How does running Aquarium-a-server rack impact local wildlife or species loss?

00:18:12: well It comes down to the indirect impacts at the physical power grid.

00:18:16: over ninety five percent of AIs bio diversity impact Is driven by the sheer scale of electricity generation required to sustain.

00:18:23: Generating that much power, even from seemingly clean sources requires massive land use changes.

00:18:30: Mining for battery minerals and it creates localized pollution like acid rain and freshwater toxicity.

00:18:36: Oh

00:18:36: I see!

00:18:36: It's the footprint of the power itself.

00:18:38: Right And here is a complex part.

00:18:40: The location with lowest carbon footprint Is not always the location With the lowest biodiversity impact.

00:18:45: Christopherson pointed out that while a country like Norway might rank best for low-carbon emissions due to its grid, A Country Like France actually ranks better for overall biodiversity protection based on it's specific energy mix and land management.

00:18:58: Wow!

00:18:58: That is a tricky tradeoff...

00:19:00: It IS!

00:19:01: Optimizing purely for zero carbon metric can inadvertently devastate the local physical ecosystem.

00:19:07: We

00:19:13: are racing to build the infrastructure for a hundred trillion tokens per day.

00:19:17: We're figuring out the carbon, trying to solve water but what happens to physical biodiversity of land beneath and around this machinery?

00:19:25: That is an invisible bill we haven't even begun to calculate!

00:19:28: If you enjoyed today's episode new episodes drop every two weeks.

00:19:31: also check our other editions on cloud digital products services artificial intelligence ICT tech insights health tech defense tech.

00:19:40: Thanks so much for joining us on this deep dive.

00:19:41: Don't forget to subscribe.