Pat Lynch: The game has really changed from bringing the power to the site to bringing the site to the power. We were building facilities at 75 watts a square foot, and people thought we were crazy. Five or 10 years ago, you'd build them at 150 and people thought you were crazy. Now it's north of 300, and we're starting to talk about it in terms of KW per rack, three to five KW per rack, not too long ago, was the norm. We're now hearing, with AI and others, things north of 30 KW per rack. It really changes the game, first of all, on what you're building, but also how much power you need and the density that you're building to. It really is all about power when you think about data center space and what we’re building today.
Teri Viswanath: That’s Pat Lynch, the global head of CBRE’s data center division and he’s talking about how the AI surge has big tech scrambling for power supply.
I’m Teri Viswanath, the energy economist at CoBank and your co-host of Power Plays. Today we are going to talk about U.S. data center growth, financing considerations and the implications for electric distribution utilities. As always, I’m joined by my co-host, a managing director here at CoBank, Tamra Reynolds. Hello Tamra.
Tamra Reynolds: Hey Teri. This is an exciting conversation. Pat Lynch will discuss why sourcing enough power is the number one priority these days for data center operators. We’ll also hear from CoBank’s Chris Shaffner and Jackie Bove; they manage our electric distribution and digital infrastructure teams. And they’ll discuss financing considerations for data centers and the related balancing act for electric distribution cooperatives.
But let’s step back, and first understand the scope of investment driving new data center development. Jackie Bove will share her perspective and explain CoBank’s role in financing the nation’s new facilities.
Jackie Bove: Based on an estimate of approximately 4 to 6 gigawatts of new capacity to be added per year over the next two to three years, this really creates significant financing opportunities for CoBank and the farm credit system, especially if you consider 10% to 20% of this new capacity going into rural communities. While again a considerable amount of this new capacity will be developed in the U.S., there is also about $400 billion of capital globally being earmarked for data centers.
I also wanted to mention Jeff Johnston, who is our digital infrastructure analyst, has a great podcast called All Day Digital that spends a lot of time talking about what is actually driving data center development in this country, and I'm going to boil it down into two really interrelated developments. The first one is growth in spending on artificial intelligence or AI.
Bloomberg really predicts that generative AI is poised to expand its impact from less than 1% of total IT spend last year to 10% in the next decade. Jeff's most recent All Day Digital podcast talked about what the breakthrough for AI actually means. His guests talked about how the computation requirements change from people engaging with computers to the computers engaging with themselves. There's really a multiplier effect at play here.
The other point I wanted to make is that cloud and AI are both competing for data center bandwidth. One of the key drivers of cloud adoption growth has been remote work, and this segment isn't simply maturing or going away with greater AI adoption. What's more, the lines are blurring between these two segments. All told, this is leading to a marketplace imbalance for capacity. Or simply put the current and near-term supply of data center capacity simply can't keep up.
Viswanath: The real estate market for data centers is having a moment and Pat Lynch has a front row seat on this activity. Here’s what Pat has to say about the growing importance of this segment of business and its growth potential.
Lynch: I will tell you in the top 20+ years that I've been in this space, we've continued to underestimate the growth. One of the interesting things I've heard, when you think about AI, three or four years ago, 5G was a big topic of conversation.
When I think about projecting the future of data centers, a quote I heard the other day was, "Much of what we're going to use AI and 5G for has not yet been invented." It won't be on our computers or our phones, it'll be in our house or our automobiles. It'll be on our watches or wearables. That part of it, to me, is terribly exciting. But all of that is going to require a significant amount of data centers and related connectivity to make it all operational. So I think the growth remains and will be significant.
Reynolds: Pat makes a really important point. We are fundamentally unclear on just how much power will be required in the future because the most popular applications for AI have yet to be built. Let’s be clear, the most popular generative AI model — OpenAI’s ChatGPT — was just released in November 2022. So it’s in its infancy. But, we do know that AI models are typically much more energy-intensive than the data retrieval, streaming, and communications applications that drove data center growth over the past two decades.
Viswanath: That’s right. At 2.9 watts per ChatGPT request, AI queries are estimated to require 10x the electricity of a traditional Google query, which uses about 0.3 watts per hours each. And emerging, computation-intensive capabilities such as image, audio, and video generation have no precedent. Still, AI applications are estimated to use only about 10%, maybe 20% of data center electricity today, but that percentage is growing rapidly.
So, while we don’t get have a full grasp of the “use cases,” what do we know about the players? Here’s Pat again.
Lynch: I think there's a couple of big buckets. In some cases, when you think about the hyperscale end user, big cloud providers, more recently, the emerging large AI companies that are buying these facilities, and occupying them themselves. CBRE works with a lot of them, on site selection, and we will sometimes operate those facilities and do project management for them as well. The end user-based would be one bucket, Teri, to think about.
Then, the other big bucket would be the developer operator investor. When you think about publicly traded companies, like an Equinix, or Digital Realty Trust, or some of the private equity companies, Vantage, Aligned Data Centers, Prime Data Centers, they're all large players in the space. They're also out, acquiring some of these sites, building at scale, and then looking for tenants, single tenants, sometimes multi-tenant. These are multibillion-dollar projects when they're completed.
We do have an emerging group. When I started with CB, 13 years ago, our client list was predominantly banks, insurance companies, and health care. They're still critically important clients to us, but most of them have moved away from the build, own and operate to a co-location, I'm going to be a tenant in multiple sites. I think about my company, CBRE, Fortune 120, 120,000 people spread around the globe. I'll use, on any day, 10 to 15 different cloud applications. I work a third from home, a third on the road, and a third in the office, and I need to access all of those systems wherever I'm at. I think oftentimes, we take that for granted, but it really requires a diverse global infrastructure for companies like CB and others.
That's the importance of these sites and why you see large cloud companies, large tech companies, and then, also, the large investor operators as a predominant developer of these sites.
Viswanath: While the U.S. growth estimates are significant, it is even more striking to consider the geographic concentration of the industry and the local challenges this presents for distribution systems.
According to a recent report from EPRI, analyzing power consumption trends from data centers, 15 states currently account for 80% of the national data center load. In Virgina alone, data centers consume about a quarter of that state’s electricity. Will the concentration remain static, or are we going to see greater geographic dispersion? Here is what Pat had to say.
Lynch: When you think about northern Virginia, which continues — I think our report indicated 400 megawatts of new delivered power last year, new data center space, when, supposedly, the market's out of power. We are still delivering large capacities of data center space in the northern Virginia area, but not everything that's required. We've seen migrations to Columbus, New Albany, Ohio.
I talked about Richmond earlier. When the clients draw the circles around a particular area, the old thought was, it has to be within X amount. Each client was a little bit different. Out of necessity and, I'm sure, a lot of ingenuity, those circles are getting broader and broader. We think Atlanta broadly has expanded, significantly. It's picking up some of the demand from northern Virginia, Richmond, Columbus, Ohio.
If we jump to the other coast, you think about Silicon Valley, which we're hearing, from some of our clients that have an immediate need for power, that it could be 10 years out before they get the power they need. You've got to go elsewhere, and elsewhere, on the West Coast, recently has included places like Reno, Nevada, Portland or Hillsborough, Oregon. I think the geographic dynamics have shifted somewhat out of necessity, but also, technology is allowing us to diversify and broaden those circles.
You can talk about Texas as well, where everything used to center around Dallas-Fort Worth. We've seen a significant amount of activity outside of the immediate area. Now you see a lot of activity in Austin and San Antonio as well. You could talk about, similarly, in Europe and Asia, and LATAM as well, where those circles are broadening into new markets and new regions that have utility capacity in the time frames that people require them.
Reynolds: New areas of development will bring new challenges for financing these facilities. Even three years ago, investors looking to raise capital for these investments faced a particular set of challenges. Many traditional lenders were unfamiliar with the asset class. Jackie walks us through the particular credit challenges related to financing data center development.
Bove: There are really a lot of nuances to these deal structures, so I don't want to oversimplify, but for purposes of our conversation, I'll break it down into three major categories of risks that we will look at and consider. The first one would be the quality of cash flows supporting the data center. For example, we need to understand the type of anchor lease and overall creditworthiness of the tenants. I mentioned that we are experiencing a very tight market. Most of the data center transactions we see are 100% pre-lease to a hyper-scaler such as an Amazon, Google, Microsoft, Meta, et cetera.
Amazon alone has committed $150 billion in the next 15 years to data centers. This spending spree is a show of force as the company looks to maintain its grip on the cloud services market where it really holds twice the share of the No. 2 player which is Microsoft. The next main category that we will generally focus on is the procurement of power. We've been financing this sector for quite some time, and we see that these data center projects are really only getting bigger and bigger. When you layer in AI, the demand for power is exponential, so the ability for a data center developer to secure the necessary power is critical.
Lastly, we consider what the tail risk for the project will look like, and the tail risk is what is the period of time remaining really to fully repay the loan after the lease is terminated. And if the full amount of the loan can be repaid through free cash flow within the initial lease period, then the tail risk is considered low. If that's not the case, then we have to consider what the marketplace might look like at the end of the lease. Are we still experienced, for example, tight supply and demand imbalance?
Viswanath: Just how tight is the current data center market? Pat Lynch weighed in here, mentioning that approximately 83% of the massive data center capacity currently under construction in the U.S. has already been pre-leased. Why? Mainly to ensure these facilities have adequate power supply.
Reynolds: But just how much electricity do data centers need?
EPRI estimates that data centers currently use about 4% of total U.S. electricity, with that number rising to 9% by the end of this decade. In just 4 years, from 2017 to 2021, electricity used by the biggest players in this space — Meta, Amazon, Microsoft, and Google — more than doubled. And the largest hyperscale data centers can use as much energy as 80,000 U.S. households.
From a power supply perspective, can’t we just build our way out of this problem? Here’s what CoBank’s Chris Shaffner had to say.
Chris Shaffner: Data centers represent clearly a really important source of new electricity growth for the co-ops. It's also important to understand that growth is happening in a number of other fronts as well. There's a massive new investment cycle that's been unfolding and continues to unfold for U.S. heavy industry. The catalyst there has been probably no surprise. It's the hundreds of billions of dollars allocated by new legislation. That includes the Inflation Reduction Act, the Bipartisan Infrastructure Law, the CHIPS and Science Act.
But it also has to do with significant pent-up demand that exists and has existed. We can point to a couple of things there. The upstream supply frustrations that we've all encountered. We've seen in a variety of corners in the industry as well as the prior 20 years of underinvestment, frankly. It's that period, those last two decades that's coinciding with this period of flat growth in that same period of time in the electric industry. But all that is changing. While this conversation here we're going to be talking about I think revolves around data centers and you asked about data centers.
I manage CoBank's electric distribution banking segment and I know there are obviously important growth stories for every consuming segment here that we serve. Given the rapid acceleration of demand, we're going to be faced with the need to make some pretty important resource choices related specifically to power supply, that's a critical story here too.
Reynolds: The most serious challenges that Chris alludes to are going to be very localized in nature and, in part, result from the scale of the centers themselves and the mismatch in infrastructure development timing. Chris outlines the challenges ahead.
Shaffner: We're becoming more and more reliant on a grid that is aging by the day.
And so asking more of a grid that has experienced some underinvestment in the last couple of decades creates important context for what we're anticipating to see come our way. According to the U.S. Department of Energy, we've got 70% of transmission lines that are well over 25 years old. In fact, more frighteningly approaching the end of their typical 50 to 80-year life cycle. Over 90% electricity that's consumed in the U.S. passes through a large power transformer. The average age of installed large power transformers in the United States is about 40 years, which is also at the end of that expected operational life.
So you've got the intersection of demand and capacity, and stress on an aging grid infrastructure. That's the headline news here. Now, in response to the need to make those reinvestments for future generations, that's back to what I was mentioning earlier, Congress approved the Bipartisan Infrastructure Law, which made the largest federal infrastructure investment in decades.
The $30 billion is meaningful and historic, but that doesn't change the fact that it's still not going to fill that gap. I guess the last thing I would just mention, Teri, is that spending on electric distribution systems by major U.S. utilities, that's about 70% of the total U.S. load. Spending there has already risen 54%, 55% over the last two decades. We're making those investments. In all three of the major components, I would also say of the electric grid, the generation, the transmission, and the distribution all have critical investment gaps. I think you look at that $30 billion investment we've gotten from the federal government, a conservative view of that is that they're still only covering about 10% of the total bill.
We still got to figure out where the rest of that investment is going to come from — and how much of that can be carried by consumers.
Viswanath: So, it’s clear as a country that we need to re-invest in our infrastructure. And, the numbers tell us that utilities’ spending on technologies to help tackle the challenge of more complex grids has been soaring. In fact, the 50 largest investor-owned utilities in the U.S. have raised their proposed spending on grid modernization at an annual average rate of 72% since 2018. That’s according to a Wood Mackenzie.
But we know that consumers will ultimately pay that bill. The question is which consumers? And how do we ensure that these future payments are fair and equitable? I asked Chris these tough questions and here’s what he had to say.
Shaffner: I don't know the answer to that. I hope you didn't invite me here to have an answer to that one. I'll tell you how I'm thinking about it. I think part of it I should acknowledge is that there is no one-size-fits-all answer to that. I don't think there's a real solution yet, but I do believe that directionally, there must be some expectation and preparation. The electric co-ops are going to have to shoulder their fair share of it. I think we're still trying to figure out what that is and how to structure that.
I was listening to the podcast interview you did with Northern Virginia Electric's CEO in the last year or so, Dave Schleicher. He mentioned that NOVEC had begun to standardize their intake process for data centers to really start to formalize some of that work, and made sure that there were financial safeguards in place, so that, exactly what we're talking about, the rest of the membership is kept whole. I think it's a great effort and critically important as we start to design new models on how co-ops can react to the demand.
Reynolds: We had an interesting conversation earlier in the year with Ken Hester from NVIDIA.
We asked Ken how artificial intelligence might help solve the energy-crunch problem with data centers. He mentioned that leveraging best-in-class GPUs, CPUs, networking and reference architectures, the industry would drive better performance per watt. But another important solution will be cooling efficiencies, which is an area that NVIDIA is investing in.
In our conversation, Jackie mentioned that roughly 40% of the facility load is focused on cooling and according to Ken, direct liquid cooling could result in 50% or more load reduction. Still, the collective numbers for energy consumption are beginning to scare people.
Viswanath: I came across an article in a Semiconductor Engineering magazine that suggested machine learning is on track to consume all the energy currently supplied in the world — a path that is costly, inefficient, and unsustainable. But there are opportunities for data centers to help solve this problem as Jackie explains.
Bove: Some of the biggest advancements in energy efficiency have taken place in the data center infrastructure, that would be cooling. But in reality, the lion's share of electricity use occurs with the computational equipment. There's real energy savings that we could accrue here as well. There is intense scrutiny and focus on improving the compute performance to drive this datacenter efficiency. I'm really optimistic that we'll see some savings here.
Reynolds: The combination of technological advancements, strategic planning, and energy-conscious practices are contributing to the ongoing improvement in data center efficiency. But there might be a not-so-distant moment when tough decisions on resource allocation will be made.
In our podcast with NOVEC’s Dave Schleicher, he mentioned the importance of balancing member requirements. Acknowledging that NOVEC is not going to give up the obligation to serve their membership, but that co-op staff would do so in a way that safeguards the co-op’s other 180,000 members. He also saw the benefits of the data center growth on his system. He mentioned that as residential sales flatten out, data center demand probably represents the next-generation growth engine for the community.
Viswanath: Our guest Pat Lynch also talked about the need to strike the right balance within communities — here’s what he had to say.
Lynch: It's very apparent, in some of the conversations we have early stage, where markets, regions and the respective utility companies are embracing the data center user, and in some places, not embracing them. I think having alignments between your economic development, your state, and your utility company, is really important.
We're going to come out here in a month with a piece that's going to talk about the economic advantages of data centers. I think the industry — We've spent a lot of time talking about the demand on the grid, and in a lot of other things. I think it's lost, the economic impact that data centers have in a community. When you think about sales tax, personal property tax, they're significant, even in an incentive-based environment.
And you think, "Well, they don't create jobs." They create a lot of jobs on the construction site, and some very high-end jobs once they're operational. They also don't put cars on the road. They don't put people in your schools. They're really a nice economic engine with a very long-term staying power within these communities, once they're built.
Viswanath: The U.S. is home to a third of the world’s 8,000 data centers, and electric cooperatives are either serving or will soon serve these emerging members.
A collaborative approach involving stakeholders, informed decision-making, and proactive planning can help communities address concerns related to data center expansion. What’s more, there are more sharp minds at the table paying attention to this grid challenge. The U.S. Department of Energy established a task group to study growing AI resource demand and the recommendations come out this month.
Reynolds: So, this is definitely a ‘work-in-progress’ discussion for us. I do hope all of you have enjoyed this conversation and will join us again next month as we explore the changing fundamentals driving U.S. natural gas prices.
Viswanath: That’s right, Tamra. We’ll be joined by Dr. Robert Brooks and Bethel King from RBAC, a leading software and energy consulting company, as they explore how rising LNG exports, limitations on building new interstate gas pipelines in the U.S. and energy transition will impact domestic natural gas prices. I hope all of you will join us then. Bye for now.
