Electric utilities increasingly find themselves at war with their own customers. Homeowners and commercial building owners have all sorts of new choices for how to generate, store, and manage their own electricity — solar panels, home batteries, electric vehicles, smart thermostats and appliances, and more gizmos coming along every day.

The more they take advantage of these new “distributed energy resources,” they less power they buy from their utility. Turns out utilities don’t like that!

Right now, utilities don’t control any of those distributed energy resources (DERs), which tend to be located on the customer side of the electricity meter (“behind the meter”). Utilities have no visibility into DERs, they don’t know how value or make use of them, and they don’t know how to plan for them.

What’s more, the regulatory model under which utilities operate puts them at odds with DERs. (This explains the many fights playing out across the country over “net metering,” which requires utilities to compensate rooftop solar owners, i.e., their competitors.)

It’s an unfortunate situation, because DERs could be useful to utilities, providing grids not only energy but additional flexibility and stability. Utilities just have to figure out how to adapt.

That’s easier said than done, though. Electric utilities tend to be heavily regulated monopolies with guaranteed returns, operating in a market where the basic underlying technologies have, until recently, not changed substantially in decades. It’s not a model that encourages nimble innovation.

The sector is now faced with the need for rapid, responsive change, but it lacks the mindset and resources.

To help confused utilities, a team of researchers at the MIT Energy Initiative has put together a guide: The Utility of the Future, a magisterial, 360-page report that exhaustively documents the shortcomings of the current system and various tools and models available to utilities and regulators to help them cope with changes that are, by now, unstoppable.

I called Jesse Jenkins, a doctoral student working on the project at MIT, to walk me through it. We were joined in the beginning by Bob Armstrong, the director of the MIT Energy Initiative.

Just as a friendly warning: The conversation gets pretty deep in the weeds, and uses some terminology that might not be familiar to non-nerds. Along the way, I will explain key terms and concepts in {curly brackets}, to help everyone follow along. To be clear: everything inside the {curly brackets} is my language, added after the fact.

Let’s jump in. An edited transcript of our conversation follows.

David Roberts

What, in essence, is the problem with today’s electricity system?

Jesse Jenkins

Individual households and businesses are starting to face a whole new set of options for how they provide or consume electricity, everything from rooftop solar panels to electric vehicles to smart thermostats. That unprecedented degree of choice requires a new set of policies and regulations, to make sure that people are making decisions that are smart for themselves and also help lower costs or improve the performance of the power system as a whole.

Bob Armstrong

Yes, so the challenge is not so much that the industry is looking to change the system, but it’s changing under their feet. They understand that it’s changing, but are unsure how to respond to those changes. The idea in this study is to inform them how best to take advantage of these distributed energy resources, where best to put them to optimize cost and impact, and how to exploit information and communications technologies.

David Roberts

My sense is that most US utilities are fighting rearguard battles against DERs. How much of that is real hostility versus just not knowing what else to do?

Bob Armstrong

It’s uncertainty about the future, not knowing what their business models are going to look like going forward. Holding on to what they’ve got is more comfortable.

Jesse Jenkins

A variety of frictions emerge any time the fundamental set of technologies and choices in a marketplace change.

And in a regulated marketplace like electricity, where regulation and policy have a pervasive role, it’s difficult for utilities to adapt to those changes and implement their own strategies unless regulation and policy are as proactive and innovative as we want regulated businesses to become.

So that’s a big piece of what we’re trying to do in this report: lay out the regulatory and institutional framework to allow that revolution to occur in a cost-effective way, both on the utilities’ side of things and on the electricity customer side of things.

David Roberts

One recurring theme in the report is that the value of power varies based on time and location. Explain why that’s true and why it matters so much.

Jesse Jenkins

The cost of delivering electricity to end consumers changes across time and location due to the variability of electricity demand. {Electricity use rises and falls throughout the day, typically reaching a mid-afternoon “peak.” It also rises and falls throughout the year, spiking during times of particular heat or cold.}

We have certain power plants that are only run to meet peak demand, and those tend to be more expensive when they’re running.

Here in New England, we still burn oil for about 7 percent of our electricity, mostly during peak-demand periods in winter and summer. In other places, it’s natural gas plants that are less efficient, or old coal plants. So inefficient power plants, around to meet those peak-demand periods, have a high cost for every megawatt hour that they produce.

We basically build the power system to meet our peak requirements. It would be like trying to build a highway that never has congestion, or traffic jams. You might need forty lanes, to make sure that on no day during the year would you ever have a traffic jam. That’s how we build the grid today, to meet peak demand and have power 24/7.

Meeting demand in periods of the year when electricity demand is really high can become very costly, and lead to investments in new capacity or generation from power plants that are very expensive.

And then power grids can become congested, so sometimes we have to turn on more expensive power plants on the other side of that congestion. That leads to different costs for delivering energy in different parts of the grid.

So both time and location matter for the cost of energy. Then why does that matter to consumers?

Well, it didn’t matter much when we didn’t have any ability to respond. We were passive consumers of electricity. But if I’m trying to decide whether to buy a smart thermostat and use it to control my AC load, or when to charge my electric vehicle, or whether it’s worth it to me to put solar panels on my house facing south, where they produce the most power, or facing west, where they produce power later in the afternoon — I don’t have the signals to guide those investments in an efficient way, unless I know whether it’s more valuable for me to generate power at a given time or given location.

David Roberts

So we want to get those price signals to consumers and distributed power producers.

{Today, residential customers tend to pay a flat “volumetric rate” for electricity -- “volumetric” just means that the more you use, the more you pay. It’s a flat per-kilowatt-hour rate, the same for all customers, no matter where they are located or the time of day. MIT’s report suggests making that price signal more granular. There are a variety of ways to do this, but the basic idea is that the price of electricity would vary throughout the day, and from place to place, based on real-time grid conditions.}

Jesse Jenkins

We tend to have very good, granular price signals in [wholesale] electricity markets.

{Wholesale electricity markets are where power producers — generation companies, or gencos, which own big power plants — sell their power to distribution utilities, which then sell it to customers. Such markets exist in “deregulated” areas of the country, where generation and distribution companies have been split off from one another. About 60 percent of US electricity is sold in deregulated areas. Wholesale power travels on high-voltage transmission lines; distribution utilities run the low-voltage grid that connects to customers.}

[Those price signals] reflect, at least at the transmission level, these variations in price across time and location. Locational-marginal pricing is used in most wholesale electricity markets in the United States.

{“Locational-marginal pricing,” or LMP, just means electricity pricing that varies across time and location, depending on the state of the grid.}

But we don’t send those price signals on to the demand side, and that means we’re probably overbuilding the amount of generation we need, or making costly decisions about transmission expansions. We’re not unlocking the flexibility in demand and distributed resources that we could if we had better price signals. That would lead to smarter decisions by everyone and lower costs for electricity.

David Roberts

In terms of the technology required to send and received those signals, is it just smart meters?

{Most “dumb” electricity meters on homes and buildings today only report monthly use — not much help to customers trying to make hour-by-hour decisions. “Smart meters” measure at more regular intervals and communicate the data wirelessly — no need for meter readers.}

Jesse Jenkins

We need some form of interval metering, something that can measure consumption not just every month, but on an hourly or even more frequent basis. That could be the traditional utility smart meters or it could be some new ICT [internet and communications technology] solutions that emerge in the future.

One thing we want to note is that this report is intended to be about the utility of the future, 10 years from now or so, not tomorrow morning. So we’re expecting a lot of additional change and particularly improvements in the cost of ICT.

But yeah, we need some capability to measure and record the pattern of consumption and injection into the grid each user has.

Bob Armstrong

Of course, there are going to be issues with cybersecurity and personal data security — those need to get resolved at the same time. It’s not just a matter of having smart meters, but there’s going to be a lot of system capability and infrastructure that needs to be put in place before this can work efficiently and securely.

David Roberts

Another distinction that comes up repeatedly is between utility capital expenditures (capex) and operating expenditures (opex).

{Capex refers to investments in physical infrastructure like power lines and transformers. Opex refers to the operational costs of managing the grid and serving power to customers.}

You talk about how costs are shifting from the former to the latter, but the current utility regulatory model inhibits that. Say a little bit about that shift.

Jesse Jenkins

We’re focused here on distribution companies that run the lower-voltage power grid that delivers energy to consumers — how they meet demand for the ways people use their networks.

Traditionally, utilities have invested in things like more transformers or new power lines and other hard equipment, to meet growth and peak demand on their grid.

Going forward, there’s more flexibility on the consumer side of the meter and more opportunities for embedding distributed resources throughout the grid in the locations that are most valuable. Utilities have the opportunity to contract with, incentivize, or otherwise pay distributed resources to help solve grid-management problems — without necessarily investing in their own assets.

So that would look like a shift from a traditional capital expenditure, like a new substation, to a series of contracts or payments for performance to distributed energy resource owners or aggregators or other intermediaries.

{“Aggregators” contract with dozens or hundreds of owners of DERs to pool their resources together — so, for example, an aggregator can deploy a few hundred home batteries as one giant grid-scale battery, or a few thousand smart appliances as one big source of controllable demand.}

David Roberts

Some utilities say you don’t need to change the basic regulatory model, you just need to let them invest in DERs and rate-base them like they rate-base their infrastructure investments.

{“Rate-basing” means recovering the costs through volumetric rates.}

Jesse Jenkins

Outside of vertically integrated {i.e., non-deregulated} markets, we generally caution against having regulated network utilities owning their own distributed resources.

That’s for a couple reasons, mostly related to concerns about the potential for conflicts of interest, or the exercise of market power. [Utilities] play a central role in every electricity market transaction — the network itself, the system that they operate, and increasingly the market platforms that they might run.

In order for that core platform function of the utility to be performed in an efficient and neutral manner, it’s ideal that the utilities responsible for managing those functions don’t have any particular interest or stake in the outcomes of competitive [market] decisions.

We addressed these issues head-on in the reform of wholesale energy markets and bulk power systems in the 1990s and 2000s — the restructuring era. That ultimately led to the creation of independent system operators (ISOs), independent third parties that had the responsibility for managing the [transmission] grid in the United States. In Europe, it was transmission system operators that both owned the grid and operated it but were barred from being active participants in competitive markets.

We outline a set of similar concerns about the role of distribution utilities, which are becoming more important; they are now at the heart of the electricity marketplace more than they were in the past.

If regulation is established well, and utilities are neutral about capital investments or operational investments, then they should be agnostic as to whether they own the asset or whether they contract with a third party. And it’s better from a regulatory perspective that they don’t have any financial stake in the competitive provision of distributed energy resources.

Instead, [they should] focus on procuring the services they need to run the grid from whoever is the cheapest provider of that service.

David Roberts

Say you have electricity prices that capture all the relevant time and location value of energy — locational-marginal pricing for retail electricity markets. Then you would have, theoretically, a perfectly transparent and efficient retail market for electricity.

Doesn’t that mean that any other costs or charges you introduce — grid maintenance costs, policy costs, what have you — are, almost by definition, going to distort the price signal and make the market less efficient?

{“Policy costs” refers to the utility’s cost of complying with government regulations and standards.}

Jesse Jenkins

Your question is well framed: If we have the right price signals, that accurately convey the marginal cost of consuming electricity, anything we add on top of that is going to change what people want to do and move us away from the efficient outcome.

And so for residual network costs and policy costs, we recommend removing those from volumetric charges and adding them to a fixed lump-sum payment, which wouldn’t distort short-term behavior.

{Today, the cost of electricity is generally lumped in with other costs — fixed costs to maintain the grid, policy costs — in the same flat volumetric charge, which means customers that use more power pay more of those costs as well.}

David Roberts

Aren’t there equity concerns with a lump-sum charge? By using more energy, wealthy people use the grid more. Shouldn’t they pay more?

Jesse Jenkins

We do acknowledge an important equity concern. We don’t comment on it in this report because there are any number of different ways that the charge can be allocated to different customers. There’s not really a guiding principle from economics, as far as efficiency, for how best how to do that, except to do it in a way that is minimally distortive of the efficient pricing signal.

So basically, society has to decide: What’s a fair way to pay for the public services that we get from the grid, or the public policy objectives that we tend to tack on to our electricity bills?

Protecting low-income ratepayers, raising funds for energy-efficiency programs, or paying for sunk network assets that everybody benefits from — these are all important goals. And they can be achieved without sacrificing the efficient price signals that we need to guide all of the myriad distributed decision makers on the power grid now.

For example, if we’re concerned about low-income residents paying too much in their fixed charges, we can have a means-tested credit, or reduction in the fixed charge, and a corresponding increase the fixed charge for everyone else to cover for that. There are lots of ways to address these equity concerns without blunting or distorting efficient price signals.

The other possibility that we raise, a little outside the box today, is that some of these costs, particularly the policy costs, might be best and most equitably distributed via taxes.

David Roberts

It does seem like public purposes {like pollution reduction} are properly funded through taxes. That has always struck me as the fairest way to do it — shift policy costs out of electricity rates, into a progressive tax.

Jesse Jenkins

We only pay for a portion of our roads with user fees, in the form of gas taxes. Some comes from general funds, raised by income and property taxes. Everyone pays for public schools, regardless of whether they send their kids to those schools. So there are certainly plenty of precedents for allocating public policy objectives, or infrastructure that has an amenity benefit, to everybody,

What makes this an important conversation, now or maybe in 10 years, is if somebody defects from the grid entirely, to avoid their share of sunk costs or policy costs, those costs don’t go away. They get shifted to someone else — other ratepayers or utility shareholders. It’s important we think through those implications carefully.

{“Defecting” from the grid means disconnecting entirely, supplying all your own power with DERs. It’s expensive today, but is expected to get cheaper as technologies progress. The worry is that high fixed charges would drive more people to defect; shifting those costs to taxes would avoid that danger.}

David Roberts

There’s a lot of activism right now around net metering.

{“Net metering” refers to policies that require utilities to buy power from owners of rooftop solar panels, usually at retail rates.}

But it actually violates a number of the principles in your report. How would you explain that to a net-metering fan?

Jesse Jenkins

The problem with net-metering isn’t necessarily the “net” part, it’s the fact that it’s netting out flat volumetric rates that don’t accurately reflect the value of electricity production or the cost of consumption. There’s basically no incentive to have flexible AC loads or install a storage device when you are charged the same exact price every hour of the day and every day of the year.

The second challenge is that [net metering] doesn’t help us understand the best place to put solar, and at what scale. Flat volumetric rates don’t convey either of those things.

For example, in California, or Germany, where there’s a lot of solar already deployed, the value of production from solar panels in the middle of the day is getting quite low, lower every year. But it’s still pretty high in the afternoon at 3 or 4 or 5 pm.

So it might actually be more beneficial to put your panels facing west, so they produce more power in the afternoon, than to put them facing south, so they produce more kilowatt hours in total. But with flat volumetric rates, we have no way of knowing that, or incentive to do so.

David Roberts

There are several “value of solar” studies going on for utilities.

{Utilities do not like paying rooftop solar producers retail rates under net metering. They think that overvalues the energy. So many utilities are studying, in a more detailed way, exactly what grid services solar panels provide, and what they are worth — the “value of solar.”}

It seems like the beginning of what could be an endless series of “value of x” distributed-technology hearings.

Jesse Jenkins

Exactly.

Those efforts are an important step, in that they start breaking down the individual value streams or services that solar might provide, and how much each is worth, to the individual and to the grid as a whole. It’s important because it recognizes that the rate we use now, the simple volumetric average rate, bears no resemblance to that value or cost.

But we recommend shifting from thinking about the value of a specific technologies to thinking about the value of electricity consumption or injection at different times and different places — basically the value of the service, rather than the value of the technology.

That’s important for practical reasons, because, like you said, if we try to define the value-of-solar tariff, then we need the value-of-storage tariff, the value-of-solar-and-storage-together tariff, the value-of-smart-home tariff, etc.

It becomes very impractical to define all those different tariffs and the value of each of them in a way that’s consistent and doesn’t cause distortions or encourage people to game those systems.

It is important to recognize that multiple technologies and behaviors can all provide the same limited set of services to the grid.

For example, there’s a lot of focus right now on energy storage, which is becoming a more affordable and potentially valuable part of the grid, and that’s leading to specific policy supports for energy storage. But a lot of the services that energy storage provides could also be provided by demand response or flexible demand.

{“Demand response” is electricity consumption that can be controlled and dialed back during times of high demand or grid congestion. It is typically run by aggregators.}

We don’t know a priori what’s going to be cheaper to provide those services. Is it going to be energy storage? If so, what kind of storage? Is it going to be flexible demand or demand response? Fuel cells that generate during the right times?

This isn’t just about solar. It’s about giving people the right set of incentives to make the right choices across a whole range of different options. And that requires us to think about the value of services to the grid, rather than trying to add up all the particular services that an individual technology might provide.

David Roberts

You talk about ways utilities could boost research and experimentation. They don’t seem well-suited to it, at least today.

Jesse Jenkins

Utilities have little financial incentive, if any, to pursue longer-term, riskier innovations, unless the regulatory environment directs them to do so. Remember, these are regulated monopolies.

We need mechanisms that encourage utilities to engage in, not fundamental research or basic science, but the kinds of later-stage, applied-research projects and demonstration projects that can accelerate the uptick of novel technologies.

And just as importantly, learning about the capabilities of those technologies and disseminating that knowledge across the industry helps make the (properly) risk-averse utilities more comfortable with these new technologies, and more likely to integrate them into their service.

David Roberts

But aren’t regulators bred to the bone to focus on reliability and cost-effectiveness? Experimentation makes them nervous.

Jesse Jenkins

I don’t think pursuing innovation has to undermine reliability. In fact, it could generate new tools to improve the reliability of the grid and fortify it against threats like stronger storms, flooding, and cyberattacks.

The basic regulatory principle is that we want to ensure that electricity service doesn’t cost any more than it should — particularly, any more than it would if the utility were acting like a competitive business.

And any competitive business in a changing marketplace, where technology and options for providing their services are evolving, would have some degree of research and development and demonstration, learning activities to understand what the future might look like.

Lower costs for ratepayers doesn’t just mean lower costs today, it also means lower costs over time. If we don’t invest in learning and innovation, ratepayers are going to pay more in the long term than they should for their energy.