3. The landscape gets politically messy and technologically diverse, varying by locale, as utilities, customers and politicians battle over new ways to produce and harvest energy.

Whichever scenario we end up with, solar power is an odds-on favorite source, because of its abundance. Every hour, our sun bombards the Earth with enough light to satisfy humanity’s energy needs for an entire year. But at least three barriers stand between us and that sunny future.

Problem One: Cell Cost

For solar power to meet 30% of the world’s electricity needs, it will need to fall from its current cost of a dollar per watt of electricity to 25 cents per watt, says Varun Sivaram, a science and technology expert at the Council on Foreign Relations, a nonprofit think tank.

A maintenance team in Oxford, Mass., changes out a faulty solar inverter on a 16.5-megawatt solar array owned and operated by BlueWave Solar on De. 4, 2017. PHOTO:ROBERT NICKELSBERG/GETTY IMAGES

The only way to get there, Dr. Sivaram argues in his forthcoming book, is by bringing to market solar-cell technologies that are currently still far from mass production, such as perovskite-based solar cells.

Perovskite cells can be made from materials that could be radically cheaper than conventional silicon. They can also take on novel forms, such as a tint on windows or thin printable sheets. But they still face significant barriers to commercialization: They tend to rapidly degrade when wet, and scientists can’t create large cells with the same efficiency as the small ones they can make in a lab.

While perovskite is promising, there’s no guarantee we’ll get it or any other better, cheaper technologies when we need them, since the energy industry isn’t investing enough in R&D to bring them to market, says Dr. Sivaram.

Energy companies tend to spend 1% to 2% of their revenue on R&D, he says, whereas semiconductor companies can easily spend ten times as much. The U.S. federal budget for energy research, $5 billion a year, is likely to be eclipsed by China’s budget for such research by 2020, he adds.

Problem 2: Energy Management

One reason we’re going to need cheaper solar cells is that the more solar there is on the grid, the less valuable it is to add more. This happens because sunlight is intermittent. It isn’t hard to get to the point where solar is producing too much power at some times of day, and none at all when it’s needed most. The first solar panel added to the grid helps offset mid-day consumption, but the last one to be added may be completely unnecessary, because the grid may already be saturated when it’s capable of producing the most power.

A worker moves a Vivint Solar Inc. solar panel during a home installation in Bergenfield, N.J., in Dec. 2017. PHOTO: DAVID WILLIAMS/BLOOMBERG NEWS

California, which gets about 10% of its electricity from solar power, already has this problem. On some sunny days, it has to pay other states to take electricity off its hands.

One solution is utility-scale power storage. But putting enough batteries on the grid to make a meaningful dent is a truly gargantuan feat, and batteries are still far too costly to address it at scale. Batteries currently handle only 1.7% of energy storage on the grid, according to the U.S. Department of Energy; the rest is almost entirely pumped hydro storage.

A more immediate solution to this problem could be a bigger and more spread-out electrical grid, says Ramez Naam, a lecturer on energy and the environment at Singularity University and an angel investor.

Some studies suggest that with bigger power grids and a continued drop in the price of existing solar technology, it is possible to get 30% of global electricity from current solar technology. That’s assuming panels continue to get cheaper as manufacturers scale up.

Problem Three: ‘Soft’ Utility Costs

Some are skeptical that technology is the real roadblock to the spread of solar. It could be the high “soft costs” related to building utility-scale solar power plants, including project design, permitting, siting and interconnection to the grid.

Tesla's Solar Roof with Powerwall home battery PHOTO: NICHOLA GROOM/REUTERS

“People in the tech community either conveniently ignore or truly don’t understand that they could honestly just give away solar panels for free now and soft costs would remain the bigger problem,” says Rob Day, a general partner at Spring Lane Capital, which invests in clean water, energy, food and waste projects.

The Department of Energy estimates that soft costs contribute up to 64% of the cost of a solar installation. The rest of the cost is split between mounting hardware for solar panels and the cells themselves.

One tantalizing possibility is that through a combination of rooftop solar panels and home batteries, individual consumers could just start harvesting their own electricity. This is one of the goals of Tesla Energy, which in January began production of solar panels and slate-like solar roof tiles at its Buffalo, N.Y. solar “Gigafactory 2.”

The technology is too expensive now, however, and even when it becomes affordable, we’ll probably still want that grid as a backup. That’s why the future, as usual, may look something like California—where in 2017 rooftop solar and local power providers took 25% of the business that would otherwise have gone to big utility companies.

Write to Christopher Mims at christopher.mims@wsj.com