It’s not obvious where the Left and Right got their ideas about where good technologies come from, but it certainly wasn’t by studying history.

For over a generation, there has been a bipartisan consensus against the public and private sectors working together. On the Left, dating at least as far back as Ralph Nader, exists an endemic skepticism of corporations. Big business has undue influence in politics and policymaking, and must be reined in and regulated to protect consumers and the environment.

Conversely, to those on the Right, the government commands too large a role in the marketplace. Better a small government than a clumsy leviathan, picking winners and losers according to pork barrel priorities of clueless politicians.

These are familiar ideological arguments, central to the identities of the two major parties in the United States. But they are highly misinformed when it comes to innovation.

Both the Left’s anti-corporate zeal and the Right’s anti-statist bias are challenged by the shale revolution, perhaps the most significant American technology success this side of the Internet.

The U.S. fracking revolution was the result of over 30 years of public and private investment, patiently and persistently swimming against conventional wisdom. There are many lessons to learn from the shale revolution, but a primary takeaway is this: neither the government nor the industry could have succeeded on their own.

It’s arguable that the entire saga of shale fracking innovation began with pork barrel politics. The 1976 Eastern Gas Shales Project, which was essential for proving the resource potential of shales, was the pet project of Senator Robert Byrd (D-W.Va.), who wanted the jobs for his constituents. The ESGP eventually sparked the interest of Texas oilman George Mitchell, who is now considered the forefather of the shale revolution.

Against widespread concerns about the corrupt ‘revolving door’ between government and industry, fracking shows why it’s essential that government agencies work closely with the private sector. The Department of Energy and the Gas Research Institute were extremely successful in recycling national labs research to meet industry needs. Because they had no ‘skin in the game,’ they also acted as intermediaries between potential rivals. None of this could have happened if public and private were keep at arms length.

America’s expansive property rights, often demonized by the Left, were also an essential part of this story. In the 1980s, Mitchell Energy bought significant chunks of land in the Fort Worth Basin. Mitchell gambled that, were it to prove the viability of fracking, the firm would then sell the land at a much higher price. Indeed, that’s exactly what happened when it merged with Devon Energy in 2002. Such a strategy would not have been possible had the state restricted underground property rights, which is the case in many other countries.

Perhaps the most remarkable aspect of the shale story is that, for decades, almost no one thought shale gas would be a winner. George Mitchell faced opposition and ridicule from the oil and gas industry, even within his own company. (Fortunately for the rest of us, he owned 51 percent of Mitchell Energy). The Department of Energy invested far more in advanced fossil technologies like coal gasification and synthetic fuels than in shale gas, the red-headed stepchild of other serious fossil energy investments. The shale revolution tells us emphatically that investments in technological innovation should embrace a broad portfolio approach…and have patience.

A longshot bet on a risky energy technology, made possible by the cozy relationship between government and industry, was neither a drain on public finances nor a case of corporate irresponsibility causing environmental catastrophe. The shale revolution puts $100 billion a year into Americans’ pockets in the form of cheaper energy, according to a Yale University study (which, incidentally, morethan pays for the investments that were ‘wasted’ on synfuels and coal gasification). Meanwhile, for the first time, cheap natural gas has overtaken dirty coal in US electric generation. Average damages from coal in the United States, from sulfur and nitrogen oxides and other pollutants, are estimated at $156 million per plant, compared to $1.5 million per gas plant. Natural gas consumes less water, emits less carbon dioxide, and has a lighter impact on landscapes than does coal.  

To be sure, there has been some bad behavior by frackers in the last decade, behavior that necessitates appropriate government regulation. Likewise, the government must adopt some limits on how it spends its money and avoid undue influence from rent-seekers in the private sector. Just as we argue for more collaboration and freedom between the public and private sector in the innovation process, no one should argue for endless government budgets or a completely untethered ‘free market.’

But the shale revolution shows us that innovation requires less dogmatism on the Left and Right and more acceptance of risk, and, perhaps more critically, some mess, along the way.

King and Trembath are senior analysts at the Breakthrough Institute, a research organization in Oakland, Calif., that has recently published a report on the policy lessons of the "shale gas revolution."