Morningside Campus Access Updates

Updated July 10, 2024

The following color-coded campus status levels have been developed to simplify communication regarding the current circumstances on and around the Morningside campus. The campus status level is based upon the potential disruption to our academic mission and/or campus operations. The status level will be adjusted up or down to reflect then-current circumstances, including information received from law enforcement and other sources regarding potential events that could affect Columbia. Continued adherence to university rules and policies remains fundamental to maintaining an open campus. Read more.



Explore our expert insights and analysis in leading energy and climate news stories.

Energy Explained

Get the latest as our experts share their insights on global energy policy.


Hear in-depth conversations with the world’s top energy and climate leaders from government, business, academia, and civil society.


Find out more about our upcoming and past events.

Energy Markets

Uncertainties in Forecasting US Tight Oil Production

Commentary by Marianne Kah • January 25, 2018


The United States is enjoying substantial economic and political benefits from reclaiming its role as an oil superpower. Retaining those benefits is contingent upon the nation’s ability to grow its oil output to help supply rising global oil demand. The main source of production growth in the United States is from tight oil that is produced from impermeable reservoirs through hydraulic fracturing. One of the biggest uncertainties about long-term growth in US tight oil production is the rate of technological progress and whether productivity improvements will offset rising unit costs companies will face as they move beyond the locations with the best geology—the so-called sweet spots. If cost declines from productivity improvement do not offset cost increases from moving to locations with weaker geology, then the cost of bringing the oil to market will rise with less supply available at a given oil price. The rising cost of supply would constrain US tight oil production growth unless the global oil price rose to enable investment in higher-cost supply.

A recent study by M.I.T. attempts to address the issue of the technological improvements in the U.S. tight oil sector. While there was some confusion about the findings in the media,[1] some analysts had already expressed concern about the robustness of US tight oil production given that EIA’s weekly production estimates had been overestimating US monthly crude production for more than half of 2017 and because investors drove down US independent producers’ equity prices in the first half of the year, fearing a lack of capital discipline. It should be noted that most recently, EIA’s initial weekly estimates have underestimated US monthly crude production and that independent producers’ equity prices have generally recovered over the last four months.

Given the consequential impacts of the US shale revolution, both at home and abroad, it is important to understand what the MIT study[2] actually said and its implications for US production. This commentary attempts to highlight and clarify some of the critical findings of this important study for a broader audience. It also addresses more broadly the many uncertainties regarding forecasting future US tight oil production. Finally, this commentary provides a rationale for why tight oil technology improvement is likely to continue.

The MIT Study

The MIT study advanced the statistical methodology of isolating the rate of historical technology improvement from other factors improving productivity (defined as initial production from wells) in the Bakken tight oil play. The historical rate of technology improvement is an important reality check on the assumed improvement in forecasting future production. Their contribution was to remove the impact of the location of the well and its unique geology from the analysis. Without removing this effect, the statistical analysis would attribute location-driven improvement such as changing the mix of wells drilled to only the best ones (referred to as high grading) to technology improvement, thereby overstating the pace of historical technology improvement. Putting it simply, one needs to subtract the effects of all other causes of productivity improvement to be able to pinpoint changes arising from technology advancement.

The MIT study is critical of other statistical methodologies that lack sufficient spatial or locational variability, including the EIA methodology that they reviewed at the time of their study. However, the MIT study implicitly referenced—but did not explicitly cite—EIA’s 2014 methodology, which used county-level estimates of oil recovery per well.[3], [4] EIA has changed its production forecasting methodology since then in a direction that could mitigate some of MIT’s earlier concerns. EIA’s 2017 production forecast[5] has a substantially greater level of locational variability than its 2014 methodology as it delineates plays and subplays by county. It is important to note that the MIT article did not reference or criticize EIA’s current methodology or production forecast.

The MIT study concluded that technology improvement was responsible for about half of the improvement in initial production per well over the period analyzed, which is within the range of mainstream estimates. While some readers may perceive that as being a small impact, given the number of factors that impact well productivity, I would consider half to be a large contribution.

The MIT study had limited scope to assess the trend in historical productivity-related cost improvements. This is because it focused solely on initial well production (IP) rates, which (1) may not be a good proxy for recovery over the life of the field, although it is unclear whether using IP rates over- or underestimates technology-related improvements; and (2) does not include improvements in drilling days and other operational improvements that have structurally benefitted the economics of tight oil plays but not increased initial production rates. Unfortunately, no standard definition exists on what types of activities should be included under the umbrella of “technology improvement.” I believe it is more useful to separate unit-cost improvements into those that are structural (long-lasting change) and those that are cyclical (change with oil prices and activity levels) rather than to attempt to define what is and isn’t a technology improvement. This classification ensures that all structural improvements are considered in long-term production forecasts. Without including structural improvements besides those that impact initial production rates, the MIT methodology could underestimate future production.

The historical rate of technology-based productivity improvement may not be a good predictor of future improvement. Technology change is episodic, with periods of incremental change and periods of step changes, which are difficult to predict. For example, there was a major switch in completion styles in the Bakken in 2015,[6] which increased the number of fracture initiation points and proppant volumes, probably causing a break from the productivity trend and relationship of proppant use to initial production rates observed in MIT’s study period (Q1 2012 to Q1 2015). Relationships should be estimated over as long a time period as possible to capture episodes of change.

Technology to Continue to Improve Tight Oil Production

At some point, the rate of technology improvement will decline. However, that decline might not happen anytime soon because the science of producing these impermeable reservoirs is still immature, leaving significant room for improvement. Understanding of these unconventional reservoirs is still relatively limited. The oil industry is producing from sweet spots using primary production, similar to how it was operating in its early days 100 years ago. While improvement from any specific activity may come to an end, there should still be a long way to go in overall technology advancement.

To date, larger completions and increased drilling efficiency have been responsible for most of the technology improvements. The oil and gas production and service industries are now working to improve fracturing systems and reduce completion costs, driven by such factors as better reservoir evaluation techniques, proppant placement, and precision of fracks. The industry is also continuing to improve drilling efficiency with such examples as the use of “intelligent” drill bits and multilateral wells. Data analytics is also being used to improve well site selection and the efficiency of operations. Work will also continue on water availability, movement, and processing and reducing methane emissions.

The sector will also benefit from the ability of tight oil producers to undertake scientific experimentation, due simply to the large number of onshore wells being drilled and their relatively low cost compared to offshore wells. There are presently over 1,000 rigs drilling for oil and gas on US land and less than 20 rigs drilling in the US offshore. Each onshore unconventional well costs about $8 million, while costs for offshore wells can reach several hundred million dollars. Thus, the scope for experimentation is greater for tight oil wells than offshore wells. In addition, the sheer number of companies working to improve operations also supports innovation, and further improvements may come from majors as they take on a bigger role in tight oil development. These large energy companies could accelerate basic scientific advancement since they tend to invest more in R&D than smaller companies.

Broader Uncertainties about the Pace of US Tight Oil Production Growth

There are many additional factors besides technology improvement that will determine the magnitude and pace of US tight oil production growth. Indeed, the MIT researchers agree that a much broader scope of work would be required to forecast tight oil production, including building an understanding of all the components of supply cost. The factors that impact supply growth include:

  • the size of sweet spots or geological attractiveness of available plays (here it is critical to remember that technology improvement can create new sweet spots or expand existing ones);
  • the magnitude and pace of structural cost improvement, including technology advancement;
  • US oil industry reinvestment rates, which vary with the cost and availability of investment capital;
  • the oil price, which determines how much cash the oil industry has to invest and which areas are economic to produce;
  • service industry cost escalation as onshore US oil and gas industry activity accelerates;
  • government taxes;
  • costs of environmental compliance and water use; and
  • the pace of infrastructure development and whether it keeps up with production.

In conclusion, the MIT study’s findings do not imply that the shale boom is coming to an end. We should expect to see continued structural improvement and considerable growth in US tight oil production for the foreseeable future, with the pace of growth influenced by the factors discussed.


Marianne Kah is an Adjunct Senior Research Scholar and Advisory Board Member at the Center on Global Energy Policy. She is the former Chief Economist for ConocoPhillips.

The views represented in this commentary represent those of the author not the Center on Global Energy Policy or Columbia University.

[1] “U.S. Vastly Overstates Oil Output Forecasts, M.I.T. Study Suggests,” Bloomberg News, December 1, 2017,

[2] J. B. Montgomery and F. M. O’Sullivan, “Spatial Variability of Tight Oil Well Productivity and the Impact of Technology,” Applied Energy 195 (2017): 344–355,

[3] “Annual Energy Outlook 2014,” US EIA, April 2014, IF-10 and IF-11,

[4] “Oil and Gas Supply Module of the National Energy Modeling System: Model Documentation 2014,” US EIA, July 2014,

[5] “Assumptions to the Annual Energy Outlook 2017,” US EIA, July 2017, 134,

[6] The style change was from using uncemented pipe with swell packers to maintain zonal isolation to cemented pipe with a “plug-n-perf” process in which a plug isolates zones and perforating guns are fired, allowing proppant and fluid to move into the formation. The “plug-n-perf” process allows more learning between stages.

Our Work


See All Work
Energy Markets

Uncertainties in Forecasting US Tight Oil Production

Commentary by Marianne Kah • January 25, 2018