Do journal impact factors distort science?

From my inbox:

An ad hoc coalition of unlikely insurgents -- scientists, journal editors and publishers, scholarly societies, and research funders across many scientific disciplines -- today posted an international declaration calling on the world scientific community to eliminate the role of the journal impact factor (JIF) in evaluating research for funding, hiring, promotion, or institutional effectiveness.

Here's the rest of the story at Science Daily.

And a link to DORA, the "ad hoc coalition" in question.

It seems fairly obvious that impact factors do distort science.  But I wonder how much, and I also wonder if there are realistic alternatives that would do a better job of encouraging good science.

There are delicate tradeoffs here: some literatures seem to become mired within their own dark corners, forming small circles of scholars that speak a common language.  They review each others' work, sometimes because no one else can understand it, or sometimes because no one else cares to understand it.  The circle has high regard for itself, but the work is pointless to those residing outside of it.

At the same time, people obviously have very different ideas about what constitutes good science.

So, what does the right model for evaluating science look like?

Consensus Statements on Sea-Level Rise

In my mailbox from the AGU:

After four days of scientific presentations about the state of knowledge on sea-level rise, the participants reached agreement on a number of important key statements. These statements are the reflection of the participants of the conference and not official positions from the sponsoring societies.

 

Earth scientists agree that the global sea level is rising at an accelerated rate overall in response to climate change.

Scientists have a professional responsibility to inform government, the public, and the private sector about the impacts of rising sea levels and extreme events, and the risks they pose.

 

The geological record indicates that the current rates of sea-level rise in many regions are unprecedented relative to rates of the last several thousand years.

Global sea-level rise has changed rapidly in the past and scientific projections show it will continue to rise over the course of this century, altering our coasts.

 

Extreme events and their associated impacts will be more damaging and pose higher risks in the immediate future than sea-level rise.

Increasing human activity, such as land use change and water management practices, adds stress to already fragile ecosystems and can affect coasts just as much as sea-level rise.

 

Sea-level rise will exacerbate the impacts of extreme events, such as hurricanes and storms, over the long-term.

Extreme events have contributed to loss of life, billions of dollars in damage to infrastructure, massive taxpayer funding for recovery, and degradation of our ecosystems.

 

In order to secure a sustainable future, society must learn to anticipate, live with, and adapt to the dynamics of a rapidly evolving coastal system.

Over time feasible choices may change as rising sea level limits certain options. Weighing the best decisions will require the sharing of scientific information, the coordination of policies and actions, and adaptive management approaches.

 

Well-informed policy decisions are imperative and should be based upon the best available science, recognizing the need for involvement of key stakeholders and relevant experts.

As we work to adapt to accelerating sea level rise, deep reductions in emissions remain one of the best ways to limit the magnitude and pace of rising seas and cut the costs of adaptation.

 

Spatial Econometric Peeves (wonkish)

Nearly all observational data show strong spatial patterns.  Location matters, partly due to geophysical attributes, partly because of history, and partly because all the things that follow from these two key factors tend to feedback and exaggerate spatial patterns.  If you're a data monkey you probably like to look at cool maps that illustrate spatial patterns, and spend a lot of time trying to make sense of them.  I know I do.

Most observational empirical studies in economics and other disciplines need to account for this general spatial connectedness of things.  In principal, you can do this two ways:  (1) develop a model of the spatial relationship; (2) account for the spatial connectedness by appropriately adjusting the standard errors of your regression model.

The first option is a truly heroic one, and most all attempts I've seen seem foolhardy.  Spatial geographic patters are extremely complex and follow from deep geophysical and social histories (read Guns, Germs and Steal). One is unlikely to uncover the full mechanism that underlies the spatial pattern.  When one "models" this spatial pattern, assumptions drive the result, and the assumptions are, almost always, a heroic leap of faith.

That leaves (2), which shouldn't be all that difficult using modern statistical techniques, but does take some care and perhaps a little experimentation.  It seems to me many are a little too blithe about it, and perhaps select methods that falsely exaggerate statistical significance.

Essentially, the problem is that there's normally a lot less information in a large data set than you think, because most observations from a region and/or time are correlated with other observations from that region and/or time.  In statistical speak, the errors are clustered.

To illustrate how much this matters, I'll share some preliminary regressions from a current project of mine.  Here I am predicting the natural log of corn yield using field-level data that span about 15 years on most of the corn fields in three major corn-producing U.S. states. I've got several hundred thousand observations. Yes, you read that right--it's a very rich data set.

But corn yields, as you can probably guess, tend to have a lot of spatial correlation. This happens in large part because weather, soils, and farming practices are spatially correlated.  However, there isn't a lot of serial correlation in weather from year to year, so, my data are highly correlated within years, and average outcomes have strong geographic correlation, but errors are mostly independent between years in a fixed location.

Where the amount of information in the data normally scales with the square root of the sample size, when the data are clustered spatially or otherwise, a conservative estimate for the amount of information is the square root of the number of clusters you have.  In this data set, we don't really have fixed clusters.  It's more like smooth overlapping clusters.  But we might proxy the "number" of clusters  around the square root of 45, the number of years X states I have, because most spatial correlation in weather fades out after about 500 miles.  Although these states border each other, so it may be even less than 45.  Now, I do have weather matched to each field depending on the field's individual planting date, which can vary a fair amount.  That adds some statistical power. So, I hope it's a bit better than the square root of 45.  Either way, in the ballpark of 45 is a whole lot less than several hundred thousand.

I regress the natural log of corn yield on

YEAR:             a time trend
log (potential):  (output of a crop model calibrated from daily weather inputs),
gdd:                  growing degree days (a temperature measure),
DD29:              degree days above 29C (a preferred measure of extreme heat),
prec & prec^2:  season precipitation and precipitation squared,
PDay:                number of days since Jan 1 until planting
interaction between DD29 and CO2 exposure.

CO2 exposure varies a little bit spatially, and also temporally, both due to a trend from burning fossil fuels and other emissions, as well as seasonal fluctuations following from tree and leaf growth (earlier planting tends to have higher CO2, and higher CO2 can improve radiation water use efficiency in corn, which can effectively make the plants more drought tolerant).

The standard regression output gives:

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)     2.320e+00  3.014e-02   76.98   <2e-16 ***
I(YEAR - 2000)  1.291e-02  4.600e-04   28.06   <2e-16 ***
log(Potential)  5.697e-01  5.470e-03  104.14   <2e-16 ***
gdd             1.931e-04  4.177e-06   46.24   <2e-16 ***
DD29           -2.477e-02  1.149e-03  -21.56   <2e-16 ***
Prec            1.787e-02  9.424e-04   18.96   <2e-16 ***
I(Prec^2)      -4.939e-04  2.038e-05  -24.24   <2e-16 ***
PDay           -6.798e-03  6.269e-05 -108.45   <2e-16 ***
DD29:AvgCO2     6.229e-05  2.953e-06   21.09   <2e-16 ***

Notice the huge t-statistics: all the parameters look precisely identified.  But you should be skeptical.

Most people now use White "robust" standard errors, which uses a variance-covariance matrix constructed from the residuals to account for arbitrary heteroscedasticity.  Here's what that gives you:


                    Estimate   Std. Error   t value      Pr(&gt;|t|)
(Intercept)     2.319894e+00 3.954834e-02  58.65970  0.000000e+00
I(YEAR - 2000)  1.290703e-02 5.362464e-04  24.06922 5.252870e-128
log(Potential)  5.696738e-01 7.161458e-03  79.54718  0.000000e+00
gdd             1.931294e-04 5.058033e-06  38.18271  0.000000e+00
DD29           -2.477002e-02 1.397239e-03 -17.72783  2.557376e-70
Prec            1.786707e-02 1.099087e-03  16.25627  2.016306e-59
I(Prec^2)      -4.938967e-04 2.327153e-05 -21.22321 5.830391e-100
PDay           -6.798270e-03 7.381894e-05 -92.09386  0.000000e+00
DD29:AvgCO2     6.229397e-05 3.616307e-06  17.22585  1.698989e-66

The standard errors are larger and the T-values smaller, but this standard approach still gives us extraordinary confidence in our estimates.

You should remain skeptical.  Here's what happens when I use robust standard errors clustered by year:


                Estimate Std. Error t value Pr(&gt;|t|)    
(Intercept)     2.32e+00  5.57e-01  4.17 3.094e-05 ***
YEAR            1.29e-02  8.57e-03  1.52   0.12920    
log(Potential)  5.70e-01  9.11e-02  6.25 4.000e-10 ***
gdd             1.93e-04  7.89e-05  2.45   0.01443 *  
DD29           -2.48e-02  1.35e-02 -1.83   0.06719 .  
Prec            1.79e-02  1.06e-02  1.68   0.09243 .  
I(Prec^2)      -4.94e-04  2.15e-04 -2.29   0.02178 *  
PDay           -6.80e-03  8.17e-04 -8.32   2.2e-16 ***
DD29:AvgCO2     6.23e-05  3.50e-05  1.78   0.07510 .  


Standard errors are an order of magnitude larger and T-values are more humbling.  Planting date and potential yield come in very strong, but now everything else is just borderline significant.  It seems robust standard errors really aren't so robust.

But even if we cluster by year, we are probably missing some important dependence, since geographic regions may have similar errors across years, and in clustering by year, I assume all errors in one year are independent of all errors in other years.

If I cluster by state, the standard robust/clustering procedure will account for both geographic and time-series dependence within a state.  Since I know from earlier work that one state is about the extent of spatial correlation, this seems reasonable.  Here's what I get:



                  Estimate  Std. Error  t value  Pr(>|t|)    
(Intercept)     2.32e+00  1.1888e+00   1.9514 0.0510065 .  
YEAR            1.29e-02  4.6411e-03   2.7810 0.0054194 ** 
log(Potential)  5.70e-01  1.6938e-01   3.3632 0.0007706 ***
gdd             1.93e-04  2.2126e-04   0.8729 0.3827338    
DD29           -2.48e-02  2.6696e-02  -0.9279 0.3534781    
Prec            1.79e-02  1.2786e-02   1.3974 0.1622882    
I(Prec^2)      -4.94e-04  2.7371e-04  -1.8045 0.0711586 .  
PDay           -6.80e-03  4.9912e-04 -13.6205 < 2.2e-16 ***
DD29:AvgCO2     6.23e-05  6.8565e-05   0.9085 0.3635962    

Oops.  Now most of the weather variables have lost their statistical significance too.  But since I'm explicitly limiting assumed dependence in the cross section within years, now the time trend (YEAR) is significant, and it wasn't when clustering by YEAR.  We probably shouldn't take that significance very seriously, since some kinds of dependence (like technology) probably spans well beyond one state.

Note that this strategy of using large clusters combined with robust SE treatment (canned in STATA, for example) is what's recommended in Angrist and Pischke's Mostly Harmless Econometrics.

There are other ways of dealing with these kinds of problems.  For example, you can use a "block bootstrap" that resamples residuals whole years as a time, which preserves spatial correlation.  This is great in agricultural applications since weather is pretty much IID across years in a fixed locations and we should feel reasonably comfortable that there is little serial correlation.  One can also adapt the method by Conley for panel data.  Soloman Hsiang has graciously provided code here.  In earlier agriculture-related work, Wolfram Schlenker and I generally found that clustering by state gives similar standard errors as these methods.

The overarching lesson is: try it different ways and err on the side of least significance, because it's very easy to underestimate your standard errors and very hard to overestimate them.

And watch out for data errors: these have a way of screwing up both estimates and standard errors, sometimes quite dramatically.

If you had to patience to follow all of this, you might appreciate the footnotes and appendix in our recent comment on Deschenes and Greenstone.

Is peace needed in the climate/conflict debate?

Yes, argues Andy Solow in a comment in Nature today.  He bemoans the "fierce battle that has broken out within the research community" over whether there is a link between climate and conflict, highlighting the protracted argument we've been having with Halvard Buhaug and colleagues over whether there is a link between temperature and conflict in Africa (see here, here, and here).  Solow's main point seems to be that these sorts of "reduced form" analyses of climate and conflict reveal little about the true underlying processes that drive conflict, and thus that people running these models ("quants") need to engage with folks that are studying individual conflicts in depth ("quals") to make the analyses meaningful.

Overall his comments are quite reasonable, and if I can speak for my co-authors I think we are sympathetic to many of his arguments.  I have a few main points of disagreement, however.  First, I would not characterize the disagreements between Halvard (and colleagues) and us as a disagreement between "quants" and "quals".  We would certainly put ourselves in the former category and I'm guessing Halvard would too.  The disagreements between us have really been about how to do the quantitative analysis correctly, something over which we continue to disagree and will continue to disagree as long as folks are estimating panel models without fixed effects, or estimating models that include outcome variables as covariates.  But contrary to what Solow suggests, we definitely have been engaging with Halvard and his colleagues.  For instance, at Halvard's generous invitation I participated in a couple sessions on climate and conflict at last month's annual meeting of the International Studies Association, sessions organized by Halvard and colleagues. 

More importantly, though, I think Solow misperceives what estimating reduced form models is all about in this setting, and does not allow that the very reason a "quants" might want to engage with a "quals" is because a reduced form effect has been established.  A typical quant approach would be to start with an initial hunch that conflict and (say) temperature might be related, put together some data, and estimate a regression of conflict on temperature.  If there is a relationship in this "reduced form", then this would motivate more careful study of the underlying cases and mechanism - i.e. an engagement with the quals.  On the other hand, if there is no relationship in the reduced form, there is arguably no reason to take a "deeper look behind the statistics", as Solow puts it.  This is because the reduced form effect is going to encompass all possible ways in which climate will affect conflict (even if it doesn't illuminate them), and a null result in the reduced form would tell you that there is no mechanism linking the two phenomena.  

(Some initial hint at the underlying mechanism can certainly motivate how the reduced form is estimated, and this is something people are already attentive to.  For instance, suppose you think that shortfalls in income are what link climate variables to conflict events; in a setting where income is earned in agriculture, then focusing on climate during growing season months might make sense.  This approach for example is pursued in a nice paper by Harari and La Ferrara, in which they show that drought during the growing season matters a lot more for conflict than drought in the non-growing season.)  

The argument with Halvard and colleagues is about whether there is a reduced form effect.  Because my colleagues and I think that the reduced form relationship between temperature and conflict in Africa has been pretty well established by different research groups working at multiple scales (e.g. here, here, here, here, and here), we are in full agreement with Solow that a closer look at mechanism is now warranted, and that we need to learn a lot from quals about what might be going on.  But if you do not believe this reduced form, then it doesn't make sense to engage with quals on how climate might affect conflict.  

But a reduced form relationship is also much more than just a reason to talk to qualitative folks.  For the sake of argument, let's imagine that no manner of further engagement with quals is ever able to fully isolate the mechanism linking climate to conflict - perhaps there are just too many factors that are both affected by climate and that are potential contributors to conflict.  Does this mean that we learn nothing from the reduced form, or that it should be ignored by policy-makers?  

Probably not, for two reasons.  First, the reduced form effect of climate on conflict, when properly estimated, is estimating the causal effect of climatic fluctuations (typically inter-annual) on conflict outcomes.  This is because year-to-year variation in climate is pretty random and unlikely correlated with other unobserved variables that also affect conflict.  We know climate has "caused" conflict, we just don't know why.  

Second, even if we don't yet understand the mechanism linking the two variables, the fact that warmer temperatures and conflict are linked would appear useful to policymakers interested in whether we should take action to mitigate future climate change.  Clearly, as Solow points out, past responses to year-to-year variation and future response to long-run changes in means are not the same thing.  But given that adaptation to long-run temperature changes appears very slow at best, that sensitivity to extreme heat does not appear to be diminishing, and that past relationships are all we really have to go on, then assuming these relationships might carry into the future does not seem like a crazy place to start.  Moreover, important decisions are often made solely on "reduced form" evidence.  For example, I don't need to know how snake venom kills me to figure that, if I find myself in a room full of poisonous snakes, it might make sense to either invest in snake-proof boots or see if I can find a door.  The "adaptation" strategy here - buying boots - does involve knowing a little something about the mechanism: i.e. snake --> bite --> die;  the "mitigation" strategy - leaving the room - does not.  As a less prosaic example, aspirin was discovered before 1900 and was prescribed for decades before the mechanism underlying its efficacy was understood in the 1960s. Similarly, quinine has been used as a malaria drug since the 1600s although we still do not understand why it works.  So failing to understand a mechanism does not typically keep us from taking meaningful action when we've decided a reduced form effect is real.

Nevertheless, I hope (with Solow) that we can stop arguing about a few of the primary reduced forms in question -- e.g. the link between temperature and conflict in Africa -- and do more to understand what's going on and what to do about it.  Doing so will help inform adaptation investments in particular, and Solow is right that detailed case-study knowledge from the "quals" will be critical in building this understanding.  But moving in this direction involves establishing that there is a reduced form relationship between temperature and conflict, and it is for this reason we are arguing with Halvard and co.  

Irrigation in South Africa

Somewhat buried in a recent USDA update on the corn situation in South Africa are some interesting statistics on center pivot irrigation in South Africa. It’s a region where climate change is projected to have very negative effects on rainfed corn, so the trend towards irrigation could represent an important adaptation response, especially if it continues:

Irrigated corn area in South Africa has more than doubled during the past 12 years. It has increased from 100,000 hectares in 2000/01 to 240,500 hectares in 2011/12, as shown in Figure 5. Irrigated corn area in South Africa is also dependent on corn price, with the 2005/06 irrigated corn area being reduced by 73,000 hectares from the previous year when corn prices were low in 2005/06. In contrast, corn prices this year have been high and USDA/FAS forecast the 2012/13 irrigated corn area to be greater than last year’s record irrigated corn area of 240,500 hectares. By assuming average irrigated corn yields at 10-tons/hectare on 250,000 hectares of irrigated corn, USDA/FAS-Washington estimates the 2012/13 irrigated corn output to be nearly 2.5 million tons, or about 20 percent of South Africa’s total corn output of 12.5 million tons.

Mitigation with magic

This week marks a milestone of sorts. The carbon dioxide (CO2) levels measured at Mauna Loa in Hawaii have passed 400ppm for the firsttime. For all of the concerns and talk about global warming over the years, the bottom line is there have been very few successes yet at the scale required to slow down global emissions rates. If anything they are going faster than expected.

There have been tons of studies on potential mitigation strategies, and a clear message that multiple changes are needed to make a real dent in CO2 growth rates. For example, see here for a nice recent update by Steve Davis and colleagues on the concept of mitigation "wedges", where a wedge is enough to avoid 1 Gt of C emissions per year. The short story is that lots (about 9) of wedges are needed just to stabilize emissions as energy consumption grows, and another 10 are needed to reduce emissions enough to stabilize climate (in the figure below, "hidden" wedges refer to progress that is expected based on business as usual improvements in the emission intensity of economic growth).

One thing that has always puzzled me about mitigation discussion is how agriculture is treated. It's usually included, but mainly in the role of sequestering carbon in the soil, or reducing nitrous oxide and methane emissions. Not that these aren't worthwhile, but I think this emphasis tends to miss a more fundamental role -- demand for agricultural land drives much of deforestation, which remains a substantial (~10%) driver of global emissions.

As people search for investments to slow climate change, it seems useful to think about how to reduce demand for agricultural land. This roughly translates to a question of how to keep agricultural prices low enough that the incentives to clear new land don't outweigh the costs. On that, the evidence is pretty clear that investments in research and development for agriculture is a good bet. In an important report in 2001, Phil Pardey called agricultural R&D “slow magic” because it was both powerful but relatively slow to materialize. More recently, Keith Fuglie has provided some further evidence on the benefits of capacity to generate and deploy new technologies – countries with greater capacity consistently have higher growth in total factor productivity (TFP).

So investments in R&D Help, but do they really help enough to matter in terms of lowering land use change and slowing emissions? First, in the context of emissions, it's important that the technologies and practices that emerge from R&D are improving yield, and not just overall productivity of inputs. For example, reducing inputs to achieve the same yield is great for many reasons and will be measured as a rise in TFP, but will actually tend to increase land clearing because it lowers production costs without changing yield. To folks like Fuglie, who are mainly concerned about TFP and associated changes in prices, yields are a secondary issue. But in the context of land use and emission rates, it is yield and not overall TFP that is most critical.

Second, it’s possible that yield improvements lead to counteracting "rebound" effects that reduce their effectiveness in lowering emissions. For example, if yield increases are localized in land rich regions, they will tend to increase profits, raising the incentives to clear. Or if demand is elastic with respect to prices, so that small price reductions lead to higher demand, then total land savings are likely to be small.

Capturing all of these requires a reasonable model of agricultural markets and land use. In recent work with Uris Baldos and Tom Hertel of Purdue, we made an attempt to quantify how much mitigation benefit you get from investment in agriculture. In this case, we were focused specifically on investments related to climate change adaptation. To make a long story short, we estimated that broad-based investments in yield improvements reduce emissions at a cost of roughly $15 per metric ton of CO2. There's lots of assumptions embedded in that number, and we try our best to look at a full range of possible conditions at each link in the chain, coming up with a 95% confidence interval of $11-$22 per ton (see full paper here). Even the high end is pretty competitive with other mitigation actions. So it seems like “slow magic” may be a good mitigation strategy after all.  

How farmers, and the environment, could benefit from fertilizer taxes

Some of the worst water quality problems result from nutrient leaching and runoff from agricultural lands.  Nitrogen and phosphorus applied to cropland and not absorbed by crops in the process of photosynthesis will, one way or another, one day or another, end up in the water.  The same goes for animal waste. The nutrients cause algae blooms, reduced concentrations of dissolved oxygen, and diminished fisheries and ecosystem health (called eutrophication). 

While there has been some effort to deal with these problems, I know of no great success stories, and water quality continues to decline in the Mississippi, the Gulf of Mexico, and the Chesapeake, the Great Lakes, and countless other water bodies.

One obvious remedy would be to tax fertilizer.  This would be a nearly Pigouvian solution.  Better would be to tax runoff and leaching directly, but that’s basically impossible for practical reasons.

The obvious but rarely stated problem is that it would probably require an extraordinarily large tax to have any real influence on the quantity of fertilizer used.  And politically powerful farmers would cry foul, which is why this kind of tax will probably never happen.

But I wonder: What would the incidence of a fertilizer tax, broadly applied, really be?  Agriculture is fairly competitive.  And demand for agricultural commodities is nearly vertical—about as inelastic as anything.  The econ 101 analysis would suggest that burden of the tax would fall mainly on consumers.  That is, food commodity prices would go up enough to compensate for most all of the tax.

Now, I’ve seen some economists propose fertilizer taxes on a graduated scale.  If fertilizer is applied at a sufficiently low rate, no tax would be levied, but the tax would then rise sharply with higher application levels (which is where most runoff and leaching comes from).  This would be a little harder to monitor, but probably not too bad.  If done this way, the total tax bill would cost farmers far less, but cause the same reduction in fertilizer use.  And farmers would still get the full compensating price increase, since less output would be collectively produced.

I think it’s possible—indeed, very probable—that the induced rise in commodity prices would more than compensate farmers for the fertilizer taxes they would have to pay under the graduated tax system.  That is, a statutory tax on farmers could cause their profits to go up.

Anyway, I don’t think anyone has made this point or emphasized it very well.  And it’s an important one, at least politically speaking, because maybe farmers could get on board with a tax that actually benefits them.  I’m not sure if it would save the Chesapeake Bay or Great Lakes from eutrophication, but I bet it would do a lot more good than anything else that’s been tried.

Of course, there is no free lunch here: consumers would pay higher prices for food.

Are yields becoming less or more sensitive to temperature?

One of the hopes for adaptation is that farmers and breeders can somehow make crop yields less sensitive to temperature. Given that many agricultural areas have already seen significant amounts of warming, it’s fair to ask whether this is in fact happening. At the same time, it’s worth considering the possibility that things may be going in the opposite direction – that yields may in fact be getting more sensitive to weather. There’s a lot of history on this topic, which I’ll try to summarize briefly here before delving into some recent research summaries.

Before history, though, it’s useful to keep in mind three distinctions. First, yield sensitivity is not the same as yield variability. The latter can change if weather variability changes, even if sensitivity to weather stays the same. Second, an increase in yield sensitivity (or, for that matter, variability) is not necessarily a bad thing from a farmer’s or even a consumer’s point of view. If increased variability comes as a cost of increased average productivity, then both producers and consumers may benefit from the newer, more variable yield levels.

Third, this is potentially a VERY scale dependent question. Yield at a national scale, for example, can change for a lot of reasons apart from a change in yield variability for individual fields. For example, work by PeterHazell 30 years ago showed that changes in national yield variability were often driven by increased correlation of yields across different parts of the country. This could happen if production becomes more concentrated in a given geographic region, if practices or varieties become more uniform, or if weather becomes more correlated. There are lots of interesting questions related to sensitivity of aggregate yields to weather, some of which may be a topic for future posts. But here I want to focus on the evidence for yield sensitivity at the field level.

In thinking about changes in sensitivity, it might be useful to lump things into two bins – things that we would expect to reduce sensitivity to temperature, and things we’d expect to increase it. In the first category are many agronomic changes, particularly a stable supply of irrigation water and pest and disease control measures. It’s also possible that varietal changes could reduce sensitivity, for example if they are more resistant to pest or disease damage (especially if these disease pressures are different from year to year), or if they are more “hardy”, by which breeders often mean the ability to handle extreme moisture or temperature conditions. One example of this would be the flood tolerant rice lines recently developed. Finally, there is increased atmospheric CO₂, which one would expect to lower impacts of droughts by improving water use efficiency.

In the other bin are things that increase sensitivity. Some agronomic changes could increase sensitivity, for example high fertilizer rates can allow crops to be more responsive to good weather conditions. Similarly, new varieties can be more responsive to good conditions. The “responsiveness” of varieties to water and nutrients is in fact one of the main reasons that current varieties are so high yielding on average.

The net change in yield sensitivity of a cropping system is the result of all of these factors pushing in different directions. So which changes matter most in modern cropping systems? I recently came across a very nice volume from an IFPRI meeting held 25 years ago, which included several papers on the topic of yield variability (here’s a link, but beware of large file size). It seems many of their comments are just as relevant today as then. In particular, most of them emphasize the “responsiveness” issue – that well-fertilized modern hybrids are increasingly capable of taking advantage of good years. Especially in rainfed systems, this seems to dominate the other factors.

For example, here’s Donald Duvick (a world renowned corn breeder) on the experience of US corn:

“The yield advantage of the new hybrids is greatest when environmental conditions are most favorable. When environmental factors are severely limiting, as in drought, the new hybrids outyield the old ones, but by a smaller margin. It is likely, therefore, that present-day hybrids introduce the possibility of greater year-to-year variation in U.S. maize yields than used to occur, since they can expand their yields so much further in environmentally favorable seasons. When environmental factors are overwhelmingly limiting, the fallback in yield of the new hybrids is correspondingly greater than it would have been for the older hybrids, even though the new hybrids, under poor conditions, yield more than the old ones.”

He goes on to conclude

“I expect that any changes in U.S. maize farming practices will be made in concert. The tendency for the nation's maize plantings to be handled like one big farm will continue. Reactions to varying climatic conditions will be amplified, and some measure of instability in year-to-year national expectations for maize yields must continue. This may be the price that must be paid for high average yield in the long term.”

Similar conclusions are made by other authors in the volume on tropical maize, temperate and tropical wheat, and other crops. More recent work has made similar points, for example David Connor and others have written about how Australian wheat yields are now more variable than in previous generations, because farmers and cultivars are now so good at taking advantage of high rainfall years, but still face low yields in dry years because of insurmountable water constraints (compare 1920-1940 and 1980-2000 in figure below, taken from here. not shown here are the major recent droughts which would further show the remarkable amount of current variability in the system).

Of course, in some cropping systems things can go the other direction. For example, a recent study of maize yields in France showed a decline in rainfall sensitivity associated with increased irrigation. And another study at the grid-cell level for major maize and soybean regions found both areas of increases and decreases in temperature sensitivity. Although in that study, it seems plausible that a lot of the variation was due to noise, because cells of increasing sensitivity were often adjacent to cells with decreasing sensitivity.

In a recent study, which was a collaboration with CIMMYT (the International maize and wheat improvement center), we attempted to look at how heat sensitivity has changed for one particular set of cropping systems that are of wide interest – irrigated spring wheat. To make a long story very short, we looked at two breeding nurseries run by CIMMYT, and evaluated yield changes for differing temperature conditions. In the main nursery aimed at producing “elite” varieties, there has been steady yield progress at cool temperatures but no significant progress at warmer temperatures (left panel in figure below shows yield trends for 4 bins defined by grain filling temperatures. “climate corrected” trends account for shifts in locations and weather within each temperature bin over time). This seems to be a similar story of cultivars becoming more responsive to good conditions (low temperatures in the case of wheat), with a resulting increase in the sensitivity to bad conditions (high temperatures). Again, this isn't necessarily a bad thing, but does suggest that wheat farmers are getting more not less sensitive to high temperatures. Interestingly, when looking at nurseries targeted at drought stress (right panel), the gains appear more evenly distributed and maybe even higher under worse conditions (albeit with more noise because of a smaller sample size). This is a reminder that it is possible to create more hardy plants. But those won’t necessarily be the ones that farmers choose to grow.

Land Prices

There's a nice article in the New York Times about farmland prices.  The article suggests (as do most NYT articles on land values) that it's just another bubble.  Since we've had a couple bubbles in relatively recent memory, now it seems like everything's a bubble.

I'm not so sure.  There is a lot that's different about farmland values as compared to houses.  For one, we don't have quite the same level of debt relative to asset value.  It's reported as 25.5% for farmland.  That's the highest since the 1980s boom.  But then given today's low interest rates, it's not really a fair comparison to the 1980s.  It's also less than half the 60%+ level of debt we had for housing at the peak of the bubble.  Also, we don't have stated-income mortgages.  Banks are more cautious with farms than they were with houses during the Countrywide Mortgage years.

The simple analysis compares the price to rent ratio to current long-run interest rates. Today it looks like land that sells for around $10,000/acre rents for $500-600/year.  That's a 5-6% yield where 10 year T-bill rates at this writing are a little under 2% and 30-year T-bill rates are a little over 3%.

Now, there are costs and benefits to owning land instead of T-bills.  A benefit to land is that is has built-in inflation protection.  Inflation will push up commodity and land prices like everything else. So, the better comparison may be real rates, which are currently negative for 10-20 years out, which makes a 5-6% yield look pretty good.

A cost to owning land is that it's risky.  Rents are high because commodity prices are high.  If commodity prices fall, so will rents and yield.  So, what's the forecast for prices?  I'd say that's hard to tell.  They could go up further, they might crash if rapid productivity growth resumes.  The best bet is to look at futures prices, which suggests prices will fall somewhat from today's levels, but not that much.  Then there is the question of how to price uncertainty around that best guess.

It's not clear how costly that risk really is.  For a family with all their wealth piled into the farm, the risk is a big deal.  Most family farms own their land outright, or have relatively small mortgages.  A downturn in commodity prices, rents and land values would hurt for sure.  But typically there's not the kind of leverage we had in homes (the Times article has a notable exception).

Consider, in contrast, the risk to outside investors.  Wall Street types are pilling into land, probably because they see a reasonable yield.  But their exposure to risk is quite different.  These are highly diversified investors who have large portfolios of assets.  If farmland values crash, it's unlikely to be closely associated with the rest of the economy.  The fact that agriculture is a small share of our economy, and that commodity prices have little association with the rest of the economy, means that risk is far less important to these outside investors than it is to the family farm.  This basic point was implicitly made by Gary Gorton and Greet Rouwenhorst awhile back.

So, the basic math suggest that land investment is a reasonable deal right now.  With prices this high, it may not be quite as good a deal as buying a home in some post-bubble cities with low price-to-rent ratios.  But it's not bad, and probably not as difficult for an arms-length investor to get into.

So, I don't think this is a bubble. However, you should be forewarned that I've been wrong about this kind of thing before.

Why heat hurts

In a recent post I wrote about why rainfall is sometimes given too much credit for variations in crop production. Or, put differently, temperature deserves more credit than it often gets. A paper we have out this week delves into the reasons for this. Below is a brief background and summary, but see full paper here.

As background, there are now several studies showing close correlations between crop yields and temperature, in both rainfed or irrigated areas. More specifically, we now see that a lot of these correlations are driven by the tails of the temperature distribution – more very hot days means lower yields. Those of you familiar with our blog will know we often represent this effect by summing up degree days above some threshold, like 29 or 30 °C. See posts here, and here.

But correlations don’t tell us much about mechanisms of causation. And as a result, many people are wary about using correlations to project future impacts. So why might hot days in particular be so important? Agronomists are usually quick to talk about the key time of flowering when some really hot days can spell disaster. This is confirmed by many experimental studies, including one recent one here. But there are other things that could be going on in farmers’ fields in addition to this. One is the fact that hot air can hold more moisture, so that hotter days tend to have higher vapor pressure deficits (VPD). When air has higher VPD, plants lose more water per amount of CO₂ uptake, which lowers their water use efficiency. The response of most plants is to then slow down growth, which avoids losing too much water during the parts of the day where efficiency is lowest. Tom Sinclair, among others, has a boatload of interesting papers on this dynamic and the tradeoffs involved with selecting varieties that slow down more or less under high VPD. 

Now to the study. We wanted to see if the VPD effect could explain the observed importance of hot days for corn in the U.S. So we took an Australian crop model, APSIM, that handles the VPD effect and simulated some long time series of yields at different locations. We then look at whether the model can reproduce the observed relationships, and the match was quite good:

 We then look at some diagnostics in the model to confirm that it’s high VPD causing growth to slow, which causes lower yields. Then, to be sure it is not just that hot days or years tend to happen when rainfall is low, we artificially manipulate temperature or rainfall one at a time, and see how the model responds. The figure below shows the change in water stress (lower values on the y-axis mean higher amounts of water stress) for 3 key months for corn (July being the most important for final yield). This shows that a warming of 2°C causes a much bigger (3x) increase in water stress than a drop of in-season rainfall by 20%.

Why does this matter? First, it suggests that a lot of the effects of extreme heat (at least for this crop, in this region, in today’s climate) are related to drought stress. So when the media refers to the big drought, that is technically correct (at least in this case). But it’s important to be clear what we mean by drought – we don’t necessarily mean low rainfall (a common meteorological definition of drought), or even low soil moisture (a common definition of “agricultural drought”), but we mean a more agronomic definition of drought, such as “not enough water to grow as fast as a plant can”. The key is that water stress is not just about water supply to the plant, but about how much water it has relative to how much water it needs to maintain growth rates. The “needs” or water demand part depends on VPD, and hotter days on average tend to have higher VPD (and extended heat waves tend to have much higher VPD).

Second, it implies that efforts to adapt to warming in U.S. maize should probably, at least for the near future, focus on dealing with water stress associated with high VPD, rather than, say, the direct effects of heat damage during flowering.  

One thing we didn’t have the horsepower to do for this study would be to repeat the analysis with other types of crop models, most of which handle water stress slightly differently than APSIM. That would tell us how many models that have been used to project climate change impacts on U.S. agriculture are actually getting the key process right. It’s the type of model comparison that hopefully AGMIP will tackle – not just comparing projections of different models, but seeing how well they perform in reproducing historical responses to extreme heat.