Taking crop analysis to MARS

I couldn’t bear to watch the clock wind down on October without a single post this month on G-FEED. So here goes a shot at the buzzer…

A question I get asked or emailed fairly often by students is whether they should use a linear or quadratic model when relating crop yields to monthly or growing season average temperatures. This question comes from all around the world so I guess it’s a good topic for a post, especially since I rarely get the time to email them back quickly.  If you are mainly interested in posts about broad issues and not technical nerdy topics, you can stop reading now.

The short answer is you can get by with a linear model if you are looking over a small range of temperatures, such as year to year swings at one location. But if you are looking across a bigger range, such as across multiple places, you should almost surely use something that allows for nonlinearity (e.g., an optimum temperature somewhere in the middle of the data).

There are issues that arise if using a quadratic model that includes fixed effects for location, a topic which Wolfram wrote about years ago with Craig McIntosh. Essentially this re-introduces the site mean into the estimation of model coefficients, which creates problem of interpretation related to a standard panel model with fixed effects.

A bigger issue that this question points to, though, is the assumption by many that the choice is simply between linear and quadratic. Both are useful simple approaches to use, especially if data is scarce. But most datasets we work with these days allow much more flexibility in functional form. One clear direction that people have gone is to go to sub-monthly or even sub-daily measures and use flexible spline or degree day models to compute aggregate measures of temperature exposure throughout the season, then use those predictors in the regression.  I won’t say much about that here, except that it makes a good deal of sense and people who like those approaches should really blog more often.

Another approach, though, is to use more flexible functions with the monthly or seasonal data itself. This can be useful in cases where we have lots of monthly data, but not much daily data, or where we simply want something that is faster and easier than using daily data. One of my favorite methods of all time are multivariate adaptive regression splines, also called MARS. This was developed by Jerome Friedman at Stanford about 20 years ago (and I took a class from him about 10 years ago). This approach is like a three-for-one, in that it allows for nonlinearities, can capture asymmetries, and is a fairly good approach to variable selection. The latter is helpful in cases where you have more months than you think are really important for crop yields.

The basic building block of MARS is the hinge function, which is essentially a piecewise linear function that is zero on one side of the hinge, and linearly increasing on the other side. Two examples are shown below, taken from the wikipedia entry on MARS.

MARS works by repeatedly trying different hinge functions, and adds whichever one gives the maximum reduction in the sum of squared errors. As it adds hinge functions, you can have it added by itself or have it multiply an existing hinge in the model, which allows for interactions (I guess that makes it a four-for-one method). Despite searching all possible hinge functions (which covers all variables and hinges at all observed values of each variable), it is a fairly fast algorithm. And like most data mining techniques, there is some back-pruning at the end so it isn’t too prone to overfitting.

For a long time I liked MARS but couldn’t figure out how to apply it to data where you want to include spatial fixed effects to account for omitted variables. Unlike models with pre-determined predictors, such as monthly average temperature or squared temperature, MARS has to search for the right predictors. And before you know what the predictors are, you can’t substract out the site-level means as you would in a fixed-effect model. So you can’t know what the predictors are until you search, but you can’t search if you can’t compute the error of the model correctly (because you haven’t included fixed-effects.)

One semi-obvious solution would be to just ignore fixed-effects, find the hinge-function predictors, and then rerun the model with the selected predictors but including fixed effects. That seems ok but all the problems of omitted variables would still be affecting the selection process.

Recently, I settled on a different idea – first use a crop simulation model to develop a pseudo-dataset for a given crop/region, then run MARS on this simulated data (where omitted variables aren’t an issue) to find the predictors, and then use those predictors on an actual dataset, but including fixed effects to account for potential omitted variables.

I haven’t had much time to explore this, but here’s an initial attempt. First, I used some APSIM simulations for a site in Iowa that we ran for a recent paper on U.S. maize. Running MARS on this, allowing either monthly or seasonal average variables to enter the model, results in just four variables that are able to explain nearly 90% of the yield variation across years. Notice the response functions (below) show the steepest sensitivity for July Tmax, which makes sense. Also, rainfall is important but only up to about 450mm over the May-August period. In both cases, you can see how the results are definitely not linear and not symmetric. And it is a little surprising that only four variables can capture so much of the simulated variation, especially since they all contain no information at the sub-monthly time scale.

Of course this approach relies on assuming the crop model is a reasonable representation of reality. But recall we aren’t using the crop model to actually define the coefficients, just to define the predictors we will use. The next step is to then compute these predictors for actual data across the region, and see how well it works at predicting crop yields. I actually did that a few months ago but can’t find the results right now, and am heading off to teach. I’ll save that for a post in the near future (i.e. before 2015).

Climate Change and Resource Rents

With the next IPCC report coming out, there's been more reporting on climate change issues.  Brad Plumer over a Wonkblog has nice summary that helps to illustrate how much climate change is already "baked in" so to speak.

I'd like to comment one point.  Brad writes "Humans can only burn about one-sixth of their fossil fuel reserves if they want to keep global warming below 2ºC."

I'd guess some might quibble with the measurement a bit, since viable reserves depends on price and technology, plus many unknowns about much fossil fuel there really is down there.  But this is probably in the ballpark, and possibly conservative.

Now imagine you own a lot of oil, coal and/or natural gas, you're reading Brad Plumber, and wondering what might happen to climate policy in the coming years.  Maybe not next year or even in the next five or ten years, but you might expect that eventually governments will start doing a lot more to curb fossil fuel use.  You might then want to sell your fossil fuels now or very soon, while you can.   If many resource owners feel this way, fossil fuel prices could fall and CO2 emissions would increase.  

This observation amounts to the so-called "green paradox."  Related arguments suggest that taxing carbon may have little influence on use, and subsidizing renewable fuels and alternative technologies, without taxing or otherwise limiting carbon-based fuels, might make global warming worse, since it could push emissions toward the present.

Research on these ideas, mostly theoretical, is pretty hot in environmental economics right now.  It seems like half the submissions I manage at JEEM touch on the green paradox in one way or another.  

All of it has me thinking about a point my advisor Peter Berck often made when I was in grad school. At the time, we were puzzling over different reasons why prices for non-renewable resources--mostly metals and fossil fuels--were not trending up like Hotelling's rule says they should.  Peter suggested that we may never use the resources up, because if we did, we'd choke on all the pollution.  Resource use would effectively be banned before it could be used. If resource owners recognized this, they'd have no incentive to hold or store natural resources and the resource rent (basically the intrinsic value based on its finite supply) would be zero, which could help explain non-increasing resource prices.

For all practical purposes, Peter understood the green paradox some 15-20 years ago.  Now the literature is finally playing catch up.  

David is a confirmed genius

The MacArthur Foundation just confirmed what we've known all long: that G-FEED's very own David Lobell is a genius.  Hopefully the $625k that MacArthur is forking over will free up David to do some additional blogging on his preferred choice of tennis racquet.

Big congrats to David!!

The noise lovers

I try not to use this blog much for rants, but it’s an easy way to post more frequently. So… we’ve been getting some feedback on our recent paper on drought stress in U.S. maize, some of it positive, some not. One thing that comes up, as with a lot of prior work, is doubts about how important temperature is. Agronomists often talk about how important the timing of rainfall is. And about how heat doesn’t hurt nearly as much if soils are very moist. To me this is another way of saying (1) other factors matter too and (2) there are interactions between temperature and other factors. The answer to both of these is “Of course!”

I am struck by the similarity of these discussions to those that Marshall posted about the empirical work on conflict. They go something like this:

Person A: “We’ve looked at the data and see a clear response that people tend to wake up when you turn the lights on”
Person B: “But people also wake up because they went to bed early last night and aren’t tired any more.”
A: “Yeah, ok”
B: “And the lights probably won’t wake them up if they are passed out drunk.”
A: “Ok, great point”
B: “I’ve woken up thousands of times in my 30-year career, and rarely did I get woken because someone turned the light on”
A: “ok”
B: “So then how can you possibly claim that turning on lights causes people to wake up”
A: “What? Are you serious? Did you even read the paper?”
B: “I don’t need to, I am an expert on waking up.”

And so on. I sometimes don’t know whether people seriously don’t understand the difference between explaining some vs. all of the variance, or if they just look for any opportunity to plug their own area of expertise. When we claim to see a clear signal in the data, it is not a claim that there is no noise around that signal. And some of the noise might include interactions with other variables. In fact, if there wasn’t any noise then the signal would have been known long ago.

The other day I was thinking of replacing my old tennis racquet, so I went to Google and typed in “prince thunder” (the name of my current racquet).  Turns out the top results were about a song that the artist Prince wrote called Thunder. Does that mean that Google is entirely useless? No, it means there is some error if you try to predict what someone wants based only on a couple of words they type. But with their enormous datasets, they are pretty good at picking up signals and getting the answer right more often than not. For most people typing those words, they were probably looking for the song.

Back to the crop example. Of course, heat will matter less or more depending on the situation. And of course getting rainfall right before key stages is more important than getting it after. These are both reasons for the scatter in any plot of heat (or any other variable) against yields. But neither of those refutes the fact that higher temperatures tend to result in more water stress, and lower yields. Or in Sol and Marshall’s case, that higher temperatures tend to increase the risk of conflict.

I sometimes think if one of us were to discover some magic combination of predictors that explained 95% of the variance in some outcome, there would be a chorus of people saying “you left out my favorite 5%!” Don’t get me wrong, there are lots of legitimate questions about cause and effect, stationarity, etc. that are worth looking into. But how much time should we really spend having the same old conversation about the difference between a signal and noise? 

Yet more cleverness: getting ambient temperature data from cellphones

Following up on an earlier post about some smarty-pantses that figured out how to use cell phone towers to extract estimates of local rainfall, many of these same smarty-pantses have now figured out how to use those same cell phones to provide information on local temperatures.  Here's the new paper, just out in Geophysical Research Letters [HT: Noah Diffenbaugh]:

Crowdsourcing urban air temperatures from smartphone battery temperatures
A. Overeem, J. C. R. Robinson, H. Leijnse, G. J. Steeneveld, B. K. P. Horn, and R. Uijlenhoet

Accurate air temperature observations in urban areas are important for meteorology and energy demand planning. They are indispensable to study the urban heat island effect and the adverse effects of high temperatures on human health. However, the availability of temperature observations in cities is often limited. Here we show that relatively accurate air temperature information for the urban canopy layer can be obtained from an alternative, nowadays omnipresent source: smartphones. In this study, battery temperatures were collected by an Android application for smartphones. A straightforward heat transfer model is employed to estimate daily mean air temperatures from smartphone battery temperatures for eight major cities around the world. The results demonstrate the enormous potential of this crowdsourcing application for real-time temperature monitoring in densely populated areas.

They validate their technique in a few big cities around the world, and it looks pretty neat.  As shown in their Fig 2, which shows temperatures for London over a 4-month period and is reproduced below, raw changes in battery temperature are highly correlated with variation in ambient temperature (compare the orange line and black line, reported correlation r=0.82), and their heat transfer model is able to get the levels close to right (compare the blue dots with the black line).

What we really want to know, of course, is whether this can also work in places where the weather-observing infrastructure is currently really poor (e.g. most of Africa), and thus were techniques like this could be extra useful.  It seems like there are a couple hurdles.  First, you need a lot of people with smartphones.  According to this article, smartphone penetration in Africa is currently around 20%, but Samsung (who might know) puts it at less than 10%.  Nevertheless, smartphone adoption appears to be growing rapidly (you can find them in just about any tiny rural market in western Kenya, for instance), and so this might not be such a limitation in a few years.  And something the authors worry about in colder and richer climes -- that their battery temperature readings are biased because people are in heated or air-conditioned buildings a lot -- is much less of a worry in places where people are outside more and don't keep their houses at a perfect 70F.

Second, to get temperature levels right, it appears that the authors have to calibrate battery temperatures in a given area to data on observed temperatures in that area -- which is obviously not helpful if you don't have observed data to start with.  But if all you care about is temperature deviations -- e.g. if you're running a panel model that is linear in average temperature -- then it seems like the raw battery temperature data give you this pretty well (see figure).  Then if you really need levels -- say you want to estimate how a crop responds to temperatures above a given threshold -- you could do something like David did in his 2011 paper on African maize and add these deviations back to somebody else's estimate of the climatology (David used WorldClim).

Given this, the authors' optimism on future applications seems fitting:


"In the end, such a smartphone application could substantially increase the number of air temperature observations worldwide. This could become beneficial for precision agriculture, e.g., for optimization of irrigation and food production, for human health (urban heat island), and for energy demand planning."

But hopefully the expansion of this technique into rural areas won't have to wait for observed data with which to calibrate their heat transfer model.  If that London plot above is representative, it seems like just getting the raw battery data could be really helpful.

The future of crop demand

It is common to hear statements about needing to increase food production by 70%, or even to double it, by 2050. In the past I’ve tried to avoid such statements, mainly because I hadn’t really dug into the sources behind them. But as part of prepping for a new class I’m teaching in the fall, I decided to take a closer look. And what I found, maybe not surprisingly, is a lot of confusion.

For starters, the baselines are often different, with some using 2000, some 2005-07, and some 2010. More importantly, the definition of “food” can be quite different, with some referring to cereal production, some to all calorie production, and some simply to the total value of agricultural output. And the studies use different assumptions about drivers of demand, like incomes or biofuel policies, so it’s often not clear how much differences are driven by assumptions in the models themselves vs. the inputs into the models.

Here’s a quick rundown of the common citations. First and foremost is the FAO, which produced the commonly cited 70% number. Last year they actually revised this down to 60% in their new projections, but not because the projected demand changed very much, but because they up-revised their estimate of actual output in 2006. The baseline for the FAO number is still 2006, so the 60% refers to an increase over a 44-year period. And the 60% refers to price-weighted aggregate of output, so that part of the 60% is simply a shift toward producing higher value stuff. Total cereal production only rises 46%, from roughly 2 to 3 billion tons per year. About two-thirds of that increase occurs by 2030. In terms of calorie consumption, global per capita consumption rises by 11%, and total calorie consumption rises by 54%.

The “doubling” statement, as far as I can tell, comes mainly from a 2011 paper by David Tilman and colleagues that said calorie demand would double between 2010 and 2050, and protein demand would rise by 110%. That was mainly based on extrapolating historical patterns of cereal and protein demand as a function of income, combined with projections of income growth. Coincidentally, doubling of production is also what we found in the baseline simulations we did for a climate adaptation analysis published earlier this year in ERL, and discussed in a prior post.

I won’t take time here to bore you with details of the methods in FAO vs. Tilman vs. others. But it seems a lot of the disparity is not so much the methods as the input assumptions. For example, FAO has world GDP per capita growing at an average rate of 1.4%, which they acknowledge as conservative. In contrast, Tilman has a global per capita GDP growth rate of 2.5% per year. Over a 40-year period, that translates to an increase of about 75% for FAO but 170% for Tilman! In our paper, we had a rate in between of 2% per year based on USDA projections. The reason we still get a doubling with lower income growth than Tilman is probably because we had a larger biofuel demand. (Note that my coauthors Uris Baldos and Tom Hertel have since switched to using income projections from a French group, which - maybe not surprisingly - are a little more pessimistic than the American ones.)  Now, global per capita growth rates only tell us so much, because what mainly matters for demand is how incomes grow at lower and middle income levels where people most rapidly increase consumption with higher income. Unfortunately, studies don’t usually report for the same sets of countries, and I’m too lazy to try to recompute. But the global numbers suggest pretty important differences at all income levels.

To me, it’s always useful to compare these projections to historical growth rates. Below I plot global cereal production from FAO since 1961. A couple of things are clear. First, production was about 150% higher in 2010 than 50 years earlier. Second, the growth rate appears pretty linear at a clip of roughly 28 million tons per year. A naive extrapolation gives an increase of 1.1 billion tons over a 40 year period from 2010 to 2050. For reference, I show what a 50% and 100% increase from 2010 trend levels would be. Obviously the 50% number (or the FAO’s 46%) are closer to this naive extrapolation than a doubling. 

This isn’t to say that the doubling number is definitely wrong, but just that it would mean a significant acceleration of cereal demand, and/or a significant shift of calorie consumption away from cereal-based products. It would be really nice if someone could systematically explore the sources of uncertainty, but my guess for now is that income growth is a big part of it. Unfortunately, this means our hope for narrowing uncertainty is largely in the hands of economists, and we know how good they or anyone else are at predicting GDP growth . But for those who work mainly on supply side questions, it’s mostly good enough just to know that demand for crop production will rise by 50% or more, because even 50% is a pretty big challenge.

(Note: for anyone interested in a summary of an unrelated recent paper on extreme heat, see here. And for an exchange we had about adaptation in the US see here. Wolfram told me a blog about the latter is coming, but as I told him, so is Christmas.)

Crop insurance under climate change

How should crop insurance premiums adjust to a changing climate in order to remain actuarial fair?

Short answer:  Very slowly.

That seems pretty obvious to me, and hopefully to anyone who thinks about it for a few minutes, even if you think climate change is ultimately going to have big impacts.  Moreover, the way crop insurance premiums are already determined---as a function a farmer's own recent yield history---gradual adjustment of premiums will take place naturally.

So, what should USDA's Risk Management Agency do, if we think nasty crop outcomes like last year are going to be more frequent going forward?

Well, I'll abstain from making a recommendation, but I will say that if they do absolutely nothing, there will be no significant budgetary implications.

None of this is to say that there might not be other ways to improve crop insurance.

Update: So, if this issue is so unimportant, why do I mention it?  Because I'm seeing and hearing the question a lot, and my general sense is that energy and resources might be better spent on other issues.

A climate for conflict

Sol, Ted Miguel, and I are excited to have a paper we've been working on for a while finally come out. The title is "Quantifying the impact of climate on human conflict", and it's out in this week's issue of Science.  Seems like this should get Sol's tenure package at Berkeley off to a pretty nice start.  

Our goal in the paper is rather modest, and it's all in the title:  we collect estimates from the rapidly growing number of studies that look at the relationship between some climate variable and some conflict outcome, make sure they are being generated by a common and acceptable methodology (one that can credibly estimate causal effects), and then see how similar these estimates are.  We want to understand whether all these studies spread across different disciplines are actually telling us very different things, as the ongoing debate on climate and conflict might seem to suggest.

We compute things in terms of standardized effects -- percent change in conflict per standard deviation change in climate -- and find that, by and large, estimates across the studies are not all that different.  All the exciting/tedious details are in the paper and the supplement (here's a link to the paper for those without access, and here is a link to all the replication data and code).  We've also gotten some nice (and not so nice) press today that discusses different aspects of what we've done.  See here, here, here, here, here, here, here, here, here, and here for a sampler.  Most importantly, see here. 

We want to use this space to answer some FAQ about the study.  Or, really, some FHC:  Frequently Heard Criticisms.  A bunch of these have popped up already in the press coverage.  Some are quite reasonable, some (we believe) stem from a misunderstanding or misrepresentation of what we're trying to do in the paper, and some are patently false. 

So, in no particular order, some FHC we've gotten (in bold), and our responses.  We include direct quotes from our critics where possible.  Apologies for the lengthy post, but we are trying to get it all out there. 

UPDATE (Friday Aug 2, 4pm): We have now heard from 3 different folks - Andy Solow, Idean Salehyan, and Cullen Hendrix - who each noted that what they were quoted as saying in these articles wasn't the full story, and that they had said lots of nice things too.  After talking to them, it seems there is much more agreement on many of these issues than the press articles would have you believe, and in many cases it looks like the journalists went far out of their way to highlight points of disagreement. I guess this shouldn't be surprising, but we believe it does the debate a pretty serious disservice.  So our responses below should be interpreted as responses to quotes as they appeared, not necessarily responses to particular individuals' viewpoints on these issues, which the quotes might not adequately represent.  We are not trying to pick fights, just to clarify what our paper does and doesn't do.

1.  You confuse weather and climate. 
(verbatim from Richard Tol, courtesy Google Translate)

This is an old saw that you always get with these sorts of papers.  The implied concern is that most of the historical relationships are estimates using short-run variation in temperature and precip (typically termed "weather"), and then these are used to say something about future, longer-run changes in the same variables ("climate").  So the worry is that people might respond to future changes in climate -- and in particular, slower-moving changes in average temperature or precip -- differently than they have to past short-run variation.

This is a very sensible concern.  However, there are a couple reasons why we think our paper is okay on this front. First, we document in the paper hat the relationship between climate variables and conflict shows up at a variety of time scales, from hourly changes in temperature to century-scale changes in temperature and rainfall.  We use the word "climate" in the paper to refer to this range of fluctuations, and we find similar responses across this range, which provides some evidence that today's societies are not that much better at dealing with long-run changes than short-run changes.  This is consistent with evidence on climate effects in other related domains (e.g. agriculture).  

Second, we do not to make explicit projections about future impacts - our focus is on the similarity in historical responses.  Nevertheless, the reader is definitely given the tools to do their own back-of-the-envelope projections:  e.g. we provide effects in terms of standard deviations of temperature/ rainfall, and then provide a map of the change in temperature by 2050 in terms of standard deviations (which are really really large!).  That way, the reader can assume whatever they want about how historical responses map into future responses.  If you think people will respond in the future just like they did in the past, it's easy multiplication.  If you think they'll only be half as sensitive, multiply the effect size by 0.5, etc etc.  People can adopt whatever view they like on how future long-run responses might differ from past short-run responses; our paper does not take a stand on that. 

2. Your criterion for study inclusion are inconsistently applied, and you throw out studies that disagree with your main finding.
(Paraphrasing Halvard Buhaug and Jurgen Sheffran)

This one is just not true.  We define an inclusion criterion -- which mainly boils down to studies using standard techniques to account for unobserved factors that could be correlated with both climate and conflict --  and include every study that we could find that meets this criterion. In more technical terms, these are panel or longitudinal studies that include fixed effects to account for time-invariant or time-varying omitted variables.

In a few cases, there were multiple studies that analyzed the exact same dataset and outcomes, and in those cases we included either the study that did it first (if the studies were indistinguishable in their methods), or the study that did the analysis correctly (if, as was sometimes the case, one study met the inclusion criteria and another did not).  

So, the inclusion criterion had nothing to do with the findings of the study, and in the paper we highlight estimates from multiple studies whose do not appear to agree with the main findings of the meta-analysis (see Figure 5). We did throw out a lot of studies, and all of these were studies that could not reliably identify causal relationships between climate and conflict -- for instance if they only relied on cross-sectional variation.  We also throw out multiple studies that agreed very strongly with our main findings!  We provide very detailed information on the studies that we did and did not include in Section A of the SOM.

Sheffran refers to other recent reviews, and complains that our paper does not include some papers in those reviews.  This is true, for the methodological reasons just stated.  But what he does not mention is that none of these reviews even attempt to define an inclusion criterion, most of them review a very small percentage of the number of papers we do, and no make any attempt to quantitatively compare results across papers.  This is why our study is a contribution, and presumably why Science was interested in publishing it as a Research Article.

3. You lump together many different types of conflict that shouldn't be lumped together.  Corollary: There is no way that all these types of conflict have the same causal mechanism.  
(Idean Salehyan: "It’s hard to see how the same causal mechanism that would lead to wild pitches would be linked to war and state collapse")

This quote is very substantial misrepresentation of what we do.  First, nowhere do we make the claim that these types of conflict have the same causal mechanism.  In fact we go to great lengths to state that climate affects many different things that might in turn affect conflict, and the fact that the effect sizes are broadly similar across different types of conflict could be explained by climate's pervasive effect on many different potential intervening variables (economic conditions, food prices, institutional factors, ease of transportation, etc). 

Second, we take great pains to separate out conflicts into different categories, and only make comparisons within each category.  So we calculate effect sizes separately for individual-level conflict (things like assault and murder), and group level conflict (things like civil war).  So, again contra Idean's quote, we are not actually comparing baseball violence (which we term individual-level) with war and state collapse (which are group conflicts).  Read the paper, Idean! 

But the whole point of the paper is to ask whether effect sizes across these different types of conflict are in fact similar!  What we had in the literature was a scattering of studies across disciplines, often looking at different types of conflict and using different methodologies.  This disarray had led to understandable confusion about what the literature as a whole was telling us.  Our goal was to put all the studies on the same footing and ask, are these different studies actually telling us that different types of conflict in different settings respond very differently to climate?  Our basic finding is that there is much more similarity across studies than what is typically acknowledged in the debate.  

Whether this similarity is being driven by a common underlying mechanism, or by multiple different mechanisms acting at the same time, is something we do not know the answer to -- and what we highlight very explicitly as THE research priority going forward.  

4. You cherry pick the climate variables that you report
(paraphrasing Halvard Buhaug).

We try really hard not to do this.  Where possible, we focus on the climate variable that the authors focused on in the original study.  However, the authors themselves in these studies are almost completely unrestricted in how they want to parameterize climate.  You can run a linear model, a quadratic model, you can include multiple lags, you can create binary measures, you can create fancy drought measures that combine temperature and rainfall, etc etc.  Authors do all these things, and often do many of them in the same paper.  Since we can't include all estimates from every single paper, we try to pick out the author's preferred measure or estimate, and report that one.  In the cases where authors tested many different permutations and did not hint at their "preferred" estimate (e.g. in Buhaug's comments on our earlier PNAS paper), we pick the median estimate across all the reported estimates.  Section A2 in the SOM provides extra detail on all of these cases.

5. This paper is not based on any theory of conflict, so we learn nothing.

This is very related to FHC #3 above, and we get this from the political science crew a lot.  The thing is, there are a ton of different theories on the causes of conflict, and empirical work so far has not done a great job of sorting them out.  In some sense, we are being atheoretical in the paper -- we just want to understand whether the different estimates are telling us very different things.  As noted above, though, the fact that they're generally not would seem to be very important to people interested in theory! 

Claire Adida, a political science prof at UCSD (and @ClaireAdida on twitter), put it really nicely in an email:  "I don't understand this ostrich politics. How about saying something like 'we really want to thank these economists for doing a ton of work to show how confident we can be that this relationship exists. It's now our turn - as political scientists - to figure out what might be the causal mechanisms underlying this relationship.' " (Btw, I have no idea what "ostrich politics" means, but I really like it!) 

6. People will adapt, so your results are a poor guide for impacts under climate change. 
(paraphrasing Jurgen Sheffran, courtesy Google Translate; Cullen Hendrix: "I'm optimistic.  Unlike glaciers, humans have remarkable adaptive capacity"; as well as audience members in every seminar we've ever given on this topic).

This is very related to FHC #1 above. It is definitely possible that future societies will become much better at dealing with extreme heat and erratic rainfall.  However, to just assume that this is the case seems to us a dangerous misreading of the existing evidence.  As stated above, available evidence suggests that human societies are remarkably bad at dealing with deviations from average climate, be it a short-lived temperature increase or a long-term one.  See here and here for other evidence on this topic.

And it has to be the case that knowing something about how yesterday's and today's world respond to climate tells us more about future impacts than knowing nothing about how societies have responded to climate.  The alternative - that the present world tells us nothing about the future world - just does not appear consistent with how nearly anybody sees things. 

7.  A lot of your examples - e.g. about civilization collapse - do not pertain to the modern world.

We got this in peer review. It's true that a lot of these collapse stories are from way back.  The Akkadian empire collapsed before 2000 BC, after all!  In a companion paper, forthcoming in Climatic Change, Sol and I look a little more carefully at this, and it actually turns out that per capita incomes in many of these societies, pre-collapse, were remarkably similar to incomes in many developing countries today.  To the extent that economic conditions shape conflict outcomes -- a common belief in economics and political science -- then this provides at least some evidence that these historical events are not completely irrelevant to today. 

More basically, though, it seems like hubris to just assume that "this time things are different".  At the time of their collapse, each of these societies (the Akkadians, the Maya, some of the Chinese dynasties, Angkor Wat) were incredibly advanced by global standards, and they probably also did not figure that climate would play a role in their demise.  Because we don't yet have a firm grasp on why climate affects conflict, it again seems dangerous to assume that things are completely different today -- just as it seems dangerous to conclude that modern societies are going to be completely destroyed by climate change, a claim we make nowhere in the paper. 

However, we do hope that "this time is different"!  It would be quite nice if the Mayan and Angkor Wat examples did not, in fact, pertain to the modern world. 

8. You can't claim that there is an overall significant relationship between climate and conflict if many of the studies you analyze do not show a statistically significant effect. 
(Halvard Buhaug: "I struggle to see how the authors can claim a remarkable convergence of quantitative evidence when one-third of their civil conflict models produce a climate effect statistically indistinguishable from zero, and several other models disagree on the direction of a possible climate effect")

This is a basic misunderstanding of what a meta-analysis does.  The beauty of a meta-analysis is that, by pooling a bunch of different studies, you can dramatically increase statistical power by increasing your sample size. It's even possible to find a statistically significant result across many small studies even if no individual study found a significant result.  This happens in medical meta-analyses all the time, and is why they are so popular in that setting:  each individual study of some expensive drug or procedure often only includes a few individuals, and only by combining across studies do you have enough statistical power to figure out what's going on.

So the fact that some of the individual studies were statistically significant, and others were not, does not necessarily affect the conclusions you draw when you average across studies.  In our case, it did not:  the mean across studies can be estimated very precisely, as we show in Figures 4 and 5, and discuss in detail in the SOM.

A final point:  we find a striking consistency in findings in the studies that look at temperature in particular.  Of the 27 modern studies that looked at a relationship between temperature and conflict, all 27 estimated a positive coefficient.  This is extremely unlikely to happen by chance - i.e. very unlikely to happen if there were in fact no underlying relationship between temperature and conflict.  Think of flipping a coin 27 times and getting heads all 27 times.  The chance of that is less than 1 in a million.  This is not a perfect analogy -- coin flips of a fair coin are independent, our studies are not fully independent (e.g. many studies share some of the same data) -- but we show on page 19 in the SOM that even if you assume a very strong dependence across studies, our results are still strongly statistically significant. 

9. Conflict has gone down across the world, as temperatures have risen. This undermines the claims about a positive relationship between temperature and conflict.
(Idean Salehyan: "We've seen rising temperatures, but there's actually been a decline in armed conflict".)

There are a couple things wrong with this one. First, many types of conflict that we look at have not declined at all over time.  Here is a plot of civil conflicts and civil wars since 1960 from the PRIO data, summed across the world.  As coded in these data, civil conflicts are conflicts that result in at least 25 battle deaths (light gray in the plot), and civil wars are those that result in at least 1000 deaths (dark gray).  As you can see, both large wars and smaller conflicts peaked in the mid-1990s, and while the incidence of larger wars have fallen somewhat, the incidence of smaller conflicts is currently almost back up to its 1990s peak.  These types of conflicts are examined by many of the papers we study, and have not declined.   

As another check on this, I downloaded the latest version of the Social Conflict in Africa Dataset, a really nice dataset that Idean himself was instrumental in assembling.  This dataset tracks the incidence of protests, riots, strikes, and other social disturbances in Africa.  Below is a plot of event counts over time in these data.  Again, you'd be very hard pressed to say that this type of conflict has declined either.  So I just don't understand this comment.

Second, and more importantly, there are about a bazillion other things that are also trending over this period.  The popularity of the band New Kids On The Block as also fallen fairly substantially since the 1990s, but no-one is attributing changes in conflict to changes in NKOTB popularity (although maybe this isn't implausible).  The point is that identifying causal effects from these trends is just about impossible, since so many things are trending over time.  

Our study instead focuses on papers that use detrended data - i.e. those that use variation in climate over time in a particular place.  These papers, for instance, compare what happens to conflict in a hot year in a given country, to what happens in a cooler year in that country, after having account for any generic trends in both climate and conflict that might be in the data.  Done this way, you are very unlikely to erroneously attribute the effects of changes in conflict to changes in climate. 


10. You don't provide specific examples of conflicts that were caused by climate.  

(Halvard Buhaug: "Surprisingly, the authors provide no examples of real conflicts that plausibly were affected by climate extremes that could serve to validate their conclusion. For these and other reasons, this study fails to provide new insight into how and under what conditions climate might affect violent conflict")

I do not understand this statement.  We review studies that look at civil conflict in Somalia, studies that look at land invasions in Brazil, studies that look at domestic violence in one city in one year in Australia, studies that look at ethnic violence in India, studies that look at murder in a small set of villages in Tanzania.  The historical studies looking at civilization collapses in particular try to match single events to large contemporaneous shifts in climate.  We highlight these examples in both the paper and in the press materials that we released, and they were included in nearly every news piece on our work that we have seen.  So, again, this comment just does not make sense. 

Perhaps implicit in this claim is often some belief that we are climate determinists.  But as we say explicitly in the paper, we are not arguing that climate is the only factor that affects conflict, nor even that it is the most important factor affecting conflict.  Our contribution is to quantify its role across a whole host of settings, and our findings we hope will help motivate a bunch more research on why climate should shape conflict so dramatically (see Claire's quote above).

11.  You are data mining. Corollary: What you guys are demonstrating is a severe publication bias problem -- only studies that show a certain result get published.
(Andy Solow: "In the aggregate, if you work the data very hard, you do find relationships like this. But when you take a closer look, things tend to be more complicated." As an aside, Andy sent us a very nice email, noting in reference to the press coverage of our article: "From what I've seen so far, all the nice things I said - that you are smart, serious researchers working on an important and difficult problem, that your paper will contribute to the discussion, that you may well be right - have been lost in favor of concerns I have and that, as I took pains to point out, you are already aware of.")

This is related to FHC #2 and #4 above. We have defined a clear inclusion criterion, and only include studies that meet this criterion.  As detailed in the SOM Section A2, we do not include a number of studies that agree very strongly with our main findings - for instance Melissa Dell's very nice paper on the Mexican Revolution.  Again, our inclusion criteria is based on methodology, not findings. 

The publication bias issue is a tougher one, and one which we explicitly address in the paper -- it even gets its own section, so it's not something we're trying to hide from.  We test formally for it in the SOM (Section C), finding limited evidence that publication bias is behind our results.  We also note that it's not clear where the professional incentives now lie in terms of the sorts of results that are likely to get published or noticed.  The handful of climate/conflict skeptics have garnered a lot of press by very publicly disagreeing with our findings, and this has presumably been good for their careers.  Had they instead published papers that agreed with our findings, it's likely that the press would not have had these folks as their go-to this time around.  Similarly, journals are probably becoming less interested in publishing yet another paper that shows that higher temperatures lead to more conflict.  Because so many of the papers we review are really recent (the median publication date across our studies was 2011), we feel that it is unlikely that all of these results are false positives.

GMOs: Franken food or technological savior?

Amy Harmon has a great in-depth story in the New York Times about the science and controversy surrounding GMO crops.  The article is nominally built around the worldwide problem of citrus greening, which is huge, but nicely builds in a much broader story about GMOs.

Another great source for learning more about the GMO controversy is the book Tomorrow's Table, by Pamela Ronald and Raoul Adamchak.

My own take on GMOs so far: The hysteria against them is likely overblown, but the extraordinary promises by technological optimists are overblown too.  Traditional breeding is a solid and, over the long run, often superior and less costly substitute to GMOs.  What's more worrisome to me is that intellectual property laws and regulatory costs may be acting to concentrate the seed business and make it less competitive.  These later issues are complex, not exactly my forte, and I don't presently see clear answers to any of it.

Anyhow, it's nice to see good reporting on an evocative topic.

Commodity Speculation or Market Power

After seeing how much Goldman profited from selling MBS that they knew were junk, it's hard to feel sorry for Goldman receiving so much grief for its commodity storage and trading activities.  The worry seems to be that because Goldman has become increasingly involved in commodities markets that they must be manipulating prices for profit, and in the process pushing prices away from their fundamental values---ie., supply and demand.

Do we actually know whether there is a problem here? It's possible that Wall Street is trying to manipulate the market.  But this is a hard thing to do, even for a really big company, especially one that doesn't produce the stuff it's trying to monopolize.  Also bear in mind that anyone can buy and store commodities, so it's not like there are huge barriers to entry.  Those who have tried to corner commodity markets in the past haven't fared well.

My sense is that cornering a commodity market via hoarding is basically impossible once the market realizes what the major player(s) is doing.  And if they're having senate hearings about Goldman's storage and trading activities, I think it's fair to say the cat's out of the bag.

So, what is Goldman doing? If it's not a market power story I'd guess they're trying to buy low and sell high, just like everybody else. They probably believe they have a better handle on market fundamentals than other commodity speculators.  Perhaps they do.  But if this is all they are doing, then they are effectively reducing price volatility and helping to make the market work more efficiently.

On public radio this morning a reporter (sorry, I forget who), asked Omarova whether Goldman's profits just meant that consumers were paying higher prices.  Omarova said "that's absolutely right." But it's absolutely wrong if Goldman's just speculating.  Goldman's profits are coming out of the pockets of speculators who bet prices would fall when they rose, and vice versa.  In fact, that's probably the case if it's a market power issue too.

Anyway, if this is about Goldman trying to corner the storage market, that's a problem and Goldman deserves the grief they're receiving. But that strikes me as unlikely as it would be foolhardy.  My guess is that this is just speculation, which means Goldman's profits translate directly to better allocation of commodities over time, less commodity price volatility, and basically zero influence on average prices.