Claims of Bias in Climate-Conflict Research Lack Evidence [Uncut Version]

Marshall and I had an extremely brief Correspondence published in Nature last week. We were reacting to an article "Sampling bias in climate–conflict research" by Adams et al. in Nature Climate Change and an Editorial published in Nature discussing and interpreting some of the statements by Adams et al.  Below is the full 300-word unedited "director's cut" of our original submission, which was edited down by the journal. Two other short comments in the same issue (actually same page!) provided other perspectives.

Butler & Kefford pointed out that prior literature describes climate stress as amplifying pre-existing conflict risk, rather than being the "sole cause" as Adams et al. suggest previous studies suggest (yes, this is getting confusing). We agree with this interpretation and the evidence seem to back it up pretty clearly. Our analyses indicate fairly consistent percentage changes in conflict risk induced by climatic shifts, so places with high initial risk get more of a boost from climatic events.

Glieck, Lewandowsky & Kelly argued that the earlier articles were an oversimplification of prior research, and that focusing on locations where conflict occurs is important for helping to trace out how climate induces conflict. I would think that this point would resonate with Adams et al, since some of those authors do actual case studies as part of their research portfolio. It does kind of seem to me that case studies would be the most extreme case of "selection on the outcome" as Adams et al define it.

Finally, here's what we originally wrote to fit on a single MS Word page:

Claims of Bias in Climate-Conflict Research Lack Evidence 

Solomon Hsiang and Marshall Burke 

A recent article by Adams et al. [1] and accompanying editorial [2] criticize the field of research studying links between climate and conflict as systematically biased, sowing doubt in prior findings [3,4]. But the underlying analysis fails to demonstrate any evidence of biased results. 

Adams et al. claim that because most existing analyses focus on conflict-prone locations, the conclusions of the literature must be biased. This logic is wrong. If it were true, then the field of medicine would be biased because medical researchers spend a disproportionate time studying ill patients rather than studying each of us every day when we are healthy. 

Adams et al.’s error arises because they confuse sampling observations within a given study based on the dependent variable (a major statistical violation) with the observation that there are more studies in locations where the average of a dependent variable, the conflict rate, is higher (not a violation). Nowhere does Adams et al. provide evidence that any prior analysis contained actual statistical errors. 

We are also concerned about the argument advanced by Adams et al. and repeated in the editorial that it is “undesirable” to study risk factors for populations at high risk of conflict because it may lead to them being “stigmatized.” Such logic would imply that study of cancer risk factors for high risk patients should not proceed because success of these studies may lead to the patients being stigmatized.  We believe that following such recommendations will inhibit scientific research and lead to actual systematic biases in the literature. 

Research on linkages between climate and conflict is motivated by the desire to identify causes of human suffering so it may be alleviated. We do not believe that shying away from findings in this field is an effective path towards this goal. 

References 

1. Adams, Ide, Barnett, Detges. Sampling bias in climate–conflict research. Nature Climate Change (2018).
2. Editorial. Don’t jump to conclusions about climate change and civil conflict. Nature, 555, 275-276 (2018).
3. Hsiang, Burke, Miguel. Science (2013) doi:10.1126/science.1235367
4. Burke, Hsiang, Miguel. Ann. Rev. Econ. (2015) doi:10.1146/annurev-economics-080614-115430

Climate change, crop failure, and suicides in India (Guest post by Tamma Carleton)

[This is a guest post by Tamma Carleton, a Doctoral Fellow at the Global Policy Lab and PhD candidate in the Ag and Resource Econ department here at Berkeley]

Last week, I published a paper in PNAS addressing a topic that has captured the attention of media and policymakers around the world for many years – the rising suicide rate in India. As a dedicated student of G-FEED contributors, I focus on the role of climate in this tragic phenomenon. I find that temperatures during India’s main growing season cause substantial increases in the suicide rate, amounting to around 65 additional deaths if all of India gained a degree day. I show that over 59,000 suicides can be attributed to warming trends across the country since 1980. With a range of different approaches I’ll talk about here, I argue that this effect appears to materialize through an agricultural channel in which crops are damaged, households face economic distress, and some cope by taking their own lives. It’s been a pretty disheartening subject to study for the last couple years, and I’m glad to see the findings out in the world, and now here on G-FEED.

First, a little background on suicides in India. The national suicide rate has approximately doubled since 1980, from around 6 per 100,000 to over 11 per 100,000 (for reference, the rate in the U.S. is about 13 per 100,000). The size of India’s population means this number encompasses many lives – today, about 135,000 are lost to suicide annually. There have been a lot of claims about what contributes to the upward trend, although most focus on increasing risks in agriculture, such as output price volatility, costly hybrid seeds, and crop-damaging climate events like drought and heat (e.g. here, here, and here). While many academic and non-academic sources have discussed the role of the climate, there was no quantitative evidence of a causal effect. I wanted to see if this relationship was in the data, and I wanted to be able to speak to the ongoing public debate by looking at mechanisms, a notoriously thorny aspect of the climate impacts literature.

The first finding in my paper is that while growing season temperatures increase the annual suicide rate, also hurting crops (as the G-FEED authors have shown us many times over), these same temperatures have no effect on suicides outside the growing season. While the results are much less certain for rainfall (I’m stuck with state-by-year suicide data throughout the analysis), a similar pattern emerges there, with higher growing season rainfall appearing to cause reductions in suicide: 

These effects seem pretty large to me. As I said above, a degree day of warming throughout the country during the growing season causes about 65 suicides throughout the year, equivalent to a 3.5% increase in the suicide rate per standard deviation increase in growing season degree days. 

The fact that the crop response functions are mirrored by the suicide response functions is consistent with an agricultural mechanism. However, this isn’t really enough evidence. Like in other areas of climate impacts research, it’s difficult to find exogenous variation that turns on or shuts off the hypothesized mechanism here –  I don’t have an experiment where I randomly let some households’ farm income be unaffected by temperature, as others’ suffer. Therefore, aspects of life that are different in the growing and non-growing seasons could possibly be driving heterogeneous response functions between temperature and suicide.  Because this mechanism is so important to policy, I turn to a couple additional tests.

I first show that there are substantial lagged effects, which are unlikely to occur if non-economic, direct links between the climate and suicidal behavior were taking place (like the psychological channels linking temperature to violence discussed in Sol and Marshall’s work). I also estimate spatial heterogeneity in both the suicide response to temperature, as well as the yield response, and find that the locations where suicides are most sensitive to growing season degree days also tend to be the locations where yields are most sensitive: 

The fact that higher temperatures mean more suicides is troubling as we think about warming unfolding in the next few decades. However, I’m an economist, so I should expect populations to reallocate resources and re-optimize behaviors to adapt to a gradually warming climate, right? Sadly, after throwing at the data all of the main adaptation tests that I’m aware of from the literature, I find no evidence of adaptation, looking both across space (e.g. are places with hotter average climates less sensitive?) and across time (e.g. has the response function flattened over time? What about long differences?):

Keeping in mind that there is no evidence of adaptation, my last calculation is to estimate the total number of deaths that can be attributed to warming trends observed since 1980. Following the method in David, Wolfram and Justin Costa-Roberts’ 2011 article in Science, I find that by end of sample in 2013, over 4,000 suicides per year across the country can be attributed to warming. Integrating from 1980 to today and across all states in India, I estimate that over 59,000 deaths in total can be attributed to warming. With spatially heterogeneous warming trends and population density, these deaths are distributed very differently across space:

While the tools I use are not innovative by any means (thanks to the actual authors of this blog for developing most of them), I think this paper is valuable to our literature for a couple reasons. First, while we talk a lot about integrating our empirical estimates of the mortality effects of climate change into policy-relevant metrics like the SCC, this is a particular type of death I think we should be incredibly concerned about. Suicide indicates extreme distress and hardship, and since we care about welfare, these deaths mean something distinct from the majority of the deaths driving the mortality rate responses that we often study. 

Second, mechanisms really matter. The media response to my paper has been shockingly strong, and everyone wants to talk about the mechanism and what it means for preventative policy. While I have by no means nailed the channel down perfectly here, a focus on testing the agricultural mechanism has made my findings much more tangible for people battling the suicide epidemic on the ground in India. I look forward to trying to find ways to improve the tools at our disposal for identifying mechanisms in this context and in others.

Finally, as climate change progresses, I think we could learn a lot from applying David, Wolfram, and Justin’s method more broadly. While attribution exercises have their own issues (e.g. we can’t, of course, attribute with certainty the entire temperature trend in any location to anthropogenic warming), I think it’s much easier for many people to engage with damages being felt today, as opposed to those likely to play out in the future. 

Food fights

Eoin McGuirk and I have a new NBER working paper out that (we argue) sheds important new light on the economic origins of conflict in Africa.  This is a topic that has received a lot of scrutiny over the years, but one which has remained tricky to sort out empirically, as conflict could be both a cause and a consequence of deteriorating economic conditions.

Our approach is to utilize plausibly random variation in local-level food prices induced by (1) variation in global food prices and (2) variation in what crops are grown and consumed in different regions across Africa.  There's a lot of variation in both, which is helpful.  For instance, here's a plot from the paper of where some of the main crops in Africa are grown, with the colors showing the percentage of each cell planted to different crops (using the SAGE data):

Clearly, world price movements for maize are going to have geographically distinct impacts from global price movements in rice or coffee.

We find strong evidence that changes in economic conditions induced by food price movements have substantial effects on conflict across the continent.  Eoin wrote a nice summary of the paper for the IGC blog, which I shamelessly copy below:

+++++++++++++++++++++++++

Food Fights: Food Prices and Civil Conflict in Africa

Over one million people have died from direct exposure to civil conflict in Africa since the end of the Cold War alone. What causal role do economic factors play in this tragedy? We study the effect of world food prices on violence across the continent, finding that the impact depends on both the type of conflict and whether a region mainly produces or consumes food.

The role that economic conditions have in shaping conflict risk is notoriously difficult to identify. Civil conflict can cause enormous economic damage by destroying human and physical capital, so any correlations between economic conditions and conflict could be explained by reverse causality. Moreover, the type of institutional environment in which conflict occurs is likely to be the type that also depresses economic activity, raising the additional concern of omitted factors that confound the analysis.

Identifying economic effects using high-resolution data on prices and conflict

To overcome these challenges, we study the impact of food price shocks on local conflict using panel data constructed at the level of a 0.5 x 0.5-degree subnational grid cell (roughly 55km x 55km at the equator) across the African continent. Studying conflict at this local level has several advantages over the more traditional country-level approach. Since food crops represent a higher average share of both production and consumption in Africa than in any other region, a price spike can be expected to significantly raise income for agricultural producers and reduce real income for consumers at the same time. With the help of spatial data on where specific crops are grown and consumed, we are able to separate these producer and consumer effects within countries. Moreover, we can confidently dismiss the potentially confounding concern that African civil conflict affects world cereal prices, given that the entire continent accounts for less than 6% of global production over our sample period. The recent emergence of highly detailed data on African conflict also allows us to consider the effects of price movements on different varieties of violence within countries.

Factor conflict: for permanent control of land

We first look at the type of large-scale armed conflict that tends to be the subject of most attention among researchers. We label this “factor conflict”, as these battles typically relate to the permanent control of land. Our results identify the differences that one would expect to see between places with crop agriculture and those without. A rise in food prices of one standard deviation leads to a 17% decline in conflict in regions where food production is common and farmers benefit from higher prices, and a 9% increase in conflict in populated areas with no food production – areas where consumers are hurt by rising food prices.

This finding is consistent with the idea that opportunity costs are important drivers of violence: when food prices are high, it makes more sense for producers to harvest crops; when they are low, farmers must look for other ways to make ends meet. At the same time, high prices can push some of the poorest consumers toward joining armed conflict groups in order to maintain a living wage; a fall in prices, on the other hand, pushes their real wages up and allows them to return to the productive sector. In both cases, real income is a fundamental factor in the decision to fight.

Output conflict: the taking of movable goods

The second type of violence we consider relates not to the control of territory but to the appropriation of output. This “output conflict”, as we call it, is more transitory, more atomistic, and less organised than factor conflict. The goal of output conflict is simply to take food, money, or other movable goods. It can appear in our dataset as looting, raiding, rioting, or interpersonal theft.

As with factor conflict, higher food prices cause an increase in output conflict in areas without crop agriculture (e.g., urban centres). This finding is consistent with the idea that declining real wages push some consumers towards appropriation in order to make ends meet. The more interesting distinction, however, comes in areas with crop agriculture. In contrast to factor conflict, we see clear evidence that output conflict increases with rising food prices in regions that produce the associated crops. What can explain this difference? Even in food-producing regions, not everyone makes a living from farming. For net consumers in these regions, higher food prices simultaneously (i) lower real income, as a given wage buys less food, and (ii) increase the value of appropriable output, as nearby farmers have assets that have appreciated in value. These forces provide incentives for consumers to engage in output conflict when prices rise.

We corroborate this finding using a large multi-country survey dataset with information on self-reported crime. Commercial farmers who grow food crops in food-producing regions are more likely to report being victims of theft and physical assault following a spike in food prices. Interestingly, farmers who instead grow cash crops such as cotton or coffee do not report being victimised following an equivalent price spike. In this case, higher prices are unlikely to have reduced real income for consumers, who tend not to purchase cash crops in order to get by.

Key relationships and policy implications

In sum, the effect of food prices on conflict in Africa depends on both the sub-national location and the type of conflict. Global price increases lower large-scale factor conflict in agricultural areas, and increase it in urban areas. For output conflict, the effect is more straightforward: higher prices will increase the incidence of conflict in both rural and urban areas. Overall, our results provide clear evidence that much of the observed conflict in Africa has economic roots. Our approach allows us to isolate the role of changing economic conditions from other factors that might affect conflict risk (such as governance), and to show that these changes can have quantitatively large effects on different types of conflict.

Our findings suggest that policies to minimise conflict risk in the face of food-price shocks must be tailored to sub-national areas. Incentives to farm rather than fight can reduce the risk of factor conflict in rural areas when global prices are critically low. Similarly, policy makers ought to be prepared for potential instability in urban areas when global prices are critically high. Potential policy responses include guaranteed prices for producers and well-timed releases of buffer-stock food for consumers when global prices reach critical levels in either direction. A complementary approach could take the form of local “workfare” programmes—welfare conditional upon work—that shift from urban to rural regions as prices fall, and vice versa.

Everything we know about the effect of climate on humanity (or "How to tell your parents what you've been doing for the last decade")

Almost exactly ten years, Nick Stern released the his famous global analysis on the economics of climate change.  At the time, I was in my first year of grad school trying to figure out what to do research on and remember fiercely debating all aspects of the study with Ram and Jesse in the way only fresh grad students can.  Almost all the public discussion of the analysis revolved around whether Stern had chosen the correct discount rate, but that philosophical question didn’t seem terribly tractable to us.  We decided instead that the key question research should focus on was the nature of economic damages from climate change, since that was equally poorly known but nobody seemed to really be paying attention to it.  I remember studying this page of the report for days (literally, days) and being baffled that nobody else was concerned about the fact that we knew almost no facts about the actual economic impact of climate—and that it was this core relationship that drove the entire optimal climate management enterprise:

click to enlarge

So we decided to follow in the footsteps of our Sensei Schlenker and to try and figure out effects of climate on different aspects of the global economy using real world data and rigorous econometrics. Together with the other G-FEEDers and a bunch of other folks from around the world, we set out to try and figure out what the climate has been doing and will do to people around the planet. (Regular readers know this.)

Friday, Tamma Carleton and I published a paper in Science trying to bring this last decade of work all together in one place. We have learned a lot, both methodologically and substantively. There is still a massive amount of work to do, but it seemed like a good idea to try and consolidate and synthesize what we’ve learned at least once a decade…

Here’s one of the figures showing some of the things we’ve learned from data across different contexts, it’s kind of like a 2.0 version of the page from Stern above:

click to enlarge

Bringing all of this material together into one place led to a few insights. First, there are pretty clear patterns across sectors where adaptation appears to either be very successful (e.g. heat related mortality or direct losses from cyclones) or surprisingly absent (e.g. temperature losses for maize, GDP-productivity losses to heat). In the latter case, there seem to be “adaptation gaps” that are persistent across time and locations, something that we might not expect if adaptation is costless in the long run (as many people seem to think). We can’t say exactly what’s going on that’s causing these adaptation gaps to persist, for example, it might be that all actors are behaving optimally and this is simply the best we can do with current technology and institutions, or alternatively there might be market failures (such as credit constraints) or other disincentives (like subsidized crop insurance) that prevent individuals from adapting. Figuring out (i) whether current adaptation is efficient, or (ii) if it isn’t, what’s wrong so we can fix it, is a multi-trillion-dollar question and the area where we argue researchers should focus attention.

Eliminating adaptation gaps will have a big payoff today and in the future. To show this, we compute the total economic burden borne by societies today because they are not perfectly adapted today. Even before one accounts for climate change, our baseline climate appears to be a major factor determining human wellbeing around the world.

For example, we compute that on average the current climate

- depresses US maize yields by 48%
- increases US mortality rates 11%
- increases US energy use by 29%
- increases US sexual assault rates 6%
- increases the incidence of civil conflict 29% in Sub-Saharan Africa
- slows global economic growth rates 0.25 percentage points annually

These are all computed by estimating a counterfactual where climate conditions at each location are whatever historically observed values at that location are most ideal.

Our first reaction to some of these numbers were that they were too big. But then we reflected on the numbers more and realized maybe they are pretty reasonable. If we could grow all of US maize in greenhouses where we control the temperature, would our yields really be 48% higher? That’s not actually too crazy if you think about cases where we have insulated living organisms much more from their environment and they do a whole lot better because of it. For example, life expectancy for people has more than double in the last few centuries as we started to protect ourselves from all the little health insults that use to plague people. For similar reasons, if you just look at pet cats in the US, indoor cats live about twice as long as more exposed outdoor cats on average (well, at least according to Petco and our vet). Similarly, lot of little climate insults, each seemingly small, can add up to generate substantial costs—and they apparently do already.

We then compared these numbers (on the current effect of the current climate) to (i) the effects of climate change that has occurred already (a la David and Wolfram) and (ii) projected effects of climate change.

When it comes to effects of climate change to date, these numbers are mostly small. The one exception is that warming should have already increased the integrated incidence of civil conflict  since 1980 by >11%.

When it comes to future climate change, that we haven’t experienced yet, the numbers are generally large and similar-ish in magnitude to the current effect of the current climate. For example, we calculate that future climate change should slow global growth by an additional 0.28 percentage points per year, which is pretty close in magnitude to the 0.25 percentage points per year that temperatures are already slowing things down. For energy demand in the US, current temperatures area actually doing more work today (+29%) than the additional effect of future warming (+11%), whereas for war in Sub-Saharan Africa, current temperatures are doing less (+29%) than near term warming (+54%).

All these numbers are organized in a big table in the paper, since I always love a big table. There's also a bit of history and summary of methods in there as well, for those of you who, like Marshall, don't want to bother slogging through the sister article detailing all the methods.

Conflict in a changing climate: Adaptation, Projection, and Adaptive Projections (Guest post by Tamma Carleton)

Thanks in large part to the authors of G-FEED, our knowledge of the link between climatic variables and rates of crime and conflict is extensive (e.g. see here and here). The wide-ranging effects of high temperatures on both interpersonal crimes and large-scale intergroup conflict have been carefully documented, and we’ve seen that precipitation shortfalls or surpluses can upend social stability in locations heavily dependent on agriculture. Variation in the El Niño cycle accounts for a significant share of global civil conflict, and typhoons lead to higher risks of property crime.

Much of the research in this area is motivated by the threat of anthropogenic climate change and the desire to incorporate empirically derived social damages of climate into policy tools, like integrated assessment models. Papers either explicitly make projections of conflict under various warming scenarios (e.g. Burke et al., 2009), or simply refer to future climate change as an impetus for study. While it’s valuable from a risk management perspective to understand and predict the effects that short-run climate variation may have on crime and conflict outcomes today, magnitudes of predicted climate changes and the lack of clear progress in climate policy generally place future projections at the heart of this body of work.

However, projections are also where much of the criticism of climate-conflict research lies. For example, see Frequently Heard Criticisms #1 and #6 in this post by Marshall. These criticisms are often reasonable, and important (unlike some other critiques you can read about in that same post). The degree to which impacts identified off of short-run climate variation can effectively predict future effects of long-run gradual climate change involves a lot of inherent (statistical and climatological) uncertainty and depends critically on likely rates of adaptation. Because the literature on adaptation is minimal and relies mostly on cross-sectional comparisons, we are limited in our ability to integrate findings into climate policy, which is often the motive for conducting research in the first place. 

Recently, Sol, Marshall and I published (yet another) review of the climate and conflict literature, this time with an emphasis on adaptation and projection, to try to address the concerns discussed above. I’m going to skip the content you all know from their detailed and impressive previous reviews, and talk about the new points we chose to focus on.

Do guns kill people or do people kill people? An economist's perspective

Marshall and I think a lot about the economics of violence, so when the debate about gun control started on House of Cards, I started considering the classic,

"Guns don't kill people, people kill people." 

I mulled over this a bit and got in a debate with my wife since it didn't seem immediately obvious to me what this statement meant and whether it was testable.

I think the economist's take on the statement is that guns are a "technology" used to "produce" murder. This framing made it easier for me to think about what people meant in a way that had some testable predictions.

If "people kill people," that means there are other technologies out there that are pretty similar to guns and can easily be used to produce murder. In econospeak: guns have substitutes. This could happen for two reasons. First, either there are other technologies that produce murder at similar cost, where costs include both the psychological burden of committing murder and the convenience of gun technology, in addition to actual dollar costs. Alternatively, there might be technologies out that that are much more costly than guns (think: committing murder with a knife probably has different psychological costs), but the "demand" for murder is so high that people are willing to use those much more costly technologies to get the job done, i.e. demand for murder is inelastic. If this is true, then raising the cost of gun technology (e.g. strengthening gun control measures) won't save lives since people will be so motivated to produce murder they'll just use the alternative technologies, regardless of whether they they have to pay a higher cost of doing so.

If "guns kill people," I think that means gun technology is so much better than the next closest substitute that simply the presence of guns affects the likelihood that murder is produced. In order for this to happen, it seems you need two conditions to hold. (1) People would have to be willing to commit murder if they can use a gun, but not if they can't (demand for murder would have to elastic) and (2) gun technology must substantially lower the cost of committing murder relative to the closest substitute (seems likely to me). If these conditions are true, then one could effectively reduce the total production of murder by raising the cost of using gun technology, forcing people to use costly alternative technologies. If demand for murder is elastic, then some marginal would-be-murderers will find it's not worth the trouble and lives will be saved.

This led me to an alternative framing of the original statement, which seemed much more tractable to me:

"Are guns and knives substitutes?"

My curiosity piqued, I stayed up late hunting down some data to see if there were any obvious patterns.

I found the FBI Uniform Crime Reports, which tabulate homicides by weapon used for each state in Table 20. I created a state-by-year panel for 2005-2014, converting homicide counts into rates with census data, which goes up until 2010. I needed a measure of the "cost" of gun technology, and after a little hunting around (there is amazingly little data on gun-related anything) found Okoro et al (Pediatrics, 2005) which estimated the fraction of homes with a loaded firearm present using a randomized survey.  Obviously, there is a lot of other stuff that goes into making gun use costly, but it seemed to me that having a loaded gun in the house would dramatically lower the cost of using the gun for murder compared to the alternative situation where a gun had to be located and obtained prior to the act. The data set I put together is posted here in case you want to mess around with it.  (I imagine an enterprising graduate student can analyze the time series variation in this data, but I ended up collapsing this to a cross-section for simplicity.)

The first thing I did was to plot the homicide rate (where a gun was the weapon used) for each state against the fraction of homes with a loaded weapon present. Maybe someone has done this before, but I was struck by the plot:

The slope of the fitted line is 0.18, implying that a 1% increase in homes with a loaded firearm is associated with and average 0.18 additional annual gun murders per 100,000 state residents. This might not be a causal relationship, it's just a correlation. But the the idea that "guns don't kill people, people kill people" has the testable prediction that "guns and knives are substitutes." If this is true, then we would expect that in the states where guns are less accessible, people simply commit murder with other weapons. If we assume for the moment that people in all different states have similar demand for murder, then the substitutability of guns and other weapons would lead us to expect to see this:

From the archives: Friends don't let friends add constants before squaring

I was rooting around in my hard-drive for a review article when I tripped over this old comment that Marshall, Ted and I drafted a while back.

While working on our 2013 climate meta-analysis, we ran across an interesting article by Ole Thiesen at PRIO where he coded up all sorts of violence at a highly local level in Kenya to investigate whether local climatic events, like rainfall and temperature anomalies, appeared to be affecting conflict. Thiesen was estimating a model analogous to:

and reported finding no effect of either temperature or rainfall. I was looking through the replication code of the paper to check the structure of the fixed effects being used when I noticed something, the  squared terms for temperature and rainfall were offset by a constant so that the minimum of the squared terms did not occur at zero:



(Thiesen was using standardize temperature and rainfall measures, so they were both centered at zero). This offset was not apparent in the linear terms of these variables, which got us thinking about whether this matters. Often, when working with linear models, we get used to shifting variables around by a constant, usually out of convenience, and it doesn't matter much. But in non-linear models, adding a constant incorrectly can be dangerous.

After some scratching pen on paper, we realized that

for the squared term in temperature (C is a constant), which when squared gives:

because this constant was not added to the linear terms in the model, the actual regression Thiesen was running is:

which can be converted to the earlier intended equation by computing linear combinations of the regression coefficients (as indicated by the underbraces), but directly interpreting the beta-tilde coefficients as the linear and squared effects is not right--except for beta-tilde_2 which is unchanged. Weird, huh? If you add a constant prior squaring for only the measure that is squared, then the coefficient for that term is fine, but it messes up all the other coefficients in the model.  This didn't seem intuitive to us, which is part of why we drafted up the note.

To check this theory, we swapped out the T-tilde-squared measures for the correct T-squared measures and re-estimated the model in Theisen's original analysis. As predicted, the squared coefficients don't change, but the linear effects do:

This matters substantively, since the linear effect of temperature had appeared to be insignificant in the original analysis, leading Thiesen to conclude that Marshall and Ted might have drawn incorrect conclusions in their 2009 paper finding temperature affected conflict in Africa. But just removing the offending constant term revealed a large positive and significant linear effect of temperature in this new high resolution data set, agreeing with the earlier work. It turns out that if you compute the correct linear combination of coefficients from Thiesen's original regression (stuff above the brace for beta_1 above), you actually see the correct marginal effect of temperature (and it is significant).

The error was not at all obvious to us originally, and we guess that lots of folks make similar errors without realizing it. In particular, it's easy to show that a similar effect shows up if you estimate interaction effects incorrectly (after all, temperature-squared is just an interaction with itself).

Thiesen's construction of this new data set is an important contribution, and when we emailed this point to him he was very gracious in acknowledging the mistake. This comment didn't get seen widely because when we submitted it to the original journal that published the article, we received an email back from the editor stating that the "Journal of Peace Research does not publish research notes or commentaries."

This holiday season, don't let your friends drink and drive or add constants the wrong way in nonlinear models.

What we know about climate change, conflict, and terrorism

Ever since Bernie Sanders' remarks about climate change causing terrorism, a lot of folks have been asking about what we know on this and related issues. I worked with Marshall and Tamma Carleton to put together this short brief for those interested in knowing what we know quickly. (For those looking for a long answer, see here.)

Summary points:

1. Research clearly demonstrates that hotter temperatures cause more individual level violence (e.g. homicides in the US) and more large-scale violence (e.g. civil wars in Africa), and that extreme rainfall leads to violence in agrarian contexts.
2. Climate change to date, via warmer temperatures, has likely increased the risk of conflict, although this has not yet been empirically proven.
3. Attributing the Syrian conflict to climate change is difficult.  What we can say is that drought and hot temperatures increase the likelihood of these types of conflict.
4. There is currently little evidence for or against a systematic relationship between climate and terrorism. 

El Niño and Global Inequality (guest post by Kyle Meng)

El Niño is here, and in a big way. Recent sea-surface temperatures in the tropical Pacific Ocean, our main indicator of El Niño intensity, is about as high as they were prior to the winter of 1997/98, our last major El Niño (and one of the biggest in recorded history). Going forward, median climate forecasts suggest this intensity will be sustained over the coming months, and could even end up stronger than the 1997/98 El Niño. This will have important global consequences over the next 12 months not just on where food is produced but also how it is traded around the planet.

First, a quick primer. The El Niño Southern Oscillation (ENSO) is a naturally occurring climatic phenomenon, arguably the most important driver of global annual climate variability. It is characterized by two extreme states: El Niño and La Niña. Warm water piles up along the western tropical Pacific during La Niña. During El Niño, the atmospheric and oceanographic forces that maintain this pool of warm water collapse resulting in a large release of heat into the atmosphere that is propagated around the planet within a relatively short period. ENSO’s impact on local environmental conditions around the planet are known as its “teleconnections.” To a rough first order, El Niño makes much of the tropics (from 30N to 30S latitude, shown in red in figure below) hotter and dryer and the temperate regions (shown in blue in figure below) cooler and wetter.

Countries where the majority of the population experience hotter conditions under El Niño are shown in red. Countries that get cooler under El Niño are shown in blue (reproduced from Hsiang and Meng, American Economic Review, May 2015).

There are two features of El Niño that have important implications on global food markets. First, El Niño creates winners and losers across the planet. Sol Hsiang and I have shown in a paper published in the American Economic Review Papers and Proceedings, that between 1960-2010, country-level cereal output in the tropics drop on average by 3.5% for every degree increase in the winter ENSO index. For a large event like 1997/98, or the one anticipated this winter, we estimate a 7% decrease in cereal output across the tropics. Conversely, the relatively more favorable environmental conditions experienced by temperate countries during the same year results in a 5% increase in cereal output. Interestingly, if you sum up the gains and losses across the world, you end up with a positive number: El Niño actually increases global cereal output. 

Second, El Niño impacts are highly spatially correlated, organizing winners and losers to roughly two spatially contiguous blocks across the planet: temperate and tropical countries (which are also mostly just countries south of 30N because there are few countries south of 30S). This means under El Niño, countries suffering crop failures deep in the tropics are also surrounded by neighbors that are likely experiencing similar food shortages at the same time. Why is the spatial scale of El Niño impacts important? Basic economics tells us that the primary driver for international trade is productivity differences across countries. When El Niño occurs, tropical neighbors that normally engage in bilateral trade experience similar crop losses and thus may be less likely to trade with each other. To find an exporter that experiences bumper yields under El Niño, tropical countries have to source food from temperate countries (i.e. North America, Europe, North Asia), that are much further away, for which the cost of trade is higher. The predicted result is that El Niño has two effects on countries in the tropics: it causes direct crop losses and limits the ability of imports to offset such losses. 

Is this happening? In ongoing work with Sol and Jonathan Dingel, we detect exactly these trade effects. From 1960-2010, when an El Niño occurs, cereal output fall in the tropics with some extra imports arriving. However, these imports do not offset all losses such that countries deep in the tropics experience large spikes in food prices. Stay tuned for that paper.

We think this is important beyond food prices during El Niño. In an article published in Nature in 2011, Sol and I, together with Mark Cane, detected that the likelihood of civil wars breaking out in the tropics doubles during strong El Niño years relative to strong La Niña years. Many have asked us about the mechanisms behind this large effect. We now think that direct crop losses together with the limits of trade during El Niño are important parts of the story.

What can be done? Sol and I recently wrote an op-ed in the Guardian on El Niño and its impact on global equality with some policy prescriptions. In the short-term, we argue that aid agencies, peacekeeping groups, refugee organizations, and other international institutions should be prepared to send food to the tropics as local conditions deteriorate. We also argue, in the long-term, that investments should be made to better integrate global food markets and improved access to other financial instruments such as like crop insurance. 

Finally, the spatial nature of El Niño events has similarities with that of anthropogenic climate change, which we know from Marshall, Sol, and Ted’s work is expected to generate winners and losers across the planet. As such adaptation to climate change will involve not just local investments but also global efforts to improve how markets redistribute the unequal effects of climate change.

This is a guest post by Kyle Meng, an Assistant Professor at UC Santa Barbara.

Fanning the flames, unnecessarily

This post is a convolution of David's post earlier today and Sol's from a few days ago.  Our debate with Buhaug et al that Sol blogged about has dragged on for a while now, and engendered a range of press coverage, the most recent by a news reporter at Science.  "A debate among scientists over climate change and conflict has turned ugly", the article begins, and goes on to catalog the complaints on either side while doing little to engage with the content.

Perhaps it should be no surprise that press outlets prefer to highlight the mudslinging, but this sort of coverage is not really helpful.  And what was lost in this particular coverage were the many things I think we've actually learned in the protracted debate with Buhaug.

We've been having an ongoing email dialog with ur-blogger and statistician Andrew Gelman, who often takes it upon himself to clarify or adjudicate these sorts of public statistical debates, which is a real public service.  Gelman writes in an email:

In short, one might say that you and Buhang are disagreeing on who has the burden of proof.  From your perspective, you did a reasonable analysis which holds up under reasonable perturbations and you feel it should stand, unless a critic can show that any proposed alternative data inclusion or data analytic choices make a real difference.  From their perspective, you did an analysis with a lot of questionable choices and it’s not worth taking your analysis seriously until all these specifics are resolved.

I'm sympathetic with this summary, and am actually quite sympathetic to Buhaug and colleagues' concern about our variable selection in our original Science article.  Researchers have a lot of choice over how and where to focus their analysis, which is a particular issue in our meta-analysis since there are multiple climate variables to choose from and multiple ways to operationalize them.  Therefore it could be that our original effort to bring each researcher's "preferred" specification into our meta-analysis might have doubly amplified any publication bias -- with researchers of the individual studies we reviewed emphasizing the few significant results, and Sol, Ted, and I picking the most significant one out of those.  Or perhaps the other researchers are not to blame and the problem could have just been with Sol, Ted, and my choices about what to focus on.

One effect to rule them all? Our reply to Buhaug et al's climate and conflict commentary

For many years there has been a heated debate about the empirical link between climate and conflict. A year ago, Marshall, Ted Miguel and I published a paper in Science where we reviewed existing quantitative research on this question, reanalyzed numerous studies, synthesized results into generalizable concepts, and conducted a meta-analysis of parameter estimates (watch Ted's TED talk).  At the time, many researchers laid out criticism in the press and blogosphere, which Marshall fielded through G-FEED. In March, Halvard Buhaug posted a comment signed by 26 authors on his website strongly critiquing our analysis, essentially claiming that they had overturned our analysis by replicating it using an unbiased selection of studies and variables. At the time, I explained numerous errors in the comment here on G-FEED. 

The comment by Buhaug et al. was published today in Climatic Change as a commentary (version here), essentially unchanged from the version posted earlier with none of the errors I pointed out addressed. 

You can read our reply to Buhaug et al. here. If you don't want to bother with lengthy details, our abstract is short and direct:

Abstract: A comment by Buhaug et al. attributes disagreement between our recent analyses and their review articles to biased decisions in our meta-analysis and a difference of opinion regarding statistical approaches. The claim is false. Buhaug et al.’s alteration of our meta-analysis misrepresents findings in the literature, makes statistical errors, misclassifies multiple studies, makes coding errors, and suppresses the display of results that are consistent with our original analysis. We correct these mistakes and obtain findings in line with our original results, even when we use the study selection criteria proposed by Buhaug et al. We conclude that there is no evidence in the data supporting the claims raised in Buhaug et al.

When evidence does not suffice

Halvard Buhaug and numerous coauthors have released a comment titled “One effect to rule them all? A comment on climate and conflict” which critiques research on climate and human conflict that I published in Science and Climatic Change with my coauthors Marshall Burke and Edward Miguel. 

The comment does not address the actual content of our papers.  Instead it states that our papers say things they do not say (or that our papers do not say thing they actually do say) and then uses those inaccurate claims as evidence that our work is erroneous.

Below the fold is my reaction to the comment, written as the referee report that I would write if I were asked to referee the comment.

(This is not the first time Buhaug and I have disagreed on what constitutes evidence. Kyle Meng and I recently published a paper in PNAS demonstrating that Buhaug’s 2010 critique of an earlier paper made aggressive claims that the earlier paper was wrong without actually providing evidence to support those claims.)

Violence is expensive

We've blogged before (ad nauseam?) about our ongoing research that suggests that changes in climate could substantially affect patterns of human violence.  Imagine, for a moment, that you buy these results.  A natural question is, how much should we care?

One way to answer this question it to try to calculate the added economic cost of a climate-induced change in conflict.  E.g., if temperatures were to rise 1 degree, what would be the economic cost of the ensuing increase in conflict?  Expressing the cost in dollars then allows us to compare it against other things we spend money on to give a sense of how "large" the costs of climate-induced violence might be.  And to the extent that we actually think future changes in climate could increase conflict risk, such a calculation could also inform estimates of the "social costs of carbon" -- essentially, the overall cost of emitting one more ton of CO2 today.  

Clearly it is not easy to calculate how much a climate-induced change in conflict would cost. It's going to be some combination of the economic damage wrought by different types of conflict, and the increase in each type of conflict due to a change in climate.   Our paper provides some estimates of the latter, but figuring out the former seems like a real bear.

This is why it was very interesting to see a new report entitled "The Economic Cost of Violence Containment", put out by a group called the Institute for Economics and Peace [ht: Tom Murphy].  As the title suggests, the goal of this report is to calculate the aggregate economic costs of violence and what we spend to contain it.   Violence, they calculate, is very very expensive.  Their headline number is that we spend about $10 trillion a year in "violence containment", which they define as "economic activity that is related to the consequences or prevention of violence… directed against people or property." For those scoring at home, $10 trillion is about 10% of the total value of stuff the world produces (the so-called Gross World Product).  For those of you who think only in Benjamins, it's 100 billion of them. 

Their estimate is the result of a big adding-up exercise where they use existing estimates from the literature on how much each type of violence costs directly -- from what we spend to house and feed conflict refugees, to the economic cost of a homicide -- and add to them estimates of how much we spend to "contain" violence more broadly, which for them includes all military expenditure (more on that below).  The figure below shows their assessment of breakdown of the different costs, with the shares given by the exploding pie chart (yeah!) on the left, and the absolute values on the right.  

There are clearly some questions about what the authors have decided to include and not include.  For instance, it doesn't seem like military expenditure can just be thought of as a cost. Sure, with a fixed budget, increased expenditure on the military necessarily reduces the investments we could otherwise make, many of which could be higher return.  But this doesn't mean that investment in the military has no economic benefit.  Irrespective of one's feelings about military spending, the military employs a lot of people, both directly and indirectly, so this sort of expenditure has benefits as well as costs.

Then as you can see in the table on the right, to get from $5 trillion in direct costs to $10 trillion in total costs, they literally multiply the direct costs by 2.  The claim is that the economic spillover from reduced violence -- e.g. from investments made elsewhere with the money you save, and from people no longer having to protect themselves from violence -- is as large as the direct cost of violence.  This number seems to come out of nowhere.

Nevertheless, if we that military expenditure is actually a wash in terms of total costs, and assume that there is no multiplier effect, we still have a cost of violence estimate of $2.3 trillion dollars.  If you drop out the "private security" and "internal security" categories for similar reasons (e.g. they are employing people), you are down closer to $1.3 trillion dollars.  So call it $1 trillion dollars in annual costs of violence.  I.e., to be conservative, let's assume that the report was off by an order of magnitude.

So what of the economic costs of climate-induced increase in conflict?  For a quick back-of-the-envelope, we can combine this $1 trillion estimate with our earlier estimates on how conflict risk responds to increases in temperature.  We had calculated these latter estimates as standardized effects -- i.e. a percentage change in conflict per 1 standard deviation change in temperature or precipitation -- and came up with numbers between about 4% and 14%, depending on the type of violence.  And since we're not used to thinking of temperature changes in terms of standard deviations, we made the map below (Fig 6 in our Science paper) to show the projected change in temperature between 2000 and 2050, expressed as multiples of the historical temperature standard deviation at each location.

So putting this all together:  $1 trillion annual cost of violence, say a 5% increase in violence for every SD increase in temperature to be conservative, and say a 2SD increase in temperature by 2050 (most populated regions are higher than that, as shown in the figure).  Under the assumption that future societies will respond to temperature increases as societies have in the past, then this would give us a $100 billion increase in the annual cost of violence by 2050.  Assuming a linear temperature increase between now and 2050 (and assuming effects stop after 2050), and setting the discount rate at 2%, you can calculate the present value of a future increase in climate-induced conflict by adding up the effects in each year and discounting them back to the present.

The number I get is just over $1.5 trillion, or a little over 1% of current Gross World Product. That is a large number.  As a simple calibration, it is about 1/5th of the total cost of climate change calculated in the Stern Review a few years back (which did not consider costs from violence).

Clearly there are a ton of assumptions that go into these sorts of calculations, but these order-of-magnitude exercises can be useful for getting a basic answer to the "should we care" question.  And even if this new report is off by an order of magnitude about the costs of violence and conflict, I think the answer to whether we should care about the potential costs of climate-induced violence is a simple "Yes".  These are big numbers.

A climate for conflict

Sol, Ted Miguel, and I are excited to have a paper we've been working on for a while finally come out. The title is "Quantifying the impact of climate on human conflict", and it's out in this week's issue of Science.  Seems like this should get Sol's tenure package at Berkeley off to a pretty nice start.  

Our goal in the paper is rather modest, and it's all in the title:  we collect estimates from the rapidly growing number of studies that look at the relationship between some climate variable and some conflict outcome, make sure they are being generated by a common and acceptable methodology (one that can credibly estimate causal effects), and then see how similar these estimates are.  We want to understand whether all these studies spread across different disciplines are actually telling us very different things, as the ongoing debate on climate and conflict might seem to suggest.

We compute things in terms of standardized effects -- percent change in conflict per standard deviation change in climate -- and find that, by and large, estimates across the studies are not all that different.  All the exciting/tedious details are in the paper and the supplement (here's a link to the paper for those without access, and here is a link to all the replication data and code).  We've also gotten some nice (and not so nice) press today that discusses different aspects of what we've done.  See here, here, here, here, here, here, here, here, here, and here for a sampler.  Most importantly, see here. 

We want to use this space to answer some FAQ about the study.  Or, really, some FHC:  Frequently Heard Criticisms.  A bunch of these have popped up already in the press coverage.  Some are quite reasonable, some (we believe) stem from a misunderstanding or misrepresentation of what we're trying to do in the paper, and some are patently false. 

So, in no particular order, some FHC we've gotten (in bold), and our responses.  We include direct quotes from our critics where possible.  Apologies for the lengthy post, but we are trying to get it all out there. 

UPDATE (Friday Aug 2, 4pm): We have now heard from 3 different folks - Andy Solow, Idean Salehyan, and Cullen Hendrix - who each noted that what they were quoted as saying in these articles wasn't the full story, and that they had said lots of nice things too.  After talking to them, it seems there is much more agreement on many of these issues than the press articles would have you believe, and in many cases it looks like the journalists went far out of their way to highlight points of disagreement. I guess this shouldn't be surprising, but we believe it does the debate a pretty serious disservice.  So our responses below should be interpreted as responses to quotes as they appeared, not necessarily responses to particular individuals' viewpoints on these issues, which the quotes might not adequately represent.  We are not trying to pick fights, just to clarify what our paper does and doesn't do.

1.  You confuse weather and climate. 
(verbatim from Richard Tol, courtesy Google Translate)

This is an old saw that you always get with these sorts of papers.  The implied concern is that most of the historical relationships are estimates using short-run variation in temperature and precip (typically termed "weather"), and then these are used to say something about future, longer-run changes in the same variables ("climate").  So the worry is that people might respond to future changes in climate -- and in particular, slower-moving changes in average temperature or precip -- differently than they have to past short-run variation.

This is a very sensible concern.  However, there are a couple reasons why we think our paper is okay on this front. First, we document in the paper hat the relationship between climate variables and conflict shows up at a variety of time scales, from hourly changes in temperature to century-scale changes in temperature and rainfall.  We use the word "climate" in the paper to refer to this range of fluctuations, and we find similar responses across this range, which provides some evidence that today's societies are not that much better at dealing with long-run changes than short-run changes.  This is consistent with evidence on climate effects in other related domains (e.g. agriculture).  

Second, we do not to make explicit projections about future impacts - our focus is on the similarity in historical responses.  Nevertheless, the reader is definitely given the tools to do their own back-of-the-envelope projections:  e.g. we provide effects in terms of standard deviations of temperature/ rainfall, and then provide a map of the change in temperature by 2050 in terms of standard deviations (which are really really large!).  That way, the reader can assume whatever they want about how historical responses map into future responses.  If you think people will respond in the future just like they did in the past, it's easy multiplication.  If you think they'll only be half as sensitive, multiply the effect size by 0.5, etc etc.  People can adopt whatever view they like on how future long-run responses might differ from past short-run responses; our paper does not take a stand on that. 

2. Your criterion for study inclusion are inconsistently applied, and you throw out studies that disagree with your main finding.
(Paraphrasing Halvard Buhaug and Jurgen Sheffran)

This one is just not true.  We define an inclusion criterion -- which mainly boils down to studies using standard techniques to account for unobserved factors that could be correlated with both climate and conflict --  and include every study that we could find that meets this criterion. In more technical terms, these are panel or longitudinal studies that include fixed effects to account for time-invariant or time-varying omitted variables.

In a few cases, there were multiple studies that analyzed the exact same dataset and outcomes, and in those cases we included either the study that did it first (if the studies were indistinguishable in their methods), or the study that did the analysis correctly (if, as was sometimes the case, one study met the inclusion criteria and another did not).  

So, the inclusion criterion had nothing to do with the findings of the study, and in the paper we highlight estimates from multiple studies whose do not appear to agree with the main findings of the meta-analysis (see Figure 5). We did throw out a lot of studies, and all of these were studies that could not reliably identify causal relationships between climate and conflict -- for instance if they only relied on cross-sectional variation.  We also throw out multiple studies that agreed very strongly with our main findings!  We provide very detailed information on the studies that we did and did not include in Section A of the SOM.

Sheffran refers to other recent reviews, and complains that our paper does not include some papers in those reviews.  This is true, for the methodological reasons just stated.  But what he does not mention is that none of these reviews even attempt to define an inclusion criterion, most of them review a very small percentage of the number of papers we do, and no make any attempt to quantitatively compare results across papers.  This is why our study is a contribution, and presumably why Science was interested in publishing it as a Research Article.

3. You lump together many different types of conflict that shouldn't be lumped together.  Corollary: There is no way that all these types of conflict have the same causal mechanism.  
(Idean Salehyan: "It’s hard to see how the same causal mechanism that would lead to wild pitches would be linked to war and state collapse")

This quote is very substantial misrepresentation of what we do.  First, nowhere do we make the claim that these types of conflict have the same causal mechanism.  In fact we go to great lengths to state that climate affects many different things that might in turn affect conflict, and the fact that the effect sizes are broadly similar across different types of conflict could be explained by climate's pervasive effect on many different potential intervening variables (economic conditions, food prices, institutional factors, ease of transportation, etc). 

Second, we take great pains to separate out conflicts into different categories, and only make comparisons within each category.  So we calculate effect sizes separately for individual-level conflict (things like assault and murder), and group level conflict (things like civil war).  So, again contra Idean's quote, we are not actually comparing baseball violence (which we term individual-level) with war and state collapse (which are group conflicts).  Read the paper, Idean! 

But the whole point of the paper is to ask whether effect sizes across these different types of conflict are in fact similar!  What we had in the literature was a scattering of studies across disciplines, often looking at different types of conflict and using different methodologies.  This disarray had led to understandable confusion about what the literature as a whole was telling us.  Our goal was to put all the studies on the same footing and ask, are these different studies actually telling us that different types of conflict in different settings respond very differently to climate?  Our basic finding is that there is much more similarity across studies than what is typically acknowledged in the debate.  

Whether this similarity is being driven by a common underlying mechanism, or by multiple different mechanisms acting at the same time, is something we do not know the answer to -- and what we highlight very explicitly as THE research priority going forward.  

4. You cherry pick the climate variables that you report
(paraphrasing Halvard Buhaug).

We try really hard not to do this.  Where possible, we focus on the climate variable that the authors focused on in the original study.  However, the authors themselves in these studies are almost completely unrestricted in how they want to parameterize climate.  You can run a linear model, a quadratic model, you can include multiple lags, you can create binary measures, you can create fancy drought measures that combine temperature and rainfall, etc etc.  Authors do all these things, and often do many of them in the same paper.  Since we can't include all estimates from every single paper, we try to pick out the author's preferred measure or estimate, and report that one.  In the cases where authors tested many different permutations and did not hint at their "preferred" estimate (e.g. in Buhaug's comments on our earlier PNAS paper), we pick the median estimate across all the reported estimates.  Section A2 in the SOM provides extra detail on all of these cases.

5. This paper is not based on any theory of conflict, so we learn nothing.

This is very related to FHC #3 above, and we get this from the political science crew a lot.  The thing is, there are a ton of different theories on the causes of conflict, and empirical work so far has not done a great job of sorting them out.  In some sense, we are being atheoretical in the paper -- we just want to understand whether the different estimates are telling us very different things.  As noted above, though, the fact that they're generally not would seem to be very important to people interested in theory! 

Claire Adida, a political science prof at UCSD (and @ClaireAdida on twitter), put it really nicely in an email:  "I don't understand this ostrich politics. How about saying something like 'we really want to thank these economists for doing a ton of work to show how confident we can be that this relationship exists. It's now our turn - as political scientists - to figure out what might be the causal mechanisms underlying this relationship.' " (Btw, I have no idea what "ostrich politics" means, but I really like it!) 

6. People will adapt, so your results are a poor guide for impacts under climate change. 
(paraphrasing Jurgen Sheffran, courtesy Google Translate; Cullen Hendrix: "I'm optimistic.  Unlike glaciers, humans have remarkable adaptive capacity"; as well as audience members in every seminar we've ever given on this topic).

This is very related to FHC #1 above. It is definitely possible that future societies will become much better at dealing with extreme heat and erratic rainfall.  However, to just assume that this is the case seems to us a dangerous misreading of the existing evidence.  As stated above, available evidence suggests that human societies are remarkably bad at dealing with deviations from average climate, be it a short-lived temperature increase or a long-term one.  See here and here for other evidence on this topic.

And it has to be the case that knowing something about how yesterday's and today's world respond to climate tells us more about future impacts than knowing nothing about how societies have responded to climate.  The alternative - that the present world tells us nothing about the future world - just does not appear consistent with how nearly anybody sees things. 

7.  A lot of your examples - e.g. about civilization collapse - do not pertain to the modern world.

We got this in peer review. It's true that a lot of these collapse stories are from way back.  The Akkadian empire collapsed before 2000 BC, after all!  In a companion paper, forthcoming in Climatic Change, Sol and I look a little more carefully at this, and it actually turns out that per capita incomes in many of these societies, pre-collapse, were remarkably similar to incomes in many developing countries today.  To the extent that economic conditions shape conflict outcomes -- a common belief in economics and political science -- then this provides at least some evidence that these historical events are not completely irrelevant to today. 

More basically, though, it seems like hubris to just assume that "this time things are different".  At the time of their collapse, each of these societies (the Akkadians, the Maya, some of the Chinese dynasties, Angkor Wat) were incredibly advanced by global standards, and they probably also did not figure that climate would play a role in their demise.  Because we don't yet have a firm grasp on why climate affects conflict, it again seems dangerous to assume that things are completely different today -- just as it seems dangerous to conclude that modern societies are going to be completely destroyed by climate change, a claim we make nowhere in the paper. 

However, we do hope that "this time is different"!  It would be quite nice if the Mayan and Angkor Wat examples did not, in fact, pertain to the modern world. 

8. You can't claim that there is an overall significant relationship between climate and conflict if many of the studies you analyze do not show a statistically significant effect. 
(Halvard Buhaug: "I struggle to see how the authors can claim a remarkable convergence of quantitative evidence when one-third of their civil conflict models produce a climate effect statistically indistinguishable from zero, and several other models disagree on the direction of a possible climate effect")

This is a basic misunderstanding of what a meta-analysis does.  The beauty of a meta-analysis is that, by pooling a bunch of different studies, you can dramatically increase statistical power by increasing your sample size. It's even possible to find a statistically significant result across many small studies even if no individual study found a significant result.  This happens in medical meta-analyses all the time, and is why they are so popular in that setting:  each individual study of some expensive drug or procedure often only includes a few individuals, and only by combining across studies do you have enough statistical power to figure out what's going on.

So the fact that some of the individual studies were statistically significant, and others were not, does not necessarily affect the conclusions you draw when you average across studies.  In our case, it did not:  the mean across studies can be estimated very precisely, as we show in Figures 4 and 5, and discuss in detail in the SOM.

A final point:  we find a striking consistency in findings in the studies that look at temperature in particular.  Of the 27 modern studies that looked at a relationship between temperature and conflict, all 27 estimated a positive coefficient.  This is extremely unlikely to happen by chance - i.e. very unlikely to happen if there were in fact no underlying relationship between temperature and conflict.  Think of flipping a coin 27 times and getting heads all 27 times.  The chance of that is less than 1 in a million.  This is not a perfect analogy -- coin flips of a fair coin are independent, our studies are not fully independent (e.g. many studies share some of the same data) -- but we show on page 19 in the SOM that even if you assume a very strong dependence across studies, our results are still strongly statistically significant. 

9. Conflict has gone down across the world, as temperatures have risen. This undermines the claims about a positive relationship between temperature and conflict.
(Idean Salehyan: "We've seen rising temperatures, but there's actually been a decline in armed conflict".)

There are a couple things wrong with this one. First, many types of conflict that we look at have not declined at all over time.  Here is a plot of civil conflicts and civil wars since 1960 from the PRIO data, summed across the world.  As coded in these data, civil conflicts are conflicts that result in at least 25 battle deaths (light gray in the plot), and civil wars are those that result in at least 1000 deaths (dark gray).  As you can see, both large wars and smaller conflicts peaked in the mid-1990s, and while the incidence of larger wars have fallen somewhat, the incidence of smaller conflicts is currently almost back up to its 1990s peak.  These types of conflicts are examined by many of the papers we study, and have not declined.   

As another check on this, I downloaded the latest version of the Social Conflict in Africa Dataset, a really nice dataset that Idean himself was instrumental in assembling.  This dataset tracks the incidence of protests, riots, strikes, and other social disturbances in Africa.  Below is a plot of event counts over time in these data.  Again, you'd be very hard pressed to say that this type of conflict has declined either.  So I just don't understand this comment.

Second, and more importantly, there are about a bazillion other things that are also trending over this period.  The popularity of the band New Kids On The Block as also fallen fairly substantially since the 1990s, but no-one is attributing changes in conflict to changes in NKOTB popularity (although maybe this isn't implausible).  The point is that identifying causal effects from these trends is just about impossible, since so many things are trending over time.  

Our study instead focuses on papers that use detrended data - i.e. those that use variation in climate over time in a particular place.  These papers, for instance, compare what happens to conflict in a hot year in a given country, to what happens in a cooler year in that country, after having account for any generic trends in both climate and conflict that might be in the data.  Done this way, you are very unlikely to erroneously attribute the effects of changes in conflict to changes in climate. 


10. You don't provide specific examples of conflicts that were caused by climate.  

(Halvard Buhaug: "Surprisingly, the authors provide no examples of real conflicts that plausibly were affected by climate extremes that could serve to validate their conclusion. For these and other reasons, this study fails to provide new insight into how and under what conditions climate might affect violent conflict")

I do not understand this statement.  We review studies that look at civil conflict in Somalia, studies that look at land invasions in Brazil, studies that look at domestic violence in one city in one year in Australia, studies that look at ethnic violence in India, studies that look at murder in a small set of villages in Tanzania.  The historical studies looking at civilization collapses in particular try to match single events to large contemporaneous shifts in climate.  We highlight these examples in both the paper and in the press materials that we released, and they were included in nearly every news piece on our work that we have seen.  So, again, this comment just does not make sense. 

Perhaps implicit in this claim is often some belief that we are climate determinists.  But as we say explicitly in the paper, we are not arguing that climate is the only factor that affects conflict, nor even that it is the most important factor affecting conflict.  Our contribution is to quantify its role across a whole host of settings, and our findings we hope will help motivate a bunch more research on why climate should shape conflict so dramatically (see Claire's quote above).

11.  You are data mining. Corollary: What you guys are demonstrating is a severe publication bias problem -- only studies that show a certain result get published.
(Andy Solow: "In the aggregate, if you work the data very hard, you do find relationships like this. But when you take a closer look, things tend to be more complicated." As an aside, Andy sent us a very nice email, noting in reference to the press coverage of our article: "From what I've seen so far, all the nice things I said - that you are smart, serious researchers working on an important and difficult problem, that your paper will contribute to the discussion, that you may well be right - have been lost in favor of concerns I have and that, as I took pains to point out, you are already aware of.")

This is related to FHC #2 and #4 above. We have defined a clear inclusion criterion, and only include studies that meet this criterion.  As detailed in the SOM Section A2, we do not include a number of studies that agree very strongly with our main findings - for instance Melissa Dell's very nice paper on the Mexican Revolution.  Again, our inclusion criteria is based on methodology, not findings. 

The publication bias issue is a tougher one, and one which we explicitly address in the paper -- it even gets its own section, so it's not something we're trying to hide from.  We test formally for it in the SOM (Section C), finding limited evidence that publication bias is behind our results.  We also note that it's not clear where the professional incentives now lie in terms of the sorts of results that are likely to get published or noticed.  The handful of climate/conflict skeptics have garnered a lot of press by very publicly disagreeing with our findings, and this has presumably been good for their careers.  Had they instead published papers that agreed with our findings, it's likely that the press would not have had these folks as their go-to this time around.  Similarly, journals are probably becoming less interested in publishing yet another paper that shows that higher temperatures lead to more conflict.  Because so many of the papers we review are really recent (the median publication date across our studies was 2011), we feel that it is unlikely that all of these results are false positives.