Let there be light? Estimating the impact of geoengineering on crop productivity using volcanic eruptions as natural experiments (Guest post by Jonathan Proctor)

[This is a guest post by Jonathan Proctor, a Doctoral Fellow at the Global Policy Lab and PhD candidate in the Ag and Resource Econ department here at Berkeley]

On Wednesday I, and some notorious G-FEEDers, published a paper in Nature exploring whether solar geoengineering – a proposed technology cool the Earth by reflecting sunlight back into space—might be able to mitigate climate-change damages to agricultural production. We find that, as intended and previously described, the cooling from geoengineering benefits crop yields. We also find, however, that the shading from solar geoengineering makes crops less productive. On net, the damages from reduced sunlight wash out the benefits from cooling, meaning that solar geoengineering is unlikely to be an effective tool to mitigate the damages that climate change poses to global agricultural production and food security. Put another way, if we imagine SRM as an experimental surgery, our findings suggest that the side effects are as bad as the cure.

Zooming out, solar geoengineering is an idea to cool the earth by injecting reflective particles --usually precursors to sulfate aerosols -- into the high atmosphere. The idea is that these particles would bounce sunlight back into space and thus cool the Earth, similarly to how you might cool yourself down by standing in the shade of a tree during a hot day. The idea of such sulfate-based climate engineering was, in part, inspired by the observation that the Earth tends to cool following massive volcanic eruptions such as that of Pinatubo in 1991, which cooled the earth by about half a degree C in the years following the eruption.

Our visualization of the stratospheric aerosols (blue) that scattered light and shaded the planet after the eruption of Mount Pinatubo in 1991. Each frame is one month of data. Green on the surface indicates global crop lands. (The distance between the aerosol cloud and the surface is much larger than in real life.) 

A major challenge in learning the consequences of solar geoengineering is that we can’t do a planetary-scale experiment without actually deploying the technology. (Sol’s questionably-appropriate analogy is that you can’t figure out if you want to be a parent through experimentation.) An innovation here was realizing that we could learn about the impacts of solar geoengineering without incurring the risks of an outdoor experiment by using giant volcanic eruptions as natural experiments. While these eruptions are not perfect proxies for solar geoengineering in every way, they give us the necessary variation we need in high atmosphere aerosol concentrations to study some of the key effects on agriculture. (We expand on how we account for the important differences between the impacts of volcanic eruptions and solar geoengineering on agricultural production later in this post). This approach builds on previous work in the earth science community which has used the eruptions to study solar geoengineering’s impact on climate.  Here’s what we found:

Result 1: Pinatubo dims the lights

First, we find that the aerosols from Pinatubo had a profound impact on the global optical environment. By combing remotely sensed data on the eruption’s aerosol cloud with globally-dispersed ground sensors of solar radiation (scraped from a Russian website that recommends visitors use Netscape Navigator) we estimate that the Pinatubo eruption temporarily decreased global surface solar radiation (orange) by 2.5%, reduced direct (i.e. unscattered, shown in yellow) insolation by 20% and increased diffuse (i.e. scattered, shown in red) sunlight by 20%.

Effect of El Chichon (1982) and Mt Pinatubo (1991) on direct (yellow), diffuse (red) and total (orange) insolation for global all-sky conditions.

These global all-sky results (i.e. the average effect on a given day) generalize previous clear-sky estimates (the effect on a clear day) that have been done at individual stations. Like a softbox or diffusing sheet in photography, this increase in diffuse light reduced shadows on a global scale. The aerosol-scattering also made redder sunsets (sulfate aerosols cause a spectral shift in addition to a diffusion of light), similar to the volcanic sunsets that inspired Edvard Munch’s “The Scream.” Portraits and paintings aside, we wanted to know: how did these changes in sunlight impact global agricultural production?

Isolating the effect of changes in sunlight, however, was a challenge. First, the aerosols emitted by the eruption alter not only sunlight, but also other climatic variables such as temperature and precipitation, which impact yield. Second, there just so happened to be El Nino events that coincided with the eruptions of both El Chichon and Pinatubo. This unfortunate coincidence has frustrated the atmospheric science community for decades, leading some to suggest that volcanic eruptions might even cause El Niños, as well as the reverse (the former theory seems to have more evidence behind it).

To address the concern that volcanoes affect both light and other climatic conditions, we used a simple “condition on observables” design – by measuring and including potential confounds (such as temperature, precipitation and cloud cover) in the regression we can account for their effects. To address the concurrent El Nino, we do two things. First, we directly condition on the variables though which an El Nino could impact yields – again temperature, precipitation and cloud cover. Second, we condition on the El Nino index itself, which captures any effects that operate outside of these directly modeled channels. Essentially, we isolate the insolation effect by partitioning out the variation due to everything else – like looking for your keys by pulling everything else out of your purse.

The above figure schematically illustrates our strategy. The total effect (blue) is the sum of optical (red) and climatic components (green). By accounting for the change in yields due to the non-optical factors, we isolate the variation in yields due to stratospheric aerosol-induced changes in sunlight.

Result 2: Dimming the lights decreases yields

Our second result, and the main scientific contribution, is the finding that radiative scattering from stratospheric sulfate aerosols decreases yields on net, holding other variables like temperature constant. The magnitude of this impact is substantial – the global average scattering from Pinatubo reduced C4 (maize) yields by 9.3% and C3 (soy, rice and wheat) yields by 4.8%, which is two to three times larger than the change in total sunlight. We reconstruct this effect for each country in the figure below:

Each line represents one crop for one country. These are the reconstructed yield losses due to the estimated direct optical effects of overhead aerosols.

My young cousins dismissed the sign of this effect as obvious – after all plants need sunlight to grow. But, surprising to my young cousins, the prevailing wisdom in the literature tended to be that scattering light should increase crop growth (Sol incessantly points out that David even said this once). The argument is that the reduction in yields from loss of total light would be more than offset by gains in yield through an increase in diffuse light. The belief that diffuse light is more useful to plants than direct light stems from both the observation that the biosphere breathed in carbon dioxide following the Pinatubo eruption and the accompanying theory that diffusing light increases plant growth by redistributing light from the sun-saturated leaves at the top of the canopy to the light-hungry leaves below. Since each leaf has diminishing photosynthetic productivity for each incremental increase in sunlight, the theory argues, a more equal distribution of light should promote growth.

Aerosols scatter incoming sunlight, which more evenly distributes sunlight across the leaves of the plant. We test whether the loss of total sunlight or the increase in diffuse light from aerosol scattering has a stronger effect on yield.

While this “diffuse fertilization” appears to be strong in unmanaged environments, such as the Harvard Forest where the uptake of carbon increased following the Pinatubo eruption, our results find that, for agricultural yields, the damages from reduced total sunlight outweigh the benefits from a greater portion of the light being diffuse.

We cannot tell for sure, but we think that this difference between forest and crop responses could be due to either their differences in geometric structure (which could affect how deeply scattered light might penetrate the canopy):

Or to a re-optimization towards vegetative growth at the cost of fruit growth in response to the changes in light:

Two radishes grown in normal (left) and low light (right) conditions.  Credit: Nagy Lab, University of Edinburgh

This latter re-optimization may also explain the relatively large magnitude of the estimated effect on crop yield.

Result 3: Dimming damages neutralize cooling benefits from geo

Our final calculation, and the main policy-relevant finding of the paper, is that in a solar geoengineering scenario the damages from reduced sunlight cancel out the benefits from warming. The main challenge here was figuring out how to apply what we learned from volcanic eruptions to solar geoengineering, since the climate impacts (e.g. changes in temperature, precipitation or cloud cover) of a short-term injection of stratospheric aerosols differ from those of more sustained injections (e.g. see here and here). To address this, we first used an earth system model to calculate the impact of a long-term injection of aerosols on temperature, precipitation, cloud cover and insolation (measured in terms of aerosol optical depth). We then apply our crop model that we trained on the Pinatubo eruption (which accounts for changes in temperature, rainfall, cloud cover, and insolation independently) to calculate how these geoengineering-induced changes in climate impact crop yields. This two-step process allows us to disentangle the effects of solar geoengineering on climate (which we got from the earth system model) and of climate on crops (which we got from Pinatubo). Thus, we can calculate the change in yields due to a solar geoengineering scenario even though volcanic eruptions and solar geoengineering have different climatic fingerprints. Still, as with any projection to 2050, caveats abound such as the role of adaptation, the possibility of optimized particle design, or the possibility that variables other than sunlight, temperature, rainfall and cloud cover could play a substantial role.

Estimated global net effect of a geo-engineering on crop yields through four channels (temperature, insolation, cloud cover, precipitation) for four crops. The total effect is the sum of these four partial effects.

So, what should we do? For agriculture, our findings suggest that sulfate-based solar geoengineering might not work as well as previously thought to limit the damaging effects of climate change. However, there are other sectors of the economy that could potentially benefit substantially from geoengineering (or be substantially damaged, we just don’t know). To continue the metaphor from earlier, just because the first test of an experimental surgery had side effects for a specific part of the human body does not mean that the procedure is always immediately abandoned. There are many illnesses that are so harmful that procedures known to cause side effects are sometimes still worth the risk. Similarly, research into geoengineering should not be entirely abandoned because our analysis demonstrated one adverse side effect, there may remain good reasons to eventually pursue such a strategy despite some known costs. With careful study, humanity will eventually gain a better understanding of this powerful technology. We hope that the methodology developed in this paper might be extended to study the effects of sulfate aerosol injection on ecosystem or human health and would be open to collaborate on future studies. Thanks for reading, and I’m excited to hear any thoughts the community may have.

Warming makes people unhappy: evidence from a billion tweets (guest post by Patrick Baylis)

Everyone likes fresh air, sunshine, and pleasant temperatures. But how much do we like these things? And how much would we be willing to pay to gain more of them, or to prevent a decrease in the current amount that we get?

Clean air, sunny days, and moderate temperatures can all be thought of as environmental goods. If you're not an environmental economist, it may seem strange to think about different environmental conditions as "goods". But, if you believe that someone prefers more sunshine to less and would be willing to pay some cost for it, then a unit of sunshine really isn't conceptually much different from, say, a loaf of bread or a Playstation 4.

The tricky thing about environmental goods is that they're usually difficult to value. Most of them are what economists call nonmarket goods, meaning that we don't have an explicit market for them. So unlike a Playstation 4, I can't just go to the store and buy more sunshine or a nicer outdoor temperature (or maybe I can, but it's very, very expensive). This also makes it more challenging to study how much people value these goods. Still, there is a long tradition in economics of using various nonmarket valuation methods to study this kind of problem.

New data set: a billion tweets

Searching for critical thresholds in temperature effects: some R code

If google scholar is any guide, my 2009 paper with Wolfram Schlenker on the nonlinear effects of temperature on crop outcomes has had more impact than anything else I've been involved with.

A funny thing about that paper: Many reference it, and often claim that they are using techniques that follow that paper.  But in the end, as far as I can tell, very few seem to actually have read through the finer details of that paper or try to implement the techniques in other settings.  Granted, people have done similar things that seem inspired by that paper, but not quite the same.  Either our explication was too ambiguous or people don't have the patience to fully carry out the technique, so they take shortcuts.  Here I'm going to try to make it easier for folks to do the real thing.

So, how does one go about estimating the relationship plotted in the graph above?

Here's the essential idea:  averaging temperatures over time or space can dilute or obscure the effect of extremes.  Still, we need to aggregate, because outcomes are not measured continuously over time and space.  In agriculture, we have annual yields at the county or larger geographic level.  So, there are two essential pieces: (1) estimating the full distribution of temperatures of exposure (crops, people, or whatever) and (2) fitting a curve through the whole distribution.

The first step involves constructing the distribution of weather. This was most of the hard work in that paper, but it has since become easier, in part because finely gridded daily weather is available (see PRISM) and in part because Wolfram has made some STATA code available.  Here I'm going to supplement Wolfram's code with a little bit of R code.  Maybe the other G-FEEDers can chime in and explain how to do this stuff more easily.

First step:  find some daily, gridded weather data.  The finer scale the better.  But keep in mind that data errors can cause serious attenuation bias.  For the lower 48 since 1981, the PRISM data above is very good.  Otherwise, you might have to do your own interpolation between weather stations.  If you do this, you'll want to take some care in dealing with moving weather stations, elevation and microclimatic variations.  Even better, cross-validate interpolation techniques by leaving one weather station out at a time and seeing how well the method works. Knowing the size of the measurement error can also help correcting bias.  Almost no one does this, probably because it's very time consuming... Again, be careful, as measurement error in weather data creates very serious problems (see here and here).

Second step:  estimate the distribution of temperatures over time and space from the gridded daily weather.  There are a few ways of doing this.  We've typically fit a sine curve between the minimum and maximum temperatures to approximate the time at each degree in each day in each grid, and then aggregate over grids in a county and over all days in the growing season.  Here are a couple R functions to help you do this:

# This function estimates time (in days) when temperature is
# between t0 and t1 using sine curve interpolation.  tMin and
# tMax are vectors of day minimum and maximum temperatures over
# range of interest.  The sum of time in the interval is returned.
# noGrids is number of grids in area aggregated, each of which 
# should have exactly the same number of days in tMin and tMax
 
days.in.range <- function( t0, t1 , tMin, tMax, noGrids )  {
  n <-  length(tMin)
  t0 <-  rep(t0, n)
  t1 <-  rep(t1, n)
  t0[t0 < tMin] <-  tMin[t0 < tMin]
  t1[t1 > tMax] <-  tMax[t1 > tMax]
  u <- function(z, ind) (z[ind] - tMin[ind])/(tMax[ind] - tMin[ind])  
  outside <-  t0 > tMax | t1 < tMin
  inside <-  !outside
  time.at.range <- ( 2/pi )*( asin(u(t1,inside)) - asin(u(t0,inside)) ) 
  return( sum(time.at.range)/noGrids ) 
}

# This function calculates all 1-degree temperature intervals for 
# a given row (fips-year combination).  Note that nested objects
# must be defined in the outer environment.
aFipsYear <- function(z){
  afips    = Trows$fips[z]
  ayear    = Trows$year[z]
  tempDat  = w[ w$fips == afips & w$year==ayear, ]
  Tvect = c()
  for ( k in 1:nT ) Tvect[k] = days.in.range(
              t0   = T[k]-0.5, 
              t1   = T[k]+0.5, 
              tMin = tempDat$tMin, 
              tMax = tempDat$tMax,
              noGrids = length( unique(tempDat$gridNumber) )
              )
  Tvect
}

The first function estimates time in a temperature interval using the sine curve method.  The second function calls the first function, looping through a bunch of 1-degree temperature intervals, defined outside the function.  A nice thing about R is that you can be sloppy and write functions like this that use objects defined outside of the environment. A nice thing about writing the function this way is that it's amenable to easy parallel processing (look up 'foreach' and 'doParallel' packages).

Here are the objects defined outside the second function:

w       # weather data that includes a "fips" county ID, "gridNumber", "tMin" and "tMax".
        #   rows of w span all days, fips, years and grids being aggregated
 
tempDat #  pulls the particular fips/year of w being aggregated.
Trows   # = expand.grid( fips.index, year.index ), rows span the aggregated data set
T       # a vector of integer temperatures.  I'm approximating the distribution with 
        #   the time in each degree in the index T

To build a dataset call the second function above for each fips-year in Trows and rbind the results.

Third step:  To estimate a smooth function through the whole distribution of temperatures, you simply need to choose your functional form, linearize it, and then cross-multiply the design matrix with the temperature distribution.  For example, suppose you want to fit a cubic polynomial and your temperature bins that run from from 0 to 45 C.  The design matrix would be:

D = [    0          0          0   
            1          1           1
            2          4           8
             ...
           45     2025    91125]

These days, you might want to do something fancier than a basic polynomial, say a spline. It's up to you.  I really like restricted cubic splines, although they can over smooth around sharp kinks, which we may have in this case. We have found piecewise linear works best for predicting out of sample (hence all of our references to degree days).  If you want something really flexible, just make D and identity matrix, which effectively becomes a dummy variable for each temperature bin (the step function in the figure).  Whatever you choose, you will have a (T x K) design matrix, with K being the number of parameters in your functional form and T=46 (in this case) temperature bins. 

To get your covariates for your regression, simply cross multiply D by your frequency distribution.  Here's a simple example with restricted cubic splines:

library(Hmisc)
DMat <- rcspline.eval(0:45)
XMat <- as.matrix(TemperatureData[,3:48])%*%DMat
fit <- lm(yield~XMat, data=regData)
summary(fit)

Note that regData has the crop outcomes.  Also note that we generally include other covariates, like total precipitation during the season,  county fixed effects, time trends, etc.  All of that is pretty standard.  I'm leaving that out to focus on the nonlinear temperature bit. 

Anyway, I think this is a cool and fairly simple technique, even if some of the data management can be cumbersome.  I hope more people use it instead of just fitting to shares of days with each maximum or mean temperature, which is what most people following our work tend to do.  

In the end, all of this detail probably doesn't make a huge difference for predictions.  But it can make estimates more precise, and confidence intervals stronger.  And I think that precision also helps in pinning down mechanisms.  For example, I think this precision helped us to figure out that VPD and associated drought was a key factor underlying observed effects of extreme heat.

Fanning the flames, unnecessarily

This post is a convolution of David's post earlier today and Sol's from a few days ago.  Our debate with Buhaug et al that Sol blogged about has dragged on for a while now, and engendered a range of press coverage, the most recent by a news reporter at Science.  "A debate among scientists over climate change and conflict has turned ugly", the article begins, and goes on to catalog the complaints on either side while doing little to engage with the content.

Perhaps it should be no surprise that press outlets prefer to highlight the mudslinging, but this sort of coverage is not really helpful.  And what was lost in this particular coverage were the many things I think we've actually learned in the protracted debate with Buhaug.

We've been having an ongoing email dialog with ur-blogger and statistician Andrew Gelman, who often takes it upon himself to clarify or adjudicate these sorts of public statistical debates, which is a real public service.  Gelman writes in an email:

In short, one might say that you and Buhang are disagreeing on who has the burden of proof.  From your perspective, you did a reasonable analysis which holds up under reasonable perturbations and you feel it should stand, unless a critic can show that any proposed alternative data inclusion or data analytic choices make a real difference.  From their perspective, you did an analysis with a lot of questionable choices and it’s not worth taking your analysis seriously until all these specifics are resolved.

I'm sympathetic with this summary, and am actually quite sympathetic to Buhaug and colleagues' concern about our variable selection in our original Science article.  Researchers have a lot of choice over how and where to focus their analysis, which is a particular issue in our meta-analysis since there are multiple climate variables to choose from and multiple ways to operationalize them.  Therefore it could be that our original effort to bring each researcher's "preferred" specification into our meta-analysis might have doubly amplified any publication bias -- with researchers of the individual studies we reviewed emphasizing the few significant results, and Sol, Ted, and I picking the most significant one out of those.  Or perhaps the other researchers are not to blame and the problem could have just been with Sol, Ted, and my choices about what to focus on.

One effect to rule them all? Our reply to Buhaug et al's climate and conflict commentary

For many years there has been a heated debate about the empirical link between climate and conflict. A year ago, Marshall, Ted Miguel and I published a paper in Science where we reviewed existing quantitative research on this question, reanalyzed numerous studies, synthesized results into generalizable concepts, and conducted a meta-analysis of parameter estimates (watch Ted's TED talk).  At the time, many researchers laid out criticism in the press and blogosphere, which Marshall fielded through G-FEED. In March, Halvard Buhaug posted a comment signed by 26 authors on his website strongly critiquing our analysis, essentially claiming that they had overturned our analysis by replicating it using an unbiased selection of studies and variables. At the time, I explained numerous errors in the comment here on G-FEED. 

The comment by Buhaug et al. was published today in Climatic Change as a commentary (version here), essentially unchanged from the version posted earlier with none of the errors I pointed out addressed. 

You can read our reply to Buhaug et al. here. If you don't want to bother with lengthy details, our abstract is short and direct:

Abstract: A comment by Buhaug et al. attributes disagreement between our recent analyses and their review articles to biased decisions in our meta-analysis and a difference of opinion regarding statistical approaches. The claim is false. Buhaug et al.’s alteration of our meta-analysis misrepresents findings in the literature, makes statistical errors, misclassifies multiple studies, makes coding errors, and suppresses the display of results that are consistent with our original analysis. We correct these mistakes and obtain findings in line with our original results, even when we use the study selection criteria proposed by Buhaug et al. We conclude that there is no evidence in the data supporting the claims raised in Buhaug et al.

People are not so different from crops, turns out

A decade of strong work by my senior colleagues here at G-FEED have taught us that crops don’t like it hot: 

• Wolfram and Mike have the original go-to paper on US ag [ungated copy here], showing that yields for the main US field crops respond very negatively to extreme heat exposure
• David, Wolfram, Mike + coauthors have a nice update in Science using even fancier data for the US, showing that while average corn yields have continued to increase in the US, the sensitivity of corn to high temperatures and moisture deficits has not diminished. 
• And Max et al have a series of nice papers looking at rice in Asia, showing that hot nighttime temperatures are particularly bad for yields.

The results matter a lot for our understanding of the potential impacts of climate change, suggesting that in the absence of substantial adaptation we should expect climate change to exert significant downward pressure on future growth in agricultural productivity.

But we also know that for many countries of the world, agriculture makes up a small share of the economy.  So if we want to say something meaningful about overall effects of climate change on the economies of these countries (and of the world as a whole), we're also going to need to know something about how non-agricultural sectors of the economy might respond to a warmer climate. 

Thankfully there is a growing body of research on non-agricultural effects of climate -- and there is a very nice summary of some of this research (as well as the ag research) just out in this month's Journal of Economic Literature, by heavyweights Dell, Jones, and Olken. [earlier ungated version here].

I thought it would be useful to highlight some of this research here -- some of it already published (and mentioned elsewhere on this blog), but some of it quite new.  The overall take-home from these papers is that non-agricultural sectors are often also surprisingly sensitive to hot temperatures

First here are three papers that are already published:

1. Sol's 2010 PNAS paper was one of the first to look carefully at an array of non-agricultural outcomes (always ahead of the game, Sol...), using a panel of Caribbean countries from 1970-2006. Below is the money plot, showing strong negative responses of a range of non-ag sectors to temperature.  Point estimate for non-ag sectors as a whole was -2.4% per +1C, which was higher than the comparable estimate for the ag sector (-0.1% per 1C).

From Hsiang (2010)

2. Using a country-level panel, Dell Jones and Olken's instaclassic 2012 paper [ungated here] shows that both ag and non-ag output responds negatively to warmer average temperatures -- but only in poor countries. They find, for instance, that growth in industrial output in poor countries falls 2% for every 1C increase in temperature, which is only slightly lower than the -2.7% per 1C decline they find for ag. They find no effects in rich countries. 

3. Graff Zivin and Neidell (2014) use national time use surveys in the US to show that people work a lot less on hot days.  Below is their money fig:  on really hot days (>90F), people in "exposed" industries (which as they define it includes everything from ag to construction to manufacturing) work almost an hour less (left panel).  The right panels show leisure time.  So instead of working, people sit in their air conditioning and watch TV. 

from Graff Zivin and Neidell 2014.  Left panel is labor supply, right two panels are outdoor and indoor leisure time. 

And here are three papers on the topic you might not have seen, all of which are current working papers:

4.  Cachon, Gallino, and Olivares (2012 working paper) show, somewhat surprisingly, that US car manufacturing is substantially affected by the weather.  Using plant-level data from 64 plants, They show that days above 90F reduce output on that day by about 1%, and that production does not catch up in the week following a hot spell (i.e. hot days did not simply displace production).

5. Adhvaryu, Kala, and Nyshadham (2014 working paper)  use very detailed production data from garment manufacturing plants to show that hotter temperatures reduce production efficiency (defined as how much a particular production line produces on a given day, relative to how much engineering estimates say they should have produced given the complexity of the garment they were producing that day).  Not sure if I have the units right, but I think they find about a 0.5% decrease in production efficiency on a day that's +1C hotter.

6. Finally, in a related study, Somanathan et al (2014 working paper) use a nation-wide panel of Indian manufacturing firms and show that output decreases by 2.8% per +1C increase in annual average temperature.  They show that this is almost all coming from increased exposure above 25C, again pointing to a non-linear response of output to temperature.  For a subset of firms, they also collect detailed worker-level daily output data, and show that individual-level productivity suffers when temperatures are high -- but that this link is broken when plants are air conditioned.

So apparently it's not just crops that do badly when it's hot.  Most of the studies just mentioned cite the human physiological effects of heat stress as the likely explanation for why non-agricultural output also falls with increased heat exposure, and this seems both intuitive and plausible -- particularly given how similar the effect sizes are across these different settings.  But what we don't yet know is how these mostly micro-level results might aggregate up to the macro level. Do they matter for the projected overall effect of climate change on economies?  This is something Sol and I have been working on and hope to be able to share results on soon.  In the meantime, I will be setting my thermostat to 68F. 

Yet more cleverness: getting ambient temperature data from cellphones

Following up on an earlier post about some smarty-pantses that figured out how to use cell phone towers to extract estimates of local rainfall, many of these same smarty-pantses have now figured out how to use those same cell phones to provide information on local temperatures.  Here's the new paper, just out in Geophysical Research Letters [HT: Noah Diffenbaugh]:

Crowdsourcing urban air temperatures from smartphone battery temperatures
A. Overeem, J. C. R. Robinson, H. Leijnse, G. J. Steeneveld, B. K. P. Horn, and R. Uijlenhoet

Accurate air temperature observations in urban areas are important for meteorology and energy demand planning. They are indispensable to study the urban heat island effect and the adverse effects of high temperatures on human health. However, the availability of temperature observations in cities is often limited. Here we show that relatively accurate air temperature information for the urban canopy layer can be obtained from an alternative, nowadays omnipresent source: smartphones. In this study, battery temperatures were collected by an Android application for smartphones. A straightforward heat transfer model is employed to estimate daily mean air temperatures from smartphone battery temperatures for eight major cities around the world. The results demonstrate the enormous potential of this crowdsourcing application for real-time temperature monitoring in densely populated areas.

They validate their technique in a few big cities around the world, and it looks pretty neat.  As shown in their Fig 2, which shows temperatures for London over a 4-month period and is reproduced below, raw changes in battery temperature are highly correlated with variation in ambient temperature (compare the orange line and black line, reported correlation r=0.82), and their heat transfer model is able to get the levels close to right (compare the blue dots with the black line).

What we really want to know, of course, is whether this can also work in places where the weather-observing infrastructure is currently really poor (e.g. most of Africa), and thus were techniques like this could be extra useful.  It seems like there are a couple hurdles.  First, you need a lot of people with smartphones.  According to this article, smartphone penetration in Africa is currently around 20%, but Samsung (who might know) puts it at less than 10%.  Nevertheless, smartphone adoption appears to be growing rapidly (you can find them in just about any tiny rural market in western Kenya, for instance), and so this might not be such a limitation in a few years.  And something the authors worry about in colder and richer climes -- that their battery temperature readings are biased because people are in heated or air-conditioned buildings a lot -- is much less of a worry in places where people are outside more and don't keep their houses at a perfect 70F.

Second, to get temperature levels right, it appears that the authors have to calibrate battery temperatures in a given area to data on observed temperatures in that area -- which is obviously not helpful if you don't have observed data to start with.  But if all you care about is temperature deviations -- e.g. if you're running a panel model that is linear in average temperature -- then it seems like the raw battery temperature data give you this pretty well (see figure).  Then if you really need levels -- say you want to estimate how a crop responds to temperatures above a given threshold -- you could do something like David did in his 2011 paper on African maize and add these deviations back to somebody else's estimate of the climatology (David used WorldClim).

Given this, the authors' optimism on future applications seems fitting:


"In the end, such a smartphone application could substantially increase the number of air temperature observations worldwide. This could become beneficial for precision agriculture, e.g., for optimization of irrigation and food production, for human health (urban heat island), and for energy demand planning."

But hopefully the expansion of this technique into rural areas won't have to wait for observed data with which to calibrate their heat transfer model.  If that London plot above is representative, it seems like just getting the raw battery data could be really helpful.

A climate for conflict

Sol, Ted Miguel, and I are excited to have a paper we've been working on for a while finally come out. The title is "Quantifying the impact of climate on human conflict", and it's out in this week's issue of Science.  Seems like this should get Sol's tenure package at Berkeley off to a pretty nice start.  

Our goal in the paper is rather modest, and it's all in the title:  we collect estimates from the rapidly growing number of studies that look at the relationship between some climate variable and some conflict outcome, make sure they are being generated by a common and acceptable methodology (one that can credibly estimate causal effects), and then see how similar these estimates are.  We want to understand whether all these studies spread across different disciplines are actually telling us very different things, as the ongoing debate on climate and conflict might seem to suggest.

We compute things in terms of standardized effects -- percent change in conflict per standard deviation change in climate -- and find that, by and large, estimates across the studies are not all that different.  All the exciting/tedious details are in the paper and the supplement (here's a link to the paper for those without access, and here is a link to all the replication data and code).  We've also gotten some nice (and not so nice) press today that discusses different aspects of what we've done.  See here, here, here, here, here, here, here, here, here, and here for a sampler.  Most importantly, see here. 

We want to use this space to answer some FAQ about the study.  Or, really, some FHC:  Frequently Heard Criticisms.  A bunch of these have popped up already in the press coverage.  Some are quite reasonable, some (we believe) stem from a misunderstanding or misrepresentation of what we're trying to do in the paper, and some are patently false. 

So, in no particular order, some FHC we've gotten (in bold), and our responses.  We include direct quotes from our critics where possible.  Apologies for the lengthy post, but we are trying to get it all out there. 

UPDATE (Friday Aug 2, 4pm): We have now heard from 3 different folks - Andy Solow, Idean Salehyan, and Cullen Hendrix - who each noted that what they were quoted as saying in these articles wasn't the full story, and that they had said lots of nice things too.  After talking to them, it seems there is much more agreement on many of these issues than the press articles would have you believe, and in many cases it looks like the journalists went far out of their way to highlight points of disagreement. I guess this shouldn't be surprising, but we believe it does the debate a pretty serious disservice.  So our responses below should be interpreted as responses to quotes as they appeared, not necessarily responses to particular individuals' viewpoints on these issues, which the quotes might not adequately represent.  We are not trying to pick fights, just to clarify what our paper does and doesn't do.

1.  You confuse weather and climate. 
(verbatim from Richard Tol, courtesy Google Translate)

This is an old saw that you always get with these sorts of papers.  The implied concern is that most of the historical relationships are estimates using short-run variation in temperature and precip (typically termed "weather"), and then these are used to say something about future, longer-run changes in the same variables ("climate").  So the worry is that people might respond to future changes in climate -- and in particular, slower-moving changes in average temperature or precip -- differently than they have to past short-run variation.

This is a very sensible concern.  However, there are a couple reasons why we think our paper is okay on this front. First, we document in the paper hat the relationship between climate variables and conflict shows up at a variety of time scales, from hourly changes in temperature to century-scale changes in temperature and rainfall.  We use the word "climate" in the paper to refer to this range of fluctuations, and we find similar responses across this range, which provides some evidence that today's societies are not that much better at dealing with long-run changes than short-run changes.  This is consistent with evidence on climate effects in other related domains (e.g. agriculture).  

Second, we do not to make explicit projections about future impacts - our focus is on the similarity in historical responses.  Nevertheless, the reader is definitely given the tools to do their own back-of-the-envelope projections:  e.g. we provide effects in terms of standard deviations of temperature/ rainfall, and then provide a map of the change in temperature by 2050 in terms of standard deviations (which are really really large!).  That way, the reader can assume whatever they want about how historical responses map into future responses.  If you think people will respond in the future just like they did in the past, it's easy multiplication.  If you think they'll only be half as sensitive, multiply the effect size by 0.5, etc etc.  People can adopt whatever view they like on how future long-run responses might differ from past short-run responses; our paper does not take a stand on that. 

2. Your criterion for study inclusion are inconsistently applied, and you throw out studies that disagree with your main finding.
(Paraphrasing Halvard Buhaug and Jurgen Sheffran)

This one is just not true.  We define an inclusion criterion -- which mainly boils down to studies using standard techniques to account for unobserved factors that could be correlated with both climate and conflict --  and include every study that we could find that meets this criterion. In more technical terms, these are panel or longitudinal studies that include fixed effects to account for time-invariant or time-varying omitted variables.

In a few cases, there were multiple studies that analyzed the exact same dataset and outcomes, and in those cases we included either the study that did it first (if the studies were indistinguishable in their methods), or the study that did the analysis correctly (if, as was sometimes the case, one study met the inclusion criteria and another did not).  

So, the inclusion criterion had nothing to do with the findings of the study, and in the paper we highlight estimates from multiple studies whose do not appear to agree with the main findings of the meta-analysis (see Figure 5). We did throw out a lot of studies, and all of these were studies that could not reliably identify causal relationships between climate and conflict -- for instance if they only relied on cross-sectional variation.  We also throw out multiple studies that agreed very strongly with our main findings!  We provide very detailed information on the studies that we did and did not include in Section A of the SOM.

Sheffran refers to other recent reviews, and complains that our paper does not include some papers in those reviews.  This is true, for the methodological reasons just stated.  But what he does not mention is that none of these reviews even attempt to define an inclusion criterion, most of them review a very small percentage of the number of papers we do, and no make any attempt to quantitatively compare results across papers.  This is why our study is a contribution, and presumably why Science was interested in publishing it as a Research Article.

3. You lump together many different types of conflict that shouldn't be lumped together.  Corollary: There is no way that all these types of conflict have the same causal mechanism.  
(Idean Salehyan: "It’s hard to see how the same causal mechanism that would lead to wild pitches would be linked to war and state collapse")

This quote is very substantial misrepresentation of what we do.  First, nowhere do we make the claim that these types of conflict have the same causal mechanism.  In fact we go to great lengths to state that climate affects many different things that might in turn affect conflict, and the fact that the effect sizes are broadly similar across different types of conflict could be explained by climate's pervasive effect on many different potential intervening variables (economic conditions, food prices, institutional factors, ease of transportation, etc). 

Second, we take great pains to separate out conflicts into different categories, and only make comparisons within each category.  So we calculate effect sizes separately for individual-level conflict (things like assault and murder), and group level conflict (things like civil war).  So, again contra Idean's quote, we are not actually comparing baseball violence (which we term individual-level) with war and state collapse (which are group conflicts).  Read the paper, Idean! 

But the whole point of the paper is to ask whether effect sizes across these different types of conflict are in fact similar!  What we had in the literature was a scattering of studies across disciplines, often looking at different types of conflict and using different methodologies.  This disarray had led to understandable confusion about what the literature as a whole was telling us.  Our goal was to put all the studies on the same footing and ask, are these different studies actually telling us that different types of conflict in different settings respond very differently to climate?  Our basic finding is that there is much more similarity across studies than what is typically acknowledged in the debate.  

Whether this similarity is being driven by a common underlying mechanism, or by multiple different mechanisms acting at the same time, is something we do not know the answer to -- and what we highlight very explicitly as THE research priority going forward.  

4. You cherry pick the climate variables that you report
(paraphrasing Halvard Buhaug).

We try really hard not to do this.  Where possible, we focus on the climate variable that the authors focused on in the original study.  However, the authors themselves in these studies are almost completely unrestricted in how they want to parameterize climate.  You can run a linear model, a quadratic model, you can include multiple lags, you can create binary measures, you can create fancy drought measures that combine temperature and rainfall, etc etc.  Authors do all these things, and often do many of them in the same paper.  Since we can't include all estimates from every single paper, we try to pick out the author's preferred measure or estimate, and report that one.  In the cases where authors tested many different permutations and did not hint at their "preferred" estimate (e.g. in Buhaug's comments on our earlier PNAS paper), we pick the median estimate across all the reported estimates.  Section A2 in the SOM provides extra detail on all of these cases.

5. This paper is not based on any theory of conflict, so we learn nothing.

This is very related to FHC #3 above, and we get this from the political science crew a lot.  The thing is, there are a ton of different theories on the causes of conflict, and empirical work so far has not done a great job of sorting them out.  In some sense, we are being atheoretical in the paper -- we just want to understand whether the different estimates are telling us very different things.  As noted above, though, the fact that they're generally not would seem to be very important to people interested in theory! 

Claire Adida, a political science prof at UCSD (and @ClaireAdida on twitter), put it really nicely in an email:  "I don't understand this ostrich politics. How about saying something like 'we really want to thank these economists for doing a ton of work to show how confident we can be that this relationship exists. It's now our turn - as political scientists - to figure out what might be the causal mechanisms underlying this relationship.' " (Btw, I have no idea what "ostrich politics" means, but I really like it!) 

6. People will adapt, so your results are a poor guide for impacts under climate change. 
(paraphrasing Jurgen Sheffran, courtesy Google Translate; Cullen Hendrix: "I'm optimistic.  Unlike glaciers, humans have remarkable adaptive capacity"; as well as audience members in every seminar we've ever given on this topic).

This is very related to FHC #1 above. It is definitely possible that future societies will become much better at dealing with extreme heat and erratic rainfall.  However, to just assume that this is the case seems to us a dangerous misreading of the existing evidence.  As stated above, available evidence suggests that human societies are remarkably bad at dealing with deviations from average climate, be it a short-lived temperature increase or a long-term one.  See here and here for other evidence on this topic.

And it has to be the case that knowing something about how yesterday's and today's world respond to climate tells us more about future impacts than knowing nothing about how societies have responded to climate.  The alternative - that the present world tells us nothing about the future world - just does not appear consistent with how nearly anybody sees things. 

7.  A lot of your examples - e.g. about civilization collapse - do not pertain to the modern world.

We got this in peer review. It's true that a lot of these collapse stories are from way back.  The Akkadian empire collapsed before 2000 BC, after all!  In a companion paper, forthcoming in Climatic Change, Sol and I look a little more carefully at this, and it actually turns out that per capita incomes in many of these societies, pre-collapse, were remarkably similar to incomes in many developing countries today.  To the extent that economic conditions shape conflict outcomes -- a common belief in economics and political science -- then this provides at least some evidence that these historical events are not completely irrelevant to today. 

More basically, though, it seems like hubris to just assume that "this time things are different".  At the time of their collapse, each of these societies (the Akkadians, the Maya, some of the Chinese dynasties, Angkor Wat) were incredibly advanced by global standards, and they probably also did not figure that climate would play a role in their demise.  Because we don't yet have a firm grasp on why climate affects conflict, it again seems dangerous to assume that things are completely different today -- just as it seems dangerous to conclude that modern societies are going to be completely destroyed by climate change, a claim we make nowhere in the paper. 

However, we do hope that "this time is different"!  It would be quite nice if the Mayan and Angkor Wat examples did not, in fact, pertain to the modern world. 

8. You can't claim that there is an overall significant relationship between climate and conflict if many of the studies you analyze do not show a statistically significant effect. 
(Halvard Buhaug: "I struggle to see how the authors can claim a remarkable convergence of quantitative evidence when one-third of their civil conflict models produce a climate effect statistically indistinguishable from zero, and several other models disagree on the direction of a possible climate effect")

This is a basic misunderstanding of what a meta-analysis does.  The beauty of a meta-analysis is that, by pooling a bunch of different studies, you can dramatically increase statistical power by increasing your sample size. It's even possible to find a statistically significant result across many small studies even if no individual study found a significant result.  This happens in medical meta-analyses all the time, and is why they are so popular in that setting:  each individual study of some expensive drug or procedure often only includes a few individuals, and only by combining across studies do you have enough statistical power to figure out what's going on.

So the fact that some of the individual studies were statistically significant, and others were not, does not necessarily affect the conclusions you draw when you average across studies.  In our case, it did not:  the mean across studies can be estimated very precisely, as we show in Figures 4 and 5, and discuss in detail in the SOM.

A final point:  we find a striking consistency in findings in the studies that look at temperature in particular.  Of the 27 modern studies that looked at a relationship between temperature and conflict, all 27 estimated a positive coefficient.  This is extremely unlikely to happen by chance - i.e. very unlikely to happen if there were in fact no underlying relationship between temperature and conflict.  Think of flipping a coin 27 times and getting heads all 27 times.  The chance of that is less than 1 in a million.  This is not a perfect analogy -- coin flips of a fair coin are independent, our studies are not fully independent (e.g. many studies share some of the same data) -- but we show on page 19 in the SOM that even if you assume a very strong dependence across studies, our results are still strongly statistically significant. 

9. Conflict has gone down across the world, as temperatures have risen. This undermines the claims about a positive relationship between temperature and conflict.
(Idean Salehyan: "We've seen rising temperatures, but there's actually been a decline in armed conflict".)

There are a couple things wrong with this one. First, many types of conflict that we look at have not declined at all over time.  Here is a plot of civil conflicts and civil wars since 1960 from the PRIO data, summed across the world.  As coded in these data, civil conflicts are conflicts that result in at least 25 battle deaths (light gray in the plot), and civil wars are those that result in at least 1000 deaths (dark gray).  As you can see, both large wars and smaller conflicts peaked in the mid-1990s, and while the incidence of larger wars have fallen somewhat, the incidence of smaller conflicts is currently almost back up to its 1990s peak.  These types of conflicts are examined by many of the papers we study, and have not declined.   

As another check on this, I downloaded the latest version of the Social Conflict in Africa Dataset, a really nice dataset that Idean himself was instrumental in assembling.  This dataset tracks the incidence of protests, riots, strikes, and other social disturbances in Africa.  Below is a plot of event counts over time in these data.  Again, you'd be very hard pressed to say that this type of conflict has declined either.  So I just don't understand this comment.

Second, and more importantly, there are about a bazillion other things that are also trending over this period.  The popularity of the band New Kids On The Block as also fallen fairly substantially since the 1990s, but no-one is attributing changes in conflict to changes in NKOTB popularity (although maybe this isn't implausible).  The point is that identifying causal effects from these trends is just about impossible, since so many things are trending over time.  

Our study instead focuses on papers that use detrended data - i.e. those that use variation in climate over time in a particular place.  These papers, for instance, compare what happens to conflict in a hot year in a given country, to what happens in a cooler year in that country, after having account for any generic trends in both climate and conflict that might be in the data.  Done this way, you are very unlikely to erroneously attribute the effects of changes in conflict to changes in climate. 


10. You don't provide specific examples of conflicts that were caused by climate.  

(Halvard Buhaug: "Surprisingly, the authors provide no examples of real conflicts that plausibly were affected by climate extremes that could serve to validate their conclusion. For these and other reasons, this study fails to provide new insight into how and under what conditions climate might affect violent conflict")

I do not understand this statement.  We review studies that look at civil conflict in Somalia, studies that look at land invasions in Brazil, studies that look at domestic violence in one city in one year in Australia, studies that look at ethnic violence in India, studies that look at murder in a small set of villages in Tanzania.  The historical studies looking at civilization collapses in particular try to match single events to large contemporaneous shifts in climate.  We highlight these examples in both the paper and in the press materials that we released, and they were included in nearly every news piece on our work that we have seen.  So, again, this comment just does not make sense. 

Perhaps implicit in this claim is often some belief that we are climate determinists.  But as we say explicitly in the paper, we are not arguing that climate is the only factor that affects conflict, nor even that it is the most important factor affecting conflict.  Our contribution is to quantify its role across a whole host of settings, and our findings we hope will help motivate a bunch more research on why climate should shape conflict so dramatically (see Claire's quote above).

11.  You are data mining. Corollary: What you guys are demonstrating is a severe publication bias problem -- only studies that show a certain result get published.
(Andy Solow: "In the aggregate, if you work the data very hard, you do find relationships like this. But when you take a closer look, things tend to be more complicated." As an aside, Andy sent us a very nice email, noting in reference to the press coverage of our article: "From what I've seen so far, all the nice things I said - that you are smart, serious researchers working on an important and difficult problem, that your paper will contribute to the discussion, that you may well be right - have been lost in favor of concerns I have and that, as I took pains to point out, you are already aware of.")

This is related to FHC #2 and #4 above. We have defined a clear inclusion criterion, and only include studies that meet this criterion.  As detailed in the SOM Section A2, we do not include a number of studies that agree very strongly with our main findings - for instance Melissa Dell's very nice paper on the Mexican Revolution.  Again, our inclusion criteria is based on methodology, not findings. 

The publication bias issue is a tougher one, and one which we explicitly address in the paper -- it even gets its own section, so it's not something we're trying to hide from.  We test formally for it in the SOM (Section C), finding limited evidence that publication bias is behind our results.  We also note that it's not clear where the professional incentives now lie in terms of the sorts of results that are likely to get published or noticed.  The handful of climate/conflict skeptics have garnered a lot of press by very publicly disagreeing with our findings, and this has presumably been good for their careers.  Had they instead published papers that agreed with our findings, it's likely that the press would not have had these folks as their go-to this time around.  Similarly, journals are probably becoming less interested in publishing yet another paper that shows that higher temperatures lead to more conflict.  Because so many of the papers we review are really recent (the median publication date across our studies was 2011), we feel that it is unlikely that all of these results are false positives.