A midsummer’s cross-section

Summer is going too fast. It seems like just yesterday Lebron James was being a sore loser, body slamming and stepping over people – and getting rewarded for it by the NBA. Apart from that, one interesting experience this summer was getting to visit some very different maize (corn) fields within a few weeks in July. First, I was in Kenya and Uganda at some field sites, and then I was visiting some farms in Iowa.

When talking maize, it’s hard to get much different than East Africa and East Iowa. As a national average, Kenya produces a bit less than 4 million tons of maize on 2 million ha, for a yield of about 1.75 t/ha. Iowa has about seven times higher yield (12.5 t/ha), and produces nearly twenty times more maize grain. The pictures below give a sense of a typical field in each place (Kenya on the left).

Lots of things are obviously different between the two areas. There are also some things that people might think are different but really aren’t. For example, looking at annual rainfall or summer temperatures, they are pretty similar for the two areas (figures from www.climatemps.com, note different scales):

But there are also things that are less obviously different. Earlier this year I read this interesting report trying to estimate soil water holding capacity in Africa, and I’ve also been working a bunch with soil datasets in the U.S. from the USDA. Below shows the total capacity of the soil to store water in the root zone (in mm) for the two areas, plotted on the same scale. 

It’s common for people to talk about the “deep” soils of the Corn Belt, but I don’t think people typically realize just how much better they are at storing water than many other places. There’s virtually no overlap between the distribution of root zone storage in the two areas, and on average Kenya soils have about half the capacity of Iowa’s.

How much difference can this one factor make? As a quick thought experiment I ran some APSIM-maize simulations for a typical management and weather setup in Iowa, varying only the root zone storage capacity between 150 and 330mm. Simulated yields by year are shown below, with dashed lines showing the mean for each soil. 

This suggests that having half the storage capacity translates to roughly half the average yields, with much bigger relative drops in hot or dry years like 1988 or 2012. And this assumes that management is identical, when a rational farmer would surely respond by applying much less inputs to the worse soils.

Just something to keep in mind when thinking about the potential for productivity growth in Africa. There’s certainly room for growth, and I saw a lot of promising trends. But just like when it comes to the NBA officials, there's a lot going on under the surface, and I wouldn’t expect too much. 

Introducing SCYM

In the hopes of figuring out how to raise crop yields or farmer incomes around the world, it would be really nice if we had a quick and accurate way of actually measuring yields for individual fields. That has motivated a lot of work over the years on using satellite data, and we have a paper out this week describing another step in that direction.

As I see it there are three main ingredients needed for yield remote sensing to be successful on a meaningful scale. One is the raw data. As Marshall’s recent post explained, there are several new satellite data providers that are really transforming our ability to track individual fields, even in smallholder areas.

Second is the ability to process the data at scale. Five years ago, for example, I would have to hire a research assistant to download imagery, make sure it was geometrically and radiometrically calibrated (i.e. properly lined up and in meaningful units), and then apply whatever algorithms we had. That just didn’t scale very well, in terms of labor or on-site data storage or processing. When a collaborator would ask “could you produce yield estimates for my study area,” I would have to think about how many weeks or months of work that would entail. But a couple of years ago I was introduced to Google’s Earth Engine, which is “a planetary-scale platform for environmental data & analysis.” In practical terms, it means that they have a lot of geospatial data (including all historical Landsat imagery), a lot of built-in algorithms for processing, and an interface to run your own code on the data and visualize or save the output. Part of why it works is that data providers, like the USGS for Landsat, have gotten better at providing well calibrated data. Earth Engine is very cool, and the more I’ve worked with it, the more I can see how this transforms our ability to extract value out of data already collected.

Third, and arguably the rate-limiting step nowadays, is to have algorithms that can translate satellite data into accurate yield estimates. It’s easy enough to do this if you have lots of ground data to calibrate to for a particular site, but that’s generally not scalable (unless people get clever about crowdsourcing ground “truth”). What seemed to be lacking was a very generic, scalable algorithm. So in the last 8 months or so we’ve been working to develop and test one idea about how to do this. I’m calling it a scalable satellite-based crop yield mapper (SCYM, pronounced “skim”), and a description of it has just been published in Remote Sensing of Environment. Conveniently, SCYM also stands for Steph Curry’s Your MVP.

The basic idea is that if you don’t have lots of ground data to calibrate a model, why not generate lots of fake ground data? Then for whatever combination of observations you actually have (say, for instance, satellite images on 2 or 3 specific days, and measures of daily weather), you can look into your fake data to see what the best fit model is to predict the desired variable (“yield”) from the measured predictors. The paper provides more detail, which I won’t bore readers with here. But to give a sense of the type of output, below shows an animation of our maize yield estimates over part of Iowa for 2008-2013. Red are high yields, blue are low.

The figure below shows a comparison between these and ground “truth” estimates for maize, which we take from the dataset described in a previous post.

The cool thing about this is that it’s quite generic. To illustrate that, we reran the model for soybeans, with results nearly as good as for maize.

Hopefully this type of thing will help make faster progress on understanding yields and farm productivity, and figuring out what actually works for improving them. One general lesson out of this for me is that sometimes making something really scalable requires scrapping an old approach. We had been previously running crop models for specific sites and years, but that wasn't possible within the Earth Engine system. I think SCYM (which trains a regression using simulations over lots of sites and years) is more robust than what we had, and along with the new satellite data and Earth Engine-type systems, it might just provide a way to do yield mapping at scale.

Measuring yields from space

[This post is co-written with Florence Kondylis at the World Bank, and a similar version was also posted over at the World Bank's Development Impact blog.]

One morning last August a number of economists, engineers, Silicon Valley players, donors, and policymakers met on the UC-Berkeley campus to discuss frontier topics in measuring development outcomes. The idea behind the event was not that economists could ask experts to create measurement tools they need, but instead that measurement scientists could tell economists about what was going on at the frontier of measuring development-related outcomes.  One topic that generated a lot of excitement -- likely due to David Lobell's charm at the podium -- was the potential for a new crop of satellites to remote-sense (i.e. measure) important development outcomes.

Why satellite-based remote sensing?
The potential ability to use satellites to measure common development outcomes of interest excites researchers and practitioners for a number of reasons, chief among them the amount of time and money we typically have to spend to measure these outcomes the “traditional” way (e.g. conducting surveys of households or firms).  Instead of writing large grants, spending days traveling to remote field sites, hiring and training enumerators, and dealing with inevitable survey hiccups, what if instead you could sit at home in your pajamas and, with a few clicks of a mouse, download the data you needed to study the impacts of a particular program or intervention?

The vision of this “remote-sensing” based approach to research is clearly intoxicating, and is being bolstered by the vast amount of high-resolution satellite imagery that is now being acquired and made available.  The recent rise of “nano-“ or “micro”-satellite technology – basically, fleets of cheap, small satellites that image the earth in high temporal and spatial resolution, such as those being deployed by our partner Skybox – could hold particular promise for measuring the types of outcomes that development folks often care about.  This is perhaps most obviously true in agriculture, where unlike in the manufacturing sector, most production takes place outside.

How does it work?
For most agricultural crops – particularly the staple crops grown by African smallholders, such as maize – pretty much anyone can look at a field and see the basic difference between a healthy highly productive crop and low-yielding crop that is nutrient or moisture stressed.  One main clue is color: healthy vegetation reflects and absorbs different wavelengths of light than less-healthy vegetation, which is why leaves on healthy maize plants look deep green and leaves on stressed or dead plants look brown.  Sensors on satellites can also discern these differences in the visual wavelengths, but they also measure differences at other wavelengths, and this turns out to be particularly useful for agriculture.  Healthy vegetation, in turns out, absorbs light in the visible spectrum and reflects strongly in the near infrared (which the human eye can’t see), and simple ratios of reflectance at these two wavelengths form the basis of most satellite-based measures of vegetative vigor – e.g. the familiar Normalized Difference Vegetation Index, or NDVI.   High ratios basically tell you that you’re looking at plants with a lot of big, healthy leaves.

The trick is then to be able to map these satellite-derived vegetation indices into measures of crop yields.  There are two basic approaches (see David's nice review article for more detail).  The first combines satellite vegetation indicies with on-farm yield observations as collected from the typical household or agricultural surveys.  By regressing the “true” survey-based yield measure on the satellite-based vegetation index, you get an estimated relationship between the two that can then be applied to other agricultural plots that you observe in the satellite data but did not survey on the ground.  The second approach combines the satellite data with pre-existing estimates of the relationship between final yield and vegetative vigor under various growing conditions (often as derived from a crop simulation model, which you can think of as an agronomist’s version of a structural model). Applying satellite reflectance measures to these relationships can then be used to estimate yield on a given plot.  A nice feature of this second approach is that it is often straightforward to account for the role of other time-varying factors (e.g. weather) that also affect the relationship between vegetation and final yield.

How well does it work?
These approaches have mainly been applied to larger farm plots in the developed and developing world, at least in part because until very recently the available satellite imagery was generally too coarse to resolve the very small plot sizes (e.g. less than half an acre) common in much of Africa.  For instance, the resolution of the MODIS sensor is 250m x 250m, meaning one pixel would cover more than 15 one-acre plots. Nevertheless, these approaches have been shown to work surprisingly well on these larger fields.  Below are two plots, both again from David and co-author's work, showing the relationship between predicted and observed yields for wheat in Northern Mexico, and maize (aka “corn”, for Americans) in the US Great Plains.   Average plot sizes in both cases are > 20 hectares, equivalent to at least 50 one-acre plots.

Top plot:  wheat yields in northern Mexico, from Lobell et al 2005.  Bottom plot:  corn yields in the US Great Plains, from Sibley et al 2014. 

Although success is somewhat in the eyes of the beholder here, the fit between observed and satellite-predicted yields is pretty good in both of these cases, with overall R2s of 0.63 in the US case and 0.78 in the Mexico case.  And, at least in both of these cases, the “ground truth” yield data was not actually used to construct the yield prediction – i.e. they are using the second approach described above.  This was possible in this setting because these were crops and growing conditions for which scientists have a good mechanistic understanding of how final yield relates to vegetative vigor.

From rich to poor, big to small
Applying these approaches much of the developing world (e.g. smallholder plots in Africa) has been harder.  This is not only because of the much smaller plot sizes, and thus the difficulty (impossibility, often) of resolving them in existing satellite imagery, but also because of a lack of either (i) ground truth data to develop the satellite-based predictions, and/or (ii) a satisfactory mechanistic understanding in these environments of how to map yields to reflectance measures.

New data sources from both the ground and sky are starting to make this possible.  Sensors on the new micro-satellites mentioned above often have sub-meter resolutions, meaning smallholder plots are now visible from space (a half-acre plot would be covered by over 2000 pixels).  Furthermore, this imagery is being acquired often enough to ensure at least a few cloud-free images during the growing season -- not a small problem in the rainy tropics.

Working with David and some collaborators in Kenya, Uganda, and Rwanda, we are linking this new imagery with ground-based yield data we are collecting to understand whether the satellite data can capably predict yields on heterogeneous smallholder plots.  Below is a map of some of the smallholder maize fields we have mapped and are tracking in Western Kenya, as part of an ongoing experiment with smallholder farmers in the region.

Locations of some plots we are tracking in Western Kenya

Some of the long run goals of this work are to (i) allow researchers who have already have information on plot boundaries and crop choice to use satellite images to estimate yields, and (ii) to allow researchers who do not have plot boundaries but who are interested in broader-scale agricultural performance (eg. at the village or district level) a way to track yields at that scale. This work is ongoing, but given the experience in developed countries, we are hopeful.

Some challenges.
Nevertheless, there are clear challenges to making this approach work at scale, and clear limitations (at least in the near term) to what this technology can provide.   Here are a few of the main challenges:

1. Which boundaries and which crops. To measure outcomes at the level of the individual farm plot, satellite-based measures will be most easily employable if the researcher already knows the plot boundaries and knows what crop is being grown.   As satellite imagery improves and as computer vision algorithms are developed to remotely identify plot boundaries, both of these constraints will likely be relaxed, but the researcher will still need some ground information on which plots belong to whom. 
2. Measurement error. Even with plot boundaries in hand, the fact that satellite imagery will not be able to perfectly predict yields means that using satellite-predicted yields as an outcome will likely reduce statistical power (although it’s not immediately clear how much noisier satellite estimates will be, given that survey based measures of these outcomes – e.g. from farmer self reports – are likely also measured with error.)  This almost certainly means that this technology will not be equipped to discern small effects in the smaller-sized ag RCTs that often get run.
3. Moving beyond yield.  Finally, even with plot boundaries in hand and well-powered study, satellites are going to have a hard time measuring many of the other outcomes we care about – things like profits or consumption expenditure.  Satellites might in the near term be able to get at related outcomes such as assets (something we’re also working on), but it’s clearly going to be hard to observe most household expenditures directly. 

Putting these difficulties together, should we just abandon this whole satellite thing?  We think not, for two reasons.  The first reason is that as we (hopefully) improve our ability to accurately measure smallholder yields from space, this ability would provide a clear compliment to existing surveys.  For instance, if yields are a primary outcome, imagine just being able to do a baseline survey (where field boundaries are mapped) and then measure your outcome at follow-up from the sky.  This will make an entire study both faster and cheaper, which should allow for larger sample sizes, which will in turn help deal with the measurement error issue above.

Second, we still have a surprisingly poor understanding of why some farms, and some farmers, appear to be so much more productive than others.  Is it the case that relatively time-invariant factors like soil type and farmer ability explain most of the observed variation, or are time-varying factors like weather more important?  Satellite data might be particularly useful for this question (David's review paper, and his earlier G-FEED post, gives some really nice examples), because you can assemble huge samples of farm plots that can then be easily followed over time.  Satellite data in this setting therefore might afford more power, and you can do it all in your pajamas.

Regressing to the mean

As a general rule, people like to take credit when good things happen to them, but blame bad luck when things go wrong. This probably helps us all get through the day, feeling better about ourselves and other people we like. For example, I’d like to think that the paper rejection I got last month was bad luck, while the award I got last year was totally deserved. But anyone who pays attention to professional sports, or the stock market, knows that success in any single day or even year has a lot to do with luck. It’s not that luck is all you need, but it often makes the difference between very evenly-matched competitors. That’s why people or teams who perform particularly well in one year tend to drop back the next, a.k.a. regressing to the mean.

Take tennis. There’s clearly a skill separation between professionals and amateurs, and between the top four or five professionals and everyone else. But among the top, it’s hard to know who will win on any day, and it’s very hard to sustain a streak of victories against other top players. So even when someone like Raphael Nadal, who has some remarkable winning streaks, talks about how he got a few key bounces, he’s as much being an astute observer as a gracious winner.

Or the stockmarket. It’s well known that even the best fund managers have a hard time outperforming the market for a long time. Even if someone has beaten it five years in a row, there’s a good chance it was just luck given how many fund managers are out there. I don’t tend to watch interviews with fund managers as much as athletes, but something tells me they might be a little less inclined to turn down the credit.

So what about agriculture? It’s not a source of entertainment or income for most of us, so we don’t spend much time thinking about it, and you won’t find any posts on fivethirtyeight about it. But one thing that struck me early on working in agriculture is how farmers are just as prone to thinking they are above average as the rest of us. More specifically, if you ask why some fields around them don’t look as good, they will talk about how that farmer is lazy, has another job, doesn’t take care of his soil, etc.

As far as I can tell, this isn’t a purely academic question. Understanding how much farmers vary in their ability to consistently produce a good crop is important if you want to know the best ways of improving agricultural yields. If there really are a bunch of star performers out there, then letting them take over from the laggards by buying up their land, or training other farmers to use best-practices, could be a good source of growth in the next decade. For example, here’s a cool presentation about a new effort in India to have farmers spread videos of best practices through social networks.

There’s a fairly obvious but not perfect link to the idea of yield gaps. People who say that closing yield gaps are a big “low-hanging fruit” for yield improvement often have a vision of better agronomy being able to drastically raise yields. The link isn’t perfect, because it could be that even the best farmers in a region are underperforming, agronomically speaking, for instance if fertilizers are very expensive. And it could be that some farmers consistently out-perform not because of better management, but because they are endowed with better soils (though this can be sorted out partly by using data on soil properties). Even with these caveats, understanding how much truly better the “best” farmers are could help give a more realistic view of what could be achieved with improved agronomy.

The key here is the “how much” part of it. Nobody can argue that some farmers aren’t better than others, just as nobody can say that Warren Buffet isn’t better than an average fund manager. The question is whether this is a big opportunity, or if it’s best to focus efforts elsewhere. I’ve been trying to get a handle on the “how much” over the years by using satellites to track yields over time. I’ve used various ways of trying to display this in a simple way, but nothing was too satisfying. So let me give it another shot, based on some suggestions from Tony Fischer during a visit to Canberra.

The figures below show wheat yield anomalies (relative to average) for fields ranked in the top 10% for the first year we have satellite data (green line). Each panel is for a different area, and soils don’t vary too much within each area. Then we track those fields over the following years to see if those fields are able to consistently outperform their neighbors. Similarly we can follow fields in any other group, and I show both the bottom decile (0-10% in blue) and the fifth (40-50% in orange). The two horizontal lines show the mean yield anomaly in the first group for the year they ranked in the top, and their mean yield anomaly in all the other years. If it was all skill (or something else related to a place like the soil quality) the second line would be on top of the first. If it was all luck, the second line would be at zero.

So what’s the verdict? The top performers in the first year definitely show signs of regressing to the mean, as their mean yield drops much closer to the overall average in other years. Similarly, the worst performers “regress” back up toward the mean. But neither jump all the way back to zero, which says that some of the yield differences are persistent. In the two left panels, the anomalies are a little more than one-third the size they were in the initial year. In the right panel, the anomalies are about half their original value, indicating relatively more persistence. That makes sense since we know the right panel is a region where some farmers consistently sow too late (see this paper for more detail).

So a naive look at any snapshot of performance over a year or two would be way too optimistic about how big the exploitable yield gap might be. It’s important to remember that performance tends to regress to the mean. At the same time, there are some consistent differences that amount to roughly 10% of average yields in these regions (where mean yield is around 5-6 tons/ha). And with satellites we can pinpoint where the laggards are and target studies on what might be constraining them. At least that's the idea behind some of our current work. 

Whether other regions would look much different than the 3 examples above, I really don’t know. But it shouldn’t be too hard to find out. With the current and upcoming group of satellites, we now have the ability to track performance on individual fields in an objective way, which should serve as a useful reality check on discussions of how to improve yields in the next decade.