Here a MIP, there a MIP, everywhere a MIP, MIP

A growing number of papers are looking at climate change impacts using multiple models. A few more are out this week in a special issue of PNAS. Mostly I just want to point readers of this blog to them if they are interested. I am generally a big fan of model intercomparisons (MIPs). I talk about them so much in my modeling course that I can usually hear the students’ collective sigh of relief when I move on to another topic.

Most of the strengths of MIPs have been demonstrated clearly with the climate MIPs, now on their 5^th rendition. They are useful for estimating uncertainties, they can point to important weaknesses in some models, and most of all they can create something that is more than the sum of its parts – the magical multi-model mean – where independent model errors cancel and estimates become more reliable. And a positive externality of these activities is usually that experiments and observational data to rigorously test models tends to improve, since each model isn’t in charge of testing their own model (“trust is, it’s great, we validated it years ago!”)

I think two reasons the climate MIPs were so successful is that the results were made available to the entire community, and relatedly that most of the groups performing the comparisons were not simultaneously working on model improvement. I’m not sure yet if AgMIP will follow this example. In conversations with them, I think they’d like to, but are not quite there.

With all of the positives going for it, I am still a little puzzled by a few things in the recent MIP papers. For one, it’s not clear to me why agriculture studies still do so much comparison of “no CO2” and “with CO2” runs, and conclude that the difference represents some indication of how much more work needs to be done on CO2. I’m not saying that I haven’t heard various explanations, but none of them are satisfying. The chance that CO2 has no effect on crops is about the same as the chance that Wolfram will show up to work tomorrow in a dress (that’s a very low chance, in case you were wondering).  If you are looking at uncertainty from CO2, you should look at various plausible responses to CO2, and zero isn’t one of them. I can see reasons to make estimates without CO2, including if your model doesn’t treat it, or if you are focused on effects of heat in order to test adaptations, but if you are trying to look at impacts of different emissions scenarios, why keep running a model without CO2? It reeks of trying to make the problem look worse than it is.

Another niggle is that the experimental design isn’t always set up to provide insight into what causes differences between models. I’m sure that will improve with time. But for now they are drawing some big conclusions from fairly weak comparisons. For example, the figure below shows the huge spread in model results is largely from two models from LPJ and one from GAEZ being very positive. They use this to conclude that models without N limitation have more positive impacts. But there are tons of differences between these and other models, why not conclude that models that start with L or G are more optimistic? The theory is that models with N limits can’t respond as much to CO2, but it should also be the case that they can’t respond as much to temperature, as work Wolfram and I did a while back concluded. (We also saw the GAEZ model was way positive in regions that shouldn't have much N stress). They’d need to have an experimental design to really demonstrate that it's nitrogen and not something else.  

Just to be clear, I really do like the MIPs and the people involved are high quality and have been very generous with their repeated offers to participate. Unfortunately, they have more meetings than Australia has poisonous snakes (which is a lot, in case you were wondering). I am participating in a wheat site-level intercomparison, which hopefully will be out this year.

On another note, I am now fully settled in to live in Brisbane (on sabbatical), and will try to post a little more often. I’m learning lots of interesting stuff, and not all of it about cricket (although there has been a curious uptick in the national team’s performance since I went to their match my first week – see “Australia’s resurgence as a world power in cricket has been swift, ruthless and dangerous”).  Mostly I’m deep into crop physiology, which readers of this blog (if there are any left) may or may not care about, but it may be the only thing I have to talk about for a while. Also, there’s the IPCC approval session coming in a few weeks which should be interesting. I think Max will also be there blogging for one of the other sites he actually writes things for.