- How robust are the conclusions in the AR5 Physical Science Basis report? Have the IPCC adequately addresses [sic] criticisms of previous reports? How much scope is there to question of the report’s conclusions?
- To what extent does AR5 reflect the range of views among climate scientists?
- Can any of the areas of the science now be considered settled as a result of AR5’s publication, if so which? What areas need further effort to reduce the levels of uncertainty?
In my view, the suggestion that a formal process can itself guarantee the absolute accuracy of any document and put its conclusions beyond question is a misrepresentation of the way science works. It is often suggested that, because they have been restricted to peer-reviewed sources and passed through multiple stages of review, the IPCC’s conclusions are beyond reasonable doubt. This is misleading, because it focuses attention on the process, rather than the evidence itself.
I don't think this is a convincing reply. Allen confuses absolute certainty (which you never will get in science) with the question if the knowledge is credible and trustworthy. How can this be achieved? Allen says not by procedure but by the evidence itself. But if this were the case we still would need to have someone to say what the evidence is, given that there are many research papers which come to (sometimes) different conclusions.
The reason why IPCC reports lack credibility among some observers is that it overstates what it can achieve. It says it uses a peer review mechanism and reviews only peer reviewed research papers--both claims are obviously not true. The IPCC is answering to the UNFCCC and governments appoint its reviewers. Scientific excellence is not the main criterion (in fact, the criteria are opaque), this would be the case if the Academies of Sciences were to appoint a review board. The IPCC does not perform own research but conducts an assessment and evaluation of existing work. Academically speaking this would be a systematic review. But the IPCC does not use this terminology and has no mechanism to ensure that such a review would be conducted by an independent team. Conclusion: if a proper procedure was applied, the results of such a review would be trustworthy.
Allen further notes:
The problem with IPCC’s response to criticisms of previous assessments is that the focus has been entirely on formalizing procedures, whereas the reports ultimately depend on the collective scientific judgment of IPCC authors and reviewers. In some respects, tighter procedures may even have a distorting effect, particularly when coupled with the infrequency of IPCC reports. For example, I am aware of two papers (on one of which I was a co-author) being accepted on the day of the deadline for inclusion in the IPCC 5 th Assessment, both of which provided evidence for a downward revision of the uncertainty range for climate sensitivity. A third paper, providing evidence for a higher value for climate sensitivity, was accepted two days later, and hence could not be included. The fact that the first two papers and not the third contribute to the official consensus view on climate sensitivity for the next six years is clearly absurd.
This is another red herring against proper procedure. If there are deadlines in a review process (in any review process) they will have to be enforced. The alternative would be to have ad-hoc assessments (perhaps as soon as a new paper is published?) which would just create a parallel structure (with a slight time lag) to published research. The question seems to boil down to an appropriate time frame--but not to question the procedure of systematic review.
BTW, Allen has some fairly critical remarks about temperature reconstructions (compare this to Michael Mann’s earlier comments who feels vindicated by the AR5 report):
Likewise, I can also confirm that the particular reconstruction of Northern Hemisphere temperatures over the past millennium (the so-called “Hockey Stick”), which subsequently came in for considerable criticism, was not in any way central to the conclusions of the 2001 Assessment regarding attribution of causes of recent warming. We were aware of these proxy climate reconstructions, but did not give them much weight in the attribution assessment because the statistical methods used were (and arguably still are) rather opaque. I remember specific discussions among the attribution chapter authors questioning the error budgets of those reconstructions, and concluding that it would be premature to rely on them too heavily. With the benefit of hindsight, these discussions seem remarkable prescient, and confirm the importance of scientific judgment in the IPCC process.