notropis

An opinion from an actual expert confirms my problems with this survey.

The Chairman of the Department of Biostatistics and Applied Mathematics at the University of Texas MD Anderson Cancer Center, Donald Berry, has this to say:

"Selecting clusters and households that are representative and random is enormously difficult. Moreover, any bias on the part of the interviewers in the selection process would occur in every cluster and would therefore be magnified. The authors point out the possibility of bias, but they do not account for it in their report."

and

"Incorporating the possibility of such biases would lead to a substantially wider range, the potential for bias being huge. Although there is no formal way to address bias short of having an ‘independent body assess the excess mortality,’ which the authors recommend, the lower end of this range could easily drop to the 100,000 level."

Sort of what I mentioned in my second post, items 4 - 6. But Dr. Berry notices a further problem with the teams' selection bias, and that is that, given that there were only two teams of surveyors, any selection bias will be consistent throughout the survey, skewing all results in the same direction and further widening the true confidence interval.

Also, I don't know if he's an expert or not (I know nothing about him), but Tim Blair had the same thought about the death certificates.