Variability vs uncertainty

During the Ebola review, we identified potentially a significant issue with the way we were extracting delay distributions. The issue uncovered is that the measure of uncertainty is not sufficiently well characterised, in particular, is the extracted value referring to the variability of the underlying sample (for example the standard deviation of the number of days of observed incubation period) or the uncertainty of the parameter estimate (for example the standard deviation of the estimator itself such as the standard deviation of the posterior means of the incubation period).

During the extraction, we ‘overloaded’ the Parameter uncertainty section of the extraction form and mixed both above types of uncertainty. As part of the Ebola analysis, this caused several challenges and highlighted that we do need to address this problem properly. The good news is that this has not impacted the Marburg paper and very minor impact on the Lassa work (which was taken care of).

To fix this issue, we used an intermediate fix for the SARS-CoV-1 extraction as they were already ongoing, and we were limited in terms of changes we can implement as we do not want to re-extract all the papers we have completed so far (36% complete when we uncovered the problem).

Temporary fix for SARS-CoV-1 extraction

• We added more options for both the Parameter uncertainty – single type and Parameter uncertainty – paired type by adding options to explicitly state if the measure which is being extracted refers to the sample or the estimator, see screen shots below.
• If you have a paper that reports both measures of uncertainty for the sample and estimator, please capture the measure which refer to the sample and notify the SARS leads (Christian & Thom) of the paper.

Longer-term fix

• A longer-term solution was implemented for Zika and will be used for the remaining pathogens. This basically doubled up the number of extracted parameters so for a single parameter (say the generation time, GT) you could extract in a single form:

• under what is labelled "Parameter 1": an estimate for a measure of centrality (e.g. mean GT) + uncertainty in this measure (in the form of a paired interval or a single value, e.g. 95%CI on the mean GT)
• under "Parameter 2": an estimate of population variability (e.g. standard deviation of the GT) + corresponding uncertainty (again, in the form of a paired interval or a single value, e.g. 95%CI on the std of the GT).

• In order to be as generic as possible the two entries above are not labelled as "measure of centrality" and "variability" but rather "parameter 1" and "parameter 2" so that we can also extract information reported in other formats. This could be for example if the GT distribution is parameterised as a Gamma and the shape and scale of the gamma are estimated; then we would extract:

• under "Parameter 1": the shape estimate (central estimate + uncertainty)
• under "Parameter 2": the scale estimate (central estimate + uncertainty)

If you have any questions please reach out to the current pathogen leads.