New study: unanimity is often wrong

JuryUnder ancient Jewish law, unanimous conviction in a capital crime was considered as too good to be correct. The absence of dissenting opinions among the judges was considered as a proof that the suspect is very probably innocent. Aside from the heuristic wisdom behind this ancient practice, a new research is giving it a scientific assertion.

In an article called “Too good to be true: when overwhelming evidence fails to convince“, a team of researchers from Australia and France have investigated the subject and had a very interesting finding.

The authors start by stating: “Is it possible for a large sequence of measurements or observations, which support a hypothesis, to counterintuitively decrease our confidence? Can unanimous support be too good to be true? The assumption of independence is often made in good faith, however rarely is consideration given to whether a systemic failure has occurred” to finally conclude: “We find that even with surprisingly low systemic failure rates high confidence is very difficult to achieve and in particular we find that certain analyses of cryptographically-important numerical tests are highly optimistic, underestimating their false-negative rate.