Maloca PM, Pfau M, Janeschitz-Kriegl L, Reich M, Goerdt L, Holz FG, Müller PL, Valmaggia P, Fasler K, Keane PA, Zarranz-Ventura J, Zweifel S, Wiesendanger J, Kaiser P, Enz TJ, Rothenbuehler SP, Hasler PW, Juedes M, Freichel C, Egan C, Tufail A, Scholl HPN, Denk N.
Abstract
Supervised deep learning (DL) algorithms are highly dependent on training data for which human graders are assigned, for example, for optical coherence tomography (OCT) image annotation. Despite the tremendous success of DL, due to human judgment, these ground truth labels can be inaccurate and/or ambiguous and cause a human selection bias. We therefore investigated the impact of the size of the ground truth and variable numbers of graders on the predictive performance of the same DL architecture and repeated each experiment three times. The largest training dataset delivered a prediction performance close to that of human experts. All DL systems utilized were highly consistent. Nevertheless, the DL under-performers could not achieve any further autonomous improvement even after repeated training. Furthermore, a quantifiable linear relationship between ground truth ambiguity and the beneficial effect of having a larger amount of ground truth data was detected and marked as the more-ground-truth effect.