Suppose we’ve trained a system to make medical diagnoses.
The system, perfectly trained, has specific recall and accuracy. (recall: of all the diseases that have been submitted, what percentage did I recognize? precision: of all those that I identified as diseases, in what percentage were they really?)
Accuracy and recall are a short blanket. it does not depend on a “calibration” of the machine, it is not an inherent error. it is a matter of statistical properties.
We know the system, perfectly trained, will make incorrect predictions.
Of course, even aspecialist doctor makes incorrect predictions. For this reason his predictions are according to science …