il cosidetto “automation bias” ovvero la tendenza a confermare la predizione della macchina è decisamente un problema.
In campo medico (e non solo) gli incentivi per l’operatore sono spesso allineati a confermre la predizione.
Qualche anno fa ho fatto una proposa che chiamavo “prediction poisoning“. Sono sempre piu’ convinto che sara’ necessario. (ed anche eocnomicamente vamtaggioso, BTW).
Source: The Wall Street Journal
The technologies, which can analyze massive amounts of data with a speed beyond human capacity, are making extraordinary advances in medicine, from improving the diagnosis of heart conditions to predicting protein structures that could speed drug discovery. When it is used alongside humans to help assess, diagnose and treat patients, AI has shown powerful results, academics and tech experts say.
At the same time, the tools can be flawed and are sometimes implemented without adequate training or flexibility, say nurses and healthcare workers who work with them regularly, putting patient care at risk. Some clinicians say they feel pressure from hospital administration to defer to the algorithm.
“AI should be used as clinical decision support and not to replace the expert,” said Kenrick Cato, a professor of nursing at the University of Pennsylvania and nurse scientist at the Children’s Hospital of Philadelphia. “Hospital administrators need to understand there are lots of things an algorithm can’t see in a clinical setting.”
In a survey of 1,042 registered nurses published this month by National Nurses United, a union, 24% of respondents said they had been prompted by a clinical algorithm to make choices they believed “were not in the best interest of patients based on their clinical judgment and scope of practice” about issues such as patient care and staffing.” Of those, 17% said they were permitted to override the decision, while 31% weren’t allowed and 34% said they needed doctor or supervisor’s permission.
Continua qui: When AI Overrules the Nurses Caring for You – WSJ
