The information displayed in the AIM should not be reported as representing the official views of the OECD or of its member countries.
Ontario's auditor general reported that most AI-powered medical transcription tools approved for doctors produced inaccurate, incomplete, or fabricated medical notes, including hallucinations and privacy risks. Inadequate evaluation and oversight of these systems pose direct threats to patient health and data security across the province.[AI generated]
Why's our monitor labelling this an incident or hazard?
The AI transcription tool is an AI system used to generate medical notes, which directly impacts patient care and health outcomes. The auditor general's report documents that the system was inadequately evaluated, with accuracy issues and hallucinations already occurring during testing and some actual use. This can lead to harm to patients' health (harm category a) and breaches of privacy and security (category c). The lack of proper risk assessments and oversight further supports the classification as an AI Incident. Although the government official disputes the severity, the auditor general's findings and evidence of use confirm realized harm or at least harm that has occurred indirectly through the AI system's outputs. The broader issues with AI chatbot use in the public service also raise concerns but are less directly linked to harm at this stage. Overall, the event meets the criteria for an AI Incident.[AI generated]