SYDNEY. Australia’s federal health department has warned that AI scribes used by doctors to record and transcribe patient consultations operate with “little oversight,” raising urgent questions about privacy, patient consent, and whether the tools are illegally functioning as unregulated medical devices.
The warning, contained in federal briefing documents obtained by Guardian Australia, comes as the use of AI scribes has exploded across the country’s primary care sector. An estimated 40 percent of Australian GPs now use the technology, according to polling by the Royal Australian College of General Practitioners (RACGP), up from just 22 percent a year earlier. Popular tools include Lyrebird, Heidi Health, Amplify+, i-scribe and Medilit, among a rapidly growing field of vendors.
The technology is easy to understand as a pitch. An ambient AI listens to a doctor-patient conversation, then automatically generates clinical notes, summaries, and referral letters. Proponents say it cuts administrative burnout, lets doctors look patients in the eye instead of at a screen, and recovers hours of after-hours paperwork each week.
But the health department’s internal assessments paint a more troubling picture. The briefing documents, prepared for Senate estimates hearings in February 2026, warn that many AI scribe products “fall outside” existing regulatory frameworks and are marketed in ways that may deliberately sidestep oversight. Some vendors explicitly advertise their tools as “not a medical device” or “privacy-compliant,” often with limited transparency about how patient data actually flows through their systems.
“The concern is that some suppliers may not even know their cloud platforms send data outside Australia,” one document states, raising the risk that sensitive medical records end up on servers in jurisdictions with weaker privacy protections.
The data exposure is not hypothetical. Privacy policies for some AI scribe vendors disclose that patient information may be shared with employees, third-party suppliers, related companies, government agencies, law enforcement, and transferred overseas. A single patient consultation generates a chain of data transfers through speech-to-text engines, large language model APIs, storage layers, and analytics platforms, each representing a potential leak point.
The Therapeutic Goods Administration (TGA) has flagged an even more fundamental concern: many AI scribes that claim to be simple transcription tools are actually making medical decisions. In a formal statement, the TGA said medical professionals report that AI scribes “frequently propose diagnosis or treatment for patients beyond the stated diagnosis or treatment a clinician had identified during consultations.” Under Australian law, a tool that proposes diagnoses or treatments is a medical device requiring pre-market approval from the TGA. If vendors have been supplying such tools without approval, they may be in breach of the Therapeutic Goods Act.
Patient consent is another gaping hole. Federal documents warn that informed consent requires patients to understand what they are agreeing to, including whether audio is recorded and retained, where data is processed, whether subcontractors are involved, and whether data is used to train AI models. In practice, many patients are simply told an AI scribe will be used without meaningful explanation.
The consumer group CHOICE reports that one parent described the consent process as: “You’re unlikely to know when they ask you to sign over consent. You have no control if your data is hacked.”
AI expert Dr. Kobi Leins canceled a specialist appointment for her child after being told AI would be used, citing concerns about the specific model’s privacy and security features. “There is no need for many of these tools, and fundamentally, we need to ask why they are being pushed so hard,” Leins said. She pointed to a UK-funded study that found Google’s Gemma model downplayed women’s physical and mental health issues compared to men’s in clinical case note summaries, highlighting how gender bias can be baked into AI tools that influence medical records.
The RACGP has tentatively embraced AI scribes as a way to reduce paperwork but warns that the tools are being developed by profit-seeking tech companies without clinician oversight. “Value to technology company shareholders might be prioritized over patient outcomes,” the RACGP wrote in a position statement.
Heidi Health, one of Australia’s most prominent AI scribe providers, told CHOICE that it does not share patient-identifiable information with external parties except where required by law and does not use patient data to train its AI. The company said it is the only AI scribe in Australia certified to ISO 27001, an international information security standard.
But the patchwork of responses from individual vendors is no substitute for a coherent regulatory framework. Currently, oversight is split across multiple agencies. The TGA decides whether a product is a medical device. The Australian Health Practitioner Regulation Agency (Ahpra) handles professional conduct. The Office of the Australian Information Commissioner (OAIC) enforces privacy. No single body owns the full patient experience, meaning a scribe could pass as “not a medical device” yet still be unsafe, or be “privacy compliant” while failing to secure informed consent.
There are also worrying financial incentives. Some AI scribe vendors advertise directly to health professionals that they can achieve a 30 percent revenue increase with no additional hours or patient consultations. The health department has flagged this as a potential driver of increased Medicare Benefits Schedule costs, shifting the technology’s value proposition from clinician burnout relief to billing optimization.
For the future of AI in healthcare, Australia’s experience with AI scribes offers a cautionary tale. The technology has genuine utility and has been adopted faster than almost any other AI application in clinical settings. But that speed has exposed a regulatory system designed for an era when software did not listen, summarize, diagnose, and learn from patient data in real time.
The core challenge going forward will be whether Australia can build a governance framework that matches the pace of AI deployment. Without it, patients are left bearing the risk of opaque data flows, unverified clinical outputs, and a consent process that offers the illusion of choice without the substance of protection.