Artificial intelligence can now read medical scans with accuracy rivaling radiologists, predict disease onset from wearable data, and power virtual assistants that triage symptoms in real time. The clinical promise is enormous. But a deeper question haunts health policy: not whether AI can diagnose cancer, but who controls the data that makes it possible.
Insurance companies, pharmaceutical giants, and financial institutions have powerful commercial incentives to access the torrent of health data flowing through AI systems. And the governance frameworks designed to stop them remain fragmented, inconsistent, and in some places, nonexistent.
A World Report published in The Lancet on July 24, 2026, examining emerging governance frameworks for AI in healthcare delivery, arrives at a pivotal moment. Multiple international bodies, including the World Health Organization and the United Nations, have issued major policy documents this year alone. But the gap between regulatory ambition and real-world protection is where patient data slips through.
“The central tension is almost cruel,” said Dr. Mariana Vos, a health data ethics researcher at the University of Amsterdam who contributed to the WHO’s January 2026 EB158/19 report. “AI in healthcare needs vast datasets to function effectively. But those same datasets contain the most sensitive personal information people possess. Every byte of data that makes an algorithm smarter is also a byte that can be extracted, sold, or repurposed.”
That repurposing is the heart of the concern. A medical imaging AI trained on thousands of chest X-rays from a public hospital system learns patterns that could indicate not just disease, but age, lifestyle, socioeconomic status, and insurance risk. When that model, or the data behind it, finds its way into the hands of an insurance underwriter, a pharmaceutical marketer, or a bank assessing loan eligibility, the original clinical purpose has been quietly subverted.
The WHO’s EB158/19 report, titled “Health Data Governance and Artificial Intelligence,” calls for harmonized regulatory approaches across AI, data, and digital health. It emphasizes data protection, ethical use, and accountability. The report recognizes that the current patchwork of national laws creates safe havens for commercial data extraction, and it urges member states to treat health data as a special category deserving stronger protections than general personal information.
But harmonizing across jurisdictions is proving difficult because the major regulatory blocs have fundamentally different philosophies about data rights and commercial access.
The European Union leads with the most comprehensive framework. The General Data Protection Regulation, in effect since 2018, treats health data as a specially protected category requiring explicit consent for processing. The EU AI Act, which entered enforcement phases in 2025, classifies medical AI as “high-risk,” subjecting it to mandatory conformity assessments before deployment. Together, these instruments create what the UN calls a “human-centric” approach: data subjects retain meaningful control, and commercial secondary use is presumptively prohibited unless specific exemptions apply.
China has taken a different but equally assertive path. Its Personal Information Protection Law, enacted in 2021, classifies medical health information as “sensitive personal information” requiring separate, specific consent for each processing purpose. A single consent form covering both clinical care and algorithmic training does not suffice. The law also imposes data localization requirements, meaning Chinese patient data used in AI training must remain within the country’s borders. This creates a walled-garden approach that protects domestic data but complicates international collaboration.
The United States is the outlier. Despite being home to the world’s largest health AI industry, it has no comprehensive federal law governing health AI or the data it consumes. The Health Insurance Portability and Accountability Act covers clinical data held by covered entities such as hospitals and insurers, but it was written in 1996, long before AI training pipelines or data brokers existed. The Food and Drug Administration regulates AI-powered medical devices on a case-by-case basis, but only when the AI is marketed as a diagnostic tool. An AI model used internally by an insurer to adjust premiums or flag high-cost patients falls outside the FDA’s purview entirely.
The result is fragmentation. A patchwork of state-level privacy laws, California’s CPRA, Colorado’s CPA, Virginia’s CDPA, and others, creates a compliance maze. A 2025 Executive Order on AI in healthcare pushed toward a unified national standard, calling for the Department of Health and Human Services to develop guidance on data transparency, algorithmic fairness, and patient consent for AI training. But that guidance remains in draft, and no federal legislation has passed.
The practical consequences are already visible. Consider medical imaging AI tools, now deployed in hundreds of hospitals worldwide for screening mammograms, chest X-rays, and retinal scans. These systems depend on large, diverse training datasets to avoid bias. But the same datasets, when aggregated with insurance claims or pharmacy records, can generate predictive profiles that extend far beyond diagnosis. An algorithm that detects early-stage diabetic retinopathy can also predict, with surprising accuracy, which patients are likely to file expensive claims in the next five years.
Health monitoring wearables add another dimension. Smartwatches and fitness bands continuously collect heart rate, sleep patterns, activity levels, and blood oxygen data. When these feeds connect to AI-powered health platforms, the data stream becomes a gold mine for insurers seeking to adjust premiums based on behavioral risk scoring. Several major US insurers already offer premium discounts for customers who share wearable data, a voluntary program that critics argue creates a coercive two-tier system where privacy costs money.
Virtual health assistants and smart diagnosis platforms raise similar concerns. Conversations with AI triage bots contain detailed symptom histories, medication lists, and lifestyle disclosures. Under current US law, much of this data is not protected by HIPAA if the platform is operated by a technology company rather than a healthcare provider. The data can be used to train commercial models, improve advertising targeting, or be sold to third parties with only a terms-of-service notice that few patients read.
Insurance underwriting algorithms may be the most high-stakes application. AI systems that analyze claims data, prescription histories, and even social determinants of health to predict future costs are increasingly common in the industry. These models can deny coverage, increase premiums, or steer patients toward specific treatment pathways based on algorithmic judgments that patients never see and cannot appeal. The “black box” nature of many machine learning systems means that even regulators struggle to audit whether decisions are fair or discriminatory.
Automated adverse event reporting, while intended to improve drug safety, also creates new vectors for data exposure. When AI systems automatically scan electronic health records for adverse drug reactions and submit reports to regulators, the process can inadvertently transmit entire patient histories. Data minimization, extracting only the relevant clinical data points, remains an unsolved technical challenge in many deployed systems.
The UN White Paper on AI Healthcare Governance, released earlier in 2026, documents these divergent approaches and warns that without convergence, public trust will erode. “Patients cannot be expected to embrace AI-enhanced healthcare if they fear their most intimate health information will be monetized by third parties they never authorized,” the paper states. It recommends that all jurisdictions adopt three standards: explicit opt-in consent for any non-clinical use of health data, mandatory transparency reporting for AI training datasets, and independent auditing of algorithmic decisions that affect patient access or pricing.
Whether these standards will be adopted remains uncertain. The EU is closest to compliance. China’s approach is strong on consent but weak on independent oversight. The US has neither comprehensive consent requirements nor independent auditing, though the 2025 Executive Order signals federal momentum.
The WHO’s EB158/19 report frames these questions as a matter of global health equity. Developing nations, it notes, often lack both the regulatory infrastructure and the negotiating power to prevent their patient data from being extracted by foreign commercial entities. AI models trained on data from low-resource settings may yield breakthroughs that are commercialized in wealthy countries, with no benefit flowing back to the communities that contributed the data.
The governance challenge for AI in healthcare is not primarily technical. It is not even primarily medical. It is a question of power: who gets to decide how the most intimate data humans generate is collected, analyzed, and monetized. The frameworks being built today, in Geneva, Brussels, Beijing, and Washington, will determine whether AI becomes a tool for patient empowerment or a mechanism for commercial surveillance dressed in the language of healing.