Healthcare AI API Compliance: Engineering PHI Protection Right

Most teams building medical AI products treat compliance as a documentation problem. They wire up Claude Opus 4 or GPT-5 for diagnostic assistance, append "this is not medical advice" to the UI, and call it done. That's not compliance — it's legal decoration. Real healthcare AI API compliance requires hard constraints at three distinct layers: data ingestion, API call handling, and model output. Miss any one of them and the other two don't matter.

The Data Layer: PHI De-identification Belongs in Your Pipeline, Not the Model

Patient health information is tightly regulated. In the US, HIPAA's Safe Harbor method specifies 18 identifier categories that must be removed before data can be considered de-identified — names, sub-city geographic data, dates (birth, admission, discharge), phone numbers, email addresses, SSNs, medical record numbers, IP addresses, device serial numbers, and more. In China, the Personal Information Protection Law and the Data Security Law apply jointly, with their own identifier requirements.

The critical engineering decision: don't let the LLM decide what counts as PHI. GPT-5 or Claude Opus 4 can hit 95%+ PHI detection accuracy in controlled settings. That sounds impressive until you realize a single missed name is a compliance incident. PHI scrubbing must happen before the text ever reaches an API call.

The right approach is a dedicated NER pipeline as a pre-processing gate. Microsoft Presidio is the most mature open-source option for this — 15+ languages supported, actively maintained, and straightforward to integrate:

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def sanitize_before_llm(text: str) -> str:
    results = analyzer.analyze(
        text=text,
        entities=["PERSON", "DATE_TIME", "PHONE_NUMBER", "EMAIL_ADDRESS", "US_SSN"],
        language="en"
    )
    anonymized = anonymizer.anonymize(text=text, analyzer_results=results)
    return anonymized.text

# De-identify first, then build the prompt
clean_text = sanitize_before_llm(raw_patient_note)
response = client.chat.completions.create(
    model="gpt-5",
    messages=[{"role": "user", "content": clean_text}]
)

For Chinese-language records, stack HanLP's medical NER model on top of Presidio to catch national ID numbers, patient card numbers, and other locally-specific identifiers that English-trained models routinely miss.

One more issue teams consistently overlook: API request logs. Many setups retain the full raw patient text in request logs, which is itself a compliance gap. Either configure PHI scrubbing at the API gateway layer, or disable training data usage on the provider side. Both OpenAI and Anthropic offer zero data retention (ZDR) modes for enterprise accounts, activated via a Data Processing Agreement plus the appropriate API header configuration.

The API Layer: BAA Requirements and Model Selection Are Linked

In the US, any third-party service that touches PHI requires a signed Business Associate Agreement (BAA) with the covered entity. Both OpenAI Enterprise and Anthropic's enterprise tier support BAAs. For products targeting Chinese users, the equivalent obligation is ensuring your LLM provider holds a Multi-Level Protection Scheme Level 3 certification and that data residency is domestic.

Model selection is also a compliance decision, not just a capability one:

Dimension	Cloud API (GPT-5 / Claude Opus 4)	Self-hosted (DeepSeek V3 / Qwen 3)
PHI cross-border risk	Requires BAA + ZDR; data egress has regulatory exposure	None — data stays local throughout
Model capability	Flagship-grade reasoning and long-context handling	Near-flagship; some capability gap in specific tasks
Compliance cost	Lower (vendor-backed agreements)	Higher (requires building your own audit chain)
Best fit	Q&A assistance, document summarization, non-diagnostic tasks	Diagnostic support, chart analysis, high-sensitivity workflows

A practical rule: let the sensitivity of the use case determine where the data goes. A patient asking whether ibuprofen and amoxicillin can be taken together is a general health question — cloud API is fine. OCR parsing of a specific patient's medical record, or structured extraction from an imaging report, involves identifiable information and warrants a self-hosted deployment.

The Output Layer: Disclaimer Boundaries Need Engineering, Not Just Copy

The legal weight of "this is not medical advice" depends entirely on whether it actually shapes user behavior. If your UI presents AI responses with the same visual authority as a physician's diagnosis, the disclaimer is wallpaper.

There are three concrete, engineered approaches to building real output boundaries:

Mandatory output labeling via system prompt. Define the output format explicitly and require confidence indicators and care-seeking prompts on every medical response:

system: You are a medical information assistant. Every response must end with:
[Source: medical knowledge base | Confidence: High/Medium/Low | Note: Please consult a licensed physician for any symptoms.]
Do not make specific diagnoses. Do not recommend prescription drug dosages.

Post-processing filters on model output. Run a rule engine over every response before it reaches the user. Flag and intercept definitive diagnostic language or dosage instructions:

import re

FORBIDDEN_PATTERNS = [
    r"you (have|are diagnosed with)",
    r"diagnosis (is|shows)",
    r"take \d+ (mg|milligrams|pills?) (per|each|every) (day|dose)",
    r"you('ve| have) (got|developed)",
]

def post_filter(response_text: str) -> str:
    for pattern in FORBIDDEN_PATTERNS:
        if re.search(pattern, response_text, re.IGNORECASE):
            return (
                "Based on what you've described, we recommend visiting a licensed "
                "medical provider for a proper evaluation."
            )
    return response_text

Visual de-emphasis in the UI. AI-generated responses should use smaller type and lower contrast than primary actions like "Contact a physician" or "Book an appointment." Avoid green checkmarks, "✓ Confirmed" badges, or any visual language that signals authoritative judgment. The FDA's guidance on AI/ML-based Software as a Medical Device (SaMD) explicitly includes UI design in its regulatory review scope — this is not a UX preference, it's a compliance surface.

Auditing: Compliance Is Ongoing, Not a Launch Checklist

A healthcare AI system's compliance posture needs continuous validation. The minimum viable audit setup covers four areas:

• Request-level logging: timestamp, de-identified prompt summary (never raw text), model version, and output filter result for every API call
• Anomaly detection: alert thresholds for high-frequency repeated queries, off-hours access patterns, and bulk record queries from a single account
• Model version pinning: lock the exact model version in production (e.g., claude-opus-4-20260401) to prevent silent provider updates from shifting output behavior away from your reviewed compliance baseline
• Adversarial prompt testing: quarterly red-team exercises to verify your system prompt constraints still hold under injection attempts

On incident response: HIPAA requires reporting a data breach within 60 days of discovery. China's Data Security Law requires notifying regulators within 24 hours for major incidents. Write the runbook before you need it. Having escalation contacts, notification templates, and a documented response sequence ready in advance is far more useful than assembling them under pressure.

---

For teams building at this layer, I use XycAi (https://www.xyc.ai) for API access to GPT, Claude, and 200+ other models through a single OpenAI-compatible endpoint. It carries licensed LLM algorithm filing and supports compliant global invoicing — which cuts through a lot of paperwork when you're documenting your vendor stack for a healthcare operator. Flagship models like GPT-5 and Claude Opus 4 start at around 14% of list price, so the cost pressure to cut corners on model selection largely disappears.