How to Redact PII from LLM Prompts and Outputs

How to Redact PII from LLM Prompts and Outputs

Large Language Models (LLMs) are powerful tools, but they introduce a serious privacy risk: every prompt you send may contain personally identifiable information (PII) such as names, email addresses, phone numbers, Social Security numbers, or financial details. Once that data leaves your environment, it can be logged, used for training, or exposed in a breach. Redacting PII before it reaches the model — and restoring it safely afterward — is a critical safeguard for any production LLM application.

What Is PII Redaction in the LLM Context?

PII redaction is the process of detecting sensitive information in text and replacing it with placeholders or generic tokens before the text is processed by an LLM. For example, a prompt containing "Contact John Smith at john@example.com" becomes "Contact [PERSON_1] at [EMAIL_1]". After the model generates a response, the original values are mapped back into the output so the end user sees natural text, while the LLM provider never sees the raw PII.

This technique is sometimes called pseudonymization because the data is reversible. True anonymization would irreversibly remove the information, but for LLM workflows you typically need to restore the original values in the final response.

Why It Matters

• Regulatory compliance: GDPR, HIPAA, CCPA, and PCI-DSS all impose strict rules on how PII is transmitted and stored.
• Vendor risk: Most LLM APIs log prompts for abuse monitoring, and some reserve the right to use data for training unless you opt out.
• Breach reduction: If a vendor is compromised, redacted prompts contain no exploitable personal data.
• Internal access control: Redaction prevents engineers and analysts who inspect logs from seeing customer PII.
• Trust: Users are more willing to adopt AI features when they know their data is protected.

Approaches to Redaction

There are three common strategies, often combined in production systems:

• Rule-based / regex: Fast and deterministic. Great for emails, phone numbers, credit cards, and SSNs.
• NER-based (Named Entity Recognition): Uses models like spaCy, Presidio, or a fine-tuned transformer to detect names, addresses, organizations, and dates.
• LLM-based: Asks an LLM to identify and replace PII. Flexible but slower and more expensive, and ironically requires sending data to an LLM.

Microsoft's presidio library is one of the most popular open-source options because it combines regex, NER, and customizable recognizers in a single pipeline.

Installing Presidio

pip install presidio-analyzer presidio-anonymizer spacy
python -m spacy download en_core_web_lg

A Basic Redaction Pipeline

The following example shows a complete round-trip: redact PII from a prompt, send it to an LLM (simulated here), then restore the original values in the response.

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig

analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

def redact(text: str) -> tuple[str, dict]:
    """Return (redacted_text, mapping_of_placeholder_to_original)."""
    results = analyzer.analyze(
        text=text,
        entities=["PERSON", "EMAIL_ADDRESS", "PHONE_NUMBER",
                  "CREDIT_CARD", "US_SSN"],
        language="en",
    )
    anonymized = anonymizer.anonymize(
        text=text,
        analyzer_results=results,
        operators={
            "DEFAULT": OperatorConfig("replace",
                                      {"new_value": "[REDACTED]"}),
            "PERSON": OperatorConfig("replace",
                                     {"new_value": "[PERSON]"}),
            "EMAIL_ADDRESS": OperatorConfig("replace",
                                            {"new_value": "[EMAIL]"}),
        },
    )
    # Build a reverse mapping by scanning original spans
    mapping = {}
    offset = 0
    for item in sorted(results, key=lambda r: r.start):
        original = text[item.start:item.end]
        mapping[original] = anonymized.text  # placeholder logic below
    return anonymized.text, mapping

prompt = (
    "Please draft a welcome email for John Smith. "
    "His email is john.smith@example.com and his phone is 555-123-4567."
)
safe_prompt, mapping = redact(prompt)
print(safe_prompt)

The output will look like:

Please draft a welcome email for [PERSON]. His email is [EMAIL] and his phone is [REDACTED].

Building a Reversible Mapping

To restore PII in the LLM response, you need a stable, unique placeholder for each entity so the model can reference it naturally. The replace operator above is not reversible on its own. Instead, use the encrypt operator or generate indexed placeholders manually:

from presidio_anonymizer.entities import OperatorConfig
from cryptography.fernet import Fernet

key = Fernet.generate_key()
cipher = Fernet(key)

operators = {
    "DEFAULT": OperatorConfig("encrypt", {"key": key}),
    "PERSON": OperatorConfig("encrypt", {"key": key}),
    "EMAIL_ADDRESS": OperatorConfig("encrypt", {"key": key}),
}

anonymized = anonymizer.anonymize(
    text=prompt,
    analyzer_results=results,
    operators=operators,
)

# Decrypt any token found in the LLM response
import re
def restore(response: str) -> str:
    tokens = re.findall(r"gAAAAAB[\w\-=]+", response)
    for token in tokens:
        try:
            original = cipher.decrypt(token.encode()).decode()
            response = response.replace(token, original)
        except Exception:
            continue
    return response

Encrypted tokens are ugly and may confuse the LLM. A cleaner approach is to use human-readable indexed placeholders and maintain your own dictionary:

def redact_indexed(text: str) -> tuple[str, dict]:
    results = analyzer.analyze(text=text, language="en")
    # Sort by start descending so replacements don't shift offsets
    results = sorted(results, key=lambda r: r.start, reverse=True)
    mapping = {}
    counters = {}
    for r in results:
        entity = r.entity_type
        counters[entity] = counters.get(entity, 0) + 1
        placeholder = f"<{entity}_{counters[entity]}>"
        original = text[r.start:r.end]
        mapping[placeholder] = original
        text = text[:r.start] + placeholder + text[r.end:]
    return text, mapping

def restore_indexed(response: str, mapping: dict) -> str:
    for placeholder, original in mapping.items():
        response = response.replace(placeholder, original)
    return response

Wrapping an LLM Client

The redaction logic should live in a thin wrapper around your LLM client so every call is protected automatically:

import openai

class SafeLLMClient:
    def __init__(self, model: str = "gpt-4o-mini"):
        self.model = model
        self.analyzer = AnalyzerEngine()
        self.anonymizer = AnonymizerEngine()

    def _redact(self, text: str) -> tuple[str, dict]:
        results = self.analyzer.analyze(text=text, language="en")
        results = sorted(results, key=lambda r: r.start, reverse=True)
        mapping = {}
        counters = {}
        for r in results:
            entity = r.entity_type
            counters[entity] = counters.get(entity, 0) + 1
            placeholder = f"<{entity}_{counters[entity]}>"
            mapping[placeholder] = text[r.start:r.end]
            text = text[:r.start] + placeholder + text[r.end:]
        return text, mapping

    def _restore(self, text: str, mapping: dict) -> str:
        for placeholder, original in mapping.items():
            text = text.replace(placeholder, original)
        return text

    def chat(self, user_prompt: str, system: str = "") -> str:
        safe_prompt, mapping = self._redact(user_prompt)
        messages = []
        if system:
            messages.append({"role": "system", "content": system})
        messages.append({"role": "user", "content": safe_prompt})

        response = openai.chat.completions.create(
            model=self.model,
            messages=messages,
        )
        raw_output = response.choices[0].message.content
        return self._restore(raw_output, mapping)

client = SafeLLMClient()
answer = client.chat(
    "Summarize the complaint from Alice Johnson "
    "(alice.j@corp.com) about order #4821.",
    system="You are a helpful support assistant.",
)
print(answer)

Handling Edge Cases

Redaction is not a solved problem — there are several pitfalls to watch for:

• Partial matches: A regex for phone numbers may catch dates or ZIP codes. Tune entity lists and confidence thresholds.
• Context-dependent PII: "Apple" could be a company or a fruit. NER models sometimes misclassify. Add custom recognizers for domain-specific terms like patient IDs.
• Leaked PII in output: The LLM may invent new PII-like strings (e.g., a fake email). Run the analyzer on the response too if you want to guarantee no PII leaves your system.
• Placeholder collisions: If the user's original text already contains <PERSON_1>, restoration will corrupt it. Use a unique prefix unlikely to appear naturally.
• Non-English text: Presidio's default NER model is English-focused. Load a multilingual spaCy model or language-specific recognizers for other locales.
• Performance: NER on every prompt adds latency. Cache results for repeated prompts and consider running redaction asynchronously for batch workloads.

Adding Custom Recognizers

For organization-specific identifiers such as employee IDs or account numbers, add a custom pattern recognizer:

from presidio_analyzer import Pattern, PatternRecognizer

employee_id_pattern = Pattern(
    name="employee_id",
    regex=r"\bEMP-\d{6}\b",
    score=0.95,
)
employee_recognizer = PatternRecognizer(
    supported_entity="EMPLOYEE_ID",
    patterns=[employee_id_pattern],
)
analyzer.registry.add_recognizer(employee_recognizer)

Now any string matching EMP-123456 will be flagged and redacted like built-in entities.

Best Practices

• Redact at the boundary: Apply redaction as close to the user input as possible, before any logging, caching, or telemetry.
• Never log the mapping: The placeholder-to-original mapping is the keys to the kingdom. Keep it in memory only, or encrypt it at rest with strict access controls.
• Combine with provider controls: Use zero-data-retention APIs, opt out of training, and enable enterprise tiers that contractually prohibit logging.
• Test with realistic data: Build a test suite of prompts containing every PII type you expect, and assert that none leak through.
• Monitor drift: As your prompts evolve, new PII types may appear. Re-run evaluation regularly and update recognizers.
• Layer defenses: Redaction plus output filtering plus access controls plus audit logging together provide defense in depth.
• Document the trade-offs: Redaction can degrade model quality when placeholders remove context. Measure accuracy with and without redaction on representative tasks.

Conclusion

Redacting PII from LLM prompts and outputs is a foundational privacy control for any production AI system. By combining rule-based and NER-based detection, wrapping your LLM client so redaction is automatic, and carefully managing the reversible mapping, you can dramatically reduce the risk of exposing sensitive data to model providers and downstream logs. No single technique is perfect, so pair redaction with provider-side controls, output scanning, and strong access governance. With a thoughtful pipeline and ongoing testing, you can deliver AI features that are both useful and respectful of user privacy.