Presidio accuracy on French documents: a measured comparison

How accurate is Presidio on French documents?

We ran Presidio on the corpus we use to measure AnonymFiles: 41 fictional documents written the way French law firms, notaries and accountants write them (judgments, pleadings, deeds, engagement letters, a payroll export, emails…). Each document comes with the list of items that must disappear (374 in all) and of terms that must stay readable (57: statute references, courts, public bodies). The counting rule is the same for both tools: an item is found if its value no longer appears in the redacted copy, and a term is wrongly redacted if it has disappeared.

A note on emails. The corpus uses domains reserved for examples (.example, .test, per RFC 2606), and Presidio's email recognizer rejects domains without a real public suffix. On the raw corpus it therefore finds only 12 of 21 emails, and its total drops to 309 (82.6%), or 291 (77.8%) at the 0.5 threshold. That would not happen on real files, so we reran the test with .fr domains, in the text and in the expected answers alike; the table shows these corrected figures, which favor Presidio. AnonymFiles scores the same on both versions of the French corpus.

Speed clearly goes to Presidio: about 2 seconds for the 41 documents, against 5 seconds for AnonymFiles in light mode and one to three minutes in full mode on our test Mac (Apple silicon, 8 GB of memory), which runs two heavier language models. For how we measure AnonymFiles itself, see our detailed detection benchmark.

What does Presidio miss, type by type?

• People. Presidio's strong point, driven by the spaCy model. The 9 misses are hard cases: capitalized surnames and lone first names in a payroll table, and a surname mentioned alone later in a document. Presidio does not carry a name it recognized once over to its other mentions.
• Organizations. Companies introduced by their legal form (French "SAS" or "SARL", roughly Ltd or LLC) are often missed or cut short: in 17 of the 44 counted as found, part of the name stays readable.
• Addresses. Presidio has no postal address recognizer: its list of supported entities has none, and it relies on the language model's LOCATION entity. The street name gets redacted, but the house number and street type remain ("27 rue" followed by a tag). Our rule counts these as found, since the full value is gone, yet none of the 61 is fully covered, and 11 addresses go through in clear.
• Phone numbers. All found through the phonenumbers library, France being one of its default regions. But the recognizer's base score is 0.4: set a 0.5 threshold and only 3 of 21 remain redacted.
• Dates. None of the 14 written-out French dates of birth ("3 juillet 1979") was detected. The French spaCy model has only four entity labels (LOC, MISC, ORG, PER), none for dates, and Presidio's date recognizer did not catch them.
• Identifiers. Presidio ships recognizers for the US, the UK, Spain, Italy, Germany and a dozen other countries, but not France. French company numbers (SIREN) and social security numbers are not recognized; the 4 counted as found were covered by chance by a location detection.

Most of the 9 wrongly redacted terms are city names inside court names ("tribunal judiciaire de Bordeaux"): Presidio treats a city as a LOCATION, by design. AnonymFiles avoids these with its default "never redact" list (public bodies, statutes, courts); Presidio supports an equivalent allow_list, which we did not use when measuring its defaults.

What about medical documents and other languages?

Our corpus also includes 4 fictional French medical documents and two documents per foreign language. Presidio used each language's spaCy "md" model, with email domains corrected.

On medical documents the gap widens: Presidio recognized only 2 of the 12 French health identifiers (hospital patient number, practitioner RPPS and ADELI numbers, facility FINESS number, social security number). Outside French, Presidio does much better, which makes sense given its English-first design and its national recognizers. In fairness, AnonymFiles also wrongly redacts the city names in these documents (Köln, Madrid, Roma…), 11 times out of 12, as often as Presidio. Two documents per language show a trend, not a ranking. For health data, see AnonymFiles for healthcare.

What is Presidio, and what is it great at?

Presidio is an open-source Python library for detecting and anonymizing personal data, created by Microsoft and released under the MIT license. The project is moving to an independent, community-governed organization, Data Privacy Stack, and stays MIT-licensed. It has four modules: analyzer (detection in text), anonymizer (replace, mask, encrypt), image redactor (OCR-based) and structured (tabular data).

Its architecture is its main strength. Each recognizer is a plug-in: regular expressions with validation (an IBAN checksum), context words that raise the score, language models (spaCy, Stanza, transformers, GLiNER) or a call to an external service. You add or remove recognizers, set thresholds per entity, and drop it into a data pipeline in a few lines. It is an excellent choice to scrub logs, support tickets or datasets before analysis or training, for teams that want to own and audit every rule, and when throughput matters: 41 documents in 2 seconds in our test.

Its documentation is upfront about the limit: "In its default configuration, it contains recognizers and models for English." For French, it is a starting point, not a finished product.

How can you improve Presidio for French?

Based on the errors we measured, these changes would pay off most. We have not tested them: this is the integration work to plan for.

1. A larger model. We used fr_core_news_md (43 MB) because disk space was tight on our test machine. fr_core_news_lg (545 MB) reports a slightly higher entity F-score (0.842 vs 0.832 in spaCy's published metadata) and would probably do a little better on people and organizations. A transformer or GLiNER model is another option.
2. French recognizers. SIREN and SIRET (Luhn checksum), social security number (mod-97 key), health identifiers, written-out dates, and street addresses (number, street type, name).
3. An allow list so courts, statutes and public bodies stay readable.
4. Per-entity thresholds. A global 0.5 threshold wipes out phone numbers (score 0.4).
5. Consistency across mentions. Presidio does not link "Madame Vasseur" to "Claire Vasseur". To pseudonymize a whole case file with one tag per person, you need to write that logic, plus the mapping table that goes with it.

Presidio or a ready-to-use tool: which should you choose?

Pick Presidio if you have developers, high-volume text flows (logs, databases, tickets) and time to write and test recognizers for your language. Pick a ready-to-use tool if the documents are case files, the people handling them are lawyers or accountants, and every copy must be reviewed before it goes to an AI. More options: best anonymization software compared and choosing pseudonymization software for lawyers.

How did we measure?

Measured on October 19, 2026, on an Apple silicon Mac with 8 GB of memory, Python 3.12.4.

• Corpus. 41 fictional French law-firm documents (374 expected items, 57 terms to keep), 4 medical documents (34 and 6), two documents per foreign language. Everything is invented (IBANs have valid checksums), with deliberate traps: namesakes, capitalized names, names in tables, dates of birth next to statute dates. 45 other items (amounts, docket and land-registry numbers) are left to manual review and counted for neither tool.
• Presidio. presidio-analyzer 2.2.364, spaCy 3.8.16, fr_core_news_md 3.8.0, in a separate environment. AnalyzerEngine built with a French spaCy NlpEngineProvider and supported_languages=["fr"], with its predefined recognizers loaded for French: email, IBAN, phone, URL, IP, MAC, crypto, dates, medical license and the spaCy recognizer. Each document is analyzed on its own at the default threshold (0), then keeping only detections scoring 0.5 or more. Each detected span is replaced by its type in angle brackets, as the anonymizer does by default.
• AnonymFiles. Version 0.6.0, real engine (two local models plus rules in full mode, the names model plus rules in light mode), network off, default "never redact" list on as in the app. Figures are for one document per case file, like Presidio; they are identical when all 41 documents form one case file.
• Counting rule. An item is found if its value, as a whole word and ignoring case, no longer appears in the copy; a term is wrongly redacted if it has disappeared. The rule is lenient, so we also counted partially redacted items: 82 for Presidio (including the 61 addresses), 1 for AnonymFiles in full mode, where only a courtesy title stays readable ("Monsieur" before the tag).

To test Presidio on your own fictional documents: pip install presidio-analyzer==2.2.364, install fr_core_news_md 3.8.0 from spaCy's releases, build NlpEngineProvider(nlp_configuration={"nlp_engine_name": "spacy", "models": [{"lang_code": "fr", "model_name": "fr_core_news_md"}]}) then AnalyzerEngine(nlp_engine=…, supported_languages=["fr"]), and call analyze(text=…, language="fr"). Our measurement script, the raw JSON results of every run and the corpus are available on request at contact@anonymfiles.com.

What are the limits of this comparison?

• It is our corpus. We wrote it and tuned our engine against it, so strong AnonymFiles results there are expected. On documents we have never seen it can miss things, hence the required review before export.
• Presidio is untuned. We measured its stock French setup with the "md" model. A team that adds the recognizers above will do better.
• Text only. The test says nothing about extracting text from Word, PDF or scans.
• Tiny foreign samples. Two documents per language cannot rank two tools.
• Not comparable to other studies. Hexagone AI published a Presidio comparison on 5,000 samples of the public ai4privacy dataset (internal evaluation, 2024), where Presidio scores 74.74% in French. Different dataset, metric and setup: the numbers do not compare with ours. Our take on that product: Hexagone AI alternative.

No tool reaches 100% on every real file, and a pseudonymized copy can still leak through indirect identifiers: see re-identification risk and anonymization vs pseudonymization.