Detector catalogue

Every detector that ships, what it matches, how it is validated, and - for the low-precision ones - an honest statement of when it will be wrong.

No detector claims zero false positives. Each entry states its precision posture.

Classification, rule, element, detector and policy are different things

The names are deliberately not interchangeable:

A classification is the stable tenant-visible object a policy will name, such as Finance: PCI DSS. It has its own id, semantic source version and monotonic tenant revision.
A classification contains rules combined with OR. Matching any complete rule classifies the file.
A rule contains typed elements combined with AND. Every element in that rule must match. An empty draft rule is false, never a rule that matches everything.
A detector is one implementation primitive - regex, keyword automaton, EDM set, fingerprint corpus and so on - that may satisfy an element. It reports evidence; it is not the business grouping.
A classification verdict has three states: matched, not_matched and not_evaluated. Missing indexes, detector errors, timeouts, truncation and partial extraction never become a clean negative. A complete positive rule can still prove a match; otherwise OR/AND aggregation preserves the unknown state and names only the affected detector ids, not content or matched values.
A policy action is the separately authored response after a condition matches: audit, warn, justify, block or quarantine within endpoint capability. Classification never implies an action by itself.

0026_classification_hierarchy.sql migrates the catalogue without rewriting it: every pre-existing detector becomes a classification with the same id and one rule containing one detector-reference element. Existing policy conditions therefore keep their exact result while later authoring can grow a classification into several rules and elements. A signed bundle carries both the detector definitions and this hierarchy; the former execute, the latter preserves the stable policy-facing structure.

Version coordinates

Three counters answer different audit questions and are never substituted for one another in the API or console:

Definition version (v1.5.0) identifies detector or classification semantics. Findings keep the detector version that actually evaluated them, even after the current definition advances.
Content revision (r8) is the tenant-local immutable history coordinate for a classification or custom detector edit. Built-in detectors have no tenant content revision; their semantic version belongs to the product release that shipped them.
Bundle version identifies the signed deployment artifact. It says what endpoints received, not what a detector means.

Quality figures are keyed to the exact detector definition version. A result from v1.0.0 is historical evidence and never makes current v1.1.0 read as tested.

How a detector reaches a verdict

The data categories a detector may report

Seventeen values, and the wire string is what an event carries. They drive the sensitivity weight in the risk model and the dashboard's category breakdown, which is the whole reason a category that means nothing costs something: a tender price sheet, an as-built drawing of a bridge and a network diagram used to arrive as the same confidential, so a policy could not treat them differently and a report could not count them apart (DLP-421).

Wire string	What it means	Weight
`credential`	Keys, tokens, passwords	1.00
`health`	Health data	0.95
`financial`	Account, payment and taxpayer identifiers	0.90
`security_assessment`	Findings, penetration tests, vulnerability registers - a weakness nobody has fixed yet	0.90
`network_topology`	Network and ICS topology. The one class whose disclosure is directly a map for an attacker	0.85
`pii`	Personal/contact data and national identifiers, excluding taxpayer identifiers	0.80
`engineering_design`	Drawings, models, structural calculations, design packages	0.80
`intellectual_property`	Trade secrets, proprietary material	0.75
`infrastructure_asset`	An asset register: what exists, where, in what condition, worth what	0.75
`procurement`	Tender references, bid prices, bills of quantity. Sensitive on a clock - most of it is published the week after an award	0.75
`confidential`	Marked confidential, kind unstated	0.70
`contract_commercial`	The commercial terms inside a contract: values, rates, payment schedules	0.70
`geospatial`	Coordinates, survey points, GIS layers	0.70
`legal_contract`	The contract as an instrument	0.60
`source_code`	Source code	0.55
`internal_correspondence`	Internal correspondence. The largest ordinary class in any organisation, weighted so it does not drown the rest	0.45
`other`	Unrecognised, including a category this build has never heard of	0.40

Two rules govern adding one. Every name is a property of a document, never of a person or an activity - a category that describes what somebody was doing is a different kind of record with a different justification, and none belongs here. And an older agent must survive a newer category: the contract's other arm is a serde fallback, so an agent one release behind reads an unknown value as other and keeps evaluating rather than rejecting the bundle and going quiet. That is asserted in packages/contracts, along with a test that reads the console's own copy of this list and fails when the two disagree - which they had, silently, with the authoring screen offering three values the contract has never had.

Taxpayer identifiers are deliberately financial, while national/person identifiers are pii: VÖEN, VKN and INN identify a taxpayer in fiscal records; FIN, TCKN and SNILS identify a person. This distinction is also a stable policy contract. Reclassifying a shipped detector in place would silently stop a policy scoped to its former category from matching after an endpoint update. The built-in catalogue therefore tests this mapping explicitly.

Category, severity and sensitivity labels are separate

Data category says what kind of data matched. Its documented weight is the data_sensitivity factor, worth 20% of the explainable incident risk model; the most sensitive matched category supplies that factor.
Detector severity is the five-step default operational urgency (info, low, medium, high, critical). A composite classification has no independent sensitivity field; its Overview row shows the highest default severity among its detector elements and labels it Default severity.
Restricted / Confidential / Internal are customer-owned labels observed on content. They are policy inputs and are never inferred from detector severity. For example, a high detector is not silently converted into a Restricted label.

Policy severity and the final incident risk score remain separate outputs: a policy supplies the severity of its decision, while the risk score combines category sensitivity with confidence, volume, channel, destination and the other versioned factors in packages/policy-engine/src/risk.rs.

Core pack - `core` v1.0.0, locale-independent

Financial

Detector	Matches	Validator	Base → typical confidence	Redaction
`core.payment_card`	A number in a real scheme's issuer range (IIN) at a length it issues: Visa 4 (16, 19), Mastercard 51-55 and 2221-2720 (16), Amex 34/37 (15), Diners 300-305/36/38/39 (14), and 16-19 for Discover, JCB, UnionPay, Maestro, Mir and Troy (DLP-1376). Grouped in fours, or 4-6-5 and 4-6-4 for Amex and Diners; a space, hyphen or dot between groups, mixed freely. Not a card: groups separated by a table's column gap of three or more blanks, or a separated number with a group of eight or more digits, a decimal or two ids side by side (DLP-1627: 16 false cards in 1,000 GovDocs1 files fell to 5); a card deliberately split across comma cells or wide blanks is therefore not joined. Groups separated by a tab, anywhere in the run of blanks ("2463 \t2468", DLP-1742), are a card only beside a word about cards (DLP-1750): "Card number: 4xxx<TAB>xxxx<TAB>xxxx<TAB>xxxx" is one, typed or pasted from a sheet whose groups sat in cells, and the same tabs alone are a sheet's columns. A 14-, 17-, 18- or 19-digit number counts only with a word about cards within the context window (card, PAN, CVV, expiry, a scheme's name and their Azerbaijani, Turkish and Russian forms). Six of seven card files in GovDocs1 threads 000 and 001 were such runs with no such word: district codes, PDF table rows squeezed into one run, and dated record ids (DLP-1637). A 19-digit card with no word near it is therefore missed. Nor is a stretch of a row of numbers a card without such a word: a plain number beside it on the same line, or its last digits running on into a decimal (DLP-1657: a mesh table's node ids, a flow table's cells, a chart axis and a coordinate list in GovDocs1 batch 004). A card listed one to a line, or written beside a date or a name, is not a row. A card in a space-separated dump beside its expiry as "0925" and no word about cards is missed. Nor is a line where an expiry, `MM/YY` or `MM/YYYY` with a real month, follows the card within 24 characters: that is a card record, so a fixed-width dump with an id column before the card ("1001 4xxx xxxx xxxx xxxx 09/29 J.SMITH") is read (DLP-1662). A decimal with no whole part ("-.26667", DLP-1721) and two signed decimals printed with no gap ("160.34998-64.13333") count as a number beside it; a date or a phone number does not. Nor, without such a word, is a candidate whose own groups ascend within a thousand of each other ("2655 2660 2895 2910"): river miles, years and page numbers read that way, and about one real card in six thousand does too (DLP-1707: a star catalogue row and river miles in GovDocs1 thread 019). Groups within a hundred of each other in any order read the same way ("4491-4492-4476-4475", room numbers). So do groups in equal steps either way ("4000 3000 2000 1000", a chart axis's ticks, DLP-1757), which a card's independent groups form about once in a hundred million. Nor, without such a word, is a number among at least ten of its shape that cannot be cards (outside every scheme's issuer range at that length), ten times as many as pass Luhn: an FDIC list's number columns, a column of 15-digit ids, a binary's digits (DLP-1714, GovDocs1 threads 010-013). Look-alikes inside an issuer range do not count against a card, so padding a file with failing look-alikes hides nothing (DLP-1655). Nor, without such a word, is a value inside an HTML tag whose digits recur four or more times in the page: an id a page writes into every link, as GovDocs1 043832.html's Oracle APEX session id was (DLP-1733). A card in one hidden field is still read, which is why the source is scanned at all (DLP-1526). Nor is a number written right after a unit of measure (ug/l, mg/kg, ppm, %, psi and a few more, named one by one): a lab table's "ug/l 4100 3400 5130 1670" (DLP-1738). Nor, without such a word, are two decimals side by side, a candidate whose separators are exactly ".", " ", ".": a drawing's coordinates "2232.0254 1814.2793" (DLP-1741), and two ranges side by side, "- -": "2500-3300 1400-3000" (DLP-1746). Mixed separators in general still count, since "4111-7391 0004.1111" is a card retyped to get past a scanner (DLP-1372)	Luhn (ISO/IEC 7812)	0.55 → 0.80 with context	last 4
`core.iban`	ISO 13616 IBANs	MOD-97-10 plus registered country length; since 1.1.0 (DLP-1656) not across three or more blanks or a tab (a fixed-width table's columns, which joined into an IBAN that passed MOD-97 by chance) and not in mixed case (base64), each judged on the span after an over-capture is cut back. Since 1.2.0 (DLP-1658) an IBAN from a country outside the validator's table, which is inconclusive rather than failed because the table is a subset of the registry, is reported only beside a term such as `iban`, `account`, `hesab` or `bank`, or under such a column header: an XML section id `HC557EA5...` passed MOD-97 and was reported at High severity. Since 1.3.0 (`DLP-1801`) groups may be joined by hyphens as well as spaces (`AZ45-NABZ-1822-...`), and a last short group is taken whole or not at all: an optional trailing space before a Cyrillic word made the span one character longer than `az.iban`'s, so `core.iban` won the overlap on `AZ60... Иванов`. Check digits run 02 to 98 as ISO 13616 says (`DLP-1803`): `AZ00 0000 ...` passed MOD-97 and is no IBAN	0.60 → 0.85	last 4

core.payment_card checks the issuer range in the pattern, then Luhn. Luhn alone passes one random number in ten, so without the range an IMEI, a SIM ICCID, a millisecond timestamp and an EAN-13 were each reported as a validated card on a Windows host. A 13-digit Visa is not matched: none has been issued for decades, and an EAN-13 starting with 4 has the same shape. imei, iccid, ean, gtin, epoch and timestamp nearby lower the confidence. Since 1.2.0 (DLP-1466) a card cannot follow a decimal point or a digit, so the 16 fraction digits of a double (0.1844114959239960) are not one. A model file shipped with a Windows audio driver raised a high card incident with 146 such matches. An exponent (...0706e-003) already failed the closing boundary. It additionally rejects all-same-digit sequences.

A number that fails Luhn is dropped as written, never shortened until a part of it passes (DLP-1551). The engine trims trailing groups from a failed candidate for detectors whose pattern can over-capture, such as IBANs. For a card, the groups are the number, and twelve of a mistyped sixteen pass Luhn one time in ten, so 500 mistyped grouped numbers had read as 55 cards. Digits written straight after an IBAN's head (DE12 or DE12) are that IBAN's account field, not a card, even when the IBAN fails its own check (DLP-1548). This is engine behaviour that ships with the agent, not a change to the definition, so the detector's version is unchanged.

Any Unicode dash written between two digits is read as a hyphen (DLP-1555): the en and em dash, figure dash, horizontal bar, hyphen and minus sign. Word's AutoFormat and PDF text layers write 4111–7391–0004–1111 for a typed hyphen. The fold is in the text normaliser, so it applies to every detector, and only between digits, so a dash in prose stays a dash. A slash, an underscore and a semicolon are not card separators: a slash is a date's, and an underscore joins a token (DLP-1362).

Since 1.3.0 (DLP-1509) the networks' and payment providers' published test numbers are on the detector's allowlist and are not reported, in any grouping: 4111 1111 1111 1111, 4242 4242 4242 4242, 5555 5555 5555 4444, 3782 822463 10005, 6011 1111 1111 1117 and the rest of builtin::TEST_CARD_NUMBERS (39 numbers from Stripe, Braintree, PayPal, Adyen and the networks). Developer folders, QA fixtures and payment-SDK guides are full of them, and each raised a high-severity incident at 1.0 that buried the real ones. One digit off is a card again. A word like "test" or "example" beside a number lowers its confidence and never sets it aside, because a word that did would let anyone carry a real card past the endpoint by typing it. Maestro numbers shorter than 16 digits are not matched, like 13-digit Visa: a 12 to 15 digit run starting 6759 or 5018 is far more often an identifier. A test or a demonstration of the product therefore needs a Luhn-valid number that is on no published list, such as 4111 7391 0004 1111. 0000000000000000 and 4444444444444444 satisfy Luhn arithmetic but are padding or test data in practice, and treating them as live cards is a reliable false-positive source.

core.iban returns Inconclusive, not Validated, when the checksum passes but the country is outside the registry table - the product does not claim more certainty than it has.

Contact and identity

Detector	Matches	Precision posture
`core.email`	RFC-5322-ish addresses, Unicode local parts	High. `\w` is Unicode-aware, so `rəşad@example.az` is matched rather than silently skipped
`core.phone_e164`	International-format numbers; since 1.1.0 (DLP-1653) not a `+` glued to a letter, digit, `)`, `]` or `%` before it, which was arithmetic (`getTime()+315360000000`), URL encoding (`%3A+96001654016`) or E-notation (`2.321993+04125`) in 12 of the 14 GovDocs1 batch-003 files it fired in	Moderate. The shape alone reaches the threshold (base 0.5, threshold 0.35), so a number with no label beside it is reported; a term such as `tel` or `phone` raises confidence. Requiring a term was decided against, because an unlabelled contact list is real PII. Since `DLP-1803` the country code never opens with 0 (`+0123456789` is not a number)
`core.ipv4` / `core.ipv6`	Bounded octets; for IPv4 since 1.1.0 (DLP-1462), not within 32 characters of a version word (`version`, `ver`, `DriverVer`, `ModuleVersion`, `FileVersion`, `ProductVersion`, `AssemblyVersion`, `assemblyIdentity`, `build`, and others) or of a test marker. One Windows system file in six held a version number that was reported as a validated address. Only the matches beside such a word are set aside; an address elsewhere in the file is judged on its own. Since 1.2.0 (DLP-1650) not an enzyme or chapter number (`EC 3.2.1.23`, `Chapter 8.3.1.4`; also `section`; since 1.3.0, DLP-1659, also `E.C.`, `paragraph` and `release`; since 1.5.0, DLP-1955, also `standard` and `clause`, as in "detection standard 5.1.5.1"), not four parts of a longer dotted run (`2.1.3.4.5.6.1`, `E.C.3.1.1.1`), and not an outline section whose parent stands alone in the same document (`1.5.1` above `1.5.1.1`). A host list never prints its network alone, and `10.0.0.x` or `10.0.0/24` does not count as one. In GovDocs1 batch 003 this took IPv4 from 33 files to 21, 13 of them with a real address and no real address lost; the 8 left are metadata that skips outline levels (`1.5.2.1` with no `1.5.2`), chart labels and a chapter word split by PDF letter spacing / an IPv6 address's own forms: all eight groups, or groups either side of one `::`, or groups ending in `::` (DLP-1319; until then any two colon-separated hex groups matched, so every clock time and MAC address counted, 1462 files on one endpoint's logs). With one group either side of `::` the first needs three hex digits or more (`fe80::1`), and a form ending in `::` needs two groups (`2001:db8::`), so a scope operator (`A::B()`, `std::`) or a fusion name (`pAtPRXII-D::GUS`) is not an address (DLP-1631: 26 per 1,000 GovDocs1 files had no address in them). Since 1.1.0 (DLP-1709) only an address from an assigned range counts (`ipv6_identifying`): global unicast 2000::/3, unique local fc00::/7, link-local fe80::/10, multicast ff00::/8 and NAT64 64:ff9b::/96. In 7,932 GovDocs1 files the shape matched about 1,500 times in binary noise (`008::5`), number lists (`1001::0`) and component ids (`0:0:0:0:0:0:0:0`), and twice at an address, `3ffe:ffff:100::1`, which is kept. Since IPv6 1.2.0 (`DLP-1735`) a group of two or three characters opening with `0` is no address's spelling (canonical text drops leading zeros, the full form pads to four): a log's timestamps, `2007:03:16:16:00:0:0:0`, were 139 addresses in one GovDocs1 file. IPv4 since 1.4.0 (`DLP-1725`): a browser's version in a user-agent is not an address, `rv:1.8.1.6` by the word and `Firefox/2.0.0.6` or `Chrome/120.0.6099.109` by a name and `/` directly before it; a name that is itself after a `/` is a directory, so `/srv/backups/10.0.0.5` is still an address	High for IPv4. Informational severity. Loopback (`127.0.0.0/8`), the unspecified address and broadcast are dropped (DLP-266): they identify nothing and appear in every deployment guide, and `127.0.0.1` alone fired nine times across two files of this repository's prose. RFC1918 is kept - an internal address is real information about an estate, and excluding it is a customer's decision made with the detector's `allowlist` rather than a correctness one made here
`core.passport_mrz`	Two-line, 44-character ICAO 9303 TD3 machine-readable zone The zone's lines may be separated by spaces, tabs or line breaks, and by simple HTML tags, so a zone split across paragraphs, spreadsheet cells or HTML elements is still found (DLP-2251).	High after validating the document number, birth date, expiry, optional-data when present, and composite check digits; a shape with a wrong or missing check digit is discarded
`core.id_card_mrz`	An identity card's ICAO 9303 machine-readable zone from any issuer: TD1 (three lines of 30) or TD2 (two lines of 36) (DLP-1511) The zone's lines may be separated by spaces, tabs or line breaks, and by simple HTML tags, so a zone split across paragraphs, spreadsheet cells or HTML elements is still found (DLP-2251, DLP-2326).	High after validating every field's check digit and the composite, as the passport's; ICAO's own specimen is found, and the same zone with one digit changed is not

The passport layout and composite positions follow ICAO Doc 9303 Part 4; the character values and repeating 7, 3, 1 weights follow Part 3.

Credentials - all redact fully

Detector	Matches	Validator
`core.private_key`	PEM private key headers (RSA, EC, DSA, OpenSSH, PGP, encrypted) where a key begins: at the end of a line, in a quoted or escaped string, or before base64 on the same line. Since 1.1.0 (DLP-1758) a header quoted inside a sentence ("starts with -----BEGIN RSA PRIVATE KEY----- and ends with ...") is not a key. A key saved as a file, id_rsa, .pem, .key or .txt, is read as text since DLP-1734; an OpenSSH or passphrase-protected key, whose body cannot be opened, no longer marks its file as not fully inspected	None needed - the header where a key begins is conclusive
`core.aws_access_key`	`AKIA`/`ASIA`/`AIDA`/`AROA` + 16	Allowlists AWS's own documentation key. Since 1.1.0 (`DLP-1720`, `aws_key_id`) the sixteen are base32 (A-Z, 2-7), so a 0, 1, 8 or 9 fails, and sixteen amino-acid letters alone need an AWS word: GovDocs1's BLAST alignment had `AIDAAIATAAALTVVEPTGC`, a protein. A random real id is spelled that way about one time in 1,800, and is still found beside `aws`, `iam` or `access_key`. A key whose sixteen use four or fewer distinct characters (`AKIAXXXXXXXXXXXXXXXX`, `AKIA0000000000000000`, `AKIAABCDABCDABCDABCD`) is a placeholder and fails (`DLP-2261`; the count was two until `DLP-2288`, which set it at four: a random id has that few distinct characters about one time in 8 billion)
`core.github_token`	`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, and since 1.2.0 (DLP-2150) fine-grained `github_pat_` + 22 + `_` + 59, which GitHub issues by default and which a policy reading this detector now covers. A fine-grained token has no checksum and is reported unvalidated. GitHub token (DLP-1377): a body of four or fewer distinct characters is a placeholder and dropped; a 40-character token whose base62 CRC32 holds is validated; one whose checksum does not hold is dropped (DLP-1473): the prefixed format and its checksum arrived together in 2021, and an older token is 40 hex digits with no prefix, which this pattern never matches. A body of another length has no defined checksum and is reported unvalidated
`core.mydlp_token`	This product's own tokens: `mydlp_api_`, `mydlp_sess_`, `mydlp_enroll_`, `mydlp_invite_`, `mydlp_reset_`, each with its 18-hex id and 43-character secret (DLP-1354). Another product's tokens, such as a tracker's `erp_pat_`, are not in the catalogue: a tenant that uses one adds a custom detector for its format	-
`core.google_api_key`	`AIza` + 35	-
`core.slack_token`	`xoxb-`, `xoxp-`, `xoxa-`, `xoxs-`	-
`core.jwt`	Three base64url segments	Header must decode to JSON containing `alg`
`core.db_connection_string`	A connection string with an inline password: a URI (`postgres://user:pw@host`), the ADO.NET keyword form (`Server=...;Password=...`, either order, `Pwd=` too) and JDBC (`?password=`, `;password=`, Oracle thin `user/pw@host`) (DLP-1361), and a Spring `spring.datasource.password` key in a properties or flat YAML file, a literal value (DLP-2147; the nested YAML form is not matched). `Integrated Security` and a placeholder value (`${...}`, `%...%`) are not matched. A password of mask characters only (`********`, `xxxx`, `•••`, `####`) is a redaction, not a secret, and is not matched (`DLP-2261`)	-
`core.password_assignment`	`password=`, `api_secret:`, `access_token=`, a quoted JSON key, and a prefixed key (`db_password =`, `APP_SECRET=`, `REFRESH_TOKEN=`). A bare `token =` is not matched. Since 1.3.0 (DLP-1512) the value must be a literal: quoted, or one unquoted token that ends the line or meets `;`, `,` or a comment, and never opening with `$`, `[`, `{` or `<` or holding whitespace, `(`, `)` or `$`; code that assigns a variable, a call or a member (`$password = ConvertBase64ToPlainText ...`, every PowerShell module that handles a credential) and a sentence are not matched, and a passphrase with spaces is given up for them. Only the value is reported (DLP-1380), so a credential a specific detector knows (`api_key = AKIA...`, `SLACK_BOT_TOKEN=xoxb-...`) is reported as that credential, not as an assignment. Since 1.4.0 (DLP-1546) a key may be a camel or Pascal case name ending in the word (`DbPassword`, `adminPassword`, `SqlAdminPwd`), case-sensitive at the join so `xpassword` is still a word; a .NET appSettings entry (`<add key="SmtpPassword" value="..."/>`) and PowerShell's `ConvertTo-SecureString "<literal>" -AsPlainText` are assignments of their literal. Since 1.5.0 (`DLP-1710`) the value must be a literal (`credential_literal`): not an absolute path (`PWD=/home/tmp/job`, the working directory), not a dotted reference (`document.form.password.value`), and a value a query string runs on past is cut at the next `&name=` (`Passwd=demo&PID=...` is judged as `demo`, a placeholder). Since 1.6.0 (`DLP-1717`) a sentence's closing full stop does not hide a dotted reference (`password: www.eAuth.egov.usda.gov.`), and a template's placeholder is not a literal: a value beginning `your` and ending `password`, `pass` or `pwd` (`YourDBPassword`). `Yourself2026!` is still read. Since 1.7.0 (`DLP-1724`) a value that is itself an assignment (`AUTH_TYPE=NTLM`, the line after an empty `AUTH_PASSWORD=`) is not one; base64's padding (`dGVzdA==`) is still a value. Since 1.8.0 (`DLP-1731`) a value cut at a query tail keeps the pattern's floor of eight characters: `Passwd=&PID=...`, a masked field, was reported as `` 72 times in one GovDocs1 log. Since 1.9.0 (`DLP-1736`) `anonymous`, a public service's anonymous login, is a placeholder. Since 1.10.0 (`DLP-1739`) a request line's protocol ends an unquoted value too, so a credential in a web or proxy access log (`"GET /login?user=x&password=... HTTP/1.1"`) is found; from 1.3.0 until then it was missed. Any other blank still does not end one, so `password: secret123 and` in prose stays unread. Since 1.11.0 (`DLP-1751`) a keyword argument's bare name is not a value: an unquoted identifier whose key directly follows `(` or `,` (`def Protect(self, Password=defaultNamedOptArg, ...)`, 21 times in one generated COM wrapper), since code quotes a literal. Since 1.12.0 (`DLP-1752`) that holds only inside a call's or a definition's own parentheses (a name directly before the `(`): in 1.11.0 it also dropped real passwords after a comma or parenthesis in prose and comma-separated config (`user=admin, password=Kx7mQ2pLz9r`). A closing parenthesis and a spaced dash now end a value too (`login (password=...)`, `password=... - please change it`). Since 1.13.0 (`DLP-1755`) the key may be Azerbaijani, Turkish or Russian: `şifrə` (and `sifre` without its letters), `şifre`, `parol`, `parola`, `пароль`, with the dotted capital `İ` of `ŞİFRƏ` matched as such. Since 1.14.0 (`DLP-1761`) also with the possessive a new-account e-mail uses: `şifrəniz`, `şifrən`, `şifrəsi`, `Şifreniz`, `parolunuz`, `parolu`, `parolanız`, `parolası`; `Şifrənizi unutmusunuz?` is not one	-
`core.webhook_url_secret`	Slack and Discord webhook URLs	-
`core.aws_secret_key`	An AWS secret access key (40 base64 characters) beside its name: `aws_secret_access_key`, `AWS_SECRET_ACCESS_KEY`, `SecretAccessKey` (DLP-1510). Only the value is reported; 40 base64 characters with no name beside them are not matched	Allowlists AWS's documented example secret
`core.azure_storage_key`	An Azure storage account key (86 base64 characters and `==`) after `AccountKey=` in a storage connection string, or beside `storage_account_key` / `azure_storage_key` (DLP-1510)	Allowlists the Azure storage emulator's published development key
`core.stripe_key`	Stripe live-mode secret and restricted keys, `sk_live_` and `rk_live_` (DLP-1510). Test-mode keys move no money and are not matched	-
`core.dev_platform_token`	npm access tokens (`npm_` + 36) and GitLab access tokens (`glpat-` + 20) (DLP-1510). Since 1.1.0 (DLP-2150) GitHub's fine-grained `github_pat_` tokens are `core.github_token`'s, so a policy that reads GitHub tokens covers them	-
`core.bearer_token`	An `Authorization: Bearer` header value of 20 or more characters with no `_` or `-` (DLP-1510). High rather than critical. A token in a format another detector knows opens with a word and `_` or `-`, so it is that detector's finding; a JWT is `core.jwt`'s	-
`core.url_password`	A password inside an http(s), git, ftp(s), sftp, ssh or smb URL, `scheme://user:password@host` (DLP-1510). The whole credential is the match, so the email detector's reading of `password@host` as an address loses to it. A placeholder (`${...}`, `%...%`) is not matched; database schemes are `core.db_connection_string`'s. Since 1.1.0 (`DLP-1754`) the user part holds no comma: `http://www.wispon.com,mailto:wispon@theory...`, two addresses in a crawler's user-agent, was read as user:password@host	-

core.password_assignment is the weakest of these: base confidence 0.5 with an extended negative list (changeme, your_password, ${, {{, redacted). Configuration templates will still produce occasional hits.

A test asserts that every credential detector uses RedactionMode::Full.

Organisational

core.confidential_marking matches confidentiality markings in four languages:

English - confidential, strictly confidential, internal use only, restricted distribution, restricted circulation, not for distribution, and the two-part banners "Proprietary and Confidential", "Confidential and Proprietary", "Proprietary & Confidential" and "Confidential & Privileged" (in title case, alone on a line, as well as in capitals)
Azerbaijani - məxfi, ciddi məxfi, konfidensial, konfidensial məlumat, konfidensial informasiya, daxili istifadə üçün, xidməti istifadə üçün (and xidmeti istifade ucun), kommersiya sirri, dövlət sirri
Turkish - gizli, çok gizli, kuruma özel, hizmete özel, ticari sır, kişiye özel, devlet sırrı
Russian - конфиденциально, строго конфиденциально, для служебного пользования, коммерческая тайна, государственная тайна, совершенно секретно, секретно (DLP-2368, as a word like the grades: in lower case it is also the adverb "secretly"), and ДСП only as a banner (alone on its line, or after "Гриф:"), because elsewhere it is an abbreviation, most often for chipboard

The trade-secret, state-secret and "kişiye özel" terms (DLP-1818, 1.13.0) are words, not phrases: in capitals anywhere, and in any case only as a banner or a label. Each also opens sentences in law and contracts ("Ticari sır, ... anlamına gelir"), where it is prose. A state-secret stamp is detected, which is seeing it. Labelling state-secret material is still the register's job (DLP-427).

Since 1.13.1 (DLP-1821), a trade-secret term followed on the same line by a dash and a definition is also prose, not a banner: definition cues such as Russian это, Azerbaijani/Turkish copula endings (-dır, -dir, -dur, -dür), or English is suppress it; a longer lowercase sentence ending in a full stop or ellipsis is suppressed too. Short dash labels such as Kommersiya sirri - board only remain markings. This refinement is limited to the three trade-secret terms; other markings keep their existing rules.

A marking is a banner, not a word (DLP-1478, 1.2.0). A term is reported in one of three shapes, the same in all four languages:

in capitals anywhere: CONFIDENTIAL, MƏXFİ SƏNƏD, GİZLİ, MEXFI;
in any case as a banner: alone on its line (up to three capitalised words before it, as in "Acme Confidential", and decoration such as ***), or as a label opening a line ("Confidential:", "Restricted distribution - board only");
a multi-word marking phrase opening a line ("Not for distribution outside the company");
since 1.3.0 (DLP-1515), letter-spaced on a line of its own: C O N F I D E N T I A L, one space or tab between letters and two or more between words.
since 1.12.0 (DLP-1749), a footer: one to four capitalised words for the organisation, a separator (-, –, ,, |, ·) and the term, the whole line, as Word and PowerPoint footers are written: "Acme Corp - Confidential", "NOVIAN Health, Confidential", "Kapital Bank MMC - Məxfi", "Acme Corp - Internal Use Only". A page number may follow ("| Page 3 of 12"), and so may "and Proprietary", which needs no separator ("Acme Inc. Confidential and Proprietary"). A copyright year may lead ("(c) 2026 Acme Corp. Confidential."), and the term may come first ("Confidential | Acme Corp"). Ten of fifteen footers written on YUSIF were missed before. In a sentence the term is still prose ("The Board - Confidential matters aside - ...").

The capitals are also matched when a letter is retyped as its look-alike in another script or as a digit (1.3.0, DLP-1515): Cyrillic and Greek capitals that render as Latin ones (СONFIDENTIAL with a Cyrillic С), l or 1 for I, 0 for O, and a Cyrillic marking typed with Latin letters. A person who wanted a marked document to pass only had to retype one letter. Only the all-capitals form is widened this way; a stray look-alike in mixed case is far more often a typing slip than a marking.

Since 1.6.0 (DLP-1713) the word used as a word is not a marking, which was about half of the roughly 27 GovDocs1 files it fired in: a line naming both CONFIDENTIAL and UNCLASSIFIED (a form's classification options); a line holding only the word inside a run of five or more one-word lines (a word list, which "alone on its line" otherwise took for a banner); three or more capitalised words either side on the same line, single-spaced (a shouted sentence; a layout gap of several blanks breaks the run, so "PAGE CONFIDENTIAL FOIA EXEMPT" is still a marking); and an e-mail disclaimer ("intended only for", "may contain confidential"). A banner followed by a one-word line ("CONFIDENTIAL" over "MEMORANDUM") is still a banner. Not set aside, and recorded: a statute's or a regulation's heading ("CONTRACTOR ACCESS TO CONFIDENTIAL INFORMATION"), a proper name ("Antwerp Confidential") and "YOUR CENSUS REPORT IS CONFIDENTIAL".

Since 1.7.0 (DLP-1723) "NON-CONFIDENTIAL", "NON CONFIDENTIAL" and "NOT CONFIDENTIAL" are not markings: the first is the stamp on a filing's public version, and one US trade filing in GovDocs1 carried it 47 times. Since 1.8.0 (DLP-1730) a marking in quotes is named, not applied: a contract's 'marked "CONFIDENTIAL"' or 'the legend “Confidential”'. A banner is never written in quotes. Since 1.9.0 (DLP-1740) the quotes may hold a longer phrase: 'include the phrase "CONFIDENTIAL BUSINESS INFORMATION"', with an opening quote just before the word and a closing one later on its line. LaTeX's ``quote'' and the replacement character that extraction makes of a curly quote count as quotes. Since 1.10.0 (DLP-1743) nor is a marking that a sentence names as the object of marking or handling it: directly after "marked", "labeled", "stamped", "designated", "considered", "deemed", "treated as", "kept" or "to be" ("should be marked CONFIDENTIAL so as to avoid discovery"), on its line or at the end of the line before, where a wrapped sentence puts it. Since 1.10.1 (DLP-1744) not across a blank line, and never when the marking is its whole line: a banner under an unfinished line ("must be kept" / "CONFIDENTIAL") is still a marking, while 043123's wrapped "confidential." with its full stop is not. Since 1.11.0 (DLP-1719) a reference to the label counts the same way: "identified as (being)", "classified as", "the phrase", "the words", "the legend", "the term" ("you must include the phrase CONFIDENTIAL BUSINESS INFORMATION", "marked with the legend Confidential"). Not after "is" or "are": "This document is confidential" is a footer marking the document it is in, so "the specific medical condition is CONFIDENTIAL" is still reported, as is an adjective in prose ("NONDISCLOSURE OF CONFIDENTIAL BUSINESS INFORMATION"). Since 1.14.0 (DLP-2098) nor is a sentence's last word that hard wrapping left alone on its line, in any language: GovDocs1 001761.txt (15 U.S.C. 57b-2) wraps "which is privileged or" / "confidential." and raised a policy incident. The word must be in lower case, and the line before must be in the same paragraph and end mid-sentence, with a letter, digit or comma. Then either the word ends in a full stop, which a banner does not, or the line before ends in "or", "and", "nor", "və", "yaxud", "или", "и", "либо", "veya", "ve", "ya da" or "yahut", which leaves the clause open. A reply's ">" quoting is read through. Upper and title case stay markings under an unfinished line, as does a lower-case word under a heading ("Quarterly figures" / "confidential"). Accepted cost: a real lower-case banner placed directly under a line ending in one of those conjunctions is missed. Anyone who can place it there can also delete it, so this opens no way round the marking that deleting it does not.

Since 1.5.0 (DLP-1632) a marking within its window of "security clearance" or "U.S.C." is set aside, as one beside a vendor's copyright notice is: an all-caps order naming a clearance level ("MINIMUM OF CONFIDENTIAL SECURITY CLEARANCE") and a statute's section heading ("CONFIDENTIAL INFORMATION SEC. 708. [21 U.S.C. 379]") raised policy incidents on GovDocs1. The .xls reader also no longer reads a printer driver's settings (the PLS record), where Canon drivers list their watermark names, "CONFIDENTIAL" among them, in every workbook printed on such a printer. A marking alone on its line (decoration aside) is a banner, and no term near it sets it aside: without that, "(see 15 U.S.C.)" under a banner silenced the marking policy, as "test" did before DLP-1552. The terms work only where the word sits inside a longer line.

So "it is confidential until Monday" and "The Confidential report" are prose and not markings. A bare word opening a sentence is not enough either, because "Confidential information shall mean..." is a contract's prose. Added to the vocabulary: Turkish hizmete özel and the ASCII spelling mexfi. Russian title case (Конфиденциально) is now found as a banner, as English, Azerbaijani and Turkish title case already were. Capitals are matched with both the Turkic dotted İ and the plain I. Not found: a marking typed in lower case in the middle of a line.

Another company's notice is not a marking (DLP-1464, 1.1.0). A match within 200 characters of copyright, all rights reserved, licensor(s), trade secret(s), confidentiality agreement or source code loses 0.4, which puts it under the 0.35 threshold. Fifteen of 2,885 files that Windows ships raised a high incident from vendor boilerplate such as "Qualcomm Atheros Confidential and Proprietary" and "INTEL CONFIDENTIAL Copyright ...". A document whose own marking appears anywhere else in it is still reported. Not caught: a cover line whose only "Confidential" sits in the same 200 characters as a copyright notice.

A test word never unmarks a document (DLP-1552, 1.4.0). The marking no longer takes the test words every detector starts with (test, sample, example, dummy and the rest). Before this, at the 0.4 penalty above, "CONFIDENTIAL" over a penetration test report, clinical test results or a sample contract raised nothing. Anyone could also unmark a document by typing "test" under its banner. A banner is its author's own statement, and a word beside it says what the document is about. The vendor-notice terms are the only ones that set a marking aside. Accepted cost: a style guide showing "Example: CONFIDENTIAL" is reported.

Whole words, and what that fixed (DLP-265). The terms matched as substrings until it was measured. Three consequences, all of them the kind that destroy a DLP programme faster than a missed file:

This document has unrestricted distribution was reported as confidential, at the same confidence as one shouting STRICTLY CONFIDENTIAL. The substring inverted the meaning of the sentence it found.
gizli is inside gizlilik, məxfi inside məxfilik and конфиденциально inside конфиденциальность - the words for privacy and confidentiality. Every privacy policy in three of the four languages was a confidentiality marking.
On this repository's own prose the bare word restricted accounted for ten of eleven findings, every one of them ordinary technical English about file permissions. Measured with cargo run --release -p dlp-detector-engine --example precision_probe, which went from 17 findings to 8 across the corpus, and this detector from 11 to 2.

What it costs, stated rather than left to be discovered. The bare word restricted is gone, so a cover page stamped with that word and nothing else is not matched. The phrase forms are, and so is internal use only. An organisation that stamps the bare word - some government and defence contexts use it as a classification in its own right - needs a detector of its own, and a case-sensitive one: ordinary English writes the word in lower case and a classification stamp does not.

What it still fires on. The bare word confidential is kept, because a page stamped with that word and nothing else is the case this detector exists for. The cost is that a document about confidentiality fires: the two remaining findings in the probe are exactly that, in documents discussing markings. That is a trade, not an oversight - losing the canonical stamp to gain those two would be the wrong way round.

Implementation note. Rust's to_uppercase is locale-independent and maps i to I, so məxfi would uppercase to MƏXFI and never match the real spelling MƏXFİ. The keyword automaton therefore expands each term into its Unicode lower, upper and dotted-I variants. Tested in all four languages.

Country packs

Country formats change independently of both the agent and one another. Each country therefore has its own semantic version, and every definition in that pack carries that exact version. A finding's detector_version is consequently the pack revision that produced it, not the version of the agent that happened to run it.

Azerbaijan - `az` v1.22.0

az.fin also recognises the English labels FIN code, ID FIN code and FIN number, used in Azerbaijani forms shared with foreign partners (DLP-1841). A bare FIN without a nearby label remains intentionally unsupported.

Detector	Matches	Validator	Honest precision statement
`az.fin`	7 alphanumerics in capitals (DLP-1378: `NTamd64`, in every x64 driver's INF file, was matched and validated; DLP-1463: lower case brought `ntamd64` back, so a FIN typed in lower case is not found)	Structural only - mixed letters and digits	Weak alone. No public check digit exists. Base confidence 0.10 against a 0.45 threshold, so it cannot fire without a nearby context term (`FIN`, `şəxsiyyət vəsiqəsi`), even with the +0.05 the engine gives five or more matches in a file. At 0.15 that bonus carried a list of driver model codes (`AR5B116`, `QCA9565`) over it in a quarter of Windows' Wi-Fi INF files. Tested: a bare `5AB2C71` and a list of ten model codes produce nothing Since pack 1.5.0 (`DLP-1679`) `personal identification number`, the FIN's English name on the identity card, is a supporting term too; `PIN` alone was one until 1.6.0 (`DLP-1711`), when it was dropped as an ordinary English word and name part. Since 1.7.0 (`DLP-1716`) `şv` is a term only with its diacritic: folded to `sv` it is El Salvador's country code, and supported a hotel postcode in GovDocs1. A term of two letters or fewer is never given an ASCII spelling. Since 1.9.0 (`DLP-1718`) a bare `fin` counts only when written as a label: followed by `:`, `-`, `№`, `#`, `=`, `(` or `
`az.passport_mrz`	Two 44-character ICAO TD3 passport MRZ lines with issuer `AZE` The zone's lines may be separated by spaces, tabs or line breaks, and by simple HTML tags, so a zone split across paragraphs, spreadsheet cells or HTML elements is still found (DLP-2251).	Exact issuer plus document number, birth date, expiry, optional-data when present, and composite check digits	High after validation. It deliberately does not guess at a printed passport number without the MRZ
`az.id_card_mrz`	The national ID card's machine-readable zone with issuer `AZE`: ICAO 9303 TD1 (three lines of 30, the card's back side) or TD2 (two lines of 36) (DLP-1511) The zone's lines may be separated by spaces, tabs or line breaks, and by simple HTML tags, so a zone split across paragraphs, spreadsheet cells or HTML elements is still found (DLP-2251, DLP-2326).	Every field's check digit - document number (including one that continues into the optional data), birth date, expiry - and the composite	High after validation. A block of capitals and filler with one wrong digit is not reported. The FIN printed in the zone's optional data is part of this finding and is not reported again as `az.fin`: one card, one identity
`az.id_document_number`	The serial number of the identity card, `AA` and seven digits or `AZE` and eight digits, and the passport number, `C` and eight digits, each only beside a label (DLP-2269): `şəxsiyyət vəsiqəsi`, `seriya və nömrəsi`, `pasport`, `passport no`, `паспорт` and similar. Since 1.22.0 (DLP-2365) also the forms with a case or possessive ending (`vəsiqənin nömrəsi`, `şəxsiyyət vəsiqəsinin seriyası`, `pasportun nömrəsi`, `ümumvətəndaş pasportu`), the card's own `Sənədin nömrəsi / Document No.`, `ID card No` and `Ş/v`. Each is a whole label phrase: an ending the list does not name is not a label, and `Ş/v` is not folded, because `S/V` is a vessel's prefix	None	Weak alone, and label-only by design. The printed numbers carry no check digit of their own, and the formats are those the ticket records, not checked against an issued specimen. Base confidence 0.15 against a 0.45 threshold, so without a label nothing is reported. A bare `AA1234567` or `C12345678` in running text, such as a product or part code, is not reported, and the seven digits of an `AA` serial are not also reported as an `az.fin`. Added in 1.19.0 (DLP-2269)
`az.phone_national`	Azerbaijani numbers written the national way: `0XX XXX XX XX` with a city or operator prefix (010, 012, 018, 022, 050, 051, 055, 060, 070, 077, 099), or `994` without the plus (DLP-1511)	None	Weak alone. A ten-digit run starting with 0 is common, so a phone word nearby (`tel`, `telefon`, `mobil`, `əlaqə`) is required; `nömrə` alone is not one, because it only means "number" and labels a VÖEN or a FIN as often as a phone (DLP-1535); `Invoice 050 123 45 67` is not reported, and neither is a prefix no operator or city uses. Since pack 1.8.0 (`DLP-1732`) nor is another country's number written after its calling code (`Tel : + 31(070) 310-2279`, `39 010 3071634`; four such in about 16,000 GovDocs1 files): a number opening with its trunk `0`, directly after a one-to-three digit code other than 994 on its line. `994 (050) ...` is kept, and so is a code opening its line, which is as likely a list's number. A foreign number in national form with no code before it is still read, unless it is grouped 3-3-4: since pack 1.12.0 (`DLP-1753`) the number is compact (`0505542704`) or grouped the Azerbaijani way, 3-3-2-2 (`050 554 27 04`, `(012) 498-12-34`), because the prefixes are shared (050/055 are the UAE's mobiles, 051 a UK area, 012 Pretoria) and those are written 3-3-4 (`Mobile: 050-554-2704`, 20 times in one GovDocs1 hospital directory)
`az.voen`	10 digits	None	Weak alone. Any 10-digit number matches, so `VÖEN`, `vergi ödəyicisi` or equivalent context is structurally required. Since 1.18.0 (`DLP-2266`) `ВОЕН` and `ВÖЕН`, the label in Cyrillic, support it. `ИНН` stays with `ru.inn`, which checks the number, and a bare `TIN` is not a term: it labels other countries' taxpayer numbers. A term outside a number's own label does not support it (`DLP-2267`): an empty `VÖEN: ____` field does not make the `Sifariş No:` or `HESAB-FAKTURA No` number beneath it a VÖEN
`az.iban`	`AZ` plus 26 characters: check digits, four-letter bank code and 20-character account field	MOD-97-10 and exact 28-character country length	High after validation. The structure follows the Central Bank of Azerbaijan guidance. It reports an Azerbaijani IBAN in place of `core.iban` rather than alongside it, because it demands the literal `AZ` and the generic pattern does not (`DLP-433`). Since pack 1.14.0 (`DLP-1801`) also with hyphens between groups, and whatever text follows it, Cyrillic included

Türkiye - `tr` v1.8.0

Detector	Matches	Validator	Honest precision statement
`tr.tckn`	11 digits	Both published check digits	Good. The check digits carry real signal. Until DLP-1472 the tenth-digit rule was computed as `odd7 mod 10`, which dropped the `- even` term, so only numbers whose even-position digits summed to a multiple of ten passed. The two published examples do; about nine real numbers in ten did not. Not alone, since DLP-1664:* the check digits still pass about one random 11-digit number in a hundred, and a public disc-position log reported several at 0.80 with nothing around them. A TCKN now needs `TC`, `TCKN`, `T.C. kimlik`, `kimlik no`, `vatandaşlık` or `national id` within the window, or under such a column header in a delimited file, as `az.fin` does; since pack 1.3.0 (`DLP-1679`) also `Turkish ID`, `TC ID`, `T.C. ID` or `identity number`, as English-language papers write it. A bare list with neither is missed. Since 1.3.0 a TCKN grouped 3-3-3-2 by spaces, dashes, dots or no-break spaces is read too, validated on its digits; since 1.4.0 (`DLP-1729`) with one separator throughout, so a PostScript path's coordinates (`227.172 482.16`) are not one. Since 1.5.0 (`DLP-1802`) `T.C.` with its dots (`T.C. No:`, `Müşteri T.C.:`), `kimlik` alone (`Kimlik:`, `Kimlik numarası`, `kimlik numaranız` in a sentence) and `ID number` are labels too. A non-breaking hyphen or an en dash between the groups is already a hyphen after normalisation (DLP-1555)
`tr.passport_mrz`	Two 44-character ICAO TD3 passport MRZ lines with issuer `TUR` The zone's lines may be separated by spaces, tabs or line breaks, and by simple HTML tags, so a zone split across paragraphs, spreadsheet cells or HTML elements is still found (DLP-2251).	Exact issuer plus document number, birth date, expiry, optional-data when present, and composite check digits	High after validation. It deliberately does not guess at a printed passport number without the MRZ
`tr.vkn`	10 digits	Check digit (`tr_vkn`, the Revenue Administration's published algorithm) since pack 1.2.0 (`DLP-1675`)	Weak alone. The check digit passes one random ten-digit number in ten, so `VKN`, `vergi kimlik` or equivalent context is still structurally required: a validated number alone stays under the threshold. Until 1.2.0 any ten digits beside such a word were a VKN, `0000000000` included
`tr.iban`	`TR` plus 24 characters: check digits, five-digit provider code, reserved zero and 16-character account field	MOD-97-10 and exact 26-character country length	High after validation. The structure follows the Central Bank of Türkiye regulation. Since pack 1.6.0 (`DLP-1801`) the pattern reads groups of four as a statement prints them (`TR27 2442 805S FT02 ...`), whose boundaries are not the structure's, and hyphens; the validator checks the digits whatever the grouping

Russian identifiers - `ru` v1.1.0

Since DLP-1688. Russian is the second language of much of the paperwork in Azerbaijan, and a contract or personnel file with a Russian counterparty carries these numbers. Only identifiers with published check digits are here; a passport's series and number has none and is not detected.

Detector	Matches	Validator	Honest precision statement
`ru.inn`	10 digits (an organisation) or 12 (a person)	Check digits (`ru_inn`): one for ten digits, two for twelve, each a weighted sum mod 11 mod 10; a number of one repeated digit is refused	Weak alone. The check passes one random ten-digit number in ten, so `ИНН`, `INN`, `налогоплательщик` or `taxpayer` must be within the window or the column header; a validated number alone stays under the threshold. No match in GovDocs1 batch 006's 850 English-language files
`ru.snils`	11 digits, `NNN-NNN-NNN NN`, spaced or run together	Check number (`ru_snils`) over the first nine digits; numbers up to 001-001-998, issued before the check, pass on shape	Weak alone, for the same reason: `СНИЛС`, `SNILS`, `страховой номер` or `insurance number` is required

Gulf identifiers - `gcc` v1.0.0, locale `ar`

Since DLP-2033, for tenants in Saudi Arabia, the UAE, Kuwait and Qatar. Labels are read in Arabic and in English, as Gulf documents print them.

Detector	Matches	Validator	Honest precision statement
`sa.national_id`	10 digits beginning 1 (citizen) or 2 (resident, Iqama)	Luhn over all ten (`sa_nid`); one repeated digit is refused	Weak alone. The check passes one random ten-digit number in ten, so `رقم الهوية`, `الهوية الوطنية`, `الإقامة`, `السجل المدني`, `iqama`, `national id`, `saudi id`, `id number` or `id no` must be within 120 characters or the column header. A label within that window speaks for every number of the shape in it, so a ten-digit invoice number in the same record as an Iqama is reported too
`ae.emirates_id`	15 digits, `784-YYYY-NNNNNNN-C` or run together	784, a year of birth 1900-2100 and Luhn (`ae_emirates_id`)	Reported alone. The country code, a year and a check digit in fifteen digits are rare by chance; a value whose check fails is not reported. A label (`emirates id`, `بطاقة الهوية`) adds to the confidence
`kw.civil_id`	12 digits beginning 2 or 3	Century, a date of birth that exists, and the PACI weighted mod-11 check digit (`kw_civil_id`)	Weak alone: `الرقم المدني`, `البطاقة المدنية`, `civil id`, `civil number` or `PACI` is required
`qa.qid`	11 digits beginning 2 or 3	Shape only (`qa_qid`): century, year, a nationality code that is not 000. No check digit is published	Weak alone, and weaker. Base 0.05: only `الرقم الشخصي`, `البطاقة الشخصية`, `QID`, `Qatar ID` or `Qatari ID` beside it makes it one

Measured on GovDocs1 thread 001 (976 files): no match. The same pack with the label requirement removed matched 298 numbers in 17 files (140 Saudi, 154 Qatari, 4 Kuwaiti shapes, all traffic statistics and survey coordinates, none an identifier), which is what the label requirement is for. The Emirates ID matched nothing in either run. Thread 001 is English-language US government material, so this measures false positives, not recall; recall rests on the unit tests' constructed numbers until Gulf documents are available to test against.

Delivery and activation

POST /api/v1/detector-packs accepts a complete pack under the tenant's own namespace. Import is disabled by default; activate: true makes every detector in the submitted version active together. The server refuses duplicate ids, namespace crossings, version mismatches, version downgrades, changes made without a version bump, and collisions with platform or standalone rules. It then recompiles the policy bundle. The endpoint never loads a pack from a side-channel: definitions become runnable only after the agent verifies the signature on that bundle.

builtin::tests::azerbaijan_pack_detects_each_document_family_with_only_synthetic_values and its Türkiye counterpart cover all four families without production or government-issued identifiers. The agent acceptance test a_signed_new_country_pack_activates_without_an_agent_upgrade_and_reports_its_pack_version proves that an unknown country definition becomes runnable from a signed bundle and that changing it without a new signature leaves the previous definition in force.

A test asserts that every low-precision detector has base_confidence < min_confidence, so context is structurally required rather than merely recommended.

Procurement documents - `procurement` v1.1.0, locale `multi`

Not a country pack. The identity packs answer whose data is this; this one answers a question a contracting authority asks more often - is this the commercial position we were not supposed to have shared yet. A bid price that leaves before an award is decided is the most consequential disclosure in a procurement process, and it travels as an ordinary spreadsheet.

It is versioned separately from az because these are document conventions rather than national formats: loosening one of these patterns should not force a version change on somebody's FIN detector, and the vocabulary deliberately spans Azerbaijani, Russian and English because a tender file in this region does.

Every detector here is keyword-led, and that is a decision rather than a shortcut. A tender reference has no published format this project can cite - it is whatever the contracting authority's system emits - so a pattern invented for one would be confident and wrong. The indicator is therefore the procurement term together with a reference-shaped token beside it, and precision comes from the supporting vocabulary in the window. It is the same structural defence az.voen applies to a bare ten digits.

Detector	Matches	Honest precision statement
`procurement.tender`	A tender, lot or procurement term followed by a reference containing a digit	Needs prices to fire. Base 0.40 against a 0.55 floor, so a reference in an administrative note stays below it and a reference among `təklif olunan qiymət`, `AZN` or `vahid qiymət` clears it. The digit in the reference is load-bearing: without it "Lot sizes were discussed" read as lot number `sizes`
`procurement.contract`	A contract, agreement or `müqavilə` term followed by a reference containing a digit	Same shape and same floor. `tərəflər`, `müqavilənin dəyəri` or `заказчик` is what carries it
`procurement.invoice`	`hesab-faktura`, `faktura`, `invoice`, `ödəniş tapşırığı` or `счёт-фактура` with a reference	Same floor. `ƏDV`, `VÖEN` or `к оплате` carries it. Overlaps `az.voen` on an invoice, which is correct - they are two facts about the same document
`procurement.boq`	An estimate or bill-of-quantity heading: `smeta`, `iş həcmi cədvəli`, `bill of quantity`, `ведомость объёмов`	The heading alone is not enough - base 0.35 against the same floor - because a sentence about bills of quantity is not one. The unit-rate columns (`ölçü vahidi`, `vahid qiymət`, `miqdar`) are what report it

A published notice is not a leak, and says so

Every detector in this pack suppresses rather than penalises on a published-notice marker (elan edilir, dərc edilmişdir, извещение о закупке). An agency's own announcement carries the same reference as the bids that answer it, and a notice reported at a lower confidence is still a queue somebody has to clear - which is the fastest way to have a detector switched off.

Two consequences worth knowing. The suppressing list is deliberately not the usual test-data markers: those are matched as substrings, which is right for a penalty and catastrophic for suppression - test inside latest, contest or protest would silence a real bid sheet. And a suppressed match survives the scan report as a non-finding with a count of zero and the marker that ended it named, which is what the console's test bench shows an operator running a rule. It goes no further: the agent filters to hits before anything is queued and the server drops non-hits again when it builds an incident, so a suppression is auditable by whoever is tuning the rule and is never a record about somebody's document. PRIVACY_MODEL.md §14 draws that line.

Precision floor. packages/detector-engine/tests/procurement.rs holds fifteen ordinary business sentences that mention contracts, invoices, lots and bills of quantity without being any, and asserts zero hits across them - as a number, so a pattern loosened later fails a test rather than widening quietly. Two of those fifteen were found by writing the corpus: they are the false positives the digit requirement and the self-reinforcing supporting terms above were fixed for.

Geospatial pack - `geospatial` v1.0.0, locale `multi`

For an agency whose subject is physical infrastructure, where something is is a primary data class. Alignment coordinates, survey schedules and GIS layers describe real assets, and before this pack a survey export and a budget spreadsheet were the same thing to the engine: two columns of decimals.

Detector	Matches	Honest precision statement
`geospatial.coordinate`	Decimal-degree pairs written to three or more places, and degrees-minutes-seconds with hemisphere letters, that fall inside the detector's own bounding box	Five pairs before anything is said. The gate is the box; the floor is the count

The box is a field, not a constant

The bounding box lives on the detector definition. That is deliberate and it is the part most likely to be "simplified" later, so the reason is here: it is a political boundary. Compiled into the binary it is something nobody can see, argue with or change. On the definition it travels in the signed bundle, it is printed in the console beside the detector - inside 38.30–41.95 N, 44.70–50.70 E - and an operator who works across a border edits it instead of filing a bug.

The shipped default is Azerbaijan, generous at every edge: west to Nakhchivan, east to the Caspian shore, south to the Aras, north to the Caucasus ridge. A box slightly too large costs a few more candidates for the confidence model to weigh. One slightly too small turns a survey of a border district into a price list.

Two orderings are accepted, because a spreadsheet writes latitude first and GeoJSON writes longitude first. That is only safe where the latitude and longitude ranges do not overlap, so it is not assumed - a test asserts it of the shipped box, and a deployment whose box spans the equator and the prime meridian would be halving the gate's strength without being told.

A set of coordinates, not a coordinate

min_match_count is five. A single pair is an address on a letterhead or a meeting point in an e-mail; five is an export, a track log or a schedule of survey points. This is a deliberate miss: one precise location can matter and this detector will not find it. What it will not do is turn every document containing an address into an incident, which is the failure that gets a detector switched off - and a switched-off detector finds nothing at all.

Five pairs are not five coordinates when they are coincidences in a table of numbers (DLP-1702). A 70-column sensor CSV in GovDocs1 had 34,875 adjacent decimal pairs, 51 of which fell inside the box by chance, and was reported as a survey. Bare decimal pairs are now set aside when fewer than one in twenty of a document's decimal pairs, over at least a hundred, are in the box. A coordinate export (lat,lon with perhaps an elevation and a value) is a third or more. Degrees-minutes-seconds pairs carry their own notation and are not affected. The cost: a short genuine coordinate list (five or ten pairs) embedded in a much larger numeric report falls under the share and is not reported.

The plausibility gate is the other half. Both halves of a decimal pair need three or more decimal places: a survey point is written to metres or better, and 41.2, 47.5 is two numbers in a spreadsheet. A test holds a budget table whose every figure has three decimals and a comma, and asserts it is not a survey.

How much of this is true, measured (`DLP-425`)

Every detector above carries an honest precision statement. Until this existed none of them carried a number, and a statement without one is an aspiration.

packages/detector-engine/tests/detector_quality.rs runs a labelled corpus - 58 synthetic documents in packages/detector-engine/corpus/quality-corpus.json - past every shipped pack and reports precision, recall and the confusion matrix per detector. Every built-in detector has a labelled positive and negative case. The floors live in the corpus file beside the documents, not in a comment, because the whole point is that loosening a pattern costs somebody a red test rather than a quieter Monday for whoever reads the queue.

Every document is a negative case for every detector it is not labelled for, unless that detector is explicitly marked ambiguous. A bid sheet is a positive for procurement.tender and simultaneously a negative for the thirty-odd detectors that should stay off it, and that second half is where false positives actually come from. It is also the half nobody writes by hand.

Three properties are asserted about the measurement itself, because a metric can be gamed as easily as a detector:

Silence is not precision. A detector that reports nothing has no precision rather than a perfect one. Scoring 1.0 by never firing would make the floor an incentive to break the product.
A floor with nothing to exercise it is a failure. Every floored detector needs at least one document it must match and one it must not, or the number sits in the file looking like coverage.
An uncertain example is not forced into the confusion matrix. A detector-specific ambiguous label keeps the outcome visible while excluding it from precision, recall and their floors. Assigning both expect and ambiguous to one detector on one document is invalid; such a conflict fails corpus validation.
A report names documents, never their contents. It is exactly the artefact that ends up pasted into a ticket, and PRIVACY_MODEL.md does not stop applying because the corpus is synthetic.

The corpus holds no real identifier: the card numbers are the brands' own published test values, the IBAN is MOD-97-valid and invented, and the FİN, VÖEN, coordinates and references are made up. A test asserts it.

What it found on its first run. az.iban never fired. It tied with core.iban on every axis the overlap rule then measured, the tie fell to pack order, and the core pack is first - so the country detector was dead code in the presence of the generic one, on every document, while the catalogue documented it and the console listed it as enabled. Fixed in DLP-433 by adding required-literal-text to the specificity ordering, and the corpus now labels that document az.iban with a floor of its own.

Azerbaijani written without its diacritics is matched too (DLP-438). Every term and every indicator carries its ASCII spelling - müqavilə and muqavile, satınalma and satinalma - generated from the alphabet map rather than typed, so it cannot drift from the term beside it and cannot be forgotten when one is added. This is not a nicety: a dBASE attribute table is single-byte with no Unicode, field-collection tools fold, and older systems never had the letters, so an Azerbaijani asset register is very often written exactly that way. The transliteration pass does not cover it, because that folds the document and re-runs the same patterns, which is the opposite direction. Measured: it costs nothing on the 23-document corpus and nothing on this repository's own prose - seven findings before, seven after.

Inflection is measured too, separately.examples/morphology_probe.rs puts every supporting term in every pack through the suffixes the languages actually use and reports what the matcher finds: Azerbaijani terms score 1.000, because agglutination appends and the stem survives inside every inflected form. The losses were Russian, which is fusional and replaces the ending rather than adding to it, and the procurement indicator patterns, which placed the term next to the reference and broke when the term carried a suffix. Both are fixed (DLP-435) and the reasoning is in SPIKE-020; the probe is the thing to re-run before editing a term list.

A term inside a longer word is not context (DLP-442). Supporting terms are matched as substrings and that stays deliberate - it is why Azerbaijani term recall is 1.000, because agglutination appends and the base form survives at the front of every inflected one. But fin sits at the front of FINANSLAŞDIRMA as surely as it does at the front of fin kodu, and az.fin runs a base confidence of 0.15 against a floor of 0.45 precisely because a seven-character alphanumeric token is a weak shape: one accidental hit carried it over its own threshold on a document about funding. Position cannot separate the two, because both sit at the start of a word. The tail is the discriminator: a term counts when it is the whole word, or when what follows it inside the same word decomposes into suffixes the language actually uses. dən, nin, ları, ımız do; anslaşdırma does not. The suffix set is closed and written out in src/morphology.rs - a stemmer was the wrong tool for the dictionary problem (SPIKE-020) and is the wrong tool here too: this does not need to know what a word means, only whether a tail is a plausible ending, and a list that can be read is worth more than an algorithm that cannot be audited. The same rule governs negative terms, where the harm runs the other way - an accidental negative hit suppresses a real finding rather than inventing one - because it is the same accident.

A negative term counts against a FIN only when it is written against it (DLP-2282). A test marker on the line above a labelled FIN is a heading or a noun phrase about something else - "COVID-19 PCR test nəticəsi: mənfi" above a patient and the FIN that belongs to them - so the FIN is still found. A marker on the FIN's own line is still reported as negative context when it labels the value ("Format example: FIN: …", "test FIN: …") or sits in brackets straight after it ("FIN: … (nümunə)"). It does not lower confidence or set aside a FIN that the structural validator accepted (DLP-1770); prose must not become a detection bypass. In a table the row is the context, as it was. A documented sample FIN is set aside only when its exact value is on the tenant's detector allowlist (DLP-2324); that exception expires within 90 days like every allowlist exception, and no marker in the prose sets a value aside. Only az.fin is held to this line-scoping rule; a password's "anonymous" or an IP's "test" is what those detectors report on, and they keep the whole-window rule.

A term of three letters or fewer counts only as a whole word (DLP-1661). The tail rule let fine read as fin plus the dative -e, and in GovDocs1 thread 002 "fine", "fine-rayed" and a PDF's split "re fine" supported az.fin on a grant number, a gamma-ray burst id, a program name and a land region code. A short term is an abbreviation, and Azerbaijani writes a suffix on one after a hyphen (FİN-ə) and Turkish after an apostrophe (FİN'e); both end the word, so both still count. Glued on, the ending is almost always another language's word. Longer terms keep direct inflection (ibandan, kartlarımızdan). The rule covers negative terms the same way: a short negative term glued to an ending, such as ean in a longer word, no longer sets a match aside, while EAN-13 still does.

And the whole word is what is judged (DLP-1667). A term must lie inside the context window, window_chars either side of the match, but whether it is a word and what follows it are read in the document, not in the window. Judged in the window, a word the cut went through looked whole: a window ending inside "fine" showed the tail rule a standalone fin, and az.fin fired on a trial exhibit page's anchor names. The window is not widened to whole words: that pulls in a term that begins outside it. The same holds for negative terms.

A column header is context for its whole column (DLP-443). In prose, a character window around a match is the right shape: the words near a value are what say what it is. In a table it is the wrong shape entirely. A supplier export writes voen once, in the header, and then four hundred values under it; a 120-character window reaches the first three or four rows and nothing after them, so a detector that needs context reported the top of the file and nothing else - and the longer the export, the smaller the fraction found, which is the worst direction for a failure to point in. A delimited document now gives each match one extra piece of context: the header cell of its own column. Its own, not the whole header row, because a term in column A says nothing about a value in column C. Detection is deliberately conservative - three rows must agree on a field count of two or more, quoting is not honoured so a ragged row disqualifies the file rather than mislabelling it, and prose containing commas is not a table. In a tab-separated sheet each tab is a column boundary, so a blank cell between two tabs is a column of its own (DLP-2258). A wrong header is worse than none: it is confident.

A negative indicator speaks for the value beside it, not for the file (DLP-443). The penalty used to be computed once per detector and subtracted from a confidence every other match had earned on its own, so a supplier export whose second row was a sample record reported clean - at three rows that looks like a coincidence, at five hundred it is a page of real identifiers nobody sees. Poisoned matches are now set aside and the rest are judged on their own, with negative_context_partial saying how many and how many survived. All poisoned is unchanged: that is a sample document and the penalty is the right answer.

"Beside" was the detector's character window until DLP-1699, and in a table that window reaches the neighbouring rows: in a four-row export one sample row poisoned all four, and with every match poisoned the file was set aside as a sample document, so four real VÖENs were reported as nothing. In a delimited table a negative term now speaks for its own row and, through the header, its column. Outside a table the window stands. A table is found where exports really put one: below a title or a date line, above a "prepared by" footer, and with tabs or space-aligned columns as well as commas and semicolons (the DLP-1699 reopen, on YUSIF's shapes). Normalisation turns a tab into a space, so tab and space-aligned columns are read from the gaps it records. A gap at the start or end of a line is indentation, not a column, so indented markup is not a table. The header is searched for in the first thirty lines and the body is followed only as far as rows keep its field count. A text with several tables, such as two sheets extracted one after the other, has every table read, and each table's values take that table's header (DLP-2327).

A label speaks for the value beside it, not for every look-alike in the file (DLP-1703). For a detector whose shape alone sits below its own threshold (az.fin, az.voen, az.phone_national, tr.tckn, tr.vkn, ru.inn, ru.snils and the procurement references), a match is kept only if a supporting term is in its own window or its column header. Before, the support boost was the detector's once any match had a term near it, so one "FIN" in a gene table reported 79 gene names as FINs, and one "pin" on a grocery recall reported 109 product codes. The match count now says how many labelled values there are. ru.inn reads its label within 80 bytes, about forty Cyrillic letters, because "inn" a sentence away in an English list is a hotel's name.

A value broken across a line break is still that value (DLP-444). A wrapped invoice, a PDF whose extraction breaks a line mid-number, a plain-text e-mail hard-wrapped at 72 columns - the same card, not found, because the pattern needs its digit groups on one line. The corpus used to record that as a limitation, with expect: [] on a document that plainly contains a card.

It is found now, and the shape of the fix is the point. Normalising every newline between digits to a space before scanning would also invent values out of two unrelated rows, and a price list is a page of exactly that; roughly one accidental join in ten passes the Luhn check, so validation alone would turn a five-hundred-line table into fifty card numbers nobody wrote. So the join is a second pass whose results are accepted only where the context supports the detector - Card: before it and expiry after it. A page of part numbers has no such context and yields nothing, which is asserted against three hundred lines with the word Kart in the heading so the gate is under pressure rather than merely unchallenged.

The rewrite is byte-for-byte - a newline is one byte and a space is one byte, \r\n becomes two spaces - so every offset in the joined text still points where it did, and the pass reports locations without a mapping table. A mapping table is where an off-by-one puts a redaction window over the wrong digits.

A rejoined match carries a −0.1 penalty and a rejoined_across_a_line_break signal: it is weaker evidence than a value written contiguously, and the same treatment the transliteration pass beside it gets, for the same reason.

A quoted reply is wrapped too (DLP-2105). A reply prefixes every line with > , so a wrapped number's next line starts with the quote marker, not a digit. The join steps over the run of > and blanks that opens the continuation line and blanks it with the break, still byte for byte. The closed break is then not read as a table's column gap: three blanks where a line ended and resumed are not three blanks somebody typed (DLP-1627), and those still keep a value apart.

An IBAN needs no keyword beside it. A wrapped card is accepted only with card context, because about one accidental join in ten passes Luhn. An IBAN's own shape (country, check digits, the bank code's letters) and mod-97, one in ninety-seven, are its context. So az.iban and tr.iban find a grouped IBAN that wraps, quoted or not, and a join that fails mod-97 is still nothing. On GovDocs1 thread 001 (901 files extracted), no file's findings changed.

The figures are kept, per revision (DLP-434). POST /api/v1/detectors/quality runs the same corpus against a tenant's enabled detectors - the rules its endpoints are actually running, not the platform's defaults - and stores the four counts against each detector's own version. GET /api/v1/detectors/{id}/quality returns the current revision's figures and the ones from the revision before it, which is what makes "was it better before" answerable after somebody edits a pattern. Two runs of the same revision are not offered as a comparison: that would show a change of zero every time the corpus was re-run, which teaches people to stop looking.

And precision over the tenant's own traffic, from explicit detector-level review (DLP-1678). POST /api/v1/incidents/{id}/detector-feedback records correct, false_positive or unknown against the exact detector ID and version in that incident's snapshot. Incident closure remains independent: one incident-level false-positive decision must not label every detector that contributed. GET /api/v1/detectors/quality/observed counts only latest explicit correct and false_positive reviews; unknown and unreviewed examples do not enter precision. Verdict history and audit metadata contain no matched content.

Two things it deliberately will not say. Recall is absent, because it needs the documents a detector should have matched and did not, and an incident that was never raised leaves nothing to count - the shipped corpus is where recall comes from, and showing precision alone under a heading implying both would be the more damaging error, since precision looks perfect for a detector that has stopped firing. And below ten reviewed detector contributions there is no figure at all: one explicit false-positive verdict reads as "wrong every time" and would bury a rule on a single analyst's click. That floor is priority::MIN_CLOSED_TO_JUDGE, reused rather than reinvented, because the product had already made the decision for the same reason.

The alternative - a corpus of documents a tenant uploads - would answer the wider question and would do it by keeping copies of a customer's files indefinitely. That is a retention surface with a privacy row, an access-control question and a deletion story attached, and it is not built.

Nothing about a document is stored. A run names the documents a detector missed or over-matched, because that is what makes a failure actionable, and those names are returned to the caller and thrown away. Four integers and a version reach the database. A quality history is a table somebody exports, and the shipped corpus is synthetic today only because nobody has pointed this at their own documents yet.

What this is not is a false-positive rate. Twenty-three documents is a corpus, not a fleet. examples/precision_probe.rs measures the other thing - what the core pack fires on across this repository's own prose - and the two are complementary rather than substitutes.

Transliteration

Detectors may enable a second pass over an ASCII transliteration, for documents written in Azerbaijani, Turkish or Russian where the pattern is Latin.

Implementation note. deunicode renders ə (U+0259) as @, which is actively harmful here - it turns Azerbaijani names into strings containing an at-sign and feeds the e-mail detector garbage. The az/tr alphabet is therefore mapped explicitly before deunicode handles the rest. A transliterated match carries a −0.1 confidence penalty and a transliterated_match signal, because transliteration is lossy.

Composition

Policies combine detectors with AND, OR, NOT, AtLeast(n), TotalMatches(min), category thresholds and file conditions. The bulk-export case - "more than fifty card numbers in one file" - is TotalMatches { min: 50 }.

Exact Data Match

Recognises values from a specific authorised dataset without the dataset being on the endpoint. See Privacy model §5 for the mechanism and its honest limits. A record hit requires every configured field within a proximity window, which is what makes EDM precise: an isolated date of birth is noise; a date of birth next to the matching IBAN is a customer record.

Datasets are imported, versioned and deleted from the console, and the source file never leaves the server (DLP-26). Since DLP-267 a dataset a published policy references reaches the endpoint: packed keyed tags in the signed bundle, and the HMAC key on its own authenticated request, stored beside the device key rather than in the bundle - which is signed, not encrypted, and cached on disk.

An exact data match outranks a pattern that overlaps it (DLP-276). A record whose account field is also a valid IBAN is reported as the record, and core.iban gives way rather than being reported alongside it: one finding, the precise one, because the same characters counted twice is an analyst counting the same fact twice. It used to be the other way round, which made a dataset of IBANs match nothing while one of internal ids worked.

How large a dataset can be

Measured, not estimated - SPIKE-005, reproducible with cargo run --release -p dlp-detector-engine --example edm_scale. Eight-byte tags, which is the default and the enforced floor:

records × 3 fields	bundle	resident on the endpoint
100,000	3.1 MiB	4.8 MiB
1,000,000	30.5 MiB	36.3 MiB
10,000,000	305.2 MiB	363.2 MiB

The documented maximum is 1,000,000 records across up to three fields, four times what it was. Nothing about the algorithm changed to get there: DLP-241 moved the tags from hex strings to the bytes they always were, in the bundle and in the index. It is the same fingerprints - same key, same HMAC, same truncation - in a smaller box, and packages/detector-engine/tests/edm_encoding.rs asserts that a set written either way matches exactly the same documents, in both directions.

The binding constraint is the index rather than the bundle: 36.3 MiB of resident memory for a million records, against the 34 MiB that used to buy 250,000. Larger is not refused by the code yet; it is refused by this sentence, and the ticket to make the console refuse it is open.

What the index costs, measured on both sides - a fresh process holding one representation and nothing else, because an allocator does not hand freed pages back and two measurements in one process would charge the first one's arena to the second:

tags	as hex `String`	as tag bytes	saving
300,000	22.7 MiB	4.7 MiB	4.8x
3,000,000	199.4 MiB	36.3 MiB	5.5x

A HashSet<u64> is not nine bytes a tag, and the figures above are the honest consequence: the table is a power of two, so three million tags sit in 4.19 million slots at 71 per cent load. A sorted flat buffer would be exactly 24 MiB instead of 36.3 and is deliberately not used below eight bytes: a binary search over three million entries is around twenty cache misses, which against a measured 0.5 µs HMAC (DLP-242) would cost more time than the twelve megabytes are worth. Tags wider than eight bytes do use it, because there the per-entry overhead of a hashed set is the larger of the two costs.

Tag length is not the knob it looks like. At the enforced 8-byte floor, a scan testing 200,000 distinct values against a 300,000-tag set produced zero false positives, against a predicted one scan in three hundred million. Choosing between 8 and 32 bytes trades bundle size against nothing. What decides whether an EDM dataset is useful is field entropy: a dataset fingerprinting a country column matches every document mentioning a country, at any tag length, and the import reports that per field.

Read on the endpoint

Extracted locally, with no service and no network. Each is exercised end to end in packages/detector-engine/tests/office_documents.rs - a real file through type detection, extraction and the real detector pack - because every piece can be correct while the file still arrives at the detectors as an unreadable container, and the symptom of that is not an error. It is a customer's documents never raising anything, which looks exactly like having no sensitive data.

Format	How
Plain text, CSV, JSON, XML, source	Read directly, encoding detected
DOCX, XLSX, PPTX	The OOXML package is a ZIP of XML: document body, shared strings, slides and notes, a workbook's print header and footer (`DLP-2248`, with its section and format codes removed, so `&CCONFIDENTIAL` is read as the marking), a Word watermark's text, which Word stores as VML WordArt in an attribute (`DLP-2249`), and a slide master's or layout's text boxes, where a template puts its classification, but not the prompt text of its placeholders (`DLP-2250`). Styles and themes are skipped, and text split across runs is rejoined - a card number with one digit bolded arrives as several elements, and an extractor that joined them with spaces would find nothing
PDF	Content streams, including Flate-compressed ones, which is most real PDFs
ZIP, tar, gzip, 7z, CAB	Entries are extracted and scanned, under limits on count, bytes, depth and (for ZIP) declared expansion ratio; a nested archive of any of the five is opened. An archive cut short by any of them is reported as an incomplete inspection with the reason - never as a clean file. A 7z whose decoder would allocate a dictionary above 64 MiB is refused unread (`DLP-1177`). A CAB reads its stored, MSZIP and LZX folders; its Quantum folders, and a cabinet continued into another, are refused by name (`DLP-2230`, `DLP-2274`). RAR is not read
KML, GPX, GeoJSON	XML and JSON, so they read as text and the coordinates in them reach the detectors. Recognised by structure, not by extension: a `.kml` saved as `.txt` is still reported as a KML, and the rename is reported with it, because the text branch decides by extension alone and that rename is the case worth catching (`DLP-423`)
KMZ	A ZIP whose `doc.kml` is the file worth reading, so it takes the archive path and inherits its limits. Claimed by extension, as OOXML and ODF are
DXF, IFC	Read as text, and now identified as drawings (`DLP-436`). Both are ASCII - group codes and STEP - so their contents always reached the detectors; what was missing was the label, and without it nobody could filter findings to drawings and a drawing renamed to `.txt` raised no mismatch. Recognised by structure: a DXF by its group-code opening and a named section, an IFC by the STEP envelope and its `IFC4` schema, because STEP alone is also how a machined part is exported (SPIKE-021)
DWG	Not read, and refused rather than unbuilt. The only complete open implementation is GPL-3.0, which `deny.toml` does not admit; the alternatives are a commercial SDK or a reverse-engineered binary parser on an endpoint holding the device key. It is still recognised, by its `AC10xx` version header, so the report says a DWG this build cannot read and names DXF as the way out - which sends an operator somewhere different from “an unidentified binary” (SPIKE-021 §4)
dBASE (`.dbf`)	Read (`DLP-437`): field names and every character and numeric value, which is where a shapefile's road names, asset identifiers and contract references actually are. No dependency - a dBASE table is a 32-byte header, a descriptor per field and fixed-width records, and the format has not moved in forty years. A row the table's owner marked deleted is skipped, and a header claiming more records than the file holds reads what is there
Shapefile (`.shp`)	Recognised by its header - file code 9994, version 1000 - and reported as needing a GIS reader, with the note that the attributes live in the `.dbf` beside it - which is itself read when the scan reaches it. Read as text it would be byte soup, and a file called clean because nothing could read it is the failure this column exists to prevent

Not implemented

Stated so the catalogue is not read as a claim.

Capability	Status
OCR (`core.*` on images)	Images are reported as requiring the OCR adapter, never treated as empty
NER / semantic classification	Contract exists (`DetectorKind::External`) and stays unused. Not blocked - declined: SPIKE-008 measured both candidate models and adopted neither. A customer may configure a model they hold themselves; we never fetch one for them (Open-source inventory §2.7)
Document fingerprinting	Wired into the scan path (DLP-45). A fragment of a protected document is found inside an unrelated file; the endpoint holds chunk hashes and never the document. Corpora are built, probed and deleted over `/api/v1/fingerprint-corpora`; this row said they were not, for longer than it was true
YARA byte signatures	Not in the MVP

In every case the file is reported with a reason. Nothing is silently treated as clean.

Two separate mechanisms keep that true, and it is worth knowing which is which.

A format the endpoint cannot read is reported as unsupported with the reason, and the event carries it. That is the OCR row above.

A detector the endpoint cannot run - the contract has kinds this build does not implement - is reported by the engine as not evaluated, with the reason, rather than as a detector that found nothing. scan_text returns both lists separately, and packages/detector-engine/tests/unsupported_detectors.rs pins that: an unrunnable detector reports itself, every unimplemented kind names itself, and one unrunnable detector in a pack does not stop the rest - a single bad definition must not disable detection across a fleet.

That second list is not carried on an endpoint event, and it does not need to be, because an unrunnable detector cannot reach an endpoint: detectors are seeded, and the API can list, read, enable and disable them but not create or edit one. The guard is nothing_in_the_shipped_pack_is_unrunnable, which fails if one is ever added to the seeded pack. If detector authoring is ever added, that test is the signal that this paragraph needs revisiting - and the answer then is to refuse such a detector when it is saved, not to report it per file, because the operator should find out while they are writing it rather than weeks later in a console field nobody reads.

Benchmarks

cargo bench -p dlp-detector-engine measures the full core pack against 16 KiB, 256 KiB and 1 MiB documents, plus pack compilation. Results are in Testing.

Detector catalogue ​

Classification, rule, element, detector and policy are different things ​

Version coordinates ​

How a detector reaches a verdict ​

The data categories a detector may report ​

Category, severity and sensitivity labels are separate ​

Core pack - core v1.0.0, locale-independent ​

Financial ​

Contact and identity ​

Credentials - all redact fully ​

Organisational ​

Country packs ​

Azerbaijan - az v1.22.0 ​

Türkiye - tr v1.8.0 ​

Russian identifiers - ru v1.1.0 ​

Gulf identifiers - gcc v1.0.0, locale ar ​

Delivery and activation ​

Procurement documents - procurement v1.1.0, locale multi ​

A published notice is not a leak, and says so ​

Geospatial pack - geospatial v1.0.0, locale multi ​

The box is a field, not a constant ​

A set of coordinates, not a coordinate ​

How much of this is true, measured (DLP-425) ​

Transliteration ​

Composition ​

Exact Data Match ​

How large a dataset can be ​

Read on the endpoint ​

Not implemented ​

Benchmarks ​