Filesystem monitoring, claim by claim

WIN-DLP-EPIC-007 asks for file creation, modification, rename and deletion to be observed reliably on Windows, including the cases that break naive implementations - because data in motion is where loss happens, and missing an event is silently losing coverage.

The four event kinds are observed. Several of the naive-implementation cases are handled and tested; the remaining Windows-specific gaps are named below instead of being hidden behind a general claim of reliable delivery.

Implementation: agent/core/src/monitor.rs, over ReadDirectoryChangesW on Windows and inotify on Linux through the notify crate.

---

1. The four event kinds

Kind	Channel	Proved by
Create	FileCreate	monitor::tests::file_creation_is_observed
Modify	FileModify	monitor::tests::repeated_writes_to_one_file_coalesce
Rename	FileRename	classify maps ModifyKind::Name; no test exercises a real rename - see §5
Delete	FileDelete	monitor::tests::a_deletion_is_its_own_channel_rather_than_being_ignored

Deletion has its own channel because "deletion after a copy is a common pattern, and ignoring deletions meant the second half of it was invisible" (#364). It is answered from what was already known about the file rather than by scanning, because a deleted file has no content left to read.

2. Coalescing - and the event it used to swallow

Editors write a file several times in quick succession, so drain collapses repeats. It used to collapse them by path alone, which meant the first event for a path won and everything after it was dropped.

So a file written and then deleted inside one drain window reported the write and lost the deletion. That is the same coverage gap #364 was filed to close, reappearing one layer down: cp x y && rm x is a script, not an editor, and its two halves arrive milliseconds apart.

Fixed (DLP-121): the key is now (path, channel). Within a channel the first event still wins, so ten writes to one file are still one scan - but a deletion is not a repeat of a write and no longer looks like one.

Claim	Proved by
Ten writes to one file coalesce to one event	monitor::tests::repeated_writes_to_one_file_coalesce
A deletion after a write survives the same window	monitor::tests::a_deletion_after_a_write_is_not_coalesced_away

The first test was tightened while fixing the second. It now creates the file and drains before measuring, so what it measures is ten modifications rather than a creation followed by modifications - which are two different facts about a file, and collapsing them was the bug.

3. Exclusions - the case that lost coverage silently

Claim	Proved by
An exclusion applies to its own monitored path only	monitor::tests::an_exclusion_does_not_leak_onto_another_monitored_path
Nested monitored paths both apply their exclusions	monitor::tests::nested_monitored_paths_both_apply_their_exclusions
A sibling directory sharing a prefix is not the same root	monitor::tests::a_sibling_directory_sharing_a_prefix_is_not_the_same_root
A path under no monitored root is not excluded	monitor::tests::a_path_under_no_monitored_root_is_not_excluded
A path that is not watched contributes no exclusions	monitor::tests::a_path_that_is_not_watched_contributes_no_exclusions
Matching ignores case	monitor::tests::exclusions_are_matched_without_regard_to_case
The agent's own state directory is excluded by default, wherever it is configured	monitor::tests::default_exclusions_cover_the_agents_own_state
The defaults reach the rules the monitor applies	monitor::tests::the_defaults_reach_the_rules_the_monitor_applies
A folder anybody can name (node_modules, .git, caches, build outputs) is not a default exclusion	monitor::tests::a_folder_anybody_can_name_is_not_a_default_exclusion
A .. cannot walk a file out from under an exclusion check	monitor::tests::a_dot_dot_cannot_walk_a_file_out_from_under_an_exclusion_check

Exclusions used to be pooled - every path's list applied to every path - so an exclusion written for one directory silently suppressed events under another (DLP-148). Three of the rows above exist because of that one defect, and the fourth is its mirror image: an event from a path the roots cannot explain is kept, because dropping it would be the same silent loss in a different shape.

4. Cases that break naive implementations

Case	State
A cloud placeholder is reported, never hydrated	Handled - monitor::tests::a_dehydrated_placeholder_is_reported_not_read, …::each_recall_attribute_has_its_own_sentence, …::the_most_specific_attribute_wins_when_several_are_set
An ordinary local file is read normally	monitor::tests::a_real_local_file_is_read_normally, …::an_ordinary_file_is_not_a_placeholder
Symlinks are not followed	monitor::tests::symlinks_are_not_followed
A monitored directory that does not exist	Skipped, not fatal - monitor::tests::a_missing_directory_is_skipped_not_fatal
Watcher buffer overflow and rapid change	Backend errors are classified, counted and followed by a bounded rescan (DLP-165); a scheduled bounded re-walk covers loss the Windows backend does not report (DLP-300). Physical overflow and rename-drain stress remains DLP-448.
Long paths beyond MAX_PATH	Handled (DLP-164). The scan opens every file through long_path::for_open, which adds the \\?\ prefix past 260 characters (detector_runtime.rs), and so do the monitor's walk and the runner's metadata read. Reporting and policy use the unprefixed normalised path. Pure and unit-tested (long_path::tests, e.g. a_long_local_path_gets_the_prefix, a_long_unc_path_gets_the_unc_form); not run against a real 300-character path on Windows by this audit
Alternate data streams	Known, reported, not read (DLP-166). Named streams are listed and put on the file's event as labels, so an operator sees report.txt:payload exists. The scan still reads only the default stream, by decision: reading every stream multiplies scan I/O by whatever an attacker attaches. Zone.Identifier is skipped by exact name (alternate_streams::tests::hiding_behind_the_exemption_does_not_work)
The USN change journal as a source	Handled at service start on local NTFS/ReFS volumes - usn::windows_impl::tests::a_real_ntfs_journal_finds_a_file_changed_after_the_checkpoint (DLP-167)
A file that stops the reader	Handled (DLP-1695). A read is written to reads-in-flight.jsonl in the state directory before it starts and after it ends, so a read that began and never ended is one the process died in: a parser panic aborts a release build, and SCM restarts it into the same file, which is how one public PDF kept the service in a loop for good (DLP-1619). Each such death is a strike against that version of the file (size and modification time) in poison-files.json. A death during a stop the operator asked for is not a strike, and releases the strikes that file already holds (DLP-2319). Strikes written before stop notes existed (a ledger without the rule field) are forgiven once, when the first build that knows the rule opens it; a real crash after that counts from zero again. The stop is noted on disk when it is requested, so a kill mid-read still leaves the note. At two strikes the file is not read until it changes. It is reported as not inspected, cause reader_stopped, so it is never counted clean, and it is named, masked, in the log when the walk reaches it. A read that ends clears the strikes. Both files hold a short hash of each path, never the path. Tested through the real runner, crashes simulated by the journal they leave: nothing_sensitive_leaks::a_file_the_agent_stopped_on_twice_is_announced_not_read_until_it_changes, and poison::tests. Not run against a real crash on Windows

Reading a placeholder is the one worth restating: hydrating a file to scan it would itself be a download, caused by the scanner, of the content the scanner was asked to watch for. The attribute is reported instead.

5. What is not covered

• No test exercises a real rename. classify maps ModifyKind::Name to FileRename and a unit test covers the mapping, but nothing renames a file and observes the event. On Linux a rename within a watched tree produces two Name events (from and to) and on Windows a single rename pair, and which of those the agent sees is a platform behaviour worth pinning. Not filed as a ticket: it is a test to write, and it belongs with DLP-212's Windows watcher work rather than as its own item.
• The Windows backend does not surface every overflow. Errors that reach the monitor callback are pessimistically classified, counted in watcher_losses_total, exposed as degraded health and followed by a bounded rescan of the affected monitored root (DLP-165). The notify Windows backend can still receive an empty ReadDirectoryChangesW result without calling that callback. The scheduled bounded re-walk (DLP-300) limits that unreported gap by discovering the current file-system state on the next pass. It cannot reconstruct a file created and deleted between passes. notify-rs merged the callback fix in PR #964, but it is currently available only in the 9.0.0-rc line; MyDLP remains on stable 8.2.0 rather than putting a release-candidate crate under its primary monitoring input. DLP-212 wrote the physical Windows overflow and split-rename stress test (windows-stress.yml); it has not yet passed (Windows CI status).
• Nothing here has been run on Windows by this audit. The notify crate selects ReadDirectoryChangesW there; these tests run on inotify. The windows-agent CI job is written to run the same tests on windows-latest, and it has not yet passed on windows-latest (Windows CI status).

6. Changes made while the service was stopped

On Windows the watcher is opened first, then the agent queries each local volume with FSCTL_QUERY_USN_JOURNAL and reads forward from the persisted (journal id, next USN) checkpoint using FSCTL_READ_USN_JOURNAL. Starting the watcher first closes the otherwise unavoidable gap between taking the journal snapshot and ReadDirectoryChangesW becoming live.

The journal is volume-wide, but the product is not. For every record the agent opens only its parent directory by file reference number and proves that directory is under a monitored root. Only then is the record's UTF-16 file name decoded. An out-of-scope parent is cached as Outside; neither its path nor the record name is logged, queued or retained. The only observable fact is an aggregate discarded-record count.

The persisted file usn-checkpoints.json contains no file names. A checkpoint advances only after every in-scope path from that volume was processed and the scan cache was written. A recreated journal, a wrapped checkpoint, a restored volume whose checkpoint is ahead, a disabled journal, a non-local/non-NTFS root, a corrupt checkpoint file, or a removed volume causes a stated full-walk fallback. The old checkpoint is retained when that fallback or a file scan does not complete, so restart retries the interval rather than silently blessing it.

Since DLP-1539 the journal is also asked, while the agent runs, why a path the watcher reported removed is gone. ReadDirectoryChangesW reports a deletion and a move out of the watched folders alike. For each drained batch with a removal, the agent reads the volume's rename and deletion records from where its last lookup stopped (the first lookup reads back at most 8 MiB). A record's name is decoded only under a monitored parent, and only to compare it with the path already reported. For the new name of a renamed file, the agent keeps only where it went: inside the monitored paths, into a Recycle Bin, or elsewhere. The answer is deleted_permanently, recycled or moved_out. Where there is no journal or no record, the path stays deleted, which means only "no longer here". Nothing from the lookup is written to disk.

Known boundary: USN reconciliation is a Windows start-up path and cannot be executed on the Linux development host. The Windows-only integration test uses the real volume journal and requires the administrator rights the LocalSystem service has; the GNU Windows target check proves the FFI and runner path still compile here.