mirror of
https://github.com/karatektus/pl_mail.git
synced 2026-10-25 01:54:10 +00:00
Mail deleted on the server stops existing here too
v0.0.25 made a message that moves stay one message, and said in as many words what it could not do: there is no vanished-UID detection anywhere in the syncer. Incremental sync asks for `lastSeenUid+1:*`, which is what makes polling a fifty-thousand-message folder cost nothing and is exactly why no amount of polling can notice a message leaving. A UID below the high-water mark is never looked at again. So a remote move healed only as a side effect of collapsing duplicate Message-IDs — bounded, and needing two rows and an id to work with — and a remote deletion produced neither, so it was never reflected at all. Mail deleted in Roundcube last month is still in plMail today. Learning what left requires asking what is still there, and there is no cheaper question. RFC 7162 is the right answer and is unavailable: webklex implements neither QRESYNC nor CONDSTORE — no ENABLE, no MODSEQ, no VANISHED anywhere in the library — so the choice is EXAMINE plus UID SEARCH ALL, universally supported since IMAP4rev1 and far too expensive for the poll loop. It therefore runs on its own clock, a quarter of an hour, which is also the floor on how quickly a deletion shows up locally. The discipline is v0.0.25's, applied to the one operation the next poll cannot undo. Absence from one listing is evidence and never proof, because from the folder being left a deletion, a move to a folder this cycle has not reached, and a listing that was simply wrong are indistinguishable. So a sweep never deletes: it records that the row went missing and when, and leaves the row's address alone — which is what lets the existing machinery keep working on it, since a message that turns up elsewhere is met as a duplicate Message-ID, probed with ImapUidPresence, and relocated by the path that already existed. Deletion takes two further conditions on top. Coverage: every sync-enabled folder listed in full since the row vanished, none of them producing it, which is why earliestSweepAcross() answers null — refusing to reap anything — while even one folder has never been swept. And confirmation: one last tri-state probe of the row's own address, where present clears the mark, "could not tell" changes nothing, and only an explicit no from the server erases mail. A folder that lists empty having held four thousand messages an hour ago has not been emptied by hand; it has been rebuilt, restored, or is being served by something that lost its index, and reading that as four thousand deletions is how a mail client destroys a mailbox in one poll. So a sweep refuses itself when it would mark most of a folder at once, and a refusal withholds coverage from the whole account rather than only from that folder. A changed UIDVALIDITY is handled as what it is: every stored UID is void and no mail is, so the rows are unlocated and the folder re-read from the start, which is a re-match by Message-ID rather than a wipe. Erasing now goes through one path instead of the two that had been written months apart and agreed on nothing — one deleted attachment files, the other did not, neither touched rawPath, and neither re-synchronised the thread's labels. That was survivable while the only rows being removed were duplicates whose files another row still pointed at. A message the server no longer has is not a duplicate of anything. The outgoing half gains the tests it never had: every action was asserted only by reading the match statement and believing it. They pass as written, which makes `delete` an expunge rather than a move to Trash on the record rather than by inspection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Paul LütznerandClaude Fable 5 committed
1 parent
279ac36fde
commit
3d0712389f
12 files changed
+2370
-9
No files matched your search
@@ -0,0 +1,55 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace DoctrineMigrations;
|
||||
|
||||
use Doctrine\DBAL\Schema\Schema;
|
||||
use Doctrine\Migrations\AbstractMigration;
|
||||
|
||||
/**
|
||||
* The two dates that let a sync notice mail leaving.
|
||||
*
|
||||
* `message.vanished_at` is when a folder listing last failed to produce a row
|
||||
* where it says it is, and `mailbox.swept_at` is when that folder was last
|
||||
* listed in full. Neither is a deletion on its own — the first is evidence and
|
||||
* the second is coverage, and only both together, plus a per-row confirmation
|
||||
* from the server, let VanishedMessageReconciler erase anything.
|
||||
*
|
||||
* Both nullable, and null is the safe reading of both. A row that has never
|
||||
* gone missing has no vanished_at; a folder that has never been listed in full
|
||||
* has no swept_at, and MailboxRepository::earliestSweepAcross() answers null
|
||||
* for the whole account while even one of them is in that state, which suspends
|
||||
* reaping until the first complete pass has happened. So an install that takes
|
||||
* this migration deletes nothing until it has read every folder at least once,
|
||||
* without any special case for the upgrade.
|
||||
*
|
||||
* down() drops both columns. That is genuinely reversible: they are derived
|
||||
* state which the next sweep rebuilds from the server, and nothing else reads
|
||||
* them.
|
||||
*/
|
||||
final class Version20260811090000 extends AbstractMigration
|
||||
{
|
||||
public function getDescription(): string
|
||||
{
|
||||
return 'Record when a message went missing from its folder and when each folder was last listed in full';
|
||||
}
|
||||
|
||||
public function up(Schema $schema): void
|
||||
{
|
||||
$this->addSql('ALTER TABLE message ADD vanished_at TIMESTAMP(0) WITHOUT TIME ZONE DEFAULT NULL');
|
||||
$this->addSql('ALTER TABLE mailbox ADD swept_at TIMESTAMP(0) WITHOUT TIME ZONE DEFAULT NULL');
|
||||
|
||||
// The reaper's only query: rows of one account that carry a mark,
|
||||
// oldest first. Without this it is a sequential scan of every message
|
||||
// in the account on every poll of every folder.
|
||||
$this->addSql('CREATE INDEX idx_message_vanished ON message (account_id, vanished_at)');
|
||||
}
|
||||
|
||||
public function down(Schema $schema): void
|
||||
{
|
||||
$this->addSql('DROP INDEX idx_message_vanished');
|
||||
$this->addSql('ALTER TABLE message DROP vanished_at');
|
||||
$this->addSql('ALTER TABLE mailbox DROP swept_at');
|
||||
}
|
||||
}
|
||||
@@ -58,4 +58,29 @@ readonly class RawMessageStorage
|
||||
{
|
||||
return is_file($this->getAbsolutePath($relativePath));
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove a stored .eml, when the row that named it is going away.
|
||||
*
|
||||
* Guarded the same way AttachmentStorageHelper::delete() is, and for the
|
||||
* same reason: rawPath is a column other code writes provider schemes into,
|
||||
* and a path that is not one of ours must be ignored rather than resolved
|
||||
* against the project root and unlinked.
|
||||
*/
|
||||
public function delete(?string $relativePath): void
|
||||
{
|
||||
if (null === $relativePath || false === str_starts_with($relativePath, $this->storageDir.'/raw/')) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (true === str_contains($relativePath, '..')) {
|
||||
return;
|
||||
}
|
||||
|
||||
$path = $this->getAbsolutePath($relativePath);
|
||||
|
||||
if (true === is_file($path)) {
|
||||
unlink($path);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -93,6 +93,24 @@ class Mailbox
|
||||
#[ORM\Column(nullable: true)]
|
||||
public ?\DateTimeImmutable $syncedAt = null;
|
||||
|
||||
/**
|
||||
* When this folder's full UID set was last read off the server and compared
|
||||
* against the rows claiming to be in it.
|
||||
*
|
||||
* Separate from syncedAt because it happens on a slower clock. syncedAt is
|
||||
* every poll and asks only for UIDs above the high-water mark, which is
|
||||
* what makes incremental sync cheap and is also exactly why it can never
|
||||
* notice a message leaving. The sweep asks for all of them, so it runs on a
|
||||
* cadence instead — see VanishedMessageReconciler::SWEEP_INTERVAL.
|
||||
*
|
||||
* It is also the coverage half of the deletion rule. A row that vanished is
|
||||
* only erased once every folder's sweptAt is later than the instant it
|
||||
* vanished at, which is the difference between "no folder has it" and "the
|
||||
* folders we happen to have looked at do not have it".
|
||||
*/
|
||||
#[ORM\Column(nullable: true)]
|
||||
public ?\DateTimeImmutable $sweptAt = null;
|
||||
|
||||
|
||||
|
||||
/**
|
||||
|
||||
@@ -32,6 +32,10 @@ use Doctrine\ORM\Mapping as ORM;
|
||||
// the provider-id constraints above are ordered.
|
||||
#[ORM\Index(name: 'idx_message_message_id', columns: ['message_id'])]
|
||||
#[ORM\Index(name: 'idx_message_provider_thread_key_account', columns: ['provider_thread_key', 'account_id'])]
|
||||
// The reaper's only query: rows of one account carrying a vanish mark, oldest
|
||||
// first. Without it, every poll of every folder sequentially scans the
|
||||
// account's messages to find the handful that went missing.
|
||||
#[ORM\Index(name: 'idx_message_vanished', columns: ['account_id', 'vanished_at'])]
|
||||
#[ORM\Entity(repositoryClass: MessageRepository::class)]
|
||||
#[ORM\HasLifecycleCallbacks]
|
||||
class Message extends MessageModel
|
||||
@@ -57,6 +61,25 @@ class Message extends MessageModel
|
||||
#[ORM\Column(nullable: true)]
|
||||
public ?int $imapUid = null;
|
||||
|
||||
/**
|
||||
* When a folder listing last failed to find this row where it says it is.
|
||||
*
|
||||
* Not a deletion, and deliberately not one. A UID missing from one listing
|
||||
* is evidence: the message may equally have been moved to a folder this
|
||||
* sync cycle has not reached yet, and the same absence is what both events
|
||||
* look like from the folder being left. So the syncer records the absence,
|
||||
* clears the row's address so SentCopyReconciler::claim() can re-match it
|
||||
* by Message-ID wherever it turns up, and waits.
|
||||
*
|
||||
* What converts it into a deletion is time plus coverage, never the single
|
||||
* listing: VanishedMessageReconciler::reap() removes a row only once every
|
||||
* sync-enabled folder in the account has been listed since this instant and
|
||||
* none of them produced it. Any listing that does produce it gives the row
|
||||
* an address again, and relocateTo() clears this back to null.
|
||||
*/
|
||||
#[ORM\Column(nullable: true)]
|
||||
public ?\DateTimeImmutable $vanishedAt = null;
|
||||
|
||||
#[ORM\Column(type: Types::TEXT, nullable: true)]
|
||||
public ?string $messageId = null;
|
||||
|
||||
@@ -345,6 +368,13 @@ class Message extends MessageModel
|
||||
$this->mailbox = $mailbox;
|
||||
$this->imapUid = $uid;
|
||||
|
||||
// A row that has an address again is not missing. Whatever listing
|
||||
// failed to produce it, this one did, and that is the answer that
|
||||
// counts — see $vanishedAt.
|
||||
if (null !== $uid) {
|
||||
$this->vanishedAt = null;
|
||||
}
|
||||
|
||||
return $this;
|
||||
}
|
||||
}
|
||||
@@ -52,6 +52,58 @@ class MailboxRepository extends ServiceEntityRepository
|
||||
return $this->findOneBy(['account' => $account, 'specialUse' => MailboxSpecialUse::DRAFTS]);
|
||||
}
|
||||
|
||||
/**
|
||||
* The oldest full-listing this account's folders have between them, or null
|
||||
* if any of them has never had one.
|
||||
*
|
||||
* This is the coverage guarantee behind remote deletion. A row that went
|
||||
* missing from one folder may simply be in another, so the only moment its
|
||||
* absence means anything is once *every* folder has been listed since — and
|
||||
* the folder that was listed longest ago is what that comes down to.
|
||||
*
|
||||
* Null is the important answer and is why this returns one rather than
|
||||
* skipping the folders it cannot speak for. A single never-swept folder
|
||||
* means the account has no such moment yet, and nothing may be erased on
|
||||
* the strength of the folders that have been looked in.
|
||||
*
|
||||
* Sync-disabled folders are excluded because nothing ever lists them, so
|
||||
* including them would make the answer permanently null. The cost is stated
|
||||
* where it lands: a message moved by hand into a folder the user has turned
|
||||
* sync off for is, to plMail, a message that left the account.
|
||||
*/
|
||||
public function earliestSweepAcross(Account $account): ?\DateTimeImmutable
|
||||
{
|
||||
/** @var list<array{sweptAt: ?\DateTimeImmutable}> $rows */
|
||||
$rows = $this->createQueryBuilder('mailbox')
|
||||
->select('mailbox.sweptAt')
|
||||
->where('mailbox.account = :account')
|
||||
->andWhere('mailbox.isSyncEnabled = :enabled')
|
||||
->setParameter('account', $account)
|
||||
->setParameter('enabled', true)
|
||||
->getQuery()
|
||||
->getArrayResult();
|
||||
|
||||
if (0 === count($rows)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$earliest = null;
|
||||
|
||||
foreach ($rows as $row) {
|
||||
$sweptAt = $row['sweptAt'];
|
||||
|
||||
if (null === $sweptAt) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (null === $earliest || $sweptAt < $earliest) {
|
||||
$earliest = $sweptAt;
|
||||
}
|
||||
}
|
||||
|
||||
return $earliest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Mailboxes an IDLE supervisor should hold a connection for.
|
||||
*
|
||||
|
||||
@@ -764,6 +764,152 @@ class MessageRepository extends ServiceEntityRepository
|
||||
->getSingleColumnResult();
|
||||
}
|
||||
|
||||
/**
|
||||
* Every address this mailbox holds, as row id => UID.
|
||||
*
|
||||
* findSyncedUids() answers "have I seen this UID", which is all the
|
||||
* incremental path needs. The sweep asks the opposite question — which of
|
||||
* my rows is the server no longer offering — and to answer it has to name
|
||||
* the rows, not merely count them.
|
||||
*
|
||||
* @return array<int,int> messageId => imapUid
|
||||
*/
|
||||
public function findLocatedUidsById(Mailbox $mailbox): array
|
||||
{
|
||||
$rows = $this->createQueryBuilder('m')
|
||||
->select('m.id', 'm.imapUid')
|
||||
->where('m.mailbox = :mailbox')
|
||||
->andWhere('m.imapUid IS NOT NULL')
|
||||
->setParameter('mailbox', $mailbox)
|
||||
->getQuery()
|
||||
->getArrayResult();
|
||||
|
||||
$located = [];
|
||||
|
||||
foreach ($rows as $row) {
|
||||
$located[(int) $row['id']] = (int) $row['imapUid'];
|
||||
}
|
||||
|
||||
return $located;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record that these rows were not in their folder's listing.
|
||||
*
|
||||
* A bulk update rather than hydrated entities: a sweep of a large folder
|
||||
* after a bad night can name thousands of rows, and loading a Message per
|
||||
* row to write one nullable column would turn the safety net into the
|
||||
* slowest thing in the poll.
|
||||
*
|
||||
* `vanishedAt IS NULL` in the predicate keeps the *first* absence as the
|
||||
* instant that counts. Refreshing it on every sweep would push the deadline
|
||||
* out forever and nothing would ever be reaped.
|
||||
*
|
||||
* @param list<int> $ids
|
||||
*/
|
||||
public function markVanished(array $ids, \DateTimeImmutable $at): int
|
||||
{
|
||||
return $this->updateVanishedAt($ids, $at, onlyUnmarked: true);
|
||||
}
|
||||
|
||||
/**
|
||||
* Take the mark off rows the server has just produced after all.
|
||||
*
|
||||
* @param list<int> $ids
|
||||
*/
|
||||
public function clearVanished(array $ids): int
|
||||
{
|
||||
return $this->updateVanishedAt($ids, null, onlyUnmarked: false);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<int> $ids
|
||||
*/
|
||||
private function updateVanishedAt(array $ids, ?\DateTimeImmutable $at, bool $onlyUnmarked): int
|
||||
{
|
||||
if (0 === count($ids)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
$affected = 0;
|
||||
|
||||
// Chunked because Postgres has a bind-parameter ceiling and a folder
|
||||
// that was rebuilt server-side can put every row it has in this list.
|
||||
foreach (array_chunk($ids, 1000) as $chunk) {
|
||||
$qb = $this->getEntityManager()->createQueryBuilder()
|
||||
->update(Message::class, 'm')
|
||||
->set('m.vanishedAt', ':at')
|
||||
->where('m.id IN (:ids)')
|
||||
->setParameter('at', $at)
|
||||
->setParameter('ids', $chunk);
|
||||
|
||||
if (true === $onlyUnmarked) {
|
||||
$qb->andWhere('m.vanishedAt IS NULL');
|
||||
} else {
|
||||
$qb->andWhere('m.vanishedAt IS NOT NULL');
|
||||
}
|
||||
|
||||
$affected += (int) $qb->getQuery()->execute();
|
||||
}
|
||||
|
||||
return $affected;
|
||||
}
|
||||
|
||||
/**
|
||||
* Strip every stored UID in this folder, keeping the rows.
|
||||
*
|
||||
* What a UIDVALIDITY change means and the most that may be done about it.
|
||||
* The addresses are void; the mail is not, and the server may still have
|
||||
* all of it. An unlocated row is the shape SentCopyReconciler::claim()
|
||||
* re-matches by Message-ID when the folder is re-read, so this is a
|
||||
* re-match and not a wipe.
|
||||
*
|
||||
* The vanish marks go with them: a row with no address cannot be confirmed
|
||||
* gone by anything, so leaving it marked would only give the reaper
|
||||
* questions it can never answer.
|
||||
*/
|
||||
public function unlocateAll(Mailbox $mailbox): int
|
||||
{
|
||||
return (int) $this->getEntityManager()->createQueryBuilder()
|
||||
->update(Message::class, 'm')
|
||||
->set('m.imapUid', ':nothing')
|
||||
->set('m.vanishedAt', ':nothing')
|
||||
->where('m.mailbox = :mailbox')
|
||||
->andWhere('m.imapUid IS NOT NULL')
|
||||
->setParameter('nothing', null)
|
||||
->setParameter('mailbox', $mailbox)
|
||||
->getQuery()
|
||||
->execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* Rows that went missing before every folder had been looked in.
|
||||
*
|
||||
* The cutoff is the caller's business and is the whole safety of this: it
|
||||
* passes the *earliest* sweep across the account's folders, so a row only
|
||||
* comes back from here once every one of them has been listed since it
|
||||
* vanished and none produced it. See VanishedMessageReconciler::reap().
|
||||
*
|
||||
* Ordered by the instant they vanished so the oldest evidence is acted on
|
||||
* first, and capped, because erasing is the one thing here that cannot be
|
||||
* undone by the next poll.
|
||||
*
|
||||
* @return list<Message>
|
||||
*/
|
||||
public function findReapable(Account $account, \DateTimeImmutable $vanishedBefore, int $limit): array
|
||||
{
|
||||
return $this->createQueryBuilder('m')
|
||||
->where('m.account = :account')
|
||||
->andWhere('m.vanishedAt IS NOT NULL')
|
||||
->andWhere('m.vanishedAt < :before')
|
||||
->setParameter('account', $account)
|
||||
->setParameter('before', $vanishedBefore)
|
||||
->orderBy('m.vanishedAt', 'ASC')
|
||||
->setMaxResults($limit)
|
||||
->getQuery()
|
||||
->getResult();
|
||||
}
|
||||
|
||||
/**
|
||||
* QueryBuilder on two counts: the owner is reached through the account
|
||||
* association, which findBy() cannot traverse, and only one column comes
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Service\Imap;
|
||||
|
||||
use App\Entity\Mail\Mailbox;
|
||||
use Psr\Log\LoggerInterface;
|
||||
use Throwable;
|
||||
use Webklex\PHPIMAP\Client;
|
||||
use Webklex\PHPIMAP\IMAP;
|
||||
|
||||
/**
|
||||
* What a folder actually holds right now: its UIDVALIDITY and every UID in it.
|
||||
*
|
||||
* ## Why this exists at all
|
||||
*
|
||||
* Incremental sync asks for `lastSeenUid+1:*`, which is what makes a poll of a
|
||||
* fifty-thousand-message folder cost nothing — and is also, exactly, why no
|
||||
* amount of polling can ever notice a message leaving. A UID below the
|
||||
* high-water mark is never looked at again, so a message deleted on the server
|
||||
* stayed in plMail forever, and a message *moved* on the server only ever
|
||||
* healed as a side effect of the duplicate collapse in SentCopyReconciler,
|
||||
* which needs a Message-ID and a second row to work with. Neither is available
|
||||
* when mail is simply deleted, and deletion left no trace at all.
|
||||
*
|
||||
* Learning what left requires asking what is still there. There is no cheaper
|
||||
* question.
|
||||
*
|
||||
* ## Why UID SEARCH ALL and not QRESYNC
|
||||
*
|
||||
* RFC 7162 is the right answer to this problem and plMail cannot use it:
|
||||
* webklex/php-imap implements neither QRESYNC nor CONDSTORE — there is no
|
||||
* ENABLE, no MODSEQ parsing, no VANISHED response handling anywhere in the
|
||||
* library — so there is no supported path to a server-computed diff. Adding one
|
||||
* means implementing a protocol extension inside a vendored client, which is a
|
||||
* great deal more code than this and fails closed on every server that does not
|
||||
* advertise it anyway.
|
||||
*
|
||||
* `EXAMINE` followed by `UID SEARCH ALL` is two round trips, universally
|
||||
* supported since IMAP4rev1, and hands back a few hundred kilobytes for a
|
||||
* folder of fifty thousand. That is too much to do on every poll and nothing at
|
||||
* all to do every quarter of an hour, which is why VanishedMessageReconciler
|
||||
* puts it on a cadence rather than in the poll.
|
||||
*
|
||||
* EXAMINE rather than SELECT deliberately: it is the read-only form, so
|
||||
* measuring a folder never clears \Recent on it.
|
||||
*
|
||||
* ## Three answers, again
|
||||
*
|
||||
* Null means the server did not answer, and null is never evidence of anything
|
||||
* — the same discipline ImapUidPresence is built on and for the same reason,
|
||||
* since the caller's response to "these UIDs are gone" is eventually to delete
|
||||
* mail. A folder that cannot be examined is a folder that gets swept next time.
|
||||
*/
|
||||
final readonly class ImapFolderListing
|
||||
{
|
||||
public function __construct(
|
||||
private LoggerInterface $logger,
|
||||
) {
|
||||
}
|
||||
|
||||
/**
|
||||
* @return array{uidValidity: int, uids: array<int,true>}|null null when the
|
||||
* server did not answer. `uids` is a set rather than a list: the
|
||||
* caller's only question of it is membership, once per local row.
|
||||
*/
|
||||
public function read(Client $client, Mailbox $mailbox): ?array
|
||||
{
|
||||
$path = (string) ($mailbox->fullPath ?? $mailbox->name);
|
||||
|
||||
try {
|
||||
$connection = $client->getConnection();
|
||||
|
||||
$examined = $connection->examineFolder($path)->validatedData();
|
||||
|
||||
if (false === is_array($examined) || false === isset($examined['uidvalidity'])) {
|
||||
// A server that selects a folder without telling us its
|
||||
// UIDVALIDITY has not given us enough to be safe with. Every
|
||||
// conclusion below rests on the UIDs being comparable to the
|
||||
// ones stored, and that is precisely what UIDVALIDITY asserts.
|
||||
$this->logger->info('Folder listing skipped: no UIDVALIDITY in the EXAMINE response', [
|
||||
'mailbox' => $path,
|
||||
]);
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
$found = $connection->search(['ALL'], IMAP::ST_UID)->validatedData();
|
||||
|
||||
if (false === is_array($found)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
$uids = [];
|
||||
|
||||
foreach ($found as $uid) {
|
||||
$uids[(int) $uid] = true;
|
||||
}
|
||||
|
||||
return [
|
||||
'uidValidity' => (int) $examined['uidvalidity'],
|
||||
'uids' => $uids,
|
||||
];
|
||||
} catch (Throwable $e) {
|
||||
$this->logger->info('Could not list a folder in full', [
|
||||
'mailbox' => $path,
|
||||
'error' => $e->getMessage(),
|
||||
]);
|
||||
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -37,6 +37,7 @@ class MessageSyncer
|
||||
private readonly HeaderNormalizer $headerNormalizer,
|
||||
private readonly SentCopyReconciler $sentCopies,
|
||||
private readonly ImapConnectionFactory $connections,
|
||||
private readonly VanishedMessageReconciler $vanished,
|
||||
) {}
|
||||
|
||||
/**
|
||||
@@ -81,8 +82,24 @@ class MessageSyncer
|
||||
|
||||
public function syncMailbox(Mailbox $mailbox, Client $client): void
|
||||
{
|
||||
$mailboxId = $mailbox->id;
|
||||
$accountId = $mailbox->account->id;
|
||||
$mailboxId = $mailbox->id;
|
||||
$accountId = $mailbox->account->id;
|
||||
|
||||
$folder = $client->getFolder($mailbox->name);
|
||||
|
||||
if (null === $folder) {
|
||||
$this->logger->error('Folder not found', ['mailbox' => $mailbox->name]);
|
||||
return;
|
||||
}
|
||||
|
||||
// Before anything reads lastSeenUid or the stored UIDs, because both can
|
||||
// change here: a folder rebuilt on the server has its high-water mark
|
||||
// reset to zero and every stored UID stripped, and the incremental pass
|
||||
// below has to be planned against that rather than against the state
|
||||
// this method was entered with. On the ordinary poll this is a no-op —
|
||||
// it only lists a folder every SWEEP_INTERVAL_MINUTES.
|
||||
$this->vanished->sweep($mailbox, $client);
|
||||
|
||||
$lastSeenUid = $mailbox->lastSeenUid ?? 0;
|
||||
$uidRange = ($lastSeenUid + 1) . ':*';
|
||||
|
||||
@@ -92,13 +109,6 @@ class MessageSyncer
|
||||
'lastSeenUid' => $lastSeenUid,
|
||||
]);
|
||||
|
||||
$folder = $client->getFolder($mailbox->name);
|
||||
|
||||
if (null === $folder) {
|
||||
$this->logger->error('Folder not found', ['mailbox' => $mailbox->name]);
|
||||
return;
|
||||
}
|
||||
|
||||
// Load all already-synced UIDs up front so each batch can O(1)-skip them.
|
||||
// array_flip turns [123, 456, …] into [123 => 0, 456 => 1, …].
|
||||
$syncedUids = array_flip(
|
||||
@@ -134,6 +144,14 @@ class MessageSyncer
|
||||
// because a move left the first one behind. Also one indexed query
|
||||
// that finds nothing on an account that never had the bug.
|
||||
$this->sentCopies->repairRelocated($mailbox, $presence);
|
||||
|
||||
// Last, and account-wide rather than folder-wide, because the
|
||||
// question it settles is "is this message anywhere" and no single
|
||||
// folder can answer it. Does nothing until every folder in the
|
||||
// account has been swept, and nothing then either unless the server
|
||||
// confirms the absence one row at a time. See
|
||||
// VanishedMessageReconciler.
|
||||
$this->vanished->reap($mailbox->account, $presence);
|
||||
} finally {
|
||||
$presence->close();
|
||||
}
|
||||
|
||||
@@ -0,0 +1,398 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Service\Imap;
|
||||
|
||||
use App\Entity\Mail\Account;
|
||||
use App\Entity\Mail\Mailbox;
|
||||
use App\Repository\Mail\MailboxRepository;
|
||||
use App\Repository\Mail\MessageRepository;
|
||||
use App\Service\Mail\MessageEraser;
|
||||
use DateTimeImmutable;
|
||||
use Doctrine\ORM\EntityManagerInterface;
|
||||
use Psr\Log\LoggerInterface;
|
||||
use Webklex\PHPIMAP\Client;
|
||||
|
||||
/**
|
||||
* Mail that left the server, and what plMail is allowed to conclude from that.
|
||||
*
|
||||
* v0.0.25 made a message that moves stay one message. It could not make a
|
||||
* message that is deleted stop existing, and said so: there is no vanished-UID
|
||||
* detection anywhere in the syncer, because `lastSeenUid+1:*` never looks at an
|
||||
* old UID again. A remote move healed only as a side effect of collapsing
|
||||
* duplicates — bounded, and needing a Message-ID and a second row. A remote
|
||||
* delete produced neither, so it was never reflected at all. Mail the user
|
||||
* deleted in Roundcube last month is still in plMail today.
|
||||
*
|
||||
* ## The rule
|
||||
*
|
||||
* Absence from one folder's listing is evidence. It is not proof of anything,
|
||||
* because a message missing from INBOX is equally consistent with three
|
||||
* different events, and from INBOX they are indistinguishable:
|
||||
*
|
||||
* - it was deleted;
|
||||
* - it was moved to a folder this cycle has not reached yet;
|
||||
* - the listing was wrong, because the connection wobbled or the server
|
||||
* answered a question about a folder it had just rebuilt.
|
||||
*
|
||||
* So a sweep never deletes. It writes down that the row went missing and when —
|
||||
* Message::$vanishedAt — and leaves the row's address alone, which is what lets
|
||||
* the existing v0.0.25 machinery keep working on it: if the message turns up in
|
||||
* another folder, that folder's sync meets it as a duplicate Message-ID, probes
|
||||
* the source with ImapUidPresence, finds it gone, and relocates the row. The
|
||||
* move repairs itself through the path that already existed, and relocateTo()
|
||||
* clears the mark on the way past.
|
||||
*
|
||||
* Deletion is what is left when that does not happen, and it takes two further
|
||||
* conditions, neither of which any single listing can satisfy:
|
||||
*
|
||||
* 1. **Coverage.** Every sync-enabled folder in the account has been listed
|
||||
* in full since the instant the row vanished, and none of them produced
|
||||
* it. That is what MailboxRepository::earliestSweepAcross() computes, and
|
||||
* why it answers null — refusing to reap anything — while even one folder
|
||||
* has never been swept.
|
||||
*
|
||||
* 2. **Confirmation.** One last tri-state probe of the row's own address,
|
||||
* through the same ImapUidPresence the move reconciliation uses. Present
|
||||
* clears the mark; "could not tell" changes nothing; only an explicit
|
||||
* "not there" from the server erases mail. A sweep that went wrong is
|
||||
* undone here rather than acted on.
|
||||
*
|
||||
* ## The empty listing
|
||||
*
|
||||
* A folder that answers `UID SEARCH ALL` with nothing, having held four
|
||||
* thousand messages an hour ago, is not a folder somebody emptied by hand. It
|
||||
* is a folder that has been rebuilt, restored from backup, or is being served
|
||||
* by something that has lost its index — and treating that answer as four
|
||||
* thousand deletions is how a mail client destroys a mailbox in one poll.
|
||||
*
|
||||
* So a sweep refuses itself when it would mark most of a folder as missing at
|
||||
* once. Not merely the empty case: the same guard covers "nine tenths of INBOX
|
||||
* disappeared", which is the same accident with a less obvious shape. Below
|
||||
* BULK_FLOOR rows the guard does not apply, because a small folder genuinely
|
||||
* being emptied is an ordinary thing a user does and refusing it forever would
|
||||
* mean plMail never notices.
|
||||
*
|
||||
* A refusal leaves sweptAt untouched, which is deliberate: an unswept folder
|
||||
* withholds coverage from the whole account, so a suspicious answer in one
|
||||
* folder suspends reaping everywhere until it is understood.
|
||||
*
|
||||
* ## UIDVALIDITY
|
||||
*
|
||||
* A changed UIDVALIDITY says every UID stored for this folder now addresses
|
||||
* either nothing or somebody else's mail. It says nothing whatsoever about the
|
||||
* mail, which the server may still have all of. So it is not a deletion event
|
||||
* and must never become one: the rows are *unlocated* — stripped of the UIDs
|
||||
* that stopped meaning anything, left in the folder they are in — and the
|
||||
* high-water mark is reset so the next sync walks the folder from the start and
|
||||
* SentCopyReconciler::claim() re-matches each row by the Message-ID that
|
||||
* survived. A rebuild costs one full re-read, and no mail.
|
||||
*
|
||||
* The vanish marks are cleared in the same breath, and that is not tidiness. A
|
||||
* marked row is a candidate for erasure, and after an invalidation none of them
|
||||
* has an address left to confirm against, so leaving the marks on would hand
|
||||
* the reaper a folder's worth of rows it could only ever answer "cannot tell"
|
||||
* about. Rows the re-read fails to re-match stay in place, unlocated: dead
|
||||
* weight rather than lost mail, and `app:reset` is the cure.
|
||||
*/
|
||||
final readonly class VanishedMessageReconciler
|
||||
{
|
||||
/**
|
||||
* How long a folder's full listing stays fresh enough not to redo.
|
||||
*
|
||||
* The listing is two round trips and a few hundred kilobytes for a large
|
||||
* folder, which is far too much for the poll loop and nothing at all on
|
||||
* this clock. It also sets the floor on how quickly a remote deletion shows
|
||||
* up locally, which is the trade being made: a quarter of an hour.
|
||||
*/
|
||||
private const int SWEEP_INTERVAL_MINUTES = 15;
|
||||
|
||||
/**
|
||||
* How many vanished rows one reap will ask the server to confirm.
|
||||
*
|
||||
* Each is a round trip, and unlike the sweep this one deletes, so it drains
|
||||
* rather than sprints: an account that lost a thousand messages to a
|
||||
* mailbox clean-out catches up over several polls instead of turning one
|
||||
* into a probing run.
|
||||
*/
|
||||
private const int REAP_BATCH = 200;
|
||||
|
||||
/**
|
||||
* Below this many rows in a folder, a mass disappearance is believed.
|
||||
*
|
||||
* The number is a judgement and worth naming as one. Emptying a folder of
|
||||
* twenty by hand is routine; emptying one of four thousand between two
|
||||
* polls is not something a person did through a mail client, and the
|
||||
* asymmetry of being wrong — a duplicate versus destroyed mail — is what
|
||||
* puts the floor here rather than at zero.
|
||||
*/
|
||||
private const int BULK_FLOOR = 100;
|
||||
|
||||
/**
|
||||
* And above the floor, the share of a folder that vanishing at once is
|
||||
* treated as the server being wrong rather than the user being busy.
|
||||
*/
|
||||
private const float BULK_RATIO = 0.5;
|
||||
|
||||
public function __construct(
|
||||
private ImapFolderListing $listing,
|
||||
private MessageRepository $messages,
|
||||
private MailboxRepository $mailboxes,
|
||||
private MessageEraser $eraser,
|
||||
private EntityManagerInterface $em,
|
||||
private LoggerInterface $logger,
|
||||
) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare one folder against the server and write down what is missing.
|
||||
*
|
||||
* Deletes nothing, ever. See the class docblock.
|
||||
*
|
||||
* @return bool whether the folder was actually listed and compared
|
||||
*/
|
||||
public function sweep(Mailbox $mailbox, Client $client, ?DateTimeImmutable $now = null): bool
|
||||
{
|
||||
$now = $now ?? new DateTimeImmutable();
|
||||
|
||||
if (false === $this->isDueForSweep($mailbox, $now)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$listing = $this->listing->read($client, $mailbox);
|
||||
|
||||
if (null === $listing) {
|
||||
// No answer is not an answer. sweptAt stays where it was, so this
|
||||
// folder goes on withholding coverage until it can be read.
|
||||
return false;
|
||||
}
|
||||
|
||||
return $this->apply($mailbox, $listing, $now);
|
||||
}
|
||||
|
||||
/**
|
||||
* What a folder listing means, separated from the business of obtaining
|
||||
* one.
|
||||
*
|
||||
* Public because this is where every rule in the class docblock actually
|
||||
* lives, and a rule about when mail may be deleted has to be testable
|
||||
* against a stated server rather than a real one — the same split
|
||||
* MovedMessageReconciliationTest works through, where the test supplies the
|
||||
* answer the probe would have given.
|
||||
*
|
||||
* @param array{uidValidity: int, uids: array<int,true>} $listing
|
||||
*
|
||||
* @return bool whether the listing was believed and acted on
|
||||
*/
|
||||
public function apply(Mailbox $mailbox, array $listing, ?DateTimeImmutable $now = null): bool
|
||||
{
|
||||
$now = $now ?? new DateTimeImmutable();
|
||||
|
||||
if (true === $this->isRebuilt($mailbox, $listing['uidValidity'])) {
|
||||
$this->invalidate($mailbox, $listing['uidValidity']);
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
$located = $this->messages->findLocatedUidsById($mailbox);
|
||||
|
||||
$vanished = [];
|
||||
$returned = [];
|
||||
|
||||
foreach ($located as $messageId => $uid) {
|
||||
if (true === isset($listing['uids'][$uid])) {
|
||||
$returned[] = $messageId;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
$vanished[] = $messageId;
|
||||
}
|
||||
|
||||
if (true === $this->isSuspicious($vanished, $located, $mailbox, $listing['uids'])) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$marked = $this->messages->markVanished($vanished, $now);
|
||||
$cleared = $this->messages->clearVanished($returned);
|
||||
|
||||
$mailbox->uidValidity = $listing['uidValidity'];
|
||||
$mailbox->sweptAt = $now;
|
||||
|
||||
// The sweep owns this flush. MessageSyncer clears the entity manager
|
||||
// between batches of the incremental pass that follows, and a mailbox
|
||||
// carrying unflushed changes into that would simply lose them.
|
||||
$this->em->flush();
|
||||
|
||||
if ($marked > 0 || $cleared > 0) {
|
||||
$this->logger->info('Folder listing compared against local rows', [
|
||||
'mailbox' => $mailbox->fullPath,
|
||||
'onServer' => count($listing['uids']),
|
||||
'local' => count($located),
|
||||
'vanished' => $marked,
|
||||
'returned' => $cleared,
|
||||
]);
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Erase the rows that stayed missing everywhere, and only those.
|
||||
*
|
||||
* @param (callable(Mailbox, int): ?bool)|null $stillExists the tri-state
|
||||
* probe. Null — no way to ask — reaps nothing, because the last word
|
||||
* before a delete has to be the server's.
|
||||
*
|
||||
* @return int how many rows were erased
|
||||
*/
|
||||
public function reap(Account $account, ?callable $stillExists): int
|
||||
{
|
||||
if (null === $stillExists) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
$cutoff = $this->mailboxes->earliestSweepAcross($account);
|
||||
|
||||
if (null === $cutoff) {
|
||||
// At least one folder has never been listed, so the account has no
|
||||
// moment at which "nowhere has it" is a true statement yet.
|
||||
return 0;
|
||||
}
|
||||
|
||||
$candidates = $this->messages->findReapable($account, $cutoff, self::REAP_BATCH);
|
||||
|
||||
if (0 === count($candidates)) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
$erased = 0;
|
||||
$recovered = 0;
|
||||
|
||||
foreach ($candidates as $candidate) {
|
||||
$mailbox = $candidate->mailbox;
|
||||
$uid = $candidate->imapUid;
|
||||
|
||||
if (null === $mailbox || null === $uid) {
|
||||
// Unlocated: a UIDVALIDITY rebuild, or a row waiting for the
|
||||
// copy of itself the server has not filed yet. There is no
|
||||
// address to confirm, so there is nothing to conclude.
|
||||
continue;
|
||||
}
|
||||
|
||||
$present = $stillExists($mailbox, $uid);
|
||||
|
||||
if (true === $present) {
|
||||
// The sweep was wrong, or the mail came back. Either way the
|
||||
// server has it, and that outranks anything written down here.
|
||||
$candidate->vanishedAt = null;
|
||||
++$recovered;
|
||||
|
||||
continue;
|
||||
}
|
||||
|
||||
if (null === $present) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$this->eraser->erase($candidate);
|
||||
++$erased;
|
||||
}
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
if ($erased > 0 || $recovered > 0) {
|
||||
$this->logger->info('Reconciled messages the server no longer has', [
|
||||
'account' => $account->id,
|
||||
'erased' => $erased,
|
||||
'recovered' => $recovered,
|
||||
]);
|
||||
}
|
||||
|
||||
return $erased;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether this folder is far enough past its last full listing to be worth
|
||||
* listing again.
|
||||
*
|
||||
* Public because it is the cost control, and a cost control nobody can
|
||||
* assert on is one that quietly stops working. It is also the floor on how
|
||||
* quickly a remote deletion can show up locally, which makes it a stated
|
||||
* behaviour rather than an implementation detail.
|
||||
*/
|
||||
public function isDueForSweep(Mailbox $mailbox, ?DateTimeImmutable $now = null): bool
|
||||
{
|
||||
$now = $now ?? new DateTimeImmutable();
|
||||
$sweptAt = $mailbox->sweptAt;
|
||||
|
||||
if (null === $sweptAt) {
|
||||
return true;
|
||||
}
|
||||
|
||||
return $sweptAt < $now->modify('-' . self::SWEEP_INTERVAL_MINUTES . ' minutes');
|
||||
}
|
||||
|
||||
/**
|
||||
* A UIDVALIDITY we have seen before and that has changed. An unknown one is
|
||||
* not a change — there is nothing to have changed from — and adopting it
|
||||
* silently is safe because the comparison that follows is guarded: if the
|
||||
* stored UIDs really do belong to an older validity, every one of them will
|
||||
* read as missing at once and the bulk guard refuses the sweep.
|
||||
*/
|
||||
private function isRebuilt(Mailbox $mailbox, int $uidValidity): bool
|
||||
{
|
||||
$known = $mailbox->uidValidity;
|
||||
|
||||
return null !== $known && $known !== $uidValidity;
|
||||
}
|
||||
|
||||
private function invalidate(Mailbox $mailbox, int $uidValidity): void
|
||||
{
|
||||
$unlocated = $this->messages->unlocateAll($mailbox);
|
||||
|
||||
$mailbox->uidValidity = $uidValidity;
|
||||
$mailbox->lastSeenUid = 0;
|
||||
$mailbox->sweptAt = null;
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
$this->logger->warning('Folder was rebuilt on the server; every stored UID for it is void', [
|
||||
'mailbox' => $mailbox->fullPath,
|
||||
'uidValidity' => $uidValidity,
|
||||
'unlocated' => $unlocated,
|
||||
]);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<int> $vanished
|
||||
* @param array<int,int> $located
|
||||
* @param array<int,true> $onServer
|
||||
*/
|
||||
private function isSuspicious(array $vanished, array $located, Mailbox $mailbox, array $onServer): bool
|
||||
{
|
||||
$missing = count($vanished);
|
||||
$total = count($located);
|
||||
|
||||
if ($missing < self::BULK_FLOOR) {
|
||||
return false;
|
||||
}
|
||||
|
||||
if ($missing < (int) ceil($total * self::BULK_RATIO)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
$this->logger->warning('Refusing a folder listing that would mark most of the folder as gone', [
|
||||
'mailbox' => $mailbox->fullPath,
|
||||
'local' => $total,
|
||||
'onServer' => count($onServer),
|
||||
'missing' => $missing,
|
||||
'reason' => 0 === count($onServer)
|
||||
? 'the folder listed empty, which reads as a rebuild rather than a purge'
|
||||
: 'too much of the folder disappeared between two listings',
|
||||
]);
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Service\Mail;
|
||||
|
||||
use App\Domain\Helper\AttachmentStorageHelper;
|
||||
use App\Domain\Helper\RawMessageStorage;
|
||||
use App\Entity\Mail\Message;
|
||||
use App\Entity\Mail\MessageThread;
|
||||
use App\Service\Label\ThreadLabelSynchronizer;
|
||||
use Doctrine\ORM\EntityManagerInterface;
|
||||
|
||||
/**
|
||||
* The one way a message row leaves the database.
|
||||
*
|
||||
* There were two before this, written months apart and agreeing on nothing.
|
||||
* ComposeController::discard() deleted a draft's attachment files, recounted
|
||||
* its thread and announced a destroy; SentCopyReconciler::discard() recounted
|
||||
* the thread and announced a destroy but left the files on disk, which was
|
||||
* survivable only because the rows it removed were duplicates whose files
|
||||
* another row still pointed at. Neither touched rawPath, and neither
|
||||
* re-synchronised the thread's labels.
|
||||
*
|
||||
* Remote deletion is what made that gap matter. A message the server no longer
|
||||
* has is not a duplicate of anything: it is the only row that named its bytes,
|
||||
* and deleting it while leaving the .eml and the attachments behind grows the
|
||||
* storage directory forever with files nothing can ever reach again. It is also
|
||||
* the only row that carried its labels into the thread, so a thread that loses
|
||||
* its last Inbox message has to stop wearing the Inbox label or it keeps
|
||||
* showing in a list none of its messages are in.
|
||||
*
|
||||
* So all of it happens here, in the order the pieces depend on each other:
|
||||
*
|
||||
* 1. the files, while the row still names them;
|
||||
* 2. the destroy announcement, while the id still exists — StateManager only
|
||||
* persists change rows, so they ride out on the caller's flush;
|
||||
* 3. the row, out of its thread's collection as well as out of the database,
|
||||
* because everything below reads the collection;
|
||||
* 4. the thread's counters, recounted from what is left rather than
|
||||
* decremented — the stored numbers drift, and a thread header claiming
|
||||
* "3 messages" over two is the complaint that started all of this;
|
||||
* 5. the thread's labels, re-derived from the messages that remain.
|
||||
*
|
||||
* The caller flushes. Erasing a hundred vanished messages is one flush, not a
|
||||
* hundred, and the sweep that finds them is already inside a transaction it
|
||||
* would rather commit once.
|
||||
*
|
||||
* An emptied thread is left in place. It costs a row, it is what
|
||||
* ComposeController::discard() has always done, and the sync layer reuses it if
|
||||
* the conversation comes back.
|
||||
*/
|
||||
final readonly class MessageEraser
|
||||
{
|
||||
public function __construct(
|
||||
private EntityManagerInterface $em,
|
||||
private MailChangeRecorder $changes,
|
||||
private ThreadLabelSynchronizer $threadLabels,
|
||||
private AttachmentStorageHelper $attachments,
|
||||
private RawMessageStorage $rawMessages,
|
||||
) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Take one message out of the database, with everything that hangs off it.
|
||||
*/
|
||||
public function erase(Message $message): void
|
||||
{
|
||||
$thread = $message->thread;
|
||||
$accountId = (int) $message->account->id;
|
||||
$messageId = (string) $message->id;
|
||||
|
||||
foreach ($message->messageParts as $part) {
|
||||
$this->attachments->delete($part->storagePath);
|
||||
}
|
||||
|
||||
$this->rawMessages->delete($message->rawPath);
|
||||
|
||||
// Ahead of the removal: a destroy names an id, and the id is gone the
|
||||
// moment Doctrine flushes the delete.
|
||||
$this->changes->emailDestroyed($accountId, $messageId, $thread);
|
||||
|
||||
$this->em->remove($message);
|
||||
|
||||
if (null === $thread) {
|
||||
return;
|
||||
}
|
||||
|
||||
$thread->removeMessage($message);
|
||||
|
||||
$this->recount($thread);
|
||||
$this->threadLabels->sync($thread);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param iterable<Message> $messages
|
||||
*
|
||||
* @return int how many rows were erased
|
||||
*/
|
||||
public function eraseAll(iterable $messages): int
|
||||
{
|
||||
$erased = 0;
|
||||
|
||||
foreach ($messages as $message) {
|
||||
$this->erase($message);
|
||||
++$erased;
|
||||
}
|
||||
|
||||
return $erased;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every counter the thread publishes, taken from the messages it still
|
||||
* holds. Never decremented — see the class docblock.
|
||||
*/
|
||||
private function recount(MessageThread $thread): void
|
||||
{
|
||||
$unread = 0;
|
||||
$withAttachments = 0;
|
||||
|
||||
foreach ($thread->messages as $remaining) {
|
||||
if (null === $remaining->seenAt) {
|
||||
++$unread;
|
||||
}
|
||||
|
||||
if (true === $remaining->hasAttachments) {
|
||||
++$withAttachments;
|
||||
}
|
||||
}
|
||||
|
||||
$thread->messageCount = $thread->messages->count();
|
||||
$thread->unreadCount = $unread;
|
||||
$thread->attachmentCount = $withAttachments;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,426 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Tests\Infrastructure\Messaging\Handler;
|
||||
|
||||
use App\Domain\Enum\Mail\MailboxSpecialUse;
|
||||
use App\Domain\Helper\ImapConnectionFactory;
|
||||
use App\Entity\Mail\Account;
|
||||
use App\Entity\Mail\Mailbox;
|
||||
use App\Entity\Mail\Message;
|
||||
use App\Entity\User\User;
|
||||
use App\Infrastructure\Messaging\Handler\ApplyImapFlagsHandler;
|
||||
use App\Infrastructure\Messaging\Message\ApplyImapFlagsMessage;
|
||||
use App\Repository\Mail\MailboxRepository;
|
||||
use App\Repository\Mail\MessageRepository;
|
||||
use Doctrine\ORM\EntityManagerInterface;
|
||||
use PHPUnit\Framework\TestCase;
|
||||
use Psr\Log\NullLogger;
|
||||
use ReflectionProperty;
|
||||
use Webklex\PHPIMAP\Client;
|
||||
use Webklex\PHPIMAP\Folder;
|
||||
use Webklex\PHPIMAP\Message as ImapMessage;
|
||||
use Webklex\PHPIMAP\Query\WhereQuery;
|
||||
use Webklex\PHPIMAP\Support\FlagCollection;
|
||||
use Webklex\PHPIMAP\Support\MessageCollection;
|
||||
|
||||
/**
|
||||
* What a local action actually asks the IMAP server to do.
|
||||
*
|
||||
* The outgoing half of the lifecycle had tests for its failure modes — an
|
||||
* unreachable server, a bad password, a UID that had already moved — and none
|
||||
* at all for the thing it exists to do. Every one of the six actions was
|
||||
* asserted only by reading the `match` in applyToMessage() and believing it,
|
||||
* which is how `delete` came to be the one action whose meaning nobody had
|
||||
* pinned: it is an expunge, not a move to Trash, and those differ in whether
|
||||
* the mail still exists afterwards.
|
||||
*
|
||||
* So this states each action as an operation on a server, and asserts on what
|
||||
* that server was told. It also pins the two halves of the write-back, which is
|
||||
* the part of a move that keeps the row addressable: webklex learns the
|
||||
* destination UID by predicting the folder's UIDNEXT, and a prediction adopted
|
||||
* unchecked points the row at whatever else happened to land there in the
|
||||
* meantime.
|
||||
*/
|
||||
final class ApplyImapFlagsOutboundOperationsTest extends TestCase
|
||||
{
|
||||
// ── flags, in place ──────────────────────────────────────────────────
|
||||
|
||||
public function testMarkingReadSetsSeenOnTheServer(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'seen');
|
||||
|
||||
self::assertSame(['setFlag:Seen'], $present->calls);
|
||||
}
|
||||
|
||||
public function testMarkingUnreadClearsSeenOnTheServer(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'unseen');
|
||||
|
||||
self::assertSame(['unsetFlag:Seen'], $present->calls);
|
||||
}
|
||||
|
||||
public function testStarringSetsFlaggedOnTheServer(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'flag');
|
||||
|
||||
self::assertSame(['setFlag:Flagged'], $present->calls);
|
||||
}
|
||||
|
||||
public function testUnstarringClearsFlaggedOnTheServer(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'unflag');
|
||||
|
||||
self::assertSame(['unsetFlag:Flagged'], $present->calls);
|
||||
}
|
||||
|
||||
/**
|
||||
* A flag change is not a move, and must not become one. The distinction
|
||||
* matters because a move is the one operation here that can duplicate mail
|
||||
* on the server if it is issued twice.
|
||||
*/
|
||||
public function testAFlagChangeNeverMovesTheMessage(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'seen');
|
||||
|
||||
self::assertSame(0, $present->moveCalls);
|
||||
}
|
||||
|
||||
// ── delete, which means delete ───────────────────────────────────────
|
||||
|
||||
/**
|
||||
* `delete` expunges. It is not the Trash button — that is `trash`, and it
|
||||
* moves. This action is what plMail issues when the mail is to stop
|
||||
* existing on the server, and it has to carry the expunge or the message
|
||||
* merely wears \Deleted and comes back on the next listing.
|
||||
*/
|
||||
public function testDeleteExpungesRatherThanMovingToTrash(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'delete');
|
||||
|
||||
self::assertSame(['delete:expunge'], $present->calls);
|
||||
self::assertSame(0, $present->moveCalls, 'a delete is not a move');
|
||||
}
|
||||
|
||||
// ── moves, and where they go ─────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Trash resolves to the account's Trash folder by special-use, which is the
|
||||
* mapping the whole label model rests on.
|
||||
*/
|
||||
public function testTrashingMovesToTheAccountsTrashFolder(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'trash');
|
||||
|
||||
self::assertSame(1, $present->moveCalls);
|
||||
self::assertSame('INBOX.Trash', $present->movedTo);
|
||||
}
|
||||
|
||||
public function testArchivingMovesToTheAccountsArchiveFolder(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'archive', archiveExists: true);
|
||||
|
||||
self::assertSame(1, $present->moveCalls);
|
||||
self::assertSame('INBOX.Archive', $present->movedTo);
|
||||
}
|
||||
|
||||
/**
|
||||
* An explicit `move` — what the label propagator queues when a custom
|
||||
* location label is replaced — goes exactly where the envelope says, and
|
||||
* does not re-resolve anything.
|
||||
*/
|
||||
public function testAnExplicitMoveGoesToThePathTheEnvelopeNames(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'move', destinationPath: 'INBOX.Projekte');
|
||||
|
||||
self::assertSame('INBOX.Projekte', $present->movedTo);
|
||||
}
|
||||
|
||||
/**
|
||||
* With no destination resolvable, nothing is issued at all. Moving mail to
|
||||
* a folder that could not be identified is worse than not moving it.
|
||||
*/
|
||||
public function testAMoveWithNoResolvableDestinationIssuesNothing(): void
|
||||
{
|
||||
$present = new RecordingImapMessage(uid: 6);
|
||||
|
||||
$this->issue($present, 'archive', archiveExists: false);
|
||||
|
||||
self::assertSame(0, $present->moveCalls);
|
||||
self::assertSame([], $present->calls);
|
||||
}
|
||||
|
||||
// ── the write-back ───────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A move that lands somewhere the server confirms is the same message ends
|
||||
* with the row holding the destination's UID, so it is addressable
|
||||
* immediately rather than at the next sync of the destination.
|
||||
*/
|
||||
public function testAConfirmedMoveWritesTheDestinationUidBackOntoTheRow(): void
|
||||
{
|
||||
$landed = new RecordingImapMessage(uid: 91, rfcMessageId: 'abc@example.test');
|
||||
$present = new RecordingImapMessage(uid: 6, rfcMessageId: 'abc@example.test', landsAs: $landed);
|
||||
|
||||
$row = $this->issue($present, 'trash', rowMessageId: 'abc@example.test', rowLandsIn: 'INBOX.Trash');
|
||||
|
||||
self::assertSame(91, $row->imapUid, 'the row can be addressed in Trash straight away');
|
||||
}
|
||||
|
||||
/**
|
||||
* And the case that makes checking worth the trouble. webklex finds the
|
||||
* moved message by predicting the destination's UIDNEXT; if something else
|
||||
* landed there first, the prediction names somebody else's mail. An
|
||||
* unverified answer leaves the row unlocated, which costs one sync — a
|
||||
* wrong answer would point the row at another message permanently.
|
||||
*/
|
||||
public function testAMoveWhoseDestinationCopyIsSomebodyElsesMailIsNotAdopted(): void
|
||||
{
|
||||
$someoneElse = new RecordingImapMessage(uid: 91, rfcMessageId: 'other@example.test');
|
||||
$present = new RecordingImapMessage(uid: 6, rfcMessageId: 'abc@example.test', landsAs: $someoneElse);
|
||||
|
||||
$row = $this->issue($present, 'trash', rowMessageId: 'abc@example.test', rowLandsIn: 'INBOX.Trash');
|
||||
|
||||
self::assertNull($row->imapUid, 'unlocated is reconcilable; wrong is not');
|
||||
}
|
||||
|
||||
// ── fixture ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Run one envelope against a server holding one message, and hand back the
|
||||
* row so the write-back can be asserted on.
|
||||
*/
|
||||
private function issue(
|
||||
RecordingImapMessage $present,
|
||||
string $action,
|
||||
?string $destinationPath = null,
|
||||
bool $archiveExists = true,
|
||||
?string $rowMessageId = null,
|
||||
?string $rowLandsIn = null,
|
||||
): Message {
|
||||
$account = new Account();
|
||||
$account->usr = new User();
|
||||
|
||||
$source = $this->mailbox($account, 'INBOX', 'INBOX', MailboxSpecialUse::INBOX);
|
||||
$trash = $this->mailbox($account, 'Trash', 'INBOX.Trash', MailboxSpecialUse::TRASH);
|
||||
|
||||
$row = new Message();
|
||||
$row->account = $account;
|
||||
$row->messageId = $rowMessageId;
|
||||
|
||||
// A row mid-move carries no UID at all — relocateTo() cleared it the
|
||||
// moment the user acted, and the envelope is where the source address
|
||||
// survives. Reproducing that is the point: adopt() writing a UID onto
|
||||
// such a row is the whole behaviour under test, and a row that still
|
||||
// held its old UID would hide a wrong adoption behind it.
|
||||
if (null !== $rowLandsIn) {
|
||||
$row->relocateTo($trash);
|
||||
} else {
|
||||
$row->relocateTo($source, 6);
|
||||
}
|
||||
|
||||
new ReflectionProperty(Message::class, 'id')->setValue($row, 1);
|
||||
|
||||
$archive = $archiveExists
|
||||
? $this->mailbox($account, 'Archive', 'INBOX.Archive', MailboxSpecialUse::ARCHIVE)
|
||||
: null;
|
||||
|
||||
$messageRepository = $this->createStub(MessageRepository::class);
|
||||
$messageRepository->method('findBy')->willReturn([$row]);
|
||||
|
||||
$mailboxRepository = $this->createStub(MailboxRepository::class);
|
||||
$mailboxRepository->method('find')->willReturn($source);
|
||||
$mailboxRepository->method('findOneBy')->willReturnCallback(
|
||||
static fn (array $criteria): ?Mailbox => match ($criteria['specialUse'] ?? null) {
|
||||
'\\Trash' => $trash,
|
||||
'\\Archive' => $archive,
|
||||
default => null,
|
||||
},
|
||||
);
|
||||
|
||||
$client = $this->createStub(Client::class);
|
||||
// Every folder lookup answers with the same holding folder except the
|
||||
// ones resolveDestinationPath() probes by name when there is no
|
||||
// special-use row, which must not invent an Archive that is not there.
|
||||
$client->method('getFolder')->willReturnCallback(
|
||||
static fn (string $path): ?Folder => match (true) {
|
||||
false === $archiveExists && in_array($path, ['Archive', 'Archives'], true) => null,
|
||||
default => new RecordingFolder($path, new MessageCollection([$present])),
|
||||
},
|
||||
);
|
||||
$client->method('disconnect')->willReturnSelf();
|
||||
|
||||
$connectionFactory = $this->createStub(ImapConnectionFactory::class);
|
||||
$connectionFactory->method('connect')->willReturn($client);
|
||||
|
||||
$handler = new ApplyImapFlagsHandler(
|
||||
$messageRepository,
|
||||
$mailboxRepository,
|
||||
new NullLogger(),
|
||||
$connectionFactory,
|
||||
$this->createStub(EntityManagerInterface::class),
|
||||
);
|
||||
|
||||
$handler(new ApplyImapFlagsMessage([1 => 10], $action, $destinationPath, [1 => 6]));
|
||||
|
||||
return $row;
|
||||
}
|
||||
|
||||
private function mailbox(
|
||||
Account $account,
|
||||
string $name,
|
||||
string $fullPath,
|
||||
MailboxSpecialUse $specialUse,
|
||||
): Mailbox {
|
||||
$mailbox = new Mailbox();
|
||||
$mailbox->account = $account;
|
||||
$mailbox->name = $name;
|
||||
$mailbox->fullPath = $fullPath;
|
||||
$mailbox->specialUse = $specialUse;
|
||||
|
||||
return $mailbox;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* A folder that holds what the test says it holds. Named apart from the
|
||||
* FakeFolder in ApplyImapFlagsVanishedUidTest because PHP class names are
|
||||
* global and both files load in the same suite.
|
||||
*/
|
||||
final class RecordingFolder extends Folder
|
||||
{
|
||||
public function __construct(string $path, private readonly MessageCollection $holding)
|
||||
{
|
||||
// Deliberately not parent::__construct(): it wants a live connection.
|
||||
$this->path = $path;
|
||||
$this->name = $path;
|
||||
$this->full_name = $path;
|
||||
}
|
||||
|
||||
public function messages(array $extensions = []): WhereQuery
|
||||
{
|
||||
return new RecordingQuery($this->holding);
|
||||
}
|
||||
}
|
||||
|
||||
final class RecordingQuery extends WhereQuery
|
||||
{
|
||||
public function __construct(private readonly MessageCollection $holding)
|
||||
{
|
||||
}
|
||||
|
||||
public function whereUid(int|string $uid): static
|
||||
{
|
||||
return $this;
|
||||
}
|
||||
|
||||
public function where(mixed $criteria, mixed $value = null): static
|
||||
{
|
||||
return $this;
|
||||
}
|
||||
|
||||
public function setFetchBody(bool $fetch_body): static
|
||||
{
|
||||
return $this;
|
||||
}
|
||||
|
||||
public function get(): MessageCollection
|
||||
{
|
||||
return $this->holding;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* An IMAP message that writes down every operation performed on it.
|
||||
*
|
||||
* getUid()/getMessageId() are declared outright because webklex serves those
|
||||
* through __call, which a double cannot intercept.
|
||||
*/
|
||||
final class RecordingImapMessage extends ImapMessage
|
||||
{
|
||||
/** @var list<string> */
|
||||
public array $calls = [];
|
||||
|
||||
public int $moveCalls = 0;
|
||||
|
||||
public ?string $movedTo = null;
|
||||
|
||||
private int $fakeUid;
|
||||
|
||||
private ?string $fakeMessageId;
|
||||
|
||||
private ?RecordingImapMessage $landsAs;
|
||||
|
||||
public function __construct(
|
||||
int $uid,
|
||||
?string $rfcMessageId = null,
|
||||
?RecordingImapMessage $landsAs = null,
|
||||
) {
|
||||
$this->fakeUid = $uid;
|
||||
$this->fakeMessageId = $rfcMessageId;
|
||||
$this->landsAs = $landsAs;
|
||||
}
|
||||
|
||||
public function getUid(): int
|
||||
{
|
||||
return $this->fakeUid;
|
||||
}
|
||||
|
||||
public function getMessageId(): ?string
|
||||
{
|
||||
return $this->fakeMessageId;
|
||||
}
|
||||
|
||||
public function getFlags(): FlagCollection
|
||||
{
|
||||
return new FlagCollection([]);
|
||||
}
|
||||
|
||||
public function setFlag($flag): bool
|
||||
{
|
||||
$this->calls[] = 'setFlag:' . (is_array($flag) ? implode(',', $flag) : (string) $flag);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
public function unsetFlag($flag): bool
|
||||
{
|
||||
$this->calls[] = 'unsetFlag:' . (is_array($flag) ? implode(',', $flag) : (string) $flag);
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
public function delete(bool $expunge = true, ?string $trash_path = null, bool $force_move = false): bool
|
||||
{
|
||||
$this->calls[] = 'delete:' . (true === $expunge ? 'expunge' : 'flag-only');
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
public function move(string $folder_path, bool $expunge = false, bool $utf7 = false): ?ImapMessage
|
||||
{
|
||||
++$this->moveCalls;
|
||||
$this->movedTo = $folder_path;
|
||||
|
||||
return $this->landsAs;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,944 @@
|
||||
<?php
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
namespace App\Tests\Service\Imap;
|
||||
|
||||
use App\Domain\Enum\Mail\LabelRole;
|
||||
use App\Domain\Enum\Mail\MailboxSpecialUse;
|
||||
use App\Domain\Enum\Mail\ThreadingMethod;
|
||||
use App\Domain\Helper\MessageIdHelper;
|
||||
use App\Entity\Mail\Account;
|
||||
use App\Entity\Mail\Mailbox;
|
||||
use App\Entity\Mail\Message;
|
||||
use App\Entity\Mail\MessageThread;
|
||||
use App\Entity\User\User;
|
||||
use App\Repository\Mail\MessageRepository;
|
||||
use App\Service\Imap\SentCopyReconciler;
|
||||
use App\Service\Imap\VanishedMessageReconciler;
|
||||
use App\Service\Label\LabelResolver;
|
||||
use DateTimeImmutable;
|
||||
use Doctrine\DBAL\Connection;
|
||||
use Doctrine\ORM\EntityManagerInterface;
|
||||
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
|
||||
|
||||
/**
|
||||
* Mail that left the server, and how long plMail takes to believe it.
|
||||
*
|
||||
* v0.0.25 fixed the message that moves. It left the message that is *deleted*
|
||||
* untouched and said so in as many words: incremental sync asks for
|
||||
* `lastSeenUid+1:*`, so no UID below the high-water mark is ever looked at
|
||||
* again, and mail deleted in another client stayed in plMail indefinitely.
|
||||
* External moves healed only as a side effect of collapsing duplicate
|
||||
* Message-IDs, which needs two rows and an id to work with — a deletion leaves
|
||||
* neither.
|
||||
*
|
||||
* The rule these tests pin is that absence is evidence and never proof. Each
|
||||
* one states what the server said, exactly as MovedMessageReconciliationTest
|
||||
* states what the probe answered, and asserts what plMail was entitled to
|
||||
* conclude from it. The cases that assert *nothing happened* are the important
|
||||
* half: a listing that comes back empty, a folder that was rebuilt, a probe
|
||||
* that could not answer.
|
||||
*/
|
||||
final class RemoteDeletionSyncTest extends KernelTestCase
|
||||
{
|
||||
private EntityManagerInterface $em;
|
||||
private Connection $connection;
|
||||
private VanishedMessageReconciler $vanished;
|
||||
private SentCopyReconciler $reconciler;
|
||||
private MessageRepository $messages;
|
||||
private LabelResolver $labels;
|
||||
|
||||
private User $user;
|
||||
private Account $account;
|
||||
private Mailbox $inbox;
|
||||
private Mailbox $archive;
|
||||
private Mailbox $trash;
|
||||
|
||||
protected function setUp(): void
|
||||
{
|
||||
self::bootKernel();
|
||||
|
||||
$container = self::getContainer();
|
||||
|
||||
$this->em = $container->get(EntityManagerInterface::class);
|
||||
$this->connection = $container->get(Connection::class);
|
||||
$this->vanished = $container->get(VanishedMessageReconciler::class);
|
||||
$this->reconciler = $container->get(SentCopyReconciler::class);
|
||||
$this->messages = $container->get(MessageRepository::class);
|
||||
$this->labels = $container->get(LabelResolver::class);
|
||||
|
||||
$this->connection->beginTransaction();
|
||||
|
||||
$this->seed();
|
||||
}
|
||||
|
||||
protected function tearDown(): void
|
||||
{
|
||||
if (true === $this->connection->isTransactionActive()) {
|
||||
$this->connection->rollBack();
|
||||
}
|
||||
|
||||
parent::tearDown();
|
||||
}
|
||||
|
||||
// ── the gap v0.0.25 named ─────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The reported behaviour, from the other side: incremental sync cannot see
|
||||
* a UID it has already passed, so nothing about a poll notices the message
|
||||
* is gone. This is the state before the sweep exists — and it is still the
|
||||
* state after one poll, because a poll does not sweep.
|
||||
*/
|
||||
public function testAnIncrementalPollAloneNeverNoticesAMessageHasGone(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
// The high-water mark is above this message, which is the whole point:
|
||||
// every future incremental range starts past it.
|
||||
$this->inbox->lastSeenUid = 40;
|
||||
$this->em->flush();
|
||||
|
||||
self::assertNull($message->vanishedAt, 'nothing has looked, so nothing is known');
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
/**
|
||||
* The sweep looks, and writing down that it looked is all it does. The row
|
||||
* keeps its address — which is what lets the move machinery still work on
|
||||
* it — and the mail is still there.
|
||||
*/
|
||||
public function testASweepRecordsTheAbsenceAndDeletesNothing(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertNotNull($message->vanishedAt, 'the folder did not produce it, and that is worth recording');
|
||||
self::assertSame(6, $message->imapUid, 'but the address stays: it is what a probe can still be pointed at');
|
||||
self::assertNotNull($this->messages->find((int) $message->id), 'and one listing never deletes mail');
|
||||
}
|
||||
|
||||
/**
|
||||
* A message that comes back — because it was there all along and the
|
||||
* listing was momentarily wrong — has its mark taken off by the next
|
||||
* listing that does produce it.
|
||||
*/
|
||||
public function testAMessageTheNextListingProducesLosesItsMark(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
self::assertNotNull($message->vanishedAt);
|
||||
|
||||
$this->sweep($this->inbox, [6], now: '2026-08-11 10:00:00');
|
||||
|
||||
self::assertNull($message->vanishedAt, 'the server has it, which outranks anything written down');
|
||||
}
|
||||
|
||||
// ── remote delete, the whole way through ──────────────────────────────
|
||||
|
||||
/**
|
||||
* The user's sentence: "remote deleted email should be deleted as well."
|
||||
*
|
||||
* Every folder has been listed since the message went missing, none of them
|
||||
* produced it, and the final probe of its own address confirms it. Only
|
||||
* then does the row go.
|
||||
*/
|
||||
public function testAMessageDeletedOnTheServerIsDeletedLocally(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
$rowId = (int) $message->id;
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: []), 'the server confirms it is not there');
|
||||
|
||||
self::assertNull($this->messages->find($rowId), 'the row goes');
|
||||
}
|
||||
|
||||
/**
|
||||
* Coverage is a precondition, not a formality. While any folder in the
|
||||
* account has never been listed in full, "it is nowhere" is not a statement
|
||||
* plMail is in a position to make, so nothing is reaped however old the
|
||||
* mark is.
|
||||
*/
|
||||
public function testNothingIsReapedWhileAnyFolderHasNeverBeenListed(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
// Only INBOX is swept. Archive and Trash have never been looked in.
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(0, $this->reap(server: []), 'two folders are unaccounted for');
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
/**
|
||||
* The mark has to predate the coverage, not merely coexist with it. A
|
||||
* message that went missing *after* a folder was last listed has not been
|
||||
* looked for in that folder yet.
|
||||
*/
|
||||
public function testAMarkYoungerThanTheCoverageIsNotYetReapable(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweepAll(vanishing: [], now: '2026-08-11 09:00:00');
|
||||
|
||||
// It goes missing an hour after the last folder was listed.
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 10:00:00');
|
||||
|
||||
self::assertSame(0, $this->reap(server: []), 'no folder has been listed since it vanished');
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
/**
|
||||
* The last word before a delete is the server's, and "I could not tell" is
|
||||
* not that word. This is the v0.0.25 discipline applied to the one
|
||||
* operation that cannot be undone by the next poll.
|
||||
*/
|
||||
public function testAnUnanswerableProbeNeverDeletesMail(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
$erased = $this->vanished->reap(
|
||||
$this->account,
|
||||
static fn (Mailbox $mailbox, int $uid): ?bool => null,
|
||||
);
|
||||
|
||||
self::assertSame(0, $erased);
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
public function testWithNoWayToAskTheServerNothingIsDeleted(): void
|
||||
{
|
||||
$this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(0, $this->vanished->reap($this->account, null));
|
||||
self::assertSame(1, $this->countRowsForAccount());
|
||||
}
|
||||
|
||||
/**
|
||||
* The sweep can be wrong, and the confirmation is where that gets caught.
|
||||
* A row the probe finds after all keeps its place and loses its mark.
|
||||
*/
|
||||
public function testAProbeThatFindsTheMessageUndoesTheMark(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
self::assertNotNull($message->vanishedAt);
|
||||
|
||||
self::assertSame(0, $this->reap(server: [[$this->inbox, 6]]));
|
||||
|
||||
self::assertNull($message->vanishedAt, 'the server has it, so the sweep was wrong');
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
/**
|
||||
* A remote deletion is a JMAP destroy, and has to reach the change log as
|
||||
* one.
|
||||
*
|
||||
* This is the difference between deleting a row and deleting a message. A
|
||||
* client holding this id — ltt.rs, a phone — must be told to drop it, not
|
||||
* left to discover a 404 or, worse, to go on showing mail the server threw
|
||||
* away. It is a genuine `destroyed` rather than the `updated` that
|
||||
* Email/set destroy produces, because that one moves to Trash and keeps the
|
||||
* row while this one takes the id away.
|
||||
*/
|
||||
public function testARemoteDeletionIsAnnouncedToJmapClientsAsADestroy(): void
|
||||
{
|
||||
$message = $this->incoming('Rechnung', $this->inbox, 6);
|
||||
$rowId = (int) $message->id;
|
||||
$thread = $message->thread;
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
$since = (int) $this->connection->fetchOne('SELECT COALESCE(MAX(sequence), 0) FROM jmap_change_log');
|
||||
|
||||
self::assertSame(1, $this->reap(server: []));
|
||||
|
||||
$rows = $this->connection->fetchAllAssociative(
|
||||
'SELECT object_type, change_type, entity_id FROM jmap_change_log WHERE sequence > ? ORDER BY sequence',
|
||||
[$since],
|
||||
);
|
||||
|
||||
$announced = array_map(
|
||||
static fn (array $row): string => $row['object_type'] . ':' . $row['change_type'] . ':' . $row['entity_id'],
|
||||
$rows,
|
||||
);
|
||||
|
||||
self::assertContains('Email:destroyed:' . $rowId, $announced);
|
||||
self::assertContains('Thread:updated:' . $thread->id, $announced, 'the conversation it left changed too');
|
||||
}
|
||||
|
||||
/**
|
||||
* And the thread it leaves stops wearing labels no message in it carries
|
||||
* any more — the same rule a move obeys, arrived at from the other side. A
|
||||
* conversation whose only INBOX message was deleted is not in INBOX.
|
||||
*/
|
||||
public function testAThreadStopsWearingTheLabelOfAMessageThatWasDeletedRemotely(): void
|
||||
{
|
||||
$thread = $this->thread('Gemischt');
|
||||
|
||||
$this->rowIn($this->inbox, MessageIdHelper::mint('a@example.test'), 6, 'Gemischt', $thread);
|
||||
$this->rowIn($this->archive, MessageIdHelper::mint('b@example.test'), 210, 'Gemischt', $thread);
|
||||
$this->em->flush();
|
||||
|
||||
$inboxLabel = $this->labels->systemLabel(LabelRole::Inbox, $this->account);
|
||||
$thread->addLabel($inboxLabel);
|
||||
$thread->addLabel($this->labels->systemLabel(LabelRole::Archive, $this->account));
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: [[$this->archive, 210]]));
|
||||
|
||||
self::assertFalse($thread->hasLabel($inboxLabel), 'no message of it is in INBOX any more');
|
||||
self::assertTrue($thread->hasLabel($this->labels->systemLabel(LabelRole::Archive, $this->account)));
|
||||
}
|
||||
|
||||
// ── delete versus move, which look identical from the folder left ─────
|
||||
|
||||
/**
|
||||
* The distinction the whole design turns on. A message missing from INBOX
|
||||
* and present in Archive is a move, and the existing v0.0.25 path is what
|
||||
* settles it: the row still has the address the probe needs, Archive's sync
|
||||
* meets the duplicate Message-ID, the probe says INBOX no longer has it,
|
||||
* and the row is relocated rather than doubled — or reaped.
|
||||
*/
|
||||
public function testAMessageMovedOnTheServerIsRelocatedRatherThanDeleted(): void
|
||||
{
|
||||
$message = $this->incoming('Angebot', $this->inbox, 6);
|
||||
$messageId = (string) $message->messageId;
|
||||
$rowId = (int) $message->id;
|
||||
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
self::assertNotNull($message->vanishedAt, 'INBOX no longer lists it');
|
||||
|
||||
// Archive's incremental sync meets it under a UID it has never seen.
|
||||
$claimed = $this->reconciler->claim(
|
||||
$this->archive,
|
||||
$messageId,
|
||||
210,
|
||||
$this->serverHolding([[$this->archive, 210]]),
|
||||
);
|
||||
$this->em->flush();
|
||||
|
||||
self::assertNotNull($claimed, 'the copy INBOX had is gone, so this is that message at a new address');
|
||||
self::assertSame($rowId, (int) $claimed->id);
|
||||
self::assertSame($this->archive->id, $claimed->mailbox?->id);
|
||||
self::assertNull($claimed->vanishedAt, 'and it is not missing any more — it was found');
|
||||
|
||||
self::assertSame(1, $this->countRowsFor($messageId), 'one message, one row');
|
||||
}
|
||||
|
||||
/**
|
||||
* And having been relocated, it must not then be reaped for having gone
|
||||
* missing from the folder it left. This is the ordering hazard the mark
|
||||
* exists to survive.
|
||||
*/
|
||||
public function testAMessageThatMovedIsNotReapedAfterwards(): void
|
||||
{
|
||||
$message = $this->incoming('Angebot', $this->inbox, 6);
|
||||
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
$this->reconciler->claim(
|
||||
$this->archive,
|
||||
(string) $message->messageId,
|
||||
210,
|
||||
$this->serverHolding([[$this->archive, 210]]),
|
||||
);
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweepAll(vanishing: [], now: '2026-08-11 09:30:00');
|
||||
|
||||
self::assertSame(0, $this->reap(server: [[$this->archive, 210]]));
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
/**
|
||||
* A remote move of a message plMail has no Message-ID for cannot be
|
||||
* recognised as a move — there is no identity to re-match on — so it ends
|
||||
* as a delete in the folder it left and an insert in the one it arrived in.
|
||||
*
|
||||
* Stated rather than fixed. Message-ID is the only identity that survives a
|
||||
* move, and inferring one from content is exactly the guess v0.0.23 refused
|
||||
* to make.
|
||||
*/
|
||||
public function testAMoveOfAMessageWithNoMessageIdReadsAsADelete(): void
|
||||
{
|
||||
$message = $this->incoming('Ohne Id', $this->inbox, 6);
|
||||
$message->messageId = null;
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: []), 'nothing identifies it elsewhere, so it reads as gone');
|
||||
}
|
||||
|
||||
// ── trash, both of its endings ───────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A message deleted remotely out of INBOX while plMail shows it in INBOX
|
||||
* lands in Trash if that is where the other client put it — which is a
|
||||
* move, and arrives as one.
|
||||
*/
|
||||
public function testARemoteDeleteThatIsReallyAMoveToTrashEndsInTrash(): void
|
||||
{
|
||||
$message = $this->incoming('Werbung', $this->inbox, 6);
|
||||
|
||||
$this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
$this->reconciler->claim(
|
||||
$this->trash,
|
||||
(string) $message->messageId,
|
||||
91,
|
||||
$this->serverHolding([[$this->trash, 91]]),
|
||||
);
|
||||
$this->em->flush();
|
||||
|
||||
self::assertSame($this->trash->id, $message->mailbox?->id);
|
||||
self::assertTrue(
|
||||
$message->hasLabel($this->labels->systemLabel(LabelRole::Trash, $this->account)),
|
||||
'and plMail shows it where the server put it',
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* And the other ending: expunged out of Trash, which is the point at which
|
||||
* the message stops existing anywhere and the row has to go with it.
|
||||
*/
|
||||
public function testAMessageExpungedFromTrashIsRemovedEntirely(): void
|
||||
{
|
||||
$message = $this->incoming('Werbung', $this->trash, 91);
|
||||
$rowId = (int) $message->id;
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->trash, 91]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: []));
|
||||
self::assertNull($this->messages->find($rowId), 'emptying Trash on the server empties it here');
|
||||
}
|
||||
|
||||
/**
|
||||
* \Deleted-but-not-expunged is not a deletion. The message is still in the
|
||||
* folder and still in `UID SEARCH ALL`, so the sweep sees it and nothing
|
||||
* happens — which is correct, and is why the sweep asks the folder what it
|
||||
* holds rather than asking about flags.
|
||||
*/
|
||||
public function testAMessageFlaggedDeletedButNotExpungedIsLeftAlone(): void
|
||||
{
|
||||
$message = $this->incoming('Halb geloescht', $this->inbox, 6);
|
||||
$message->flags = ['Deleted'];
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweepAll(vanishing: [], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertNull($message->vanishedAt, 'the folder still lists it, so it is still there');
|
||||
self::assertSame(0, $this->reap(server: [[$this->inbox, 6]]));
|
||||
self::assertNotNull($this->messages->find((int) $message->id));
|
||||
}
|
||||
|
||||
// ── the copy in two folders ──────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Plain IMAP lets one message exist in two folders, and deleting one copy
|
||||
* must not take the other. The rows are separate addresses and the sweep
|
||||
* treats them separately.
|
||||
*/
|
||||
public function testDeletingOneCopyOfAMultiFolderMessageLeavesTheOther(): void
|
||||
{
|
||||
$messageId = MessageIdHelper::mint('kunde@example.test');
|
||||
$thread = $this->thread('Rundschreiben');
|
||||
|
||||
$inboxCopy = $this->rowIn($this->inbox, $messageId, 6, 'Rundschreiben', $thread);
|
||||
$archiveCopy = $this->rowIn($this->archive, $messageId, 210, 'Rundschreiben', $thread);
|
||||
$this->em->flush();
|
||||
|
||||
$keptId = (int) $archiveCopy->id;
|
||||
$goneId = (int) $inboxCopy->id;
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: [[$this->archive, 210]]));
|
||||
|
||||
self::assertNull($this->messages->find($goneId), 'the copy that was deleted goes');
|
||||
self::assertNotNull($this->messages->find($keptId), 'and the copy that was not stays');
|
||||
}
|
||||
|
||||
// ── a whole conversation ─────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Deleting a thread on the server deletes its messages here, and the thread
|
||||
* they were in is recounted rather than decremented — the drift that made
|
||||
* a header claim "3 messages" over two.
|
||||
*/
|
||||
public function testDeletingAWholeThreadRemotelyEmptiesItAndRecountsIt(): void
|
||||
{
|
||||
$thread = $this->thread('Langer Verlauf');
|
||||
|
||||
$first = $this->rowIn($this->inbox, MessageIdHelper::mint('a@example.test'), 6, 'Langer Verlauf', $thread);
|
||||
$second = $this->rowIn($this->inbox, MessageIdHelper::mint('b@example.test'), 7, 'Langer Verlauf', $thread);
|
||||
$third = $this->rowIn($this->inbox, MessageIdHelper::mint('c@example.test'), 8, 'Langer Verlauf', $thread);
|
||||
$this->em->flush();
|
||||
|
||||
self::assertSame(3, $thread->messageCount);
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6], [$this->inbox, 7], [$this->inbox, 8]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(3, $this->reap(server: []));
|
||||
|
||||
self::assertSame(0, $thread->messages->count());
|
||||
self::assertSame(0, $thread->messageCount, 'recounted from what is left, not decremented');
|
||||
self::assertSame(0, $thread->unreadCount);
|
||||
}
|
||||
|
||||
/**
|
||||
* Deleting part of a thread recounts the rest, including the counters a
|
||||
* list row is rendered from.
|
||||
*/
|
||||
public function testDeletingPartOfAThreadRecountsWhatRemains(): void
|
||||
{
|
||||
$thread = $this->thread('Teilweise');
|
||||
|
||||
$this->rowIn($this->inbox, MessageIdHelper::mint('a@example.test'), 6, 'Teilweise', $thread);
|
||||
$unread = $this->rowIn($this->inbox, MessageIdHelper::mint('b@example.test'), 7, 'Teilweise', $thread);
|
||||
$unread->seenAt = null;
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweepAll(vanishing: [[$this->inbox, 6]], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertSame(1, $this->reap(server: [[$this->inbox, 7]]));
|
||||
|
||||
self::assertSame(1, $thread->messageCount);
|
||||
self::assertSame(1, $thread->unreadCount, 'the unread one is the one that stayed');
|
||||
}
|
||||
|
||||
// ── the guards ───────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The accident this class exists to not have. A folder that held thousands
|
||||
* and lists empty has been rebuilt, restored, or is being served by
|
||||
* something that lost its index — and reading that as thousands of
|
||||
* deletions is how a mail client destroys a mailbox in one poll.
|
||||
*/
|
||||
public function testAFolderThatSuddenlyListsEmptyIsTreatedAsSuspicionNotAsAPurge(): void
|
||||
{
|
||||
for ($uid = 1; $uid <= 400; ++$uid) {
|
||||
$this->rowIn($this->inbox, MessageIdHelper::mint('bulk@example.test'), $uid, 'Bulk', $this->thread('Bulk'));
|
||||
}
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
$believed = $this->sweep($this->inbox, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertFalse($believed, 'the listing is refused');
|
||||
self::assertNull($this->inbox->sweptAt, 'and the folder counts as unswept, which suspends reaping account-wide');
|
||||
self::assertSame(400, $this->countRowsForAccount(), 'nothing is even marked');
|
||||
}
|
||||
|
||||
/**
|
||||
* Not only the empty case. Nine tenths of a folder disappearing between two
|
||||
* listings is the same accident wearing a less obvious shape.
|
||||
*/
|
||||
public function testMostOfAFolderDisappearingAtOnceIsAlsoRefused(): void
|
||||
{
|
||||
for ($uid = 1; $uid <= 400; ++$uid) {
|
||||
$this->rowIn($this->inbox, MessageIdHelper::mint('bulk@example.test'), $uid, 'Bulk', $this->thread('Bulk'));
|
||||
}
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
$believed = $this->sweep($this->inbox, range(1, 40), now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertFalse($believed);
|
||||
self::assertSame(400, $this->countRowsForAccount());
|
||||
}
|
||||
|
||||
/**
|
||||
* And the counter-case, which is what keeps the guard from being a refusal
|
||||
* to ever notice anything: a small folder genuinely emptied is an ordinary
|
||||
* thing a person does, and it is believed.
|
||||
*/
|
||||
public function testASmallFolderGenuinelyEmptiedIsBelieved(): void
|
||||
{
|
||||
$this->incoming('Eins', $this->archive, 210);
|
||||
$this->incoming('Zwei', $this->archive, 211);
|
||||
|
||||
$believed = $this->sweep($this->archive, [], now: '2026-08-11 09:00:00');
|
||||
|
||||
self::assertTrue($believed);
|
||||
self::assertNotNull($this->archive->sweptAt);
|
||||
}
|
||||
|
||||
// ── UIDVALIDITY ──────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A rebuilt folder invalidates every UID stored for it and no mail at all.
|
||||
* The rows are unlocated — which is the state claim() re-matches by
|
||||
* Message-ID — and the high-water mark is reset so the re-read starts from
|
||||
* the beginning.
|
||||
*/
|
||||
public function testAUidValidityChangeVoidsTheAddressesAndKeepsTheMail(): void
|
||||
{
|
||||
$first = $this->incoming('Eins', $this->inbox, 6);
|
||||
$second = $this->incoming('Zwei', $this->inbox, 7);
|
||||
|
||||
$this->inbox->uidValidity = 111;
|
||||
$this->inbox->lastSeenUid = 7;
|
||||
$this->em->flush();
|
||||
|
||||
$believed = $this->sweep($this->inbox, [1, 2], now: '2026-08-11 09:00:00', uidValidity: 222);
|
||||
|
||||
self::assertFalse($believed, 'a rebuild is not a comparison');
|
||||
|
||||
$this->em->refresh($first);
|
||||
$this->em->refresh($second);
|
||||
|
||||
self::assertNull($first->imapUid, 'the address it had means nothing now');
|
||||
self::assertNull($second->imapUid);
|
||||
self::assertNull($first->vanishedAt, 'and an unlocated row is not a missing one');
|
||||
self::assertSame(2, $this->countRowsForAccount(), 'no mail was deleted');
|
||||
self::assertSame(0, $this->inbox->lastSeenUid, 'the folder is re-read from the start');
|
||||
self::assertSame(222, $this->inbox->uidValidity);
|
||||
}
|
||||
|
||||
/**
|
||||
* And the rows the re-read produces are re-matched onto themselves by
|
||||
* Message-ID rather than inserted beside themselves, which is the whole
|
||||
* reason unlocating is safe.
|
||||
*/
|
||||
public function testAfterARebuildTheRowsAreReMatchedByMessageId(): void
|
||||
{
|
||||
$message = $this->incoming('Eins', $this->inbox, 6);
|
||||
$messageId = (string) $message->messageId;
|
||||
$rowId = (int) $message->id;
|
||||
|
||||
$this->inbox->uidValidity = 111;
|
||||
$this->em->flush();
|
||||
|
||||
$this->sweep($this->inbox, [1], now: '2026-08-11 09:00:00', uidValidity: 222);
|
||||
|
||||
// The re-read meets it under the UID the rebuilt folder issued.
|
||||
$claimed = $this->reconciler->claim($this->inbox, $messageId, 1);
|
||||
$this->em->flush();
|
||||
|
||||
self::assertNotNull($claimed, 'the row has no address, which is exactly what claim() resolves');
|
||||
self::assertSame($rowId, (int) $claimed->id);
|
||||
self::assertSame(1, $claimed->imapUid);
|
||||
self::assertSame(1, $this->countRowsFor($messageId), 'one message, one row');
|
||||
}
|
||||
|
||||
/**
|
||||
* A folder whose UIDVALIDITY plMail has never recorded is not a folder that
|
||||
* changed — there is nothing to have changed from. It is adopted, and the
|
||||
* comparison is guarded anyway: stored UIDs belonging to an older validity
|
||||
* would all read as missing at once, which is what the bulk guard refuses.
|
||||
*/
|
||||
public function testAnUnknownUidValidityIsAdoptedRatherThanTreatedAsAChange(): void
|
||||
{
|
||||
$this->incoming('Eins', $this->inbox, 6);
|
||||
|
||||
self::assertNull($this->inbox->uidValidity);
|
||||
|
||||
$believed = $this->sweep($this->inbox, [6], now: '2026-08-11 09:00:00', uidValidity: 333);
|
||||
|
||||
self::assertTrue($believed);
|
||||
self::assertSame(333, $this->inbox->uidValidity);
|
||||
}
|
||||
|
||||
// ── cadence ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* The listing is two round trips and a response measured in hundreds of
|
||||
* kilobytes, so it is not in the poll loop. A folder swept a minute ago is
|
||||
* left alone; one swept an hour ago is not.
|
||||
*/
|
||||
public function testAFolderSweptRecentlyIsNotSweptAgain(): void
|
||||
{
|
||||
$now = new DateTimeImmutable('2026-08-11 09:00:00');
|
||||
|
||||
$this->inbox->sweptAt = new DateTimeImmutable('2026-08-11 08:59:00');
|
||||
|
||||
self::assertFalse($this->vanished->isDueForSweep($this->inbox, $now));
|
||||
|
||||
$this->inbox->sweptAt = new DateTimeImmutable('2026-08-11 08:00:00');
|
||||
|
||||
self::assertTrue($this->vanished->isDueForSweep($this->inbox, $now));
|
||||
}
|
||||
|
||||
/**
|
||||
* A folder nobody has ever listed is always due, which is what gets the
|
||||
* account to its first complete coverage at all.
|
||||
*/
|
||||
public function testAFolderNeverSweptIsAlwaysDue(): void
|
||||
{
|
||||
self::assertNull($this->inbox->sweptAt);
|
||||
self::assertTrue($this->vanished->isDueForSweep($this->inbox, new DateTimeImmutable('2026-08-11 09:00:00')));
|
||||
}
|
||||
|
||||
// ── fixture ───────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* One folder listing, stated the way the server would have answered it.
|
||||
*
|
||||
* @param list<int> $uids every UID the folder holds
|
||||
*/
|
||||
private function sweep(
|
||||
Mailbox $mailbox,
|
||||
array $uids,
|
||||
string $now,
|
||||
int $uidValidity = 1,
|
||||
): bool {
|
||||
$set = [];
|
||||
|
||||
foreach ($uids as $uid) {
|
||||
$set[$uid] = true;
|
||||
}
|
||||
|
||||
$believed = $this->vanished->apply(
|
||||
$mailbox,
|
||||
['uidValidity' => $uidValidity, 'uids' => $set],
|
||||
new DateTimeImmutable($now),
|
||||
);
|
||||
|
||||
// markVanished() and unlocateAll() are bulk updates, which by design go
|
||||
// round the identity map rather than through it — the whole reason they
|
||||
// exist is not to hydrate a folder's worth of entities to write one
|
||||
// column. So anything the test is still holding is now stale, and has
|
||||
// to be re-read before it is asserted on.
|
||||
$this->rereadMessages();
|
||||
|
||||
return $believed;
|
||||
}
|
||||
|
||||
/**
|
||||
* The account listed in full, twice, with the same answer both times.
|
||||
*
|
||||
* Two passes rather than one because that is genuinely what reaping costs.
|
||||
* A mark records the instant a row went missing, and the coverage that
|
||||
* justifies acting on it has to *postdate* that instant — a folder listed
|
||||
* at the same moment the row vanished has not been looked in since. So the
|
||||
* first pass marks and the second pass provides the coverage, which in
|
||||
* production is two poll cycles a sweep interval apart and is the floor on
|
||||
* how quickly a remote deletion can propagate.
|
||||
*
|
||||
* testAMarkYoungerThanTheCoverageIsNotYetReapable pins the other side of
|
||||
* that rule, using single passes.
|
||||
*
|
||||
* @param list<array{Mailbox, int}> $vanishing the addresses the server
|
||||
* should NOT list. Everything else the account holds is listed as
|
||||
* present, which is what makes this a whole-account listing rather
|
||||
* than a folder's.
|
||||
*/
|
||||
private function sweepAll(array $vanishing, string $now): void
|
||||
{
|
||||
$this->listAll($vanishing, $now);
|
||||
$this->listAll($vanishing, (new DateTimeImmutable($now))->modify('+1 hour')->format('Y-m-d H:i:s'));
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<array{Mailbox, int}> $vanishing
|
||||
*/
|
||||
private function listAll(array $vanishing, string $now): void
|
||||
{
|
||||
$missing = [];
|
||||
|
||||
foreach ($vanishing as [$mailbox, $uid]) {
|
||||
$missing[$mailbox->id . ':' . $uid] = true;
|
||||
}
|
||||
|
||||
foreach ([$this->inbox, $this->archive, $this->trash] as $mailbox) {
|
||||
$present = [];
|
||||
|
||||
foreach ($this->messages->findLocatedUidsById($mailbox) as $uid) {
|
||||
if (true === isset($missing[$mailbox->id . ':' . $uid])) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$present[] = $uid;
|
||||
}
|
||||
|
||||
$this->sweep($mailbox, $present, $now);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-read every Message the test still holds. See sweep().
|
||||
*/
|
||||
private function rereadMessages(): void
|
||||
{
|
||||
$map = $this->em->getUnitOfWork()->getIdentityMap();
|
||||
|
||||
foreach ($map[Message::class] ?? [] as $message) {
|
||||
$this->em->refresh($message);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<array{Mailbox, int}> $server the addresses that still exist
|
||||
*/
|
||||
private function reap(array $server): int
|
||||
{
|
||||
$erased = $this->vanished->reap($this->account, $this->serverHolding($server));
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
return $erased;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param list<array{Mailbox, int}> $present
|
||||
*
|
||||
* @return callable(Mailbox, int): bool
|
||||
*/
|
||||
private function serverHolding(array $present): callable
|
||||
{
|
||||
$addresses = [];
|
||||
|
||||
foreach ($present as [$mailbox, $uid]) {
|
||||
$addresses[$mailbox->id . ':' . $uid] = true;
|
||||
}
|
||||
|
||||
return static fn (Mailbox $mailbox, int $uid): bool
|
||||
=> true === isset($addresses[$mailbox->id . ':' . $uid]);
|
||||
}
|
||||
|
||||
private function incoming(string $subject, Mailbox $mailbox, int $uid): Message
|
||||
{
|
||||
$thread = $this->thread($subject);
|
||||
$message = $this->rowIn($mailbox, MessageIdHelper::mint('kunde@example.test'), $uid, $subject, $thread);
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
return $message;
|
||||
}
|
||||
|
||||
private function thread(string $subject): MessageThread
|
||||
{
|
||||
$thread = new MessageThread();
|
||||
$thread->account = $this->account;
|
||||
$thread->subject = $subject;
|
||||
$thread->normalizedSubject = mb_strtolower($subject);
|
||||
$thread->threadingMethod = ThreadingMethod::References;
|
||||
$thread->messageCount = 0;
|
||||
$thread->unreadCount = 0;
|
||||
$thread->attachmentCount = 0;
|
||||
|
||||
$this->em->persist($thread);
|
||||
|
||||
return $thread;
|
||||
}
|
||||
|
||||
private function rowIn(
|
||||
Mailbox $mailbox,
|
||||
string $messageId,
|
||||
int $uid,
|
||||
string $subject,
|
||||
MessageThread $thread,
|
||||
): Message {
|
||||
$message = new Message();
|
||||
$message->account = $this->account;
|
||||
$message->mailbox = $mailbox;
|
||||
$message->imapUid = $uid;
|
||||
$message->messageId = $messageId;
|
||||
$message->subject = $subject;
|
||||
$message->fromAddress = 'kunde@example.test';
|
||||
$message->bodyText = $subject;
|
||||
$message->hasAttachments = false;
|
||||
$message->seenAt = new DateTimeImmutable();
|
||||
$message->flags = [];
|
||||
$message->receivedAt = new DateTimeImmutable('2026-08-10 09:00:00');
|
||||
$message->sentAt = $message->receivedAt;
|
||||
|
||||
$mailboxLabel = $mailbox->label;
|
||||
|
||||
if (null !== $mailboxLabel) {
|
||||
$message->addLabel($mailboxLabel);
|
||||
}
|
||||
|
||||
$this->em->persist($message);
|
||||
|
||||
$thread->addMessage($message);
|
||||
$thread->messageCount = $thread->messageCount + 1;
|
||||
|
||||
return $message;
|
||||
}
|
||||
|
||||
private function countRowsFor(string $messageId): int
|
||||
{
|
||||
return (int) $this->em->createQuery(
|
||||
'SELECT COUNT(m.id) FROM ' . Message::class . ' m WHERE m.account = :account AND m.messageId = :messageId',
|
||||
)
|
||||
->setParameter('account', $this->account)
|
||||
->setParameter('messageId', $messageId)
|
||||
->getSingleScalarResult();
|
||||
}
|
||||
|
||||
private function countRowsForAccount(): int
|
||||
{
|
||||
return (int) $this->em->createQuery(
|
||||
'SELECT COUNT(m.id) FROM ' . Message::class . ' m WHERE m.account = :account',
|
||||
)->setParameter('account', $this->account)->getSingleScalarResult();
|
||||
}
|
||||
|
||||
private function seed(): void
|
||||
{
|
||||
$this->user = new User();
|
||||
$this->user->email = 'vanished-' . uniqid('', true) . '@example.test';
|
||||
$this->user->nameFirst = 'Vanished';
|
||||
$this->user->nameLast = 'Message';
|
||||
$this->user->roles = ['ROLE_USER'];
|
||||
$this->user->password = 'x';
|
||||
$this->em->persist($this->user);
|
||||
|
||||
$this->account = new Account();
|
||||
$this->account->usr = $this->user;
|
||||
$this->account->email = 'vanished-fixture@example.test';
|
||||
$this->account->username = 'vanished-fixture@example.test';
|
||||
$this->account->imapHost = 'localhost';
|
||||
$this->account->imapPort = 993;
|
||||
$this->account->imapEncryption = 'ssl';
|
||||
$this->account->smtpHost = 'localhost';
|
||||
$this->account->smtpPort = 587;
|
||||
$this->account->smtpEncryption = 'starttls';
|
||||
$this->account->password = 'x';
|
||||
$this->account->authType = 'password';
|
||||
$this->account->isActive = true;
|
||||
$this->em->persist($this->account);
|
||||
|
||||
$this->inbox = $this->mailbox('INBOX', 'INBOX', MailboxSpecialUse::INBOX);
|
||||
$this->archive = $this->mailbox('Archive', 'INBOX.Archive', MailboxSpecialUse::ARCHIVE);
|
||||
$this->trash = $this->mailbox('Trash', 'INBOX.Trash', MailboxSpecialUse::TRASH);
|
||||
|
||||
$this->em->flush();
|
||||
|
||||
$this->user->addAccount($this->account);
|
||||
|
||||
$this->labels->bindMailbox($this->labels->systemLabel(LabelRole::Inbox, $this->account), $this->inbox);
|
||||
$this->labels->bindMailbox($this->labels->systemLabel(LabelRole::Archive, $this->account), $this->archive);
|
||||
$this->labels->bindMailbox($this->labels->systemLabel(LabelRole::Trash, $this->account), $this->trash);
|
||||
|
||||
$this->em->flush();
|
||||
}
|
||||
|
||||
private function mailbox(string $name, string $fullPath, MailboxSpecialUse $specialUse): Mailbox
|
||||
{
|
||||
$mailbox = new Mailbox();
|
||||
$mailbox->account = $this->account;
|
||||
$mailbox->name = $name;
|
||||
$mailbox->fullPath = $fullPath;
|
||||
$mailbox->specialUse = $specialUse;
|
||||
$mailbox->isSyncEnabled = true;
|
||||
$mailbox->isIdleEnabled = false;
|
||||
|
||||
$this->em->persist($mailbox);
|
||||
|
||||
return $mailbox;
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user