mirror of
https://github.com/karatektus/pl_mail.git
synced 2026-10-25 10:04:18 +00:00
Paul Lützner
c8dffdf0a1
Opening a message told the sender it had been opened. Eight images loaded from images.ctfassets.net the moment a newsletter was rendered — one of them 526x5, which is what a tracking pixel wears when it is pretending to be a spacer — and each one carried the reader's IP address, their user agent, the time, and, from a per-recipient URL, which recipient it was. Read receipts, working perfectly, for anyone who cared to send one. Remote images are blocked now, and the block is not the interesting half. The interesting half is that saying yes does not undo it: an opted-in image loads through /mail/image-proxy, which fetches server-side and streams the bytes back, so the sender learns that somebody's plMail asked and nothing else. There is also "always for this sender", per user, in a table with a unique constraint — because trusting a sender is a button in a message, the same message can be open in two tabs, and an allowlist that a double click can duplicate is an allowlist that a race can lose an entry from. The proxy is a service that connects wherever a stranger's mail tells it to, from inside the container network, so its rules are stated in order and tested one by one: https only; port 443 only, which removes port-scanning rather than blacklisting the interesting ports; every address the host resolves to must be public, and the connection is then PINNED to a checked address through the client's `resolve` option, because checking and letting the client resolve again is the whole rebinding hole; redirects followed by hand, three at most, each hop re-running all of that, since max_redirects would follow a public host to 169.254.169.254 past every check; an image content type; 8 MB. Every refusal answers with the same transparent GIF, because telling the caller "refused" rather than "timed out" against an internal address hands back exactly the port-scan result the rules exist to deny. The signature on the URL is NOT the SSRF defence and says so in its own docblock — anyone with an account can mail themselves a link and be handed a valid signature for it. Deliberately not IntegrationUrlValidator, which has an allow-list and an admin-pinned base URL because a user is naming their own Nextcloud on a LAN. Importing its escape hatches here would mean an admin who allowed a LAN host for Immich had also allowed every sender's mail to reach it. The docs there claimed pinning the resolved IP "needs something Symfony's client does not expose"; that is not true, `resolve` does exactly that, and the correction is in this commit next to the code that disproves it. The body no longer renders in the app's own DOM. The sanitizer held — nothing got through it — but "no gaps, forever, against every sender" is a claim that has to keep being true, and the price of it being wrong once was a same-origin script with the session cookie. It renders in an iframe srcdoc whose value is what is absent from its sandbox: no allow-same-origin, so an opaque origin with no cookie, no storage and no reach into this document; no allow-forms, so a phishing form cannot even be submitted; no top-navigation, no modals, no downloads. allow-scripts IS present, and that is a deviation from the brief worth stating plainly: height is a fact only the framed document can measure, a frame without allow-same-origin cannot be measured from outside, and the alternatives were a fixed height or a guess. It is paid for by the frame's own CSP restricting script-src to a per-render nonce, which our thirty lines of height-and-hover reporting carry and no markup arriving in an email can. That CSP also names our origin in img-src explicitly — not 'self', which in an opaque origin matches nothing — so the blocker and the browser both have to fail before a pixel fires. Every warning is drawn OUTSIDE the frame. A banner inside the sandbox is a banner the mail can paint for itself, and a "Show images" button a message can forge is worse than no button. The hovered-link preview follows the same line: detected in the frame, where the links are, drawn by the parent, so a message cannot paint a status bar claiming a different destination. A Hetzner-branded invoice phish sat in Spam and rendered at full fidelity with nothing said about it. Messages in Spam now carry a banner, and the sender-name check is two rules that fit in a sentence each, because a warning nobody can check is a warning people learn to click through. Rule 1: the display name spells out a domain that is not the sender's. Rule 2 runs only when the display name carries a legal form as a whole word — GmbH, Inc, Ltd — which is what separates an organisation from a person; then, if none of its non-generic words of four letters or more occurs in the sender's registrable domain, the names disagree. "Hetzner Online GmbH" <support@ownkhalsick.com> fires; the same name at hetzner.com does not; "Deutsche Bahn AG" at bahn.de does not; "Jane Cooper" is never judged at all. DKIM passing for the sender's own domain suppresses both, since a domain that signed the message may write what it likes. A lookalike domain containing the brand is a known miss and has a test saying so — catching it means edit distances against a brand list, and every false positive that bought would be spent on ordinary mail. A trusted sender still gets no images inside Spam. The allowlist records a belief about a sender; a message in Spam is the provider disagreeing about whether this really is that sender, and the cautious reading of a disagreement is the right one. Blocking runs at RENDER time, not at ingest, and both reasons matter. Every message already stored was sanitized before this existed, so an ingest-time block would have protected only mail that had not arrived yet. And the answer depends on who is asking, because the allowlist is per reader — a stored form cannot carry an answer that varies by reader. Two layout fixes ride along, from the same report. Attachments rendered after the full HTML body, some 750px below the fold on a long message, and an attachment nobody scrolls to is an attachment nobody was sent; the chips move under the header, the same partial, relocated rather than duplicated. And the message header is sticky within the reading pane, so the answer to "who is this from" is not a scroll away from the sentence that raised the question. Verified in a browser, because every claim here is a claim about what the browser does. tests/e2e/rendering-security.spec.ts asserts on route interception — no connection to the sender's host, before or after opting in — and its matcher is a hostname predicate rather than the obvious glob, which would have matched the proxy's own URL because the sender's URL lives inside its query string, and reported a leak at the exact moment the feature worked. Two traps found on the way, both now in the handbook. Dom\HTMLDocument rejects LIBXML_NOWARNING with a ValueError rather than ignoring it, and the blocker fails closed, so the symptom was every message rendering as plain text: perfectly private and completely unreadable. It also needs the UTF-8 override, because MailBodySanitizer drops <head> and the body has no meta charset left.
plMail documentation
The handbook. README says what plMail is and gets you to a running instance; this covers everything after that — every feature and how to use it, how to install it on whatever you have, how to register the Google and Microsoft applications it talks to, and how the parts work underneath.
Mirrored to the GitHub wiki, which is generated from these files. Edit them here; browser edits to the wiki are overwritten by the next mirror.
| If you want to | Start at |
|---|---|
| Use a feature | Using plMail |
| Install or run it | Installing and running |
| Connect Gmail, Outlook or a CalDAV server | Providers |
| Understand or extend the internals | How it works |
Using plMail
One page per area. Each ends with links into How it works for the mechanism behind it.
| Page | What it covers |
|---|---|
| Reading, threads, labels, search and its operators, snooze, attachments, composing, drafts, undo send | |
| Accounts and aliases | Adding Gmail, Outlook and IMAP accounts, sending aliases, per-account settings |
| Filters | Condition trees, actions, the plain-English restatement, applying a rule to mail that already arrived |
| Calendar | The four views and the time grid, creating and editing events, recurrence, editing one occurrence, the docked pane |
| Invitations and events from mail | RSVP, events extracted from invitations and from ordinary prose, proposals, Happening Soon |
| Reminders | Setting alerts, how they are delivered, what a fresh install needs before they arrive |
| Connected calendars | Subscribing to Google, Microsoft and CalDAV calendars, two-way sync, ICS import, export and feed subscriptions, duplicate meetings |
| Sharing and booking | Share links and what each reveals, appointment pages, how a booking arrives |
| Files and integrations | Attaching from and saving to Drive, Photos, OneDrive, Dropbox, Nextcloud and Immich |
| Security | Two-factor authentication, recovery codes, remembered devices, app passwords, sessions |
| Other clients | Connecting a JMAP client, per-app passwords, the PWA and browser notifications |
| Appearance | Themes, custom colours and background, import and export, language |
| Administration | Users and roles, enabling integrations, monitoring, queues and the version chip |
Installing and running
| Page | What it covers |
|---|---|
| Docker Compose | The supported path, start to finish |
| Platform notes | Linux, Windows via WSL2, macOS, and NAS boxes |
| Behind a reverse proxy | TLS, APP_PUBLIC_URL, TRUSTED_PROXIES, and what breaks without them |
| Configuration reference | Every environment variable, what it does and what happens if it is wrong |
| Backup and restore | What to back up, the encryption key, and restoring onto a new host |
| Configuration backup | Carrying an install's settings and credentials to another one, as one encrypted file |
| Upgrading | Migrations run on boot, what that means, and how to tell which build is running |
| Troubleshooting | Health checks, the queue, logs, and the failures that have actually happened |
Providers
Each page names the exact console, the exact checkboxes and the exact redirect URIs.
| Page | What it covers |
|---|---|
| Cloud project, OAuth client, scopes for mail and calendar, Pub/Sub for Gmail push, calendar watch channels and domain verification | |
| Microsoft | Azure app registration, redirect URIs, delegated permissions, tenant choice, Graph subscriptions |
| IMAP and SMTP | Plain mailboxes, IDLE, and the settings servers disagree about |
| CalDAV | Discovery, app passwords, and the servers that have been tested |
| ICS feeds | Subscribing to a published calendar by URL, and why some addresses are refused |
How it works
Deeper than CONTRIBUTING's notes, and aimed at somebody auditing or extending plMail rather than running it.
| Page | What it covers |
|---|---|
| Architecture | The layers, what lives where, and the rules that keep it that way |
| Mail ingest | The pipeline from provider to database, threading, categorisation |
| The calendar model | JSCalendar in jsonb, projected columns, occurrences, recurrence and overrides |
| The sync engine | The driver contract every provider implements, push channels, deduplication |
| Event extraction | How an invitation and how a sentence become a calendar entry |
| JMAP | What is implemented, what is deliberately not, and the id spaces |
| Security model | Encryption at rest, the secrets file, tokens, and what a public link can reach |
Client authors should also read Client development, which is the protocol-level reference.
Conventions in these pages
- Commands are given as you would run them against a Compose deployment:
docker compose exec php php bin/console <command>. Drop the prefix if you run plMail directly. - Settings paths are written as
Settings → Calendar → Connected calendars. - A "things that bite" section at the end of a page collects the traps — the failure whose cause is not obvious from the symptom. They are there because they happened.