mirror of
https://github.com/karatektus/pl_mail.git
synced 2026-10-25 01:54:10 +00:00
main
49
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
13303b247e |
v0.3.9 milestone (#283)
* Run the end-to-end suite on pull requests into milestone and wip branches
A release's work is now collected on a milestone branch (milestone/v0.4.0)
before it reaches main, and the suite only ran on pull requests into main,
so nothing proposed for the milestone was tested. `milestone/**` joins
`main`, and `wip/**` comes along from wip/28, where it already runs the
suite on the phases of #28 — the same block on both, so they merge
without a conflict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Paul LütznerandClaude Opus 5.5
|
33f58ade8b |
Publish an image for every push to a milestone branch (#261)
A milestone branch collects a release before it reaches main, and nothing built it, so an install could not run that release until it was merged. docker.yml now builds pushes to milestone/** the way it builds main: amd64, tagged with the branch name (milestone-v0.3.8) and sha-..., never latest and never the catalogue. The tag is in the upgrading page's table, in both languages, and in the changelog's note on image tags. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
122c8e56f3 |
Publish a GitHub Release for every v* tag, with its CHANGELOG section as notes (#240)
A tag gave a feed reader nothing but a version number: there was no releases.atom and nothing for "Watch → Releases" to notify about. A new `release` job in docker.yml runs after the image is published, cuts the tag's section out of CHANGELOG.md with bin/release-notes.sh (unwrapped, since a release body shows every newline), and creates the release, or rewrites its notes on a re-run. Discussion #226, asked by @adrianfurgol. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b453b79eb4 |
Single sign-on only: hide the password form and refuse password sign-ins (#210)
* Single sign-on only: hide the password form and refuse password sign-ins Adds Admin -> SSO 'Single sign-on only' (and OIDC_DISABLE_PASSWORD_LOGIN). The login page drops the password form and LoginFormAuthenticator refuses a password sign-in before looking the user up. It only applies while a whole provider is in force, the saving administrator must already be linked to the provider, and app:sso:allow-password-login brings the form back from the console. Config restore leaves it off. Off by default; one boolean column on oidc_config. Closes #169 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Single sign-on only: require a link to the provider being saved The admin form let the box be ticked by an administrator linked to any provider, including the one the page held before. Switching providers and ticking it in one save made a provider they had never used the only way in. It now compares the stored issuer with the one on the page, without a trailing slash, as OidcDiscovery does. Tests for what had none: the console command (clears the setting, keeps the provider), a restore over a row that had it on, the link to another provider, and a control showing the same password signs in with the setting off. The "right credentials" test now posts the right password. Docs: there is no iOS app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Docs: put the iOS app back in the single sign-on page The previous commit dropped it on the wrong assumption that there is no iOS app. There is one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Paul Lützner <karatektus@gmail.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
0fd9e84a11 |
What's new: the ? menu above the calendar, main builds, and feedback with a screenshot (#167)
* The ? menu draws above the calendar, and a main build announces its newest release The menu was an ordinary dropdown inside the header's z-20 stacking context, so the calendar's day headings painted over it. It now opens in the top layer through ui--user-menu, like the account menu, and closes when an item is chosen. A build that is not a tag (main, sha-…, a checkout) counted as no release, so installs that follow main never saw the dialog. It now counts as the newest tagged release in its own changelog; Unreleased is still never shown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Feedback with a screenshot stays in the dialog, whatever comes back Sending with a screenshot closed the dialog without the sent screen. The form is now posted with fetch() and its answer drawn into the dialog; anything that is not the dialog (a proxy refusing a large body, a server error) is said in the form with its status, and the dialog stays. The screenshot is a JPEG at most 1600 px wide instead of a full-size PNG, a fraction of the size, so a reverse proxy's default body limit is far away. The From is the first field of the account that is an address; an account with none is refused in the form instead of throwing at send time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Find the open conversation's toolbar by type in the keyboard-shortcuts spec The spec looked for the conversation by id alone, and the messages inside it carry the same attribute with their own ids. On CI a message id equalled the conversation's, so two elements matched and the strict locator failed on the first try and the retry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
994c00b5cd |
What's new opens after an update, with a way to send feedback (#166)
* What's new opens after an update, with a way to send feedback The release notes from CHANGELOG.md, read from the image, in a dialog that opens by itself the first time somebody uses plMail after an update and never on a fresh install. A ? in the top bar opens it again, with a dot while something is unseen; "Don't show this after updates" silences it, and Settings -> General -> What's new undoes that. Highlights are marked in the changelog with an HTML comment and may carry a screenshot taken by tests/e2e/whats-new-screenshots.spec.ts. Send feedback is a mail from one of the person's own accounts to APP_FEEDBACK_TO (feedback@pluetzner.de by default), with an optional rating and a screenshot of the page behind the dialog. Existing users are recorded as having seen 0.3.1 by a migration; anyone created later starts on the release running at their first request. Closes #165. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plain changelog headlines from 0.3.2 on, and highlights with screenshots Every bold headline users see under "New to you" now says what changed in a few words. Each release gets a handful of highlights, eight of them with a picture cropped to the card or pane they are about, taken by tests/e2e/whats-new-screenshots.spec.ts. Relative links in the notes keep their words and lose the target, and the parsed changelog's cache key carries a format version so a new image never serves the previous one's parse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep highlight pictures for ten releases, and none that nothing names Every file in public/whats-new ships in every image. ChangelogReaderTest now fails on a picture no highlight names, and on one whose release is more than ten back; the release pull request that crosses the line deletes it and leaves the entry a highlight without a picture. Documented in CONTRIBUTING.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Credit #166 in the changelog Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Match the Back button by its whole name in the back-navigation specs Playwright matches a string name as a substring, so "Back" also found the new "Help and feedback" button in the top bar, which comes first, and the specs opened its menu instead of going back to the list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
ffbf7bdd9b |
Clean up the compose files (#160)
Compose files keep only a short header; the reasoning lives in docs/install. truenas.compose.yaml is gone (the TrueNAS catalog app replaces it) and so is compose.prod.yaml (compose.yaml already runs the published prod image). compose.override.yaml.dist becomes compose.dev.yaml, passed with -f, so there is no untracked copy that compose loads on its own. Docs, CODESTYLE, CONTRIBUTING and the entrypoint's error text follow. Every remaining file resolves to the same `docker compose config` as before. Closes #30 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
0139616c57 |
Document the inbox tabs for client authors, backups and the ingest internals
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul Lützner
|
7898182e00 |
Merge branch 'pr-87' into review-87
# Conflicts: # docs/de/features/mail.md # docs/de/install/config-backup.md # docs/de/internals/mail-ingest.md |
||
|
|
09c503c61c |
Sign in through an OpenID Connect provider (Authentik, Keycloak…) (#106)
plMail could only sign people in with a local password. On an install that already runs Authentik or Keycloak, that is one more account and password per person, outside the provider's access policy. A second authenticator on the `main` firewall, beside the password form. Both end in the same token for the same User, so the user checker, the login throttle and TOTP two-factor apply unchanged. Authorization code flow with PKCE, state and nonce on league/oauth2-client's GenericProvider; endpoints come from the provider's discovery document. - Set up under a new Admin → SSO tab; the OIDC_* variables are the fallback for an install configured from its compose file, and a saved provider wins whole. This lets an administrator who points it at a provider of their own sign in as any account without a second factor; the security model says so. - The ID token's signature is not verified: it only ever comes from the token endpoint over TLS (OIDC Core § 3.1.3.7 step 6). So the issuer and endpoints must be https and no request to the provider follows a redirect. Accepting an ID token from anywhere else needs JWKS verification first. - An account is linked by (issuer, sub). The address is used for the first link only, and only when the provider reports it verified. - Auto-creation is off by default and creates an ordinary account with no usable password. - app:user:oidc-unlink forgets a link from the console, for somebody deleted and re-added at the provider. Migration: two nullable columns and a unique index on `user`, and a one-row table `oidc_config`. Nothing backfilled; reversible. Not run: Playwright on this branch. No automated test makes a round trip against a real provider; that was checked by hand on a local test stack. Closes #91 Co-Authored-By: Paul Lützner <mail@pluetzner.de> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4d705a56da |
Keep remember-me tokens on the server, so signing out holds (#104)
* Keep remember-me tokens on the server, so signing out holds A remember-me cookie that carries only its own signature stays valid until it expires, and signing out only asks the browser to delete it. A request already on its way with it - the page's own background fetches - reached the server after the sign-out and signed the person straight back in. The tokens are kept on the server now (Symfony's DoctrineTokenProvider, table rememberme_token), one per device. Signing out deletes this device's token, so a late cookie opens nothing; other devices stay signed in. Symfony's CacheTokenVerifier keeps parallel requests that rotate the token from looking like theft. Existing signed cookies are no longer read: people who relied on "Keep me signed in" sign in once more. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say what remember-me is now, wherever it was described Remember-me moved from a signed cookie to tokens kept on the server, and five places still said "signature-based, not stored": the handbook's security page, the config-backup page's account of why APP_SECRET stays behind, and three comments beside the code that depends on it. English and German both. RememberMeAcrossRestoreTest proved its point against SignatureHasher, which the firewall no longer uses. It now asserts what the property rests on today: the firewall holds the persistent handler, and a backup taken while a token row exists carries neither its series nor its value. One test added for the claim the docs repeat: a changed password ends the cookie, because the token carries a digest of the hash it was issued under. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paul Lützner <mail@pluetzner.de> |
||
|
maniandClaude Sonnet 5.5
|
44d5cefec5 |
docs: describe inbox tabs of your own and the filter action that fills them
The filters page gets the action, a worked carrier example with the marketing exclusion, what the action will and will not file (authenticated senders only; on Gmail only when plMail is told to sort over Google; nothing but a tab) and three "Things that bite". The mail page gets "Tabs of your own", the backup page the new rows, the ingest page why the verdict has a column of its own, and the JMAP README why the five-value vocabulary is untouched. German pages follow with their hash markers refreshed. CHANGELOG notes the migration and the one extra inbox read. Also pins, in the JMAP category test, that a conversation in a reader's own tab is still served as its built-in category and that a custom key is refused. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
896f30eb02 |
Import an account's history without holding up new mail
A first sync used to be the import: every IMAP folder from its oldest message to its newest, every Gmail message listed before one was fetched, every Microsoft folder enumerated to its end, all in one job on the queue new mail arrives on. A large account showed an empty Inbox for most of an hour and every other account's mail waited behind it (#42). The sync now only arranges for new mail to be seen. History is MailImporter's, a page at a time on a new `ingest_backlog` queue with a worker of its own, newest mail first and the Inbox before anything else; Spam and the bin come last. Accounts take turns, a page that fails costs itself, and every sync restarts an import that has gone quiet. - IMAP walks each folder down from the top in pages of fifty. - Gmail asks for the newest 200 of the Inbox, then pages the listing. - Microsoft reads each folder's first delta round a page at a time; a folder being read is asked only what it received since. Removals that cannot be judged while a folder is unread are kept and judged later. - Sync requests a later sync has already covered are skipped. - A chip beside the search box shows that an import is running and how far it has got. The indicator never updated on mail pages, because its controller was not mounted there. Also from #42: an account can have a separate password for sending, behind a button under the SMTP fields. One field by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
da5a0bc9ee |
Carry category sorting in the config backup
User::$categorySorting (source, overrideProvider, tabs) was never written into a config backup, so a restore put everyone back on rules, the provider's categories and a tabbed inbox. It now travels through toArray()/applyArray() like Appearance and AiPreferences do. A backup without the key restores the defaults; an unknown source falls back to the rules without losing the two switches beside it. ConfigBackupCompletenessTest now fails for any #[ORM\Embedded] on User that the export does not carry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
3e0b4456cf |
Say what the default TRUSTED_PROXIES trusts, and where to report a hole
#34 points out that the default trusts every private range, so a machine on the LAN that reaches plMail directly can forge X-Forwarded-For and step around the per-address limits on the login form and the booking page. The default stays. The proxy's own address cannot be known here, and a proxy missing from the list breaks OAuth sign-in, shares one login allowance between everybody and strips `secure` off cookies, with nothing saying why — a worse failure, on every proxied install, than the one being closed. What was missing is the sentence: the comment in .env and the reverse-proxy page now say that the default is wide, what that costs, what it does not (the per-account login limit ignores the address and still holds), and how to narrow it once the proxy works. SECURITY.md is new. The report was filed publicly because there was nowhere private to send it. Refs #34 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
a8aea33874 |
Open a fresh install from the LAN, not only from the machine it runs on
The stock compose file served https://localhost and nothing else. Reached by address or hostname, Caddy had no certificate to offer and ended the handshake, which a browser reports as an SSL protocol error with no way to proceed. A headless server could not be set up without first setting SERVER_NAME. The default now answers every name and address with the self-signed localhost certificate (default_sni/fallback_sni, and `:443` in the default SERVER_NAME), so the browser shows its usual warning. Live updates follow: compose.yaml's https://localhost hub address is treated as the placeholder it is and gives way to the public address stored during setup. Any other MERCURE_PUBLIC_URL, and any SERVER_NAME an operator sets, is left alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
808b2bff53 |
Give a demo mailbox templates to pick from
A visitor who opened Settings → Templates on a fresh demo found one empty folder and a button, and the compose window's picker said there were none. Both true, neither showing what the feature does. The provisioner now seeds six: a quote follow-up, an invoice reminder, a payment receipt, a meeting follow-up, an out-of-office and a short acknowledgement, filed at the top level, under the demo account and in a folder inside it. Between them they use every variable, and the date one with offsets in both directions and three formats. app:test:seed-demo seeds the same six, for the reason it seeds the same mailbox: a screenshot of the picker should be a picture of the demo. They are strings with tokens in them, which nothing checks — a misspelt variable is braces left in the text — so DemoTemplatesTest renders every one for a recipient and fails on an open variable or a brace. It also erases a visitor with nested folders, since the eraser removes rows one by one and folders point at each other. The demo account is not given a signature. The signature variable is in two of the templates and renders as nothing until the visitor writes one, rather than a signature appearing in every new message and in the README's compose screenshots to make one variable busier. One layout bug fell out of looking at it. A long folder path — "you@example.com / Quotes and invoices" — pushed the editor's Folder dropdown out past the card's edge; the control may now shrink and the path is cut with an ellipsis. Refs #26 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
ec938bca99 |
Count the sessions a public demo hands out, without keeping an address
Admin → Demo visitors, in demo mode only: sessions started in the last hour, 24 hours, 7 days and 30 days, the different visitors behind them, the total and how many have a mailbox now. The users cannot be counted because the reaper deletes them, so /demo writes a row of its own. That row holds a keyed hash of the address with its last part cut off (a /24 or a /48), never the address, and the reaper removes the hash after 30 days. The demo's privacy notice describes it. Outside demo mode the section is not in the nav, its route answers 404 and nothing is recorded; the table exists and stays empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
9c4cebf8b9 |
Stop rewriting the cache directory at every start
Starting plMail took over two minutes on a NAS with spinning disks, and the log was silent for all of it — between "Statement statistics are enabled" and the web server's first line. An empty install on the same box, from the same image, was up in seconds. The entrypoint ended with two `setfacl -R` passes over var/cache and var/log. var/cache is part of the image: about 2,200 files. Changing anything about an image file from inside a container, an ACL included, makes the storage driver copy the whole file into the container's layer and flush it. So every new container rewrote its cache one synchronous file at a time, in every container of the stack at once, on one pool. Measured in a fresh container on an SSD: 1.2 s for the first pass, 0.03 s for the second. On the warm production container, 0.06 s — which is why timing it there showed nothing. It ran only as root, so the catalogue install (uid 568) never paid it. Nothing needed the entries: nothing runs as www-data, root needs no grant, and the Dockerfile already makes both directories writable. This is the third time that block has been the answer to a slow start. It walked all of var/, then var/cache and var/log, and now nothing. Also, on the compose files here: the database health check had no interval, so Docker's first probe came thirty seconds after start and the web server and workers could not begin before it. Five seconds now, with retries raised so a database in trouble is called unhealthy after the same two and a half minutes as before. The README sends TrueNAS users to the catalogue app instead of walking them through truenas.compose.yaml. The file stays: installs made from it are still running. Not confirmed on the box that reported it; the copy-up cost is measured locally and inferred there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
88b1744385 |
Leave the ingest worker to fetch mail, and nothing else
Adding an account put new mail behind hours of follow-up work. What
plMail does to a message once it has it — a model call where the
assistant sorts, event extraction, insight cards, receipts — was queued
on `ingest`, beside the syncs, for every message of the import, oldest
first. One consumer serves that queue. 988 envelopes pending and the
oldest an hour old was a mailbox being asked about newsletters from
2019 while mail on the server waited to be fetched.
`ingest` is now what talks to a mail provider. The rest moved:
enrich_live worker-live follow-up work for mail that has just
arrived on an account whose import is
over; never behind an import
enrich worker-enrich the same for an import's recent mail,
and search indexing
enrich_backlog worker-enrich old mail for the assistant, consumed
only when `enrich` is empty
release worker-release ends a hold (below); nothing else
Which queue is decided per batch, not per message class: the pipeline
asks InitialImportState once, carries the answer on PostIngestResult,
and EnrichmentRouter turns it into a stamp.
Only mail from the last 30 days (ENRICH_RECENT_DAYS) gets follow-up
work on arrival. Older mail is stored, threaded and filed by the rules.
The assistant is asked about it afterwards — newest first, a bounded
handful per run, and not until every account of that person has
finished importing (app:ai:classify-backlog, ClassificationCatchUp).
For the other steps `app:backfill` remains the way to read history.
ExtractInsightsStep and ProposeEventsStep ran their work inside the
sync and argued for it; both dispatch now.
New mail no longer appears in one tab and moves to another. For
somebody sorting by assistant the rules' category is a guess about to
be replaced, so where the verdict could still change it the message is
held: stamped before threading, its new thread given no category, which
every tab query already reads as "in no tab". The answer releases it.
Failing that a timed release does, after AiSettings::holdMaxSeconds —
60 by default — and the mail is shown where the rules put it; the model
call carries on and moves the mail when it answers. A message is asked
about once. Admin → AI has the switch, the ceiling, and the measured
delay over the last week beside them.
Three things about the release that were each wrong once first:
- It ran on the live worker, so it could not fire while that worker
was inside the call that was taking too long. It was moved to
export, which mixed mail appearing with mail leaving. It has a
queue and a worker of its own.
- The call was cut off at the ceiling and the message asked about
again later. The hold and the question are separate; only the hold
has a deadline.
- A late verdict was stored and moved nothing. It moves the mail.
Both workers still end at one model host, so bulk classification looks
at the live queue before each call and stands aside (LiveMailPriority).
One migration: two columns on `message`, two on `ai_settings`.
Not verified against a real import or a live model host: the tests pin
the queues, the hold and the release against in-memory transports and a
mocked host. Known and left: IMAP still imports oldest first; a reply
into an existing thread cannot be hidden; a Gmail account counts as
importing for up to an hour after its last message lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
533b773b33 |
Stop needing to be told what a catalogue can work out
Packaging plMail for a catalogue that forbids per-app workarounds turned
up four places where the image expected its compose file to compensate.
cookie name Chosen from the public address: the __Secure- name on
https, a bare one otherwise. It used to be prefixed
unless an operator knew to set MERCURE_COOKIE_NAME, and
an install opened on http://ip:port never got a live
update. The hub always reads the bare name and the web
server renames the cookie in the proxy, so the two
cannot disagree after only one of them restarted.
storage dir An absolute APP_STORAGE_DIR inside the project is
accepted and normalised to the relative form, so the
stored attachment paths do not change. One outside it
is refused at start.
hub address MERCURE_UPSTREAM names the container the web server
proxies the hub to, for a stack that does not call it
mercure.
database secret No POSTGRES_PASSWORD is minted, and no password file
written, when DATABASE_URL already carries a password:
that database is somebody else's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
be3b27cf32 |
Let an administrator change the public address after setup
Setup asked for it once and nothing could answer it again. An install restored from another machine's backup therefore kept the old machine's address: browsers were sent there for live updates and the stream indicator stayed red, with an environment variable in the compose file as the only remedy. Admin → Address edits the same value setup writes. The change is saved to the generated file and takes a restart to apply. Making it live was left out on purpose: the entrypoint exports that file into every container's environment at start, and telling a stale export from a deliberate pin needs more machinery than a restart costs. The page shows the address in use beside the saved one whenever they differ, and offers the web container's restart after every save. One thing fell out. A hub address on file that is only the previous public address with the hub's path on it is dropped on save, so it is derived from the new address again; a hub address of its own is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
0507c1e716 |
fix(truenas): chown the Postgres directory to the uid the alpine image runs as
postgres:18-alpine runs as uid 70; 999 belongs to the Debian image. The install worked anyway because the Postgres entrypoint re-chowns PGDATA as root, but the chown, the setfacl fallback and the docs named the wrong user. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
9c25eae1bf |
fix(secrets): the JWT keypair lives beside the generated secrets
.env named var/secrets/jwt outright, which is beside the generated
secrets only while APP_SECRETS_FILE keeps its default. The TrueNAS
layouts point it into their data dataset, so there the keypair landed on
each container's own throwaway filesystem: every service generated one
on every start ("Generated the JWT keypair." in the log), and a token
one of them signed would have been refused by the others and by the
next restart. Nothing issues JWTs yet - the Android app pairs into an
app password - so nothing broke, but the promise in the comments that
every service verifies what the others sign did not hold.
- config/bootstrap_generated_secrets.php derives both paths from the
directory APP_SECRETS_FILE is in, for every PHP process including a
`docker exec` console. A real environment variable still wins; the
.env defaults are empty, as for the other derived values. On the
standard compose the paths are the ones they always were.
- app:secrets:init generates under a lock, rechecked once held, as
GeneratedSecretsFile::ensure() does. lexik's --skip-if-exists is a
check followed by two writes, so services starting together could
leave one's private key beside another's public key - a narrow window
that seven concurrent runs did not hit in five rounds, and closed now
rather than left rare: with the lock every round generated exactly
once and left a matching pair.
- A reset that rotates secrets removes the two key files where they
are, instead of emptying a fixed var/secrets/jwt.
- The security.yaml note on "the future first-party app" is corrected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
a8364e34d2 |
feat(deploy): one worker container runs every background process
The stack was nine long-running containers: the web server, the database, the Mercure hub, and six of plMail's own image, one per background process. It is three now: php, worker and database, with secrets-init still running once and exiting. The worker runs `app:work`, a small supervisor keeping seven processes alive the way Docker's restart policy kept seven containers alive: the IMAP supervisor, one consumer per queue, the scheduler and the hub. A clean exit (the hourly recycle, the restart signal) comes straight back; a crash backs off from one second to thirty. Output passes through a whole line at a time, so two processes' JSON records never interleave, and SIGTERM reaches every child at once with 25s to finish. Each queue keeps a process of its own, which is the part the split was for: a consumer inside a long handler cannot take a send. And each process keeps its container's old name as APP_CONTAINER_NAME, so heartbeats, /healthz and the log filters are unchanged. The hub is the official Mercure 1.x binary, copied into the image and run as one of those processes. Not FrankenPHP's built-in hub, which is 0.24 and refuses the RFC 9068 tokens plMail has minted since v0.2.31. The worker answers to `mercure` on the network, so the Caddyfile proxy and MERCURE_URL reach it where they reached the hub container, and older compose files keep working with this image unchanged. It is configured from the application's own settings, including MERCURE_COOKIE_NAME, so a plain-HTTP install names the cookie once for both halves. A 5s shutdown grace stops it waiting on open subscriptions, which made every stop take the whole 25s. The dev override loses its per-worker blocks, one of which (worker-bulk) had been missing, so that queue ran the published image's code in development. The demo overlay runs app:work --without=imap-supervisor. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
4cbdb4f511 |
feat(admin): a version checker, with a channel to follow
Admin → Updates: an administrator picks which builds count as an update (Releases, the `latest` image; Main, the `main` image; or Off), and every hour app:updates:check asks whether that channel has a newer one. Until somebody picks, an installation follows the tag its own build came from, so a `main` image follows main and an unbuilt checkout asks nothing of anyone. The registry decides what exists: the channel's image is read the way `docker pull` finds it (OCI distribution API, anonymous token from the 401's challenge, the index entry for this architecture, the config's labels), so a commit that builds no image is never announced. GitHub's compare decides which way two builds differ, so a main build following releases is not told to go back to an older release, and its commit list is the page's "what's new". An image from anywhere but GitHub falls back to what the version numbers alone can say. Each new build is announced once: a push to every administrator's registered devices (a new UpdateAvailable payload the service worker shows), a badge beside the build chip in the admin header, and a dot on the nav entry. A check that gets no answer keeps the last one and says why, and the hourly command exits 0 for it: an installation behind a firewall is not a fault. Images are stamped with where they are published (APP_IMAGE), so a fork that publishes its own checks its own. The channel travels in config backups; what the last check found does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
1aadc53218 |
fix(security): prune finished jobs and expired trusted devices nightly
Both tables grew forever: TrustedDeviceRepository::pruneExpired() and BackgroundJobRepository::findFinishedBefore() had no caller. The nightly app:monitoring:prune now deletes expired trusted devices and background jobs finished more than 30 days ago (--job-days). The job lookup became a bulk DELETE, since hydrating would scale with the history and trip over any row whose kind the enum no longer lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
dee4257d3f |
fix(security): check every hop of user-supplied URLs against private networks
The Nextcloud, Paperless and Immich drivers followed up to twenty redirects unexamined, the URL validator let any hostname through without resolving it (container names, nip.io, 127.1, 2130706433, mapped IPv6), and push endpoints were not checked at all. Requests to user-supplied URLs (the three drivers, CalDAV, ICS feeds, Web Push) now go through UserUrlHttpClient, which wraps NoPrivateNetworkHttpClient and re-checks every hop. The validator resolves the host. LAN integrations keep working through the existing INTEGRATIONS_ALLOWED_HOSTS and admin-pinned server addresses; push endpoints get their own PUSH_ALLOWED_HOSTS for the bundled ntfy, and never redirect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
376cf4ae72 |
Add Paperless-ngx as an attachment source and a save destination
Connect it with an address and an API token and it appears beside the other services; nothing else in the UI had to change, because the picker, the destination chooser and the connect dialog are all driven by the provider taxonomy and the driver interfaces. TAGS ARE THE CONTAINER LAYER. A document archive has no directory tree — Paperless replaced folders with tags and expects the rest to be found by searching — so the picker's "descend into a container, pick a file" has nothing to descend into. The root lists every tag as a folder with the newest documents beneath it, and opening a tag lists that tag's documents. Correspondents and document types would serve equally well and are deliberately not offered: three sideways slices of one archive is a navigation problem, and search already crosses all three. Search is Paperless's own full-text index, so it reads the OCR layer as well as the metadata and a scanned invoice is findable by a line item nobody typed. That is the feature: there is no longer a reason to open Paperless, find a document and download it before writing the mail. Upload is asynchronous — post_document answers with a task id, not a document id — so upload() returns it prefixed. Ids are opaque outside the driver, which is what makes the prefix safe, and download() refuses a task id with a sentence a person can act on rather than letting it 404 later. ShareLink is not declared. Minting a Paperless share link is a permissioned feature with its own expiry semantics, not something to do silently. The taxonomy test that assumed only photo services lack share links had a false premise; it now states the real rule rather than being relaxed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
ab9ecbc9e0 |
Hand a whole-view bulk action to a worker, which it never was
RunBulkStatusHandler exists to take a mark-read or archive over a whole view out of the request — its docblock quotes the thirty-second timeout that produced it, and BulkStatusController answers with an indicator so nobody waits. Neither was true: the message was never listed in messenger.yaml's routing map, and Messenger handles an unrouted message inline in whichever process dispatched it. Every whole-view action ran inside the web request, under the limit it was written to escape, at request concurrency — which is where the production deadlock came from. It gets a queue of its own rather than joining maintenance, because somebody is watching an indicator for it and queueing it behind an embedding backfill would be slower than the inline version it replaces. That means one new container, worker-bulk, in every compose file and in the handbook. The deadlock itself is retried now instead of failing the job, three times per chunk, and a collision that outlasts those hands the job back to the queue rather than ending it. The failure path no longer writes through an EntityManager Doctrine has closed, which is what it was doing for every failure that came out of a flush. And the list catches up: nothing ever refreshed it on the strength of the job, only on the toolbar's own "the write landed" event, which is now the instant the work was QUEUED rather than finished. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
822a17bfef |
Name the last silent path, and document the proxy it points at
The performance panel said it: "Last call: Summaries · cancelled". That row is only written when the generator is ABANDONED — AiAssistant::recorded() records ERROR_CANCELLED when its consumer never reached the end — so the summary was not timing out at all. It was the controller's abort path, which breaks out of the loop when connection_aborted() fires and returns without a frame and without a word, because losing the reader is a normal thing that needs no explaining. Except it is only normal when the reader left. From inside this process, a proxy closing the stream looks exactly the same, and one of those is a fault. Silent, the two were indistinguishable: a card reading "the summary could not be written", nothing in the log, and a `cancelled` row that reads as somebody changing their mind. Four rounds of this were spent on timeouts because the one record that named the real shape was in a panel nobody thought to read. It logs now, with the elapsed seconds and how much of the answer had reached the browser. Those two tell the cases apart: somebody who gives up does it at a human moment and usually mid-answer, while a proxy does it at its configured one and during the silence before the first token — and the same round number every time is the whole diagnosis. AND THE DOCUMENTATION HAD HALF OF THIS ALREADY. reverse-proxy.md turns buffering off and raises the read timeout to 24h — for /.well-known/mercure, and nothing else. plMail serves two long-lived streams and only one of them was ever mentioned; the assistant's endpoints got nginx's default 60-second read timeout, which is shorter than a summary of a long conversation legitimately takes. That is now written down, in both languages, with the failure mode spelled out next to it. No behaviour change beyond the log line. The proxy block is the fix, and it is not one plMail can apply to somebody else's proxy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
e7f4513574 |
fix(backup): the config backup had stopped carrying what it promises
A restore looked like it worked. That is the whole shape of this: nothing threw, the review page listed a clean plan, and the parts that had gone missing produced no row to warn about, because a section that does not exist has nothing to say. ── A removed user came back to life ──────────────────────────────────────── The restorer's own docblock and the handbook both say a soft-deleted user counts as existing, "because they hold the email address against a unique index". They had stopped holding it. Removal frees the address on purpose — the admin delete action rewrites it to `deleted-<id>@invalid` — so existingId() answered null for every removed user, the plan called them Absent, and the restore created them afresh and LIVE: old password hash, TOTP secret, recovery codes, every mailbox credential. Listed as an ordinary creation, indistinguishable from a new colleague. The handbook steers operators straight into it. The rewrite destroys the only copy of the address, so nothing could match it back; `email_at_deletion` is that copy, written at the same moment. Rows tombstoned before the migration stay unrecognisable, which is stated and held by a test rather than papered over. The regression test that was supposed to catch this passed throughout, because it wrote `deleted_at` and left the email in place — a row the application cannot produce. It now tombstones the way the delete action does. Its sibling: a violated unique index escaped as an uncaught 500 with no message. api_token.token_hash is unique across the WHOLE install and a soft delete cascades nothing, so a removed user's tokens still hold those values. Translated into a stated failure with a sentence attached. Not demoted like other failed writes — a violated constraint aborts the transaction, so there is no "rest" to let through, and stopping with a reason is the honest shape. ── Four things a person configured never reached the file ────────────────── The assistant configuration in full: model host, both models, the reverse-proxy token that exists in plaintext nowhere else, the feature switches, and every prompt an administrator rewrote. The per-user half had been carried since the day it shipped; the instance-level half it depends on had not, so a restore returned everyone's preferences pointing at a host that was gone. Account signatures and read-receipt defaults — and on the instance this was found on, `compose.signature` was the ONLY account setting in use, so every export it produced held an empty settings bag for every account. The per-alias overrides key themselves by alias row id, which names somebody else's alias on the target, so they now travel inside their alias and are rewritten under the new id after the flush, the way `calendar.target_id` already was. Five user settings, the chosen log level, and the OAuth scopes that describe what the provider granted to the refresh token the file already carries. ── Why none of it was caught, which is the part worth keeping ────────────── Every guard pointed the wrong way. The env test builds its expectation by calling the constant under test. The users test is fourteen assertArrayNotHasKey calls — it fails if somebody ADDS a field, never if one goes missing. The version assertion compares a constant against its own echo. A backup's characteristic defect is omission, and nothing could fail in that direction. Two tests now compare the backup against the application rather than against itself, and fail closed: every User and Account SETTING_ constant must be carried or excluded on the record with its reason, and the database section must carry every admin-configured table. The second would have gone red the day AiSettings landed. A round-trip test pins the set of things a restore drops to exactly two, each deliberate, so the set can only change on purpose. ── Also ─────────────────────────────────────────────────────────────────── The KDF had a ceiling and no floor, so `memlimit: 1` passed every check and took a SodiumException out onto a blank error page — through the anonymous /install/restore among others. Both memzero paths now run under try/finally, the abnormal case they exist for. App passwords keep the date they were created rather than the date they were restored. And the admin handbook said the file contains "no user accounts". It contains every one, with their secrets — published in two languages. Corrected, along with the inventories, which now list what actually travels. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
755f3cf257 |
feat(users): an administrator can run the whole life of an account, and you can change your own password
Managing users was reachable and worked — list, search, create, promote, demote, remove — but three things around it did not, and one of them was load-bearing. ── You could not change your own password ────────────────────────────────── Nowhere. Not in Settings, not by mail, not at all; only an administrator setting one for you. Settings → Security now leads with a Password card that asks for the current password (an unmapped UserPassword constraint, so it is checked against the signed-in token rather than against anything the form was handed) and takes the new one twice. The route carries no id and the form has no data_class, so it structurally cannot become the admin form by accident. The floor those passwords are held to was three literal 12s in three files that happened to agree. It is now User::PASSWORD_MIN_LENGTH, read by all three. ── An account could only be removed, never switched off ──────────────────── Nothing sat between "leave it alone" and "destroy it", so the answer to "this person has left, keep their mail" was to delete them. There is a deactivated_at now, and a UserChecker registered on BOTH firewalls — the JMAP one included, because an app password outlives the browser session it was minted from and a check only on `main` would shut somebody out of the web UI while every mail client they had already connected kept syncing. The refusal is in checkPostAuth rather than checkPreAuth on purpose. checkPreAuth runs before the password is verified, so refusing there would answer "is this address a suspended account here?" to anyone who types an address — the question hide_user_not_found exists to keep unanswerable. After verification, the only person told is the one who could otherwise have signed in. Symfony runs user checkers at authentication only, so a session already open would have carried on until it happened to end. UserProvider::refreshUser() refuses the same two states on every request. Suspension also travels through config backup, which would otherwise have handed a suspended account its sign-in back on restore. ── Two smaller lies ──────────────────────────────────────────────────────── A search under three characters silently returned the unfiltered list, which a QueryBuilder has no way to complain about, so the floor was dropped rather than kept as something that could only mislead. And an admin declining to demote themselves got a success toast for a change that never happened; the refusal now says so, while the rest of the form still saves. ── Restoring a removed user is deliberately absent ───────────────────────── Removal overwrites the address, the name and the hash specifically to free the unique address for reuse, so a restore would have to invent all three and the original address may by then belong to somebody else. Doing it properly means snapshot columns and a guided flow that asks for a fresh address — a different change, which alters what removal means. Documented as absent on the screen an administrator would look for the button on, because switching an account off is the answer to the question people are actually asking. Also fixed en route: the modal form theme dropped help_translation_parameters, so the first field to use one rendered "at least %count% characters" on screen. And GermanIsInformalTest only ever globbed messages.de.yaml, leaving validators and the new security catalogue — exactly where a translator reaches for "Sie" — outside the test that exists to catch it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
2348b75621 |
feat(db): pgvector as an opt-in image, built without changing libc
The original rejection of pgvector was tested rather than reasoned, and it stands: pgvector/pgvector:pg18 means musl to glibc on an existing data volume, and 44 collation-dependent indexes — 18 of them UNIQUE — are then walked with the wrong comparator, silently, because musl records no collation version so Postgres's own mismatch check never fires. So the extension is compiled INTO the alpine image instead. libc never changes, and there is nothing to rebuild. Verified end to end rather than argued: an existing volume migrated on stock alpine, stopped, restarted on the pgvector image, then bt_index_check(heapallindexed => true) across 175 btree indexes — zero errors. Two build flags carry more weight than they look. OPTFLAGS="" because pgvector's Makefile defaults to -march=native, which bakes the build host's instruction set into vector.so and gives SIGILL mid-query on a host without it, with an empty log. with_llvm=no because a missing .bc file makes the JIT skip inlining silently, and inlining does not apply to index scans anyway. Pinned at v0.8.6, not the earliest that works. v0.8.0 does not compile against Postgres 18 at all (vacuum_delay_point gained a parameter); v0.8.1 is the first that does. But 0.8.3 fixed possible index corruption with HNSW vacuuming and 0.8.4 fixed an unrepaired HNSW graph, and message_embedding is ON DELETE CASCADE from message on a system that deletes mail in bulk — so vacuum is a hot path here and shipping the earliest that merely compiles would ship two known corruption bugs. THE HANDBOOK'S 24x WAS WRONG AND IS CORRECTED TO 1.75x. The SIMD dot product is microseconds; what dominates is that embedding is real[], so every row pays a 4KB detoast and a fresh real[] to vector conversion. 74,000 rows: 7.70s becomes 4.51s. NO INDEX IS CREATED, and that is the finding rather than an omission. An HNSW index answers ORDER BY ... LIMIT k and the search asks distance <= 0.45, which no ANN index serves. The expression form needed to build one at all is (embedding::vector(1024)), which Postgres treats as a DIFFERENT expression from the inlined function's (embedding::vector) — so the index exists, the planner ignores it, and nothing says so. And it costs 7m40s and 578MB to build at the maintenance_work_mem this stack ships, behind the one advisory lock every entrypoint waits on: an eight-minute dead stack, not a slow migration. The measured route to an index is written down in ai-assist.md rather than half taken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
5f648864ab |
feat(admin): set the log level from the page that shows the logs
Asked for directly: "let the admin panel set the log level. i do not want to mess with env vars." APP_DB_LOG_LEVEL meant editing a file on the host and restarting the stack, and the moment it is actually wanted is the moment that is most awkward — something is wrong, the answer is one level further down, and restarting throws away whatever was in flight. So the level gets turned down, forgotten, and left down. WHAT IT IS NOT Not another control in the filter row. That row narrows what is SHOWN out of what was stored; this decides what gets STORED. They look alike and mean opposite things, and side by side they would invite an admin to change the wrong one and conclude the log was broken. Its own row, its own form, its own wording. NULL IS NOT "TODAY'S VALUE" A stored level of null means "follow the environment", which is deliberately different from copying APP_DB_LOG_LEVEL's current value into the row. An install that has never chosen keeps following its configuration including later changes to it, and there is a way back to that after a level has been set — which storing the resolved value would have removed, silently, on the day the env var next changed. PER RECORD, NOT PER PROCESS Monolog fixes a handler's level at construction. That is right for one configured in a file and useless for one an administrator can turn down from the page they are reading, so DoctrineLogHandler decides in isHandling() instead, and the constructor's level is now only the answer when no resolver was given — which keeps directly-constructed handlers, and every existing test, behaving as they did. LogLevelResolver caches for ten seconds. A query per log line would put a database round-trip inside the thing that runs when the database is in trouble; a value cached for the life of the process would leave every long-lived worker ignoring the change until the next deploy, and the worker logs are usually the ones it was lowered for. It uses DBAL, not the ORM, for the same reason the handler does, and every failure — no table yet, no database, a name that is not a level — falls back to the environment rather than throwing. A logger that throws while deciding whether to log is worse than one briefly out of date. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
8c2ef7663b |
feat(ops): a demo deploy that cannot quietly do nothing
Three commands in the handbook, one of which fails silently.
`docker compose pull` can run out of disk part-way through extracting a
layer. `up -d --wait` then brings the stack up on the image that is
ALREADY there and reports every container Healthy — so a deploy that
changed nothing is indistinguishable from one that worked, and the only
evidence is a line buried in the pull output. v0.1.36 was believed
deployed for twenty minutes while the demo went on serving v0.1.35.
Piping made it worse. Through a pipe the exit status belongs to whatever
came after it, so the `pull | tail -2` in the handbook's sequence
reported success no matter what the pull did. Nothing in the script is
piped.
And the disk fills on its own: every deploy leaves its predecessor behind
and nothing collects them. Eleven orphaned images is what filled it, on a
host that also runs Apache, MySQL and ClamAV and never has more than a
few GB spare.
So: check for room, prune if short, pull and STOP if that fails, up,
then clear the images the deploy replaced.
· `-a` on the prune is load-bearing. A replaced release is a fully
tagged image nothing references any more, and dangling-only leaves
every one of them.
· `--volumes` is never passed. The demo's database is in one.
· compose.proxy.yaml is included by existence rather than by name: it
belongs to a host that already has something on port 80 and is not in
this repository.
Verified on the real demo host, both ways — a normal run reports "image
unchanged" and exits 0, and DEMO_MIN_FREE_MB=999999 refuses, exits 1, and
leaves the running stack alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5
|
748e1cdcdf |
feat(demo): a privacy notice, and two fixes found by using the demo
**The privacy notice gets its own page and its own link.** A section inside the Impressum is not one — it has to be reachable in its own right — and it is required even though this instance asks for nothing. A web server writes the caller's IP address into a log on every request and an IP address is personal data, so "we collect almost nothing" is a statement that has to be made rather than a reason to omit the page. Written from what the software does rather than from a template. The cookies named are the three it actually sets, observed on a real visit; the claim that nothing reaches a third party is the content security policy, which permits connections to this server only; and the retention period is read off APP_DEMO_TTL through DemoMode::ttlDescription() rather than typed into the prose, because a notice stating a period the reaper does not honour is worse than one saying nothing, and two copies drift the first time somebody tunes the setting. APP_DEMO_PRIVACY_HOST names whoever runs the machine, who is a processor. Unset, the page says so in red — the same rule the Impressum follows, for the same reason: a legal notice naming nobody looks compliant from a distance. **Pressing Receive twice looked broken.** The second press appeared to do nothing for up to thirteen seconds. _refreshList() coalesces bursts through a fifteen-second floor measured from the LAST refresh, so the press right after a delivery was the one certain to be inside it. The floor is right — a sync publishes one event per mailbox per account, and it turns eight fetches into one — but it was being applied to somebody who had just pressed a button and was watching for the result. refreshNow() skips it; onAccountSynced() still does not. The difference is whether anybody is waiting. **The user menu still wore its own tooltip.** The previous fix stopped a NEW bubble appearing over an open menu but left the one already on screen, which is why it vanished only when the pointer left. There is a document-level click listener whose whole job is to take it down, and ui--user-menu#toggle calls stopPropagation(), so the click never reached it. The listener runs in the CAPTURE phase now — at the document, before the event reaches its target, where no handler can hide a click from it. The same reason the scroll listener above it already uses capture. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
6e9a441333 |
fix: an install page a demo cannot close, and eleven flashes nobody translated
Two bugs found by using the thing. **A demo visitor could lock an install out of /install.** InstallGuard closes that page as soon as a user exists, and demo mode mints one per arrival — so on a fresh demo instance the first passer-by or crawler took the install page away with them, permanently, before the operator had made themselves an administrator. Nothing announced it either: the page 404s, which is exactly what it does on a correctly installed instance, so the symptom is indistinguishable from working. Both the guard and createFirstAdmin's advisory-locked re-check now count real users only. **Flash messages reached the page as raw translation keys.** The flash bag on this install holds sentences, not keys — the toast region prints what it finds, and cannot translate, because most of these interpolate a name or a count only the controller has. Eleven call sites across TwoFactorController, SecurityStepHandler and AdminDashboardController pushed keys instead, so a rejected code produced a red toast reading "two_factor.flash.code_rejected". The strings existed and had been translated into all three locales months ago; nothing looked them up. A static test now scans src/ for flash calls passing a dotted literal, because nothing about that failure is visible in review — the call site reads exactly like a correct one. That fix broke two E2E tests, which is the interesting part: they asserted on "Two-factor authentication is on" and had been matching the panel heading ALONE because the toast beside it was showing a key. Translating it made both match and strict mode rightly refused. They were passing because of the bug. **Demo mode, from running one publicly.** The account form opened in a turbo-frame, so the guard's redirect had nowhere to land and the modal span forever — it now renders a panel saying why the form is not coming. The receive button was a full browser navigation, which reloaded the page on top of mail Mercure had already delivered AND wiped motion.js's in-memory set of seen rows, so the whole list replayed its entrance; it is a stream touching only the demo bar now, and mail simply arrives. Plus an Impressum, because a public demo in Germany needs one — blank by default and loudly saying so, since § 5 TMG wants a real operator and a notice naming nobody looks compliant without being so. **Interface, all reported from use.** The snooze menu's two distant options were taller than the near ones, because anything past two days carries the date and "Sat, Aug 29, 08:00" wrapped inside min-w-56. Clicking a dropdown trigger focused it, and focus shows a tooltip — so opening the user menu painted its own hint across the menu. The eight recovery codes ran together on one line: they were joined with real newlines and <code> collapses those without whitespace-pre-line, so nothing was rendering what was already there. Confirming a 2FA code gave no sign anything was happening, and the reply carries codes shown exactly once. Layout was the last <select> on a panel where every other short choice is a segmented group. Solar and paper put a white field on a warm off-white surface, which under the boxed layout's translucency left the search box with no visible edge. Read receipts had a tab of their own coming — the card holds nothing else, and signatures left the alias tab before it for the same reasons. And /settings opened on its fourth entry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
231d0ab0da |
feat(demo): a public demo mode, with mail that arrives on demand
plMail is hard to show. It is a mail client, so everything interesting only
happens once there is mail — and a stranger cannot reasonably be asked to hand
a demo server their IMAP password to see it. APP_DEMO_MODE fills that gap: mail
that is nobody's, and a button that makes more of it arrive.
One switch for the whole install, not a per-account flag. The promise is that
nothing reaches the network, and that is only checkable if there is one place
to check; a per-account flag would move the promise into every outbound path
individually, where it holds until the first one that forgets to ask. Read
through `default:` so an install that never heard of the variable is a normal
install — the accident worth designing against is a real deployment becoming a
demo, not the reverse.
**A mailbox per visitor.** /demo provisions a throwaway user, signs them in and
drops them in a seeded inbox. One user each rather than one shared account: two
people arriving at once would otherwise be marking each other's mail read and
reading each other's drafts, and the product would look broken while working
perfectly. app:demo:reap deletes them on a timer.
**Receiving.** A bar at the foot of the page walks a scripted queue — a reply,
an invoice with a real attachment, a delay notice, an HTML newsletter — and
wraps at the end, naming what comes next. Delivery goes through the real
PostIngestPipeline rather than writing rows by hand, because what is being
demonstrated is that plMail *does* something when mail arrives: it threads, it
categorises, it runs the filters the visitor just built, and it appears without
a reload. A hand-written row would put mail on screen and none of that would be
true, which is a worse demo than no button.
**Sending.** DemoMailSender is registered above every real sender and claims
every account while demo mode is on, so SmtpMailSender is never asked — "a demo
cannot reach a relay" becomes a property of the wiring rather than a rule each
sender has to remember. It answers a few seconds later, from whoever the visitor
addressed, on the thread they are still looking at.
**What is switched off**: adding or editing a mail account, OAuth and CalDAV
connect, calendar subscribe — a stranger typing their real credentials into a
box on someone else's server has handed their mail away, and there is no version
of that form that is safe to show. Mail sync, push renewal and calendar sync
no-op, since a demo's accounts point at documentation domains. Everything else
stays clickable: locking down all the non-mail surfaces produces a tour of a
product with the interesting doors locked.
Two things this needed that did not exist. The demo mailbox moved out of
SeedDemoMailboxCommand into DemoMailbox, shared with the provisioner, so the
README screenshots and the hosted demo cannot drift into being pictures of
different things. And deleting a user — this project had no such path, because
a self-hosted install's users leave by having their password changed. The
eraser discovers owned classes from Doctrine's metadata rather than a list:
twenty entities point at User today under two different property names, and a
hand-written list would be right until the next entity was added, at which
point the reaper would start failing on a foreign key at three in the morning.
compose.demo.yaml hardcodes APP_DEMO_MODE rather than interpolating it, and
ComposeEnvironmentTest now holds it to that. Compose reads Symfony's committed
.env, which ships the variable as 0, so `${APP_DEMO_MODE:-1}` would have
resolved to 0 for everyone who cloned the repo — and the failure is the bad
kind, where the stack comes up healthy and simply is not a demo.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5
|
d3e4eced03 |
fix: three corrections the suite caught, one of them a recipient bug
**The shared dropdown escape was too broad.** Sending every ui--dropdown menu to the top layer fixed the ones trapped in a pane and broke the ones deliberately anchored to the box they live in — the compose window's emoji picker is centred ON the window and asserted to stay inside it. It is opt-in now (`data-ui--dropdown-escapes-value`), and the snooze menu on a mail row asks for it because it opens inside the mail pane. **The recipient dropdown is NOT reparented to <body>, and that is now written down.** `dropdownParent: 'body'` fixes the reported symptom in one line — the compose window grows to make room for the suggestion panel instead of the panel overlaying it — and it reintroduces something worse. Subject sits directly under To, so an overlaid panel covers it, and a click aimed at Subject lands on whichever suggestion is under the pointer and enrols it. That is a real report: it silently added a third party to a message addressed to someone else, and three specs in compose-recipients.spec.ts exist to keep it fixed. They caught this. So the panel stays in flow and the window growing is the price, with the reasoning in the code so the next person does not spend the same hour. The way out is a layout change — the panel opening where it cannot cover the next field — not a parent change. **The session guard swallowed too much.** It cancelled every 401 and 403, not just the redirect that would have swapped a login form into the shell, so the compose dock's own "your session has expired, reload and sign in" — said in the place the user just clicked — never rendered. Only the redirect is cancelled now; a bare 401 travels on to whoever asked for it, who usually has something more specific to say. compose-safety's held-shape test sets its own setting and restores it in an afterEach rather than an afterAll, so a failure there cannot leave the worker's user on a setting the rest of the file does not expect. Full Playwright suite: 563 passed, 0 failed. That is 563 rather than the 517 a green run used to report — the exclusive project depends on the main one, so every flake was also skipping those 19 specs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
8c3080d00a |
fix(security): an SVG attachment is not an image, and a shipped install gets headers
CODE-REVIEW S-02 (high), S-04, S-08, S-05, S-06. S-02 — three controllers decided what may render inline with `str_starts_with($contentType, 'image/')`, and `image/svg+xml` satisfies it while executing script as a top-level document. nosniff does not help: nothing is mislabelled, the type genuinely is what it says. The type comes from the MIME headers of an incoming mail. Reachable from the mail itself. MailBodySanitizer::resolveCids() rewrites EVERY `cid:` in the document rather than only `img src`, and the sanitiser allows relative links — so `<a href="cid:logo">Rechnung ansehen</a>` has the part id filled in by the application, and the click opens the attachment route without the `?download=1` that every template adds. Now an allow-list (InlineDisposition), which is the difference between "everything not yet known to be dangerous" and "the formats actually wanted", plus `default-src 'none'; sandbox` on the response — so a later widening of the list is a bug rather than a vulnerability. Parameters are stripped before comparing, because `image/svg+xml; charset=utf-8` would otherwise walk past the one entry meant to stop it. S-08 pulled along: the JMAP download had the same prefix test. Guarded today only by the jmap firewall being stateless and wanting a header a browser navigation does not send, which is a thin thing to depend on. S-04 pulled along: the file-picker preview carried no Content-Disposition at all — implicitly inline, in our origin, with a type chosen by Nextcloud, Immich, Dropbox, OneDrive or Drive. A file an attacker drops in a shared folder is a realistic shape for the setups these integrations exist for. A non-image now falls back to the placeholder rather than to a download: a download prompt fired by a thumbnail is worse than a grey box. S-05 — the Caddyfile now sends HSTS, X-Frame-Options, nosniff, Referrer-Policy and Permissions-Policy, and drops the Server banner. None of these existed anywhere in the repository, so the ones a live instance showed were its operator's proxy — while the README's own install instructions are `docker compose up -d` and "open https://localhost". `defer`, so the per-response policies on the attachment and image-proxy routes still win. Referrer-Policy is the one that matters most for a mail client: it sends its users to strangers' links all day. S-06 — mercure subscribe defaults to [] rather than '*'. Never reached today, since both cookie-minting paths pass explicit topics, but a future createCookie($request) without a topic argument would have issued a subscription to every user's mail events and looked entirely ordinary in review. Also: mailpit removed from the dev override. MailerInterface is injected in no service in src/ — plMail's own outgoing mail leaves through the user's account via MailSenderRegistry — and the override never set MAILER_DSN either, so the container has been catching nothing. Both configuration pages said the dev stack pointed at it; they now say what is actually true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Fable 5
|
9f9f2c9820 |
fix(backup): APP_SECRET and TRUSTED_PROXIES leave the export
Every other name left this inventory because it describes the machine. APP_SECRET left because of what a restored copy would do. Take stock of what this codebase derives from kernel.secret: remember-me is the signature-based handler, so the sixty-day cookie is signed with it; sessions and CSRF tokens die with the session; JMAP signs with the JWT pems, whose contents travel separately; URI signing and uuid47 are unused. Nothing on disk or in the database is encrypted under it -- that is APP_ENCRYPTION_KEY's job, and it does travel. So restoring it changed exactly one thing: a REMEMBERME cookie minted by the source install kept verifying on the target. Both halves of that check are in a v2 backup now that users travel -- the signature covers the app secret and the password hash -- so a browser signed in to the machine the file came from was signed in to the machine it was restored onto, without anybody deciding that. Cookies are not scoped by port, so two stacks on one host hand each other's browsers straight through. TRUSTED_PROXIES goes for DATABASE_URL's reason instead: the addresses whose X-Forwarded-* headers this install believes are a fact about what sits in front of THIS container. KeptDeliberately and External respectively, so old backups carrying either classify and stay inert. Found while reproducing a 403 that turned out to be a double submit of the 2FA form (previous commit). The stale-cookie hypothesis was falsified as the cause of that -- a pre-restore remember-me cookie was correctly challenged for a second factor, it did not bypass anything -- but reproducing it is what showed the cookie surviving the restore at all, which is the property this commit removes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Fable 5
|
d1739e6c96 |
fix(backup): MERCURE_JWT_SECRET leaves the export
It pairs the app with the hub container beside it, and the hub reads it only at container start -- so a restored value re-keys one half of a running pair and every EventSource dies until the whole stack restarts, which is how this was found. Old backups carrying it classify External, like the DSNs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Claude Fable 5
|
03a0d6b877 |
A config backup knows its own operator
Three changes to the config-backup subsystem, made together because they land on the same two pages and the same document. **Users, and everything they configured, join the backup.** Restoring onto a fresh install used to end at "this installation still has no users — create the administrator account to finish", which is a backup of a server rather than of an installation. Document version 2 adds a `users` section, keyed by email because that is what an import matches on. Per person: the password hash, the TOTP secret and its recovery codes, app passwords, mail accounts with their IMAP/SMTP and OAuth credentials, aliases, per-user integrations with their secrets, filters, labels, calendars and the share links and booking pages whose token digests keep published URLs working. Everything encrypted at rest leaves decrypted inside the envelope and is re-encrypted under the target's key on the way in, exactly as the database section already did — that is the only way a credential survives a move to a host with a different APP_ENCRYPTION_KEY. Deliberately absent: trusted devices and push subscriptions (grants to one browser or one phone, and a trusted device is a skipped second factor), label bindings and sync state (re-derived, or about a host that is not this one), avatars and background images (filenames in a volume this file does not carry), and mail, calendar entries and contacts, which have not moved an inch. Import policy: a user this install does not have is created whole; a user it does have is left completely alone, whole. Skipping is the design and not a caution — a backup is a photograph of a moment that has passed, and applying a stale one to a live account resets today's password to February's with no undo and nobody told. Soft-deleted users count as present. The ids inside a filter, a calendar or a booking page name rows in the *source* database, so the restore builds a map as it creates rows and rewrites every reference through it; a reference to something the file did not carry is dropped rather than guessed at. The /install restore page now has two endings and asks the guard which one it is in: sign in as somebody the file restored, naming the administrator, or — for a document that carried no users — create the administrator as before. **MAILER_DSN and MESSENGER_TRANSPORT_DSN leave the export.** Machine-local deployment choices that compose supplies a default for, in DATABASE_URL's category: the target's relay and queue are the ones that match the containers running beside it. Old backups carrying them classify as External and stay inert. **The restore result page stops listing non-tasks.** A restored value already identical to the live one is dropped from "These are yours to do" and counted in one muted line; the APP_ENCRYPTION_KEY block becomes a collapsed disclosure that still carries the line the one operator who needs it came for; and when nothing is actionable the page says so and puts the way out first. Two robustness fixes fell out: planning no longer hydrates every user to build a comparison (per-email now), and neither the existence check nor the comparison can 500 a review page on an install whose encryption key was rotated without re-encrypting. No schema changes — the backup is a file format, not tables. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Fable 5
|
7e3ba2b9c0 |
Reset the topbar globals per request, stop exporting the database's own keys, and offer a password
Three fixes riding one pass: LogAlertGlobal memoised per request and was reset per request only under PHP-FPM. FrankenPHP keeps the service alive, so the first badge a worker computed was the badge every later request got - an emptied log table with a yellow outline that survives refresh, seen in the wild. All four stateful Twig globals now implement ResetInterface like InviteReader always has; SidebarCounts and CalendarGlobal could additionally leak one user's counts into another user's topbar. POSTGRES_PASSWORD, postgres_password and DATABASE_URL leave the backup: machine-local, initdb-consumed, never applicable on a restore target. Old backups that carry them still import - recognised and classified external, not silently dropped. The export form gains an optional generate button: fills both fields, puts the password on the clipboard and says so; where the browser withholds its clipboard the fields unmask instead, because a masked password nobody was told is not a backup anyone can open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Fable 5
|
567db06844 |
Restore a config backup by writing it, not by printing it
The import classified every environment value as unwritable and handed
back two dozen lines to paste into .env.local. That was built on a
premise that is wrong for the way plMail is deployed: nobody hand-edits
those. frankenphp/generate-secrets.sh mints them on first run into
var/secrets/generated.env, every service mounts the volume that file is
on, the entrypoint loads it before it execs the server, and the app
process can write it. So the import writes it.
What the entrypoint actually promises, and what everything here follows
from: a real environment variable wins over the generated file, which
wins over .env.local and .env, with an empty value counting as absent at
every level. The shell (load_generated_secrets) and PHP
(config/bootstrap_generated_secrets.php) apply exactly the same rule.
ConfigBackupObstacle — "why can plMail not write this" — becomes
ConfigBackupDisposition, "what becomes of this": applied,
applied-on-restart, shadowed-by-compose, external, kept-deliberately,
not-writable. The restart instruction is now one sentence for the whole
plan rather than one line per value, because it is one action.
Three fates survive, each forced by the deployment rather than by
caution:
- POSTGRES_PASSWORD, the postgres_password file and DATABASE_URL stay
external. generate-secrets.sh does rewrite that file from
generated.env on every boot, but the Postgres image reads
POSTGRES_PASSWORD_FILE only at initdb — on a database that already
exists the role keeps the password it was created with, so writing
one without the other is a stack that cannot authenticate.
- APP_ENCRYPTION_KEY is kept on purpose: the import re-encrypts the
backup's credentials under the key in force here, and swapping it
underneath them would make the rows it just wrote unreadable. It is
a note now, not a chore — the line is still offered for the operator
who is also restoring the old database.
- A name pinned non-empty in the process environment beats the
generated file at the next start. Detected by comparing the live
value against what the file holds, because the entrypoint exports
that file's own contents and "is it in getenv" distinguishes
nothing. On the stock compose.yaml that is three names.
The /install path is the one this was for: upload and password are the
whole job, everything lands in this instance's own secrets file and
database, and the page asks for one restart instead of a wall.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Fable 5
|
1adac1fe40 |
Carry an instance's configuration in one password-sealed file
Export from the admin area: env secrets, the JWT keypair and the database-held provider credentials - FCM included, decrypted inside the envelope because the restore target holds a different encryption key - sealed with Argon2id and secretbox at parameters a Raspberry Pi can afford, in a documented format a php one-liner can open without plMail. Import shows its plan before touching anything: what it can write is written and re-encrypted under this instance's key, what it cannot - process environment, the Postgres role's own password - is rendered as the exact lines to place by hand. The install wizard offers the same restore before the first account exists, behind the same guard that closes /install forever afterwards. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
b4e3cb6afb |
Correct what translating the handbook found wrong in it
Six defects, all found by somebody reading a page closely enough to
render it in another language. Each is fixed in English and in German
together, and the German markers are re-hashed only where the German
prose now says what the corrected English says — re-hashing alone would
mark a stale translation as current, which is the one thing that check
exists to prevent.
calendar-model.md claimed the materialiser honours "exactly four
things in a patch" above a table of five, and its
traps section named a different four. The truth
is five keys read, of which four are the patch
EventInstanceEditor writes; `excluded` is not a
patch field at all and is spelled by
RecurrenceRuleConverter::exclusionOverrides(),
because the one override value whose only job is
to be exactly right should not have two places
that must be right about it.
CLIENT_DEVELOPMENT.md said snooze was "not exposed over JMAP today" and
offered itself as the canonical "ask, don't work
around" case — while Thread/set has been
implementing exactly that. Somebody asked. Its
methods table was also missing Thread/set,
SearchSnippet/get and all four calendar methods,
which the same page announces two screens above.
install/docker.md described the missing blob volumes as a live
defect. They were added to compose.yaml hours
after that page was written — by the report that
found them — and nobody updated the page. It now
documents ten volumes and keeps the story as
history, with a pointer to the CHANGELOG note for
anyone upgrading past it.
providers/imap-smtp.md named a form label no interface shows. The
hardcoded labels in AccountType are overridden by
the Twig template, so the fields are "Email
address" and "Username".
settings.nav.calendars had no German, so one item in a German settings
menu rendered in English.
CONTRIBUTING.md said "four translators" where there were six.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5
|
4d103cfbdf |
Write the handbook, and make it fail the build when it goes stale
README says what plMail is and gets you running; CONTRIBUTING says how to
change it. Nothing covered the part in between — using it, operating it,
and registering the Google and Microsoft applications it talks to. That
is docs/: 34 pages across features, installation, providers and
internals, mirrored to the GitHub wiki.
The wiki is a mirror and never a source. bin/mirror-wiki.php flattens the
tree into the wiki's flat namespace, rewrites every relative link,
generates the sidebar from the index so the two cannot disagree, and
stamps each page with a footer saying where it came from. A browser edit
survives until the next push to main. That cost is deliberate:
documentation not versioned with the code drifts from it, and the drift
is invisible until somebody follows an instruction that stopped being
true.
Which is the whole problem, so the checkable half is checked.
DocumentationCoverageTest fails the build when a command is missing from
the reference or documented and gone, when a variable in .env is absent
from the configuration page, when a relative link dangles, when a page is
unreachable from the index or promised and missing, or when a page has no
"Things that bite" section. It cannot check whether a paragraph is TRUE,
and says so — this is the floor, so review attention goes to the part a
machine cannot have an opinion about. CONTRIBUTING has claimed since it
was written that the command table "is part of the definition of done";
nothing enforced it until now, and it was wrong in three places.
Writing it found things. They are fixed here rather than documented
around, because a handbook describing a bug accurately is still a
handbook nobody can follow.
compose.yaml mounted no volume for var/attachments, var/raw
or var/uploads. Workers write those during sync
and the web container serves them back, so each
kept its own copy in its own layer — downloads
404, and everything died on a recreate. The
override and the TrueNAS file both mount them;
the file the README tells an operator to run did
not. The same omission cost real data once
before. See the CHANGELOG for moving what is
currently stranded.
app:calendar:materialise new, nightly. Occurrences are drawn when an
event is saved and nothing moved that window
again, so a weekly standup ran out of dates in
two years and its reminders stopped with them —
silently. The query it runs had existed since
the materialiser did, documented as exactly this
sweep, and had never been called.
Microsoft wizard asked for three of the nine permissions plMail
requests, and a token already issued is not
upgraded when the rest are added later.
GmailWatchService told you to grant Pub/Sub Publisher to
gmail-api@system.gserviceaccount.com. The
account is gmail-api-push@; a topic granted
exactly what the docblock asked for never
delivered a notification.
IcsExporter said it wrote no VALARM. It has written them
since alerts landed.
CLIENT_DEVELOPMENT.md said there was no JMAP calendar API and that
SearchSnippet/get was absent. Both shipped.
CONTRIBUTING said only app:calendar:push registers a push
channel; subscribing has done it since the
subscribe flow learned to.
Three docblocks named getters removed long ago, or described a
method two below the one they sat on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|