mirror of
https://github.com/karatektus/pl_mail.git
synced 2026-10-25 01:54:10 +00:00
main
151
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
13303b247e |
v0.3.9 milestone (#283)
* Run the end-to-end suite on pull requests into milestone and wip branches
A release's work is now collected on a milestone branch (milestone/v0.4.0)
before it reaches main, and the suite only ran on pull requests into main,
so nothing proposed for the milestone was tested. `milestone/**` joins
`main`, and `wip/**` comes along from wip/28, where it already runs the
suite on the phases of #28 — the same block on both, so they merge
without a conflict.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Paul LütznerandClaude Opus 5.5
|
3b78d277d7 |
A send that fails for a passing reason is retried; one that finally fails is marked on the draft (#242)
Every sender turned every failure into `false`, so a one-minute Gmail rate limit or a short outage lost the send like a refused recipient. The three senders now classify through MailDeliveryInterface::deliver(): deferred (rate limits, 5xx, a connection that never opened, SMTP 4xx), refused (other 4xx, SMTP 5xx, too large) or outcome unknown (a lost answer, 502/504). send() stays a bool for invitations, alerts and read receipts. SendMessageHandler retries a deferral for at most two minutes, keeping the provider's Retry-After. A lost answer is never resent blind: Gmail and Graph look for the Message-ID in Sent first (now committed before the first attempt), SMTP reports it as possibly sent. A final failure, from the handler or from SendGaveUpSubscriber when the ladder runs out, marks the draft, words the toast by its reason, and shows a red warning triangle at the start of the draft's row and on Drafts in the sidebar. Migration: four nullable columns and a partial index on message. Closes #125 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b453b79eb4 |
Single sign-on only: hide the password form and refuse password sign-ins (#210)
* Single sign-on only: hide the password form and refuse password sign-ins Adds Admin -> SSO 'Single sign-on only' (and OIDC_DISABLE_PASSWORD_LOGIN). The login page drops the password form and LoginFormAuthenticator refuses a password sign-in before looking the user up. It only applies while a whole provider is in force, the saving administrator must already be linked to the provider, and app:sso:allow-password-login brings the form back from the console. Config restore leaves it off. Off by default; one boolean column on oidc_config. Closes #169 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * Single sign-on only: require a link to the provider being saved The admin form let the box be ticked by an administrator linked to any provider, including the one the page held before. Switching providers and ticking it in one save made a provider they had never used the only way in. It now compares the stored issuer with the one on the page, without a trailing slash, as OidcDiscovery does. Tests for what had none: the console command (clears the setting, keeps the provider), a restore over a row that had it on, the link to another provider, and a control showing the same password signs in with the setting off. The "right credentials" test now posts the right password. Docs: there is no iOS app. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Docs: put the iOS app back in the single sign-on page The previous commit dropped it on the wrong assumption that there is no iOS app. There is one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Paul Lützner <karatektus@gmail.com> |
||
|
|
7a5a7350bf |
A conversation no longer splits when its replies are synced first (#176)
Threading looked only from a reply to the message it answers, so replies synced before that message (INBOX before Sent Items) opened threads of their own and stayed there. MessageThreader now also looks the other way: after a message gets its thread, mail that already names its Message-ID in In-Reply-To or References is folded in by the new ThreadMerger. The older thread survives, keeping its JMAP id, summary, insights, star, snooze, labels and pinned tab. A GIN expression index makes the reverse lookup an index probe. Review follow-up: moved mail is added to the surviving thread's collection so a label rule later in the same batch keeps its labels, plus tests for that, bracketed legacy references and provider-keyed survivors. Closes #172 Co-authored-by: Philippe Delodder <philippe@delodder.be> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7c9bf694ff |
Store a UIDVALIDITY above 2147483647 (#174)
mailbox.uid_validity becomes a BIGINT, so IMAP folders whose UIDVALIDITY is above the signed INT range sync again. Reversible migration; the mapping stays ?int (DBAL 4 hydrates a bigint as an int). Closes #171 Co-Authored-By: Philippe Delodder <philippe@delodder.be> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
994c00b5cd |
What's new opens after an update, with a way to send feedback (#166)
* What's new opens after an update, with a way to send feedback The release notes from CHANGELOG.md, read from the image, in a dialog that opens by itself the first time somebody uses plMail after an update and never on a fresh install. A ? in the top bar opens it again, with a dot while something is unseen; "Don't show this after updates" silences it, and Settings -> General -> What's new undoes that. Highlights are marked in the changelog with an HTML comment and may carry a screenshot taken by tests/e2e/whats-new-screenshots.spec.ts. Send feedback is a mail from one of the person's own accounts to APP_FEEDBACK_TO (feedback@pluetzner.de by default), with an optional rating and a screenshot of the page behind the dialog. Existing users are recorded as having seen 0.3.1 by a migration; anyone created later starts on the release running at their first request. Closes #165. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Plain changelog headlines from 0.3.2 on, and highlights with screenshots Every bold headline users see under "New to you" now says what changed in a few words. Each release gets a handful of highlights, eight of them with a picture cropped to the card or pane they are about, taken by tests/e2e/whats-new-screenshots.spec.ts. Relative links in the notes keep their words and lose the target, and the parsed changelog's cache key carries a format version so a new image never serves the previous one's parse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep highlight pictures for ten releases, and none that nothing names Every file in public/whats-new ships in every image. ChangelogReaderTest now fails on a picture no highlight names, and on one whose release is more than ten back; the release pull request that crosses the line deletes it and leaves the entry a highlight without a picture. Documented in CONTRIBUTING.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Credit #166 in the changelog Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Match the Back button by its whole name in the back-navigation specs Playwright matches a string name as a substring, so "Back" also found the new "Help and feedback" button in the top bar, which comes first, and the specs opened its menu instead of going back to the list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ebd2570a59 |
A draft synced from Outlook or Gmail opens in the editor, and one sent elsewhere stops being a draft (#164)
Synced drafts no longer get a sentAt, so the editor opens them. A migration clears it on rows the provider still files as drafts, and a draft that turns up in Sent is settled as sent. Takes over #123. Co-authored-by: Flouss <15911342+Flouss@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
b940bd3a31 |
The installed app says New mail only when mail arrived (#162)
The in-app Web Push device is now told only when mail lands in the Inbox of a mailbox that has finished importing, and Outlook stops recording folder moves for messages that did not move. JMAP clients and FCM phones keep every StateChange. Migration: jmap_push_subscription.arrivals_only, set once for in-app devices. Takes over #112. Co-authored-by: Flouss <15911342+Flouss@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4738d514dd |
Hear mail read, unread, starred or unstarred in Outlook (#114)
Co-authored-by: Flouss <15911342+Flouss@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
3c0769c1e6 |
Order every inbox tab, and let a built-in one be switched off
One order across the built-in tabs and the reader's own, and a set of built-ins switched off, both beside the other sorting choices on the user (CategorySorting). The order replaces the categories' own position column, which never shipped: the strip is one list, so its order is too. Primary is always first and cannot be switched off — it is where the rest falls back to. A switched-off category takes no mail. Whatever would have gone there — by the header rules, Google's label or the assistant — is in Primary, and the message panel says it would have been that category. The answer is mapped rather than the next rule consulted: "Forums is off" does not make a mailing list Promotions for its List-Unsubscribe. Until the re-sort a switch queues has run, and for a conversation somebody pinned there, Primary lists and counts the switched-off category's conversations too, so nothing is in no tab at all. InboxTabs is where order and on/off are applied, so the strip, the drop targets and every key that resolves agree. A restore rewrites the order's own-tab keys to the categories' new ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
e5420327ea |
Bring own inbox tabs up to date with main
- The migration is dated after the ones main has run since, under a name no open branch uses (#109 and #112 hold the earlier slots today). - The inbox's query ceiling is one above the shared one again: main raised that to 28 for the sidebar's change counter, and the own-tabs read is still the one statement the inbox adds on top. - French for the new strings. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul Lützner
|
7898182e00 |
Merge branch 'pr-87' into review-87
# Conflicts: # docs/de/features/mail.md # docs/de/install/config-backup.md # docs/de/internals/mail-ingest.md |
||
|
Paul LütznerandClaude Opus 5.5
|
72d3956b0d |
Keep every Gmail message in the mailbox it is in, Spam and Trash included
`app:gmail:skipped` on a real mailbox (#90) showed what the ownership filter was dropping: the account owner's own mail. 202 sent from "send mail as" addresses (From: pl@pmdevelopment.de) 89 forwarded in from an address not connected here 43 Gmail drafts 14 received, with no recipient header naming anyone 4 addressed to the googlemail.com spelling of the account 6 chats resolveOwningAccount() kept a message only when its headers named the account. It now keeps one for the account unless another connected account has the better claim (a Gmailify sibling named in its headers): SENT or DRAFT the sibling the From names, else this account received resolveReceivedAccount()'s answer, else this account; with Gmailify attribution switched off, still only mail addressed to this account CHAT nobody, and recorded as before GmailAddressFilter reads googlemail.com as gmail.com. The listing never asked for Spam and Trash although its comment said it did: messages.list leaves both out without includeSpamTrash, and getProfile's messagesTotal, the import's total, counts them. How much of the 11,200 that mailbox was short of this accounts for is not known yet: its import was still running when it was counted. A finished import does not list again, so the migration clears the record of declined messages (chats aside) and every Gmail account's backfill markers. Each lists its mailbox once more, fetching only what it does not hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
09c503c61c |
Sign in through an OpenID Connect provider (Authentik, Keycloak…) (#106)
plMail could only sign people in with a local password. On an install that already runs Authentik or Keycloak, that is one more account and password per person, outside the provider's access policy. A second authenticator on the `main` firewall, beside the password form. Both end in the same token for the same User, so the user checker, the login throttle and TOTP two-factor apply unchanged. Authorization code flow with PKCE, state and nonce on league/oauth2-client's GenericProvider; endpoints come from the provider's discovery document. - Set up under a new Admin → SSO tab; the OIDC_* variables are the fallback for an install configured from its compose file, and a saved provider wins whole. This lets an administrator who points it at a provider of their own sign in as any account without a second factor; the security model says so. - The ID token's signature is not verified: it only ever comes from the token endpoint over TLS (OIDC Core § 3.1.3.7 step 6). So the issuer and endpoints must be https and no request to the provider follows a redirect. Accepting an ID token from anywhere else needs JWKS verification first. - An account is linked by (issuer, sub). The address is used for the first link only, and only when the provider reports it verified. - Auto-creation is off by default and creates an ordinary account with no usable password. - app:user:oidc-unlink forgets a link from the console, for somebody deleted and re-added at the provider. Migration: two nullable columns and a unique index on `user`, and a one-row table `oidc_config`. Nothing backfilled; reversible. Not run: Playwright on this branch. No automated test makes a round trip against a real provider; that was checked by hand on a local test stack. Closes #91 Co-Authored-By: Paul Lützner <mail@pluetzner.de> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
4d705a56da |
Keep remember-me tokens on the server, so signing out holds (#104)
* Keep remember-me tokens on the server, so signing out holds A remember-me cookie that carries only its own signature stays valid until it expires, and signing out only asks the browser to delete it. A request already on its way with it - the page's own background fetches - reached the server after the sign-out and signed the person straight back in. The tokens are kept on the server now (Symfony's DoctrineTokenProvider, table rememberme_token), one per device. Signing out deletes this device's token, so a late cookie opens nothing; other devices stay signed in. Symfony's CacheTokenVerifier keeps parallel requests that rotate the token from looking like theft. Existing signed cookies are no longer read: people who relied on "Keep me signed in" sign in once more. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say what remember-me is now, wherever it was described Remember-me moved from a signed cookie to tokens kept on the server, and five places still said "signature-based, not stored": the handbook's security page, the config-backup page's account of why APP_SECRET stays behind, and three comments beside the code that depends on it. English and German both. RememberMeAcrossRestoreTest proved its point against SignatureHasher, which the firewall no longer uses. It now asserts what the property rests on today: the firewall holds the persistent handler, and a backup taken while a token row exists carries neither its series nor its value. One test added for the claim the docs repeat: a changed password ends the cookie, because the token carries a digest of the hash it was issued under. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Paul Lützner <mail@pluetzner.de> |
||
|
Paul LütznerandClaude Opus 5.5
|
6f24bfabc3 |
Write down the Gmail messages an import declines, and stop fetching them again
An import sat at 21,964 of 24,199 and said "waiting" (#90). The 2,235 are messages the batch handler fetches and does not store: its headers tie the message to no account here (a draft, a chat, mail sent from a "send mail as" address), or Gmail lists it and answers 404. Declining left a debug line and nothing else, and the import's only test for "still to come" is "not stored". So: every hourly listing named all 2,235 again, and each was fetched in full again: 134,100 units of the 360,000 an hour a 6,000 project has the import could only end at BACKFILL_MAX_ATTEMPTS, a day of listings later; the account in the report was on attempt 18 the progress line could never reach Google's total, which counts them gmail_skipped_message records each one when it is declined: the reason, its labels and its From, To, Cc and Delivered-To. newGmailIds() treats a recorded id like a stored one, so a listing with nothing else outstanding settles at once an onlyMissing batch queued by an earlier listing drops them too ImportProgress counts them as done app:gmail:skipped prints the record by kind, label and address; --forget clears it so the next listing asks again A row goes when its message is stored after all. Whether some of these messages ought to be stored is a separate question and is not touched: the filter decides exactly as before. The record is what that question can now be answered from. Migration: one new, empty table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
maniandClaude Sonnet 5.5
|
7adccfb6f5 |
feat(settings): let a reader make, rename and delete their own inbox tabs
A card on Settings -> General beside the sorting card: a list of the reader's categories with rename and delete, and a form to add one. A name is trimmed, at most 40 characters, not a built-in tab's name and not one the reader already has (case-insensitively); a refused name comes back as a toast saying why. Removing a category un-files its mail through the foreign keys and clears the pins of everything in it first, in one transaction: a foreign key cannot clear a pin, and a conversation pinned into a category that has gone would be skipped by every recompute for ever. A pin to a built-in tab or to another category is left alone. The filter list shows a category chip naming its category, or "(deleted category)". The unused colour column is dropped from the new table. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
maniandClaude Sonnet 5.5
|
d3e0782cf7 |
feat(mail): add inbox categories a reader makes themselves
A new inbox_category table, and two nullable columns that file mail under one: message.rule_category_id (a filter's verdict, kept apart from message.category because five writers recompute that and would erase it) and message_thread.user_category_id (the conversation's tab). Both references are ON DELETE SET NULL, so removing a category un-files its mail in the database. The built-in category columns, their enum mapping and everything JMAP serves from them are untouched. Nothing is backfilled and nobody's inbox changes until a category and a filter filing into it exist. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
Walid ManiandClaude Opus 5.5
|
99d2da3da2 |
Show each inbox tab's unread count, with a setting to switch it off (#54)
The sidebar's Inbox badge could read 1 while Primary, the tab that opens, had nothing unread: the conversation was in Promotions, and the only hint on the strip was a colour on that tab's icon. Each tab now shows how many conversations in it hold unread mail. It is the Inbox badge split by category, so the tabs add up to it, and it is patched in place on sync. This reverses the documented rule that the "3 new" pill is the only number on a tab. The badge and the bold rows say that there is unread mail and cannot say where. On for everyone, and a per-user appearance setting switches it off, beside the account corner. One column on the user table; the migration is reversible. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
88197578d5 |
Let search by meaning use a model host of its own
One host held both models. The two jobs often want different machines: writing wants the largest model somebody can run or rent, and search wants a small one that is asked constantly, for every query and once for every message of a mailbox. Writing from a hosted endpoint with search on an Ollama at home indexes the mailbox on a machine the operator owns and bills nothing per message, and one host for both could not express it. Admin → AI has a tick box that gives search its own provider, address, token and extra headers. AiSettings::connectionFor() returns that connection for search when the box is ticked, and nothing else had to change, because every call already asked for the connection of its feature. The second host is sent only its own token and headers, and none when it has none. A feature is usable when the host it runs on is filled in. Test asks both hosts, each for the model that lives on it. Stored vectors stay filed under the model name alone, so moving search to a host that serves the same model keeps the index and no stored row is touched. Five columns on ai_settings, unused until the box is ticked; the migration is reversible. The idea of a separate embedding connection comes from #58. Its vector identity, which hashed the address into every stored row, is left out: it forced a full re-embed whenever a host changed its address. Co-authored-by: shady <shady2k@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
cd7828395d |
Send the model host extra headers
A bearer token is not all that stands in front of a model. A reverse proxy with basic auth, a zero-trust gateway that wants a client id and a secret, a routing gateway with a key of its own, a provider that takes a project by header: each of those made plMail unusable with it. Admin → AI has an Extra headers box under the token, one `Name: value` per line. ConnectionHeaders reads them strictly while somebody can still fix what they typed, and forgivingly on the way to the wire, so a column that cannot be read costs the headers and not the feature. They are stored encrypted and ride on AiConnection, so both clients send them on every request without knowing they exist. The values are never shown again; the page lists the names on file. An empty box keeps them, typing replaces them, a tick box removes them, and a change of address or provider drops them as it drops the token. A typed Authorization replaces the bearer. The headers the HTTP client writes itself are refused. Fixed values only. The idea of extra headers comes from #58; its per-task session header is left out, because nothing plMail does needs one. Co-authored-by: shady <shady2k@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3c7a03b843 |
Talk to an OpenAI-compatible endpoint as well as Ollama
The language model had to be an Ollama container. Admin → AI now has a Provider field, and the second choice is the request format OpenAI published and nearly everything else has copied: llama.cpp, vLLM, LM Studio, LiteLLM, the hosted providers, and Ollama itself under /v1. OpenAiClient speaks three paths, /chat/completions, /embeddings and /models, and answers in the value objects and error categories OllamaClient already uses. AiAssistant reads the connection for the feature being served and picks the client; nothing above it knows there are two. The body carries only what the format defines, because several servers answer 400 to a field they do not know. The stream is read a line at a time, skips comment lines, does not require [DONE], and fails on an error object delivered inside a 200. Redirects are never followed. What only Ollama can be asked is not asked of the other dialect: no keep-alive is sent, the composer is not told to expect a cold start, Warm up refuses with its own reason, and the panel reads no /api/ps. The page puts those controls away while the other provider is chosen and keeps their values. A stored token is dropped when the provider changes, as it is when the address does. One column on ai_settings, defaulting to ollama; the migration is reversible. The SSE handling, the strict reading of an embedding and the rule that a key stays with its endpoint come from #58. Co-authored-by: shady <shady2k@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
896f30eb02 |
Import an account's history without holding up new mail
A first sync used to be the import: every IMAP folder from its oldest message to its newest, every Gmail message listed before one was fetched, every Microsoft folder enumerated to its end, all in one job on the queue new mail arrives on. A large account showed an empty Inbox for most of an hour and every other account's mail waited behind it (#42). The sync now only arranges for new mail to be seen. History is MailImporter's, a page at a time on a new `ingest_backlog` queue with a worker of its own, newest mail first and the Inbox before anything else; Spam and the bin come last. Accounts take turns, a page that fails costs itself, and every sync restarts an import that has gone quiet. - IMAP walks each folder down from the top in pages of fifty. - Gmail asks for the newest 200 of the Inbox, then pages the listing. - Microsoft reads each folder's first delta round a page at a time; a folder being read is asked only what it received since. Removals that cannot be judged while a folder is unread are kept and judged later. - Sync requests a later sync has already covered are skipped. - A chip beside the search box shows that an import is running and how far it has got. The indicator never updated on mail pages, because its controller was not mounted there. Also from #42: an account can have a separate password for sending, behind a button under the SMTP fields. One field by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
24d3c19b1d |
Choose which IMAP folders to sync (#52)
Mailbox::$isSyncEnabled already decides whether a folder is polled, held
with IDLE and counted by the remote-deletion check, but nothing let a
person change it. Settings → Accounts has a button on every IMAP
account's row that opens the account's folders, each with a switch.
Switching one off stops plMail fetching it; mail already stored stays,
and switching it back on resumes from where it stopped.
The inbox is listed with its switch disabled, and the controller refuses
it too. Gmail and Microsoft accounts sync over their provider's API and
have no folder rows, so neither route exists for them. The path carries
two ids: the account is checked for ownership, then the folder against
that account, then a per-folder CSRF token.
Review added, in the same change:
Tree the list is drawn as the tree the paths describe, with
connector lines, each folder under its nearest listed
ancestor.
Placeholders a \Noselect folder is greyed out with no switch to
press. Mailbox::$isSelectable records what the server
said; one migration adds the column.
Dialog a narrow modal preset and striped rows.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Paul Lützner <mail@pluetzner.de>
|
||
|
Paul LütznerandClaude Opus 5.5
|
71e2176fd1 |
Let the inbox be one list, without category tabs
Some people do not want their inbox divided. A third control on the sorting card, Inbox tabs, switches the tabs off per person; on by default and true on every existing row, so nothing changes by upgrading. One column, user.category_tabs. Off changes what is drawn and nothing about what is stored. Mail is still filed under a category as it arrives, so switching back shows sorted tabs at once and nothing has to be re-filed in either direction, which is also why the control queues no ResortMailbox. That includes the assistant: somebody who leaves it sorting into tabs they have switched off has decided that, and the card says so when both are true rather than preventing it. The inbox query takes a null category for the whole inbox, through one helper shared by the list and its count. ListViewResolver decides from the person's setting and not from the value the page posts, so "select all in this view" is the list the server would have drawn: an untabbed page posts an empty value, and reading that as Primary would have acted on part of a selection. What only exists for tabs goes with them: the strip and its drop targets, the per-category keys in the counts payload and the grouped reads behind them, and the panel in message details that explains a filing and takes "wrong tab" reports. The JMAP session publishes the choice as inboxTabs beside the appearance hint. Thread.category is served either way, so a client drawing tabs from it had no other way to learn not to. Closes #29 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
98ea3948f4 |
Only let mail change the calendar when it comes from where the event came from
Reported in #34, and both findings held up: eight tests written from the report's scenarios fail on the code as it was. An invitation was found by its UID and then judged by SEQUENCE and arrival time alone. Nothing asked who had sent the update, and a UID is not a secret — everyone invited holds it, and so does anyone a copy was forwarded to. Any of them could cancel the meeting or move it and swap its link, and the reader's earlier "accepted" was carried over on top. A meeting mirrored from a connected calendar was rewritable the same way, which the report did not mention. A booking read out of a mail's markup was keyed on the From line's domain, which nobody had checked, and drawn at once. One rule closes both, in EventReconciler::mayApplyUnasked(). It asks about the sender of the MAIL, never the ORGANIZER line, which a forger copies along with the rest of the file: an invitation's update from an address the event's content has already come from, or from the organiser it names. "Already come from" rather than "is the organiser" because a calendar service sends the invitation and its updates from one address of its own. a booking only when the receiving server vouches for the From domain. a From that failed nobody, the reader's own address included. A refused claim is held, not dropped: filed unapplied with a reason, shown on the message's card, and applied when the reader says so. The rule is a judgment about a sender and the reader may know better. SenderAuthentication reads the Authentication-Results header and checks nothing itself; its job is deciding whose header to believe, since a sender can write one too. mx.google.com for Gmail, the single nameless header for Microsoft, and for IMAP a server on the same registrable domain as the account's IMAP host — learning the name from the mailbox or asking for it in a setting were both considered and turned down in the issue. Mail in Spam or the bin is no longer read for events at all, and a message taken back out is read then. What it does not catch: on an account whose server reports nothing, a forged From is indistinguishable from the real sender, and holding every such update would hold every update a self-hosted mailbox receives. There the address is taken at its word. The handbook says so. Two things turned up on the way. A copy of a meeting on a second calendar has no source links of its own, so the first version of the rule held every update on the copy while its sibling took it — caught by SharedEventUpdateTest, and the senders are now asked across every copy under the UID. And there is no HTTP-level test of the new "apply" action; its logic is covered through the reconciler. One new nullable column, event_source_link.hold_reason. Nothing already on a calendar changes. Refs #34 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
09600802db |
Let a message be written once and inserted as a template
Settings → Templates is a new section: a tree on the left, one editor
beside it. The tree opens with one folder per mail account, and those
are not stored — an account folder is the account, drawn as a folder,
so the mirror cannot drift. Folders the user makes go inside an
account, inside one another or beside the accounts. Removing an
account or a folder keeps the templates in it and moves them up.
A template holds variables: the recipient's first name, name and
address, the sender's name and address, a signature, and dates with an
offset and a format of their own. They are stored as text tokens in
the body, because the body goes through the inbound-mail sanitiser and
a marked element would not survive it; the editor draws them as chips
and turns them back before it posts.
The compose window gets a button beside the signature one. Dates,
sender and signature are filled in by the server when a template is
inserted. The recipient is not: a template is usually picked before
anyone is addressed, so recipient variables arrive as markers and the
window fills them when To has somebody in it, and asks before sending
with one still open. A template's signature replaces the window's
rather than joining it, and one the template names is pinned so a
change of From leaves it alone.
TemplateTokens the notation, read the same way in PHP and in
assets/compose/template_variables.js
TemplateRenderer what each variable becomes, and who fills what
TemplateLibrary the tree, and the only writer of where a
template or folder sits
One bug fell out of running it in a browser. Chromium will not put a
caret between a non-editable element and the edge of its block, so a
template ending in a variable could not be given its full stop — the
keystrokes were dropped. The editor keeps a zero-width space beside
each chip and strips them before saving.
Two new tables, mail_template and template_folder. Web only so far:
nothing is published over JMAP yet.
Refs #26
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
ec938bca99 |
Count the sessions a public demo hands out, without keeping an address
Admin → Demo visitors, in demo mode only: sessions started in the last hour, 24 hours, 7 days and 30 days, the different visitors behind them, the total and how many have a mailbox now. The users cannot be counted because the reaper deletes them, so /demo writes a row of its own. That row holds a keyed hash of the address with its last part cut off (a /24 or a /48), never the address, and the reaper removes the hash after 30 days. The demo's privacy notice describes it. Outside demo mode the section is not in the nav, its route answers 404 and nothing is recorded; the table exists and stays empty. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
3a6f4db0f1 |
Measure mail arrival from when the provider accepted it, and show the slowest
Admin → Performance measured "how late mail arrives" from the date a message carries, which is the sender's clock. A newsletter dated half an hour before it was sent, or mail held back by greylisting, was reported as plMail being that late. The delay is now taken from the newest Received: header. Each message is stamped as it is first stored with when the provider accepted it, what started the sync that fetched it (push, IDLE, schedule, sync button) and where it was filed. Inbox mail and the rest are counted apart, since providers only announce the inbox, and the slowest messages of the period are listed in a section that starts collapsed. An upgrade task gives the last week of mail its accept time from the stored headers. What fetched older mail cannot be recovered and shows as a dash. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
2dfc9b3b9b |
Give the admin panel a page for how long things take
The line under the hold switch in Admin → AI said "typically 6 s, 95%
within 13 s": enough to see that one message took twice as long as the
rest, and nothing about why. System says whether plMail is working;
nothing said how long anything took.
Admin → Performance, directly below System:
How late mail arrives per account, provider-received to stored.
Seconds is push working; minutes is the
quarter-hourly poll, which "works" and so
showed up nowhere. Imports and plMail's own
sent copies are not arrivals.
What each worker has the queue numbers by the process that owns
waiting them, a worker's queues in the order it
looks at them. Live; the transport keeps no
history.
New mail waiting for one row per held message, its wait split
the assistant three ways: queued, the model call, and how
much of the call was the model loading.
Model host moved from Admin → AI, unchanged.
To split a wait, the three parts have to be recorded with the message:
ai_queue_ms, ai_call_ms, ai_load_ms. ai_call_metric has the same
numbers and no idea which message they were for, and its timestamps are
the database's clock where the message's are PHP's, so a join by time
is not one to rely on. AiAssistant::ask() returns the whole result for
that; chat() is ask()?->content.
The model host's card and the search index backfill were one fragment
polled by one endpoint. They are about different things — a measurement
and a job with Start and Pause on it — so the fragment renders either
half, each page names the one it wants, and a caller that names neither
gets both, as before.
BackgroundProcesses::consumers() is the worker-to-queue map, read by
all() to start the processes and by this page to label them. They were
two facts that had to agree.
Rows name accounts. Nothing about any mail's content is shown.
Not looked at in a browser; the E2E card-inset spec covers the section
and has been given its frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
88b1744385 |
Leave the ingest worker to fetch mail, and nothing else
Adding an account put new mail behind hours of follow-up work. What
plMail does to a message once it has it — a model call where the
assistant sorts, event extraction, insight cards, receipts — was queued
on `ingest`, beside the syncs, for every message of the import, oldest
first. One consumer serves that queue. 988 envelopes pending and the
oldest an hour old was a mailbox being asked about newsletters from
2019 while mail on the server waited to be fetched.
`ingest` is now what talks to a mail provider. The rest moved:
enrich_live worker-live follow-up work for mail that has just
arrived on an account whose import is
over; never behind an import
enrich worker-enrich the same for an import's recent mail,
and search indexing
enrich_backlog worker-enrich old mail for the assistant, consumed
only when `enrich` is empty
release worker-release ends a hold (below); nothing else
Which queue is decided per batch, not per message class: the pipeline
asks InitialImportState once, carries the answer on PostIngestResult,
and EnrichmentRouter turns it into a stamp.
Only mail from the last 30 days (ENRICH_RECENT_DAYS) gets follow-up
work on arrival. Older mail is stored, threaded and filed by the rules.
The assistant is asked about it afterwards — newest first, a bounded
handful per run, and not until every account of that person has
finished importing (app:ai:classify-backlog, ClassificationCatchUp).
For the other steps `app:backfill` remains the way to read history.
ExtractInsightsStep and ProposeEventsStep ran their work inside the
sync and argued for it; both dispatch now.
New mail no longer appears in one tab and moves to another. For
somebody sorting by assistant the rules' category is a guess about to
be replaced, so where the verdict could still change it the message is
held: stamped before threading, its new thread given no category, which
every tab query already reads as "in no tab". The answer releases it.
Failing that a timed release does, after AiSettings::holdMaxSeconds —
60 by default — and the mail is shown where the rules put it; the model
call carries on and moves the mail when it answers. A message is asked
about once. Admin → AI has the switch, the ceiling, and the measured
delay over the last week beside them.
Three things about the release that were each wrong once first:
- It ran on the live worker, so it could not fire while that worker
was inside the call that was taking too long. It was moved to
export, which mixed mail appearing with mail leaving. It has a
queue and a worker of its own.
- The call was cut off at the ceiling and the message asked about
again later. The hold and the question are separate; only the hold
has a deadline.
- A late verdict was stored and moved nothing. It moves the mail.
Both workers still end at one model host, so bulk classification looks
at the live queue before each call and stands aside (LiveMailPriority).
One migration: two columns on `message`, two on `ai_settings`.
Not verified against a real import or a live model host: the tests pin
the queues, the hold and the release against in-memory transports and a
mocked host. Known and left: IMAP still imports oldest first; a reply
into an existing thread cannot be hidden; a Gmail account counts as
importing for up to an hour after its last message lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
f0380c47fa |
Make the wax seal the icon an account starts on
The default icon was the pl mark. It is the wax seal now, in its own
design: the mark pressed into wax is still recognisably plMail, and it
is a picture rather than two letters, which is what an icon on a
catalogue page or a home screen has to be.
Three things move together:
LogoMotif::DEFAULT the seal, for new accounts and for pages
with nobody signed in
Appearance::$logoOriginal true by default, so the seal arrives the
way a picked motif does and not in the
theme's colourway
public/icons/ favicon.svg and the four PNGs redrawn as
the seal on its tile
A migration moves the column defaults and every account that never
chose an icon: the pl mark, still in berry, still linked to the theme.
That cannot tell an untouched account from one that deliberately stayed
on the mark; those get the seal too and can pick the mark again.
The static files are drawn by hand from the motif's geometry, so a test
now holds favicon.svg to the default icon's own colours.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Paul LütznerandClaude Opus 5.5
|
6edc6c324c |
fix(deploy): the cache table exists before any worker reads it
A fresh install logged a warning about once a second from the worker, until an administrator happened to press "Restart workers": Failed to fetch key "workers.restart_requested_timestamp": ... relation "cache_items" does not exist The restart signal and the notice throttle live in cache pools on Symfony's DBAL adapter, which creates cache_items on its first write. Every Messenger consumer reads the signal on every loop, long before anything writes it, and each failed read is logged. The migrations leave the table out on purpose, because the adapter owns its DDL. app:db:migrate now has each DBAL adapter create its table when it is missing, after the migrations and inside the advisory lock, so php and the worker cannot race each other to it. The adapter still owns the DDL: it names its table through configureSchema() and creates it with createTable(). Best effort, like the statement statistics: a failure is a warning, not a container that will not start. Existing installs get the table on their next boot. The adapters are collected by class in a compiler pass rather than injected by pool id. In dev and test the profiler wraps every pool in a TraceableAdapter, which has no createTable(). It is the lookup DoctrineBundle uses for its schema listener, and it covers any DBAL pool added later. Checked on two fresh compose.yaml + compose.demo.yaml stacks: 95 warnings in 75 seconds without this, none with it, and Restart workers still reaches every consumer through the table created at boot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
4cbdb4f511 |
feat(admin): a version checker, with a channel to follow
Admin → Updates: an administrator picks which builds count as an update (Releases, the `latest` image; Main, the `main` image; or Off), and every hour app:updates:check asks whether that channel has a newer one. Until somebody picks, an installation follows the tag its own build came from, so a `main` image follows main and an unbuilt checkout asks nothing of anyone. The registry decides what exists: the channel's image is read the way `docker pull` finds it (OCI distribution API, anonymous token from the 401's challenge, the index entry for this architecture, the config's labels), so a commit that builds no image is never announced. GitHub's compare decides which way two builds differ, so a main build following releases is not told to go back to an older release, and its commit list is the page's "what's new". An image from anywhere but GitHub falls back to what the version numbers alone can say. Each new build is announced once: a push to every administrator's registered devices (a new UpdateAvailable payload the service worker shows), a badge beside the build chip in the admin header, and a dot on the nav entry. A check that gets no answer keeps the last one and says why, and the hourly command exits 0 for it: an installation behind a firewall is not a fault. Images are stamped with where they are published (APP_IMAGE), so a fork that publishes its own checks its own. The channel travels in config backups; what the last check found does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
24d7bdfc07 |
feat(appearance): the logo is an icon, then its paint
Appearance gains logoMotif (default pl) and logoOriginal (default false), with Version20260923223000 adding both columns so every account keeps the mark it has. effectiveLogoPaint() says what the chosen icon wears: for the pl mark it is effectiveLogoStyle(), unchanged; a motif wears its own design while logoOriginal holds, then the theme's namesake colourway while linked and the chosen one when not. A linked motif on a classic theme falls back to its original rather than to Berry. Export, import and the settings save carry both fields; an unknown motif keeps the current one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
cf7b971d0d |
perf(db): index message_thread for badges, list order and snooze sweep
The dozen sidebar/list aggregates per page and the per-minute snooze sweep had only account_id to work with. Adds partial indexes for unread, starred and snoozed threads and (account_id, last_message_at DESC, id DESC) for the list ORDER BY. Applied against the test database in a rolled-back transaction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
12842bb3be |
fix(imap): index unread rows per folder for the post-sync recount
Every folder sync ends with countUnseenForMailbox(), `mailbox_id = ? AND seen_at IS NULL`, which visited every row of the folder to test seen_at. A partial index on message(mailbox_id) WHERE seen_at IS NULL holds only the unread rows. Migration-only, like the table's other partial indexes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
b9238ab019 |
fix(imap): stop re-probing genuine multi-folder copies every sync
repairRelocated() probed every row whose Message-ID was also filed in another folder. A message that genuinely lives in two folders survived every probe, collapse() returned 0, and it was back on the work list next sync, costing a round trip per copy per poll and filling the 100-row slice so real ghosts behind it waited. Confirmed copies now get message.copies_confirmed_at (new migration) and are left out of findMessageIdsAlsoFiledElsewhere() for 30 days. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
07b6724e8c |
fix(imap): a message that fails to build is retried instead of lost
The catch around buildMessage() logged and moved on without holding the high-water mark, so lastSeenUid passed the failed UID and it was never fetched again. The clamp that did exist for empty fetches was per batch and got overwritten by the next batch's higher UIDs. Now the lowest skipped UID is tracked for the whole run, both failure paths hold the mark below it, and a per-mailbox counter (mailbox.failed_uid / failed_uid_attempts, new migration) lets one message that always fails go after five syncs so it cannot block the folder. A present but unparseable Date header made webklex throw while building the message, failing the fetch for the whole folder. The client now configures a fallback date (the epoch), and MessageSyncer::dateFromHeader() reads that, or any throw, as "no date" and uses the arrival time. Attachment parts are persisted only after the message is fully built; before, a second attachment that threw left the first part managed and pointing at an unpersisted message, and the batch flush failed on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
63b236b134 |
fix(imap): a folder missing from one LIST no longer deletes its mail
MailboxSyncer removed every stored folder the server's LIST did not name, and message.mailbox_id cascades, so an empty or partial LIST, or a rename done in another client, deleted every message of those folders in one poll. Now an empty listing is refused outright, a listing that would leave out most of the stored folders is refused like VanishedMessageReconciler's bulk guard, and a folder that is really gone is only marked (mailbox.missing_since and missing_syncs, new migration) and skipped by the sync loop. It is removed only after three consecutive listings and 24 hours, and its messages go through MessageEraser so JMAP clients get a destroyed change for each. A new folder with the same special use or the same UIDVALIDITY as exactly one missing folder is treated as a rename: the row keeps its mail and takes the new path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
7a7673c0e8 |
perf(jmap): Email/query pages in SQL instead of reading the mailbox
Every call read and sorted every matching message of the account for PHP to slice a page from, and received_at had no index. OFFSET/LIMIT now go into the query; a collapsed list is collapsed first with DISTINCT ON so position still counts threads; total is a separate COUNT, skipped when the client sends calculateTotal: false. New index on (account_id, received_at DESC NULLS LAST, id DESC) for the default sort. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5.5
|
bdcce85acb |
fix(jmap): state tokens follow commit order per account
The change log's identity value is the state token and is drawn at INSERT, so two writers of one account could commit out of order and a client holding the later token never saw the earlier change. A BEFORE INSERT trigger now takes a per-account advisory xact lock and draws the number after it. A trigger because Doctrine fires no event inside its flush transaction before the inserts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
5a620f4be7 |
Run one-time repairs once per installation, and repair the lost bodies
The previous commit stops bodies being filed as attachments. It does nothing for the mail already in the mailbox, which stays blank — and there was nowhere for that kind of work to live. WHY NOT THE FOUR PLACES IT COULD HAVE GONE. A migration runs exactly once on update and is the right home for schema, but this repair needs the attachment store and the body sanitizer, and a migration reaching into the container breaks the day one of them moves — halfway through an update, with the schema half applied. The entrypoint is downtime: it runs before the server accepts a request, so a walk over every message put there means an install that updates at eight in the evening is unreachable until it finishes. BackfillTaskInterface is deliberately manual, which is right for "regenerate the categories because the classifier changed" and wrong for damage nobody should read a changelog to discover. BackgroundJob is per-user, and this is not owned by a user. So: a ledger, in the same shape as Doctrine's own migration_versions and readable the same way when the application will not start. A task claims its row BEFORE running, so a container killed mid-walk leaves evidence rather than looking like one that never started; three attempts and it stops rather than taking the worker down every ten minutes forever. The runner re-reads the row afterwards because tasks clear the EntityManager between batches and the claimed instance is detached by then — writing through it would flush nothing and the task would run again for ever. Scheduled every ten minutes, and what runs every ten minutes is the QUESTION, not the work: one indexed lookup per registered task against a table with single-digit rows. What it buys is the gap between an update landing and the repair happening. The repair is the first task and implements both interfaces — automatic for everybody, and app:backfill misfiled-bodies for anyone who wants it now. It needs no mail server: persistAttachment() stored the bytes on the way past, so the body has been on disk the whole time it was missing from the screen. It does lose the declared charset, which MessagePart does not keep, so CharsetHelper works it out from the bytes — exact for UTF-8 and a reasoned guess otherwise, against a round trip per message to a server with nothing else to say. EXISTS rather than a join in the lookup, and not as a preference: a multipart/alternative can have both halves misfiled, and SELECT DISTINCT cannot run at all here — Message carries json columns and Postgres has no equality operator for json. The repository test caught that on its first run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
fb97a5a6b4 |
Recognise Google's weekly Testing expiry, and warn before the next one
A Gmail account that stops every few days is almost always one thing: Google expires refresh tokens issued by an OAuth app whose consent screen is still in "Testing" publishing status, after about a week. That is the state nearly every self-hosted install is in, because leaving it there is the documented way to avoid Google's verification review for scopes of Gmail's reach. So the account works, stops seven days later, gets reconnected, and stops again. Forever. Google gives no clue which case you are in. What arrives is `invalid_grant` — the same code it returns for a revoked consent, a changed password, and a token that aged out. Four causes, one error, and only one of them is fixed by changing a setting rather than by signing in again every week. THE ONLY THING THAT SEPARATES THEM IS THE INTERVAL, and nothing was recording it. Two columns now do: when the current grant was issued, and how many hours the previous one survived. The second is kept ACROSS the reconnect on purpose — it is the only evidence that survives one, and the moment somebody has just fixed it is the one moment they are looking at the account and could act on the cause. Two cards come out of it. A sign-in that died after about a week gets one naming that cause and the real fix instead of the generic "sign in again" for the fifth time. And once the pattern has been seen once, a warning appears a day or two BEFORE the next one runs out, while nothing is broken yet. The inference needs both halves and says so. Without the history it would nag every healthy Google account weekly; without the age it would fire the moment somebody reconnected, and a warning visible five days out of seven is furniture. The window is 4 to 9½ days: wide enough for a death noticed a day late on the next sync, narrow enough to exclude a grant that ran for three months and was then revoked — which is the claim that must never be made, and is pinned in WeeklyGrantExpiryTest in both directions. An account connected before this was recorded is told nothing at all. Verified on a running stack in all three states: the warning on a live six-day grant with a weekly history, the diagnostic card on a dead one, and the plain card with no mention of Testing on a grant that lasted ninety days. The last found a real bug — the check was wired into inspectAccount(), which the settings page never calls, so the card rendered nowhere. It also must not register as a CAUSE, or every calendar on the account would grey out and blame a card whose first sentence says mail is still arriving. The handbook said "publishing and going through verification is what removes that", which conflates two separate settings and is what makes people accept the weekly reconnect as the price of not being verified. Publishing status and verification status are different fields; an app published without verification works, with a warning screen the first time. Corrected in English and German, with a new section in features/health.md describing both cards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
d53e7c4963 |
Report browser errors to the admin panel, in a card of their own
A JavaScript fault was visible only to whoever had the console open, which on a self-hosted mailbox means nobody. Uncaught exceptions and rejected promises from plMail's own scripts now reach Admin → Logs, together with whatever the Content-Security-Policy blocked. Its own card rather than rows in the log, because it holds a different kind of thing. A server error is one event in one request; a broken line in a controller runs on every page load for every user. So a row is a distinct fault — message, file, line — with a count and a lifetime, and the same fault reported again increments it. Merged into the log, one bug would be four hundred rows and the log would be unreadable past it. THE FILTER IS THE FEATURE. Most of what window.onerror sees is not ours: the console screenshot that prompted this was an ad-attribution metrics script failing inside requestIdleCallback, injected by an extension. A panel that accepted that fills with other people's bugs and stops being read, which is the same as not having one. A report is kept only when it can be attributed to a script this server sent — an injected script has no URL (Chrome's `VM947`), and a cross-origin one arrives as the bare string `Script error.` with no file, line or message. The rule is applied in the browser and again in the recorder, because the browser half is part of the code being reported on. CSP violations come to the same card through /csp-report. That endpoint cannot be CSRF-protected and is public in security.yaml, because the browser generates the request itself: no token, no header we chose, and on a login page no session. The rate limiter is the bound. It is worth having because the full policy is enforced in production, so what arrives is what this installation actually refused to load in a real browser. The count means page loads, not executions — the client reports each fault once per page, capped at five. Measured rather than assumed: throwing the same error twice on one page records one sighting. Documented that way on both sides after the first version claimed the server would count what the client never sent. `column` is a reserved word in Postgres, so the field is `column_number`. The quoted DDL created the table happily and every insert into it was a syntax error — same trap User::$usr already avoids. Verified on a running stack by throwing real errors in a real browser: an error from a script on this origin recorded, an anonymous injected one refused, the card rendering both kinds grouped with counts. Its first catch was a false alarm, and a useful one to record. It reported the Mercure connection as blocked by `connect-src` — which was true, and was me browsing `localhost` while MERCURE_PUBLIC_URL names `127.0.0.1`. Different origins, so the app's own policy refused its own hub. Browsing the host the config actually names produces no violation at all. The compose comment calling that URL "same-origin" is correct on its own terms; it just stops being true if you reach the stack by another name. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Claude
|
044aa01846 |
Reported mail is one feature: one list, one triage, one export you choose
Two kinds of report — a mail an extractor should have understood and did not, and a mail somebody found in the wrong tab — were two tables behind two panels with two export buttons. That split made sense to whoever wrote them and to nobody using them: an admin opening this section is asking "what is still mine to do", and that question does not have two answers. The tables stay apart, because the evidence really is different: a wrong tab is a disagreement between three classifiers, a missed insight is a mail body a parser could not read, and making them inherit would put a dozen always-null columns on each row and tie a schema change on one to the other for ever. What is unified is everything above them. ReportedMail carries the handful of fields a triage list is built from and the entity alongside for the fold and the export; ReportedMailProvider is the only place that knows there are two tables; the panel, the buttons and the file are written against the pair. Category reports gained handledAt, which is what makes them belong in the same list — a worklist you cannot cross anything off is not a worklist. Before this the only way to stop looking at one was to delete every one of them, taking the evidence for a rule away at the moment somebody might want to check it again. The export's scope becomes the selection. "Everything, or only the unhandled part" answers the question an admin has on their first visit and none of the ones they have afterwards; the one that comes up is "these six, the ones about the shop". So the list filters — by status, and by the problem itself, with the chips derived from the rows because for a wrong tab the problem IS a pair of categories and only this install knows which pairs it has produced — and the export takes what is ticked. Filtering never unticks anything: a row hidden by a chip is out of the selection while it is hidden and comes back ticked, so a selection built across two chips survives. An empty post still means everything, which is what a browser without JavaScript sends. Each record now carries its kind, every field its kind has, and the same one-line rendering the panel shows and the copy button copies. The document is plmail.reported-mail version 2 — a new name rather than a bumped version alone, so a consumer written against version 1 fails on the name instead of half-reading a file whose records it only understands some of. The row is a <div> and not a <label>, though a label around the whole row would give the checkbox a better hit target: a label may not contain a <form> or a <details> and both are there. Chromium happens not to forward label activation from a nested button, which means the invalid version works right up until the row is read by something that is not Chromium. Verified on a running stack: six reports of both kinds in one list, the status chip hiding the handled one, select-all writing five keys into the export form, a problem chip narrowing that to one without losing the other four, and the export returning scope:selection with the full record and its line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Claude
|
d42012cedf |
Report the category the tabs actually sort by, and the evidence behind it
The first real miscategorisation report read `filed:updates should:updates` about a conversation its owner was reading in Primary. Both halves of that were wrong in the same way: `filed` was `message.category`, and the mailbox lists threads and filters the tabs on `thread.category`. A thread takes one category from its newest message and a pin overrides it outright, so on any conversation with more than one message the two are free to differ — and it is the thread that decides which tab somebody is looking at. So the report described a column nobody can see, and the popover's "category" line did too. Both now read the thread, falling back to the message for one not yet threaded, and both say so when the two disagree: the panel explains that the conversation takes its category from its newest message, which is the difference between a panel that looks broken and one that is informative. The report row was also too thin to act on. It carried the four verdicts and none of the inputs they came from, so a line could be entirely true and still leave nobody able to say what to change. Added: the message's own category when its thread disagrees, whether a pin is holding it, whether the reader had asked for the provider to be overruled, whether the model was asked at all (`model:-` could not tell "never asked" from "asked, no answer"), whether there was any plain text for it to read, and List-Id's value — the one bulk header whose value is a rule-writing input rather than a tracking link. The card moves from Admin → AI into Reported mail, where somebody who has just pressed Report goes looking for it, and is grouped by `filed → should` rather than listed by date: twenty reports are rarely twenty problems, and the fix for one group is not the fix for another. Each group and each row can be selected, and the selection can be copied or downloaded as a text file — so whoever is changing one rule can take the reports about that rule and leave the rest. A report that agrees with where the mail already is now 409s rather than landing as a row saying nothing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
fabaf49a80 |
Let a reader report mail as filed in the wrong tab
Categorisation is decided by three things that disagree — a provider's labels, a header cascade, and a model — and whether the answer is any good is a judgement only the owner of the mailbox can make. Everything else in this feature can be measured against fixtures somebody invented. This is the one route by which real mail, wrongly filed, becomes something a rule or a prompt can be changed on the strength of. The control sits at the bottom of the header popover, where the three answers already are. A SELECT AND A BUTTON rather than five buttons: knowing a message was misfiled is a fraction as useful as knowing where it belonged — "promotions, not primary" is a rule somebody can act on, "wrong" is a shrug. The tab it is already in is not offered, because "it should be where it is" is not a report. It records the DECISION and its evidence, not the message: what each of the three said, what the rules matched on, which of them was deciding, which bulk headers were present, the sender and the subject. Not the body — the most sensitive part of a message and the least useful here, since most marketing mail has no plain-text part at all and the classifier sees "(no plain text part)", so the column would be empty for exactly the mail most often reported. Sender and subject ARE kept, and on a multi-user installation an administrator reading these sees them for other people's mail. That is a real cost, taken deliberately: a report saying only "something was wrong" is not worth the button, and it is why nothing is recorded without somebody pressing it. The reports collect in Admin → AI rather than a nav entry of their own, because that is where somebody ACTS on them — the prompt and the model are a card above. The product is the TEXT: one line per report, built in the entity rather than the template because a format assembled in Twig would change whenever the table's layout did. Copy takes the block; clearing is a deliberate press, since these are observations nothing regenerates and a pattern across twenty of them over two months is the whole value. No foreign key to `message`: a report outlives the mail it is about. Somebody deletes the message, the pattern it was evidence of does not go away. Verified on the stack: the control renders, a report returns 204, a hand-edited category is refused with 400, the line appears in the admin card, and clearing empties it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
ed4a483ac3 |
Let each person choose what sorts their mail into tabs
Two decisions, and they were both taken for everybody by a constant. WHAT DECIDES. The rules read headers — a mailing-list header, an unsubscribe link, a sender you have written to — and answer the same way every time for the same mail, on any installation, with nothing switched on. The assistant reads the message and decides what it is for: better at mail that does not announce itself, and occasionally confidently wrong. Which is preferable is genuinely a matter of taste, which is why it is now a setting rather than something argued about once. Choosing the assistant puts its verdict ABOVE the header cascade — which is what choosing it has to mean. Read only as a tie-break, a newsletter carrying List-Unsubscribe is Promotions whatever the model made of it. Below the correspondent rule, though, and that placement is the one to keep: "you have written to this person" is a fact about the reader rather than a guess about the mail, and a model above it files a colleague under Promotions on one bad guess — the only mistake in this cascade anybody would actually notice. WHETHER IT OUTRANKS GMAIL. A Gmail account arrives already sorted, and plMail has always treated those labels as authoritative — its own rules never even ran. That is the right default and it is also exactly what somebody who dislikes Google's sorting could not escape, on the account where they most wanted to. Per person, not per installation, for AiPreferences' reason: an administrator decides what is available and a reader decides what happens to their own mail. Its own embeddable rather than two more fields in the AI bag, because only half of it is about the assistant — the provider half works on an installation with no model at all, and folding it in would file a setting where nobody would look for it. ONE BEHAVIOUR CHANGE, NAMED RATHER THAN DISCOVERED. The model's verdict used to be consulted as a silent tie-break: no setting turned it on, none turned it off, and nothing on screen said it had happened. It is now what Assistant means, and Rules means what it says — so an installation with categorisation on loses that tie-break until somebody chooses. The alternative is a model quietly sorting mail for people who never asked, which is the arrangement this ends. Said in the migration, in the handbook, and in an upgrade note. Nothing is re-sorted. The category is recomputed from stored data, so a mailbox keeps what it has until `app:backfill category` runs — an operator's call, not a settings toggle's, and the card says so. Verified on the stack rather than asserted: the card renders, the assistant option is disabled with its reason when categorisation is off, and a posted change lands in the column. The four combinations of the two settings pick three different categories for one fixture, which is what makes them worth having. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
Paul LütznerandClaude Opus 5
|
2bfc30ad8d |
Offer to summarise the whole conversation, and send a context window with it
The card now says when a summary was written from part of a thread, and the way out sits beside the notice that explains why it is there. num_ctx IS THE WHOLE MECHANISM. This client has never sent one — four docblocks across the codebase size their budgets around "OllamaClient sends no num_ctx, so the model's default decides what survives", and that default is 4096 tokens with everything past it dropped silently. So "send the whole conversation" without also asking for somewhere to put it would send more, have it truncated at a point nobody chose, and spend a much longer wait to produce no more than before. It is sent only when a caller asks, so every existing call keeps its shape. FULL IS STILL BOUNDED, and that is the honest part rather than a caveat. A context window is memory: Ollama allocates the KV cache for the whole of it whether the prompt fills it or not, so an unbounded "send everything" lets one absurd thread ask a modest host for an allocation it cannot make — and a failed allocation costs the reader the entire wait and produces nothing, where a clipped transcript produces a summary and a sentence saying it is partial. The ceiling is 96,000 characters against a 32,768-token window, twelve times the ordinary budget. Past it the run is full AND partial, and the button withdraws rather than inviting the same wait twice. The window is SIZED to the transcript rather than fixed, since it is reserved whole either way and a two-message thread should not cost what a hundred-message one does. Sized at 3.0 characters per token, deliberately below the 3.55 measured on real German business mail: this divides to get tokens, so erring low overestimates the need and asks for slightly more room than strictly required. Erring the other way would silently truncate inside the one feature whose entire purpose is not to. thread_summary gains full_context, and it cannot be derived like the rest of the card. "Is this thread too long to send whole" is a question about the thread and comes free from the transcript the freshness check already builds. "Did THIS summary see all of it" is a question about the row, and the thread cannot answer it — a thread that is still too long is still too long after somebody presses the button and gets a summary that did see everything. Without the column that press would appear to do nothing: new text, same notice, much longer wait. A press and not a setting, deliberately. It is a fair thing to spend once on a thread somebody cares about and a poor default for every summary in a mailbox, so the cost is written into the label and nothing is remembered. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |