Commit Graph
118 Commits
Author SHA1 Message Date
Paul LütznerandClaude Opus 5.5 040986e344 docs: German client guide carries both of today's changes; hash
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:38:08 +02:00
Paul LütznerandClaude Opus 5.5 1aadc53218 fix(security): prune finished jobs and expired trusted devices nightly
Both tables grew forever: TrustedDeviceRepository::pruneExpired() and
BackgroundJobRepository::findFinishedBefore() had no caller. The nightly
app:monitoring:prune now deletes expired trusted devices and background
jobs finished more than 30 days ago (--job-days). The job lookup became a
bulk DELETE, since hydrating would scale with the history and trip over any
row whose kind the enum no longer lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:37:29 +02:00
Paul LütznerandClaude Opus 5.5 ebeaf8cffa fix(security): add a per-username login limit
Symfony's default login throttle counts per username+IP and per IP, so a
guesser on many addresses got five tries per address at the same account,
while the comment in security.yaml claimed it was per username. A custom
LoginRateLimiter keeps both and adds a username-only limit (20 per 15
minutes, set above what a mistyping person reaches since it can also be
spent by someone else). Comment and handbook corrected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:37:15 +02:00
Paul LütznerandClaude Opus 5.5 dee4257d3f fix(security): check every hop of user-supplied URLs against private networks
The Nextcloud, Paperless and Immich drivers followed up to twenty redirects
unexamined, the URL validator let any hostname through without resolving it
(container names, nip.io, 127.1, 2130706433, mapped IPv6), and push endpoints
were not checked at all. Requests to user-supplied URLs (the three drivers,
CalDAV, ICS feeds, Web Push) now go through UserUrlHttpClient, which wraps
NoPrivateNetworkHttpClient and re-checks every hop. The validator resolves
the host. LAN integrations keep working through the existing
INTEGRATIONS_ALLOWED_HOSTS and admin-pinned server addresses; push endpoints
get their own PUSH_ALLOWED_HOSTS for the bundled ntfy, and never redirect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:37:15 +02:00
Paul LütznerandClaude Opus 5.5 c1a6d12556 fix(jmap): the change log is pruned to sixty days
jmap_change_log grew for the life of the install; pruneOlderThan() had
no caller. app:jmap:prune-changes now runs nightly from the maintenance
schedule. It keeps the newest row of every account and object type so a
quiet type's state token never falls back to 0; clients below the new
floor already get cannotCalculateChanges and resync.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:12:24 +02:00
Paul LütznerandClaude Opus 5.5 9b5d795aab docs(jmap): German client guide follows the calculateTotal change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:12:24 +02:00
Paul LütznerandClaude Opus 5.5 7a7673c0e8 perf(jmap): Email/query pages in SQL instead of reading the mailbox
Every call read and sorted every matching message of the account for PHP
to slice a page from, and received_at had no index. OFFSET/LIMIT now go
into the query; a collapsed list is collapsed first with DISTINCT ON so
position still counts threads; total is a separate COUNT, skipped when
the client sends calculateTotal: false. New index on
(account_id, received_at DESC NULLS LAST, id DESC) for the default sort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:12:24 +02:00
Paul LütznerandClaude Opus 5.5 8289891830 feat: failed requests raise a toast instead of breaking the page
A failed Turbo response used to be rendered: the error page drawn as
the page (or merged under it — a picked PDF avatar left plMail's 500
hanging off the settings screen), or "Content missing" in a frame.

- request_errors.js catches every Turbo visit, form and frame. A failed
  response is not rendered; the page stays as it was and an error toast
  says what happened. 422 (a form with its errors), 401/403 (the
  session guard) and elements marked data-turbo-errors="self" (the
  compose dock) are left alone. A network failure gets its own words;
  a frame response of the wrong shape is visited as a page.
- requestFailed() gives controllers' own fetch() calls the same toast.
  An audit of the 60 calls fixed the 22 files whose user-initiated
  requests failed silently (star, archive, labels, drag and drop, bulk
  actions, reorders, appearance save, undo/cancel send, and more), and
  left background polling and preferences quiet on purpose.
- Every request gets an 8-hex id: X-Request-Id on the response, `id`
  on its log entries, "Ref. …" in the toast with a copy button, and a
  Reference filter in Admin → Logs. A proxy's own id is kept.
- toast.js clones the server's toast partial, so a script's toast is the
  same pane and theme as a streamed one. The sync button uses it, and its
  hardcoded English is now translated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 18:22:21 +02:00
Paul LütznerandClaude Opus 5.5 3cfcf5e5c9 fix: config backups carry the clock placement; handbook hash
The clock-placement setting (v0.2.39) was neither in a config backup
nor excluded on the record, so a restore quietly dropped it back to the
default. It rides with the clock format now. The German appearance page
was updated with the English one; its source hash was not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 17:51:00 +02:00
Paul LütznerandClaude Opus 5.5 85ef0e9ee4 fix(e2e): capture inbox-dark in a dark theme
The screenshot emulated prefers-color-scheme: dark, which the server-side
theme never sees, so the README's dark-theme picture was light. Set the
user's appearance to the dark theme for that capture through the settings
route, and post the exported appearance back afterwards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:45:51 +02:00
Paul LütznerandClaude Opus 5.5 e9c5f2940b Release v0.2.39
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:11:23 +02:00
Paul LütznerandClaude Opus 5.5 881b7026b3 feat: a running clock, in the topbar, under the name or in the sidebar
Settings → General → Clock gains "Show the time":

- On the Happening Soon button (default): the button shows the time on
  every page, falling back to a clock icon when nothing is coming up,
  and the panel it opens is headed by the time and today's date.
- Under the plMail name: weekday and time as a second line.
- At the bottom of the sidebar: a larger time, the date and the next
  thing coming up, which opens Happening Soon.
- Nowhere: the topbar as before.

The server prints the first reading in the user's zone and format;
ui--clock advances it on each minute boundary and when the tab becomes
visible, and rewrites date text only when the day changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 14:06:43 +02:00
Paul LütznerandClaude Opus 5 8dc22b49c0 feat(search): show which words a result matched on
A row asserted a match and then showed something else. The preview was the
opening hundred characters of the body, which for most hits does not contain the
search term at all, and nothing was marked — so the only way to learn why a
conversation was in the list was to open it.

Every occurrence is marked now, in the subject and in the preview, and the
preview is a window of the message AROUND the match. The fragment comes from the
same Postgres `ts_headline` the JMAP snippet has always used, over the same query
that found the mail, so the marking is stemmed exactly as the search is and
cannot mark something the search did not match on.

**It also closes a hole that was open in the JMAP API.** `ts_headline` is a
highlighter and not an escaper: it inserts the delimiters it is given and returns
the rest of the document byte for byte. Asking it for `StartSel=<mark>` produced
a string in which the sender's markup and ours were the same kind of thing, and
the check that stood in for escaping — "does it contain `<mark>`" — could not
tell them apart. A subject or text part carrying `<img src=x onerror=…>` shipped
it to every client as snippet HTML, and per RFC 8621 a client is right to render
that. The class docblock claimed the opposite and had done since it was written.

The delimiters are two control characters now. The whole string is escaped and
only then are they swapped, pairs only, in one class that owns both the options
and the conversion so they cannot be changed apart. `toHtml()` answers JMAP;
`toMarkup()` answers Twig, which is what lets the row print a fragment without
`|raw` — that file still does not contain one.

Three things worth knowing:

- `ShortWord` went from 3 to 0. At 3 `ts_headline` trims short leading words, so
  "Oak for the alcove" searched for `alcove` came back as the bare word — a row
  that has rewritten the subject rather than highlighted it.
- A highlighted subject is used only when it still equals the thread's subject
  with the markers removed; anything else falls back. And it goes to a separate
  variable, because `subject` is also spent on `aria-label`, where a screen
  reader announcing the markup would be a worse regression than the one this
  fixes.
- The headline replaces the preview only when it actually contains a marker. That
  makes the two documented `body_text` regressions self-sorting: a bulk sender's
  "this email is only available in HTML" does not contain the term, so it does
  not win.

One batched query per account per page, not one per row: the budget test is a
hard ceiling and a per-row headline would blow it instantly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 17:18:08 +02:00
Paul LütznerandClaude Opus 5 9835756aec Make the calendar hold together at every width
Seven columns used to divide whatever width there was, so a phone — or a narrow
docked pane — gave each of them about fifty pixels, and a block showed its
colour and little else. A column has a floor now: below the width where seven
of them fit the grid pans sideways under an hour gutter that stays pinned, and
four days are legible where none were. The time-grid shell's own header costed
that fix as disproportionate once — "three scroll containers kept in step by
JavaScript" — and it is not, because the three grids already share one
scroller; all that was left was making each one's first cell sticky.

A BLOCK IS TWO ROWS NOW. The time on one line with the calendar's colour as a
bar down the left, the title under it, which is a whole title in a 96px column
where one line was four characters of each. Day view keeps the single line and
is told so rather than measured for it: a day column is the whole grid and is
never the width that makes stacking worth doing.

The agenda's date rail is gone. A day is a heading with its events under it,
and the times sit in a lane that is a grid track rather than a fixed guess —
sized against every time in the view, so it is 43px where an "all day" needs it
and 33px where nothing does. The month gives up its words entirely below 5.5rem
of cell and draws one bar per entry, which is what a phone can honestly say in
forty pixels, and "N more" is gone along with a count that a variable cell
height could never make true; the date is the link into the day instead.

The grid opens on now rather than on a fixed 07:00, and a chip too small for
its own title opens to it while the pointer rests on it. Width only: growing
the height made the chip inside stop following its parent, and a click aimed at
the middle of a Day-view block landed under the button and opened nothing.

Two bugs fell out of the work. An all-day entry sorted below the morning east
of UTC, because its start is a floating midnight in UTC while a timed one's is
an instant — the time grid never showed it, since it puts all-day entries in a
band of their own and never asks. And a grid block had never had a background
at all: an inline colour was overriding the surface class written beside it, so
the only paint it carried was a tenth of a calendar colour over whatever
happened to be behind it.

Opening on now has one consequence worth naming: a fixture at 10:00 is on
screen all morning and scrolled above the viewport by the afternoon, and
`boundingBox()` answers for an off-screen element with coordinates that land
under the pinned day headings. calendar-timegrid's drag helpers scroll the
block into view before measuring it now; without that the suite passed for
hours and then did not, and the failure read as "dragging is broken".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 15:52:00 +02:00
Paul LütznerandClaude Opus 5 376cf4ae72 Add Paperless-ngx as an attachment source and a save destination
Connect it with an address and an API token and it appears beside the other
services; nothing else in the UI had to change, because the picker, the
destination chooser and the connect dialog are all driven by the provider
taxonomy and the driver interfaces.

TAGS ARE THE CONTAINER LAYER. A document archive has no directory tree —
Paperless replaced folders with tags and expects the rest to be found by
searching — so the picker's "descend into a container, pick a file" has
nothing to descend into. The root lists every tag as a folder with the newest
documents beneath it, and opening a tag lists that tag's documents.
Correspondents and document types would serve equally well and are
deliberately not offered: three sideways slices of one archive is a navigation
problem, and search already crosses all three.

Search is Paperless's own full-text index, so it reads the OCR layer as well
as the metadata and a scanned invoice is findable by a line item nobody typed.
That is the feature: there is no longer a reason to open Paperless, find a
document and download it before writing the mail.

Upload is asynchronous — post_document answers with a task id, not a document
id — so upload() returns it prefixed. Ids are opaque outside the driver, which
is what makes the prefix safe, and download() refuses a task id with a
sentence a person can act on rather than letting it 404 later.

ShareLink is not declared. Minting a Paperless share link is a permissioned
feature with its own expiry semantics, not something to do silently. The
taxonomy test that assumed only photo services lack share links had a false
premise; it now states the real rule rather than being relaxed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 21:51:58 +02:00
Paul LütznerandClaude Opus 5 ab9ecbc9e0 Hand a whole-view bulk action to a worker, which it never was
RunBulkStatusHandler exists to take a mark-read or archive over a whole view
out of the request — its docblock quotes the thirty-second timeout that
produced it, and BulkStatusController answers with an indicator so nobody
waits. Neither was true: the message was never listed in messenger.yaml's
routing map, and Messenger handles an unrouted message inline in whichever
process dispatched it. Every whole-view action ran inside the web request,
under the limit it was written to escape, at request concurrency — which is
where the production deadlock came from.

It gets a queue of its own rather than joining maintenance, because somebody
is watching an indicator for it and queueing it behind an embedding backfill
would be slower than the inline version it replaces. That means one new
container, worker-bulk, in every compose file and in the handbook.

The deadlock itself is retried now instead of failing the job, three times per
chunk, and a collision that outlasts those hands the job back to the queue
rather than ending it. The failure path no longer writes through an
EntityManager Doctrine has closed, which is what it was doing for every
failure that came out of a flush.

And the list catches up: nothing ever refreshed it on the strength of the job,
only on the toolbar's own "the write landed" event, which is now the instant
the work was QUEUED rather than finished.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 20:18:57 +02:00
Paul LütznerandClaude Opus 5 fb97a5a6b4 Recognise Google's weekly Testing expiry, and warn before the next one
A Gmail account that stops every few days is almost always one thing: Google
expires refresh tokens issued by an OAuth app whose consent screen is still in
"Testing" publishing status, after about a week. That is the state nearly every
self-hosted install is in, because leaving it there is the documented way to
avoid Google's verification review for scopes of Gmail's reach. So the account
works, stops seven days later, gets reconnected, and stops again. Forever.

Google gives no clue which case you are in. What arrives is `invalid_grant` —
the same code it returns for a revoked consent, a changed password, and a token
that aged out. Four causes, one error, and only one of them is fixed by changing
a setting rather than by signing in again every week.

THE ONLY THING THAT SEPARATES THEM IS THE INTERVAL, and nothing was recording
it. Two columns now do: when the current grant was issued, and how many hours
the previous one survived. The second is kept ACROSS the reconnect on purpose —
it is the only evidence that survives one, and the moment somebody has just
fixed it is the one moment they are looking at the account and could act on the
cause.

Two cards come out of it. A sign-in that died after about a week gets one naming
that cause and the real fix instead of the generic "sign in again" for the fifth
time. And once the pattern has been seen once, a warning appears a day or two
BEFORE the next one runs out, while nothing is broken yet.

The inference needs both halves and says so. Without the history it would nag
every healthy Google account weekly; without the age it would fire the moment
somebody reconnected, and a warning visible five days out of seven is furniture.
The window is 4 to 9½ days: wide enough for a death noticed a day late on the
next sync, narrow enough to exclude a grant that ran for three months and was
then revoked — which is the claim that must never be made, and is pinned in
WeeklyGrantExpiryTest in both directions. An account connected before this was
recorded is told nothing at all.

Verified on a running stack in all three states: the warning on a live six-day
grant with a weekly history, the diagnostic card on a dead one, and the plain
card with no mention of Testing on a grant that lasted ninety days. The last
found a real bug — the check was wired into inspectAccount(), which the settings
page never calls, so the card rendered nowhere. It also must not register as a
CAUSE, or every calendar on the account would grey out and blame a card whose
first sentence says mail is still arriving.

The handbook said "publishing and going through verification is what removes
that", which conflates two separate settings and is what makes people accept the
weekly reconnect as the price of not being verified. Publishing status and
verification status are different fields; an app published without verification
works, with a warning screen the first time. Corrected in English and German,
with a new section in features/health.md describing both cards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 09:34:54 +02:00
Paul LütznerandClaude Opus 5 d53e7c4963 Report browser errors to the admin panel, in a card of their own
A JavaScript fault was visible only to whoever had the console open, which on a
self-hosted mailbox means nobody. Uncaught exceptions and rejected promises from
plMail's own scripts now reach Admin → Logs, together with whatever the
Content-Security-Policy blocked.

Its own card rather than rows in the log, because it holds a different kind of
thing. A server error is one event in one request; a broken line in a controller
runs on every page load for every user. So a row is a distinct fault — message,
file, line — with a count and a lifetime, and the same fault reported again
increments it. Merged into the log, one bug would be four hundred rows and the
log would be unreadable past it.

THE FILTER IS THE FEATURE. Most of what window.onerror sees is not ours: the
console screenshot that prompted this was an ad-attribution metrics script
failing inside requestIdleCallback, injected by an extension. A panel that
accepted that fills with other people's bugs and stops being read, which is the
same as not having one. A report is kept only when it can be attributed to a
script this server sent — an injected script has no URL (Chrome's `VM947`), and
a cross-origin one arrives as the bare string `Script error.` with no file, line
or message. The rule is applied in the browser and again in the recorder,
because the browser half is part of the code being reported on.

CSP violations come to the same card through /csp-report. That endpoint cannot
be CSRF-protected and is public in security.yaml, because the browser generates
the request itself: no token, no header we chose, and on a login page no session.
The rate limiter is the bound. It is worth having because the full policy is
enforced in production, so what arrives is what this installation actually
refused to load in a real browser.

The count means page loads, not executions — the client reports each fault once
per page, capped at five. Measured rather than assumed: throwing the same error
twice on one page records one sighting. Documented that way on both sides after
the first version claimed the server would count what the client never sent.

`column` is a reserved word in Postgres, so the field is `column_number`. The
quoted DDL created the table happily and every insert into it was a syntax
error — same trap User::$usr already avoids.

Verified on a running stack by throwing real errors in a real browser: an error
from a script on this origin recorded, an anonymous injected one refused, the
card rendering both kinds grouped with counts.

Its first catch was a false alarm, and a useful one to record. It reported the
Mercure connection as blocked by `connect-src` — which was true, and was me
browsing `localhost` while MERCURE_PUBLIC_URL names `127.0.0.1`. Different
origins, so the app's own policy refused its own hub. Browsing the host the
config actually names produces no violation at all. The compose comment calling
that URL "same-origin" is correct on its own terms; it just stops being true if
you reach the stack by another name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 16:02:42 +02:00
Paul LütznerandClaude Opus 5 41362290bb Release v0.2.20
Twenty-six commits since v0.2.19, of which the changelog had one entry — the
rest written up here.

The handbook needed two corrections rather than additions. features/mail.md
still said a sender you have written to is pulled into Primary "regardless of
any bulk-mail header", which stopped being true when the mailing limit went in
and directly contradicted the paragraph two sections above it. features/admin.md
described Reported mail as one pile of missed insights with a scope drop-down,
which is now two kinds in one list with a selection. Both German pages follow,
and both markers are re-hashed.

README gains the sorting choice and says the summary can take a whole
conversation. Screenshots are untouched: nothing in the seven views they show
changed this cycle — the new interface is in Settings and in Admin → Reported
mail, neither of which is photographed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 15:42:52 +02:00
Paul LütznerandClaude Opus 5 87eb10beed A mailing beats having written to the sender
The correspondence override says: you have written to this address, so this
belongs in Primary. That is the right rescue for a person whose mail happens to
carry a bulk-ish header, and in practice it was doing something else — one
enquiry to a shop's info@ about a broken part pinned every anniversary sale that
shop ever sent into Primary, for ever, and nothing on the page explained why.

The distinction it was missing: a person you correspond with does not send you
mail with an unsubscribe link in it. Their marketing platform does, from the
same address, and the address was all the rule could see.

So the rescue now applies to mail that is not a MAILING, which is the case it
was written for. A list header, a list id or an explicit bulk precedence wins
over it; everything weaker — a feedback-id, an automation marker, an unfamiliar
sending domain — still loses to somebody the reader has written to. That is the
same line drawn one commit ago for what the model is shown, and for the same
reason: marketing is legally obliged to offer an unsubscribe mechanism, which
makes it the signal that actually means "sent to many".

The forum and promotion signals move above the override so it can consult them,
and are computed once rather than twice.

THREE EXISTING TESTS ASSERTED THE OLD RULE and are updated rather than deleted,
including one named "whatever the headers say" — which was the bug, stated as a
guarantee. Each keeps its point and now uses a weak marker: what they were
pinning is that the rescue works, and a mailing header would make them tests of
the new rule instead of the one they are about.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 13:19:50 +02:00
Paul LütznerandClaude Opus 5 fabaf49a80 Let a reader report mail as filed in the wrong tab
Categorisation is decided by three things that disagree — a provider's labels, a
header cascade, and a model — and whether the answer is any good is a judgement
only the owner of the mailbox can make. Everything else in this feature can be
measured against fixtures somebody invented. This is the one route by which real
mail, wrongly filed, becomes something a rule or a prompt can be changed on the
strength of.

The control sits at the bottom of the header popover, where the three answers
already are. A SELECT AND A BUTTON rather than five buttons: knowing a message
was misfiled is a fraction as useful as knowing where it belonged — "promotions,
not primary" is a rule somebody can act on, "wrong" is a shrug. The tab it is
already in is not offered, because "it should be where it is" is not a report.

It records the DECISION and its evidence, not the message: what each of the
three said, what the rules matched on, which of them was deciding, which bulk
headers were present, the sender and the subject. Not the body — the most
sensitive part of a message and the least useful here, since most marketing mail
has no plain-text part at all and the classifier sees "(no plain text part)", so
the column would be empty for exactly the mail most often reported.

Sender and subject ARE kept, and on a multi-user installation an administrator
reading these sees them for other people's mail. That is a real cost, taken
deliberately: a report saying only "something was wrong" is not worth the
button, and it is why nothing is recorded without somebody pressing it.

The reports collect in Admin → AI rather than a nav entry of their own, because
that is where somebody ACTS on them — the prompt and the model are a card
above. The product is the TEXT: one line per report, built in the entity rather
than the template because a format assembled in Twig would change whenever the
table's layout did. Copy takes the block; clearing is a deliberate press, since
these are observations nothing regenerates and a pattern across twenty of them
over two months is the whole value.

No foreign key to `message`: a report outlives the mail it is about. Somebody
deletes the message, the pattern it was evidence of does not go away.

Verified on the stack: the control renders, a report returns 204, a hand-edited
category is refused with 400, the line appears in the admin card, and clearing
empties it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 13:14:27 +02:00
Paul LütznerandClaude Opus 5 b54cdc0c85 Show the re-sort run in the jobs indicator
Fire-and-forget answered a request for a button with a button whose only
feedback was the page reloading. There is no way to tell whether it is running,
how far it has got, or whether it failed — on work that is minutes of model
calls and is invisible by construction.

So it takes a BackgroundJob row and reports through the indicator that already
exists: the topbar shows it while it runs, counts it off in batches, draws the
bar, and says so if it fails. Mercure carries the nudges, the row is the record,
and closing the settings page changes nothing.

JobKind::Reclassify is the odd one out and is admitted as such rather than
disguised — every other kind is a bulk action over a mail view, which is why
action() and readFlag() exist on that enum. It is there anyway because what a
reader needs from a run of model calls is exactly what the indicator already
gives, and a second progress mechanism beside a working one would be two things
to keep in step and one more thing to learn to read. action() throws for it:
RunBulkStatusHandler is its only caller and only ever holds a kind it was
dispatched for, so throwing beats inventing a string that some switch would
silently accept and act on somebody's mail with.

Progress is published per BATCH rather than per message — twenty model calls a
minute would be twenty Mercure publishes saying almost the same thing, and the
batch is already the unit the flush and the thread re-file happen on.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 12:44:03 +02:00
Paul LütznerandClaude Opus 5 39dd7e5857 Ask the model again about recent mail, and let a verdict reach the tab
Two things, and the button is useless without the first.

A VERDICT NEVER REACHED A TAB. Nothing recomputed a message's category after
the classifier stored one: ingest files a message the moment it arrives, the
classifier runs afterwards and asynchronously, and it wrote the answer and
stopped. That was invisible while sorting was rules-only — the verdict was a
tie-break nobody could see — and it stopped being invisible the day sorting
became a setting. Somebody chooses "the assistant", new mail keeps arriving
into whatever the rules said, and the choice appears to do nothing at all.

ClassifyMailHandler now re-files what it just asked about, using each owner's
own preference, and resolves only the threads it touched — recomputing per
ACCOUNT would scan every message the account has, after every batch of arriving
mail. recomputeCategoriesForThreads() is that statement with the threads named,
pinned conversations still excluded.

AND THE ANSWER GOES STALE WHILE THE MAIL DOES NOT. A message is asked about
once, which is right. What changes is the QUESTION: the prompt is editable, the
model is a setting, and what plMail sends alongside a message changed one commit
ago — every verdict stored before the bulk-header line was reached without
evidence the model now gets, and demonstrably differs because of it.

So: a button, bounded to the newest 100, 200 or 500 messages, clamped server
side because the field is a select and anything else arrived by hand. Nothing
re-asks on its own and nothing should — a model call per message is the most
expensive thing here, and spending an afternoon of somebody else's GPU because
an administrator edited a prompt is not a decision to take for them.

ClassifyMailMessage gains a `force` flag, defaulted and last so envelopes
already on the transport still deserialise. The stamp means "already asked",
which is the right reason to skip on ingest and the wrong one when the answer is
what has gone stale.

The re-run INVOKES the classify handler rather than dispatching to it, and the
transport is why: ClassifyMailMessage routes to `ingest`, correct for a message
being classified as it arrives and wrong for several hundred at once — arriving
mail would queue behind a button press for the best part of an hour. Called
directly, the work stays on `maintenance` and reuses the asking, the batching,
the per-message permission check and the re-filing rather than growing a second
copy of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 12:32:53 +02:00
Paul LütznerandClaude Opus 5 e3a4b78d92 Re-file the mailbox when the sorting setting changes
The previous commit shipped the setting and told people to go and find an
administrator if they wanted their existing mail re-sorted. That was wrong, and
the argument against it is the one that should have been made for it.

MessageCategorizer reads only PERSISTED data — stored headers, the provider's
labels, the sender, the model's stored verdict — which is the entire design of
the class. So a re-sort asks no other machine anything: no IMAP, no model, no
network. It is a scan of rows the database already has, a PHP cascade over each,
and a column write, followed by one statement per account that resolves each
thread from its messages.

The caution behind the old answer was real — a hundred thousand things do not
belong inside an HTTP request — but the fix for work too big for a request is a
worker, and this application has four. A summary job was written one commit
earlier on exactly that reasoning and it was not applied here.

Leaving it undone was the worse option, not the safe one. Mail arriving after
the change would be sorted the new way and everything already there the old way,
so one inbox would be filed two ways depending on when each message landed — an
incoherent state nobody chose, produced by a setting they did, with no symptom
that leads back to the radio button that caused it.

Dispatched only when an answer actually CHANGED. The card submits on change and
a browser will happily post the option already selected; re-filing for that is
minutes of a worker writing the answers back unchanged.

Threads pinned to a tab by hand are untouched — recomputeCategoriesForAccount
has always skipped `category_pinned_at IS NOT NULL`, and it is worth saying out
loud on a feature that re-files somebody's inbox.

Safe to run twice: the categoriser is a pure function of persisted state, so a
retried envelope writes the same answers over the same rows. maintenance rather
than ingest, so arriving mail never queues behind somebody's settings change.

The command stays and is unchanged — an operator re-sorting every mailbox after
a rules change wants exactly this work, for everyone at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 11:29:52 +02:00
Paul LütznerandClaude Opus 5 ed4a483ac3 Let each person choose what sorts their mail into tabs
Two decisions, and they were both taken for everybody by a constant.

WHAT DECIDES. The rules read headers — a mailing-list header, an unsubscribe
link, a sender you have written to — and answer the same way every time for the
same mail, on any installation, with nothing switched on. The assistant reads
the message and decides what it is for: better at mail that does not announce
itself, and occasionally confidently wrong. Which is preferable is genuinely a
matter of taste, which is why it is now a setting rather than something argued
about once.

Choosing the assistant puts its verdict ABOVE the header cascade — which is what
choosing it has to mean. Read only as a tie-break, a newsletter carrying
List-Unsubscribe is Promotions whatever the model made of it. Below the
correspondent rule, though, and that placement is the one to keep: "you have
written to this person" is a fact about the reader rather than a guess about the
mail, and a model above it files a colleague under Promotions on one bad guess —
the only mistake in this cascade anybody would actually notice.

WHETHER IT OUTRANKS GMAIL. A Gmail account arrives already sorted, and plMail
has always treated those labels as authoritative — its own rules never even ran.
That is the right default and it is also exactly what somebody who dislikes
Google's sorting could not escape, on the account where they most wanted to.

Per person, not per installation, for AiPreferences' reason: an administrator
decides what is available and a reader decides what happens to their own mail.
Its own embeddable rather than two more fields in the AI bag, because only half
of it is about the assistant — the provider half works on an installation with
no model at all, and folding it in would file a setting where nobody would look
for it.

ONE BEHAVIOUR CHANGE, NAMED RATHER THAN DISCOVERED. The model's verdict used to
be consulted as a silent tie-break: no setting turned it on, none turned it off,
and nothing on screen said it had happened. It is now what Assistant means, and
Rules means what it says — so an installation with categorisation on loses that
tie-break until somebody chooses. The alternative is a model quietly sorting
mail for people who never asked, which is the arrangement this ends. Said in the
migration, in the handbook, and in an upgrade note.

Nothing is re-sorted. The category is recomputed from stored data, so a mailbox
keeps what it has until `app:backfill category` runs — an operator's call, not a
settings toggle's, and the card says so.

Verified on the stack rather than asserted: the card renders, the assistant
option is disabled with its reason when categorisation is off, and a posted
change lands in the column. The four combinations of the two settings pick three
different categories for one fixture, which is what makes them worth having.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 11:16:01 +02:00
Paul LütznerandClaude Opus 5 932c3f1f6f Finish a summary whose reader has gone, and store it
The controller broke out of the loop when connection_aborted() fired and stored
nothing, on the grounds that what exists is half a summary and half a summary
reads as a finished one. That is true of the TOKENS and false of the answer:
draining the generator to the end produces the whole summary, and throwing it
away made somebody whose connection dropped at ninety per cent pay the entire
cost again.

Connections drop here for reasons plMail cannot see — a proxy, a network, a
laptop closing — and on this installation they demonstrably do, on ordinary
summaries as well as long ones. Finishing turns that from "you lost the work"
into "reload and it is there".

The frames stop at the abort, because there is nobody to send them to; the loop
does not. The store happens either way, and only the `done` frame is skipped.

The price, stated where it is paid: a reader who genuinely navigated away keeps
the model busy until it finishes, where it used to be released at once. That is
the trade, taken deliberately.

Two things fall out of it. The generator is now always exhausted by the end, so
AiAssistant::recorded() files what actually happened instead of stamping the
call `cancelled` — a word that was never true of a run the model completed. And
the abandonment path keeps its log line, reworded: it says the reader was lost
and the summary is being finished anyway, with the elapsed seconds and how much
had been delivered, which is still what distinguishes a proxy giving up from a
person navigating away.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-04 10:44:25 +02:00
Paul LütznerandClaude Opus 5 408fcd1bed Run full-conversation summaries as a job, with nothing held open
A full run is silent for minutes while the model reads, and for the whole of
that silence the answer depended on a browser connection staying open. That
dependency is what has been failing, and no timeout could have fixed it:

    full: true, transcript_chars: 67720, elapsed_seconds: 40, chars_so_far: 0

Forty seconds, four heartbeats already delivered, nothing written yet, and a
model host that was up for seven days either side of it. The connection was
alive and being written to and then was not. Nothing in plMail closed it — a
held-open connection has a proxy, a browser and a network in it, and any of them
may end it without telling this application. Three rounds went into tuning
timeouts before the metrics panel said `cancelled` and named the real shape.

So the full run stops needing a connection. It dispatches SummariseThreadMessage
and answers 202; a worker on the maintenance transport does the work, writes to
the store and publishes a Mercure nudge; the card shows it as queued and fills
in when it lands. The reader may close the conversation, the tab, or the laptop
and still find the summary waiting — which is what somebody just told "this
takes several minutes" would reasonably expect anyway.

maintenance rather than ingest, deliberately: this holds a consumer for minutes,
and mail arriving must not queue behind somebody's summary.

The nudge carries no text. Same shape as JobNotifier and the insights strip —
the store is the record and a missed message costs one stale render — with one
reason of its own: what it would be carrying is a paragraph about somebody's
mail, on a channel that has never held any.

TWO BUGS RETIRED ON THE WAY. The streamed path is now never full, so the
transcript it hashes for freshness is always the one the reader's page
recomputes; a full run used to store a hash that could never be reproduced,
which would have shown every full summary as permanently out of date. And the
handler hashes the ORDINARY transcript for the same reason while sending the
full one to the model — the hash answers "has this conversation changed", and
the only answer that matters is the one the reader will ask.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 14:28:25 +02:00
Paul LütznerandClaude Opus 5 822a17bfef Name the last silent path, and document the proxy it points at
The performance panel said it: "Last call: Summaries · cancelled". That row is
only written when the generator is ABANDONED — AiAssistant::recorded() records
ERROR_CANCELLED when its consumer never reached the end — so the summary was not
timing out at all. It was the controller's abort path, which breaks out of the
loop when connection_aborted() fires and returns without a frame and without a
word, because losing the reader is a normal thing that needs no explaining.

Except it is only normal when the reader left. From inside this process, a proxy
closing the stream looks exactly the same, and one of those is a fault. Silent,
the two were indistinguishable: a card reading "the summary could not be
written", nothing in the log, and a `cancelled` row that reads as somebody
changing their mind. Four rounds of this were spent on timeouts because the one
record that named the real shape was in a panel nobody thought to read.

It logs now, with the elapsed seconds and how much of the answer had reached the
browser. Those two tell the cases apart: somebody who gives up does it at a
human moment and usually mid-answer, while a proxy does it at its configured one
and during the silence before the first token — and the same round number every
time is the whole diagnosis.

AND THE DOCUMENTATION HAD HALF OF THIS ALREADY. reverse-proxy.md turns buffering
off and raises the read timeout to 24h — for /.well-known/mercure, and nothing
else. plMail serves two long-lived streams and only one of them was ever
mentioned; the assistant's endpoints got nginx's default 60-second read timeout,
which is shorter than a summary of a long conversation legitimately takes. That
is now written down, in both languages, with the failure mode spelled out next
to it.

No behaviour change beyond the log line. The proxy block is the fix, and it is
not one plMail can apply to somebody else's proxy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 14:01:14 +02:00
Paul LütznerandClaude Opus 5 2bfc30ad8d Offer to summarise the whole conversation, and send a context window with it
The card now says when a summary was written from part of a thread, and the way
out sits beside the notice that explains why it is there.

num_ctx IS THE WHOLE MECHANISM. This client has never sent one — four docblocks
across the codebase size their budgets around "OllamaClient sends no num_ctx, so
the model's default decides what survives", and that default is 4096 tokens with
everything past it dropped silently. So "send the whole conversation" without
also asking for somewhere to put it would send more, have it truncated at a
point nobody chose, and spend a much longer wait to produce no more than before.
It is sent only when a caller asks, so every existing call keeps its shape.

FULL IS STILL BOUNDED, and that is the honest part rather than a caveat. A
context window is memory: Ollama allocates the KV cache for the whole of it
whether the prompt fills it or not, so an unbounded "send everything" lets one
absurd thread ask a modest host for an allocation it cannot make — and a failed
allocation costs the reader the entire wait and produces nothing, where a
clipped transcript produces a summary and a sentence saying it is partial. The
ceiling is 96,000 characters against a 32,768-token window, twelve times the
ordinary budget. Past it the run is full AND partial, and the button withdraws
rather than inviting the same wait twice.

The window is SIZED to the transcript rather than fixed, since it is reserved
whole either way and a two-message thread should not cost what a hundred-message
one does. Sized at 3.0 characters per token, deliberately below the 3.55
measured on real German business mail: this divides to get tokens, so erring low
overestimates the need and asks for slightly more room than strictly required.
Erring the other way would silently truncate inside the one feature whose entire
purpose is not to.

thread_summary gains full_context, and it cannot be derived like the rest of the
card. "Is this thread too long to send whole" is a question about the thread and
comes free from the transcript the freshness check already builds. "Did THIS
summary see all of it" is a question about the row, and the thread cannot answer
it — a thread that is still too long is still too long after somebody presses
the button and gets a summary that did see everything. Without the column that
press would appear to do nothing: new text, same notice, much longer wait.

A press and not a setting, deliberately. It is a fair thing to spend once on a
thread somebody cares about and a poor default for every summary in a mailbox,
so the cost is written into the label and nothing is remembered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 12:43:18 +02:00
Paul LütznerandClaude Opus 5 ffe9457b42 Give the search model its query instruction and its own threshold
Search by meaning was answering "mails about food" with a page of login links
and newsletters, ranked confidently, with nothing anywhere saying it had gone
wrong. Two settings decide whether it works and plMail had neither.

THE QUERY INSTRUCTION. Embedding models are asymmetric: they are trained to be
told which side of a search they are looking at. An unlabelled query is treated
as one more document, and it lands nearest whatever is most boilerplate — in a
mailbox, login links and newsletters. Measured on qwen3-embedding:0.6b over
twelve German and English messages, five about food:

    query                     non-food ranked above the best food mail
    "mails about food"        5    (raw)      0  (instructed)
    "essen"                   7    (raw)      0  (instructed)
    "restaurant reservation"  2    (raw)      0  (instructed)

"essen" — the least ambiguous word available — put every food mail below every
non-food mail. With the model's own instruction, perfect separation on all
three. It goes on the QUERY ONLY, so stored vectors are untouched and nothing
has to be re-indexed.

THE THRESHOLD. 0.55 was one constant for every installation, and cosine
similarity is not comparable between models — or even between the instructed
and uninstructed forms of one model. Best threshold by F1, same corpus:

    qwen3-embedding:0.6b  no instruction    0.20   F1 0.59   precision 0.42
    qwen3-embedding:0.6b  with instruction  0.42   F1 0.85   precision 1.00
    nomic-embed-text      no instruction    0.45   F1 0.74   precision 0.65
    all-minilm            no instruction    0.20   F1 0.71   precision 0.77

0.55 is above all of them. So it travels on SemanticSearch next to the model and
width it has to agree with, and it is a setting rather than a release.

Both are columns, and EmbeddingPreset holds the measured pair for each model the
release was tested against, with a line on what it is good and bad at. The model
field stays free text — a closed dropdown would go stale and refuse something
that works — and choosing a preset fills all three fields at once, because they
are one decision written in three boxes: any two from different models give a
search that is confidently wrong rather than visibly broken.

qwen3-embedding:0.6b becomes the documented default. Multilingual matters more
than any benchmark for a mailbox that is not all in one language, and it was the
only combination measured at precision 1.00.

Existing installations keep 0.55 and no instruction. Backfilling 0.42 would
apply Qwen's number to whatever model is configured, and on all-minilm (0.20)
that is twice as tight as it should be — a migration is the wrong place to
change what somebody's search returns. The panel now shows both values and sets
them on one click.

The instruction field is trim: false, which a test caught rather than a guess:
every working instruction ends in a separator, Qwen's is "…\nQuery: ", and
Symfony's default trim turned it into "Query:mails about food".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 11:42:28 +02:00
Paul LütznerandClaude Opus 5 6a4118ddbb Stop reporting a failed semantic search as a slow model
The search page has two ways to print "The model took too long, so this
search used words only", and only one of them is about the model.

SemanticQuery answers TimedOut when Ollama does not return a query vector
inside EMBED_TIMEOUT. That one is honest. MessageThreadRepository::cheapRows()
answered the same TimedOut when the vector arm blew its 5s statement budget —
by which point the model had already answered, in milliseconds, and is sitting
in ai_call_metric marked a success. An operator reading that sentence goes to
the model host, where there is nothing to find.

SemanticQuery's own docblock guards the mirror image of this: the embedding is
done in the controller so that a slow model is never reported as a database
fault. This was the same confusion pointing the other way, and it is the worse
direction, because it names a machine that is working.

Worse, the catch was on DriverException, which is every driver fault Postgres
can raise. DBAL's PostgreSQL ExceptionConverter maps deadlocks, constraint
violations and syntax errors onto dedicated classes and lets 57014 fall through
to a plain DriverException, so "the budget fired" and "the query is broken"
arrived as the same type. A missing plmail_embed_distance(), a stored width
that disagrees with its vector, a connection dropped mid-statement: all
deterministic, all degrading every search instantly and for as long as the
cause stands, all reported as an occasional overrun — and none of them logged
anywhere at all, because the exception was discarded unnamed.

So: SearchTooSlow for a real cancellation, SearchFailed for everything else,
each with its own sentence in all three catalogues, and both logged with their
SQLSTATE. The repository gets its first logger for it. A rare give-up path that
leaves no trace is indistinguishable from a common one, which is how this
stayed invisible.

The new test drops plmail_embed_distance inside the test transaction rather
than making the query slow, so it cannot pass by accident on a busy machine:
42883 is not 57014 under any load.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-03 10:31:06 +02:00
Paul LütznerandClaude Opus 5 d8f8acf29b feat(ai): tunable keep-alive per model, and a warm-up button
The first "Help me write" after a quiet spell shows nothing for tens of
seconds while the Ollama host reads twenty gigabytes back off disk. How
long it waits before evicting a model is OLLAMA_KEEP_ALIVE — an
environment variable on a machine plMail can talk to but cannot
configure. The API takes the same setting per request and a per-request
value wins, so this is the only lever available from inside the app.

Two nullable columns on ai_settings, one per model, because the two are
opposite trades: the writing model is eighteen gigabytes and asked for
occasionally, the embedding model a few hundred megabytes and asked for
on every search and every indexed message. Both ship at 5m, which is
Ollama's own default, so an upgrade changes nothing about how a host
behaves — the boxes just say out loud what was already happening. -1 is
recommended for the search model in the help text and the handbook
rather than shipped, because plMail cannot see how much memory that host
has. Empty means send no keep_alive at all and let the host decide,
which is a real selectable state and not an unfinished one.

The wire representation is the part that is easy to get wrong: Ollama
parses a JSON string as a Go duration, and a Go duration needs a unit, so
"-1" and "3600" as strings are a 400 on every model call from a setting
that reads as correct. KeepAlive::forBody() is the only thing that puts
the value in a body — int for the numeric spellings, string for the
durations, null to omit the field entirely, since an explicit null is
read as zero and unloads immediately. One helper adds it to all four
bodies so no call site can get that wrong.

Validated with the same expression the body is built from, refused
rather than coerced, and the message lives in the validators domain
where Symfony actually looks for it.

Warm up now: POST /admin/ai/warmup issues the no-prompt /api/generate,
times the round trip and reports it. PHP's 30s execution limit is raised
for it — a cold load is not 30 seconds — and the upstream timeout sits
below that so it gives up first. A host that is unreachable and a model
that is not on the host are distinct, translated answers.

Also carries both columns through the configuration backup, and adds the
AI feature to the README, which had never mentioned it.

Fixes a search test that asserted how fast the machine was: it gave the
last-resort body pass a one-millisecond budget and one row to scan, so on
quick hardware the pass finished in time and the test failed. It now
provokes a genuine statement_timeout cancellation with pg_sleep, which
exercises the same guard without a clock in the assertion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 12:54:09 +02:00
Paul LütznerandClaude Opus 5 e7f4513574 fix(backup): the config backup had stopped carrying what it promises
A restore looked like it worked. That is the whole shape of this: nothing threw,
the review page listed a clean plan, and the parts that had gone missing produced
no row to warn about, because a section that does not exist has nothing to say.

── A removed user came back to life ────────────────────────────────────────
The restorer's own docblock and the handbook both say a soft-deleted user counts
as existing, "because they hold the email address against a unique index".
They had stopped holding it. Removal frees the address on purpose — the admin
delete action rewrites it to `deleted-<id>@invalid` — so existingId() answered
null for every removed user, the plan called them Absent, and the restore created
them afresh and LIVE: old password hash, TOTP secret, recovery codes, every
mailbox credential. Listed as an ordinary creation, indistinguishable from a new
colleague. The handbook steers operators straight into it.

The rewrite destroys the only copy of the address, so nothing could match it
back; `email_at_deletion` is that copy, written at the same moment. Rows
tombstoned before the migration stay unrecognisable, which is stated and held by
a test rather than papered over.

The regression test that was supposed to catch this passed throughout, because it
wrote `deleted_at` and left the email in place — a row the application cannot
produce. It now tombstones the way the delete action does.

Its sibling: a violated unique index escaped as an uncaught 500 with no message.
api_token.token_hash is unique across the WHOLE install and a soft delete
cascades nothing, so a removed user's tokens still hold those values. Translated
into a stated failure with a sentence attached. Not demoted like other failed
writes — a violated constraint aborts the transaction, so there is no "rest" to
let through, and stopping with a reason is the honest shape.

── Four things a person configured never reached the file ──────────────────
The assistant configuration in full: model host, both models, the reverse-proxy
token that exists in plaintext nowhere else, the feature switches, and every
prompt an administrator rewrote. The per-user half had been carried since the day
it shipped; the instance-level half it depends on had not, so a restore returned
everyone's preferences pointing at a host that was gone.

Account signatures and read-receipt defaults — and on the instance this was found
on, `compose.signature` was the ONLY account setting in use, so every export it
produced held an empty settings bag for every account. The per-alias overrides
key themselves by alias row id, which names somebody else's alias on the target,
so they now travel inside their alias and are rewritten under the new id after
the flush, the way `calendar.target_id` already was.

Five user settings, the chosen log level, and the OAuth scopes that describe what
the provider granted to the refresh token the file already carries.

── Why none of it was caught, which is the part worth keeping ──────────────
Every guard pointed the wrong way. The env test builds its expectation by calling
the constant under test. The users test is fourteen assertArrayNotHasKey calls —
it fails if somebody ADDS a field, never if one goes missing. The version
assertion compares a constant against its own echo. A backup's characteristic
defect is omission, and nothing could fail in that direction.

Two tests now compare the backup against the application rather than against
itself, and fail closed: every User and Account SETTING_ constant must be carried
or excluded on the record with its reason, and the database section must carry
every admin-configured table. The second would have gone red the day AiSettings
landed. A round-trip test pins the set of things a restore drops to exactly two,
each deliberate, so the set can only change on purpose.

── Also ───────────────────────────────────────────────────────────────────
The KDF had a ceiling and no floor, so `memlimit: 1` passed every check and took
a SodiumException out onto a blank error page — through the anonymous
/install/restore among others. Both memzero paths now run under try/finally, the
abnormal case they exist for. App passwords keep the date they were created
rather than the date they were restored.

And the admin handbook said the file contains "no user accounts". It contains
every one, with their secrets — published in two languages. Corrected, along with
the inventories, which now list what actually travels.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 17:33:22 +02:00
Paul LütznerandClaude Opus 5 755f3cf257 feat(users): an administrator can run the whole life of an account, and you can change your own password
Managing users was reachable and worked — list, search, create, promote, demote,
remove — but three things around it did not, and one of them was load-bearing.

── You could not change your own password ──────────────────────────────────
Nowhere. Not in Settings, not by mail, not at all; only an administrator setting
one for you. Settings → Security now leads with a Password card that asks for
the current password (an unmapped UserPassword constraint, so it is checked
against the signed-in token rather than against anything the form was handed)
and takes the new one twice. The route carries no id and the form has no
data_class, so it structurally cannot become the admin form by accident.

The floor those passwords are held to was three literal 12s in three files that
happened to agree. It is now User::PASSWORD_MIN_LENGTH, read by all three.

── An account could only be removed, never switched off ────────────────────
Nothing sat between "leave it alone" and "destroy it", so the answer to "this
person has left, keep their mail" was to delete them. There is a deactivated_at
now, and a UserChecker registered on BOTH firewalls — the JMAP one included,
because an app password outlives the browser session it was minted from and a
check only on `main` would shut somebody out of the web UI while every mail
client they had already connected kept syncing.

The refusal is in checkPostAuth rather than checkPreAuth on purpose. checkPreAuth
runs before the password is verified, so refusing there would answer "is this
address a suspended account here?" to anyone who types an address — the question
hide_user_not_found exists to keep unanswerable. After verification, the only
person told is the one who could otherwise have signed in.

Symfony runs user checkers at authentication only, so a session already open
would have carried on until it happened to end. UserProvider::refreshUser()
refuses the same two states on every request. Suspension also travels through
config backup, which would otherwise have handed a suspended account its sign-in
back on restore.

── Two smaller lies ────────────────────────────────────────────────────────
A search under three characters silently returned the unfiltered list, which a
QueryBuilder has no way to complain about, so the floor was dropped rather than
kept as something that could only mislead. And an admin declining to demote
themselves got a success toast for a change that never happened; the refusal now
says so, while the rest of the form still saves.

── Restoring a removed user is deliberately absent ─────────────────────────
Removal overwrites the address, the name and the hash specifically to free the
unique address for reuse, so a restore would have to invent all three and the
original address may by then belong to somebody else. Doing it properly means
snapshot columns and a guided flow that asks for a fresh address — a different
change, which alters what removal means. Documented as absent on the screen an
administrator would look for the button on, because switching an account off is
the answer to the question people are actually asking.

Also fixed en route: the modal form theme dropped help_translation_parameters,
so the first field to use one rendered "at least %count% characters" on screen.
And GermanIsInformalTest only ever globbed messages.de.yaml, leaving validators
and the new security catalogue — exactly where a translator reaches for "Sie" —
outside the test that exists to catch it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 16:02:12 +02:00
Paul LütznerandClaude Opus 5 dadab12439 feat(inbox): a tab says in colour when it is holding unread mail
The report, with a screenshot: "Unread only" on, Primary selected, the list
reading "No messages in this tab", and the sidebar's Inbox badge reading 66
next to it. Both were true — the 66 was in Social and Updates, and nothing on
the strip distinguished those tabs from the empty one being looked at. The only
way to find the mail was to click every tab in turn.

The reasoning that had kept unread off the tabs holds in the ordinary inbox:
unread is already said by the bold rows and the sidebar badge, so a number on a
tab would be a third voice saying it. Under the filter it inverts. Every row on
screen is unread, so boldness separates nothing, and the badge is one total that
cannot name a tab.

So the strip makes the one statement nothing else can, without a number: the
icon wears its category's colour when there is unread mail behind it. Drawn
outside the filter too, so the strip means the same thing wherever you meet it,
and never on the tab being looked at — that mail is already in the list
underneath, and an identity colour dropped into the accent-coloured active tab
reads as a fault rather than a signal. It was reported as one.

It rides on the existing marker machinery: rendered whatever the count, carrying
a data-count-key the counts endpoint now answers, so a Mercure sync re-tints in
place. The tone travels in an attribute because the patcher cannot know which of
the five a tab wears.

Three unified-inbox counts also stopped counting conversations the list refuses
to show. Binning a thread ADDS the Trash label rather than swapping it, and
findForUnifiedInbox() has always excluded trashed threads while the counts beside
it did not. That lit a tab for a thread the list would not show, on a mailbox
whose badge said one unread and whose tabs claimed two — the Trash-badge-versus-
list disagreement BadgeSemanticsTest exists for, arriving by another road. It
also means countForUnifiedInbox() had been feeding the paginator a total that
could offer a page that does not exist, which is the failure its own DISTINCT is
written out to prevent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-31 15:29:42 +02:00
Paul LütznerandClaude Opus 5 5f8ae26be4 feat(ai): the system prompts are the administrator's, and summaries follow them
Seven prompts were constants, so tuning one meant a deploy. Admin -> AI now
holds all of them, with the shipped wording as the placeholder and a way back
to it.

EMPTY = NULL = SHIPPED, and that one rule carries the whole design. Reset is a
DELETION rather than a copy, so a later release can still improve the wording
instead of every installation being pinned to whatever shipped the day it first
saved. It is also why the language rule can be reworded but not removed: there
is no special case, an empty box simply is not an override.

THE HAND-BUMPED VERSION HAD TO GO
thread_summary carried PROMPT_VERSION, an integer whose docblock said "bumped
whenever SYSTEM_PROMPT changes". That cannot survive an editable prompt: an
administrator tunes it, nobody bumps anything, and every stored summary keeps
the old prompt's output for ever while the page calls it fresh. It is now a
digest of the EFFECTIVE prompt, which is the same move the transcript hash
already made — edit a prompt, edit the language rule, clear either, or take an
upgrade that improves the shipped text, and each is a different string on the
wire, so a different digest, so the stale rows stop matching by construction.

WritingTask::systemPrompt() is deleted rather than kept. A method by that name
returning something that is not the system prompt on any installation that
edited one is precisely the drift this repository keeps collapsing.

The cap is 1200 and not AiPreferences' 600, because the summary prompt we ship
is 587 characters: a cap the shipped text already fills only ever bites the
person trying to improve it.

Verified against the real host — an override changed the answer and clearing
the box changed it back. Worth recording the negative result too: rewriting the
language rule to demand English did NOT make this model answer a German mail in
English. The rule is a nudge it already agrees with; the bug it was written for
was the ABSENCE of any rule, which is now unreachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 17:16:36 +02:00
Paul LütznerandClaude Opus 5 661d3bbf25 feat(ai): summarise a conversation, when somebody asks
On demand, and that is the design rather than a first step. A summary spends the
20.3 GiB writing model for about a minute cold, on one GPU already shared with
the indexer and the composer. Summarising every arriving thread would spend days
of it on conversations nobody opens.

FRESHNESS IS DERIVED, NOT MAINTAINED
The stored row carries a SHA-256 of the exact transcript that was sent, plus the
model tag and a prompt version, and freshness is that hash matching what the
thread would send today. Nothing has to remember to invalidate anything.

lastMessageAt was the obvious key and is the wrong one: it only ever moves
FORWARD, so a deleted message leaves a summary describing mail that is no longer
there, silently and for ever. The hash also makes a prompt change or a budget
change invalidate every stored summary by itself, which is correct and free.

CASCADE, following message_embedding rather than MailInsight. The two
precedents point opposite ways and both are right — an insight outlives its
message because "a deleted shipping confirmation does not un-ship the parcel",
an embedding does not because it describes one. A summary describes a thread.

THE BUDGET IS MEASURED, NOT PICKED
8000 characters, against two ceilings and the smaller one is silent. The visible
one is the wait before the first token — load plus prompt eval, about 42s at
this budget, a third of the client's idle timeout. The quiet one is the context
window: OllamaClient sends no num_ctx and Ollama's default is 4096 tokens, so
12000 characters sits exactly on the line, and an install running the default
would drop the HEAD of the transcript — the part the reader most wants and the
part the transcript is assembled forwards to protect. No error, no notice.

The fourth AiFeature case landed LAST, after every arm that reads it. Eight
exhaustive matches key on that enum and none of them fails at compile time;
AiPermissions::anyAdminEnabled() loops the cases and the settings nav calls it
on every settings page, so the case arriving early would have taken the whole
settings area down rather than the AI section.

Verified against the real host, not a mock: a German thread returns a German
summary, a reopened thread renders the stored row with no new model call, a
reply greys it and offers to run again, and changing the chat model hides it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 14:43:59 +02:00
Paul LütznerandClaude Opus 5 2348b75621 feat(db): pgvector as an opt-in image, built without changing libc
The original rejection of pgvector was tested rather than reasoned, and it
stands: pgvector/pgvector:pg18 means musl to glibc on an existing data volume,
and 44 collation-dependent indexes — 18 of them UNIQUE — are then walked with
the wrong comparator, silently, because musl records no collation version so
Postgres's own mismatch check never fires.

So the extension is compiled INTO the alpine image instead. libc never changes,
and there is nothing to rebuild. Verified end to end rather than argued: an
existing volume migrated on stock alpine, stopped, restarted on the pgvector
image, then bt_index_check(heapallindexed => true) across 175 btree indexes —
zero errors.

Two build flags carry more weight than they look. OPTFLAGS="" because
pgvector's Makefile defaults to -march=native, which bakes the build host's
instruction set into vector.so and gives SIGILL mid-query on a host without it,
with an empty log. with_llvm=no because a missing .bc file makes the JIT skip
inlining silently, and inlining does not apply to index scans anyway.

Pinned at v0.8.6, not the earliest that works. v0.8.0 does not compile against
Postgres 18 at all (vacuum_delay_point gained a parameter); v0.8.1 is the first
that does. But 0.8.3 fixed possible index corruption with HNSW vacuuming and
0.8.4 fixed an unrepaired HNSW graph, and message_embedding is ON DELETE CASCADE
from message on a system that deletes mail in bulk — so vacuum is a hot path
here and shipping the earliest that merely compiles would ship two known
corruption bugs.

THE HANDBOOK'S 24x WAS WRONG AND IS CORRECTED TO 1.75x. The SIMD dot product is
microseconds; what dominates is that embedding is real[], so every row pays a
4KB detoast and a fresh real[] to vector conversion. 74,000 rows: 7.70s becomes
4.51s.

NO INDEX IS CREATED, and that is the finding rather than an omission. An HNSW
index answers ORDER BY ... LIMIT k and the search asks distance <= 0.45, which
no ANN index serves. The expression form needed to build one at all is
(embedding::vector(1024)), which Postgres treats as a DIFFERENT expression from
the inlined function's (embedding::vector) — so the index exists, the planner
ignores it, and nothing says so. And it costs 7m40s and 578MB to build at the
maintenance_work_mem this stack ships, behind the one advisory lock every
entrypoint waits on: an eight-minute dead stack, not a slow migration. The
measured route to an index is written down in ai-assist.md rather than half
taken.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 12:48:35 +02:00
Paul LütznerandClaude Opus 5 92d6e8ffab feat(ai): index new mail when the model is warm, and once a night
New mail was embedded the moment it arrived. That is the wrong trade on a box
with one GPU: it is rare to search for mail that has just landed, and paying to
index every arrival competes with the two things somebody is actually waiting
on.

Two triggers replace it. A nightly sweep is the backstop, and a search is the
invitation — a search has just loaded the embedding model, so the cheapest
possible moment to index is immediately after one, when the model is warm and
nobody is waiting on it any more.

THE PART THAT MADE THIS MORE THAN A SCHEDULE CHANGE
The backfill stood aside after ANY interactive AI use, and a search counted.
That is precisely backwards for search, and it took both halves of the signal to
say so: lastInteractiveCallAt() matched
`feature IN ('search_query','writing_help')`, and the request subscriber stamped
`app_mail_search` alongside `app_compose_assist`.

The distinction the old code could not make is that there are two models with
completely different costs. The composer spends the 20.3 GiB writing model,
which is worth standing aside for and is the case the yielding was written for.
Search spends the sub-gigabyte embedder — the same model indexing uses. So a
finished search is not a reason to keep away, it is the best moment there is.
Both docblocks now cross-reference each other, so restoring one half alone reads
as obviously wrong.

The opportunistic trigger claims its rate-limit marker BEFORE it queries, so the
mailbox scan is inside the window rather than only the dispatch, and a search
that produced no vector at all spends nothing — one unreachable moment must not
cost the next five minutes of indexing.

EmbedMailStep is deleted rather than emptied. An empty step would still be built
and iterated on every ingest batch while carrying a name asserting that
embedding happens on arrival — an active lie in the one directory somebody reads
to find out what happens to new mail.

The finder matches on model and not on width, deliberately. What consumes its
ids asks about the model alone, so a width-aware finder would hand over a full
budget the handler then drops as already done: a nightly sweep that reports
itself busy and indexes nothing, for ever. The width mismatch already has an
owner in the coverage report's "stale" count, which tells the user to re-index.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 01:50:25 +02:00
Paul LütznerandClaude Opus 5 377d6026b8 feat(ai): make the language model observable, and the composer honest
"I click the AI button in the composer and nothing happens" turned out not to be
about the model at all. #editor() queried data-compose--toolbar-target while the
composer renders data-compose--compose-toolbar-target, so run() returned at its
first guard: no request, no status, no error. The feature had never worked once.
#subject() was wrong the same way and quieter — every prompt went out with an
empty subject. And the status element lived INSIDE the dropdown that the same
click closes, so even once working it would have written into a hidden subtree.

The rest of this is what makes the next such thing findable.

RECORDING
One row per model call, tagged by the workload that caused it, and nothing about
what was in it — no prompt, no completion, no message id, no subject, no
address. error_kind is a category from a closed set for the same reason: an HTTP
client's exception message routinely quotes the request body back, and here that
is somebody's mail. OllamaClient was discarding Ollama's whole timing block; it
now returns it, and AiAssistant — the only place that knows both the feature and
the model — writes the row.

DBAL and deliberately no entity, following EmbeddingStore: BackfillEmbeddings
clears the EntityManager after every chunk, so an unflushed ORM row would be
detached and silently dropped, fifty at a time, for as long as a backfill runs —
which is exactly the window the panel exists to show.

AiFeature could not carry this. A four-character search query somebody is
waiting on and two thousand characters of mail body written unattended a hundred
thousand times both went through embed() with no discriminator, and averaged
together the second buries the first — the panel would report backfill
throughput under the heading "search". AiCallFeature splits them.

THE PANEL
Live state from /api/ps, which is passive and safe to poll. Six states that look
different, and the one that justifies the work is "partly on the CPU":
size_vram < size means layers spilled, throughput collapses, and the model still
answers correctly — invisible everywhere else, so everything is simply slow
forever.

THE COMPOSER
Tokens stream into a provisional surface and commit in one operation, because
hundreds of per-token insertions destroy the undo stack and Ctrl+Z is the first
thing anyone tries. Waiting, generating, done and failed are distinct states
with reasons: outside the keep-alive window the first thirteen seconds produce
nothing while 20 GiB loads, and a featureless spinner for that IS the bug.
Cancellation propagates to the host — an abandoned draft otherwise holds the GPU
and everything else queues behind it.

SEARCH
Silent degradation to keyword-only is replaced by the reason it degraded, and
rows found by meaning say so. The semantic arm and its join are now matched on
model AND width: vectors from different models are not comparable, and after
somebody changes the search model every existing vector is dead weight. On a
pgvector install comparing different widths raises an error — a 500 on
/mail/search caused by an admin dropdown — and on the shipped loop it silently
compares the overlap, which is worse. Coverage is stated where results are shown,
because semantic search over an 8%-indexed mailbox is a trap.

BACKFILL
Progress is counted from the database, never accumulated, so a killed worker
resumes. It stands aside while anyone is using the AI and comes back on its own:
both compete for one integrated GPU, and a batch in flight when somebody presses
the button puts them straight back to "nothing happens".

Also carries the cron line for app:jobs:reap from the commit before this one.

Verified against the real host: reachable, both configured models installed,
timings parsed, 1024-dimension embeddings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 00:38:43 +02:00
Paul LütznerandClaude Opus 5 6c33316f33 feat(settings): a second opacity slider, for windows and menus
One slider drove every translucent surface, and translucency multiplies: a
composer at 0.7 over a pane at 0.7 left about a tenth of the wallpaper landing
in the middle of the text being read.

So the knob splits. Structural surfaces — sidebar, top bar, main and calendar
panes — keep --pane-alpha. Floating ones — the compose window, modals, confirm
dialogs, toasts and twelve menus — take --popover-alpha, and both floating
utilities re-point --pane-alpha to it for their own subtree, so an inset strip
inside the composer belongs to the composer rather than to the sidebar behind
it.

The guarantee is a floor, not just a default: 0.5, so a popover can never be
made as see-through as a pane. Boxed spends the gap at 0.9 over its panes'
0.7 — about 3% of the background survives both layers instead of 10%.

The old stored value is deliberately NOT copied across. Twelve of the sixteen
floating surfaces never honoured it: `popover` has been hardcoded opaque since
it was written, for exactly the illegibility reason being reported. Backfilling
0.7 would have introduced the fault in twelve places to preserve it in four.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 00:38:02 +02:00
Paul LütznerandClaude Opus 5 672b38d6c2 test(ai): pin the refusals, and document the feature for the people using it
TESTS

The refusals are the half worth having. This produces text that goes into
somebody's outgoing mail and vectors that decide what a search finds, so
the failure that matters is not a clumsy sentence — it is a model
inventing a message because it was handed nothing to work from, or a
search quietly getting worse for an install that never switched any of
this on.

  · WritingAssistant: the three rewriting tasks refuse an empty draft
    without spending a request, a reply can still be drafted from the
    message being answered alone, a code fence is stripped and prose that
    merely mentions backticks is not, and the task list is closed —
    asserted as a count, because it is a security property rather than
    tidiness.
  · SemanticQuery: every reason a vector might be missing — off,
    unconfigured, host refusing the connection, query too short, a vector
    that cannot be normalised — arrives at the same null, because search
    must never be worse than it was before this feature existed.

DOCS

Admin -> AI, in both handbooks, written for somebody deciding whether
they want this rather than for somebody who has already decided. The
three switches are a table with a "when it runs" column, because that is
the difference that matters: sorting runs unattended on every arriving
message and the other two do not.

Says plainly that semantic search needs one pass over the existing
mailbox and that switching it on does nothing for mail already there —
which is the thing most likely to be reported as "it does not work".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 20:21:41 +02:00
Paul LütznerandClaude Opus 5 e7394254b4 docs: bring the AI page up to the code that now exists
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 19:30:26 +02:00
Paul LütznerandClaude Opus 5 ee370d7c5f docs: how the AI features are wired, and why the vector design avoids pgvector
Written now rather than at the end because the two decisions most likely
to be argued with — the model can only break a tie, and pgvector is
refused on an upgrade path that silently corrupts indexes — are both
easier to overturn before more is built on them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 18:10:19 +02:00
Paul LütznerandClaude Opus 5 5f648864ab feat(admin): set the log level from the page that shows the logs
Asked for directly: "let the admin panel set the log level. i do not want
to mess with env vars."

APP_DB_LOG_LEVEL meant editing a file on the host and restarting the
stack, and the moment it is actually wanted is the moment that is most
awkward — something is wrong, the answer is one level further down, and
restarting throws away whatever was in flight. So the level gets turned
down, forgotten, and left down.

WHAT IT IS NOT

Not another control in the filter row. That row narrows what is SHOWN out
of what was stored; this decides what gets STORED. They look alike and
mean opposite things, and side by side they would invite an admin to
change the wrong one and conclude the log was broken. Its own row, its own
form, its own wording.

NULL IS NOT "TODAY'S VALUE"

A stored level of null means "follow the environment", which is
deliberately different from copying APP_DB_LOG_LEVEL's current value into
the row. An install that has never chosen keeps following its
configuration including later changes to it, and there is a way back to
that after a level has been set — which storing the resolved value would
have removed, silently, on the day the env var next changed.

PER RECORD, NOT PER PROCESS

Monolog fixes a handler's level at construction. That is right for one
configured in a file and useless for one an administrator can turn down
from the page they are reading, so DoctrineLogHandler decides in
isHandling() instead, and the constructor's level is now only the answer
when no resolver was given — which keeps directly-constructed handlers,
and every existing test, behaving as they did.

LogLevelResolver caches for ten seconds. A query per log line would put a
database round-trip inside the thing that runs when the database is in
trouble; a value cached for the life of the process would leave every
long-lived worker ignoring the change until the next deploy, and the
worker logs are usually the ones it was lowered for. It uses DBAL, not the
ORM, for the same reason the handler does, and every failure — no table
yet, no database, a name that is not a level — falls back to the
environment rather than throwing. A logger that throws while deciding
whether to log is worse than one briefly out of date.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 17:21:00 +02:00
Paul LütznerandClaude Opus 5 8c2ef7663b feat(ops): a demo deploy that cannot quietly do nothing
Three commands in the handbook, one of which fails silently.

`docker compose pull` can run out of disk part-way through extracting a
layer. `up -d --wait` then brings the stack up on the image that is
ALREADY there and reports every container Healthy — so a deploy that
changed nothing is indistinguishable from one that worked, and the only
evidence is a line buried in the pull output. v0.1.36 was believed
deployed for twenty minutes while the demo went on serving v0.1.35.

Piping made it worse. Through a pipe the exit status belongs to whatever
came after it, so the `pull | tail -2` in the handbook's sequence
reported success no matter what the pull did. Nothing in the script is
piped.

And the disk fills on its own: every deploy leaves its predecessor behind
and nothing collects them. Eleven orphaned images is what filled it, on a
host that also runs Apache, MySQL and ClamAV and never has more than a
few GB spare.

So: check for room, prune if short, pull and STOP if that fails, up,
then clear the images the deploy replaced.

  · `-a` on the prune is load-bearing. A replaced release is a fully
    tagged image nothing references any more, and dangling-only leaves
    every one of them.
  · `--volumes` is never passed. The demo's database is in one.
  · compose.proxy.yaml is included by existence rather than by name: it
    belongs to a host that already has something on port 80 and is not in
    this repository.

Verified on the real demo host, both ways — a normal run reports "image
unchanged" and exits 0, and DEMO_MIN_FREE_MB=999999 refuses, exits 1, and
leaves the running stack alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 11:11:09 +02:00
Paul LütznerandClaude Opus 5 e8f4033f03 docs: changelog and log-entry reference for v0.1.36
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 10:10:55 +02:00
Paul LütznerandClaude Opus 5 561fc3f372 feat(mail): a message that bounced now says so, on the message that bounced
plMail could tell you a message had been READ. It could not tell you the
opposite. A bounce arrived in the Inbox as a machine-generated mail
titled "Undelivered Mail Returned to Sender" whose body is an SMTP
transcript, and the message it was about went on sitting in Sent saying
"Sent" — which at that point is a lie. The one delivery outcome a person
has to act on was the one outcome attached to nothing.

The structure was already here. A DSN is ReadReceiptCorrelator's twin:
same multipart/report envelope, same Original-Message-ID lookup, same
"stamp the sent row" payoff. BounceCorrelator is a separate service
anyway, because the two disagree on every judgement that matters.

ONLY `Action: failed`

A DSN reports four things and three of them are not bounces. `delayed`
in particular is a message STILL IN FLIGHT — Postfix emits one after
four hours and keeps trying for five days — and stamping it as
undelivered would tell the user their mail failed at the exact moment it
is being retried, with nothing to clear the stamp when it went through.
`delivered` and `relayed` are good news and set nothing.

THE BOUNCE IS NOT FILED AWAY

A read receipt is noise once correlated and gets marked read and taken
off the Inbox. A bounce stays exactly where it landed, unread. Its body
is routinely the only readable statement of what went wrong, and a
failure the user may have to act on is not something to tidy up on their
behalf.

NOTHING IS CLASSIFIED

The status is stored as the reporting MTA wrote it — "5.1.1", "4.4.7" —
and never reduced to a boolean "permanent". No address is disabled,
nothing is removed from a contact list, no resend is attempted. plMail
records what one server said about one delivery attempt. Deciding an
address is dead on that basis is a conclusion this codebase has been
wrong about before, and the sweep that led here said so repeatedly.

A BOUNCE WAS BEING READ AS A READ RECEIPT

Found by the first fixture written against this, and reachable before
it. Both are multipart/report and both carry Original-Message-ID, so a
DSN satisfied ReadReceiptCorrelator's body fallback, got correlated
there, and put "Read at …" on a message that was never delivered — not
merely missing information but the precise opposite of the truth.

The two field blocks have mutually exclusive discriminators, so that is
what tells them apart now: `Disposition:` is an MDN, `Action:` is a DSN.
Both arms of the detection refuse a delivery-status report first.

ON THE DEMO

The demo seeds a reply sent to a typo'd domain and the MAILER-DAEMON
answer four minutes later, correlated by the real BounceCorrelator
rather than by writing the columns directly. A demo that stamps the
result itself would stay green while the feature broke; this way the red
panel disappearing from the demo IS the alarm.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-26 00:52:32 +02:00
Paul LütznerandClaude Opus 5 cd6be7f6a1 fix: a partial OAuth grant was accepted in silence, and broke things quietly
The report: three Google calendars saying "stopped syncing" with a 403,
and selecting 5,455 unread conversations, pressing mark-as-read, waiting,
and finding nothing changed.

One cause. Google's consent screen offers sensitive scopes as individual
tick boxes and a declined one does not fail the sign-in — the account
connects, plMail says so, and the withheld permission is discovered days
later as something that does not work. plMail asked for the scopes and
threw the answer away: the token response says what was actually granted
and nothing read it.

WHAT IT COST

Calendars never synced, because the calendar scope was never given.
Worse, and this is the half that had no explanation at all: the grant
could not WRITE, so Gmail refused messages.batchModify with
insufficientPermissions. The change applied locally, never reached
Gmail, and was undone by the next sync. Five thousand conversations
marked read, and afterwards not one of them was.

WHAT NOW HAPPENS

Account::$oauthGrantedScopes records what the provider gave us, and
MailProvider::missingScopes() compares it with what was asked for.
Account health raises a card naming the account with a reconnect on it.
A flash says the same thing at the moment of connecting, when the user
still remembers the screen they just clicked through.

Backfilled by the refresh, which is the only place it can come from: the
granted set is in the response that delivered the token, and that
response is long gone for every existing account. Both providers return
`scope` on a refresh, so a working account fills itself in within the
hour, with no migration inventing an answer.

AND IT NO LONGER FAILS SILENTLY

A permanently refused export was a log line and nothing else, so the
local-versus-remote divergence was invisible to the person it happened
to. ApplyGmailLabelsHandler records the refusal on the account now and
clears it on the next export that works; the health card reads that too,
which is what makes this useful on an install that has been broken for
weeks rather than only on the next account somebody connects.

FOUR CARDS BECOME ONE

Calendars blocked by an account-level cause were already demoted to
consequences — and still printed one card each, so one missing
permission read as four problems, three of them offering a retry whose
entire behaviour is to fail. They collapse into a single card that names
them. A lone blocked calendar keeps its own card: "Familie has stopped
syncing" says more than "1 calendar is not syncing".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 19:08:42 +02:00
Paul LütznerandClaude Opus 5 e00fe25a14 feat: save a signature once instead of drawing it on every document
Settings → Profile gains a pad beside the avatar. Draw a signature
there and the PDF reader offers "Use saved signature", which places it
without asking again; remove it from the same place.

It goes through exactly the same placement path as a fresh scribble,
because a stamp has been an IMAGE rather than a set of strokes since the
day signing was written. That was the point of expressing it that way:
this is six lines, not a second implementation to keep in step. The two
pads share trimAlpha from assets/pdf_geometry.js for the same reason —
they have to agree about what a stamp is.

Deliberately NOT the "Signature" section in the sidebar. That one is the
block of text appended to outgoing mail; the two are unrelated and the
wording keeps them apart.

SignatureStorage is AvatarStorage's shape — one file per user under
var/uploads, replaced in place, random filename so a changed image is
not served from cache. PNG only, and that is a requirement rather than a
narrow allow-list: the image is stamped onto somebody else's document
and anything without alpha lands as a white card over their text.

It is not a credential. There is nothing to verify against it and it is
encrypted by nothing, which is the honest treatment for a picture of a
squiggle.

FILES OUTLIVE ROWS

Erasing a demo visitor now deletes their avatar and signature too.
DemoUserEraser finds what to delete from Doctrine's metadata, and a PNG
under var/uploads is not a row — so on a public demo minting a throwaway
user per visitor, every signature drawn was staying on disk for good.

TWO BUGS THE TESTS HID

The save endpoint redirected to a route named `app_settings`, which does
not exist — every POST was a 500. Three of the four tests passed anyway,
satisfied by a signature an earlier failed run had left on disk. They
clear any saved signature in beforeEach now, so "it saved" cannot pass
without anything being saved.

And the first version could not draw at all: the pad sits below the
fold, page.mouse works in viewport coordinates, and boundingBox()
happily reports a box off-screen. The strokes landed nowhere.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 16:38:46 +02:00