On demand, and that is the design rather than a first step. A summary spends the
20.3 GiB writing model for about a minute cold, on one GPU already shared with
the indexer and the composer. Summarising every arriving thread would spend days
of it on conversations nobody opens.
FRESHNESS IS DERIVED, NOT MAINTAINED
The stored row carries a SHA-256 of the exact transcript that was sent, plus the
model tag and a prompt version, and freshness is that hash matching what the
thread would send today. Nothing has to remember to invalidate anything.
lastMessageAt was the obvious key and is the wrong one: it only ever moves
FORWARD, so a deleted message leaves a summary describing mail that is no longer
there, silently and for ever. The hash also makes a prompt change or a budget
change invalidate every stored summary by itself, which is correct and free.
CASCADE, following message_embedding rather than MailInsight. The two
precedents point opposite ways and both are right — an insight outlives its
message because "a deleted shipping confirmation does not un-ship the parcel",
an embedding does not because it describes one. A summary describes a thread.
THE BUDGET IS MEASURED, NOT PICKED
8000 characters, against two ceilings and the smaller one is silent. The visible
one is the wait before the first token — load plus prompt eval, about 42s at
this budget, a third of the client's idle timeout. The quiet one is the context
window: OllamaClient sends no num_ctx and Ollama's default is 4096 tokens, so
12000 characters sits exactly on the line, and an install running the default
would drop the HEAD of the transcript — the part the reader most wants and the
part the transcript is assembled forwards to protect. No error, no notice.
The fourth AiFeature case landed LAST, after every arm that reads it. Eight
exhaustive matches key on that enum and none of them fails at compile time;
AiPermissions::anyAdminEnabled() loops the cases and the settings nav calls it
on every settings page, so the case arriving early would have taken the whole
settings area down rather than the AI section.
Verified against the real host, not a mock: a German thread returns a German
summary, a reopened thread renders the stored row with no new model call, a
reply greys it and offers to run again, and changing the chat model hides it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
