Offer to summarise the whole conversation, and send a context window with it

The card now says when a summary was written from part of a thread, and the way
out sits beside the notice that explains why it is there.

num_ctx IS THE WHOLE MECHANISM. This client has never sent one — four docblocks
across the codebase size their budgets around "OllamaClient sends no num_ctx, so
the model's default decides what survives", and that default is 4096 tokens with
everything past it dropped silently. So "send the whole conversation" without
also asking for somewhere to put it would send more, have it truncated at a
point nobody chose, and spend a much longer wait to produce no more than before.
It is sent only when a caller asks, so every existing call keeps its shape.

FULL IS STILL BOUNDED, and that is the honest part rather than a caveat. A
context window is memory: Ollama allocates the KV cache for the whole of it
whether the prompt fills it or not, so an unbounded "send everything" lets one
absurd thread ask a modest host for an allocation it cannot make — and a failed
allocation costs the reader the entire wait and produces nothing, where a
clipped transcript produces a summary and a sentence saying it is partial. The
ceiling is 96,000 characters against a 32,768-token window, twelve times the
ordinary budget. Past it the run is full AND partial, and the button withdraws
rather than inviting the same wait twice.

The window is SIZED to the transcript rather than fixed, since it is reserved
whole either way and a two-message thread should not cost what a hundred-message
one does. Sized at 3.0 characters per token, deliberately below the 3.55
measured on real German business mail: this divides to get tokens, so erring low
overestimates the need and asks for slightly more room than strictly required.
Erring the other way would silently truncate inside the one feature whose entire
purpose is not to.

thread_summary gains full_context, and it cannot be derived like the rest of the
card. "Is this thread too long to send whole" is a question about the thread and
comes free from the transcript the freshness check already builds. "Did THIS
summary see all of it" is a question about the row, and the thread cannot answer
it — a thread that is still too long is still too long after somebody presses
the button and gets a summary that did see everything. Without the column that
press would appear to do nothing: new text, same notice, much longer wait.

A press and not a setting, deliberately. It is a fair thing to spend once on a
thread somebody cares about and a poor default for every summary in a mailbox,
so the cost is written into the label and nothing is remembered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Paul LütznerandClaude Opus 5 committed 2026-09-03 12:43:18 +02:00
1 parent 5ca9aab2ee
commit 2bfc30ad8d
16 files changed
+441 -50

No files matched your search

@@ -69,7 +69,7 @@ export default class extends Controller {
"status",
"pending",
"output",
"stale", "partial",
"stale", "partial", "full",
"run",
"runLabel",
"stop",
@@ -200,6 +200,26 @@ export default class extends Controller {
this.#start();
}
/**
* Write one from the whole conversation, however long it is.
*
* Offered only beside the notice saying the last one was not — the reader
* learns something was left out and the way to fix it is in the same
* breath, rather than in a setting they would have to know exists.
*
* A SEPARATE ACTION AND NOT A CHECKBOX, because it is not a preference. It
* costs a much longer wait and asks the model host to reserve a far larger
* context window, which on a modest machine is slow and can fail to
* allocate. That is a fair thing to spend once on a thread somebody cares
* about and a poor default for every summary in the mailbox, so it is a
* press with a warning on it and nothing is remembered.
*/
full(event) {
event?.preventDefault();
this.#start(true);
}
/**
* Stop, and keep nothing.
*
@@ -217,7 +237,7 @@ export default class extends Controller {
this.#say(this.stoppedLabelValue);
}
async #start() {
async #start(full = false) {
// Every run past the first is a REPLACEMENT, so the previous one's
// fetch has to go before this one's counter moves — otherwise two
// streams write into the same card.
@@ -260,12 +280,14 @@ export default class extends Controller {
"Content-Type": "application/x-www-form-urlencoded",
"X-CSRF-Token": this.tokenValue,
},
// A form-encoded body with nothing in it. Everything the server
// needs is the thread id already in the URL and the person
// already in the session: what gets summarised is never taken
// from the page, or anything that could post here would choose
// what the model is told about somebody's mail.
body: new URLSearchParams({}),
// ONE field, and it is not "what to summarise". Everything the
// server needs to decide THAT is the thread id already in the
// URL and the person already in the session: what gets
// summarised is never taken from the page, or anything that
// could post here would choose what the model is told about
// somebody's mail. How much of their own thread to send is a
// different question, and it is the one the card asks.
body: new URLSearchParams(true === full ? { full: "1" } : {}),
signal: this.#controller.signal,
});
@@ -362,6 +384,13 @@ export default class extends Controller {
// transcript is the only one that describes what was just read.
this.#partial(true === frame.partial);
// The offer is only worth making while it would change something.
// A run that already sent everything has nothing more to send, and
// a thread past the ceiling is still partial after a full run — so
// the button goes either way, and the notice standing alone says
// the honest thing: this is as much as it can see.
this.#offerFull(true === frame.partial && true !== frame.full);
// No sentence at all. A finished summary explains itself, and a
// status line reading "Summarised" over the summary is furniture.
this.#settle("");
@@ -585,6 +614,13 @@ export default class extends Controller {
this.partialTarget.hidden = false === on;
}
/** The "send the whole thing" offer, which only stands while it would help. */
#offerFull(on) {
if (false === this.hasFullTarget) return;
this.fullTarget.hidden = false === on;
}
/** Stop replaces Summarise for the length of a run, and never sits beside it. */
#stopping(on) {
if (true === this.hasStopTarget) {
+29 -5
View File
@@ -1,4 +1,4 @@
<!-- translated-from: features/mail.md sha1:75906ba33b88ccd8efd121f0ba99cfa471a42b57 -->
<!-- translated-from: features/mail.md sha1:58b0384fc5c26659556b26d1f222ea00bc7d69ce -->
# Mail
@@ -236,10 +236,34 @@ Stück an, du siehst also, dass etwas passiert. **Anhalten** bricht ab und behä
Die Zusammenfassung entsteht aus den Nachrichten selbst und aus sonst nichts — nicht aus deinen
Notizen unter **Einstellungen → Assistent**, die dafür da sind, wie *du* angeschrieben werden
möchtest, und die in einer Beschreibung fremder Mail nichts zu suchen haben. Ein langes Gespräch
wird aus seinem Anfang und seinen jüngsten Nachrichten zusammengefasst, und wenn die Mitte
wegfallen musste, sagt die Zusammenfassung das und sagt, um wie viele Nachrichten es ging. Ein
deutsches Gespräch bekommt eine deutsche Zusammenfassung.
möchtest, und die in einer Beschreibung fremder Mail nichts zu suchen haben. Ein deutsches Gespräch
bekommt eine deutsche Zusammenfassung.
#### Wenn das Gespräch zu lang ist, um es ganz zu schicken
Es gibt eine Grenze dafür, wie viel von einem Gespräch auf einmal zum Modell geht, und die sucht
plMail sich nicht aus: Ein Modell kann nur eine bestimmte Menge Text auf einmal halten, und alles
darüber fällt kommentarlos weg — eine Zusammenfassung eines übergroßen Gesprächs würde also
stillschweigend beschreiben, welches Ende zufällig übrig geblieben ist. Statt das passieren zu
lassen, kürzt plMail vorher und sagt, dass es gekürzt hat.
Zwei Arten von Kürzung, und die Karte benennt beide:
- Ein langes Gespräch wird aus **Anfang und jüngsten Nachrichten** zusammengefasst; die Mitte fällt
weg, mit einem Hinweis, um wie viele Nachrichten es ging.
- Eine einzelne riesige Nachricht — eine Kette, die ein Dutzend Mal weitergeleitet und zitiert wurde,
ist *eine* Nachricht und nicht zwölf — wird **abgeschnitten**.
In beiden Fällen steht auf der Karte *„Das Gespräch war zu lang, um es ganz zu schicken“*, und
darunter **Stattdessen das ganze Gespräch zusammenfassen**.
Dieser Knopf tut, was er sagt: Er schickt alles und bittet den Modell-Host, dafür Platz zu machen.
Rechne mit deutlich mehr Zeit, und damit, dass es der Maschine hinter dem Modell mehr abverlangt —
auf einer schwächeren kann das langsam genug sein, dass es sich nur für ein Gespräch lohnt, das du
wirklich brauchst. Es wird nichts gemerkt: Es gilt für die eine Zusammenfassung, um die du gebeten
hast, und für keine andere. Ist das Gespräch so lang, dass selbst das nicht reicht, bekommst du
trotzdem eine Zusammenfassung, der Hinweis bleibt stehen, und der Knopf verschwindet, statt dich
zweimal dasselbe abwarten zu lassen.
Einmal geschrieben, wird sie aufbewahrt und beim nächsten Öffnen der Konversation ohne Wartezeit
wieder angezeigt. Kommt danach eine Antwort, siehst du sie ausgegraut mit dem Hinweis, dass sich das
+25 -3
View File
@@ -212,9 +212,31 @@ it and keeps nothing.
The summary is written from the messages themselves and nothing else — not from your notes in
**Settings → Assistant**, which are for how *you* want to be written for and have no business
shaping a description of somebody else's mail. A long conversation is summarised from its opening
and its most recent messages, and if the middle had to be left out the summary says so and says how
many messages it was. A German conversation gets a German summary.
shaping a description of somebody else's mail. A German conversation gets a German summary.
#### When the conversation is too long to send whole
There is a limit on how much of a conversation goes to the model at once, and it is not plMail's
choice: a model can only hold so much text at a time, and anything past that is dropped without a
word — so a summary of a conversation that overran would quietly describe whichever end happened to
survive. Rather than let that happen silently, plMail trims first and says that it did.
Two kinds of trimming, and the card names either:
- A long conversation is summarised from its **opening and its most recent messages**, and the middle
is left out with a note saying how many messages it was.
- A single enormous message — a chain that has been forwarded and quoted a dozen times is one
message, not a dozen — is **cut short**.
When either happens, the card says *"This conversation was too long to send in full, so the summary
was written from part of it"*, with **Summarise the whole conversation instead** underneath.
That second button does what it says: it sends everything and asks the model host to make room for
it. Expect it to take noticeably longer, and expect it to ask more of the machine running the model —
on a modest one it may be slow enough to be worth doing only for a conversation you actually need.
Nothing is remembered, so it applies to the one summary you asked for and no others. If the
conversation is so long that even this cannot hold it, you still get a summary, the note stays, and
the button withdraws rather than inviting the same wait twice.
Once written, it is kept and shown the next time the conversation is opened, at no cost. If a reply
arrives afterwards, the summary is shown greyed with a note that the conversation has changed and a
+47
View File
@@ -0,0 +1,47 @@
<?php
declare(strict_types=1);
namespace DoctrineMigrations;
use Doctrine\DBAL\Schema\Schema;
use Doctrine\Migrations\AbstractMigration;
/**
* Remember whether a stored summary was written from the whole conversation.
*
* WHY THIS CANNOT BE DERIVED LIKE THE REST OF THE CARD
* ───────────────────────────────────────────────────
* "Is this conversation too long to send in full" is a question about the
* thread, and MailController answers it for free from the transcript the
* freshness check already builds. "Was this particular summary written from all
* of it" is a question about the ROW, and the thread cannot answer it: a thread
* that is still too long is still too long after somebody presses "summarise
* the whole conversation" and gets a summary that did see all of it.
*
* Without this column that press would appear to do nothing. The summary would
* change, the notice saying the model had not seen everything would stay, and
* the reader would have spent a much longer wait for a card that still says the
* work was not done.
*
* DEFAULT FALSE, which is true of every row already stored: nothing before this
* release could send a full conversation, because nothing before it sent
* num_ctx at all.
*/
final class Version20260904140000 extends AbstractMigration
{
public function getDescription(): string
{
return 'Record whether a stored thread summary was written from the whole conversation.';
}
public function up(Schema $schema): void
{
$this->addSql('ALTER TABLE thread_summary ADD full_context BOOLEAN DEFAULT FALSE NOT NULL');
}
public function down(Schema $schema): void
{
$this->addSql('ALTER TABLE thread_summary DROP full_context');
}
}
@@ -119,7 +119,15 @@ final class ThreadSummaryController extends AbstractController
// read, and captured into the closure. It is needed twice — as the
// prompt and as the thing hashed — and building it twice would be two
// versions of one fact that disagree the first time either is tuned.
$transcript = $this->transcript->forThread($thread);
// WHAT THE READER ASKED FOR, and the only thing this endpoint takes
// off the request. The thread comes from the URL and the person from
// the session — see the body comment in the controller that calls this
// — so a post that chose WHICH mail to summarise would be a way to aim
// the model at somebody else's. Choosing how much of their own thread
// to send is not that, and it is the one decision the card offers.
$full = $request->request->getBoolean('full');
$transcript = $this->transcript->forThread($thread, $full);
$sourceHash = ThreadTranscript::hash($transcript);
$model = $this->summariser->model();
@@ -158,8 +166,8 @@ final class ThreadSummaryController extends AbstractController
// ComposeAssistController captures its user: reading a property inside
// the callback is free, while a repository lookup there would be a
// query against a kernel that has finished with the request.
$response = new StreamedResponse(function () use ($user, $thread, $transcript, $sourceHash, $model, $promptHash): void {
$this->generate($user, $thread, $transcript, $sourceHash, $model, $promptHash);
$response = new StreamedResponse(function () use ($user, $thread, $transcript, $sourceHash, $model, $promptHash, $full): void {
$this->generate($user, $thread, $transcript, $sourceHash, $model, $promptHash, $full);
});
// Not application/json: this is a sequence of JSON documents and never
@@ -190,7 +198,7 @@ final class ThreadSummaryController extends AbstractController
* had gone. A local in a method is freed when the method returns, and
* freeing it is what cancels the call.
*/
private function generate(User $user, MessageThread $thread, string $transcript, string $sourceHash, string $model, string $promptHash): void
private function generate(User $user, MessageThread $thread, string $transcript, string $sourceHash, string $model, string $promptHash, bool $full = false): void
{
// PHP's default is to kill the script the moment a write finds the
// browser gone, which sounds like what we want and is the opposite of
@@ -231,7 +239,7 @@ final class ThreadSummaryController extends AbstractController
// cold load, so this only ever fires when that has failed to.
set_time_limit(self::STREAM_TIME_LIMIT_SECONDS);
$tokens = $this->summariser->stream($user, $thread, $transcript);
$tokens = $this->summariser->stream($user, $thread, $transcript, $full);
if (null === $tokens) {
// Switched off between the check above and here, or a thread whose
@@ -313,7 +321,7 @@ final class ThreadSummaryController extends AbstractController
// nothing the session holds. Its failure is logged and swallowed inside
// the store: the reader already has their summary on screen, and a
// caching miss must not become a 500 on a response that is half sent.
$this->store->save((int) $thread->id, $result->content, $sourceHash, $model, $promptHash);
$this->store->save((int) $thread->id, $result->content, $sourceHash, $model, $promptHash, $full);
$this->frame([
'type' => 'done',
@@ -323,6 +331,12 @@ final class ThreadSummaryController extends AbstractController
// "written from part of it" is the one caveat they cannot get from
// reading the paragraph.
'partial' => ThreadTranscript::isPartial($transcript),
// Whether this run sent the whole conversation, so the card knows
// the offer has been taken up and stops making it. Distinct from
// `partial`: a thread past FULL_TRANSCRIPT_CEILING is both — sent
// in full mode and still trimmed — and the reader needs to be told
// that pressing it again will not help.
'full' => $full,
// The whole answer again, tidied. The browser has the raw tokens
// and could join them itself, but tidy() strips a code fence that
// can only be recognised from both ends — so what is SHOWN from
+2 -1
View File
@@ -248,7 +248,7 @@ final readonly class AiAssistant
*
* @return \Generator<int, string, void, AiChatResult>|null
*/
public function chatStream(AiFeature $feature, array $messages, ?float $temperature = null): ?\Generator
public function chatStream(AiFeature $feature, array $messages, ?float $temperature = null, ?int $numCtx = null): ?\Generator
{
if (AiFeature::Search === $feature) {
$this->logger->error('AiAssistant: a streamed chat was requested on behalf of the search feature', [
@@ -279,6 +279,7 @@ final readonly class AiAssistant
$messages,
$temperature,
$settings->keepAliveFor($feature),
$numCtx,
),
);
}
+27 -2
View File
@@ -461,7 +461,7 @@ final readonly class OllamaClient
*
* @return \Generator<int, string, void, AiChatResult>
*/
public function chatStream(string $baseUrl, string $model, array $messages, ?float $temperature = null, ?string $keepAlive = null): \Generator
public function chatStream(string $baseUrl, string $model, array $messages, ?float $temperature = null, ?string $keepAlive = null, ?int $numCtx = null): \Generator
{
$payload = self::withKeepAlive([
'model' => $model,
@@ -469,8 +469,33 @@ final readonly class OllamaClient
'stream' => true,
], $keepAlive);
$options = [];
if (null !== $temperature) {
$payload['options'] = ['temperature' => $temperature];
$options['temperature'] = $temperature;
}
// THE FIRST num_ctx THIS CLIENT HAS EVER SENT, and its absence was
// load-bearing: four docblocks across the codebase size their budgets
// around "OllamaClient sends no num_ctx, so the model's default decides
// what survives". That default is Ollama's 4096, and everything past it
// is dropped with no error, no warning and no way for the caller to
// find out — which is why every budget here is small and why a thread
// too long to fit was summarised from whichever end happened to remain.
//
// Sent ONLY when a caller asks for it, so nothing that worked before
// changes shape. It is not free: the KV cache is allocated for the
// whole window whether the prompt fills it or not, so a large value on
// a modest host is slower and can fail to allocate at all. That is why
// no default is applied here and why the one caller that passes it —
// the full-conversation summary — is a button somebody presses rather
// than a setting that applies to every request.
if (null !== $numCtx) {
$options['num_ctx'] = $numCtx;
}
if ([] !== $options) {
$payload['options'] = $options;
}
$whole = '';
+79 -2
View File
@@ -92,6 +92,71 @@ final readonly class ThreadSummariser
*/
public const int TRANSCRIPT_BUDGET = 8000;
/**
* The most a full-conversation run will send, in characters.
*
* "Full" is bounded, and the bound is the point rather than a caveat. The
* whole reason this option exists is that the ordinary budget is sized for
* a context window plMail cannot see and does not set — but the cure is
* asking for a bigger window, and a window is MEMORY. Ollama allocates the
* KV cache for the whole of num_ctx whether the prompt fills it or not, so
* an unbounded "send everything" would let one absurd thread ask a modest
* host for an allocation it cannot make, and the failure is the host
* refusing or thrashing rather than anything plMail can catch.
*
* 96,000 characters is about 32,000 tokens at the 3.0 chars/token this
* sizes with, which lands the request just inside FULL_MAX_CONTEXT_TOKENS
* once the prompt and the answer are reserved. It is twelve times the
* ordinary budget: a conversation that does not fit in it is not a thread
* anybody is going to read either, and it still gets a summary — clipped,
* and the card still says so.
*/
public const int FULL_TRANSCRIPT_CEILING = 96000;
/**
* The largest context window a full run will ask a host for.
*
* Chosen as a number the models plMail is used with actually support and a
* mid-range GPU can still allocate. Above this the honest move is to send
* less rather than to ask for a window the host may not have — a request
* that fails to allocate costs the reader the whole wait and produces
* nothing, where a clipped transcript produces a summary and a sentence
* saying it is partial.
*/
private const int FULL_MAX_CONTEXT_TOKENS = 32768;
/**
* Characters per token, for sizing num_ctx only.
*
* DELIBERATELY BELOW the 3.55 measured on real German business mail, and
* the direction matters: this divides characters to get tokens, so a
* smaller number OVERESTIMATES how many tokens the transcript needs and
* asks for a slightly larger window than strictly necessary. Erring the
* other way would silently truncate the conversation inside a feature whose
* entire purpose is not to, which is the failure this exists to end.
*/
private const float CHARS_PER_TOKEN = 3.0;
/** The system prompt plus room for the summary being written, in tokens. */
private const int FULL_CONTEXT_RESERVE_TOKENS = 800;
/**
* The context window to ask for, for a transcript of this size.
*
* Rounded up to a whole number of 1024-token blocks, which is how these
* windows are conventionally sized and keeps the request from looking like
* a number somebody typed. Never below Ollama's own 4096 default: asking
* for less than the host would have used anyway can only make a short
* conversation worse.
*/
public static function contextFor(string $transcript): int
{
$tokens = (int) ceil(mb_strlen($transcript) / self::CHARS_PER_TOKEN) + self::FULL_CONTEXT_RESERVE_TOKENS;
$blocks = (int) ceil($tokens / 1024) * 1024;
return max(4096, min(self::FULL_MAX_CONTEXT_TOKENS, $blocks));
}
/**
* A conversation with one message in it is not summarised.
*
@@ -262,7 +327,7 @@ final readonly class ThreadSummariser
*
* @return \Generator<int, string, void, AiChatResult>|null
*/
public function stream(User $user, MessageThread $thread, string $transcript): ?\Generator
public function stream(User $user, MessageThread $thread, string $transcript, bool $full = false): ?\Generator
{
$messages = $this->messagesFor($user, $thread, $transcript);
@@ -270,7 +335,19 @@ final readonly class ThreadSummariser
return null;
}
$tokens = $this->ai->chatStream(AiFeature::Summary, $messages, self::TEMPERATURE);
// num_ctx ONLY for a full run, and sized from the transcript in hand.
//
// Not sent otherwise, so every ordinary summary keeps the shape it has
// always had — the budget above is chosen to fit the 4096 default and
// sending a window would change nothing except the memory the host
// reserves. A full run is the case where the default is the problem,
// and it is the reader's own deliberate press.
$tokens = $this->ai->chatStream(
AiFeature::Summary,
$messages,
self::TEMPERATURE,
true === $full ? self::contextFor($transcript) : null,
);
// Not a generator function, for the reason AiAssistant::chatStream()
// gives: the refusals above have to happen when this is CALLED, not
+19 -11
View File
@@ -97,7 +97,7 @@ final readonly class ThreadSummaryStore
try {
$row = $this->connection->fetchAssociative(
<<<'SQL'
SELECT summary, source_hash, created_at
SELECT summary, source_hash, created_at, full_context
FROM thread_summary
WHERE thread_id = :id
AND model = :model
@@ -141,7 +141,13 @@ final readonly class ThreadSummaryStore
// one is reachable by an attacker.
hash_equals((string) $row['source_hash'], $sourceHash),
$writtenAt,
$isPartial,
// Partial only if the conversation is trimmable AND this row was
// not written from all of it. Both halves are needed: the thread
// answers the first and only the row can answer the second, and a
// full run on a thread that is still too long to trim normally
// would otherwise keep a notice saying the model had not seen
// everything — after a much longer wait during which it did.
$isPartial && false === (bool) $row['full_context'],
);
}
@@ -156,19 +162,20 @@ final readonly class ThreadSummaryStore
*
* @return bool whether anything was stored
*/
public function save(int $threadId, string $summary, string $sourceHash, string $model, string $promptHash): bool
public function save(int $threadId, string $summary, string $sourceHash, string $model, string $promptHash, bool $fullContext = false): bool
{
try {
$this->connection->executeStatement(
<<<'SQL'
INSERT INTO thread_summary (thread_id, summary, source_hash, model, prompt_hash, created_at)
VALUES (:id, :summary, :hash, :model, :prompt, :now)
INSERT INTO thread_summary (thread_id, summary, source_hash, model, prompt_hash, full_context, created_at)
VALUES (:id, :summary, :hash, :model, :prompt, :full, :now)
ON CONFLICT (thread_id) DO UPDATE
SET summary = EXCLUDED.summary,
source_hash = EXCLUDED.source_hash,
model = EXCLUDED.model,
prompt_hash = EXCLUDED.prompt_hash,
created_at = EXCLUDED.created_at
SET summary = EXCLUDED.summary,
source_hash = EXCLUDED.source_hash,
model = EXCLUDED.model,
prompt_hash = EXCLUDED.prompt_hash,
full_context = EXCLUDED.full_context,
created_at = EXCLUDED.created_at
SQL,
[
'id' => $threadId,
@@ -176,9 +183,10 @@ final readonly class ThreadSummaryStore
'hash' => $sourceHash,
'model' => $model,
'prompt' => $promptHash,
'full' => $fullContext,
'now' => (new DateTimeImmutable())->format('Y-m-d H:i:s'),
],
['id' => ParameterType::INTEGER],
['id' => ParameterType::INTEGER, 'full' => ParameterType::BOOLEAN],
);
return true;
+28 -12
View File
@@ -83,9 +83,9 @@ final readonly class ThreadTranscript
* thread whose replies were all at the end would describe an argument in
* which one side spoke only after the other had finished.
*/
public function forThread(MessageThread $thread): string
public function forThread(MessageThread $thread, bool $full = false): string
{
return $this->forMessages($this->messages->forThreadInConversationOrder($thread));
return $this->forMessages($this->messages->forThreadInConversationOrder($thread), $full);
}
/**
@@ -99,15 +99,26 @@ final readonly class ThreadTranscript
*
* @param list<Message> $messages oldest first
*/
public function forMessages(array $messages): string
public function forMessages(array $messages, bool $full = false): string
{
$blocks = array_map(self::block(...), $messages);
// FULL IS STILL BOUNDED, and calling it "full" anyway is deliberate: it
// is the whole conversation for every thread anyone actually has, and
// the ceiling exists so that the one thread that is not cannot ask the
// host for a context window it has no memory to allocate. Above it the
// clip still applies and the card still says the summary is partial,
// which is the honest answer rather than a promise this cannot keep.
// See ThreadSummariser::FULL_TRANSCRIPT_CEILING.
$limit = $full ? ThreadSummariser::FULL_TRANSCRIPT_CEILING : self::turnLimit();
$blocks = array_map(static fn (Message $turn): string => self::block($turn, $limit), $messages);
if ([] === $blocks) {
return '';
}
return implode("\n\n", self::trimmed($blocks));
// Whole turns are only ever dropped to fit the ordinary budget. In full
// mode the point is that nothing is dropped, so every turn stands and
// the ceiling above is what holds the total down.
return implode("\n\n", $full ? $blocks : self::trimmed($blocks));
}
/**
@@ -161,7 +172,13 @@ final readonly class ThreadTranscript
* a gap in the conversation, and "somebody replied here and we cannot show
* you what they said" is more use to a model than a turn that never appears.
*/
private static function block(Message $turn): string
/** Two thirds of the ordinary budget. See block() for why a turn is capped at all. */
private static function turnLimit(): int
{
return intdiv(ThreadSummariser::TRANSCRIPT_BUDGET * 2, 3);
}
private static function block(Message $turn, int $limit): string
{
$body = trim((string) ($turn->bodyText ?? ''));
@@ -183,12 +200,11 @@ final readonly class ThreadTranscript
// whatever end of the chain survived the truncation and summarises
// that, and the summary is confidently about the wrong message.
//
// Two thirds of the whole budget, so a single turn can still be most of
// a conversation — a thread that really is one long mail and one "yes"
// should summarise the long mail — while two oversized turns together
// cannot exceed it by more than a third.
$limit = intdiv(ThreadSummariser::TRANSCRIPT_BUDGET * 2, 3);
// Two thirds of the whole budget by default, so a single turn can still
// be most of a conversation — a thread that really is one long mail and
// one "yes" should summarise the long mail — while two oversized turns
// together cannot exceed it by more than a third. The caller raises it
// for a full-conversation run.
if (mb_strlen($body) > $limit) {
// Cut from the END, keeping the head. Same argument as trimmed()
// makes for keeping the opening turn: a forwarded chain is newest
+23
View File
@@ -226,6 +226,29 @@ that is there on reopening cannot drift apart.
class="text-[11px] leading-snug text-ink-muted"
>{{ 'thread.summary.partial'|trans }}</p>
{# The way out, beside the notice that explains why
it is there. Not a setting: it costs a much
longer wait and asks the host to reserve a far
larger context window, which on a modest machine
is slow and can fail outright. Fair to spend once
on a thread somebody cares about, wrong as a
default for every summary in the mailbox — so the
cost is in the label rather than in a tooltip
nobody opens.
Shown on exactly the same condition as the notice
at first paint, and withdrawn by the controller
once a full run has answered: a thread that is
still partial AFTER one is past the ceiling, and
pressing again would spend the wait to be told
the same thing. #}
<button type="button"
data-mail--thread-summary-target="full"
data-action="click->mail--thread-summary#full"
{{ summary is not null and summary.isPartial ? '' : 'hidden' }}
class="self-start text-[11px] leading-snug underline underline-offset-2 text-ink-muted hover:text-ink transition-colors cursor-pointer"
>{{ 'thread.summary.send_full'|trans }}</button>
{# A LIST, because three findings in a block of
prose is three findings nobody separates while
skimming — which is the only way a summary is
@@ -109,6 +109,45 @@ final class ThreadSummaryStreamTest extends KernelTestCase
self::assertSame('They agreed.', $tokens->getReturn()->content);
}
/**
* num_ctx goes on the wire for a full run, and only for a full run.
*
* THIS IS THE ONLY THING THAT MAKES THE BUTTON MEAN ANYTHING. Ollama's
* default window is 4096 tokens and everything past it is dropped with no
* error and no way for the caller to notice — four docblocks in this
* codebase size their budgets around exactly that. So a "send the whole
* conversation" that did not also ask for a window to put it in would send
* more, have it silently truncated at a point nobody chose, and cost the
* reader a much longer wait for a summary written from no more than before.
*
* The other half is that an ordinary summary is UNCHANGED. Sending a window
* makes the host reserve memory for the whole of it whether the prompt
* fills it or not, and every summary in the mailbox paying that so one
* thread can be complete is the wrong trade — which is why the option is a
* press and not a setting.
*/
public function testTheContextWindowIsSentOnlyForAFullRun(): void
{
$sent = null;
$summariser = $this->summariser($this->ndjson(['done.']), sent: $sent);
iterator_to_array($summariser->stream($this->reader(), $this->thread, 'a transcript'), false);
self::assertArrayNotHasKey(
'num_ctx',
$sent['options'] ?? [],
'an ordinary summary must not make the host reserve a window it did not need',
);
$transcript = str_repeat('a', 30000);
iterator_to_array($summariser->stream($this->reader(), $this->thread, $transcript, true), false);
self::assertSame(
ThreadSummariser::contextFor($transcript),
$sent['options']['num_ctx'] ?? null,
'a full run asks for a window sized to what it is actually sending',
);
}
/** One row, on the ordinary ending, tagged as the summary workload. */
public function testAFinishedStreamIsRecordedOnceAsAThreadSummary(): void
{
+56
View File
@@ -248,6 +248,62 @@ final class ThreadTranscriptTest extends KernelTestCase
);
}
/**
* A full run sends the conversation, and is still bounded.
*
* Both halves matter. It has to actually send more — otherwise the button
* spends a much longer wait to produce the same summary — and it must still
* stop somewhere, because the whole mechanism behind it is asking the host
* to reserve a context window, and a window is memory. An unbounded
* "send everything" would let one absurd thread ask a modest machine for an
* allocation it cannot make, and a failed allocation costs the reader the
* entire wait and produces nothing.
*/
public function testAFullRunSendsMoreButStillStopsSomewhere(): void
{
$this->seedThread([
'FIRST ' . str_repeat('a', ThreadSummariser::TRANSCRIPT_BUDGET * 40),
'LAST ' . str_repeat('b', ThreadSummariser::TRANSCRIPT_BUDGET * 40),
]);
$normal = $this->transcript->forThread($this->thread);
$full = $this->transcript->forThread($this->thread, true);
self::assertGreaterThan(
mb_strlen($normal) * 4,
mb_strlen($full),
'a full run has to send substantially more, or the wait buys nothing',
);
self::assertLessThanOrEqual(
ThreadSummariser::FULL_TRANSCRIPT_CEILING * 2 + 500,
mb_strlen($full),
'and it still has a ceiling, because num_ctx is memory on somebody else\'s machine',
);
// Past the ceiling it is STILL partial, and says so. The button is
// withdrawn on that answer rather than inviting a second identical wait.
self::assertTrue(ThreadTranscript::isPartial($full));
}
/**
* The context window asked for follows the transcript, and is bounded.
*
* Sized rather than fixed: a window is allocated whole whether the prompt
* fills it or not, so asking for the maximum on every full run would make
* a two-message thread as expensive as a hundred-message one.
*/
public function testTheContextWindowIsSizedToTheTranscriptAndCapped(): void
{
$small = ThreadSummariser::contextFor(str_repeat('a', 1000));
$large = ThreadSummariser::contextFor(str_repeat('a', 90000));
self::assertGreaterThanOrEqual(4096, $small, 'never below the default the host would have used');
self::assertGreaterThan($small, $large, 'a longer conversation asks for a longer window');
self::assertLessThanOrEqual(32768, $large, 'and never more than a host can be expected to allocate');
self::assertSame(0, $large % 1024, 'windows are sized in whole blocks');
}
/**
* A turn with no text part still appears, as its sender line.
*
+1
View File
@@ -3404,6 +3404,7 @@ thread:
heading: 'Zusammenfassung'
regenerate: 'Neu zusammenfassen'
partial: 'Das Gespräch war zu lang, um es ganz zu schicken — die Zusammenfassung ist aus einem Teil davon geschrieben.'
send_full: 'Stattdessen das ganze Gespräch zusammenfassen — dauert länger und verlangt dem Modell-Host mehr ab.'
stale: 'Das Gespräch hat sich geändert, seit das hier geschrieben wurde.'
state:
sent: 'An das Modell geschickt …'
+1
View File
@@ -3396,6 +3396,7 @@ thread:
heading: 'Summary'
regenerate: 'Summarise again'
partial: 'This conversation was too long to send in full, so the summary was written from part of it.'
send_full: 'Summarise the whole conversation instead — slower, and it asks more of the model host.'
stale: 'The conversation has changed since this was written.'
state:
sent: 'Sent to the model…'
+1
View File
@@ -2980,6 +2980,7 @@ thread:
heading: "Th' short of it"
regenerate: 'Boil her again'
partial: "This yarn were too long t' send whole, so th' short of it were writ from but a part."
send_full: "Boil down th' whole yarn instead — slower, an' it asks more o' th' oracle's berth."
stale: "Th' yarn has shifted since this were writ."
state:
sent: "Sent t' th' oracle…"