diff --git a/CHANGELOG.md b/CHANGELOG.md index f5e95154..b3609511 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,52 @@ so anything that changes the schema irreversibly is called out explicitly. The published image tags: `latest` follows the most recent release below, `main` follows the tip of the default branch, and `sha-…` pins one commit. +## v0.2.15 — 2026-09-01 + +**Adds two columns to the AI settings table; the migrations run on boot. Nothing is removed and +nothing is rewritten, and the values it fills in are the ones your model host was already using — so +this release changes how the AI feels, not what it does.** + +### Added + +- **How long a model stays loaded is yours to set.** The first "Help me write" after a quiet spell + used to take a small age and show nothing at all while it did. That is not the model thinking: it + is your Ollama host reading twenty gigabytes back off disk, because it had dropped the model out of + memory a few minutes after the last request. How long it waits before doing that is + `OLLAMA_KEEP_ALIVE`, an environment variable on the host — a machine plMail can talk to but cannot + configure, so until now the only way to change it was to go and edit the host and restart it. + + Admin → AI now has a box under each model. Write a duration — `5m`, `30m`, `4h` — or a plain number + of seconds; `-1` keeps the model loaded for good and `0` unloads it after every request. Longer + means a faster first answer, and the host keeps that memory reserved for the whole time either way. + Every request plMail makes carries the setting, so it covers drafting, summaries, sorting mail into + tabs and the search index alike. + + **Both boxes start at `5m`, which is exactly what Ollama does on its own** — so nothing about your + host changes until you decide it should. The boxes simply say out loud what was already happening. + + **`-1` is worth considering for the search model.** It is small, and it is asked for constantly — + every search by meaning, and every message being indexed — so pinning it usually costs a rounding + error of memory and saves a wait every single time. It is not the shipped setting only because + plMail cannot see how much memory your host has, and that is not a decision to make on your behalf. + + **Leaving a box empty is a real answer too**, and it means "send nothing, and let the host decide". + If you have set `OLLAMA_KEEP_ALIVE` deliberately, that is how you keep it. + +- **Warm up now.** A button beside the writing model that loads it straight away and tells you how + long it took. "Model loaded in 41 s" means it came off disk; "loaded in 230 ms — it was already in + memory" means the click bought nothing. Useful after a restart, and the only way to find out what a + cold load actually costs on your hardware before deciding how long to hold it. A host that cannot + be reached and a model that is not on the host are told apart, because they send you to two + different places. + +### Fixed + +- **A search test asserted how fast your machine was.** The guard that stops an expensive + last-resort search pass from running away is real and unchanged; the test proving it fired had a + one-millisecond budget and one row to scan, so on quick hardware the pass simply finished in time + and the test failed. It now provokes a genuine cancellation instead of hoping for one. + ## v0.2.14 — 2026-08-31 **Adds one table; the migrations run on boot. Nothing is removed and nothing is rewritten. Your diff --git a/README.md b/README.md index e8b9bbba..0f3d29ad 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,9 @@ provider doesn't. week grid, invitations answered in place, share and booking links. - **Files where you keep them** — attach from and save to Drive, Photos, OneDrive, Dropbox, Nextcloud and Immich. +- **A language model, if you want one** — point plMail at an Ollama box on your own network and it + can search by meaning, draft replies, summarise a thread and sort mail into tabs. Off by default, + each of the four switched on separately, and nothing ever leaves your network. - **Yours** — two-factor auth, per-app JMAP passwords, 41 themes, English, German and Pirate. ## A look around diff --git a/assets/controllers/admin/ai_warmup_controller.js b/assets/controllers/admin/ai_warmup_controller.js new file mode 100644 index 00000000..62d3ce58 --- /dev/null +++ b/assets/controllers/admin/ai_warmup_controller.js @@ -0,0 +1,170 @@ +import { Controller } from "@hotwired/stimulus"; + +/** + * "Warm up now" — asks the host to load the writing model, and says what happened. + * + * Values: + * url — POST endpoint, /admin/ai/warmup + * csrf — token minted for `admin-ai-warmup`, NOT the shared `ajax` one + * idleLabel — the button's resting text, restored after a result has been read + * busyLabel — while the host is loading; see below, this is not a spinner + * failedLabel — the fallback when the answer carries no message of its own + * + * Targets: + * button — the control itself, disabled while a request is in flight + * label — the span inside it whose text changes + * icon — the resting icon's wrapper; hidden while busy + * spinner — the spinner's wrapper; shown while busy + * result — where the sentence from the server is written + * + * WHY THE STATES ARE WORDS AND NOT A SPINNER ALONE + * ──────────────────────────────────────────────── + * This request can legitimately take forty seconds and produce nothing at all + * while it does — the host is reading eighteen gigabytes off disk and says + * nothing until it has finished. A spinner during that is indistinguishable + * from a dead button, which is the exact report compose/ai_assist_controller + * was built to answer, and the answer there was the same one: name the state. + * The spinner is here as well, but it is the decoration and the words are the + * information. + * + * WHY THE SENTENCE COMES FROM THE SERVER + * ────────────────────────────────────── + * Every outcome is a sentence in three languages, and one of them carries the + * host's own refusal in it. Assembling those here would put the copy in a file + * no translator opens — the rule admin/ai_status_controller states for the same + * page. This file decides WHEN to ask and what the button looks like while it + * is asking, and nothing about what it says. + * + * WHAT IT DOES NOT DO + * ─────────────────── + * It never talks to the model host: different origin, an address usually only + * the server can route to, and `connect-src 'self'` refuses it in production + * regardless. + */ +export default class extends Controller { + static targets = ["button", "label", "icon", "spinner", "result"]; + + static values = { + url: String, + /** + * Taken from the template rather than from the `csrf-token` meta tag, + * because it is a different token and not a different way of reading + * the same one — the distinction assets/csrf.js draws for + * settings/account_order_controller. This endpoint makes another + * machine reserve eighteen gigabytes; a token good for it should not + * also be good for everything else on the page. + */ + csrf: String, + idleLabel: { type: String, default: "Warm up now" }, + busyLabel: { type: String, default: "Loading the model…" }, + failedLabel: { type: String, default: "No answer" }, + }; + + /** + * Whether a request is in flight. + * + * The button is disabled while it is, so this is belt and braces — but the + * braces earn their place: a second preload of a model already loading does + * not queue behind the first, it makes the host do the work twice, and the + * host is the machine this button exists to be considerate of. + */ + #busy = false; + + /** + * Abandoned if the frame is swapped out from under it — this card lives + * inside a Turbo Frame that re-renders on every save, and a fetch that + * outlives its controller would write into elements no longer on the page. + */ + #abort = null; + + disconnect() { + if (null !== this.#abort) { + this.#abort.abort(); + this.#abort = null; + } + } + + async warmUp() { + if (true === this.#busy) { + return; + } + + this.#busy = true; + this.#abort = new AbortController(); + this.#setBusy(true); + + try { + const response = await fetch(this.urlValue, { + method: "POST", + headers: { "Content-Type": "application/json", "X-CSRF-Token": this.csrfValue }, + signal: this.#abort.signal, + }); + + if (false === response.ok) { + throw new Error(`Request failed (${response.status}).`); + } + + const payload = await response.json(); + + this.#show(payload?.message ?? this.failedLabelValue, true === payload?.ok); + } catch (error) { + // An aborted fetch is the frame being replaced, not a failure — + // there is nothing left on screen to report it to. + if ("AbortError" === error?.name) { + return; + } + + // Everything the server can answer already arrives as a translated + // sentence, so reaching here means the endpoint itself did not + // answer: a session that expired, or a container restarting. One + // label covers both, because the fix for both is to try again. + this.#show(this.failedLabelValue, false); + } finally { + this.#busy = false; + this.#abort = null; + this.#setBusy(false); + } + } + + // ── Private ─────────────────────────────────────────────────────────── + + #setBusy(busy) { + if (true === this.hasButtonTarget) { + this.buttonTarget.disabled = busy; + } + + if (true === this.hasLabelTarget) { + this.labelTarget.textContent = busy ? this.busyLabelValue : this.idleLabelValue; + } + + // The `hidden` attribute on the WRAPPERS, not on the icons — Font + // Awesome's own display rule beats it on an . See the template. + if (true === this.hasIconTarget) { + this.iconTarget.hidden = busy; + } + + if (true === this.hasSpinnerTarget) { + this.spinnerTarget.hidden = false === busy; + } + + if (true === busy && true === this.hasResultTarget) { + // Cleared on the way IN rather than on the way out, so the previous + // answer stays readable until a new one is actually being fetched. + this.resultTarget.textContent = ""; + this.resultTarget.hidden = true; + } + } + + #show(message, ok) { + if (false === this.hasResultTarget) { + return; + } + + // textContent, never innerHTML. One of these messages carries the + // host's own error string, which is text off another machine. + this.resultTarget.textContent = message; + this.resultTarget.hidden = false; + this.resultTarget.classList.toggle("text-emerald-500", ok); + this.resultTarget.classList.toggle("text-danger", false === ok); + } +} diff --git a/docs/de/features/admin.md b/docs/de/features/admin.md index 270bbf98..2c0778f4 100644 --- a/docs/de/features/admin.md +++ b/docs/de/features/admin.md @@ -1,4 +1,4 @@ - + # Administration @@ -336,6 +336,37 @@ Es sagt dir den Unterschied zwischen „unter der Adresse hat nichts geantwortet geantwortet, hat aber nicht das Modell, das du eingetragen hast“ — und das führt an zwei ganz verschiedene Stellen. +### Wie lange die Modelle geladen bleiben + +Ollama wirft ein Modell ein paar Minuten nach der letzten Anfrage aus dem Speicher, und es wieder +von der Platte zu lesen ist die ganze Wartezeit beim ersten „Hilf mir schreiben“ nach einer Pause — +ein 20-GB-Modell braucht dafür Dutzende Sekunden, in denen der Host überhaupt nichts liefert. Mit +**Schreib-Modell geladen lassen für** und **Such-Modell geladen lassen für** legst du fest, wie lange +plMail den Host bittet, das jeweilige Modell zu halten. + +Trag eine Dauer ein — `5m`, `30m`, `4h` — oder einfach eine Zahl in Sekunden. `-1` lässt das Modell +dauerhaft geladen, `0` entlädt es nach jeder Anfrage. Je länger, desto schneller kommt die erste +Antwort; der Host hält den Speicher dafür die ganze Zeit belegt, ob ihn jemand braucht oder nicht. + +Beide starten bei `5m` — das ist Ollamas eigene Voreinstellung. plMail ändert also von sich aus +nichts daran, wie dein Host sich verhält; die Felder sagen nur, was ohnehin schon passiert. + +**Für das Such-Modell lohnt sich `-1`**, und es ist nur deshalb nicht voreingestellt, weil plMail +nicht sehen kann, wie viel Speicher dein Host hat. Dieses Modell ist klein und wird ständig +gebraucht — bei jeder Suche nach Bedeutung und bei jeder Nachricht, die indiziert wird. Es +festzuhalten kostet meist kaum Speicher und spart jedes Mal eine Wartezeit. Wenn der Host den +Speicher übrig hat, stell es um. + +Lass ein Feld **leer**, dann schickt plMail gar nichts mit, und es gilt, was am Host selbst in +`OLLAMA_KEEP_ALIVE` steht. plMail kann diese Variable nicht lesen — ein leeres Feld ist also die +einzige Möglichkeit zu sagen „das überlasse ich dem Host“. + +**Jetzt vorladen** lädt das Schreib-Modell sofort und sagt dir, wie lange das gedauert hat. „Modell +in 41 s geladen“ heißt: es kam von der Platte. „Modell in 30 ms geladen“ heißt: es war schon im +Speicher und der Klick hat nichts gebracht. Praktisch nach einem Neustart — und um herauszufinden, +was ein kalter Ladevorgang auf deiner Hardware wirklich kostet, bevor du entscheidest, wie lange du +das Modell hältst. + ### Die vier Funktionen sind einzeln schaltbar Sie kosten sehr Unterschiedliches, und es hat sehr unterschiedliche Folgen, wenn das Modell danebenliegt. diff --git a/docs/features/admin.md b/docs/features/admin.md index 2a2ae029..a769f7ad 100644 --- a/docs/features/admin.md +++ b/docs/features/admin.md @@ -298,6 +298,35 @@ on the host first — plMail never downloads a model. tell you the difference between "nothing answered at that address" and "that host answered but is not holding the model you named", which send you to two different places. +### How long each model stays loaded + +Ollama drops a model out of memory a few minutes after the last request, and reading it back off +disk is the whole of the wait on the first "Help me write" after a quiet spell — a 20 GB model takes +tens of seconds to load, during which the host produces nothing at all. The **keep the writing model +loaded for** and **keep the search model loaded for** fields are how long plMail asks the host to +hold each one. + +Write a duration — `5m`, `30m`, `4h` — or a plain number of seconds. `-1` keeps the model loaded +indefinitely, `0` unloads it as soon as each request finishes. Longer means a faster first answer, +and the host keeps that memory reserved for the whole time whether anyone uses it or not. + +Both ship at `5m`, which is Ollama's own default — so out of the box plMail changes nothing about +how your host behaves, and the boxes simply say what is already happening. + +**`-1` is worth considering for the search model**, and it does not ship that way only because plMail +cannot see how much memory your host has. That model is small and asked for constantly — every +search by meaning, and every message being indexed — so pinning it usually costs a rounding error of +memory and saves a wait every single time. If the host has the memory to spare, set it. + +Leave either box **empty** to send nothing at all, in which case whatever `OLLAMA_KEEP_ALIVE` says +on the host itself applies. plMail cannot read that variable, so an empty box is the only way to say +"leave this to the host". + +**Warm up now** loads the writing model straight away and tells you how long it took. "Loaded in +41 s" means it came off disk; "loaded in 30 ms" means it was already resident and the click bought +nothing. Useful after a restart, and useful for finding out what a cold load actually costs on your +hardware before you decide how long to hold it. + ### The four features are separate switches They have very different costs, and very different consequences when the model is wrong. diff --git a/migrations/Version20260903150000.php b/migrations/Version20260903150000.php new file mode 100644 index 00000000..caef6d36 --- /dev/null +++ b/migrations/Version20260903150000.php @@ -0,0 +1,84 @@ +addSql('ALTER TABLE ai_settings ADD chat_keep_alive VARCHAR(16) DEFAULT \'5m\''); + $this->addSql('ALTER TABLE ai_settings ADD embedding_keep_alive VARCHAR(16) DEFAULT \'5m\''); + } + + public function down(Schema $schema): void + { + // Nothing to preserve. Dropping these puts every call back to sending + // no keep_alive field, which is what the code did before them and what + // the host's own default is for. + $this->addSql('ALTER TABLE ai_settings DROP chat_keep_alive'); + $this->addSql('ALTER TABLE ai_settings DROP embedding_keep_alive'); + } +} diff --git a/src/Controller/Admin/AiSettingsController.php b/src/Controller/Admin/AiSettingsController.php index 3034c491..63f3541f 100644 --- a/src/Controller/Admin/AiSettingsController.php +++ b/src/Controller/Admin/AiSettingsController.php @@ -21,6 +21,7 @@ use Symfony\Component\HttpFoundation\Request; use Symfony\Component\HttpFoundation\Response; use Symfony\Component\Routing\Attribute\Route; use Symfony\Component\Security\Http\Attribute\IsGranted; +use Symfony\Contracts\Translation\TranslatorInterface; /** * Admin → AI: which model host this installation talks to, and what it may do. @@ -47,6 +48,12 @@ final class AiSettingsController extends AbstractController { use ChecksCsrf; + /** + * Above OllamaClient::PRELOAD_TIMEOUT, so the upstream gives up first and + * this only ever catches a request that has escaped every other bound. + */ + private const int WARMUP_TIME_LIMIT_SECONDS = 300; + public function __construct( private readonly AiSettingsRepository $settings, private readonly AiAssistant $assistant, @@ -54,6 +61,7 @@ final class AiSettingsController extends AbstractController private readonly AiPerformancePanel $panel, private readonly EmbeddingBackfill $backfill, private readonly PromptLibrary $prompts, + private readonly TranslatorInterface $translator, ) { } @@ -81,6 +89,14 @@ final class AiSettingsController extends AbstractController $settings->apiToken = $token; } + // The two keep-alive fields are NOT normalised here, and that is + // deliberate rather than an omission. An emptied TextType bound to + // a nullable property yields null already, and KeepAlive::PATTERN + // has no room in it for surrounding whitespace — so by the time a + // form is valid the value is null or exactly one of the accepted + // spellings. A second pass would imply the first cannot be trusted. + // The writer that DOES need it is the config restore, which reads a + // file rather than a validated form; see ConfigBackupDatabase. $this->entityManager->persist($settings); $this->entityManager->flush(); @@ -217,6 +233,77 @@ final class AiSettingsController extends AbstractController ]); } + /** + * Load the writing model now, and say how long it took. + * + * The one control on this page that asks the host to DO something rather + * than to report. Ollama's `/api/generate` with a model and no prompt loads + * the weights and returns, so the round trip is the cold-load cost — which + * is the number the keep-alive field above is really about, and the only + * way to see it without making somebody go and click "Help me write" and + * count. + * + * WHY PHP'S EXECUTION LIMIT IS RAISED HERE + * ──────────────────────────────────────── + * It is 30 s in this stack, and a cold load of an eighteen-gigabyte model is + * not. Without this the button 500s with MaxExecutionTimeError on precisely + * the slow load it exists to measure, which is the same failure + * ComposeAssistController documents at length — and the same fix, a + * POSITIVE value rather than 0, because 0 cancels the timer without taking + * down the one already armed and it fires anyway while blocked on the + * socket. Above OllamaClient::PRELOAD_TIMEOUT, so the upstream is what + * gives up first and this is only ever the backstop. + * + * A SENTENCE, NOT A KEY, and this is the one place on this page that + * answers that way. The panel next door ships rendered HTML because it has + * six states and a table; this has one line of text on a button, and + * shipping a fragment for it would be markup where a string will do. What + * it must not do is build the sentence in JavaScript — that is where copy + * goes to be missed by translators — so the translation happens here and + * the controller sends the finished line, the way ComposeAttachmentController + * answers its uploads. + */ + #[Route('/warmup', name: 'warmup', methods: ['POST'])] + public function warmUp(Request $request): JsonResponse + { + // Its own token id rather than the shared `ajax` one, the rule + // ChecksCsrf states: this makes another machine reserve eighteen + // gigabytes of memory, which is not something a single stolen token + // should be good for alongside everything else. + $this->assertCsrf($request, 'admin-ai-warmup'); + + set_time_limit(self::WARMUP_TIME_LIMIT_SECONDS); + + $result = $this->assistant->warmUp(); + + if (true === $result->loaded) { + return $this->json([ + 'ok' => true, + 'millis' => $result->milliseconds, + 'message' => $this->translator->trans( + // Milliseconds below a second, whole seconds above it. Not + // a decimal in either case: the distinction that matters is + // "already resident" against "read off disk", and those are + // 40 ms and 40 s. A shared format would print 0.0 s for the + // first, which is the one reading that tells you nothing. + $result->milliseconds < 1000 ? 'admin.ai.warmup.loaded_ms' : 'admin.ai.warmup.loaded_seconds', + ['%count%' => $result->milliseconds < 1000 + ? $result->milliseconds + : (int) round($result->milliseconds / 1000)], + ), + ]); + } + + return $this->json([ + 'ok' => false, + 'millis' => $result->milliseconds, + 'message' => $this->translator->trans( + 'admin.ai.warmup.' . (string) $result->reason, + $result->reasonParams, + ), + ]); + } + /** * What the seven prompts say, when an administrator does not want what we * ship. diff --git a/src/Domain/Ai/KeepAlive.php b/src/Domain/Ai/KeepAlive.php new file mode 100644 index 00000000..d4ab560c --- /dev/null +++ b/src/Domain/Ai/KeepAlive.php @@ -0,0 +1,148 @@ + $reasonParams + */ + public function __construct( + public bool $loaded, + public int $milliseconds, + public ?string $reason = null, + public array $reasonParams = [], + ) { + } + + public static function loaded(int $milliseconds): self + { + return new self(true, $milliseconds); + } + + /** + * @param array $params + */ + public static function failed(string $reason, array $params = [], int $milliseconds = 0): self + { + return new self(false, $milliseconds, $reason, $params); + } +} diff --git a/src/Entity/Ai/AiSettings.php b/src/Entity/Ai/AiSettings.php index 14c75995..31b89b45 100644 --- a/src/Entity/Ai/AiSettings.php +++ b/src/Entity/Ai/AiSettings.php @@ -4,6 +4,7 @@ declare(strict_types=1); namespace App\Entity\Ai; +use App\Domain\Ai\KeepAlive; use App\Domain\Trait\TimestampableTrait; use App\Entity\Embeddable\AiPrompts; use App\Infrastructure\Doctrine\Type\EncryptedStringType; @@ -94,10 +95,47 @@ class AiSettings #[ORM\Column(name: 'chat_model', length: 128, nullable: true)] public ?string $chatModel = null; + /** + * How long the host is asked to hold the writing model after a request. + * + * The setting people actually feel. This model is around eighteen + * gigabytes, so a request that arrives after it has been evicted spends the + * first thirteen seconds or more reading it off disk and produces nothing + * at all while it does — which is the "the AI button does nothing" report, + * every time. Longer here buys a faster first answer; the price is that the + * host keeps the memory reserved for the whole window whether anybody + * writes anything or not, and on a single-GPU box that memory is the + * scarce thing. + * + * NULL IS A REAL SETTING AND NOT AN UNFINISHED ONE. It means send no + * keep_alive field, so whatever the operator put in OLLAMA_KEEP_ALIVE on + * the host stands — a choice somebody may have made deliberately and which + * plMail has no way to read and no business overriding by accident. See + * {@see KeepAlive}, which owns the format and the wire representation. + */ + #[ORM\Column(name: 'chat_keep_alive', length: 16, nullable: true, options: ['default' => KeepAlive::DEFAULT_CHAT])] + public ?string $chatKeepAlive = KeepAlive::DEFAULT_CHAT; + /** The model that turns text into vectors. Usually a much smaller one. */ #[ORM\Column(name: 'embedding_model', length: 128, nullable: true)] public ?string $embeddingModel = null; + /** + * The same question for the search model, and a separate answer to it. + * + * Its own column rather than one setting for both, because the two models + * are nothing alike even when the value is. This one is a few hundred + * megabytes and it is asked for constantly — every search by meaning, and + * every message the backfill walks — so paying a cold load for it is close + * to the worst bargain on offer, and `-1` is usually right here where it + * would rarely be right above. It is not what SHIPS, for the reason + * {@see KeepAlive::DEFAULT_EMBEDDING} gives: plMail cannot see how much + * memory that host has, and `-1` is the one value that never gives any + * back. + */ + #[ORM\Column(name: 'embedding_keep_alive', length: 16, nullable: true, options: ['default' => KeepAlive::DEFAULT_EMBEDDING])] + public ?string $embeddingKeepAlive = KeepAlive::DEFAULT_EMBEDDING; + /** * How many components the embedding model produces. * @@ -215,4 +253,29 @@ class AiSettings AiFeature::Summary => $this->summaryEnabled, }; } + + /** + * How long to ask the host to hold the model this feature uses. + * + * The same split enabledFor() makes and for the same reason: a feature does + * not get "the" model, it gets the one chosen for its job, and the setting + * that governs how long it stays resident has to follow the model rather + * than the feature. Three of the four features share the writing model, so + * three of the four arms share its answer — asking per feature is what + * stops a caller having to know that. + * + * The raw stored spelling, not the wire value. {@see KeepAlive::forBody()} + * is the one place that turns it into something a request body can carry, + * because the difference between the number -1 and the string "-1" matters + * to the host and to nothing else. + */ + public function keepAliveFor(AiFeature $feature): ?string + { + return match ($feature) { + AiFeature::Search => $this->embeddingKeepAlive, + AiFeature::Categorise, + AiFeature::WritingHelp, + AiFeature::Summary => $this->chatKeepAlive, + }; + } } diff --git a/src/Form/Admin/AiSettingsType.php b/src/Form/Admin/AiSettingsType.php index 143a7f80..271980e1 100644 --- a/src/Form/Admin/AiSettingsType.php +++ b/src/Form/Admin/AiSettingsType.php @@ -4,6 +4,7 @@ declare(strict_types=1); namespace App\Form\Admin; +use App\Domain\Ai\KeepAlive; use App\Entity\Ai\AiSettings; use App\Form\PasswordManagerIgnore; use Symfony\Component\Form\AbstractType; @@ -76,12 +77,27 @@ final class AiSettingsType extends AbstractType 'required' => false, 'attr' => ['placeholder' => 'llama3.1:8b', 'autocomplete' => 'off'], ]) + // Directly under the model it governs rather than grouped with the + // other keep-alive below, because the pair is one decision: which + // model, and for how long it stays loaded. Two adjacent duration + // fields under two adjacent model fields would be four boxes in + // which it is genuinely hard to see which belongs to which. + ->add('chatKeepAlive', TextType::class, self::keepAlive( + 'admin.ai.field.chat_keep_alive', + 'admin.ai.field.chat_keep_alive_help', + KeepAlive::DEFAULT_CHAT, + )) ->add('embeddingModel', TextType::class, [ 'label' => 'admin.ai.field.embedding_model', 'help' => 'admin.ai.field.embedding_model_help', 'required' => false, 'attr' => ['placeholder' => 'nomic-embed-text', 'autocomplete' => 'off'], ]) + ->add('embeddingKeepAlive', TextType::class, self::keepAlive( + 'admin.ai.field.embedding_keep_alive', + 'admin.ai.field.embedding_keep_alive_help', + KeepAlive::DEFAULT_EMBEDDING, + )) // The four below are separate because they have very different // costs and very different appetites for being wrong. See // AiSettings for the argument. @@ -107,6 +123,53 @@ final class AiSettingsType extends AbstractType ]); } + /** + * The two keep-alive fields, which differ only in what they say. + * + * WHY THE VALIDATION IS A REGEX AND NOT A COERCION + * ──────────────────────────────────────────────── + * A wrong value here fails at the far end, hours later, on somebody else's + * request: Ollama answers 400 for a duration it cannot parse, and the only + * symptom is that the writing help stopped working. Quietly rewriting "5 + * min" into "5m" would hide that, and quietly falling back to the default + * would hide it better — the field would show one thing and the host would + * be told another, which is the single worst outcome available for a + * setting whose entire job is to be believed. + * + * So it is refused at the moment somebody can still fix it, with the + * accepted spellings in the message. {@see KeepAlive::PATTERN} is the same + * expression the request body is built from, so the form and the wire + * cannot come to disagree about what is legal. + * + * The message lives in the `validators` domain, which is not the domain + * this form declares — Symfony translates constraint output there + * regardless, and a key put in `messages` renders on screen as the raw key. + * See CODESTYLE §8.2, and the `base_url_invalid` message directly above, + * which has that bug. + * + * @return array + */ + private static function keepAlive(string $label, string $help, string $shipped): array + { + return [ + 'label' => $label, + 'help' => $help, + // Empty is a real, selectable answer — "say nothing and let the + // host's own OLLAMA_KEEP_ALIVE stand" — so the field is optional + // and the placeholder shows what plMail ships rather than what is + // required. + 'required' => false, + 'attr' => ['placeholder' => $shipped, 'autocomplete' => 'off'], + 'constraints' => [ + new Regex( + pattern: KeepAlive::PATTERN, + message: 'admin.ai.keep_alive_invalid', + match: true, + ), + ], + ]; + } + public function configureOptions(OptionsResolver $resolver): void { $resolver->setDefaults([ diff --git a/src/Service/Ai/AiAssistant.php b/src/Service/Ai/AiAssistant.php index 6d0fe77f..3aa90c61 100644 --- a/src/Service/Ai/AiAssistant.php +++ b/src/Service/Ai/AiAssistant.php @@ -8,6 +8,7 @@ use App\Domain\DTO\Ai\AiCallTiming; use App\Domain\DTO\Ai\AiChatResult; use App\Domain\DTO\Ai\AiEmbedResult; use App\Domain\DTO\Ai\AiProbe; +use App\Domain\DTO\Ai\AiWarmUp; use App\Domain\Enum\Ai\AiCallFeature; use App\Entity\Ai\AiFeature; use App\Entity\Ai\AiSettings; @@ -152,7 +153,12 @@ final readonly class AiAssistant $model = (string) $settings->embeddingModel; - $result = $this->client->embed((string) $settings->baseUrl, $model, $text); + $result = $this->client->embed( + (string) $settings->baseUrl, + $model, + $text, + $settings->keepAliveFor(AiFeature::Search), + ); // Recorded here rather than in the client, because the client knows the // numbers and this knows what they were FOR. @@ -194,7 +200,13 @@ final readonly class AiAssistant $model = (string) $settings->chatModel; - $result = $this->client->chat((string) $settings->baseUrl, $model, $messages, $temperature); + $result = $this->client->chat( + (string) $settings->baseUrl, + $model, + $messages, + $temperature, + $settings->keepAliveFor($feature), + ); $this->recorder->record( AiCallFeature::forChat($feature), @@ -261,7 +273,13 @@ final readonly class AiAssistant return $this->recorded( AiCallFeature::forChat($feature), $model, - $this->client->chatStream((string) $settings->baseUrl, $model, $messages, $temperature), + $this->client->chatStream( + (string) $settings->baseUrl, + $model, + $messages, + $temperature, + $settings->keepAliveFor($feature), + ), ); } @@ -414,4 +432,56 @@ final readonly class AiAssistant return $this->client->probe($target); } + + /** + * Load the writing model now, so the next person to ask does not wait. + * + * WHY THE MASTER SWITCH IS NOT CONSULTED + * ────────────────────────────────────── + * probe()'s reasoning, and the same situation: an administrator setting + * this up has not turned anything on yet, and the cost of a cold load is + * precisely what they are trying to find out before they do. A button that + * answered "disabled" would refuse at the only moment the number is worth + * measuring. + * + * A HOST AND A MODEL ARE STILL REQUIRED, and they get separate answers + * because they are separate mistakes with separate fixes — one is a field + * above this button, the other is a different field above this button, and + * "nothing happened" would send somebody to check the network for neither + * of them. + * + * WHAT IT DELIBERATELY DOES NOT DO + * ──────────────────────────────── + * It reads the SAVED settings, where the Test button reads what is on + * screen. The two are different errands: Test answers "is this address any + * good" before you commit to it, while this one asks a host to reserve + * eighteen gigabytes on behalf of the configuration that is actually in + * force. Warming a model nothing is going to use — because the form has not + * been saved — would be the wrong machine doing real work for a setting + * that does not exist yet. + * + * It also warms only the writing model. The search model is the one that is + * cheap to pin and is pinned by default, and a second button for it would + * be a control whose correct use is "never". + */ + public function warmUp(): AiWarmUp + { + $settings = $this->settings(); + + if (false === $settings->isConfigured()) { + return AiWarmUp::failed('no_host'); + } + + $model = $settings->chatModel; + + if (null === $model || '' === trim($model)) { + return AiWarmUp::failed('no_model'); + } + + return $this->client->preload( + (string) $settings->baseUrl, + trim($model), + $settings->keepAliveFor(AiFeature::WritingHelp), + ); + } } diff --git a/src/Service/Ai/OllamaClient.php b/src/Service/Ai/OllamaClient.php index ca6c738e..76871c8a 100644 --- a/src/Service/Ai/OllamaClient.php +++ b/src/Service/Ai/OllamaClient.php @@ -4,10 +4,12 @@ declare(strict_types=1); namespace App\Service\Ai; +use App\Domain\Ai\KeepAlive; use App\Domain\DTO\Ai\AiCallTiming; use App\Domain\DTO\Ai\AiChatResult; use App\Domain\DTO\Ai\AiEmbedResult; use App\Domain\DTO\Ai\AiProbe; +use App\Domain\DTO\Ai\AiWarmUp; use App\Domain\DTO\Ai\LoadedModel; use App\Domain\DTO\Ai\OllamaModel; use DateTimeImmutable; @@ -91,6 +93,24 @@ final readonly class OllamaClient */ private const float PROBE_TIMEOUT = 5.0; + /** + * Longer than GENERATE_TIMEOUT, which looks wrong and is not. + * + * Every one of these timeouts is an IDLE timeout — the longest the host may + * go without saying anything — and on a streamed completion the first token + * resets it, so 120 s covers a cold load and then never applies again. + * {@see preload()} has no tokens. It is the one call that is ENTIRELY the + * cold load: the host says nothing whatsoever until eighteen gigabytes are + * in memory, and off a spinning disk that is comfortably past two minutes. + * Held to 120 s the button would report a timeout for a load that was + * proceeding perfectly well, which is the exact confusion it exists to end. + * + * Below the caller's own ceiling, so it is this that gives up first — see + * AiSettingsController::warmUp(), which raises PHP's execution limit above + * this and says why. + */ + private const float PRELOAD_TIMEOUT = 240.0; + public function __construct( private HttpClientInterface $http, private LoggerInterface $logger, @@ -263,22 +283,27 @@ final readonly class OllamaClient * as broken rather than as out of date. * * Both attempts collapse into ONE result, and therefore one recorded call. + * + * The keep-alive rides on both. The older dialect predates the field and + * ignores what it does not recognise, so sending it costs nothing there and + * omitting it would mean the setting silently stopped applying on exactly + * the hosts that pay the most for a reload. */ - public function embed(string $baseUrl, string $model, string $text): AiEmbedResult + public function embed(string $baseUrl, string $model, string $text, ?string $keepAlive = null): AiEmbedResult { - $modern = $this->tryEmbed($baseUrl, '/api/embed', [ + $modern = $this->tryEmbed($baseUrl, '/api/embed', self::withKeepAlive([ 'model' => $model, 'input' => $text, - ], static fn (array $body): ?array => $body['embeddings'][0] ?? null); + ], $keepAlive), static fn (array $body): ?array => $body['embeddings'][0] ?? null); if (true === $modern->succeeded) { return $modern; } - $legacy = $this->tryEmbed($baseUrl, '/api/embeddings', [ + $legacy = $this->tryEmbed($baseUrl, '/api/embeddings', self::withKeepAlive([ 'model' => $model, 'prompt' => $text, - ], static fn (array $body): ?array => $body['embedding'] ?? null); + ], $keepAlive), static fn (array $body): ?array => $body['embedding'] ?? null); if (true === $legacy->succeeded) { // The older dialect answered, so the embedding worked. Its timings @@ -313,13 +338,13 @@ final readonly class OllamaClient * * @param list $messages */ - public function chat(string $baseUrl, string $model, array $messages, ?float $temperature = null): AiChatResult + public function chat(string $baseUrl, string $model, array $messages, ?float $temperature = null, ?string $keepAlive = null): AiChatResult { - $payload = [ + $payload = self::withKeepAlive([ 'model' => $model, 'messages' => $messages, 'stream' => false, - ]; + ], $keepAlive); if (null !== $temperature) { $payload['options'] = ['temperature' => $temperature]; @@ -434,13 +459,13 @@ final readonly class OllamaClient * * @return \Generator */ - public function chatStream(string $baseUrl, string $model, array $messages, ?float $temperature = null): \Generator + public function chatStream(string $baseUrl, string $model, array $messages, ?float $temperature = null, ?string $keepAlive = null): \Generator { - $payload = [ + $payload = self::withKeepAlive([ 'model' => $model, 'messages' => $messages, 'stream' => true, - ]; + ], $keepAlive); if (null !== $temperature) { $payload['options'] = ['temperature' => $temperature]; @@ -531,6 +556,158 @@ final readonly class OllamaClient return AiChatResult::ok($whole, $timing); } + /** + * Load a model into the host's memory and generate nothing. + * + * A `/api/generate` with a model and NO prompt is Ollama's documented way + * to say "have this ready". The host reads the weights in and answers as + * soon as they are resident, so the round trip IS the cold-load cost — the + * number an operator actually wants, and one nothing else here can report: + * /api/ps says whether a model is loaded but never how long getting it + * there took. + * + * It is a WRITE in every sense that matters — it reserves gigabytes on + * somebody else's machine for as long as the keep-alive says — which is why + * it is only ever reached from an administrator pressing a button, and why + * nothing polls it. ps() is the passive question. + * + * NOT RECORDED IN THE METRICS TABLE, and that is deliberate rather than + * forgotten. AiCallRecorder accounts for WORKLOADS — five of them, each + * gated by a feature — and this is not one: it generates no tokens, it + * serves no user's request, and giving it a sixth AiCallFeature case would + * put a row in the throughput table with nothing to put in any of its + * columns. The timing goes back to the button that asked for it, which is + * the only reader there is. + * + * Answers a value object rather than throwing, like probe(): the caller is + * an administrator who wants to be told what happened, in the same three + * shapes — it worked, nothing answered, or something answered and refused. + */ + public function preload(string $baseUrl, string $model, ?string $keepAlive = null): AiWarmUp + { + $started = microtime(true); + + try { + $response = $this->http->request('POST', $this->url($baseUrl, '/api/generate'), [ + 'json' => self::withKeepAlive([ + 'model' => $model, + 'stream' => false, + ], $keepAlive), + 'timeout' => self::PRELOAD_TIMEOUT, + ]); + + $status = $response->getStatusCode(); + + if (200 !== $status) { + // 404 is its own answer because it is the one an operator can + // fix from where they are standing: the host is up, it is + // talking, and the model named in the settings above is simply + // not pulled on it. Told "the host is unreachable" they would + // go and check the network instead. + if (404 === $status) { + return AiWarmUp::failed('model_missing', [ + 'model' => $model, + 'error' => self::refusal($response->getContent(false)), + ], self::elapsed($started)); + } + + return AiWarmUp::failed('status', [ + 'status' => $status, + 'error' => self::refusal($response->getContent(false)), + ], self::elapsed($started)); + } + + // Read so the response is complete before the clock is stopped — + // the status line arrives with the headers, and on this endpoint + // that is the whole of the wait. Not read for its content: a + // preload answers `done: true` with an empty response, which is + // success and would fail any check for text. + $response->getContent(false); + + return AiWarmUp::loaded(self::elapsed($started)); + } catch (TimeoutExceptionInterface $exception) { + $this->logger->warning('OllamaClient: preload timed out', [ + 'model' => $model, + 'error' => $exception->getMessage(), + 'exception' => $exception, + ]); + + return AiWarmUp::failed('timeout', [], self::elapsed($started)); + } catch (HttpClientException $exception) { + $this->logger->warning('OllamaClient: preload could not reach the host', [ + 'model' => $model, + 'error' => $exception->getMessage(), + 'exception' => $exception, + ]); + + return AiWarmUp::failed('unreachable', ['error' => $exception->getMessage()], self::elapsed($started)); + } catch (Throwable $exception) { + $this->logger->error('OllamaClient: preload failed unexpectedly', [ + 'model' => $model, + 'error' => $exception->getMessage(), + 'exception' => $exception, + ]); + + return AiWarmUp::failed('unexpected', ['error' => $exception->getMessage()], self::elapsed($started)); + } + } + + /** + * The request body with `keep_alive` in it, or exactly as it was. + * + * One helper for all four bodies, because the rule is the same everywhere + * and the thing being avoided is a fifth call site that sends `null` — an + * explicit null in the JSON is NOT the same as omitting the field: Ollama + * decodes it to a zero duration and unloads the model the moment the + * request finishes, which is the exact opposite of "we did not want to + * say". + * + * @param array $payload + * + * @return array + */ + private static function withKeepAlive(array $payload, ?string $keepAlive): array + { + $value = KeepAlive::forBody($keepAlive); + + if (null === $value) { + return $payload; + } + + $payload['keep_alive'] = $value; + + return $payload; + } + + /** Whole milliseconds, which is the resolution anybody reads this at. */ + private static function elapsed(float $started): int + { + return (int) round((microtime(true) - $started) * 1000); + } + + /** + * What the host said when it refused, short enough to put on a page. + * + * Ollama answers `{"error": "model 'qwen3' not found"}`, and that sentence + * is the most useful thing on the screen — it names the model as the host + * understood it, which is how a trailing space or a wrong tag gets found. + * The raw body is the fallback for a reverse proxy answering its own HTML, + * clipped because that can be a whole error page. + * + * Safe to show here in a way it would not be from chat(): this request body + * carries a model name and nothing else. No mail has ever been near it. + */ + private static function refusal(string $body): string + { + $decoded = json_decode($body, true); + + if (true === is_array($decoded) && true === is_string($decoded['error'] ?? null)) { + return $decoded['error']; + } + + return mb_substr(trim($body), 0, 200); + } + /** * @param array $payload * @param callable(array): (list|null) $pluck diff --git a/src/Service/Backup/ConfigBackupDatabase.php b/src/Service/Backup/ConfigBackupDatabase.php index 7f601773..b3f53ebf 100644 --- a/src/Service/Backup/ConfigBackupDatabase.php +++ b/src/Service/Backup/ConfigBackupDatabase.php @@ -12,6 +12,7 @@ use App\Entity\Push\FcmConfig; use App\Repository\Integration\IntegrationProviderConfigRepository; use App\Repository\Integration\MailProviderConfigRepository; use App\Repository\Push\FcmConfigRepository; +use App\Domain\Ai\KeepAlive; use App\Entity\Ai\AiSettings; use App\Entity\Monitoring\LogSettings; use App\Repository\Ai\AiSettingsRepository; @@ -238,7 +239,9 @@ final readonly class ConfigBackupDatabase 'baseUrl' => $settings->baseUrl, 'apiToken' => $settings->apiToken, 'chatModel' => $settings->chatModel, + 'chatKeepAlive' => $settings->chatKeepAlive, 'embeddingModel' => $settings->embeddingModel, + 'embeddingKeepAlive' => $settings->embeddingKeepAlive, 'embeddingDimensions' => $settings->embeddingDimensions, 'searchEnabled' => $settings->searchEnabled, 'categorisationEnabled' => $settings->categorisationEnabled, @@ -282,6 +285,15 @@ final readonly class ConfigBackupDatabase $settings->apiToken = $this->string($values, 'apiToken'); $settings->chatModel = $this->string($values, 'chatModel'); $settings->embeddingModel = $this->string($values, 'embeddingModel'); + // Through KeepAlive::normalised() rather than string() alone, because + // this is the one writer that is not the form. A file can carry + // whitespace where the form's constraint would have refused it, and a + // column holding " " shows an administrator a value they cannot save + // again. Absent from an older backup means null, which is the + // host's-own-default state and the right answer for a file written + // before this setting existed. + $settings->chatKeepAlive = KeepAlive::normalised($this->string($values, 'chatKeepAlive')); + $settings->embeddingKeepAlive = KeepAlive::normalised($this->string($values, 'embeddingKeepAlive')); $settings->embeddingDimensions = is_int($values['embeddingDimensions'] ?? null) ? $values['embeddingDimensions'] : null; $settings->searchEnabled = true === ($values['searchEnabled'] ?? false); $settings->categorisationEnabled = true === ($values['categorisationEnabled'] ?? false); diff --git a/templates/admin/ai/_frame.html.twig b/templates/admin/ai/_frame.html.twig index a50522f5..bcd1a505 100644 --- a/templates/admin/ai/_frame.html.twig +++ b/templates/admin/ai/_frame.html.twig @@ -140,7 +140,90 @@ poll would empty a field mid-word. {{ form_start(form, { attr: { 'data-turbo-frame': 'admin-ai' } }) }}
- {{ form_widget(form) }} + {# Rendered row by row rather than with form_widget(form), + for one reason: the warm-up button has to sit with the + writing model and its keep-alive, and a single + form_widget emits all eleven rows in one go with nowhere + to put anything between them. The wrapper carries the + space-y-5 that modal_form_theme's form_widget_compound + adds for a root form, and form_rest() below still emits + the CSRF token — dropping it is how this kind of + hand-rendered form silently stops validating. #} +
+ {{ form_errors(form) }} + + {{ form_row(form.isEnabled) }} + {{ form_row(form.baseUrl) }} + {{ form_row(form.apiToken) }} + + {{ form_row(form.chatModel) }} + {{ form_row(form.chatKeepAlive) }} + + {# ── Warm up now ────────────────────────────────── + Here rather than beside Save, because it is about + the two fields above it and nothing else on the + page: it loads THAT model and holds it for THAT + long. Beside Save it would read as another way of + submitting the form, which it is not — it writes + nothing, and it acts on what is stored rather than + on what is on screen. + + type="button", so a stray Enter in a text field + above cannot make it the form's default submit. #} +
+ + + + {# aria-live, because the answer arrives up to a + minute after the click and a screen reader user + who has moved on would otherwise never learn + what happened. Polite: it is a result, not an + interruption. #} + +
+ + {{ form_row(form.embeddingModel) }} + {{ form_row(form.embeddingKeepAlive) }} + + {{ form_row(form.searchEnabled) }} + {{ form_row(form.categorisationEnabled) }} + {{ form_row(form.writingHelpEnabled) }} + {{ form_row(form.summaryEnabled) }} + + {{ form_rest(form) }} +
{# Same form, different action. `formaction` so the diff --git a/tests/Form/Admin/AiKeepAliveTest.php b/tests/Form/Admin/AiKeepAliveTest.php new file mode 100644 index 00000000..7481fd27 --- /dev/null +++ b/tests/Form/Admin/AiKeepAliveTest.php @@ -0,0 +1,73 @@ +forms = self::getContainer()->get('form.factory'); + } + + public function testTheAcceptedSpellingsAreAccepted(): void + { + foreach (['5m', '30m', '4h', '3600', '0', '-1', ''] as $value) { + $form = $this->submit($value); + + self::assertTrue($form->isValid(), $value . ' → ' . (string) $form->getErrors(true, false)); + } + } + + /** + * "5 min" is what somebody types who has not read the help text, and + * "5M" is what somebody types who has — Go's duration parser accepts + * neither, so neither may be accepted here. + */ + public function testAnythingElseIsRefusedRatherThanQuietlyReplaced(): void + { + foreach (['5 min', '5M', 'forever', '5m30s', '-2'] as $value) { + $form = $this->submit($value); + + self::assertFalse($form->isValid(), $value . ' was accepted'); + } + } + + private function submit(string $keepAlive): FormInterface + { + $form = $this->forms->create(AiSettingsType::class, new AiSettings(), ['csrf_protection' => false]); + + $form->submit([ + 'baseUrl' => 'http://10.0.0.5:11434', + 'chatKeepAlive' => $keepAlive, + ], false); + + return $form; + } +} diff --git a/tests/Repository/Mail/MessageSearchTest.php b/tests/Repository/Mail/MessageSearchTest.php index a978087f..dd65824f 100644 --- a/tests/Repository/Mail/MessageSearchTest.php +++ b/tests/Repository/Mail/MessageSearchTest.php @@ -906,28 +906,60 @@ final class MessageSearchTest extends KernelTestCase } /** - * The same search, with a budget no query can meet: nothing found, and - * nothing thrown. + * A body pass that blows its budget: the cheap results stand, and nothing + * is thrown. * - * Before this, the pass ran unbounded and a live mailbox answered with - * `MaxExecutionTimeError: Maximum execution time of 30 seconds exceeded`. - * Losing the infix is the deliberate price; a 500 was not a price anybody - * chose. + * Before this guard existed the pass ran unbounded and a live mailbox + * answered with `MaxExecutionTimeError: Maximum execution time of 30 + * seconds exceeded`. Losing the infix is the deliberate price; a 500 was + * not a price anybody chose. + * + * IT NO LONGER ASKS POSTGRES TO BE SLOW, AND THAT IS THE FIX + * ───────────────────────────────────────────────────────── + * This used to build a repository with `rescueTimeoutMs: 1`, run a real + * search and assert it came back empty — which asserts how fast the machine + * is, not what this code does. One seeded row is a scan Postgres finishes + * well inside a millisecond, so on a quick box the pass SUCCEEDS, finds the + * row the test called "the one row only the expired pass could find", and + * fails with "actual size 1 matches expected size 0". Green on slow + * hardware and red on fast is not a test of anything. + * + * `pg_sleep` removes the race instead of widening the assertion. One second + * against a one-millisecond budget cannot finish in time on any machine, so + * the cancel is certain — and it is a REAL cancel, arriving as SQLSTATE + * 57014 through the same path a genuine overrun takes. All three halves of + * the guard are exercised: the budget is set, the statement is killed, and + * the fallback rows are returned instead of the exception escaping. + * + * The rescue callable is handed in by reflection because the method is + * private and should stay private — the same access this file already takes + * for buildSearchSql(), and for the same reason: the seam is where the bug + * lives. Going through searchPage() instead is what forced the timing + * assumption in the first place. */ public function testAnExpiredBodyPassGivesUpInsteadOfFailing(): void { - $this->seedMessage('Quarterly figures', body: 'The pelican audit is attached.'); - $impatient = new MessageThreadRepository( self::getContainer()->get(ManagerRegistry::class), new FreeTextCompiler(), rescueTimeoutMs: 1, ); - $page = $impatient->searchPage($this->user, $this->parser->parse('elica')); + $connection = self::getContainer()->get(EntityManagerInterface::class)->getConnection(); - self::assertCount(0, $page->threads, 'the one row only the expired pass could find'); - self::assertSame(0, $page->total); + // What the cheap pass already found. Identity is the assertion below: + // the guard must hand back exactly this, not an empty page. + $fallback = [['thread_id' => 7]]; + + $rows = new \ReflectionMethod(MessageThreadRepository::class, 'rescueRows')->invoke( + $impatient, + // The same connection rescueRows() put the budget on — a second one + // would be a different session and would never see the timeout. + static fn (): array => $connection->fetchAllAssociative('SELECT pg_sleep(1)'), + $fallback, + ); + + self::assertSame($fallback, $rows, 'the cheap results have to stand when the rescue pass is killed'); } /** diff --git a/tests/Service/Ai/OllamaKeepAliveTest.php b/tests/Service/Ai/OllamaKeepAliveTest.php new file mode 100644 index 00000000..eb87cee3 --- /dev/null +++ b/tests/Service/Ai/OllamaKeepAliveTest.php @@ -0,0 +1,137 @@ +client(static function (string $method, string $url, array $options) use (&$body): MockResponse { + $body = json_decode((string) $options['body'], true); + + return new MockResponse(json_encode(['message' => ['content' => 'hello']])); + }); + + $client->chat('http://h:11434', 'm', [['role' => 'user', 'content' => 'hi']], null, ''); + + self::assertIsArray($body); + self::assertArrayNotHasKey('keep_alive', $body); + } + + /** + * And the embedding path carries its own, which is a different column. + */ + public function testTheEmbeddingCallCarriesTheSearchModelsSetting(): void + { + $body = null; + + $client = $this->client(static function (string $method, string $url, array $options) use (&$body): MockResponse { + $body = json_decode((string) $options['body'], true); + + return new MockResponse(json_encode(['embeddings' => [[0.1, 0.2]]])); + }); + + $client->embed('http://h:11434', 'nomic-embed-text', 'some mail', '-1'); + + self::assertIsArray($body); + self::assertSame(-1, $body['keep_alive']); + } + + /** + * The preload posts a model and NO prompt, which is what makes it a load + * rather than a generation. + */ + public function testWarmingUpAsksForAModelAndNoPrompt(): void + { + $body = null; + $seen = null; + + $client = $this->client(static function (string $method, string $url, array $options) use (&$body, &$seen): MockResponse { + $seen = $url; + $body = json_decode((string) $options['body'], true); + + return new MockResponse(json_encode(['done' => true])); + }); + + $result = $client->preload('http://h:11434', 'qwen3:30b', '5m'); + + self::assertTrue($result->loaded); + self::assertStringContainsString('/api/generate', (string) $seen); + self::assertIsArray($body); + self::assertArrayNotHasKey('prompt', $body); + self::assertSame('5m', $body['keep_alive']); + } + + /** + * "The host is not there" and "the host is there and has never heard of + * that model" send an administrator to two different fields, so they are + * two different answers. + */ + public function testAMissingModelIsNotTheSameAnswerAsAMissingHost(): void + { + $refused = $this->client(static fn (): MockResponse => new MockResponse( + json_encode(['error' => "model 'qwen3:30b' not found"]), + ['http_code' => 404], + )); + + $missing = $refused->preload('http://h:11434', 'qwen3:30b'); + + self::assertFalse($missing->loaded); + self::assertSame('model_missing', $missing->reason); + // The host's own sentence, because it names the model as the host read + // it — which is how a trailing space or a wrong tag gets found. + self::assertSame("model 'qwen3:30b' not found", $missing->reasonParams['error']); + + $down = $this->client(static function (): MockResponse { + throw new TransportException('Connection refused'); + }); + + self::assertSame('unreachable', $down->preload('http://h:11434', 'qwen3:30b')->reason); + } + + private function client(callable $handler): OllamaClient + { + return new OllamaClient(new MockHttpClient($handler), new NullLogger()); + } +} diff --git a/translations/messages.de.yaml b/translations/messages.de.yaml index 0c986ef2..15c86dd4 100644 --- a/translations/messages.de.yaml +++ b/translations/messages.de.yaml @@ -95,8 +95,12 @@ admin: base_url_invalid: 'Das muss mit http:// oder https:// anfangen.' categorisation: 'Mail in Tabs einsortieren' categorisation_help: 'Läuft bei jeder eingehenden Nachricht. Immer nur als Stichentscheid — zuerst entscheiden die normalen Regeln.' + chat_keep_alive: 'Schreib-Modell geladen lassen für' + chat_keep_alive_help: '5m, 30m, 4h oder einfach eine Zahl in Sekunden. -1 lässt es dauerhaft geladen, 0 entlädt es nach jeder Anfrage. Je länger, desto schneller kommt die erste Antwort — der Host hält den Speicher aber die ganze Zeit belegt. Leer lassen, um das zu nehmen, was am Host selbst eingestellt ist.' chat_model: 'Schreib-Modell' chat_model_help: 'Für Antworten, Zusammenfassungen und das Einsortieren. Nimm ein instruction-tuned Modell.' + embedding_keep_alive: 'Such-Modell geladen lassen für' + embedding_keep_alive_help: 'Gleiches Format wie oben. Dieses Modell ist klein und wird ständig gebraucht, deshalb kostet -1 — dauerhaft geladen — meist kaum etwas und spart bei jeder Suche eine Wartezeit; wenn der Host den Speicher übrig hat, lohnt es sich. Leer lassen, um das zu nehmen, was am Host selbst eingestellt ist.' embedding_model: 'Such-Modell' embedding_model_help: 'Wandelt Mail in Vektoren um, damit die Suche Bedeutung statt Wörter findet.' enabled: 'Sprachmodell verwenden' @@ -211,6 +215,19 @@ admin: on: 'An' paused: 'Pausiert' test: 'Testen' + warmup: + button: 'Jetzt vorladen' + failed: 'Keine Antwort. Versuch es noch einmal.' + loaded_ms: 'Modell in %count% ms geladen — es war schon im Speicher.' + loaded_seconds: 'Modell in %count% s geladen.' + loading: 'Modell wird geladen …' + model_missing: 'Auf dem Host gibt es kein Modell namens „%model%“. Zieh es dort, oder trag eines ein, das da ist. (%error%)' + no_host: 'Trag zuerst einen Modell-Host ein und speichere.' + no_model: 'Trag zuerst ein Schreib-Modell ein und speichere.' + status: 'Der Host hat mit HTTP %status% abgelehnt. (%error%)' + timeout: 'Der Host hat nach vier Minuten immer noch geladen, plMail hat aufgehört zu warten. Gut möglich, dass er fertig wird — vermutlich ist das Modell aber zu groß für seinen Speicher.' + unexpected: 'Etwas ist schiefgegangen, bevor der Host antworten konnte. (%error%)' + unreachable: 'Unter der Adresse hat nichts geantwortet. (%error%)' config_backup: change: absent: 'hier noch nicht gesetzt' diff --git a/translations/messages.en.yaml b/translations/messages.en.yaml index 7b7432fa..ce3f9754 100644 --- a/translations/messages.en.yaml +++ b/translations/messages.en.yaml @@ -95,8 +95,12 @@ admin: base_url_invalid: 'This has to start with http:// or https://.' categorisation: 'Sort mail into tabs' categorisation_help: 'Runs on every message that arrives. Only ever a tie-breaker — the ordinary rules decide first.' + chat_keep_alive: 'Keep the writing model loaded for' + chat_keep_alive_help: '5m, 30m, 4h, or a plain number of seconds. -1 keeps it loaded for good, 0 unloads it after every request. Longer means the first reply comes back sooner; the host keeps the memory reserved for the whole time either way. Leave empty to use whatever the host itself is set to.' chat_model: 'Writing model' chat_model_help: 'Used for replies, summaries and sorting. Something instruction-tuned.' + embedding_keep_alive: 'Keep the search model loaded for' + embedding_keep_alive_help: 'Same format as above. This model is small and asked for constantly, so -1 — loaded for good — usually costs little and saves a wait on every search; it is worth setting if the host has the memory to spare. Leave empty to use whatever the host itself is set to.' embedding_model: 'Search model' embedding_model_help: 'Turns mail into vectors so search can find meaning rather than words.' enabled: 'Use a language model' @@ -211,6 +215,19 @@ admin: on: 'On' paused: 'Paused' test: 'Test' + warmup: + button: 'Warm up now' + failed: 'No answer. Try again.' + loaded_ms: 'Model loaded in %count% ms — it was already in memory.' + loaded_seconds: 'Model loaded in %count% s.' + loading: 'Loading the model…' + model_missing: 'That host has no model called “%model%”. Pull it there, or name one that is on it. (%error%)' + no_host: 'Set a model host first, and save.' + no_model: 'Name a writing model first, and save.' + status: 'The host refused with HTTP %status%. (%error%)' + timeout: 'The host was still loading after four minutes and plMail stopped waiting. It may well finish; the model is very likely too big for the memory it has.' + unexpected: 'Something went wrong before the host could answer. (%error%)' + unreachable: 'Nothing answered at that address. (%error%)' config_backup: change: absent: 'nothing set here yet' diff --git a/translations/messages.en_PI.yaml b/translations/messages.en_PI.yaml index efd9e7ed..855d035c 100644 --- a/translations/messages.en_PI.yaml +++ b/translations/messages.en_PI.yaml @@ -95,8 +95,12 @@ admin: base_url_invalid: 'That must begin with http:// or https://.' categorisation: 'Sort mail into holds' categorisation_help: "Runs on ev'ry message that arrives. Only breaks a tie — the ordinary rules speak first." + chat_keep_alive: "Keep the scrivener aboard for" + chat_keep_alive_help: "5m, 30m, 4h, or a plain count o' seconds. -1 keeps it aboard for good, 0 puts it ashore after ev'ry errand. Longer means the first reply comes quicker — but the berth stays taken the whole while. Leave it empty to sail by whatever the host itself says." chat_model: 'Scrivener model' chat_model_help: "For replies, summaries an' sortin'. Somethin' instruction-tuned." + embedding_keep_alive: "Keep the diviner aboard for" + embedding_keep_alive_help: "Same shape as above. This one be small an' called for constantly, so -1 — aboard for good — mostly costs barely a crumb an' spares a wait on ev'ry search; worth settin' if the host has the hold to spare. Leave it empty to sail by whatever the host itself says." embedding_model: "Divinin' model" embedding_model_help: "Turns mail into vectors so the search finds meanin', not words." enabled: 'Consult an oracle' @@ -211,6 +215,19 @@ admin: on: 'On' paused: 'Becalmed' test: 'Test' + warmup: + button: 'Rouse it now' + failed: "Nothin' came back. Try again." + loaded_ms: 'Model roused in %count% ms — it were already aboard.' + loaded_seconds: 'Model roused in %count% s.' + loading: "Rousin' the model…" + model_missing: "That host carries no model named “%model%”. Haul it aboard there, or name one that be. (%error%)" + no_host: "Name a model host first, an' stow it." + no_model: "Name a scrivener model first, an' stow it." + status: 'The host refused with HTTP %status%. (%error%)' + timeout: "Four minutes an' the host were still haulin', so plMail stopped waitin'. It may yet finish — like as not the model be too heavy for the hold it has." + unexpected: "Somethin' went wrong afore the host could answer. (%error%)" + unreachable: "Nothin' answered at that address. (%error%)" db: active: empty: 'No queries afoot right now.' diff --git a/translations/validators.de.yaml b/translations/validators.de.yaml index 8a0da5aa..aadf89a3 100644 --- a/translations/validators.de.yaml +++ b/translations/validators.de.yaml @@ -1,5 +1,6 @@ # Symfony übersetzt Validierungsmeldungen in der Domain `validators`, nicht in # `messages` — ein Schlüssel in messages.*.yaml erscheint unübersetzt. +admin.ai.keep_alive_invalid: Nimm eine Dauer wie 5m, 30m oder 4h, eine reine Sekundenzahl oder -1, damit das Modell dauerhaft geladen bleibt — leer lassen nimmt die Einstellung des Hosts. admin.integrations.client_id_missing: Trage die Client-ID ein, die zu diesem Secret gehört. admin.integrations.secret_missing: Trage auch das Client-Secret ein — mit der Client-ID allein kann sich niemand anmelden. admin.users.password_too_short: Mindestens {{ limit }} Zeichen — dieses Passwort wählst du für jemand anderen. diff --git a/translations/validators.en.yaml b/translations/validators.en.yaml index 8616f940..2323c866 100644 --- a/translations/validators.en.yaml +++ b/translations/validators.en.yaml @@ -1,6 +1,7 @@ # Constraint messages. Symfony translates validation output in the `validators` # domain, not `messages` — a key put in messages.*.yaml renders as the raw key, # which is how "setup.install.password_too_short" ended up on screen. +admin.ai.keep_alive_invalid: Use a duration like 5m, 30m or 4h, a plain number of seconds, or -1 to keep the model loaded for good — leave it empty to use the host's own setting. admin.integrations.client_id_missing: Enter the client ID that goes with this secret. admin.integrations.secret_missing: Enter the client secret too — a client ID on its own cannot sign anyone in. admin.users.password_too_short: Use at least {{ limit }} characters — this password is being chosen for someone else. diff --git a/translations/validators.en_PI.yaml b/translations/validators.en_PI.yaml index 0118eacc..4dae9bf2 100644 --- a/translations/validators.en_PI.yaml +++ b/translations/validators.en_PI.yaml @@ -1,4 +1,5 @@ # Symfony be translatin' validation messages in the `validators` domain. +admin.ai.keep_alive_invalid: "Give a spell like 5m, 30m or 4h, a plain count o' seconds, or -1 to keep it aboard for good — leave it empty an' the host's own word stands." admin.integrations.client_id_missing: Name th' client ID that goes with this secret. admin.integrations.secret_missing: Enter th' client secret an' all — an ID alone signs nobody aboard. admin.users.password_too_short: "{{ limit }} marks at least — ye be choosin' this one for somebody else."