* Single sign-on only: hide the password form and refuse password sign-ins
Adds Admin -> SSO 'Single sign-on only' (and OIDC_DISABLE_PASSWORD_LOGIN).
The login page drops the password form and LoginFormAuthenticator refuses a
password sign-in before looking the user up. It only applies while a whole
provider is in force, the saving administrator must already be linked to the
provider, and app:sso:allow-password-login brings the form back from the
console. Config restore leaves it off. Off by default; one boolean column on
oidc_config.
Closes#169
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* Single sign-on only: require a link to the provider being saved
The admin form let the box be ticked by an administrator linked to any
provider, including the one the page held before. Switching providers and
ticking it in one save made a provider they had never used the only way in.
It now compares the stored issuer with the one on the page, without a
trailing slash, as OidcDiscovery does.
Tests for what had none: the console command (clears the setting, keeps the
provider), a restore over a row that had it on, the link to another
provider, and a control showing the same password signs in with the setting
off. The "right credentials" test now posts the right password.
Docs: there is no iOS app.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Docs: put the iOS app back in the single sign-on page
The previous commit dropped it on the wrong assumption that there is no
iOS app. There is one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: Paul Lützner <karatektus@gmail.com>
Compose files keep only a short header; the reasoning lives in
docs/install. truenas.compose.yaml is gone (the TrueNAS catalog app
replaces it) and so is compose.prod.yaml (compose.yaml already runs the
published prod image). compose.override.yaml.dist becomes compose.dev.yaml,
passed with -f, so there is no untracked copy that compose loads on its
own. Docs, CODESTYLE, CONTRIBUTING and the entrypoint's error text follow.
Every remaining file resolves to the same `docker compose config` as
before.
Closes#30
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
plMail could only sign people in with a local password. On an install that
already runs Authentik or Keycloak, that is one more account and password
per person, outside the provider's access policy.
A second authenticator on the `main` firewall, beside the password form.
Both end in the same token for the same User, so the user checker, the login
throttle and TOTP two-factor apply unchanged. Authorization code flow with
PKCE, state and nonce on league/oauth2-client's GenericProvider; endpoints
come from the provider's discovery document.
- Set up under a new Admin → SSO tab; the OIDC_* variables are the fallback
for an install configured from its compose file, and a saved provider wins
whole. This lets an administrator who points it at a provider of their own
sign in as any account without a second factor; the security model says so.
- The ID token's signature is not verified: it only ever comes from the
token endpoint over TLS (OIDC Core § 3.1.3.7 step 6). So the issuer and
endpoints must be https and no request to the provider follows a redirect.
Accepting an ID token from anywhere else needs JWKS verification first.
- An account is linked by (issuer, sub). The address is used for the first
link only, and only when the provider reports it verified.
- Auto-creation is off by default and creates an ordinary account with no
usable password.
- app:user:oidc-unlink forgets a link from the console, for somebody deleted
and re-added at the provider.
Migration: two nullable columns and a unique index on `user`, and a one-row
table `oidc_config`. Nothing backfilled; reversible.
Not run: Playwright on this branch. No automated test makes a round trip
against a real provider; that was checked by hand on a local test stack.
Closes#91
Co-Authored-By: Paul Lützner <mail@pluetzner.de>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stock compose file served https://localhost and nothing else. Reached
by address or hostname, Caddy had no certificate to offer and ended the
handshake, which a browser reports as an SSL protocol error with no way
to proceed. A headless server could not be set up without first setting
SERVER_NAME.
The default now answers every name and address with the self-signed
localhost certificate (default_sni/fallback_sni, and `:443` in the default
SERVER_NAME), so the browser shows its usual warning.
Live updates follow: compose.yaml's https://localhost hub address is
treated as the placeholder it is and gives way to the public address
stored during setup. Any other MERCURE_PUBLIC_URL, and any SERVER_NAME an
operator sets, is left alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Starting plMail took over two minutes on a NAS with spinning disks, and
the log was silent for all of it — between "Statement statistics are
enabled" and the web server's first line. An empty install on the same
box, from the same image, was up in seconds.
The entrypoint ended with two `setfacl -R` passes over var/cache and
var/log. var/cache is part of the image: about 2,200 files. Changing
anything about an image file from inside a container, an ACL included,
makes the storage driver copy the whole file into the container's layer
and flush it. So every new container rewrote its cache one synchronous
file at a time, in every container of the stack at once, on one pool.
Measured in a fresh container on an SSD: 1.2 s for the first pass,
0.03 s for the second. On the warm production container, 0.06 s — which
is why timing it there showed nothing.
It ran only as root, so the catalogue install (uid 568) never paid it.
Nothing needed the entries: nothing runs as www-data, root needs no
grant, and the Dockerfile already makes both directories writable.
This is the third time that block has been the answer to a slow start.
It walked all of var/, then var/cache and var/log, and now nothing.
Also, on the compose files here: the database health check had no
interval, so Docker's first probe came thirty seconds after start and
the web server and workers could not begin before it. Five seconds now,
with retries raised so a database in trouble is called unhealthy after
the same two and a half minutes as before.
The README sends TrueNAS users to the catalogue app instead of walking
them through truenas.compose.yaml. The file stays: installs made from
it are still running.
Not confirmed on the box that reported it; the copy-up cost is measured
locally and inferred there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adding an account put new mail behind hours of follow-up work. What
plMail does to a message once it has it — a model call where the
assistant sorts, event extraction, insight cards, receipts — was queued
on `ingest`, beside the syncs, for every message of the import, oldest
first. One consumer serves that queue. 988 envelopes pending and the
oldest an hour old was a mailbox being asked about newsletters from
2019 while mail on the server waited to be fetched.
`ingest` is now what talks to a mail provider. The rest moved:
enrich_live worker-live follow-up work for mail that has just
arrived on an account whose import is
over; never behind an import
enrich worker-enrich the same for an import's recent mail,
and search indexing
enrich_backlog worker-enrich old mail for the assistant, consumed
only when `enrich` is empty
release worker-release ends a hold (below); nothing else
Which queue is decided per batch, not per message class: the pipeline
asks InitialImportState once, carries the answer on PostIngestResult,
and EnrichmentRouter turns it into a stamp.
Only mail from the last 30 days (ENRICH_RECENT_DAYS) gets follow-up
work on arrival. Older mail is stored, threaded and filed by the rules.
The assistant is asked about it afterwards — newest first, a bounded
handful per run, and not until every account of that person has
finished importing (app:ai:classify-backlog, ClassificationCatchUp).
For the other steps `app:backfill` remains the way to read history.
ExtractInsightsStep and ProposeEventsStep ran their work inside the
sync and argued for it; both dispatch now.
New mail no longer appears in one tab and moves to another. For
somebody sorting by assistant the rules' category is a guess about to
be replaced, so where the verdict could still change it the message is
held: stamped before threading, its new thread given no category, which
every tab query already reads as "in no tab". The answer releases it.
Failing that a timed release does, after AiSettings::holdMaxSeconds —
60 by default — and the mail is shown where the rules put it; the model
call carries on and moves the mail when it answers. A message is asked
about once. Admin → AI has the switch, the ceiling, and the measured
delay over the last week beside them.
Three things about the release that were each wrong once first:
- It ran on the live worker, so it could not fire while that worker
was inside the call that was taking too long. It was moved to
export, which mixed mail appearing with mail leaving. It has a
queue and a worker of its own.
- The call was cut off at the ceiling and the message asked about
again later. The hold and the question are separate; only the hold
has a deadline.
- A late verdict was stored and moved nothing. It moves the mail.
Both workers still end at one model host, so bulk classification looks
at the live queue before each call and stands aside (LiveMailPriority).
One migration: two columns on `message`, two on `ai_settings`.
Not verified against a real import or a live model host: the tests pin
the queues, the hold and the release against in-memory transports and a
mocked host. Known and left: IMAP still imports oldest first; a reply
into an existing thread cannot be hidden; a Gmail account counts as
importing for up to an hour after its last message lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Packaging plMail for a catalogue that forbids per-app workarounds turned
up four places where the image expected its compose file to compensate.
cookie name Chosen from the public address: the __Secure- name on
https, a bare one otherwise. It used to be prefixed
unless an operator knew to set MERCURE_COOKIE_NAME, and
an install opened on http://ip:port never got a live
update. The hub always reads the bare name and the web
server renames the cookie in the proxy, so the two
cannot disagree after only one of them restarted.
storage dir An absolute APP_STORAGE_DIR inside the project is
accepted and normalised to the relative form, so the
stored attachment paths do not change. One outside it
is refused at start.
hub address MERCURE_UPSTREAM names the container the web server
proxies the hub to, for a stack that does not call it
mercure.
database secret No POSTGRES_PASSWORD is minted, and no password file
written, when DATABASE_URL already carries a password:
that database is somebody else's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stack was nine long-running containers: the web server, the
database, the Mercure hub, and six of plMail's own image, one per
background process. It is three now: php, worker and database, with
secrets-init still running once and exiting.
The worker runs `app:work`, a small supervisor keeping seven processes
alive the way Docker's restart policy kept seven containers alive: the
IMAP supervisor, one consumer per queue, the scheduler and the hub. A
clean exit (the hourly recycle, the restart signal) comes straight back;
a crash backs off from one second to thirty. Output passes through a
whole line at a time, so two processes' JSON records never interleave,
and SIGTERM reaches every child at once with 25s to finish. Each queue
keeps a process of its own, which is the part the split was for: a
consumer inside a long handler cannot take a send. And each process
keeps its container's old name as APP_CONTAINER_NAME, so heartbeats,
/healthz and the log filters are unchanged.
The hub is the official Mercure 1.x binary, copied into the image and
run as one of those processes. Not FrankenPHP's built-in hub, which is
0.24 and refuses the RFC 9068 tokens plMail has minted since v0.2.31.
The worker answers to `mercure` on the network, so the Caddyfile proxy
and MERCURE_URL reach it where they reached the hub container, and older
compose files keep working with this image unchanged. It is configured
from the application's own settings, including MERCURE_COOKIE_NAME, so a
plain-HTTP install names the cookie once for both halves. A 5s shutdown
grace stops it waiting on open subscriptions, which made every stop take
the whole 25s.
The dev override loses its per-worker blocks, one of which (worker-bulk)
had been missing, so that queue ran the published image's code in
development. The demo overlay runs app:work --without=imap-supervisor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
v0.2.31 could not mint a token at all on a TrueNAS install, and nothing an
operator set would have fixed it.
That release bound `iss` and `aud` to MERCURE_PUBLIC_URL. truenas.compose.yaml
leaves that BLANK ON PURPOSE — the setup screen asks for the public URL on first
boot and stores it in the database, which is long after the hub container has
had to be told what to trust. An empty issuer is not a weak issuer: the factory
refuses it outright with `The "iss" additional claim is required by RFC 9068
access tokens`, so there was no subscriber cookie, every publish threw, and the
symptom was a red dot indistinguishable from a hub that was simply down.
The mistake underneath was deriving a value that must exist at CONTAINER START
from one that only exists after SETUP. So it is a literal now, shared by the
application and every compose file, and no installation has to know it. An
identifier does not have to resolve — `.invalid` is reserved by RFC 2606 for
exactly this — and all that matters is that both halves agree on the same
string, which a literal guarantees and a derived value cannot. The same string
everywhere costs nothing: the signing secret is generated per installation, so
one install's token has never been usable against another's hub however it
identifies itself.
truenas.compose.yaml had been left behind twice more. It still carried the
"/healthz" check that v0.2.30 replaced in the other two files, so its hub would
have gone unhealthy the moment the image moved; and it had no pull_policy, so
the image never moved at all. That is the combination that produced the original
report: an application on the 1.0 protocol talking to a 0.x hub that nothing had
ever pulled.
The regression test empties MERCURE_PUBLIC_URL rather than reading the config,
because what broke was the derivation — a test that read the same placeholder
the code did would have passed throughout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v0.2.31 moved plMail to the Mercure 1.0 protocol and assumed the hub had moved
too. On a real install it had not, and the result is the mirror image of the
bug that release fixed: plMail speaking 1.0 to a 0.x hub, red dot, no live
updates, nothing in plMail's own logs to say why.
A TAG IS A POINTER, NOT A SUBSCRIPTION, which is the whole of it. Compose
defaults to pull_policy: missing — it pulls only when the image is absent, so
`dunglas/mercure` present on the box means "use it" and the registry is never
asked whether the tag has moved. Restarting pulls nothing either. The app image
updates because the app is the thing being updated; the hub is a sidecar nobody
mentions, and it sits on the digest it first resolved. Leaving the tag unpinned
therefore bought exactly nothing: the tag floated and the image did not.
pull_policy: always is the missing half of the decision not to pin. It costs one
registry round trip per `up` that answers "same digest" almost every time.
The changelog entry said "Nothing to configure" and was wrong three ways: the
hub image must be pulled, an install with its own compose needs
MERCURE_TRUSTED_ISSUERS and a pinned resource_identifier that no image pull
supplies, and a plain-HTTP install needs MERCURE_COOKIE_NAME. It now says all
three, and leads with the pull, because that is the one that bites an install
that changed nothing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Live updates were dead: every subscription came back 401 and the inbox sat on
"Live updates unavailable" while the hub answered its own healthcheck happily.
The hub had moved to the 1.0 protocol and now wants RFC 9068 access tokens — an
`iss` it has been told to trust, an `aud` naming the hub the token is for, and
`authorization_details` in place of the bare `mercure` claim. plMail was minting
the older shape.
NOT HAND-ROLLED CLAIMS. symfony/mercure already implements 1.0, so this is
configuration: `protocol_version: '1.0'` plus the three claims the bundle
requires (it refuses to build the container without `iss`, `sub` and
`client_id`, which is how the requirement was found). They are merged into the
token FACTORY as well as the publisher provider, so the subscriber cookie gets
them too — one block covers both halves.
The protocol switch also changes the subscription query parameter from `topic`
to `match` and the cookie's name. Both belong to the bundle — the `mercure()`
Twig helper writes the URL, Authorization writes the cookie — which is why no
JavaScript and no controller is touched here.
`resource_identifier` IS PINNED ON THE HUB, and that is not decoration. A 1.0
hub works its expected audience out from each request, and plMail reaches it by
two roads: the browser subscribes at MERCURE_PUBLIC_URL, the application
publishes to the internal MERCURE_URL. One token cannot name both, so a hub
left to derive its own answer would refuse every publish. Issuer and audience
are both read from MERCURE_PUBLIC_URL so the app and the hub cannot be
configured into disagreeing.
THE `__Secure-` COOKIE is the one thing an operator may have to know about. 1.0
names the cookie `__Secure-mercure_access_token`, and a browser drops a
`__Secure-` cookie that did not arrive over TLS. Right for every real install,
fatal for the test stack, which serves plain HTTP on 127.0.0.1 — so the name is
a parameter defaulting to the secure one, and only the test stack overrides it,
with the hub told the same name.
Verified in a browser against the current hub image: the indicator reads
"connected", and the subscription is GET /.well-known/mercure?match=mail%2Fuser
%2F1 → 200. Full suite green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-up to eca0f8b7, which pointed our healthcheck at the hub's new
readiness path. Correct, and still the wrong shape: it leaves plMail asserting
something about the internals of an image that is deliberately unpinned so it
can update itself. The next time the hub moves that path, CI goes red for
another three days.
The image has always shipped its own HEALTHCHECK — wget against
127.0.0.1:2019/mercure/health/ready, 60s grace, 5s timeout, 5 retries — which
is the same grace and retries compose.yaml was spelling out by hand. Inheriting
it costs nothing there and cannot drift, because it travels with the image it
describes.
The test stack wanted to keep its five-second interval and inherit the command,
and Compose refuses: a healthcheck block without a `test` is rejected with
"must define a healthcheck command for all healthchecks". So owning the cadence
means owning the command again, which is the thing that rotted. It asks every
30s now — a few more seconds before a job starts, against never again waiting
for ever.
Verified against the current image: the running container carries the image's
healthcheck verbatim and reports healthy, and the test stack comes up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every E2E job has failed since 18 September with "dependency failed to start:
container pl_mail_test-mercure-1 is unhealthy". The hub was fine. Our
healthcheck was not.
dunglas/mercure is pinned to a floating tag, and the image moved its readiness
probe. "/healthz" now answers 404; `curl -f` treats a 404 as failure; the
container is marked unhealthy for ever while serving every subscription put to
it. Verified against the current image: /healthz gives 404, and
127.0.0.1:2019/mercure/health/ready gives 200 — which is the path the image's
OWN healthcheck has always used, and a container left to use it reports healthy.
Overriding the image's healthcheck with a URL of our choosing is precisely what
let the two drift apart, so the fix is to ask where it asks.
compose.yaml carried the same line, and that half was shipped. Nothing there
depends on the hub being healthy, so live updates kept working and the symptom
was an install permanently showing "(unhealthy)" in docker ps and in the
TrueNAS view — about nothing at all.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RunBulkStatusHandler exists to take a mark-read or archive over a whole view
out of the request — its docblock quotes the thirty-second timeout that
produced it, and BulkStatusController answers with an indicator so nobody
waits. Neither was true: the message was never listed in messenger.yaml's
routing map, and Messenger handles an unrouted message inline in whichever
process dispatched it. Every whole-view action ran inside the web request,
under the limit it was written to escape, at request concurrency — which is
where the production deadlock came from.
It gets a queue of its own rather than joining maintenance, because somebody
is watching an indicator for it and queueing it behind an embedding backfill
would be slower than the inline version it replaces. That means one new
container, worker-bulk, in every compose file and in the handbook.
The deadlock itself is retried now instead of failing the job, three times per
chunk, and a collision that outlasts those hands the job back to the queue
rather than ending it. The failure path no longer writes through an
EntityManager Doctrine has closed, which is what it was doing for every
failure that came out of a flush.
And the list catches up: nothing ever refreshed it on the strength of the job,
only on the toolbar's own "the write landed" event, which is now the instant
the work was QUEUED rather than finished.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docker compose resolves ${VAR} against the .env file beside compose.yaml,
and the .env beside ours is Symfony's — committed, full of development
defaults, and never written with compose in mind. So APP_ENV: ${APP_ENV:-prod}
was not a default: .env says APP_ENV=dev, and that is what every operator who
ran `git clone && docker compose up -d` actually got.
The published image is built --no-dev and has no DebugBundle, which
bundles.php loads in dev. Every bin/console call died on the missing class
before it could do anything — including the entrypoint's database probe, which
then reported "The database is not up or not reachable" while the database sat
there healthy. Six services in a restart loop, and the message pointing at the
cause was not the one printed.
APP_ENV is hardcoded to prod here now. Running this stack in dev is what
compose.override.yaml.dist is for, and it still merges over the top.
Verified the way it broke: a fresh clone, `docker compose up -d`, nothing else,
against the published image — all eight containers healthy, APP_ENV=prod
inside, and the setup screen served.
ComposeEnvironmentTest holds every ${VAR} in the shipped compose files to the
rule this turned on: interpolate only what .env leaves alone, or leaves at
exactly the value written in the compose file. DATABASE_URL is the one
documented exemption. Text parsing rather than `docker compose config`, so it
fails in CI without a Docker daemon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README says what plMail is and gets you running; CONTRIBUTING says how to
change it. Nothing covered the part in between — using it, operating it,
and registering the Google and Microsoft applications it talks to. That
is docs/: 34 pages across features, installation, providers and
internals, mirrored to the GitHub wiki.
The wiki is a mirror and never a source. bin/mirror-wiki.php flattens the
tree into the wiki's flat namespace, rewrites every relative link,
generates the sidebar from the index so the two cannot disagree, and
stamps each page with a footer saying where it came from. A browser edit
survives until the next push to main. That cost is deliberate:
documentation not versioned with the code drifts from it, and the drift
is invisible until somebody follows an instruction that stopped being
true.
Which is the whole problem, so the checkable half is checked.
DocumentationCoverageTest fails the build when a command is missing from
the reference or documented and gone, when a variable in .env is absent
from the configuration page, when a relative link dangles, when a page is
unreachable from the index or promised and missing, or when a page has no
"Things that bite" section. It cannot check whether a paragraph is TRUE,
and says so — this is the floor, so review attention goes to the part a
machine cannot have an opinion about. CONTRIBUTING has claimed since it
was written that the command table "is part of the definition of done";
nothing enforced it until now, and it was wrong in three places.
Writing it found things. They are fixed here rather than documented
around, because a handbook describing a bug accurately is still a
handbook nobody can follow.
compose.yaml mounted no volume for var/attachments, var/raw
or var/uploads. Workers write those during sync
and the web container serves them back, so each
kept its own copy in its own layer — downloads
404, and everything died on a recreate. The
override and the TrueNAS file both mount them;
the file the README tells an operator to run did
not. The same omission cost real data once
before. See the CHANGELOG for moving what is
currently stranded.
app:calendar:materialise new, nightly. Occurrences are drawn when an
event is saved and nothing moved that window
again, so a weekly standup ran out of dates in
two years and its reminders stopped with them —
silently. The query it runs had existed since
the materialiser did, documented as exactly this
sweep, and had never been called.
Microsoft wizard asked for three of the nine permissions plMail
requests, and a token already issued is not
upgraded when the rest are added later.
GmailWatchService told you to grant Pub/Sub Publisher to
gmail-api@system.gserviceaccount.com. The
account is gmail-api-push@; a topic granted
exactly what the docblock asked for never
delivered a notification.
IcsExporter said it wrote no VALARM. It has written them
since alerts landed.
CLIENT_DEVELOPMENT.md said there was no JMAP calendar API and that
SearchSnippet/get was absent. Both shipped.
CONTRIBUTING said only app:calendar:push registers a push
channel; subscribing has done it since the
subscribe flow learned to.
Three docblocks named getters removed long ago, or described a
method two below the one they sat on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Seventeen message types shared one transport and one worker, so they were
ordered by arrival and nothing else. Pressing Send behind a Gmail batch meant
waiting for the batch. The queue was deciding by accident what should be
decided on purpose.
Three now, named for who is waiting. export is anything leaving plMail — the
send itself, the outgoing label and flag pushes, notifier traffic — and is the
only queue a person is watching. ingest is mail arriving and the work that
immediately follows it. maintenance is bulk and deliberate: backfills, a rule
applied to existing mail, admin sweeps.
Three worker processes rather than three transports on one, and that is the
part that matters. Priority ordering does not help: a worker already inside a
long handler cannot pick up a send however the queues are ranked. Only a
separate process makes sending independent of syncing. Measured before
choosing — the existing worker holds 76MB, so two more cost about 150MB
against the 1.1GB the IMAP supervisor already uses, and running three
processes in one container would have meant adding a supervisor the image does
not have, with multiplexed logs and hangs Docker cannot see.
export retries faster than the other two. Its failures are a relay refusing a
connection while somebody watches the outcome, not a rate limit that needs
minutes; five attempts over about ninety seconds rather than eight minutes.
async stays declared and consumed by the maintenance worker, because envelopes
queued before the upgrade still need somewhere to land. Nothing routes there.
Remove both once a release has drained.
Four tests asked the async transport what had been sent and now ask export.
That coupling is deliberate: which queue an outgoing push lands on is the
behaviour, and routing one onto ingest would put a user's archive back behind a
mailbox sync.
Verified by dispatching one of each against the dev stack: send landed on
export, sync on ingest, rule run on maintenance.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NTFY_BASE_URL was one more thing to get right, and it did not need to be: the
host a user sets at first boot is already the address phones can reach, which
is the only property this URL has to have. It now defaults to
http://$SERVER_NAME:8090, so enabling push is the compose command and nothing
else. An explicit NTFY_BASE_URL still wins, for a separate hostname or TLS
terminated in front.
The obvious tidier answer does not work. ntfy cannot be hosted under a
sub-path -- it refuses a base-url containing one at startup -- so it cannot be
folded in behind the app's own Caddy at /push the way the Mercure hub sits at
/.well-known/mercure. Hence a port of its own, and that is a limitation of ntfy
rather than a choice worth revisiting.
The derived URL is http, which is right for the LAN and Tailscale deployments
this is usually reached over and wrong for the open internet: the endpoint URL
is itself the secret, so anyone able to watch the traffic could learn a topic
and publish noise to it. They still could not read anything -- the payload is
encrypted to the device's key before it arrives -- but facing the internet you
want TLS in front and NTFY_BASE_URL set to the https address. The comment says
so where someone deploying it will read it.
Verified with nothing but SERVER_NAME set: the container comes up with the
derived URL, the distributor issues an endpoint on it, and the subscription
comes back from PushSubscription/get verified.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Web Push assumes a push service: something that owns the endpoint URL, holds
the connection to the device and receives our encrypted POST. Browsers ship
one. A native Android app does not, and Android's own is FCM, which speaks its
own protocol and cannot be reached by WebPushSender at all.
That left a self-hoster with a gap this repository was well placed to close and
did not: to get push without Firebase they had to find and run a push service
themselves, having already been asked to self-host their mail.
So ntfy is available here, behind a `push` profile and off by default:
NTFY_BASE_URL=https://push.example.com docker compose --profile push up -d ntfy
It adds no application code. ntfy is simply a push service speaking the
protocol WebPushSender already emits -- RFC 8030 transport, RFC 8291 payload
encryption, RFC 8292 VAPID -- so a UnifiedPush distributor on the phone can
receive what this server already sends today. Nobody who does not want push
pays for a container they never start.
Two settings are worth reading before exposing it. NTFY_BASE_URL is the URL
*phones* use, so it must be reachable from outside this machine, and it is
baked into every endpoint issued -- changing it later forces every device to
re-register. And access is open by default, which is a considered choice rather
than an oversight: UnifiedPush topics are random and unguessable, and payloads
are encrypted to the device's own key before they arrive, so this service
cannot read mail and someone who guessed a topic could only send noise the app
discards. The comment says how to lock it down for an internet-facing
deployment.
Verified end to end against the isolated Android test stack: the app registers
with the distributor, ntfy issues an endpoint, the app registers it through
PushSubscription/set, this server POSTs its encrypted PushVerification there,
the connector decrypts it and echoes the code back. PushSubscription/get then
returns the subscription with no verificationCode, which is what live means.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three faults, one story: the stream could die silently, nothing told the
user, and nothing tested any of it.
**It did not recover.** EventSource retries on its own only when a
connection drops cleanly. An HTTP error response is fatal per spec —
readyState goes to CLOSED and the browser never tries again — and our
Caddy answers 502 for the seconds a hub restart takes. Measured: an open
stream, a 20s outage, and the tab stayed dead after the hub was back.
Every deploy did this to every open tab. There is now a reconnect loop
with jittered backoff, plus an immediate retry when a tab is refocused
or the network returns.
**Nothing said so.** The mail list has no polling fallback, so a dead
stream looks exactly like a quiet mailbox — the inbox stops changing and
nothing on screen explains why. A glow behind the logo now carries the
state, quiet when healthy, and announced rather than colour-only. While
the stream is down the list falls back to a 60s poll: strictly the worse
mechanism, which is why it runs only when the good one is unavailable.
**The hub accepted anyone.** dev.Caddyfile was in use in every
environment including TrueNAS, and it sets `anonymous`. Since
/.well-known/mercure* is proxied publicly, anyone who could reach an
instance could subscribe to mail/user/<any id> and watch that account's
sync activity. Both stacks now run the image's default config, which
requires the subscriber JWT.
That JWT expires — an hour by default, well inside a remember-me
session — and the hub only checks it when a connection is made, so the
expiry is invisible until the next reconnect is refused. /mercure/auth
re-issues the cookie, and the client refreshes it before every retry.
The cookie is attached by a response subscriber rather than by
`mercure(topics, {subscribe: …})` in the layout, because that helper
throws for a hub on another second-level domain and a throw in a
template takes the whole page with it. PHPUnit caught it at once: its
client requests as localhost against a 127.0.0.1 hub, and every
page-rendering test turned 500. An external hub is a legitimate
deployment; it should cost live updates, not the page.
Tests: compose.test.yaml grew a real hub, running the same non-anonymous
config as production so the suite exercises the authorization path
instead of one that accepts anything. The recovery spec stops the
container — `setOffline` does not sever an established SSE connection
and route interception never fires without a request, so both looked
like passing tests of nothing. It lives in the exclusive project: there
is one hub, and taking it away mid-run breaks specs that never asked.
Verified by disabling the reconnect and watching the recovery test fail,
then restoring it and watching it pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
