Upgrade Runs
A run is a mass upgrade of web applications across accounts, started with
elpanctl webapp:upgrade and followed on Upgrade Runs in the System
menu (also linked from Web Applications). It's how
an advisory against WordPress gets answered on every site of the server in
one command.
The screen is read-only; a run starts only from a root shell, with
--no-dry-run. An upgrade an account holder starts in their own panel isn't
a run; it appears on the installation.
Two authorities
The nightly sweep and webapp:upgrade upgrade the same sites under
different rules. The sweep is the account holder's opt-in, run while nobody
watches. webapp:upgrade is yours, run while you watch.
Nightly sweep (webapp:auto-upgrade) | webapp:upgrade | |
|---|---|---|
| Owner turned Automatic Updates off | Skips | Proceeds |
The no-upgrade tag | Skips | Skips |
| No recent backup of the account | Skips | Proceeds, unless --require-backup-within-days |
| A new major version | Never | Only with --allow-major |
| Plugins, themes, modules | Never | Never: they're the account holder's, from their panel |
| Re-applies the application's file permissions after a success | Yes | Yes |
An account holder who declines unattended updates hasn't declined a
security fix. The no-upgrade tag is the one way to say "never touch
this", and both paths honour it. The account holder sets it on the
installation (the account docs'
Applications and updates).
Start a run
--type is required: a sweep across every application type at once is not
a thing anybody means to do. The dry run is on by default and prints the
rows the real run would touch.
elpanctl webapp:upgrade --type=wordpress # dry run
elpanctl webapp:upgrade --type=wordpress --version-below=<version> --no-dry-run
elpanctl webapp:upgrade --type=drupal --version-below=<version> --limit=20 --no-dry-run
elpanctl webapp:upgrade --type=wordpress --account=<account> --no-dry-run --sync
| Option | Effect |
|---|---|
--version-below | Only installations below this dotted version. The number an advisory names. |
--to | A target version. Default: each installation's newest release in its own branch, from the release feed. |
--account | One account, by username or UUID. |
--limit | At most this many installations queued in one run. Default 100. |
--allow-major | Allow a target in a newer major version. Off by default: a major upgrade breaks themes, plugins and extensions. |
--include-unserved | Also installations no website serves: backups, staging copies, unpacked tarballs. Off by default. |
--require-backup-within-days | Skip installations whose account has no home-directory backup in the last N days. |
--sync | Run each upgrade inline, in this shell, instead of leaving it to the worker. For small batches and debugging; needs --no-dry-run. |
The command prints one row per matched installation (account, type, path, from, to, and what it would do or did), then a summary: how many matched, how many are queued, how many refused and how many deferred, and the run's id. A refusal names its reason (Skipped and failed).
Every cap announces itself. When more installations matched than --limit
allowed, the command says so and tells you to run it again for the next
batch. When installations were held back because another upgrade on the
same account is already queued or running (one at a time per account is
deliberate), it says that too. A truncated sweep reads like a complete one
otherwise, and that is how a sweep gets declared finished with half the
server still vulnerable.
What can be upgraded
| Application | Core upgrade | Notes |
|---|---|---|
| WordPress | Yes, with wp-cli | Plugins and themes are the account holder's, from their panel. |
| Drupal 8 and later | Yes, with Composer and the project's own Drush | A tarball install has no composer.json and is refused. Modules update one at a time from the account's Modules tab. |
| Drupal 7 | No | End of life, no automatic path. |
| Laravel, Symfony | No, by design | Frameworks. What is installed is the account holder's own application, and moving it forward is development work. |
| Everything else | Not yet | Listed with its version; no upgrader. |
Before every upgrade the panel exports the installation's database to
~/.local/share/elpan/webapp-upgrade/<installation id>.sql in the
account's home directory, outside the document root, one file per
installation, overwritten each run.
Follow a run
Upgrade Runs lists every run, newest first: when it started and who
started it (cli: and the Unix user of the shell), the selection (type,
version bound, account), the target, how many upgrades started out of how
many installations matched, the succeeded, failed and skipped
counts so far, and when it finished, or in progress.
A run's page has the same facts in full, including whether major upgrades were allowed and whether the scope was Installations served by a website only or Every installation, served or not, and while it runs, how many upgrades are still queued or active. Below, one row per installation: account, path and type, the version it went from and to, the outcome, the detail, and when the result was recorded. A row whose installation was deleted since stays, marked deleted since: a sweep's report must not lose the site that failed.
A run stays in progress until every result is written. When the page
says nothing is queued and nothing runs, and that doesn't change, no worker
is reading the webapp transport, and the run won't finish on its own.
When a run finishes, a plain-text digest of how it went goes to the server's root mailbox (System mail).
Skipped and failed
Skipped: nothing was touched. The detail names the reason, among them:
- the installation is tagged
no-upgrade - the account is suspended or expired
- the last scan could not read the installation
- it can't be upgraded in place in its state (a WordPress without
wp-config.php, a Drupal withoutsettings.php) - it's already at or newer than the requested version
- the target is a new major version and
--allow-majorwasn't given - the panel doesn't know which version to upgrade to (no feed, or a stale one)
- an update of the application was already in progress
- there is no recent backup, when
--require-backup-within-daysasked for one - the panel recognizes the application but can't upgrade it
Failed: the upgrade started and stopped part-way. The detail carries the tool's own output, the command that puts the files back to the earlier version, and the location of the database export. The earlier version is kept on the installation for that reason.
A half-updated tree was never in a valid state, so an automatic rollback is more likely to break the site than repair it. The panel stops, records what happened, and names the way back. Restoring the files and the database export is a decision for a person, made on that site.
A successful upgrade re-applies the application's declared file permissions, because an upgrade is what moves them. A permission the account holder set by hand inside the installation is replaced; a failure to apply is recorded in the detail and doesn't fail the upgrade.
Plugins and themes with open advisories
webapp:remediate is the sibling command for the other half of WordPress
security: it updates the plugins and themes that open advisories are
published against, across accounts, with the same dry-run default and the
same --account, --type, --limit, --include-unserved and --sync
options, and selects by --advisory, --component (e.g.
wp:plugin/akismet), --kind (plugin, theme) or --severity.
elpanctl webapp:remediate # every finding the panel can act on
elpanctl webapp:remediate --advisory=<CVE> --no-dry-run
elpanctl webapp:remediate --severity=high --kind=plugin --no-dry-run
Most of the advisory store is Composer packages and application cores,
which this command can't fix: cores are webapp:upgrade's work, packages
are the developer's, and the command says how many findings have no remedy.
Many plugin advisories name no fixed version, so an update can't clear
them; --include-unpatched runs it anyway, for a vendor that shipped a fix
the feed hasn't recorded. A succeeded job re-reads the tree and re-matches
it, and its message leads with the answer: all findings resolved, or how
many are still open.
Housekeeping of the jobs
Every upgrade, plugin update and other action on an installation is a job
row. Two sweeps keep them honest: housekeeping:webapplicationtask:reap
settles the jobs whose worker died before writing an outcome, and
housekeeping:webapplicationtask:purge deletes old settled jobs
(Housekeeping).
The reaper is the one to reach for during an incident. A stranded job leaves its installation permanently busy, so the account holder is refused every further action on that site. When the journal shows the worker was killed, don't wait for the hourly pass:
elpanctl housekeeping:webapplicationtask:reap --after-hours=1
One hour is the floor: settling a job that started a second ago, on a server whose workers are healthy, would let a second job start beside a live one on the same tree.