Telemetry

Every Elpan server can report on itself to a collection system of your choice, by an open contract: metrics over OTLP/HTTP or Prometheus remote write, logs over OTLP, and a stable set of names and labels. The agent is the OpenTelemetry Collector, shipped as elpan-telemetry and installed on every server; it runs nothing until the server is enrolled.

What receives the data is outside the platform. Anything that accepts OTLP works: a plain Prometheus, VictoriaMetrics, Grafana's stack, SigNoz, a SaaS vendor. elpan-observability is one receiver, the one Elpan's authors run, written to be forked.

What the server emits

Every metric (and every log record) carries these resource attributes:

Attribute	Value
service.name	elpan
service.version	The elpan-panel package version.
host.name, host.id	The hostname and /etc/machine-id.
elpan.server.id	The id you give at enrolment. It survives a re-IP and a rename, and is the label to group by.
elpan.server.tier	Free text from enrolment, such as production or staging.
elpan.telemetry.version	The contract's version, 1.

The metric families:

• Host metrics under the OpenTelemetry semantic-convention names (system.cpu.*, system.memory.*, system.disk.*, system.filesystem.*, system.paging.*, system.uptime), every 15 s.
• One job per service the agent reads (node, postgres-system, postgres-userdb, mariadb-userdb, valkey, valkey-cache, rspamd-in, rspamd-out, pdns, postfix-in, postfix-out, dovecot, nginx, haproxy, httpd, panel-web, unbound, panel), each under its upstream names. A job whose service is not installed is left out entirely. up{job="node"} is the server's heartbeat.
• The panel's own state as elpan_*: pending migrations and deploy tasks, stale workers, messenger backlogs and failed messages, pending DNS publications, certificate expiry per host, backups (last success, failed and overdue assets per type), the mail queue, accounts and hosts, the account closest to its storage quota, outdated web applications, recent sign-in failures, and when each scheduled task last succeeded.
• OS updates and the pending kernel: elpan_os_updates_pending, elpan_os_security_updates_pending, elpan_kernel_reboot_required and their timestamps, from elpan-telemetry-dnf, which checks the repositories four times a day and recounts after every package transaction.

What is never emitted

Account holders' logs and journals, per-site access logs, per-home PHP logs, any message body, any password or token, and any label that names an account, a domain, a mailbox or a uid. This is a cardinality rule and a privacy rule in one.

Every server also answers GET /.well-known/elpan/ping?n=<nonce> on every address and name, unauthenticated and from the edge itself, for monitoring from outside; the answer echoes the nonce and reports the server's address, the client's address and the server's clock.

Enrol the server

Push is the default: one outbound HTTPS connection to your endpoint, with a credential per server, and nothing opened inbound. The credential is a basic-auth password; the user is the server id.

echo '<credential>' | elpanctl server:telemetry:enroll --endpoint https://<receiver> --server-id <id> --credential-stdin --tier <tier>

--credential-stdin keeps the secret out of the process list and the shell history; --credential <value> is the alternative. --exporter remotewrite sends Prometheus remote write to the endpoint instead of OTLP (then --endpoint is the full remote-write URL). Running the command again changes the enrolment.

The command writes /etc/elpan/telemetry/remote.env and features.env through a privhelper; the agent restarts by itself, and the firewall admits the endpoint's port as the only destination the agent may reach outside the server. OTLP pushes queue on disk for about a day of the server's output, so a receiver outage over a weekend loses nothing; remote write queues in memory only.

elpanctl server:telemetry:status
elpanctl server:telemetry:disable

status shows the enrolment and whether the agent is active, never the credential. disable removes the endpoint and the credential; the agent stops.

Pull

A pull-based collector (a plain Prometheus) scrapes the agent instead:

echo '<credential>' | elpanctl server:telemetry:enroll --mode pull --server-id <id> --credential-stdin --pull-from <addresses> [--pull-port 9464]

The agent serves everything it gathered as Prometheus text on https://<server>:<port>/metrics, and the firewall admits only the IPv4 addresses or networks in --pull-from. The certificate is self-signed, made on the first start in pull mode; --pull-tls-cert and --pull-tls-key give it one of yours. The collector authenticates with the server id and credential, pins the certificate, and must scrape with honor_labels: true.

Logs

Logs are built in and off by default. --logs turns them on, and --log-streams narrows them:

Stream	Source
journal	The system journal, the system slice only.
maillog	The mail log.
panel-access	The panel's access log.
web-error	The server-wide copies of the web servers' error logs, not the accounts' own.

A secret after password, token, authorization and the like, and a token, key or code parameter in a URL, becomes ***. The file and journal privileges are given only while logs are on, and taken back when they are turned off. Only the OTLP push variant carries logs.

Memory

The agent refuses new data above its memory limit, 400 MiB on a standard host and less on the small profiles (Sizing profiles); telemetry_memory_limit_mib in /etc/elpan/sizing.conf raises it when the agent's log says it is refusing data.

elpan-observability

elpan-observability is a reference implementation of a receiver, not part of the platform: one compose.yaml on a small VPS with Caddy, an OpenTelemetry Collector gateway, VictoriaMetrics, vmalert and Alertmanager, Grafana, and probes that check every server from outside (its public ports, its zone, the blocklists). It takes 2 vCPU and 4 GB of RAM for about 30 servers.

Enrolling a server there is three steps: add the server to its fleet.yaml (on the server, elpan-telemetry-fleet-entry prints the entry from the panel's own records), ./obsctl enroll <id> on the receiver, which stores only the credential's hash and prints the server:telemetry:enroll command to run on the server, and that command. ./obsctl enroll <id> --rotate replaces a credential and ./obsctl revoke <id> withdraws it.

A receiver of your own

A Prometheus-format consumer sees the host metrics under job="elpan" and must honor the datapoint's labels. Prometheus 3's OTLP endpoint overwrites job with service.name unless told not to; VictoriaMetrics keeps it.