Without a migration address libvirt sends a migrating VM to the
destination's own host name, which the source host often cannot resolve:
a live migration to a host added without one failed ("Unable to resolve
address ..."), also when the other host had one. The destination's
address on the Computes page, which the panel reaches it at, is now the
default; the migration address stays for another network. A host the
panel reaches on its loopback keeps libvirt's default.
gstfsd, the root service on each compute that rewrites a shut-off VM's
/etc/shadow and authorized_keys, did what any client on its port asked.
Now it needs GSTFSD_KEY (64 hexadecimal characters, in the unit's
/etc/default/webvirt-gstfsd) and does not start without it. Each
connection gets a new challenge; the panel signs its request for it
with HMAC-SHA256 and gstfsd checks the signature before any change, so
a request recorded on one connection is refused on another. gstfsd
signs its reply for the same challenge and the panel's nonce, and the
panel accepts only that reply.
Each compute has a gstfsd key, the host's GSTFSD_KEY: the compute forms
and the API take it and never show it, and an empty value on an edit
keeps it. The bootstrap script makes the key file once, readable by
root only; the README shows how to make it and where the panel takes
it. The traffic is still not encrypted: a remote panel's port stays
open to the panel's address only.
libvirt measures a pool's size and free space only when the pool is
refreshed and keeps those figures until the next refresh. The pool
lists read them without a refresh, so they showed a pool's space as of
whenever libvirt last refreshed it: two hosts sharing one pool's
storage told different free space for it, each its own refresh's. On a
test host, a 1 GiB file left libvirt's free space unchanged while df
showed 1 GiB less, until the pool was refreshed.
get_storages_info refreshes each active pool before reading it; a
refresh libvirt refuses (while a volume is cloned in the pool) leaves
the last figures, as the pool pages already do. A refresh scans the
pool's volumes, so pools are read only where one host's are shown: a
host's overview (views.host_pools, its Storage card) and its Storages
page and API. The Computes list and the new-VM dialog's host lines,
which read every host at once, no longer show pools; Compute.capacity
is the vCPUs and memory given.
A host the panel could not reach made its overview the 500 page,
without the host's sections, and the Computes list gave such a host an
Overview button that led nowhere. The page now keeps the host's name,
sections, "Edit connection" and "Remove from panel" (the existing
confirmation), and says "Not connected" with the reason and a "Try
again". Details that cannot be read are said in an alert; the VMs,
capacity and load still show, as they are read on their own.
A reachable host's page leads with what it runs and what it gave:
- its VMs by state (running, shut off, paused, other), linked to
Instances;
- the vCPUs, memory and storage pools given, as on the Computes list,
which now shares those parts (computes/capacity_*.html) and their
preparation (views.shown_capacity) with it;
- CPU and memory in use, side by side, with the current reading in
words, a note on what the charts hold and when they were last
updated, or since when not; memory in GB like the rest of the page;
- Hardware and Software and connection: logical CPUs and clock from
virNodeGetInfo, but physical cores, sockets and NUMA nodes counted
from the CPUs of the capabilities' NUMA cells, as virNodeGetInfo
makes a topology up for hosts it cannot describe (asymmetric NUMA);
the hypervisor's own name and version, libvirt, emulator, the guest
architectures as text instead of menus of dead links. A host without
sysinfo shows no processor instead of failing.
Empty notes no longer show "None"; labels stack over their values on a
phone instead of running into them.
Pools that cannot be read no longer drop a host's vCPUs and memory from
its capacity: the storage part says so. get_capacity also counts the
VMs by state.
- The host graphs and the machine/bus lists connect inside their libvirt
error handling, so a down host gives empty answers instead of a 500 page
the polling script drops.
- The Secrets page lists vTPM secrets instead of failing on them, and a
secret without usage data shows an empty cell instead of "None".
- A TCP/TLS host takes a SASL password longer than 14 characters and a
login up to the Linux limit of 32.
- The compute edit page says "Edit Compute"; a host without VMs no longer
closes an extra page element; the memory graph is labelled MiB.
Under each host's CPU and memory load, the dialog shows one line:
Given to VMs: 24 vCPUs (1.5:1), 48.0 GB memory.
Pool default: 120.0 GB free.
The line is read when the dialog opens, from computes/<id>/capacity/,
as plain text, so the VM list itself reads no VM definitions. The CPU
bars and the lines share one queue, host by host and two reads at a
time, as each read holds a worker. A host that cannot be read says so.
Compute.capacity now also gives the vCPUs per CPU and its pools with
the default one first, so the Computes page and the dialog share them.
The capacity read lived in its own module with raw libvirt calls and a
thread pool with a deadline, beside helpers that already read the same
things. It now uses them:
- wvmHostDetails.get_capacity() sums the vCPUs and memory of all and of
the running VMs from get_host_instances(), against the host's info.
- get_storages_info() also gives each pool's free space.
- Compute.capacity joins the two, like cpu_count and ram_size, and is
None when the host is down or the read fails.
The page reads the hosts one after another, as it already connected to
each of them; a host that cannot be reached is given up after the
connect timeout. computes/capacity.py and its thread handling are gone.
The page looks and sorts as before.
Two computes decide the owners of the same UUID: the one the VM left
buries them in a tombstone (or a compute delete does), the one it went
to restores them. On PostgreSQL the two ran at once: the restore still
saw the old row of the uncommitted bury, skipped, and the VM ended up
without owners while the tombstone stayed. SQLite was never affected:
its IMMEDIATE transactions take the write lock at the start.
The reconcile and the compute delete now take one transaction-level
advisory lock (pg_advisory_xact_lock) when their database step starts,
so ownership decisions run one after another on PostgreSQL as they do
on SQLite. Reading libvirt stays outside the lock.
Deleting a compute removed its VMs from the panel together with their
owners, while the VMs kept running on the host: added again, the host's
VMs came back with no owners. A compute's delete now treats each VM as
one that left its compute: its owners follow the same UUID on another
compute or wait in a tombstone for the retention period. One function,
forget(), does this for the refresh and for the delete; the API's
compute delete goes through the same model delete.
The delete holds the compute exclusive and locks its row before its
VMs', as the reconcile does: a reconcile, VM operation or job on the
compute makes it busy instead of losing owners to a concurrent restore
or deadlocking with it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The CPU bar showed the average since the host booted, read for every
compute on every load of the VM list. The dialog now reads each compute's
graph endpoint (load over one second) when it opens, two computes at a
time so the reads hold at most two workers, and ignores answers of an
earlier opening. A host that cannot be read shows a dash. The since-boot
reading had no other caller and is removed.
A compute's hostname and login go unquoted into the libvirt connection URI
(qemu+ssh://login@host/system, qemu+tcp://host/system, qemu+tls://...) and,
for an SSH compute, into the ssh command line of the DRBD status view. They
were barely validated: validate_hostname used re.match with a negated class,
so it caught only a leading special character, and login was never checked.
A hostname like "192.0.2.1/system?command=/path" injected a libvirt URI
parameter (verified: connecting ran the named command through the ssh
wrapper), and TLS/SSH parameters such as no_verify, pkipath, socket or
netcat could be added the same way; a login like "-oProxyCommand=..." became
an ssh option. The DRF ComputeSerializer applied neither check, and
validate_name tested a string literal instead of its argument, so it never
raised.
validate_hostname now accepts only an IPv4 address or a host name with
fullmatch (no scheme, port, path, query, whitespace or IPv6 literal; IPv6 is
left out until the URI building brackets it). validate_login accepts an empty
value (TCP falls back to the server's authentication, the socket transport
ignores the login) or a plain name with no leading "-". Both sit on the
Compute model fields, so the forms and the DRF serializer (which copies model
validators onto its fields) reject the values; a migration records the
metadata. validate_name checks its argument. drbd_status validates the login
and hostname before building login@host and ends ssh option parsing with
"--"; an existing or bypassed bad row is skipped rather than run.
Tests: hostname/login validators accept real values and reject URI and
option injection; the SSH form and the serializer (create, and an empty TCP
login) enforce them; drbd_status skips an injecting target. Checked the real
test computes still validate. The new tests fail on the previous code.
Migration is client-driven: libvirt sends the VM's memory to the
destination's own host name unless a migration URI is given, and the
app never gave one. Where the hosts cannot resolve each other's names
(or the names point at a management network) every migration failed
with "Unable to resolve address '<host name>' service '49152'".
A compute now has an optional migration address (an IP address or a
host name, no scheme or port), editable in the compute form and the
API. A VM migrating to that compute is sent to tcp://<address>
(bracketed for IPv6). Empty, the default for existing computes, keeps
the previous behavior. The compute's host name is not reused, because
for SSH computes it can be an ssh_config alias.
The migration log entry names the address when one is set.
libvirt has no timeout for opening a connection. Measured in the test
setup: an unreachable host blocked qemu+tcp for 135 s and qemu+ssh for
133 s; a daemon that accepts and never answers blocks forever. Gunicorn
runs sync workers and the instance list connects to every compute, so
one unreachable compute held a worker for minutes per page load.
- A connection is opened in a helper thread, at most one per
connection at a time. The request waits LIBVIRT_CONNECT_TIMEOUT
(default 5 s). An open nobody waits for any more keeps running; until
it ends, requests get its error at once, and if it succeeds later the
connection is used. A failed open is not repeated for
LIBVIRT_RETRY_AFTER (default 30 s). A stuck host holds one thread and
does not keep other hosts from connecting.
- qemu+ssh runs ssh through vrtManager/libvirt-ssh (the URI's command
parameter), which adds ConnectTimeout=10 and BatchMode=yes, so an
abandoned ssh process ends and no prompt can hang. The user's ssh
configuration (aliases, ProxyJump, keys) still applies.
- The close callback acts only for the connection it was registered
for: a late callback of a replaced connection dropped its successor.
The reason is reported as text. A connection that closed before it
was published is an error.
- Compute.status is a bool (it returned the exception object, so the
"host is down" check in refresh_instance_database never fired); the
message is Compute.connection_error.
Unit tests cover the timeout, retry delay, late success, single open,
a stuck host next to a healthy one and the close callback. Live tests
(vrtManager/test_live_connection.py), for TCP and SSH computes of the
test setup: an unreachable address (no ssh left after 10 s), an endpoint
that never answers, a slow open completing later, a host that goes
silent on an established connection (keepalive ends calls within 45 s,
the next request reconnects), an ssh_config alias and a ProxyJump host.
