Each 32 MiB chunk went out as 1024 SFTP writes, each waiting for its
answer: 19 MiB/s on a gigabit LAN, ~1.9 s per chunk with the SSH session.
The writes now go out without waiting and their answers are read as they
come and at the end: ~0.53 s per chunk, the same file (checked by sha256).
paramiko's own pipelined writes (set_pipelined) drop the errors in their
answers, so a full disk would leave a short ISO behind a successful
upload. Each write is registered to its file instead, the way paramiko
does for prefetched reads: every answer, in any order, reaches the file
and an error is raised as soon as one is read.
Each stats answer carries a signed sample of the counters, bound to the VM,
its compute and UUID. The next poll sends it back and the rates are the
averages since then; a sample from another run of the VM (domain ID),
another host CPU count or device list, or older than 30 s falls back to
one second as before. Windows are measured on the monotonic clock.
Now that a poll is cheap, the Stats tab polls every 5 s like the host
graph, the first time as soon as it opens (pollWhileVisible takes the
delay of the first request).
A counter the host reports as unsupported (-1) or that went back gives no
value instead of 0 or a jump; a NIC without a target keeps its chart, a
pool volume disk is measured, and a disk or NIC attached after the page
loaded no longer stops the other charts. Disk rates are MB/s (they were
MiB/s labelled Mb/s), network Mbit/s; the CPU tooltip names the window.
Each poll of a host's overview graph held a worker for one second between
two CPU samples. The answer now carries its sample, signed with the time
and the compute; the next poll sends it back and the usage is taken since
then, so a point is the average since the previous one (normally 5 s, at
most 15 s). Without a sample, or with one that is altered, older or from
another host, or when the host restarted, it is measured over one second
as before.
A sample is one reading; the busy share was mixed from separate readings,
counted guest time twice (it is part of user) and added half a percent.
A host that cannot be read leaves a gap in both graphs instead of a zero
CPU point and a broken memory axis.
Measured on a VM with two disks, a CD-ROM and a NIC: 70 libvirt calls per
page, now 43. Within the page's cached_reads block the VM's state, the
host's info and domain capabilities, and each disk path's volume are read
once; the running and the persistent disk lists share a path's volume.
The ISO list takes the active pools in one call instead of looking each up
by name; it still refreshes them, so an image copied in by hand shows up.
The opt-in DRBD status waits at most 10 s for the compute's ssh, then
leaves the status unknown instead of holding the page.
The ARP lookup passed 3 as the address source; libvirt's
VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_ARP is 2 and libvirt refused 3 as an
unknown source. The error was suppressed, so a VM without a guest agent
never showed an address.
Destroy deletes the NVRAM file only when asked (VIR_DOMAIN_UNDEFINE_NVRAM,
otherwise KEEP_NVRAM): a unit test checks the flags, live tests check the
file on the host both ways. A VM whose disk is an overlay with a snapshot
goes with its own image and the snapshot metadata, and the base image
stays, since other images may grow from it.
libvirt undefines the source domain after a migration but keeps its NVRAM
file. A VM created later on that host with the same name reuses the file
and starts with the old VM's UEFI variables (boot entries, enrolled keys):
checked on libvirt 11.10, where only VIR_DOMAIN_START_RESET_NVRAM
recreates it from the template.
After a successful migration the file is now deleted on the source, as a
volume of the pool on libvirt's NVRAM directory or of a transient one.
Only a file in /var/lib/libvirt/qemu/nvram, and only when the destination
cannot create a marker file the source just created there (an exclusive
create also fails on shared storage whose listing an NFS client caches).
A failed removal is logged with the migration, which stands and is
recorded.
The resize tab enabled disk growth only for a shut-off VM, although
resize_disk already grows a running domain's disk through blockResize.
The button is now enabled for running and paused VMs too. A live test
grows qcow2 and raw disks of a running and of a paused VM and checks the
size the guest sees, the volume size, the unchanged persistent definition
and the state.
A new bridge always declared its member as type='ethernet'. On a host
with the netcf backend that rewrote the member's configuration: a bond
lost its BONDING_OPTS and a vlan its VLAN=yes. The member is now taken
from the host's own definition of the device (a bond keeps its mode and
devices, a vlan its tag), without the start mode, MTU and addresses that
only a top-level interface has; a device the host does not define stays
ethernet.
When the serial console process exited on its own (a bad token, an
unreachable host, a killed consolecallback), the read task died on EIO
while fd and child_pid stayed set: the child was a zombie and every later
connection wrote to the dead pty until socketiod was restarted. A child
that failed before exec also returned into the server code and sent its
traceback to the client.
Now the read loop treats EOF/EIO as the end: it reaps the child, closes
the pty, tells the clients "[console closed]" and disconnects them, so
the next connection starts a new console. The child always leaves with
os._exit, also without a token; output is decoded incrementally, and the
unused imports are gone.
A VM that disappears from its compute keeps its owners in a tombstone
until INSTANCE_OWNERSHIP_RETENTION_DAYS pass, but nothing showed them.
Superusers now see them on the "Removed VMs" admin page (name, UUID,
owners, removed, expires) and can remove one, so a VM deleted on
purpose no longer hands its owners to a later VM with the same UUID.
echodebug compared _ECHO_DEBUG and BS_TRUE with -eq, but neither is set
anywhere, so each step printed "[: : integer expression expected". It
now prints debug lines only when _ECHO_DEBUG=1 is set.
Files that libguestfs creates have no SELinux label. On an enforcing
guest (RHEL, Rocky, Alma, Fedora) sshd may not read the unlabeled
/root/.ssh/authorized_keys, so a key added from the panel did not let
anyone log in ("avc: denied { read } ... unlabeled_t").
After writing a key, gstfsd now relabels /root/.ssh with the guest's own
file contexts (/etc/selinux/<SELINUXTYPE>/contexts/files/file_contexts),
as virt-customize does. Guests without an SELinux policy or with SELinux
disabled are left as they are. If the guest needs labels but the host's
libguestfs cannot set them, the request fails before anything is written.
A root password needs no labelling: /etc/shadow keeps its file and label.
The host commands hard-coded www-data, which does not exist on RHEL and
openSUSE (the panel runs as nginx there), so the first command failed
with "invalid user". The user is now set once in U and its home is read
with getent, since ~$U is not expanded. The host block also ends with
the same connection check as the Docker one.
Both blocks are labelled, and the step says to use only the one for the
installation type: a key set up on the host is not seen by the panel in
Docker.
AGENTS.md is a local agent file and is ignored.
The README gave host commands (sudo, ~www-data) and one sentence for
Docker, where there is no sudo and exec runs as root: a key made that way
lands in /root/.ssh, which the panel (www-data) does not use, and the
first connection fails with "Host key verification failed". Docker now
has its own commands, run as www-data, ending with a connection check.
Both variants accept a new host key once (accept-new) instead of
disabling host key checking.
Two tests released a hanging open and slept 0.1 s before expecting its
connection; on a slow CI runner the open had not published it yet and
the stored timeout error came back. They now wait until the connection
is published.
A clone copies the disks of the VM's persistent definition, but its disk
quota and the disk names chosen with automatic naming came from the
running VM's disks, which differ after a disk is attached or detached on
the running VM only.
The Destroy button was disabled and the confirmation page refused a
paused VM ("Instance is suspended, cannot destroy!"), while the destroy
request itself stops a paused VM like a running one. Both now offer it.
The form required a bridge member for an ethernet interface, which does
not use one. When the host's network devices could not be read, the error
was swallowed and every creation failed with "the device is not on this
host"; the page now shows the real error and does not try.
Without instances.snapshot_instances (snapshots) or is_vnc (console
settings) the request was accepted and silently did nothing, although
the documented answer for a missing right is 403. delete_external_snapshot
also asked libvirt for the VM state before checking the permission.
Redirects to #access (root password, SSH key) and other top-level panes
opened nothing, and some sub-tab ids did not exist. static/js/instance_tabs.js
opens the pane and the tab that holds it from the page structure, and
opens Settings on its first section, without leaving the current tab;
guest OS info is loaded whenever it is the section opened. The bottom bar linked to
host pages, which only administrators may open, and showed host names to
everyone. A stray </li> is removed; the page's inline scripts and the tab
logic are checked with Node.
Power cycle force-stopped and started a template, which power on refuses.
A non-staff owner with is_delete could delete a template that the
template rule otherwise leaves to superusers and staff owners;
get_instance(perm_type="delete") now refuses it, and the page offers
Destroy by the same rule.
A VM operation (or destroy and clone, which hold the whole compute) waits
for its compute's lock while a migration holds it. When the migration
moved the VM's record to the destination, the waiting operation went on
there holding only the source compute's lock, unordered with a
reconciliation, migration or disk attach on the destination. After taking
the locks it now checks the VM is still on the locked compute and
otherwise answers busy, so the retry locks the right one. Clone also
checks again under the lock whether the user may clone the VM, and
checks the quota and the clone's settings against the VM as it is then:
its template mark, size or the user's rights may have changed meanwhile.
gstfsd mounted every partition it listed: a root on LVM was never found,
a disk with two OS roots got the change twice and two concatenated
replies, and no root, an error or a request longer than 1024 bytes ended
the connection without a reply. It now lets libguestfs inspect the guest,
mounts its one OS the way the guest does (a broken filesystem off the
changed path is left out), and sends exactly one reply, keeping the first
error. The root entry is found on any line of /etc/shadow and must be
unique; a key needs no shadow file. A request is at most 64 KiB. Each
change runs in a worker process with a directly started appliance; both
are killed when the request is not done 100 s after gstfsd accepted it
(GSTFSD_DEADLINE may only lower that), before the panel's 120 s timeout
releases the VM, or when the panel's connection ends, and the reply waits
until they are gone, so nothing touches the disk once the panel has given
up. A worker that dies says why. The panel
connects over IPv4 or IPv6, sends the whole request and reads the reply
until gstfsd closes.
The documented rule held only for snapshots: an owner with is_change could
still resize a template, set its root password and SSH keys, change its
options and console settings. get_instance(perm_type="change") now
refuses a template to owners who are not staff, which covers every change
view and the API; the per-view snapshot checks it replaces are removed,
and the page offers those controls by the same rule. Viewing, the console,
power (a no-op on a template) and cloning are unchanged.
The page showed controls the backend then refused: power to global
viewers, the console, virt-viewer and VDI controls and the root password
and SSH key forms to viewers, resize to staff, per-vCPU hotplug, video
model and guest agent to non-superusers, clone and options by the clone
permission alone, snapshots without change permission. The page now asks
the views' own rules (owner, is_change, clone and snapshot permissions,
the template rules). OS info, opened from the header, is there for every
viewer; the XML editor and tab restoring after a redirect no longer throw
when a pane is absent; Settings opens on the first section the user has.
The authorization matrix also covers a template VM, the template clone
rule and the template snapshot rule.
The IPv4 address and gateway were required even for DHCP or no IPv4, so
such an interface could not be created; malformed addresses passed and a
missing prefix crashed the XML builder. They are validated like the IPv6
fields now (netmask notation accepted, IPv6 scope ids rejected), and a
static address without a gateway gets no empty route. Interface names
follow the Linux rule (up to 15 letters, digits, . _ -). A bridge member,
or the device an ethernet interface configures, must be a device of the
host (the page no longer falls back to made-up names), a bridge cannot
contain itself, and needs STP and a non-negative delay.
The README grew as a list of per-change notes. Reorder it along the
operator's path (install, first login, compute node, configuration,
update, usage, development) and correct what no longer matched the code:
supported distros and Python, the openSUSE nginx step that installed a
full nginx.conf as a vhost, the disabled serial console listed as a
feature, missing ALLOWED_HOSTS and file-mode steps, services not
restarted, and an update path that copied settings blocks by hand
instead of rebuilding settings.py from the template with the panel
stopped.
The first appsettings migrations insert the default rows with fixed ids,
which a PostgreSQL id sequence does not see; the login lockout migration,
the first to insert without an id, then collided with id 1. It now moves
the sequence past the existing rows first (a no-op on sqlite), which also
covers a database upgraded from any earlier migration. A test compute
password exceeded its 14-character column, which only PostgreSQL enforces.
Reviewing the error handlers outside instances/ turned up three that masked or
swallowed the failure they caught:
- appsettings/settings.get_settings() runs on every request (middleware) and
loads app_settings from the database. Its bare `except: pass` left `entries`
unbound on a database error, so the next line raised UnboundLocalError
instead; it also caught KeyboardInterrupt/SystemExit. It now catches
DatabaseError and returns, keeping the already-loaded values.
- vrtManager.network.refresh_dhcp_leases raised a formatted string
(`raise f"..."`), which Python rejects with TypeError, hiding the real
DHCPLeases failure. It now raises util.OperationError, which the API/HTML
error handling already renders as a libvirt error, and clears self.leases.
- storages.views.get_volumes caught libvirtError and answered 200 with an empty
body, so the caller could not tell a failed pool read from an empty pool. It
now returns 400 with the error.
The other broad handlers reviewed (appsettings/nwfilters/virtsecrets views,
storages/upload cleanup and recovery paths, vrtManager display getters) report
the error or are best-effort and were left as they are.
Tests: get_settings() survives a database error during the query and during
iteration without raising and still loads a real value; refresh_dhcp_leases
raises OperationError (keeping the cause) not TypeError; get_volumes returns 400
on a listing failure and the sorted volumes on success. The tests fail on the
previous code.
Several instance mutation views crashed with a bare HTTP 500 when a form field
the UI always sends was absent or malformed (a direct or crafted request);
ExceptionMiddleware turns only libvirt and timeout errors into 4xx, so these
reached a 500. Fixed:
- add_network parsed a None network source ("'NoneType' has no attribute
'split'"); it now requires a selected network.
- set_qos read an unbound local 'mac' when no field name contained "mac", and
passed an invalid direction or non-numeric rate to the proxy (ValueError); it
now requires a MAC, an inbound/outbound direction and whole-number rates.
- unset_qos passed an empty direction into the proxy's XPath (XPathEvalError);
it now requires a MAC and a valid direction.
- resizevm_cpu and resize_memory ran int() over an empty or non-numeric value;
they now validate with int() (not str.isdigit(), which accepts "²" that
int() rejects).
- add_public_key passed a missing/non-numeric sshkeyid into an integer-id
lookup (ValueError); it now requires a numeric id.
Each reports the problem and returns to the VM page without calling libvirt or
logging, as change_network already does.
Tests: each view with a field missing, non-numeric or a unicode digit returns
302 with an error message, writes no log, and does not attempt the mutation.
The tests fail on the previous code.
OTP (OTP_ENABLED) was enforced only by the HTML login form, but two other
doors let an enrolled user authenticate with a password and no token, so a
leaked password defeated the second factor. Both were reproduced with a
confirmed TOTP device: /api-auth/login/ (DRF's browsable-API login) created a
site-wide session, and DRF's default BasicAuthentication let every /api/v1/
endpoint accept HTTP Basic credentials.
The api-auth/ include is removed, so the only interactive login is the
OTP-aware form at /accounts/login/; the browsable API still works with the
session that form sets. REST_FRAMEWORK now pins DEFAULT_AUTHENTICATION_CLASSES
to SessionAuthentication only, so the API requires a session and no longer
accepts a password over HTTP Basic. A session therefore implies the OTP factor
was met. Password-based programmatic API access is intentionally dropped.
The authorization matrix drops its rest_framework:login/logout allowlist
entries (the routes are gone). New tests assert the browsable login route is
removed, HTTP Basic is rejected on the API, Basic is not a default auth class,
and a session still reaches the API. Verified live with OTP_ENABLED that
/api-auth/login/ is 404 and an API Basic request is 403.
With OTP_ENABLED the login used django-otp's OTPAuthenticationForm. Under the
pinned django-otp 1.7.3 that form could not log anyone in through the shipped
page: the page posts only a 6-digit otp_token, but the form now needs the
device named explicitly, so an enrolled user got "Please select a device.",
and a user who never enrolled got "Please enter your OTP token." with no
device to produce one. Verified in an isolated test database.
SingleDeviceOTPForm makes OTP per user: a user with a confirmed TOTP device
must enter a valid code; a user without one logs in with the password alone.
The form chooses the device itself (the user's oldest confirmed TOTP device,
locked with select_for_update inside clean_otp's transaction so replay and
throttling counters cannot race) and ignores any posted otp_device, which
django-otp does not check for confirmed status or type. The login page drops
the HTML "required" on the token field (the field is optional for an
unenrolled user) and says so.
The "Lost OTP?" page (email_otp) created a confirmed device when none existed.
With per-user OTP that let anyone who knows an address turn a password-only
account into an OTP-required one and, if the mail never arrived, lock it out.
It now resends the QR only to a user who already enrolled and never creates a
device (get_existing_totp_device); admin enrollment (admin_email_otp) still
creates one. Email QR distribution and confirmed-on-create enrollment are
unchanged by user decision.
Tests: per-user login (no device -> password only; device -> valid token
passes, wrong/empty token fails), a posted or another user's device is not
used, a token cannot be replayed, and the real login page over HTTP with
OTP_ENABLED; email_otp sends nothing and creates no device for an unenrolled
user. The new tests fail on the previous code.
A compute's hostname and login go unquoted into the libvirt connection URI
(qemu+ssh://login@host/system, qemu+tcp://host/system, qemu+tls://...) and,
for an SSH compute, into the ssh command line of the DRBD status view. They
were barely validated: validate_hostname used re.match with a negated class,
so it caught only a leading special character, and login was never checked.
A hostname like "192.0.2.1/system?command=/path" injected a libvirt URI
parameter (verified: connecting ran the named command through the ssh
wrapper), and TLS/SSH parameters such as no_verify, pkipath, socket or
netcat could be added the same way; a login like "-oProxyCommand=..." became
an ssh option. The DRF ComputeSerializer applied neither check, and
validate_name tested a string literal instead of its argument, so it never
raised.
validate_hostname now accepts only an IPv4 address or a host name with
fullmatch (no scheme, port, path, query, whitespace or IPv6 literal; IPv6 is
left out until the URI building brackets it). validate_login accepts an empty
value (TCP falls back to the server's authentication, the socket transport
ignores the login) or a plain name with no leading "-". Both sit on the
Compute model fields, so the forms and the DRF serializer (which copies model
validators onto its fields) reject the values; a migration records the
metadata. validate_name checks its argument. drbd_status validates the login
and hostname before building login@host and ends ssh option parsing with
"--"; an existing or bypassed bad row is skipped rather than run.
Tests: hostname/login validators accept real values and reject URI and
option injection; the SSH form and the serializer (create, and an empty TCP
login) enforce them; drbd_status skips an injecting target. Checked the real
test computes still validate. The new tests fail on the previous code.
