A VM that changes its memory faster than the network carries it never
finishes a live migration without auto converge: measured with a guest
rewriting about 1.2 GB/s over an 8.5 Gbit/s link. The migration then
ran until gunicorn ended the request after 600 s. libvirt cancelled it
when the request went away and the VM stayed on the source, but the
user got an error page and nothing was logged.
The app now cancels a migration that has not finished after 540 s
(MIGRATION_TIME_LIMIT, below the 600 s of gunicorn.conf.py; a test
keeps it below). The failure is reported and logged like any other:
the migration was cancelled, the VM stays on the source host, and a
VM like that needs auto converge. A timer that fires as the migration
ends finishes before the compute locks are released, and an error
the cancel did not cause keeps its own message.
What migrate_instance did around the migration itself:
- It checked the destination with a 1 s TCP probe of the libvirt port,
which passes while libvirt is down or refuses the login. It now uses
the compute's connection status, as the rest of the app does.
- Two requests for the same VM waited for the compute locks one after
the other; the second then migrated with the VM's old location. The
VM's record is read again under the locks, and the request is
refused when the VM is no longer on the source.
- Restoring autostart on the destination came after the migration but
before the database move; when it failed, the VM ran on the
destination while the app recorded it on the source. Autostart is
now read from libvirt and a failure to restore it is only logged.
- A database failure after a successful migration was logged as
nothing; the log now says the VM migrated and recording it failed.
- When the destination's list had already made a record for the VM,
the merge kept the owners but dropped the template mark.
moveto read the VM state for each flag and dropped options that did
not fit it, so a request could run as something else than asked; the
log then named the mode from the form ("live" or "offline"), not the
one used. The API did not log at all and answered "instance migrate is
started" after the migration had finished.
- moveto reads the state once. A running VM migrates live, or without
live paused for the transfer (non-live); a paused VM stays paused; a
shut-off VM moves offline. Live for a shut-off VM, offline for a
running or paused one, and any other state are refused before
anything changes. unsafe, auto converge (live only) and compressed
apply to a running VM.
- migrate_instance logs the outcome for the VM page and the API: the
mode and options used, or the error. The API answers
"instance is migrated (<mode>)" and accepts requests that leave out
an optional option.
- Post copy is gone from the form, the view and the API: only the
flag was set, and without migrateStartPostCopy, which the app never
called, the migration never switched to post copy.
Migration is client-driven: libvirt sends the VM's memory to the
destination's own host name unless a migration URI is given, and the
app never gave one. Where the hosts cannot resolve each other's names
(or the names point at a management network) every migration failed
with "Unable to resolve address '<host name>' service '49152'".
A compute now has an optional migration address (an IP address or a
host name, no scheme or port), editable in the compute form and the
API. A VM migrating to that compute is sent to tcp://<address>
(bracketed for IPv6). Empty, the default for existing computes, keeps
the previous behavior. The compute's host name is not reused, because
for SSH computes it can be an ssh_config alias.
The migration log entry names the address when one is set.
Migration left the VM defined on the source host unless "Delete
original" was ticked (VIR_MIGRATE_UNDEFINE_SOURCE). With the shared
storage migration needs, that definition points at the disks the
destination VM is running on: the VM showed on both hosts and could be
started (or autostarted) a second time against the same disks; only a
configured lock manager stops that.
moveto now always passes VIR_MIGRATE_UNDEFINE_SOURCE with
VIR_MIGRATE_PERSIST_DEST. The option is gone from the form, the view,
the API (MigrateSerializer.xml_del) and the clone auto-migration. It
removes the definition, not the disk files.
check_user_quota added up the vCPUs, memory and disks of the user's
VMs, but skipped a VM whose host failed a TCP probe. A host that was
down therefore let users exceed their quota, and the probe always failed
for ssh computes named by an ssh_config alias or reached through
ProxyJump, so their VMs were never counted. A VM that could not be read
turned the page into the error page instead.
- Hosts are checked with Compute.status, the connection WebVirtCloud
uses, once per host. A VM that cannot be read (host down, VM gone)
makes the usage unknown: clone and resize up are refused with their
own message (fail closed).
- A request that adds nothing is not checked, so a user over quota can
still shrink a VM. Users without limits, and an exceeded instance
count (from the database), need no host either.
- Memory resize of a running VM changes only the current memory, but
the quota was charged with the submitted maximum. It is now charged
with no change, and resize_mem gets the current maximum, so a VM that
shuts down meanwhile cannot take an unchecked one.
- The VM page no longer computes the quota on every load (it read every
VM of the user on every host for a "quota reached" banner). The limit
is reported when a change is refused.
The test that expected a down host to pass the quota now expects it to
be refused. Checked against the test hosts: with a VM behind an
ssh_config alias the previous code counted 29 vCPUs, the new one 31;
with a VM on an unreachable host the previous code allowed +1 vCPU,
the new one refuses it after 5.1 s and allows -1 at once. The new
tests fail on the previous code, except the ones guarding unchanged
behaviour.
An optional dhcpd.conf next to the application supplies clone names and
their MAC addresses. get_dhcp_mac_address read it from BASE_DIR, but
guess_clone_name from a fixed /srv/webvirtcloud/dhcpd.conf, so on any
other install path the Guess button found no name while the MAC lookup
still worked. Both use utils.dhcpd_conf() (BASE_DIR) now.
Creating each option on a RHEL qemu-kvm host showed:
- CD-ROM: the create page offered every disk bus, but the empty CD-ROM
of a new VM works only on ide, sata and scsi. fdc and virtio failed
("Invalid bus type", "does not support ejectable media"), usb failed
with a device named 'Nonea' because vol_dev_type() had no prefix for
usb, and after that libvirt refuses an empty usb disk. Only ide, sata
and scsi are offered now, and another value is refused before any
storage is allocated. vol_dev_type() names usb disks sdX, which also
fixes data volumes on the usb bus.
- USB controller: any input device added a bare <controller
type='usb'/>, which libvirt turns into piix3-uhci (USB 1.1) on q35
instead of its default qemu-xhci. The bare controller is gone; libvirt
adds the machine's default for USB inputs.
- MACs: with more networks than MAC addresses, create_instance indexed
past the list and the request failed with a 500. NICs without a given
MAC now get one from libvirt. MAC addresses are validated (format, not
more than networks) by the page and the API before anything is
created; they went into the XML unchecked.
Unit tests cover the refused values, the MAC rule and the usb prefix.
Live tests create a VM with every offered CD-ROM bus and start it,
check the USB controller with a USB input device and create a VM with
two networks and one MAC.
Web views:
- 34 views returned to request.META["HTTP_REFERER"] + "#tab": without
a Referer they failed with a TypeError (500), and a Referer from
another site was followed (open redirect). They return through
_back(), which keeps a same-site Referer and otherwise opens the VM
page at the tab. get_safe_redirect uses the same check. Rejected disk
options also return to the VM's disks tab instead of the VM list.
- migrate_instance returned silently when the target was the current
compute or unreachable, so the web view logged "Instance is migrated"
and the API answered "instance migrate is started". It now raises;
the view reports the error and logs only a completed migration, and
an unknown compute id is a 404 instead of a 500. The automatic
migration after a clone skips a target that is the current compute.
- add_owner and del_owner failed with a 500 without their ids, add_owner
also when the user was already an owner, and del_owner deleted the
given ownership record even when it belonged to another VM. Ids are
validated, a record of another VM is a 404, and a repeated owner is
reported.
- Broad handlers that hid programming errors are narrowed: clone catches
libvirt, value and OS errors (a lock timeout included), ownership
lookups catch DoesNotExist. The DRBD status no longer needs its bare
except: the VM name is escaped in the regex, which was the only thing
that could fail there.
API:
- A libvirt error under /api/ is JSON with status 400, and a lock
timeout 409, instead of the HTML error page. This lives in the
existing ExceptionMiddleware, so existing settings need no change.
- Volume create/delete on an inactive pool answer 409, and invalid
volume data 400 with the serializer errors, instead of 200.
The new tests cover every view that returns with _back() without a
Referer and with a foreign one, the migrate failures on web and API,
the owner ids, the API error format and the volume statuses; nine of
the ten fail on the previous code.
Comments, docstrings and live test names cited item IDs of a roadmap
that is not part of the repository. They now say what is meant, or
nothing where the sentence stands without them. Test assertions and
behavior are unchanged.
Instance.proxy looked the domain up by its stored name. After a rename
or a name swap outside WebVirtCloud, an operation on one VM changed
another; after a VM was replaced under the same name, operations hit
the new VM.
wvmInstance takes the UUID and uses lookupByUUIDString, with no
fallback to the name: a missing UUID fails instead of picking another
domain. Every caller that has an Instance passes it: proxy, console,
quota, post-migration autostart and the VDI URL. Migration uses the
domain the proxy already resolved. getvvfile and the gstfsd requests
use the host's current name of that UUID.
Live tests cover a name swap and a VM replaced under the same name;
both fail on the previous code.
Clone refused VMs that cloned fine before: a disk without a volume got a
None disk name, and a source image with characters such as a space or "@"
produced a derived name outside the allowed pattern. Disks without a
volume (which clone_instance does not copy) are no longer validated, and
derived disk names are sanitized to the allowed charset. Names typed by a
superuser are still validated strictly.
The console settings form on the VM detail page renders the VNC password
(PasswordInput(render_value=True)) and was shown to every is_staff user.
A staff user with the global view_instances permission, or a staff owner
without is_vnc, could read the VNC password of every VM they could see.
Add instances.utils.can_manage_console (active superuser, or an active
owner with is_change and is_vnc), which matches what update_console
already enforced. The detail view only builds the form, and so only puts
the password into the page, when it returns True; the template and
update_console use the same rule.
Add webvirtcloud/test_authorization_matrix.py, which walks every named URL
and checks that a user without any VM/compute relation only reaches an
explicit allowlist, and that the global view_instances permission does not
open consoles. It found:
- /computes/<id>/... views without any check (compute_graph,
get_compute_*, get_dom_capabilities, storages.get_volumes): the whole
computes URL namespace is now wrapped with superuser_only, so a view
added there later cannot be exposed by a missing decorator.
- guess_mac_address (leaks dhcpd.conf host/MAC pairs), random_mac_address
and guess_clone_name: superuser-only, like the buttons that use them.
- check_instance: cross-tenant VM name lookup with no caller; removed.
- getvvfile and get_vdi_url: reachable with only view_instances, and the
.vv file contains the VNC password.
Console access now follows one rule, instances.utils.can_open_console:
an active superuser or an active owner of the VM. The global
view_instances permission is read-only. It is used by the console page,
novncd, getvvfile and get_vdi_url (getvvfile and the VDI tab are kept for
virt-viewer users). The console page answers 403 instead of 500 when
access is denied, with the same answer for a missing VM.
