Every VM list load reconciles each compute with libvirt. Even when nothing
had changed, that took the compute's claims and a transaction with
SELECT FOR UPDATE (and the ownership advisory lock on PostgreSQL): about
11 queries a compute, 4 of them writes. On SQLite each load queued behind
every other write of the app.
The reconcile now first compares libvirt's VM list with the database,
without a claim or a transaction, and stops when they match and no
removed record has expired. Otherwise it runs as before, reading libvirt
again under its claims, so two reconciles still cannot commit in reverse
order. With two matching hosts a list load goes from 38 queries (8
writes) to 18 (none).
test_default_storage and test_default_storage_volumes opened a pool
named default, which a compute need not have (an openEuler host had
none, and the two tests failed there). They now use the wvc-test pool
the live suite creates and removes, and run only on a named test host.
The host pages lose their last inline scripts: the overview's charts
(host_graphs.js), the New pool, New network and New interface dialogs
(pool_dialog.js, network_dialog.js, interface_dialog.js), a network's
and a network filter's XML editors and dialogs (network_page.js,
nwfilter_page.js, nwfilters_page.js), the secrets page's labels
(secrets_page.js, its texts from the view as usage_texts) and a pool's
page with the ISO upload (storage_page.js). Their URLs and messages are
data attributes of the elements they belong to; the upload reads its
token and address from its form. No template has an inline script or
an inline event handler any more.
Without a migration address libvirt sends a migrating VM to the
destination's own host name, which the source host often cannot resolve:
a live migration to a host added without one failed ("Unable to resolve
address ..."), also when the other host had one. The destination's
address on the Computes page, which the panel reaches it at, is now the
default; the migration address stays for another network. A host the
panel reaches on its loopback keeps libvirt's default.
The host form said "Add Compute" or "Edit Compute" whatever the
connection, set its labels in a narrow column, and on an edit showed an
empty password box as if the password were gone; the migration address
and the gstfsd key sat among the basics with their long help, Cancel
went back in the browser history and Save was green.
Now the heading names the host and the connection type, with the line
the add menu uses for it; a Connection card holds the name, the address,
the login and the password in two columns (SASL login and password for
TCP and TLS, SSH user for SSH) and the description the hosts list
shows; the migration address and the gstfsd key fold under Advanced,
whose summary says what is set, open when one has a value or an error.
A stored password or key says so in its box. Cancel returns to the
hosts list and Save is the primary action.
gstfsd, the root service on each compute that rewrites a shut-off VM's
/etc/shadow and authorized_keys, did what any client on its port asked.
Now it needs GSTFSD_KEY (64 hexadecimal characters, in the unit's
/etc/default/webvirt-gstfsd) and does not start without it. Each
connection gets a new challenge; the panel signs its request for it
with HMAC-SHA256 and gstfsd checks the signature before any change, so
a request recorded on one connection is refused on another. gstfsd
signs its reply for the same challenge and the panel's nonce, and the
panel accepts only that reply.
Each compute has a gstfsd key, the host's GSTFSD_KEY: the compute forms
and the API take it and never show it, and an empty value on an edit
keeps it. The bootstrap script makes the key file once, readable by
root only; the README shows how to make it and where the panel takes
it. The traffic is still not encrypted: a remote panel's port stays
open to the panel's address only.
A migration needed every disk on storage both hosts share: it was refused
when the destination lacked a disk file, and a file of the same name
there was taken for the VM's own disk.
The Migrate tab now lists the VM's disk files for the chosen destination.
For each disk it offers the destination's pools that keep disk files
(directory, filesystem and NFS pools). A disk on storage both hosts share
can also stay where it is.
A directory counts as shared when both hosts have it as a storage pool
and a marker file created on the source exists on the destination. A
directory the probe cannot tell is never taken as shared.
libvirt copies the chosen disks during the migration: migrate3 with
NON_SHARED_DISK for those targets, and destination and persistent
definitions naming the new paths, where libvirt creates the files.
- A shut-off VM is started paused for the copy and shut off on the
destination; its CPUs never run. Its UEFI variables are copied first,
as the destination would start from the firmware's template. If the
migration fails, the VM is shut off on the source again.
- Refused before anything changes, each disk with its reason:
- a disk neither shared nor given a pool, also one only the next
start's definition has;
- CD-ROM images, read-only or encrypted disks, formats other than raw
and qcow2, and disks with a backing file;
- a pool that is not of disk files, or that has a file of that name,
which libvirt would reuse and overwrite;
- two disks copied to one file.
- The job claims each copy's name in its pool, and the source volume.
- A failed or cancelled migration deletes the copies libvirt made, and
the UEFI variables copy, unless a VM on the destination has them.
- The source files stay and are named in the log. With "Delete the
source files of copied disks" they go, except one another VM or volume
uses; a failed delete is logged and the migration stands.
The NVRAM sharing check uses the same probe: a shut-off move stops when
it cannot be told, and the source file stays.
The API's migrate call takes disk_pools ({target: pool}) and
delete_source.
- The host navigation takes one font size and spaces its dividers alike on
both sides in every theme: Zephyr doubled the space after each divider,
and with the Devices section its eight sections took two lines.
- Its labels are shorter: Storage, Filters.
- Measured in the 27 themes at 992, 1200 and 1440 px: one line in each,
with at least 73 px to spare (Lux had 20 px before, Zephyr was 13 px
short).
- A Devices section of each host lists its PCI devices by IOMMU group and
its USB devices: what each is (vendor, product, IDs, class, driver), what
the host itself has on it (its network interfaces, disks and graphics
card, anywhere below it) and the VMs whose definitions name it.
- A VM names a PCI device by its address, a USB device by its address or
by its model only; a VM that names a model only is shown on every device
of that model, which one it takes is known only when it starts.
- A host without IOMMU groups says that no PCI device can be given to a VM
and how to turn the IOMMU on.
- One listing of the host's node devices, one XML read each (a device gone
meanwhile is left out), and the VMs' definitions read once; nothing is
shown when they cannot be read, as a device would look free.
- Read only: giving devices to VMs comes in the next layers.
- The pool list is a card per pool, two to a row, as the host
Overview's Hardware and Software cards: its name with Active or
Stopped and Autostart badges in the header, then its type, its path
(an RBD pool's Ceph pool), its volume count and its space as the
Overview shows it (computes/pool_bar.html, without the name again);
a stopped pool's space is unavailable, not zero.
wvmStorages.get_storages_info gives the path, autostart and percent
used, which computes.views.host_pools computed for itself.
- A pool's page has a Space card (percent, used of size, of what: the
filesystem of its path, its volume group or the Ceph cluster, which
other pools may share; free) and a Pool card (state, autostart,
type, path) instead of a two-column list mixing facts and forms.
Start or Stop is its heading's button; autostart and deleting a
stopped pool are in its menu; the confirmations stay.
- The volume table drops the row number; a volume's capacity and what
it has allocated share a column hidden on a phone; its format is a
badge; its actions are one group, Clone and a menu with Delete,
named for the volume.
- "New pool", "New volume" and "Upload ISO" are labelled buttons
instead of bare green pluses; their dialogs' titles name them.
- The New pool dialog is one form (design reviewed with Codex): the
pool's type as radio cards, as the VM create page chooses its disk
source (their style moves to webvirtcloud.css), one name, and only
the chosen type's fields, the others hidden and disabled.
- A directory's or a share's mount path follows the name
(/var/lib/libvirt/<name>) until it is edited: no default path of
another pool any more.
- ISO images is a directory preset named iso; when the host has an
iso pool, a card links to it instead.
- Ceph offers the host's Ceph secrets by their names, not every
secret by UUID; without one it says so and links the Secrets page.
- LVM asks to agree that the device is erased, the server too.
- AddStgPool checks each type's fields, a name or a path another
pool has, an RBD secret that is not one of the host's Ceph secrets,
and the names "." and "..". A refused pool, or one libvirt refuses,
opens the dialog again with what was entered, the reasons beside
the fields and focus on the first; the audit row for a taken name
stays.
- Totals above the list: hosts connected of all, VMs running of all,
vCPUs given on the CPUs, memory given of the memory of the hosts the
panel reads, as cards; the connected count is red while a host is
not.
- A host's name links its overview; its state is a Connected or Not
connected badge (the dot goes), beside a badge of how the panel
reaches it (TCP, SSH, TLS, Socket); its address in monospace.
- A VMs column: the host's VMs, linked to its Instances, and how many
run.
- The vCPUs have a bar like the memory's: running VMs' and shut-off
VMs' vCPUs against the CPUs, amber over 2:1 and red over 4:1 as the
ratio is; the overview's vCPU card shares it.
- The three coloured buttons of each row are one menu: Overview, Edit
connection and Remove from panel (its popper fixed, as the table's
scrolling box would cut it off).
- The four green add buttons are one "Add host" menu that says what
each kind of connection is.
- The table is table-hover; a host the panel cannot reach has a
table-warning row.
libvirt measures a pool's size and free space only when the pool is
refreshed and keeps those figures until the next refresh. The pool
lists read them without a refresh, so they showed a pool's space as of
whenever libvirt last refreshed it: two hosts sharing one pool's
storage told different free space for it, each its own refresh's. On a
test host, a 1 GiB file left libvirt's free space unchanged while df
showed 1 GiB less, until the pool was refreshed.
get_storages_info refreshes each active pool before reading it; a
refresh libvirt refuses (while a volume is cloned in the pool) leaves
the last figures, as the pool pages already do. A refresh scans the
pool's volumes, so pools are read only where one host's are shown: a
host's overview (views.host_pools, its Storage card) and its Storages
page and API. The Computes list and the new-VM dialog's host lines,
which read every host at once, no longer show pools; Compute.capacity
is the vCPUs and memory given.
The memory given read "24.0 GB given of 32.0 GB": long, and the share
of the host's memory, what one looks for, had to be worked out. It is
now told as the vCPUs are: the share large ("75 %", amber over 90 %,
red over 100 %), then "24.0 GB / 32.0 GB", the bar and "12.0 GB
running". The Computes list's memory column shares it.
Every card of the overview has a Bootstrap card-header with an icon:
VMs, vCPUs, Memory, Storage, CPU in use and Memory in use (with the
current reading at its right), Hardware, Software and connection. The
last two were label and value lists under indented headings; they are
list-group-flush rows now, the label at the left, the value at the
right: counts in semibold, the hypervisor's and libvirt's versions as
badges, the guest architectures as light badges, the emulator and the
connection as code. The section headings, indented by the panel's
page-header margin unlike the cards, are gone: each card names itself.
The status line and notes are not paragraphs, which the panel indents.
#241 made the host sections pills: on a desktop they took two lines,
unlike the breadcrumb the pages had. computes/host_nav.html is the
breadcrumb again, one line at 1200 px and wider, still shared by the
eleven host pages: the section's own page is its bold item with
aria-current="page", an item page (a pool, a network) links its
section in bold with aria-current="true".
The capacity lines were kept on one line, made for the Computes
table: in the overview's narrower cards a long memory line ran out of
its card. They now wrap.
A host the panel could not reach made its overview the 500 page,
without the host's sections, and the Computes list gave such a host an
Overview button that led nowhere. The page now keeps the host's name,
sections, "Edit connection" and "Remove from panel" (the existing
confirmation), and says "Not connected" with the reason and a "Try
again". Details that cannot be read are said in an alert; the VMs,
capacity and load still show, as they are read on their own.
A reachable host's page leads with what it runs and what it gave:
- its VMs by state (running, shut off, paused, other), linked to
Instances;
- the vCPUs, memory and storage pools given, as on the Computes list,
which now shares those parts (computes/capacity_*.html) and their
preparation (views.shown_capacity) with it;
- CPU and memory in use, side by side, with the current reading in
words, a note on what the charts hold and when they were last
updated, or since when not; memory in GB like the rest of the page;
- Hardware and Software and connection: logical CPUs and clock from
virNodeGetInfo, but physical cores, sockets and NUMA nodes counted
from the CPUs of the capabilities' NUMA cells, as virNodeGetInfo
makes a topology up for hosts it cannot describe (asymmetric NUMA);
the hypervisor's own name and version, libvirt, emulator, the guest
architectures as text instead of menus of dead links. A host without
sysinfo shows no processor instead of failing.
Empty notes no longer show "None"; labels stack over their values on a
phone instead of running into them.
Pools that cannot be read no longer drop a host's vCPUs and memory from
its capacity: the storage part says so. get_capacity also counts the
VMs by state.
Eleven host pages (overview, instances, storages, networks, interfaces,
nwfilters, secrets and their item pages) each carried their own copy of
the links to the host's sections, as a breadcrumb: the sections are
siblings, not a path, and on a phone the separators were left at the
start of wrapped lines. The item pages marked no section at all.
computes/host_nav.html is the one nav, labelled "Host sections": pills
that wrap, the page's section marked active, with aria-current="page"
on the section's own page and "true" on an item of it.
- An API volume takes the name and size the web form takes. Its name went
into the volume XML unchecked, so a name could add elements such as a
backing store pointing at another VM's disk; the name is also escaped
there now. A size of 0 and metadata preallocation on a non-qcow2 volume
are refused or dropped, as on the web form.
- An API compute update with an empty or null password keeps the stored
one, as the web form does; it emptied it and cut the host off.
- The architectures endpoint answers 404 for an unknown compute instead of
500; an unrouted copy of it is removed.
- The host graphs and the machine/bus lists connect inside their libvirt
error handling, so a down host gives empty answers instead of a 500 page
the polling script drops.
- The Secrets page lists vTPM secrets instead of failing on them, and a
secret without usage data shows an empty cell instead of "None".
- A TCP/TLS host takes a SASL password longer than 14 characters and a
login up to the Linux limit of 32.
- The compute edit page says "Edit Compute"; a host without VMs no longer
closes an extra page element; the memory graph is labelled MiB.
A tombstone kept the VM's name and UUID but not its compute, so the page
could not say where the VM had been. It now stores the compute's name (as
text: the compute may be the one deleted) and the list shows it; records
made before this have no host. Deleting a record says so, and the admin
tables use scope="col" instead of the invalid span="col".
Closes#217
Under each host's CPU and memory load, the dialog shows one line:
Given to VMs: 24 vCPUs (1.5:1), 48.0 GB memory.
Pool default: 120.0 GB free.
The line is read when the dialog opens, from computes/<id>/capacity/,
as plain text, so the VM list itself reads no VM definitions. The CPU
bars and the lines share one queue, host by host and two reads at a
time, as each read holds a worker. A host that cannot be read says so.
Compute.capacity now also gives the vCPUs per CPU and its pools with
the default one first, so the Computes page and the dialog share them.
The capacity read lived in its own module with raw libvirt calls and a
thread pool with a deadline, beside helpers that already read the same
things. It now uses them:
- wvmHostDetails.get_capacity() sums the vCPUs and memory of all and of
the running VMs from get_host_instances(), against the host's info.
- get_storages_info() also gives each pool's free space.
- Compute.capacity joins the two, like cpu_count and ram_size, and is
None when the host is down or the read fails.
The page reads the hosts one after another, as it already connected to
each of them; a host that cannot be reached is given up after the
connect timeout. computes/capacity.py and its thread handling are gone.
The page looks and sorts as before.
Each host is a row of what it has committed, read live from the host (in
parallel, by a 10 s deadline):
- Host: a status dot, the name, its address and details; when it cannot
be reached, why.
- vCPUs: the overcommit ratio, large (warning above 2:1, red above
4:1), over "N vCPUs on M CPUs" and the running VMs' share.
- Memory: memory given to all VMs against the host's memory, as one bar:
running VMs dark, shut-off VMs light (they take their memory again
when they start), and a line where the host's memory ends; red when
the VMs were given more than the host has.
- Storage: the default pool (or the first) with its used share and free
space; the others, a host may have dozens, open on a click.
A host that cannot be reached, or does not answer in time, says so in
one line instead. vCPUs and Memory sort by ratio and share. A host whose
read hangs is not read again until that read ends, so a stuck host keeps
at most one thread. The table's hidden status texts stay inside it, so
the page does not widen on a phone.
Two computes decide the owners of the same UUID: the one the VM left
buries them in a tombstone (or a compute delete does), the one it went
to restores them. On PostgreSQL the two ran at once: the restore still
saw the old row of the uncommitted bury, skipped, and the VM ended up
without owners while the tombstone stayed. SQLite was never affected:
its IMMEDIATE transactions take the write lock at the start.
The reconcile and the compute delete now take one transaction-level
advisory lock (pg_advisory_xact_lock) when their database step starts,
so ownership decisions run one after another on PostgreSQL as they do
on SQLite. Reading libvirt stays outside the lock.
The Logs page shows who added, changed or deleted a compute, from web
or API, and every login, failed login, lockout and logout, with the
client address and a Result column (Done, Failed, Unknown for older
rows). A failed change is recorded with its reason and the error still
reaches the user; a refused one is recorded too.
- logs.audit.write saves each row in its own savepoint and never
raises, so a log write cannot break or undo the action it records.
The audit() context manager writes one row when the change ends.
- Logs gains success and ip; user, host and instance are wider and
date is indexed. The page orders by date then id.
- A compute edit names the changed fields; an edit that changes
nothing writes no row, and an empty password field is no change.
- A compute's VMs leaving the panel get a system row each: owners
kept on the Removed VMs page, moved to another compute, restored,
left without owners, or the removed record expired.
- Job rows carry the result. A job's log entry that cannot be
computed is logged and skipped instead of rolling back the finish
(test_service.py changes this on purpose).
- Logins are recorded by signal receivers; CustomLoginView is removed
and the plain LoginView is used. A wrong OTP code is a failed login,
counted by django-axes toward the lockout.
Panel records and host-side objects follow in the next layers.
Deleting a compute removed its VMs from the panel together with their
owners, while the VMs kept running on the host: added again, the host's
VMs came back with no owners. A compute's delete now treats each VM as
one that left its compute: its owners follow the same UUID on another
compute or wait in a tombstone for the retention period. One function,
forget(), does this for the refresh and for the delete; the API's
compute delete goes through the same model delete.
The delete holds the compute exclusive and locks its row before its
VMs', as the reconcile does: a reconcile, VM operation or job on the
compute makes it busy instead of losing owners to a concurrent restore
or deadlocking with it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
No page or script called it, and it answered with the disk device
types for both videos and buses. The create view keeps reading the
domain capabilities through wvmConnect.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The bus list of a disk on the VM create page came from the host's disk
device types: choosing "disk" for an added image offered cdrom, disk and
lun as its buses. The disk branch of get_disk_buses now returns the
host's disk bus types, the list the page starts each disk with.
131 test modules sat next to the code they test; instances alone had
51 of them beside views.py and utils.py. They move into a
tests/ package per app, and the JavaScript checks into
webvirtcloud/tests/js. Each non-empty tests.py becomes tests/test_views.py;
the empty generated stubs are removed.
Imports of the app's own modules become absolute, paths built from
__file__ go one level up, the live helper is instances.tests.livetest,
and coverage omits */tests/*. The suite discovers the same 960 tests as
before.
