python-ldap waits for a server without end, to connect and for each
answer: a directory that stopped answering held every login's thread.
The settings template's LDAP block sets 5 seconds to connect and 10 for
each answer.
EMAIL_TIMEOUT was not set, and Django's SMTP connection then waits for
a server without end: one that accepts the connection and stops
answering held the request's thread for good, also on the login page's
Email OTP form, which anyone may post. The settings template sets 10
seconds. An administrator whose mail fails is told why on the account
page instead of getting an error page.
gunicorn writes a line of its access log for every request to the
container's output, and Docker's json-file log driver keeps all of it
by default: an open VM list alone adds a line every 5 seconds until the
disk is full. The compose file now keeps 5 files of 10 MB, as the
panel's own log.
Nothing removed the Logs page's rows, which every action and every
failed login adds, or the database sessions past their expiry. The job
worker's housekeeping now deletes the rows older than Log Retention
Days, a new setting under Lists and pages (1-3650, 365 by default), and
the expired sessions.
django-axes wrote an AccessLog row at every successful login and nothing
removed them; the Logs page already records each login. Its access log
is now off; failed logins are still counted for the lock.
The clone form's checks read the source's NICs from the live and the
persistent definition, and each NIC read the running VM's addresses
again since b2b2386d: a guest agent that is connected but hung held the
request for its timeout once a NIC and definition. The checks now run
in one cached_reads block, as the VM page does.
other_hosts counted the request on its own host again only when the
block ended normally: a quota check that raised left the request
counted on the other host while it went on with its own. The count is
now restored however the block ends.
upload_volume opens the libvirt stream before it creates the volume, but
created the volume outside the block that aborts the stream: a pool that
refused the volume (full, a name taken) left an unfinished stream to be
freed. The creation is now inside that block.
When the worker stopped after it defined a clone and before it resized
and started it, recovery recorded the clone at the source's size and
shut off, and the job read as a plain success. Recovery now finishes a
defined clone that is shut off as the run does; one that runs went
through the last step, starting, and is left as it is.
clone_data checked the clone's vCPUs and memory with str.isdigit(), which
takes characters such as a superscript two that int() refuses: the user
saw Python's "invalid literal for int()" instead of the refusal. It now
checks with str.isdecimal(), whose characters int() reads.
#394 read each host's states in one call and every domain's XML once,
for the title, memory and vCPUs. A user's list shows only their own VMs,
but it read the XML of every VM on those hosts: a user with two VMs on a
host of 300 made about 301 libvirt calls a list load, where the old
per-VM reads made 6.
get_domain_rows takes the UUIDs the list shows and reads only their XML;
the states still come in one call. The capacity reads every domain, as
before.
607c8381 sent each memory and disk resize back to its own sub-tab, but
the disk view's last redirect, the one after a resize was tried, still
went to #resize, which opens on the CPU sub-tab. Only the live tests
reach it, with a disk to grow: the full live suite failed there on every
connection type.
The live vCPU test built its post from the page's form, and the page now
disables the CPU fields while the VM runs, so the post lacked cur_vcpu
and was refused as not a number before the running VM was. It now posts
both fields, as the direct post it checks.
The page where a user changes their own password and the one where an
administrator sets a user's password shared an old card: a horizontal
form, a blue Cancel and a green Change. The administrator's had Django's
"Password-based authentication: Enabled/Disabled" with a paragraph about
other backends, and its password fields stayed while Disabled was chosen.
Both pages now have the user and group forms' layout, each its own
template. The administrator chooses how the user signs in, With a
password or Without a password (such as LDAP), and the password fields
and "must change it" show only with a password; Cancel goes back to the
user. A user's own page says that their other sessions end, or, when
they must change it, that they choose one before going on, without a
Cancel. A refused password is said at its field only, without the
general "Wrong Data Provided". The section styles of these forms move
into webvirtcloud.css.
The group form put every user as a checkbox in one unsorted list, the
members among them, with no way to find one: past a few dozen users a
wall of names. Its name field ran the width of the page.
It now has the user form's sections (Group, Permissions, Members), each
with what it is for beside its fields. Members come first, then everyone
else, each by username with their full name; a search box filters them
by username, name or e-mail (table_filter.js). A form's search, marked
data-forget, starts empty: the list pages remember theirs per page title,
which here is the same for every group. Hidden users stay in the form,
so saving while filtered keeps them. The button says Create group or
Save changes.
The create and edit pages of a user had one form for both, ordered for
editing: permissions first, the name and password folded below them and
the quota folded and closed at the end; a new user's name came last.
Both pages now go Account (name, password, may sign in), Role, Quota,
then Groups and permissions, each section with what it is for beside its
fields. Template editor and Superuser become one Role choice with User:
choosing Superuser hides the quota and permissions, which a superuser
does not need (no quota check applies, every permission is theirs); it
keeps the is_staff it had. The quota is always open: each limit takes a
number from 0, or No limit, which is kept as -1 as before; memory is
entered in GiB and kept in MiB, shown exactly so saving a user again
changes nothing. The button says Create user or Save changes.
The VM's keys page (instances:sshkeys) and the cloud-init user data
collected the owners' keys with one query an owner. Both now read them
in one query joined over the VM's owners, ordered owner by owner as
before.
Each row of the Users page asked has_perm whether its user may clone:
two queries a user (the user's permissions and the groups'). The list's
query now annotates it with one Exists over the clone permission, given
to the user or to a group of the user; a blocked user is shown without
it, as has_perm did. The page's query count no longer grows with the
users.
A host's capacity (Computes page, new VM dialog) came from
get_host_instances, which listed the VMs by ID and by name and then
looked each up, read its XML and its info: four calls a VM, for every
host on the page. It now uses the VM lists' get_domain_rows: the states
in one call and each VM's XML once. get_host_instances had no other
caller and is gone.
Checked on QEMU against the per-VM info of every domain (running, paused,
shut off): the same vCPUs and memory. A VM gone while the host is read
is left out instead of making the host's capacity unreadable.
A VM list row showed the VM's state, memory, vCPUs and title, read
through the VM's own connection object: a lookup by UUID, virDomainGetInfo
and the live XML, one after the other for every VM on the page.
The lists (Instances, grouped or not, and a compute's VMs) now read each
host once: the states in one getAllDomainStats call (the state group
only, as the lists' poll; no other group, so a VM busy in the QEMU
monitor holds nothing up) and the memory, vCPUs and title from each
VM's live XML, which the title already read. Checked on QEMU against
virDomainGetInfo for running, paused and shut-off VMs: the same values.
A VM gone from its host, or a host that cannot be read or fails its
listing, shows its VMs unknown as before. Only the grouped list, a
superuser's, prefetches every host's VMs and owners.
Every VM list load reconciles each compute with libvirt. Even when nothing
had changed, that took the compute's claims and a transaction with
SELECT FOR UPDATE (and the ownership advisory lock on PostgreSQL): about
11 queries a compute, 4 of them writes. On SQLite each load queued behind
every other write of the app.
The reconcile now first compares libvirt's VM list with the database,
without a claim or a transaction, and stops when they match and no
removed record has expired. Otherwise it runs as before, reading libvirt
again under its claims, so two reconciles still cannot commit in reverse
order. With two matching hosts a list load goes from 38 queries (8
writes) to 18 (none).
Nothing loads it: gunicorn (systemd, runit) and Django use
webvirtcloud.wsgi. It exec'd the shell script venv/bin/activate as
Python, which cannot run, and left that file open.
libvirt requires a stream that did not finish to be aborted before it is
freed. The NVRAM download of a shut-off move and upload_volume (NVRAM
upload, cloud-init seed) freed theirs after a failed send, receive or
finish without aborting, on the shared host connection. They abort now;
upload_volume opens its stream before it makes the volume, so the
cleanup has both.
The New password field sits under the "Set a new one" choice and shows
only while it is chosen, so its help no longer has to say so. The Console
heading above the Console type label goes, and the save button has the
size of the other tabs' main buttons.
The VM's XML editor takes most of the window's height and is framed like
a form field. The review dialog shows removed and added lines on a red and
green tint, and each part of the diff starts with its line number instead
of the unified diff's @@ header.
Unsafe migration and deleting the source files of copied disks sit below
a line, apart from how the memory moves, with their help in red. The disk
lines read "vda 10.0 GB" instead of a dot-joined fragment.
Random and Guess join the MAC field as one input group, the Clone tab's
secondary buttons are outlined like the other tabs', and the NIC and disk
lines read as plain text ("default pool, qcow2") instead of dot-joined
fragments.
Edit, Move, Detach, Unmount and a CD-ROM drive's Detach are outlined
secondary buttons and Delete an outlined danger one, as on the Network
and Snapshots tabs; Mount and Add disk stay the filled primary actions.
A NIC's row put its name and host device in one column, "Source",
"IPv4 unknown" and the link switch in another, and its model, MAC and
filter behind Device details; every NIC also had a "QoS limits · none"
fold and solid Edit, Add QoS and Delete buttons.
Now a row reads eth0, a cable, then the source: the cable is solid green
while the link is up and dashed grey while it is down. Under it are the
IPv4 addresses (or "No IPv4 address known") beside the link switch, and
a line of model, MAC, filter and whether bandwidth is limited. Device
details keep the type, the host device and IPv6; QoS limits show only
for a NIC that has one. The buttons are outlined. Dialogs, form names
and posts are unchanged.
The tab opened on "Snapshots · Active" above the form, and listed the
snapshots below it as rows of "name · description · external, current
· date" with a solid red Delete on each. Which snapshot the VM goes on
from was the word "current" among the others.
Now a line runs down the tab, newest first. On top is Now, the VM in
its state's color, with the form that takes a snapshot there. Below it
each snapshot is a point: its name and date, its description on a line
of its own, its kind under them. The one the VM goes on from is filled
and says so; the others are rings. Revert and Delete are small outlined
buttons. Forms, confirmations and the rules that disable them are
unchanged.
The tab listed seven settings at one weight, each a two-column card with
a heading and a paragraph beside its control. The controls disagreed:
three switches posted on change, while the video model, the crash action
and the guest agent (an Enabled/Disabled select) each had their own
Save, and "From the next start" was repeated under three of them.
Now it has three parts. VM details: title and description with their
Save, and the template switch below them. Devices the guest sees: video,
virtio RNG, TPM 2.0 and the guest agent as rows of name, control and a
one-line note, under one line saying when changes apply; the video model
posts on change and the guest agent is a switch, with its connection
shown beside it as the Power tab's dot. When the guest crashes:
Restart / Stop as two buttons that post on click. Form names, posts and
who sees what are unchanged.
The Turkish catalog now calls the guest agent "konuk ajanı" everywhere.
Each sub-tab (CPU, Memory, Disks) opened on a two-column card that put a
heading and a sentence of current values beside the forms, and the CPU
tab added two more rows: Live vCPUs as a row of "0 · Fixed / 1 · On"
buttons and vCPU hot plug as a select with its own Save.
Now each sub-tab opens on the VM's size. CPU: one cell per vCPU up to
the maximum, filled when on, with "2 on, up to 4" and the host's logical
CPUs; on a running VM with hot-pluggable vCPUs the cells are the
buttons that turn them on and off. Memory: a bar of the current memory
within the maximum, with the host's memory. Disks: a row per disk with
its format, size now and a new size. The forms below keep their names
and posts; the CPU fields are disabled while the VM runs, and hot plug
is a switch that posts on change. A memory or disk resize returns to
its own sub-tab.
The Resize tab offered vCPU counts only up to the host's logical CPUs.
A VM whose maximum was higher (4 on a 2-CPU host, say) found no option
for it, so the Maximum list showed 1, and a resize of the shut-off VM
lowered its maximum to that without a word.
get_max_cpus() now takes the VM's own maximum as a floor, and the
Instance's vcpu_range passes it. The host's CPU count shown on the page
and used by the clone form is read from the host alone, not from that
range.
The tab put the console, a remote viewer address and two root forms at
one weight under "Access · Active · VNC", named buttons after their
implementation (VDI, Get console.vv), and repeated why the root forms
were disabled under each of them.
Now it has two sections. Screen: Open console with what it does, and
the remote viewer on a row of its own (address, Open, Download .vv
file). Log in as root: one line says how a password or key reaches the
guest now, written to the disk of a shut-off VM or sent through a
running VM's guest agent, or what to do when neither works; the
password and key rows follow with aligned labels. Ids, field names and
posts are unchanged.
A checkbox marked a device as booting and also chose what Up and Down
moved, which moved every checked device at once: to move one device the
others had to be unchecked and checked again. The buttons sat apart
from the rows, and a VM booting by device type got a note that did not
say what to do.
Now the devices the VM tries are a numbered chain, each with its own
up, down and Remove, and the others are listed below it with Add; the
posted order is unchanged. Devices read by kind first (CD-ROM sda,
Disk vda) with their file or network below; a VM booting by device
type says so and how to choose an order. Autostart and the boot menu
sit below, lighter. Focus stays on the device that moved, and on a
narrow screen a device's buttons wrap below its name.
The Turkish catalog says "Ön yükleme" and "sanal makine" throughout,
also in the Power tab texts.
The tab put Power Off and Suspend, then Power Cycle and Force Off, in
joined button groups with a paragraph under each, so which sentence
told which button was left to the reader, and the two power cuts
looked like one control as large as the rest.
Now the tab shows the three states a VM moves between, Active,
Suspended and Off, the current one marked, and under each the action
that takes the VM there with one line on what it does. The power cuts
are small outlined buttons on a line of their own with one warning;
their confirmations stay. A state outside the three (shutting down,
crashed) is named above them. The test of the tab now finds the
outlined power-cut buttons, which its class match missed.
A quota refusal put the English name of the quota (cpu, memory, disk,
instance) into a translated sentence, so a Turkish page read "memory
kotası doldu". check_user_quota now returns which quota and its numbers
(QuotaExceeded), each quota has a whole sentence, and the refusals put
that sentence before what is refused; QUOTA_DEBUG adds the numbers as
before. The clone job stores the refusal as a message, so the Jobs page
shows it in its reader's language.
The connection errors of vrtManager (did not answer within, still
waited for, closed, failed with libvirt's text) were English f-strings;
they are messages now, each close reason a whole one. A connection
that died without a close callback raised an error reading "None"; it
now names the host. Logs keep the English text.
test_default_storage and test_default_storage_volumes opened a pool
named default, which a compute need not have (an openEuler host had
none, and the two tests failed there). They now use the wvc-test pool
the live suite creates and removes, and run only on a named test host.
On an AlmaLinux 10 host with SELinux enforcing, the installer finished
with exit 0 but the panel's four services never started: chcon labelled
the virtualenv httpd_sys_rw_content_t, which systemd may not execute
(status 203/EXEC, Permission denied), and the login page answered 502.
The README's manual steps did the same with httpd_sys_content_t. chcon
labels also went on a relabel, after which nginx got 403 for the
static files.
Now the installer writes file context rules, httpd_sys_content_t for
the panel and bin_t for venv/bin, restores the labels from them, and
does so whenever SELinux is enabled, not only when it enforces. The
manual steps add the bin_t rule and policycoreutils-python-utils (for
semanage).
The README said external snapshots need libvirt 9.9 or newer. The live
suite against an openEuler 24.03 LTS compute (libvirt 9.10) showed that
reverting a snapshot taken without memory of a running or paused VM is
refused there ("Invalid target domain state 'disk-snapshot'"); libvirt
11.10 on the test computes reverts it.
Opening /computes/<id>/storage/<pool>/ for a pool the host does not
have (or a host that does not answer) raised libvirt's error out of
the view: a 500. Now the pools list shows the reason, as the network
page does.
The compute bootstrap found openEuler but had no install function for
it, so it stopped. Now it installs qemu, libvirt, python3-libguestfs
and cyrus-sasl (openEuler's names; it has no EPEL), applies the RHEL
family's qemu.conf, SASL and gstfsd steps, and runs the monolithic
libvirtd with its TCP socket: openEuler 24.03 ships modular daemons
only for qemu and the proxy.
Tested in an openEuler 24.03 container with /dev/kvm: the full live
suite over TCP (SASL) ran 2145 tests. What failed is libvirt 9.10's
(reverting a running VM's external snapshot, live clone needs 11.10),
the test host's (no pool named default), and its SeaBIOS, which
prints no banner on the serial port.
The removed openEuler branch only took 20.03, whose Python 3.7 cannot
run the panel. 24.03 LTS has Python 3.11: it installs like the RHEL
family, with the packages under their openEuler names (iproute holds
tc, cyrus-sasl the MD5 mechanism) and the RHEL family's main nginx
config. Other openEuler versions are refused with the supported one.
Tested on Ubuntu 22.04, 24.04 and 26.04, Debian 12 and 13, AlmaLinux 10
and openSUSE Leap 15.6. On Leap the panel did not start: the secret key
was written with python3 before the installer looked for Python 3.10+,
and Leap has only python3.11, so the key was empty. The installer still
said "Finished!" and exited 0, as it did for any failed step.
Now Python is found first. The package list update, each package, the
clone, the key, pip, migrate and collectstatic stop the installer with
a message when they fail (pipefail makes a logged step's status count).
Also:
- Ubuntu 26.04 is installed; Ubuntu 18/20 and Debian 9-11 are refused
up front (their Python is older than 3.10, so they failed later).
- The uos and openEuler branches are gone: untested, and their Python
is too old as well.
- Only `apt-get update`, no upgrade of the whole system.
- Debian no longer asks for smbios-utils (gone in Debian 13) and uuid,
which nothing uses; Debian and Ubuntu take python3-libvirt.
- RHEL reads the time zone like the others, and installs hostname for
ALLOWED_HOSTS.
- The unused wvcuser account is no longer created.
- The end of the run says where the admin password is.
It described Django 4.2 on Python 3.11, a distribution list the
installer does not support and no job worker. Now: the stack, the
processes and their units, what each app and vrtManager module holds,
how jobs claim, recover and share a host, and the API's routes; all
read from the code.
Behavior changes held one note per past fix; it keeps what an update
changes for operators and users. Migration is a section of bullets.
Docker cannot keep options that exist only in settings.py (LDAP,
OTP_ENABLED, LOGIN_TRUSTED_PROXIES): the README shows how to mount
one. The OpenLDAP examples use OpenLDAP names, the variables table
has ADMIN_USERNAME and ADMIN_PASSWORD, Contents lists every section,
and features name backups and the languages.
Removed: screenshots of the 2020 interface, the Gitpod badge and its
TODO-only config, and doc/readme.md, which held one word.
The daily backups stayed on the panel's host, so losing the host lost
them too. webvirt-offsite.timer now copies them at 03:45 with restic
over SFTP to another server, encrypted with a password only the
administrator and the host hold, and keeps 7 daily, 4 weekly and 6
monthly copies. It runs on the host as root, with Docker too, outside
the panel and its image. It fails when the newest backup is older
than 26 hours, so a stopped panel backup shows; an optional heartbeat
URL is called after each good copy.
The README has the setup, how to get a backup back from the copies,
and a fix found by the drill: moving to a new machine stopped when
the backup had no SSH keys (a panel without SSH computes).
The Backups section said only which files to put back on a new
machine. Now Restore and rollback has the steps, for Docker and for
installer and manual installations: restore the database, put back the
secret key, the worker's id and the SSH keys, and start. Users stay
signed in, the worker knows its old jobs and the computes accept the
panel again; the configuration outside the backup is brought over by
hand.
