The VNC page's token travels in the websocket URL, so nginx's access log
and anyone who saw the URL had it for 120 s, and it opened the console as
a bearer token. novncd now also wants the session cookie of the token's
user. Its refusals were a warning without a user or an address: they now
name the client address (nginx sends X-Real-IP to novncd too) and leave
a Logs row "VNC console refused: reason".
After the pivot the job asks every compute whether the old file is used;
libvirt has no call timeout, so one stuck host held the job, and a cancel
sent meanwhile held the VM host's control slot, for as long as the host
stayed stuck. The check now waits at most 60 s; a host that has not
answered by then might use the file, which is left behind.
The page checked only that the VM's host answers; a VM undefined with
virsh then failed at its first read and the page was a server error. It
now says the VM is no longer on its host and goes back to the list, whose
sync forgets the record (tr and zh too).
It looked the user's ownership up by the VM's name on its compute; names
are not unique there (a VM renamed outside the panel and a migration can
leave two), which gave a server error or another VM's rights on the page.
The VM and compute claim decorators checked only that the user sees the
VM, took the claim, and left the action's own right to the view inside:
a read-only viewer, or an owner who may not delete the VM, briefly held
the VM's or the whole compute's claim and could make other requests
there busy. They now check the least right first: power for a VM change,
delete for the compute-wide claim (destroy; its other views are
superuser-only).
The worker's checks refuse with PermissionDenied or Http404, which carry
no text: the job failed with an empty error and its Logs row gave no
reason. They now read "You may no longer do this." and "The virtual
machine is no longer in the panel." (tr and zh too).
Deleting a VM with its disks checks every other compute for the files. A
compute that could not be read raised an OperationError whose text already
held that compute's name and libvirt's error, so owners who do not see
hosts got both. It is now a ConnectionFailed, which users who do not see
hosts get as "cannot be reached"; libvirt's text comes from the error
itself, so a connection that never opened no longer reads "None".
The API's VM actions call the web views and only looked at the status
code; the views report a partial result (a disk that was not deleted or
was kept, a template that was not started) as a message for the page
they redirect to. The API reported plain success and left the messages in
the session. Its answer now carries them in "messages".
The API's create serializer leaves mac and nwfilter optional, but the
job's normalize gave them no default: without mac the request ended in an
HTML 500 (KeyError), without nwfilter it was queued and failed in the
worker. Both now default to empty, as the web form sends them.
Every other job re-checks its permission and state in precheck before any
effect; these two checked only when queued. A queued create of a user who
is no longer a superuser, or whose VM name a host took meanwhile, ran; so
did a volume clone whose name a volume made outside the panel took.
Only migrate_instance, inside the job's run, wrote the migration's row:
a migration refused by precheck, cancelled while queued, or ended by
recovery after a worker restart left none. The job now has a log entry
for the ends migrate_instance did not log; run marks those it did.
The internal revert's shut-off check ran in the job's run, after its
intent was recorded, so the refusal went to recovery: the job ended
uncertain and kept the VM's claim until a superuser resolved it. The check
(check_internal_revert) now runs in precheck too, so a page older than the
VM's start, or a direct POST, fails the job with the reason.
The list page synchronised and connected to every compute for every user,
while the poll and the API list read only the hosts of the user's own VMs.
Hosts a user has no VMs on no longer get a listing per page load, which
also took one of their request slots.
A web request refused because this process's requests to the host
(HOST_REQUESTS) all still wait for it raised the same ConnectionFailed as
a host that is down, so the VM page said the host cannot be reached. It
now raises HostBusy; the VM page and the error users who do not see hosts
get say the host is busy and to try again in a moment.
The token now names the console type it was made for: novncd accepts only
a VNC page's token and socketiod only a serial page's, so a VNC page's
token no longer opens the serial console without the VNC password.
A VNC session re-checks every 10 s, as the serial one did, that its user
may still open the console; a deactivated user or a removed owner loses
it. The check (may_still_open) moves to console.tokens for both daemons.
libvirt's default handler printed each error on stderr without saying
where it came from: in Docker those lines went to docker logs, and a
VM page read of a shut-off VM left one there. Every process that talks to
libvirt imports vrtManager.connection, which now registers a silent
handler; the errors still reach their callers as exceptions. The
console callback's own registration is dropped.
Each 401 or 403 answered to a signed-in user or to a request with
credentials (an API token) writes "Access denied (status): method path"
with the user and the address. Anonymous refusals are left out, so a
scanner cannot fill the table.
A run of failed opens is logged once, at its first failure, and the next
successful open ends it; a close callback and a request refused because the
host's requests still wait are logged each time. The lines go to the app log
of the process (web or job worker).
A request gets nginx's $request_id (or a new one when it comes without a
valid one) and sends it back in X-Request-ID. The app log's lines carry
the logger, process and thread, the request ID, the signed-in user and the
client address; gunicorn's access lines carry the client address from
X-Real-IP and the same request ID, so the two logs can be joined.
js-cookie served only table_filter.js, which kept a list's search in a
cookie for a day; the cookie went to the server with every request.
The search is now kept in the tab's sessionStorage: it stays across
reloads and pages in that tab, a new tab starts empty, and the server
never sees it. The library goes.
The updates on the login, SSH and serial console paths:
- django-otp 1.7.5 revokes the session of a device that is no longer
confirmed.
- python-engineio 4.15.1 and python-socketio 5.18.1 refuse requests and
events before the connection completes, HTTP requests after the
upgrade to WebSocket, and ../ in static file paths.
- paramiko 5.1.0 fixes agent keys and the default RSA signature
algorithm.
django-rest-knox 5.1.1 changes only its project's references, and
django-auth-ldap 5.4.0 drops Python 3.10, which the panel supports;
both stay.
django-bootstrap-icons (last release July 2024) only read the SVG of
each {% bs_icon %} from static/icon_cache; for an icon not there it
fetched the SVG from a CDN while rendering the page, and it brought
defusedxml along. A 20-line template tag in webvirtcloud reads the
same files; the package, its CDN path and the BS_ICONS_CACHE setting
go. requests stays: websockify needs it.
libvirt takes the same disk file or MAC twice in a persistent
definition. A double click or a resent request then attached an
existing volume twice, and a shut-off VM did not start again (QEMU
opens no file twice for writing), or added a second NIC with the MAC
the form had filled in. Attaching a volume that is already a disk of
the VM, live or in its next start, and adding a NIC with a MAC the VM
has are now refused with the reason.
gstfsd_request connected with the 120 seconds it waits for the change:
a host that was down held the request's thread two minutes before its
page said so, a wait no host bound counts. It now connects within
libvirt's CONNECT_TIMEOUT and gives the change its 120 seconds after.
libpq connects without a timeout of its own: a database server that
does not answer held each request for the operating system's TCP
timeout, minutes. The settings template sets connect_timeout to 5
seconds.
python-ldap waits for a server without end, to connect and for each
answer: a directory that stopped answering held every login's thread.
The settings template's LDAP block sets 5 seconds to connect and 10 for
each answer.
EMAIL_TIMEOUT was not set, and Django's SMTP connection then waits for
a server without end: one that accepts the connection and stops
answering held the request's thread for good, also on the login page's
Email OTP form, which anyone may post. The settings template sets 10
seconds. An administrator whose mail fails is told why on the account
page instead of getting an error page.
gunicorn writes a line of its access log for every request to the
container's output, and Docker's json-file log driver keeps all of it
by default: an open VM list alone adds a line every 5 seconds until the
disk is full. The compose file now keeps 5 files of 10 MB, as the
panel's own log.
Nothing removed the Logs page's rows, which every action and every
failed login adds, or the database sessions past their expiry. The job
worker's housekeeping now deletes the rows older than Log Retention
Days, a new setting under Lists and pages (1-3650, 365 by default), and
the expired sessions.
django-axes wrote an AccessLog row at every successful login and nothing
removed them; the Logs page already records each login. Its access log
is now off; failed logins are still counted for the lock.
The clone form's checks read the source's NICs from the live and the
persistent definition, and each NIC read the running VM's addresses
again since b2b2386d: a guest agent that is connected but hung held the
request for its timeout once a NIC and definition. The checks now run
in one cached_reads block, as the VM page does.
other_hosts counted the request on its own host again only when the
block ended normally: a quota check that raised left the request
counted on the other host while it went on with its own. The count is
now restored however the block ends.
upload_volume opens the libvirt stream before it creates the volume, but
created the volume outside the block that aborts the stream: a pool that
refused the volume (full, a name taken) left an unfinished stream to be
freed. The creation is now inside that block.
When the worker stopped after it defined a clone and before it resized
and started it, recovery recorded the clone at the source's size and
shut off, and the job read as a plain success. Recovery now finishes a
defined clone that is shut off as the run does; one that runs went
through the last step, starting, and is left as it is.
clone_data checked the clone's vCPUs and memory with str.isdigit(), which
takes characters such as a superscript two that int() refuses: the user
saw Python's "invalid literal for int()" instead of the refusal. It now
checks with str.isdecimal(), whose characters int() reads.
#394 read each host's states in one call and every domain's XML once,
for the title, memory and vCPUs. A user's list shows only their own VMs,
but it read the XML of every VM on those hosts: a user with two VMs on a
host of 300 made about 301 libvirt calls a list load, where the old
per-VM reads made 6.
get_domain_rows takes the UUIDs the list shows and reads only their XML;
the states still come in one call. The capacity reads every domain, as
before.
607c8381 sent each memory and disk resize back to its own sub-tab, but
the disk view's last redirect, the one after a resize was tried, still
went to #resize, which opens on the CPU sub-tab. Only the live tests
reach it, with a disk to grow: the full live suite failed there on every
connection type.
The live vCPU test built its post from the page's form, and the page now
disables the CPU fields while the VM runs, so the post lacked cur_vcpu
and was refused as not a number before the running VM was. It now posts
both fields, as the direct post it checks.
The page where a user changes their own password and the one where an
administrator sets a user's password shared an old card: a horizontal
form, a blue Cancel and a green Change. The administrator's had Django's
"Password-based authentication: Enabled/Disabled" with a paragraph about
other backends, and its password fields stayed while Disabled was chosen.
Both pages now have the user and group forms' layout, each its own
template. The administrator chooses how the user signs in, With a
password or Without a password (such as LDAP), and the password fields
and "must change it" show only with a password; Cancel goes back to the
user. A user's own page says that their other sessions end, or, when
they must change it, that they choose one before going on, without a
Cancel. A refused password is said at its field only, without the
general "Wrong Data Provided". The section styles of these forms move
into webvirtcloud.css.
The group form put every user as a checkbox in one unsorted list, the
members among them, with no way to find one: past a few dozen users a
wall of names. Its name field ran the width of the page.
It now has the user form's sections (Group, Permissions, Members), each
with what it is for beside its fields. Members come first, then everyone
else, each by username with their full name; a search box filters them
by username, name or e-mail (table_filter.js). A form's search, marked
data-forget, starts empty: the list pages remember theirs per page title,
which here is the same for every group. Hidden users stay in the form,
so saving while filtered keeps them. The button says Create group or
Save changes.
The create and edit pages of a user had one form for both, ordered for
editing: permissions first, the name and password folded below them and
the quota folded and closed at the end; a new user's name came last.
Both pages now go Account (name, password, may sign in), Role, Quota,
then Groups and permissions, each section with what it is for beside its
fields. Template editor and Superuser become one Role choice with User:
choosing Superuser hides the quota and permissions, which a superuser
does not need (no quota check applies, every permission is theirs); it
keeps the is_staff it had. The quota is always open: each limit takes a
number from 0, or No limit, which is kept as -1 as before; memory is
entered in GiB and kept in MiB, shown exactly so saving a user again
changes nothing. The button says Create user or Save changes.
The VM's keys page (instances:sshkeys) and the cloud-init user data
collected the owners' keys with one query an owner. Both now read them
in one query joined over the VM's owners, ordered owner by owner as
before.
Each row of the Users page asked has_perm whether its user may clone:
two queries a user (the user's permissions and the groups'). The list's
query now annotates it with one Exists over the clone permission, given
to the user or to a group of the user; a blocked user is shown without
it, as has_perm did. The page's query count no longer grows with the
users.
A host's capacity (Computes page, new VM dialog) came from
get_host_instances, which listed the VMs by ID and by name and then
looked each up, read its XML and its info: four calls a VM, for every
host on the page. It now uses the VM lists' get_domain_rows: the states
in one call and each VM's XML once. get_host_instances had no other
caller and is gone.
Checked on QEMU against the per-VM info of every domain (running, paused,
shut off): the same vCPUs and memory. A VM gone while the host is read
is left out instead of making the host's capacity unreadable.
A VM list row showed the VM's state, memory, vCPUs and title, read
through the VM's own connection object: a lookup by UUID, virDomainGetInfo
and the live XML, one after the other for every VM on the page.
The lists (Instances, grouped or not, and a compute's VMs) now read each
host once: the states in one getAllDomainStats call (the state group
only, as the lists' poll; no other group, so a VM busy in the QEMU
monitor holds nothing up) and the memory, vCPUs and title from each
VM's live XML, which the title already read. Checked on QEMU against
virDomainGetInfo for running, paused and shut-off VMs: the same values.
A VM gone from its host, or a host that cannot be read or fails its
listing, shows its VMs unknown as before. Only the grouped list, a
superuser's, prefetches every host's VMs and owners.
Every VM list load reconciles each compute with libvirt. Even when nothing
had changed, that took the compute's claims and a transaction with
SELECT FOR UPDATE (and the ownership advisory lock on PostgreSQL): about
11 queries a compute, 4 of them writes. On SQLite each load queued behind
every other write of the app.
The reconcile now first compares libvirt's VM list with the database,
without a claim or a transaction, and stops when they match and no
removed record has expired. Otherwise it runs as before, reading libvirt
again under its claims, so two reconciles still cannot commit in reverse
order. With two matching hosts a list load goes from 38 queries (8
writes) to 18 (none).
Nothing loads it: gunicorn (systemd, runit) and Django use
webvirtcloud.wsgi. It exec'd the shell script venv/bin/activate as
Python, which cannot run, and left that file open.
libvirt requires a stream that did not finish to be aborted before it is
freed. The NVRAM download of a shut-off move and upload_volume (NVRAM
upload, cloud-init seed) freed theirs after a failed send, receive or
finish without aborting, on the shared host connection. They abort now;
upload_volume opens its stream before it makes the volume, so the
cleanup has both.
