The addresses were checked for their form only: a clone could keep the
VM's address, or give two NICs one, and the NICs on a network then take
each other's traffic. Such an address is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
The name's length was not checked: libvirt defined a clone of a name
longer than the panel's records hold (120), and the job failed to record
it, leaving a VM the panel did not list. The form and the job take 64,
as for a new VM.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
libvirt refused the copy only after the job had begun; recovery then
found a volume of the planned name and ended the job uncertain, holding
the VM. The names are checked when the job is queued and before it runs,
as a live clone did; that check now refreshes the pool first, so a live
clone's free space is read current too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
A worker that stopped during the copy left the partial files on the
destination: libvirt ends the migration and the VM stays, but the files
took the space and refused the next try, as a copy never replaces a
file. The job records the copies before libvirt makes them, and
recovery deletes them while the destination has no VM of it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
A libvirt error after a cancel ended the job cancelled at once; a
migration that finished but lost its last reply left the VM on the
destination and its record on the source. The error goes to recovery,
which ends the job cancelled when the VM stayed and succeeded when it
moved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
The source host alone was asked: a file on storage the computes share,
used by a VM or volume of another compute, was deleted under it. The
check the disk move makes, every compute and a host that cannot tell
counting as using it, moves to instances.utils and serves both. The
destination is not asked when its copy has the source's path: its VM
uses that copy, another file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kk4h2TcUiaWmU5MUszYSea
- The root password's SHA-512 crypt hash comes from libpass (the maintained
passlib fork, pure Python) instead of crypt-r, a copy of the stdlib crypt
module gone in Python 3.13: crypt-r ships no wheels and its build needs
crypt.h, which Ubuntu 26.04 no longer brings with python3-dev, so the
installer and the manual installation failed there. The hash is the same
format ($6$, now with libpass's 656000 rounds), checked with glibc's crypt.
- The README's RHEL compute packages left out python3-libguestfs, which
gstfsd imports with the system python: setting the password or key of a
shut-off VM failed there.
- The tests read YAML themselves and had PyYAML only through
drf-spectacular.
- The VM list with no VMs no longer also says that no VM matches the
search and the state.
- The New VM dialog with no compute yet says to add the first host, with a
link to the Computes page (and links there when none is reachable).
- The new VM page on a host without a running storage pool or a network
says what to add or start and where, instead of "You haven't defined any
storage pools" (also when pools exist but are stopped) and "network
pools".
A disk of the user's VMs whose size could not be read (its volume not in a
pool, the pool stopped) counted as 0 GiB, so a disk-limited user's increase
could pass over the limit. A disk no pool holds now has the size libvirt
reads from the file or device (blockInfo, also for a shut-off VM); one whose
size is still unknown makes the usage unverified and the increase is
refused, as when a host cannot be reached. Users without a disk limit are
not affected.
A revert gives the VM the vCPUs and memory it had when the snapshot was
taken. An owner could take a snapshot, shrink the VM, give the freed share
to another VM and revert, ending over the quota. The revert now checks
what it adds to the VM (against the larger of its running and next-start
sizes) as a resize does: on the page when it is queued, and again in the
worker before it runs, as a resize may have used the quota meanwhile.
A running VM's page read its addresses while it rendered: the guest agent,
when its channel is connected but it does not answer, holds the page (and
a gunicorn worker) for its whole 10 s timeout, on every load. The page now
shows them once it has loaded: vm_addresses.js asks the new
instances/<pk>/addresses/ for each NIC's addresses and fills the header's
IPv4 and each network card's IPv4 and IPv6.
A job keeps the client address of its submission (Operation.client_ip,
migration operations.0003); the rows the worker writes for it, including the
migration's and the VM delete's own rows, carry it. A cancel is written under
the user who asked and from their address, also when an administrator
cancels another user's job; a running job's cancel gets a row when asked,
and its end a row when it stops. A resolution carries the administrator's
address.
- A migration whose libvirt call failed but which recovery finds moved (or
cannot tell) gets a row of that end; only a failure already logged by
migrate_instance is not logged twice.
- A clone made with its resize or start failed, and a recovered migration
whose autostart or NVRAM step failed, write a failed row of that step, as
migrate_instance does for its later steps.
- Setting the link state of an interface or the video model of a VM that
has none is refused: it changed nothing and was logged as done.
- The rows of a removed VM record (deleted or expired) name its host, VM
and UUID, so the VM's history has them.
- libvirt errors all answered 400. A VM, pool or volume the host does not
have now answers 404, a volume in use 409 (vrtManager.util.InUse) and a
host that cannot be reached 503, which a client may retry.
- A template's power on or cycle and a stopped pool answered
{"status": ...} with 409; they answer {"detail": ...}, as every other
refusal. The schema documents these 409s (ErrorSerializer).
- The volume list and a volume of a stopped pool answered 200 with an
empty list or a made-up volume; they answer 409.
- An API URL that names nothing answered the HTML error page; it answers
a JSON 404 after the API's authentication.
- A session that must change its password was redirected to the HTML
form; an API request of it gets the JSON 403 a token gets.
English-only texts users read, now in their language:
- the serial console's four refusals, which socketiod sends in the
language LocaleMiddleware would choose (cookie, then browser) and logs
in English; the texts live in console/refusals.py, as makemessages
reads only .py files;
- a submission key reused for another request, a malformed MAC address,
the moved-VM refusal of a script's request, the API's "Templates
cannot be started" (now the page's sentence), "Pool is not active"
and the ISO upload's checks;
- "Problem occurred with host" was a fragment with the host's name and
error after it: one sentence with both.
- A failed cancel hook logged "worker task cancel failed": it now names
the job, its kind, resource and user, with the traceback for a fault
of the panel.
- novncd logged "Console connection failed" without the VM or the user.
- django-axes wrote four INFO lines a login ("Reset 0 access
attempts"...); logins are on the Logs page, so it logs warnings only.
- The new VM form's machine types and disk buses answered 200 with an
empty object when libvirt failed, and nothing was logged: the list
stayed empty, the form could not go on and said nothing. They answer
400 with the error, as the pool's volume list does, the warning is
logged with the compute, and the form's lists show an error they get
with the page's messages.
A job that failed on an exception that is not libvirt's or a refusal
showed its raw text to users (a SQL error such as "UNIQUE constraint
failed: instances_instance.uuid", a KeyError's key), and the worker
logged only that text. Users now read "The panel failed unexpectedly;
its log has the details." and the worker logs the exception with its
traceback.
A live clone whose recovery found the clone made but could not thaw the
source's file systems kept that in the error of a succeeded job, which
the job panel does not show. It is a warning of the clone's result, as
when the job itself made the clone.
Since the VM create became a job (6f0c7300) every new VM was defined with
no VNC password, and a clone copied its source's password (or its lack
of one) exactly: every clone of a template shared it. Computes listen
for VNC on all their addresses (dev/libvirt-bootstrap.sh), so anyone who
could reach a compute's VNC ports opened any of these consoles directly,
without the panel; the README's firewall rule was the only guard.
The create job asks for create.py's random 8-character password again,
and a clone's VNC graphics get a new random one. The console page sends
the password to a user with passwordless_console, which the user create
form now ticks by default next to change_password; a user without it is
asked for the password, as the permission intends. Existing VMs keep
their settings.
Live create, clone and VM page tests pass on a TCP compute.
Since deleting a VM became a job (#431), the delete page queues it and
returns to the VM page; the live VM page tests still expected the VM
gone at once and a redirect to the list, so they failed, and the VMs
they left behind failed later tests. They now run the queued job and
check the VM is gone.
- console settings were split over four files; who sees the form, the
listen address, and the settings themselves are test_console_settings,
and who may open a console is test_console_access;
- the three template files are test_templates;
- the quota lock test joins test_quota;
- accounts' test_security_fixes held two topics: the SSH key tests are
test_ssh_keys, the owner-row delete redirect is in test_views;
- instances' test_security_mutations, who may change a VM, is
test_permissions.
The suite runs the same 2097 tests.
Tests that build vrtManager objects and touch no view, form or model of
an app lived in that app; vrtManager/tests already holds such tests, so
the convention was split. They move to vrtManager/tests: the firmware,
CPU mode and PCIe port definitions of a new VM (test_create), USB and PCI
host devices, the console listener, host CPU reads, interface reads,
deletes and XML, network filters, PCI passthrough rules, node devices,
network changes, addresses and leases, pool XML, secrets, and the live
tests that use vrtManager only (disk resize, ISO, snapshots, XML, live
clone). livetest.py, the live-test helper six apps import, and the
network test driver move with them.
Others go to their app: the quota with a host down to instances, the
401/403 Logs rows (webvirtcloud's middleware) to webvirtcloud, the VM
page's tab script test to instances, novncd's own tests out of the
console views tests, the compute lock and input validation tests out of
the computes views tests, the interface views out of the forms tests.
instances/tests/test_views.py, entirely live, is test_live_views.py.
Test modules imported fixtures from each other; shared ones are now in
vrtManager/tests/fakes.py (migration fakes, node devices, IOMMU groups,
leases, a NAT network, a Ceph secret), interfaces/tests/fakes.py,
console/tests/daemons.py (novncd loaded once), livetest.wait_for and
instances/tests/create_page.BODY. Every live test checks
livetest.enabled(). The suite runs the same 2097 tests.
Ten classes ran on TestCase or TransactionTestCase although they read
files, templates or scripts only. SimpleTestCase skips the transaction
and refuses a database query, so a test that starts to use the database
says so. LoggingConfigTestCase also removes the temporary directory it
makes.
- create_admin had a branch for `"test" in sys.argv` that gave the admin
the password "admin"; 17 places signed in with it, and failed silently
with ADMIN_PASSWORD set in the environment. The tests now make their
own superuser and force_login it, and the shipped code has no test
branch: the test runner gives migrate a password (also when
ADMIN_PASSWORD is set but empty), so no data/admin_password is written
into the checkout.
- AccountsTestCase builds its rows in setUpTestData instead of
setUpClass and deleting them by hand.
- accounts tests set OTP_ENABLED with override_settings instead of
assigning settings (one assignment was never undone, so later tests in
the process ran with OTP on), and undo their app_settings change. The
OTP login class reloads the URLs again only after the setting is off.
- computes/tests/test_views.py reached the machine's own libvirt
(localhost over the socket) when TEST_LIBVIRT_HOST was unset; its live
tests now run only with TEST_LIBVIRT_HOST, like the other live tests,
as the README says.
Networks, Interfaces, Storages, Storage, Volume, MigrateInstance and
CreateInstance were unmanaged models with no table, there only so that
ModelSerializers and the new VM ModelForm could take their fields. They
made content types and migrations, and their DO_NOTHING foreign keys
added reverse accessors that meant nothing (Compute.host,
Compute.target_host, Instance.source_host).
The serializers are plain Serializers and NewVMForm a Form, field for
field as the models gave them (types, lengths, required, defaults, the
minimum of 1 for vCPUs, memory and disk size; the form's labels are
still derived from the field names). Compared field by field before and
after, the API schema changes only where the models were wrong:
- no int64 bounds on the listing fields;
- the interface fields lose the titles that named the wrong column
(type was "Status", state "Device", mac "Forward");
- a pool's volumes is a list of objects (it was described as an
integer) and a volume's allocation an integer (it was untyped).
One migration per app deletes the models; none touches a table.
instances/forms.py and instances/models.py imported values from the
webvirtcloud.settings module itself, which bypasses DJANGO_SETTINGS_MODULE
(settings-dev, test overrides) and runs the module a second time. They
now read django.conf.settings. webvirtcloud/urls.py also loses its
fallback that set REST_FRAMEWORK's schema class at import for settings.py
copies without it: the template sets it, and an older settings.py is
rebuilt from the template when updating.
The root password hash for a shut-off VM used crypt_r where installed
(Python 3.13) and fell back to the stdlib crypt, which Python 3.11 and
3.12 deprecate: `python -W error::DeprecationWarning manage.py check`
failed there. crypt-r (the same module, it requires Python 3.11) is now
installed from 3.11, and the module is chosen by the Python version;
3.10's crypt is not deprecated.
- noVNC's own web client (app/ except the bell sounds, vnc.html,
vnc_lite.html, defaults.json, mandatory.json): the console imports
only noVNC's core, and novncd serves no web pages. AUTHORS, LICENSE.txt
and package.json (the version) stay.
- 24 cached icons no {% bs_icon %} asks for; the icon cache test now
also fails on an icon nothing uses.
- Ace's ext-textarea.js, which nothing loads.
- instances/fixtures/initial_data.json, for an app ("create") that no
longer exists; migration 0006 seeds the flavors.
- webvirtcloud/asgi.py: gunicorn runs the WSGI application only.
- instances/templatetags/tags_active.py: a copy of common_tags'
class_active that no template loads.
- .github/linters/.hadolint.yaml, from super-linter, which no workflow
runs anymore.
Each was checked with git grep across every tracked file (templates, JS,
shell, unit files and the extensionless console daemons) and for
dynamic use (URL names, template names, registries):
- vrtManager: host_is_up, is_kvm_supported, get_emulators,
get_file_extensions, both get_network_forward, get_snapshots,
get_storages_images, get_os_type, get_host_arch, the get_pretty_* trio
and, in util, is_kvm_available, randomMAC, get_max_vcpus, compareMAC,
pretty_mem, pretty_bytes and validate_uuid; with them the port
constants only host_is_up read. Some were reached only by their tests,
which go too.
- The flavor edit page: nothing linked to it; flavors are created and
deleted on the Flavors page and edited through the API.
- computes.validate_name, UserForm.identity_has_errors, the has_perm
template tag, the ALLOW_EMPTY_PASSWORD setting nothing reads,
hidePleaseWaitDialog, default_app_config (ignored since Django 4.1)
and the Python < 3.6 lines of secret_generator.py.
The delete ran in the request: undefine, then the VM's record, then its
disk files. A request that died in between (a gunicorn worker past its
timeout, OOM, a restart), or a database write that failed after the
undefine, left a record that later showed as removed outside the panel,
and the disk files the user asked to delete were never deleted.
Deleting a VM is now an instance.delete job. It holds the compute
exclusively, as the request did, so no other VM attaches a file it is
about to delete. The job chooses the files before it stops the VM (a host
that cannot be checked still stops nothing) and records them first. It
then stops and undefines the VM, removes its record, and deletes the
files, each with its own Logs row. Recovery decides from libvirt: a VM
still on its host means the delete failed; a VM gone means libvirt did
it, so recovery removes the record and deletes the recorded files that
are still there.
The page queues the job and returns to the VM page, where the job panel
follows it and opens the VM list when it ends; disks that were kept are
the result's warning. The API's delete answers 202 with the job and
needs a submission_key. backing_path moves to wvmConnect, as recovery
has no domain to ask.
The worker sends cancels only to jobs its own threads run. After a
worker restart, a migration or live clone that libvirt still ran was
watched by recovery; a cancel then moved the job to cancelling, the page
said so, but nothing told libvirt, and the job ran to its end. Recovery
now sends the cancel once while libvirt still runs the job, and a
migration it aborted ends cancelled rather than failed.
migrate_instance sets autostart on the destination (it is per host and
the migration does not carry it) and deletes the NVRAM file libvirt
keeps on the source. A worker that stopped after libvirt moved the VM
skipped both: recovery only recorded the move, so an autostart VM no
longer started after the destination rebooted, and a later VM of the
same name on the source started with these UEFI variables. The job now
records both before the move, and recovery does them after a proven
move; a step that fails is a warning of the job's result.
A shut-off VM with a TPM or disks to copy migrates started paused
(migrate_running), and the job shuts it off afterwards on whichever host
has it. A worker that stopped in between left it running paused, on the
source when the migration did not happen or on the destination when it
did; recovery reported the outcome but left the VM running. The job now
records whether the VM was shut off, and recovery destroys it again
where it ended up.
After a successful copy, recovery defined the clone; a define that
failed raised, and recovery retried it with the same result. One case
is certain: a worker killed after define_clone copied the UEFI
variables but before it defined the VM leaves <clone>_VARS.fd, and every
retry is refused because the volume exists. The job then waited for
recovery with its claims on the source VM and the clone's name. A
libvirt error now ends it uncertain, naming the error and the copied
volumes; an unreachable host still waits.
A CD-ROM form sent twice (a double click, a resent request) added a
second drive with the same ISO. The form is refused when the ISO is in a
CD-ROM of the VM's running or persistent definition; an empty drive may
still be added more than once. tr and zh too.
The workbook kept every cell as an object: a 100 000-row Logs download
took some 400 MB in a web worker. openpyxl's write-only mode writes the
rows as they come (85 MB for the same file, measured), and the Logs rows
are read as tuples, not model objects. The bold, frozen, filtered header,
the column widths and the text cells stay as they were.
record_migration moved or merged a VM's owners without lock_ownership(),
which every other step that moves owners takes; on PostgreSQL a sync of
a third compute with the same UUID could move them at the same time, and
owners moved onto the source record went with its delete.
The name was checked only when the job was queued; recovery counts any
snapshot of that name as the job's, so one taken outside the panel
meanwhile made a failed job succeed. precheck now refuses the name
before any effect.
A drbdadm query that timed out, could not start ssh or failed showed
"None DRBD", as for a VM without DRBD resources. It now shows Unknown,
in the same red badge as a DRBD status that is not OK.
Every connection got a thread before any authentication, and every
signed request a guestfs appliance, with no limit: with remote access
allowed, anyone on that network could make the host run out of threads
or memory. A connection over MAX_CLIENTS (4) now gets an unsigned busy
line and is closed before it gets a thread; the panel shows it as the
gstfsd error.
No view used sensitive_post_parameters, so once ADMINS is set the error
mail of a failing POST would carry the password or key in plain text.
The user create, edit and password views, the password change, the
compute forms, the host secrets, the VM's root password and its console
settings now mark their POST data sensitive.
After the pivot the job asks every compute whether the old file is used;
libvirt has no call timeout, so one stuck host held the job, and a cancel
sent meanwhile held the VM host's control slot, for as long as the host
stayed stuck. The check now waits at most 60 s; a host that has not
answered by then might use the file, which is left behind.
The page checked only that the VM's host answers; a VM undefined with
virsh then failed at its first read and the page was a server error. It
now says the VM is no longer on its host and goes back to the list, whose
sync forgets the record (tr and zh too).
It looked the user's ownership up by the VM's name on its compute; names
are not unique there (a VM renamed outside the panel and a migration can
leave two), which gave a server error or another VM's rights on the page.
The VM and compute claim decorators checked only that the user sees the
VM, took the claim, and left the action's own right to the view inside:
a read-only viewer, or an owner who may not delete the VM, briefly held
the VM's or the whole compute's claim and could make other requests
there busy. They now check the least right first: power for a VM change,
delete for the compute-wide claim (destroy; its other views are
superuser-only).
Deleting a VM with its disks checks every other compute for the files. A
compute that could not be read raised an OperationError whose text already
held that compute's name and libvirt's error, so owners who do not see
hosts got both. It is now a ConnectionFailed, which users who do not see
hosts get as "cannot be reached"; libvirt's text comes from the error
itself, so a connection that never opened no longer reads "None".
The API's VM actions call the web views and only looked at the status
code; the views report a partial result (a disk that was not deleted or
was kept, a template that was not started) as a message for the page
they redirect to. The API reported plain success and left the messages in
the session. Its answer now carries them in "messages".
The API's create serializer leaves mac and nwfilter optional, but the
job's normalize gave them no default: without mac the request ended in an
HTML 500 (KeyError), without nwfilter it was queued and failed in the
worker. Both now default to empty, as the web form sends them.
Every other job re-checks its permission and state in precheck before any
effect; these two checked only when queued. A queued create of a user who
is no longer a superuser, or whose VM name a host took meanwhile, ran; so
did a volume clone whose name a volume made outside the panel took.
