A revert gives the VM the vCPUs and memory it had when the snapshot was
taken. An owner could take a snapshot, shrink the VM, give the freed share
to another VM and revert, ending over the quota. The revert now checks
what it adds to the VM (against the larger of its running and next-start
sizes) as a resize does: on the page when it is queued, and again in the
worker before it runs, as a resize may have used the quota meanwhile.
A running VM's page read its addresses while it rendered: the guest agent,
when its channel is connected but it does not answer, holds the page (and
a gunicorn worker) for its whole 10 s timeout, on every load. The page now
shows them once it has loaded: vm_addresses.js asks the new
instances/<pk>/addresses/ for each NIC's addresses and fills the header's
IPv4 and each network card's IPv4 and IPv6.
A job keeps the client address of its submission (Operation.client_ip,
migration operations.0003); the rows the worker writes for it, including the
migration's and the VM delete's own rows, carry it. A cancel is written under
the user who asked and from their address, also when an administrator
cancels another user's job; a running job's cancel gets a row when asked,
and its end a row when it stops. A resolution carries the administrator's
address.
- A migration whose libvirt call failed but which recovery finds moved (or
cannot tell) gets a row of that end; only a failure already logged by
migrate_instance is not logged twice.
- A clone made with its resize or start failed, and a recovered migration
whose autostart or NVRAM step failed, write a failed row of that step, as
migrate_instance does for its later steps.
- Setting the link state of an interface or the video model of a VM that
has none is refused: it changed nothing and was logged as done.
- The rows of a removed VM record (deleted or expired) name its host, VM
and UUID, so the VM's history has them.
- A failed cancel hook logged "worker task cancel failed": it now names
the job, its kind, resource and user, with the traceback for a fault
of the panel.
- novncd logged "Console connection failed" without the VM or the user.
- django-axes wrote four INFO lines a login ("Reset 0 access
attempts"...); logins are on the Logs page, so it logs warnings only.
- The new VM form's machine types and disk buses answered 200 with an
empty object when libvirt failed, and nothing was logged: the list
stayed empty, the form could not go on and said nothing. They answer
400 with the error, as the pool's volume list does, the warning is
logged with the compute, and the form's lists show an error they get
with the page's messages.
A job that failed on an exception that is not libvirt's or a refusal
showed its raw text to users (a SQL error such as "UNIQUE constraint
failed: instances_instance.uuid", a KeyError's key), and the worker
logged only that text. Users now read "The panel failed unexpectedly;
its log has the details." and the worker logs the exception with its
traceback.
A live clone whose recovery found the clone made but could not thaw the
source's file systems kept that in the error of a succeeded job, which
the job panel does not show. It is a warning of the clone's result, as
when the job itself made the clone.
Since the VM create became a job (6f0c7300) every new VM was defined with
no VNC password, and a clone copied its source's password (or its lack
of one) exactly: every clone of a template shared it. Computes listen
for VNC on all their addresses (dev/libvirt-bootstrap.sh), so anyone who
could reach a compute's VNC ports opened any of these consoles directly,
without the panel; the README's firewall rule was the only guard.
The create job asks for create.py's random 8-character password again,
and a clone's VNC graphics get a new random one. The console page sends
the password to a user with passwordless_console, which the user create
form now ticks by default next to change_password; a user without it is
asked for the password, as the permission intends. Existing VMs keep
their settings.
Live create, clone and VM page tests pass on a TCP compute.
Since deleting a VM became a job (#431), the delete page queues it and
returns to the VM page; the live VM page tests still expected the VM
gone at once and a redirect to the list, so they failed, and the VMs
they left behind failed later tests. They now run the queued job and
check the VM is gone.
- console settings were split over four files; who sees the form, the
listen address, and the settings themselves are test_console_settings,
and who may open a console is test_console_access;
- the three template files are test_templates;
- the quota lock test joins test_quota;
- accounts' test_security_fixes held two topics: the SSH key tests are
test_ssh_keys, the owner-row delete redirect is in test_views;
- instances' test_security_mutations, who may change a VM, is
test_permissions.
The suite runs the same 2097 tests.
Tests that build vrtManager objects and touch no view, form or model of
an app lived in that app; vrtManager/tests already holds such tests, so
the convention was split. They move to vrtManager/tests: the firmware,
CPU mode and PCIe port definitions of a new VM (test_create), USB and PCI
host devices, the console listener, host CPU reads, interface reads,
deletes and XML, network filters, PCI passthrough rules, node devices,
network changes, addresses and leases, pool XML, secrets, and the live
tests that use vrtManager only (disk resize, ISO, snapshots, XML, live
clone). livetest.py, the live-test helper six apps import, and the
network test driver move with them.
Others go to their app: the quota with a host down to instances, the
401/403 Logs rows (webvirtcloud's middleware) to webvirtcloud, the VM
page's tab script test to instances, novncd's own tests out of the
console views tests, the compute lock and input validation tests out of
the computes views tests, the interface views out of the forms tests.
instances/tests/test_views.py, entirely live, is test_live_views.py.
Test modules imported fixtures from each other; shared ones are now in
vrtManager/tests/fakes.py (migration fakes, node devices, IOMMU groups,
leases, a NAT network, a Ceph secret), interfaces/tests/fakes.py,
console/tests/daemons.py (novncd loaded once), livetest.wait_for and
instances/tests/create_page.BODY. Every live test checks
livetest.enabled(). The suite runs the same 2097 tests.
Ten classes ran on TestCase or TransactionTestCase although they read
files, templates or scripts only. SimpleTestCase skips the transaction
and refuses a database query, so a test that starts to use the database
says so. LoggingConfigTestCase also removes the temporary directory it
makes.
- create_admin had a branch for `"test" in sys.argv` that gave the admin
the password "admin"; 17 places signed in with it, and failed silently
with ADMIN_PASSWORD set in the environment. The tests now make their
own superuser and force_login it, and the shipped code has no test
branch: the test runner gives migrate a password (also when
ADMIN_PASSWORD is set but empty), so no data/admin_password is written
into the checkout.
- AccountsTestCase builds its rows in setUpTestData instead of
setUpClass and deleting them by hand.
- accounts tests set OTP_ENABLED with override_settings instead of
assigning settings (one assignment was never undone, so later tests in
the process ran with OTP on), and undo their app_settings change. The
OTP login class reloads the URLs again only after the setting is off.
- computes/tests/test_views.py reached the machine's own libvirt
(localhost over the socket) when TEST_LIBVIRT_HOST was unset; its live
tests now run only with TEST_LIBVIRT_HOST, like the other live tests,
as the README says.
Each was checked with git grep across every tracked file (templates, JS,
shell, unit files and the extensionless console daemons) and for
dynamic use (URL names, template names, registries):
- vrtManager: host_is_up, is_kvm_supported, get_emulators,
get_file_extensions, both get_network_forward, get_snapshots,
get_storages_images, get_os_type, get_host_arch, the get_pretty_* trio
and, in util, is_kvm_available, randomMAC, get_max_vcpus, compareMAC,
pretty_mem, pretty_bytes and validate_uuid; with them the port
constants only host_is_up read. Some were reached only by their tests,
which go too.
- The flavor edit page: nothing linked to it; flavors are created and
deleted on the Flavors page and edited through the API.
- computes.validate_name, UserForm.identity_has_errors, the has_perm
template tag, the ALLOW_EMPTY_PASSWORD setting nothing reads,
hidePleaseWaitDialog, default_app_config (ignored since Django 4.1)
and the Python < 3.6 lines of secret_generator.py.
The delete ran in the request: undefine, then the VM's record, then its
disk files. A request that died in between (a gunicorn worker past its
timeout, OOM, a restart), or a database write that failed after the
undefine, left a record that later showed as removed outside the panel,
and the disk files the user asked to delete were never deleted.
Deleting a VM is now an instance.delete job. It holds the compute
exclusively, as the request did, so no other VM attaches a file it is
about to delete. The job chooses the files before it stops the VM (a host
that cannot be checked still stops nothing) and records them first. It
then stops and undefines the VM, removes its record, and deletes the
files, each with its own Logs row. Recovery decides from libvirt: a VM
still on its host means the delete failed; a VM gone means libvirt did
it, so recovery removes the record and deletes the recorded files that
are still there.
The page queues the job and returns to the VM page, where the job panel
follows it and opens the VM list when it ends; disks that were kept are
the result's warning. The API's delete answers 202 with the job and
needs a submission_key. backing_path moves to wvmConnect, as recovery
has no domain to ask.
The worker sends cancels only to jobs its own threads run. After a
worker restart, a migration or live clone that libvirt still ran was
watched by recovery; a cancel then moved the job to cancelling, the page
said so, but nothing told libvirt, and the job ran to its end. Recovery
now sends the cancel once while libvirt still runs the job, and a
migration it aborted ends cancelled rather than failed.
migrate_instance sets autostart on the destination (it is per host and
the migration does not carry it) and deletes the NVRAM file libvirt
keeps on the source. A worker that stopped after libvirt moved the VM
skipped both: recovery only recorded the move, so an autostart VM no
longer started after the destination rebooted, and a later VM of the
same name on the source started with these UEFI variables. The job now
records both before the move, and recovery does them after a proven
move; a step that fails is a warning of the job's result.
A shut-off VM with a TPM or disks to copy migrates started paused
(migrate_running), and the job shuts it off afterwards on whichever host
has it. A worker that stopped in between left it running paused, on the
source when the migration did not happen or on the destination when it
did; recovery reported the outcome but left the VM running. The job now
records whether the VM was shut off, and recovery destroys it again
where it ended up.
After a successful copy, recovery defined the clone; a define that
failed raised, and recovery retried it with the same result. One case
is certain: a worker killed after define_clone copied the UEFI
variables but before it defined the VM leaves <clone>_VARS.fd, and every
retry is refused because the volume exists. The job then waited for
recovery with its claims on the source VM and the clone's name. A
libvirt error now ends it uncertain, naming the error and the copied
volumes; an unreachable host still waits.
A CD-ROM form sent twice (a double click, a resent request) added a
second drive with the same ISO. The form is refused when the ISO is in a
CD-ROM of the VM's running or persistent definition; an empty drive may
still be added more than once. tr and zh too.
record_migration moved or merged a VM's owners without lock_ownership(),
which every other step that moves owners takes; on PostgreSQL a sync of
a third compute with the same UUID could move them at the same time, and
owners moved onto the source record went with its delete.
The name was checked only when the job was queued; recovery counts any
snapshot of that name as the job's, so one taken outside the panel
meanwhile made a failed job succeed. precheck now refuses the name
before any effect.
A drbdadm query that timed out, could not start ssh or failed showed
"None DRBD", as for a VM without DRBD resources. It now shows Unknown,
in the same red badge as a DRBD status that is not OK.
Every connection got a thread before any authentication, and every
signed request a guestfs appliance, with no limit: with remote access
allowed, anyone on that network could make the host run out of threads
or memory. A connection over MAX_CLIENTS (4) now gets an unsigned busy
line and is closed before it gets a thread; the panel shows it as the
gstfsd error.
After the pivot the job asks every compute whether the old file is used;
libvirt has no call timeout, so one stuck host held the job, and a cancel
sent meanwhile held the VM host's control slot, for as long as the host
stayed stuck. The check now waits at most 60 s; a host that has not
answered by then might use the file, which is left behind.
The page checked only that the VM's host answers; a VM undefined with
virsh then failed at its first read and the page was a server error. It
now says the VM is no longer on its host and goes back to the list, whose
sync forgets the record (tr and zh too).
It looked the user's ownership up by the VM's name on its compute; names
are not unique there (a VM renamed outside the panel and a migration can
leave two), which gave a server error or another VM's rights on the page.
The VM and compute claim decorators checked only that the user sees the
VM, took the claim, and left the action's own right to the view inside:
a read-only viewer, or an owner who may not delete the VM, briefly held
the VM's or the whole compute's claim and could make other requests
there busy. They now check the least right first: power for a VM change,
delete for the compute-wide claim (destroy; its other views are
superuser-only).
Deleting a VM with its disks checks every other compute for the files. A
compute that could not be read raised an OperationError whose text already
held that compute's name and libvirt's error, so owners who do not see
hosts got both. It is now a ConnectionFailed, which users who do not see
hosts get as "cannot be reached"; libvirt's text comes from the error
itself, so a connection that never opened no longer reads "None".
The API's VM actions call the web views and only looked at the status
code; the views report a partial result (a disk that was not deleted or
was kept, a template that was not started) as a message for the page
they redirect to. The API reported plain success and left the messages in
the session. Its answer now carries them in "messages".
The API's create serializer leaves mac and nwfilter optional, but the
job's normalize gave them no default: without mac the request ended in an
HTML 500 (KeyError), without nwfilter it was queued and failed in the
worker. Both now default to empty, as the web form sends them.
Every other job re-checks its permission and state in precheck before any
effect; these two checked only when queued. A queued create of a user who
is no longer a superuser, or whose VM name a host took meanwhile, ran; so
did a volume clone whose name a volume made outside the panel took.
Only migrate_instance, inside the job's run, wrote the migration's row:
a migration refused by precheck, cancelled while queued, or ended by
recovery after a worker restart left none. The job now has a log entry
for the ends migrate_instance did not log; run marks those it did.
The internal revert's shut-off check ran in the job's run, after its
intent was recorded, so the refusal went to recovery: the job ended
uncertain and kept the VM's claim until a superuser resolved it. The check
(check_internal_revert) now runs in precheck too, so a page older than the
VM's start, or a direct POST, fails the job with the reason.
The list page synchronised and connected to every compute for every user,
while the poll and the API list read only the hosts of the user's own VMs.
Hosts a user has no VMs on no longer get a listing per page load, which
also took one of their request slots.
A web request refused because this process's requests to the host
(HOST_REQUESTS) all still wait for it raised the same ConnectionFailed as
a host that is down, so the VM page said the host cannot be reached. It
now raises HostBusy; the VM page and the error users who do not see hosts
get say the host is busy and to try again in a moment.
js-cookie served only table_filter.js, which kept a list's search in a
cookie for a day; the cookie went to the server with every request.
The search is now kept in the tab's sessionStorage: it stays across
reloads and pages in that tab, a new tab starts empty, and the server
never sees it. The library goes.
libvirt takes the same disk file or MAC twice in a persistent
definition. A double click or a resent request then attached an
existing volume twice, and a shut-off VM did not start again (QEMU
opens no file twice for writing), or added a second NIC with the MAC
the form had filled in. Attaching a volume that is already a disk of
the VM, live or in its next start, and adding a NIC with a MAC the VM
has are now refused with the reason.
gstfsd_request connected with the 120 seconds it waits for the change:
a host that was down held the request's thread two minutes before its
page said so, a wait no host bound counts. It now connects within
libvirt's CONNECT_TIMEOUT and gives the change its 120 seconds after.
The clone form's checks read the source's NICs from the live and the
persistent definition, and each NIC read the running VM's addresses
again since b2b2386d: a guest agent that is connected but hung held the
request for its timeout once a NIC and definition. The checks now run
in one cached_reads block, as the VM page does.
When the worker stopped after it defined a clone and before it resized
and started it, recovery recorded the clone at the source's size and
shut off, and the job read as a plain success. Recovery now finishes a
defined clone that is shut off as the run does; one that runs went
through the last step, starting, and is left as it is.
clone_data checked the clone's vCPUs and memory with str.isdigit(), which
takes characters such as a superscript two that int() refuses: the user
saw Python's "invalid literal for int()" instead of the refusal. It now
checks with str.isdecimal(), whose characters int() reads.
#394 read each host's states in one call and every domain's XML once,
for the title, memory and vCPUs. A user's list shows only their own VMs,
but it read the XML of every VM on those hosts: a user with two VMs on a
host of 300 made about 301 libvirt calls a list load, where the old
per-VM reads made 6.
get_domain_rows takes the UUIDs the list shows and reads only their XML;
the states still come in one call. The capacity reads every domain, as
before.
607c8381 sent each memory and disk resize back to its own sub-tab, but
the disk view's last redirect, the one after a resize was tried, still
went to #resize, which opens on the CPU sub-tab. Only the live tests
reach it, with a disk to grow: the full live suite failed there on every
connection type.
The live vCPU test built its post from the page's form, and the page now
disables the CPU fields while the VM runs, so the post lacked cur_vcpu
and was refused as not a number before the running VM was. It now posts
both fields, as the direct post it checks.
A VM list row showed the VM's state, memory, vCPUs and title, read
through the VM's own connection object: a lookup by UUID, virDomainGetInfo
and the live XML, one after the other for every VM on the page.
The lists (Instances, grouped or not, and a compute's VMs) now read each
host once: the states in one getAllDomainStats call (the state group
only, as the lists' poll; no other group, so a VM busy in the QEMU
monitor holds nothing up) and the memory, vCPUs and title from each
VM's live XML, which the title already read. Checked on QEMU against
virDomainGetInfo for running, paused and shut-off VMs: the same values.
A VM gone from its host, or a host that cannot be read or fails its
listing, shows its VMs unknown as before. Only the grouped list, a
superuser's, prefetches every host's VMs and owners.
The tab opened on "Snapshots · Active" above the form, and listed the
snapshots below it as rows of "name · description · external, current
· date" with a solid red Delete on each. Which snapshot the VM goes on
from was the word "current" among the others.
Now a line runs down the tab, newest first. On top is Now, the VM in
its state's color, with the form that takes a snapshot there. Below it
each snapshot is a point: its name and date, its description on a line
of its own, its kind under them. The one the VM goes on from is filled
and says so; the others are rings. Revert and Delete are small outlined
buttons. Forms, confirmations and the rules that disable them are
unchanged.
The tab listed seven settings at one weight, each a two-column card with
a heading and a paragraph beside its control. The controls disagreed:
three switches posted on change, while the video model, the crash action
and the guest agent (an Enabled/Disabled select) each had their own
Save, and "From the next start" was repeated under three of them.
Now it has three parts. VM details: title and description with their
Save, and the template switch below them. Devices the guest sees: video,
virtio RNG, TPM 2.0 and the guest agent as rows of name, control and a
one-line note, under one line saying when changes apply; the video model
posts on change and the guest agent is a switch, with its connection
shown beside it as the Power tab's dot. When the guest crashes:
Restart / Stop as two buttons that post on click. Form names, posts and
who sees what are unchanged.
The Turkish catalog now calls the guest agent "konuk ajanı" everywhere.
Each sub-tab (CPU, Memory, Disks) opened on a two-column card that put a
heading and a sentence of current values beside the forms, and the CPU
tab added two more rows: Live vCPUs as a row of "0 · Fixed / 1 · On"
buttons and vCPU hot plug as a select with its own Save.
Now each sub-tab opens on the VM's size. CPU: one cell per vCPU up to
the maximum, filled when on, with "2 on, up to 4" and the host's logical
CPUs; on a running VM with hot-pluggable vCPUs the cells are the
buttons that turn them on and off. Memory: a bar of the current memory
within the maximum, with the host's memory. Disks: a row per disk with
its format, size now and a new size. The forms below keep their names
and posts; the CPU fields are disabled while the VM runs, and hot plug
is a switch that posts on change. A memory or disk resize returns to
its own sub-tab.
The Resize tab offered vCPU counts only up to the host's logical CPUs.
A VM whose maximum was higher (4 on a 2-CPU host, say) found no option
for it, so the Maximum list showed 1, and a resize of the shut-off VM
lowered its maximum to that without a word.
get_max_cpus() now takes the VM's own maximum as a floor, and the
Instance's vcpu_range passes it. The host's CPU count shown on the page
and used by the clone form is read from the host alone, not from that
range.
A checkbox marked a device as booting and also chose what Up and Down
moved, which moved every checked device at once: to move one device the
others had to be unchecked and checked again. The buttons sat apart
from the rows, and a VM booting by device type got a note that did not
say what to do.
Now the devices the VM tries are a numbered chain, each with its own
up, down and Remove, and the others are listed below it with Add; the
posted order is unchanged. Devices read by kind first (CD-ROM sda,
Disk vda) with their file or network below; a VM booting by device
type says so and how to choose an order. Autostart and the boot menu
sit below, lighter. Focus stays on the device that moved, and on a
narrow screen a device's buttons wrap below its name.
The Turkish catalog says "Ön yükleme" and "sanal makine" throughout,
also in the Power tab texts.
