1
0
Fork 0
mirror of https://github.com/retspen/webvirtcloud synced 2026-11-08 11:16:47 +00:00
Commit graph

1484 commits

Author SHA1 Message Date
catborise
99eadaab31 fix(datasource): cloud-init endpoints work without login, keys need a unique name
The OpenStack-style cloud-init endpoints (index, meta_data.json,
user_data) are fetched by booting VMs, which do not log in. Since the
login-required middleware they redirected to the login page, so the
datasource has not worked. They are login_not_required now; they expose
a hostname and the public SSH keys of a VM's owners.

A VM is found by the reverse DNS name of the client address, and VM
names are not unique (only compute and UUID are). user_data merged the
owners of every VM with that name, so a user who named their VM like
another user's VM got their key into that VM at its next cloud-init
run. Keys are now given only when exactly one VM has the name.

The client address is the last X-Forwarded-For entry, which nginx in
the default setup appends; gunicorn listens on 127.0.0.1 only. The bare
except around the reverse lookup is narrowed to OSError.
2026-10-04 20:46:15 +03:00
catborise
99dabbdcec fix(accounts): migrate no longer overrides the change-password permission
SHOW_PROFILE_EDIT_PASSWORD was replaced in 2020 by the change_password
permission, settable per user or group. A post_migrate hook applied the
old setting to all users so existing installs kept their behavior, and
asked to remove the setting afterwards; but the settings template still
set it to True. Every migrate, so every upgrade, therefore granted the
permission to all users again and undid an admin's per-user or
per-group choice (with False it would have revoked it from everyone).

- Migration accounts.0008 applies the setting one last time, if a
  settings.py still has it; the post_migrate hook is gone.
- New users get the permission when they are created (any path,
  including LDAP); the admin user form ticks it and saves the admin's
  choice afterwards.
- The template no longer sets SHOW_PROFILE_EDIT_PASSWORD; a check
  (accounts.W001) asks to remove it from existing settings, and the
  README upgrade notes say so.

The test that removes the permission and runs migrate fails on the
previous code. An upgrade from master with users was simulated: the
setting is applied once, and a permission removed afterwards stays
removed through the next migrate.
2026-10-04 20:26:53 +03:00
catborise
026391907a chore(computes): remove the unrouted video models view
computes.views.get_compute_video_models had no URL and no caller, and
ComputeManager.get_video_models was used only by it. The video models
of the create page and the VM settings come from wvmConnect directly.
2026-10-04 19:41:56 +03:00
catborise
5c4eb97410 fix(instances): clone name and MAC read the same dhcpd.conf
An optional dhcpd.conf next to the application supplies clone names and
their MAC addresses. get_dhcp_mac_address read it from BASE_DIR, but
guess_clone_name from a fixed /srv/webvirtcloud/dhcpd.conf, so on any
other install path the Guess button found no name while the MAC lookup
still worked. Both use utils.dhcpd_conf() (BASE_DIR) now.
2026-10-04 19:39:29 +03:00
catborise
80a94a36b6 fix(instances): the XML editor saves only while the VM is shut off
The editor shows its Change button only for a shut-off VM, but
change_xml did not check the state: a page loaded while the VM was off
could still be saved after it was started, and the posted XML then
overwrote every change made to the definition in between. The view now
refuses unless the VM is shut off, with a message.
2026-10-04 19:13:20 +03:00
catborise
03ac93ea53 fix(instances): CD-ROM bus, USB controller and MACs of a new VM
Creating each option on a RHEL qemu-kvm host showed:

- CD-ROM: the create page offered every disk bus, but the empty CD-ROM
  of a new VM works only on ide, sata and scsi. fdc and virtio failed
  ("Invalid bus type", "does not support ejectable media"), usb failed
  with a device named 'Nonea' because vol_dev_type() had no prefix for
  usb, and after that libvirt refuses an empty usb disk. Only ide, sata
  and scsi are offered now, and another value is refused before any
  storage is allocated. vol_dev_type() names usb disks sdX, which also
  fixes data volumes on the usb bus.
- USB controller: any input device added a bare <controller
  type='usb'/>, which libvirt turns into piix3-uhci (USB 1.1) on q35
  instead of its default qemu-xhci. The bare controller is gone; libvirt
  adds the machine's default for USB inputs.
- MACs: with more networks than MAC addresses, create_instance indexed
  past the list and the request failed with a 500. NICs without a given
  MAC now get one from libvirt. MAC addresses are validated (format, not
  more than networks) by the page and the API before anything is
  created; they went into the XML unchecked.

Unit tests cover the refused values, the MAC rule and the usb prefix.
Live tests create a VM with every offered CD-ROM bus and start it,
check the USB controller with a USB input device and create a VM with
two networks and one MAC.
2026-10-04 18:33:28 +03:00
catborise
525018dd99 fix(interfaces): validate IPv6 address and gateway of host interfaces
The form checked both with re.match("^[0-9a-f./:]+|^$"), which tests
only how a value starts: "2001:db8::10/64'/><evil/>" passed and went
into the interface XML, which is built with f-strings, while upper-case
hex was refused. An address without a prefix passed too and then failed
with a ValueError when the XML was built.

The address is parsed with ipaddress.IPv6Interface and stored as
address/prefix (/128 when none is given), the gateway with
IPv6Address. A static configuration needs an address, and the XML gets
a route only when a gateway is given instead of gateway=''. The HTML
patterns accept upper-case hex.

Eight of the ten form and XML checks fail on the previous code.
2026-10-04 17:29:59 +03:00
catborise
78ab85f211 fix(instances): a secure boot loader on q35 gets secure boot
create_instance set secure='yes' (and with it SMM) only while switching
a non-q35 machine to q35. With q35 already chosen, a secure boot loader
was defined with secure='no' and without SMM. Secure boot is now
requested whenever the loader is a secure boot build; the switch to q35
is unchanged.

libvirt 11 on RHEL fills in secure='yes' and SMM from the firmware
descriptor of a known loader, so there the domain was already correct;
without such a descriptor (older libvirt, custom loader) it was not.
A unit test checks what WebVirtCloud requests (fails on the previous
code); a live test creates a q35 VM with the host's secure boot loader
and starts it.
2026-10-04 17:26:25 +03:00
catborise
cdbff38223 fix(instances): the create page offers the UEFI loader of the chosen machine
create_instance passed the loader list where find_uefi_path_for_arch
expects the machine type. The domain capabilities were then read for
the default machine (pc) instead of the chosen one: on a RHEL host a q35
VM was offered /usr/share/OVMF/OVMF_CODE.secboot.fd, which q35 does not
list, instead of /usr/share/edk2/ovmf/OVMF_CODE.fd. The machine type is
passed now.

The page also read dom_caps["loader_enums"] into an unused variable;
on a host without loader support the key is missing and the page failed
with a KeyError. The line is removed.

A unit test covers the call and the host without loader support (it
fails on the previous code). A live test creates a q35 VM with the
offered UEFI firmware, checks that libvirt lists that loader for q35 and
starts the VM; it fails on the previous code.
2026-10-04 17:22:54 +03:00
catborise
16e1a334d0 fix(instances): disk and agent channel changes on paused VMs
attach_disk (new or existing volume, CD-ROM), add_guest_agent and
remove_guest_agent acted only when the status was running (1) or shut
off (5). On a paused VM they changed neither definition and the request
reported success. They now use the same rule as the NIC edits: the
running definition while the VM is active, paused included, and the
persistent one always. remove_guest_agent reads each definition from its
own XML, so a channel present only in the persistent one is removed too.

Unit tests check paused, running and stopped VMs (paused fails on the
previous code). The live effects test adds attach_disk and
add_guest_agent, and now also checks that edits meant for the running VM
reach it, which showed that an ejected CD-ROM keeps an empty <source>
in the live XML; NIC unplug is left to the guest tests, as these VMs
have no OS to acknowledge it.
2026-10-04 17:10:49 +03:00
catborise
9931b73b2c test(instances): the guest sees the hardware WebVirtCloud configures
New live guest tests compare what the guest agent reports (online vCPUs,
memory blocks, disk serials and buses, NIC MACs) with the persistent
definition, through the web views:

- at boot, with a data disk;
- after force off, CPU and memory resize and power on (2 vCPUs, 3 GiB);
- a hotplugged disk at once; a bus and serial change after powercycle;
- a NIC MAC change after powercycle, not before;
- a clone boots with its own MAC and disk copy;
- reverting a running snapshot removes a NIC added after it, in the
  guest and in the definition, and leaves the VM running;
- a suspended guest does not answer and answers again after resume.

Every request in these tests must redirect (302), so a failing view is
not mistaken for a no-op. Only NICs with QEMU's 52:54: prefix are
compared, as a guest may create its own interfaces. All 13 guest tests
pass against a Rocky Linux 9 guest.
2026-10-04 16:39:02 +03:00
catborise
a47fb9d825 test(instances): live tests with a running guest OS
The live tests so far ran VMs without an OS, so everything that needs the
guest's cooperation was untested. With TEST_LIBVIRT_GUEST_IMAGE set to an
installed Linux image with qemu-guest-agent, instances/test_live_guest.py
boots wvc-test VMs from qcow2 overlays of it (the image is only read) and
checks through the web views:

- delete_vol on a running VM: the guest releases the disk, so it leaves
  the live and persistent definitions and the volume is deleted;
- detach_vol removes the disk from the running guest and keeps the volume;
- add_network/delete_network: the guest sees the hotplugged NIC, and the
  unplug completes in both definitions;
- set_vcpu brings a hotplugged vCPU online in the guest;
- osinfo returns the guest agent's OS, time zone and host name, and the
  agent reports the VM's NIC by MAC;
- poweroff shuts the guest down through libvirt.

The image must be a BIOS-bootable qcow2 volume in a storage pool other
than wvc-test; the tests are skipped while a running VM uses it. They pass
against a Rocky Linux 9 guest on a RHEL qemu-kvm host; the guest had no
address on the NAT network (its network configuration is static), so only
the agent's NIC report is checked, not an address.
2026-10-04 15:39:45 +03:00
catborise
b150a9dd36 fix: stop ignoring SIGCHLD process-wide
The web app (wsgi.py), novncd and socketiod set SIGCHLD to SIG_IGN at
import. The kernel then reaps every child at once, so subprocess loses
the exit status and a failing command reports 0, in the server and in
anything else run in that process: the test suite had to reset the
signal in two tests.

It was added in 2023 against one defunct bash per gunicorn worker and
per novncd. That bash came from the settings template of the time,
which ran .dec_ldap_pwd.sh with Popen and never waited for it; the
template stopped doing that in 561fedf. A settings.py still holding
that line leaves at most one zombie per process, not a growing number.

- wsgi.py, novncd, socketiod: the global SIG_IGN is removed. subprocess
  reaps its own children, websockify installs its own SIGCHLD reaper
  for novncd's connection processes, and novncd's SSH tunnels wait for
  their ssh process.
- socketiod: the pty child execs consolecallback instead of running it
  and then returning into the server's code; disconnect reaps that
  child (waitpid) instead of any child (wait).
- The settings template drops its unused subprocess import; the two
  test workarounds are removed.

New tests load the web app and novncd in a fresh interpreter and check
that SIGCHLD is untouched and a failing command still returns 1; both
fail on the previous code. A patched novncd in the test container
served five connections and left no child behind.
2026-10-04 13:40:53 +03:00
catborise
792487e65e fix(instances): failures reach the user; no redirect to other sites
Web views:
- 34 views returned to request.META["HTTP_REFERER"] + "#tab": without
  a Referer they failed with a TypeError (500), and a Referer from
  another site was followed (open redirect). They return through
  _back(), which keeps a same-site Referer and otherwise opens the VM
  page at the tab. get_safe_redirect uses the same check. Rejected disk
  options also return to the VM's disks tab instead of the VM list.
- migrate_instance returned silently when the target was the current
  compute or unreachable, so the web view logged "Instance is migrated"
  and the API answered "instance migrate is started". It now raises;
  the view reports the error and logs only a completed migration, and
  an unknown compute id is a 404 instead of a 500. The automatic
  migration after a clone skips a target that is the current compute.
- add_owner and del_owner failed with a 500 without their ids, add_owner
  also when the user was already an owner, and del_owner deleted the
  given ownership record even when it belonged to another VM. Ids are
  validated, a record of another VM is a 404, and a repeated owner is
  reported.
- Broad handlers that hid programming errors are narrowed: clone catches
  libvirt, value and OS errors (a lock timeout included), ownership
  lookups catch DoesNotExist. The DRBD status no longer needs its bare
  except: the VM name is escaped in the regex, which was the only thing
  that could fail there.

API:
- A libvirt error under /api/ is JSON with status 400, and a lock
  timeout 409, instead of the HTML error page. This lives in the
  existing ExceptionMiddleware, so existing settings need no change.
- Volume create/delete on an inactive pool answer 409, and invalid
  volume data 400 with the serializer errors, instead of 200.

The new tests cover every view that returns with _back() without a
Referer and with a foreign one, the migrate failures on web and API,
the owner ids, the API error format and the volume statuses; nine of
the ten fail on the previous code.
2026-10-04 12:23:54 +03:00
catborise
a2b8bb1cd5 ci: run tests and lint on pushes to every branch
Branches no longer need a pull request to master to be tested; a
branch is checked when it is pushed and lands on master by
fast-forward once it is green. Pull requests to master still run the
checks. CodeQL keeps running on master, its pull requests and its
weekly schedule.
2026-10-04 12:07:36 +03:00
catborise
f94b3cee7a chore: drop references to the local roadmap from code and tests
Comments, docstrings and live test names cited item IDs of a roadmap
that is not part of the repository. They now say what is meant, or
nothing where the sentence stands without them. Test assertions and
behavior are unchanged.
2026-10-04 11:09:25 +03:00
catborise
39feb373ed fix(vrtManager): host_is_up returns a bool; slow hosts block no others
- host_is_up returned the exception object for a down host, which is
  truthy: check_user_quota then connected to it and failed, so a user
  with a VM on a down host could not open its page, clone or resize,
  and migrate_instance's "target is down" check never fired. It returns
  True or False now, closes its socket, and accepts IPv6 and host:port.
- The connection manager no longer holds its global lock while a new
  connection is opened, so one slow host does not delay new connections
  to the others.

Bounding the connection time itself (including SSH aliases and jump
hosts) is left for a separate change.
2026-10-04 01:08:11 +03:00
catborise
d054b13a87 fix(computes): VM ownership survives a move or a short absence
The sync deleted the Instance of every VM missing from its compute, and
its owners and rights went with it. A VM moved with virsh, or undefined
and defined again, came back with no owner.

Now, when a VM disappears from a compute:
- if the same UUID is recorded on another compute, it moved: its owners
  and template flag go there;
- otherwise they are kept in an InstanceTombstone and restored when the
  UUID appears again within INSTANCE_OWNERSHIP_RETENTION_DAYS (30).
A UUID recorded on two computes at the same time is a copy and gets no
owners. VMs deleted through WebVirtCloud leave no tombstone.
2026-10-04 01:08:11 +03:00
catborise
abdd229a1f fix(instances): a failed VM creation leaves no volumes behind
Web form: the cache mode and the UEFI firmware value were checked only
after the disk volume had been created; an invalid one answered 500 and
left the volume. All checks now run before any allocation. Volumes the
request creates are removed on any failure up to the definition; ones
that cannot be removed are reported. After the definition the VM uses
them, so a later failure keeps them.

API: the new volume was never removed when the definition failed, and
the error was a 500. It is removed now and a libvirt error returns 400.
2026-10-04 01:08:11 +03:00
catborise
5def4e41fd fix(instances): NIC edits keep their settings; edits keep metadata
- change_network detached the NIC and attached one built from scratch,
  dropping its QoS, link state and other settings. It now edits the NIC
  of the persistent definition in place and defines once; a running VM
  gets the change at its next start, as the UI already says.
- add_network, delete_network and set_link_state did nothing on a
  paused VM, and set_link_state rewrote the persistent NIC from the live
  XML. They now update the running definition when the VM is active and
  the persistent one always, each from its own XML.
- Boot and console edits used xml.etree, which renamed the namespace
  prefixes of other tools' metadata (<app:info> became <ns0:info>); they
  use lxml now.
- The NIC table and the clone form are filled from the persistent
  definition (Instance.config_networks, config_disks), so a pending NIC
  change is shown and can be edited again.

A live test runs 18 domain edits on stopped, running and paused VMs
whose definition has elements WebVirtCloud does not manage, and checks
for each the effect, that nothing else in the persistent definition
changed, and that persistent-only edits leave the running VM alone.
2026-10-04 01:08:11 +03:00
catborise
57cf49262a fix(instances): clones roll back their copies; own error messages stay
Clone:
- only shut-off VMs are cloned, as the UI already required: a running
  guest keeps writing while its disks are copied; the definition comes
  from the persistent XML;
- MAC addresses and disk destinations are checked before anything is
  allocated;
- every volume the clone creates (NVRAM and disks) is tracked; if a
  later copy or the definition fails, only those are deleted, and any
  that cannot be deleted are named in the error. Before, a failed clone
  left its earlier copies behind.

libvirtError(msg) reports the thread's last libvirt error instead of
msg when there is one, so WebVirtCloud's own errors could show an
earlier, unrelated failure. util.OperationError, a libvirtError
subclass that keeps its message, replaces those raises in vrtManager,
instances and nwfilters; 'except libvirtError' still catches it.
2026-10-04 01:08:11 +03:00
catborise
e5fda79f5a feat(console): VNC is the only console type
SPICE was removed earlier; its leftovers and the console type choice go
too:
- the console settings form, update_console and the backend no longer
  offer or change the type; an invalid console form now shows its
  errors instead of being ignored;
- the create form, the API serializer and create_instance no longer take
  a graphics type, new VMs get VNC; get_graphics_types and the graphics
  entries of the domain capabilities are gone;
- migration appsettings.0013 deletes QEMU_CONSOLE_DEFAULT_TYPE;
- a VM defined elsewhere with another graphics type gets "Console type
  ... is not supported".

The locale catalogs still reference the deleted SPICE templates in
source comments; regenerating them is left to the translation work.
2026-10-04 01:08:11 +03:00
catborise
9f86330552 fix(instances): ISO mount/unmount for every VM state and CD-ROM
The old code edited the live XML, called attachDevice and redefined the
domain from it: a stopped VM lost its VNC password, a paused VM crashed
with UnboundLocalError, an unknown CD-ROM was silently ignored (on a
running VM the last disk of the loop could be updated instead), and a
missing ISO raised AttributeError.

Media changes now use updateDeviceFlags on the requested CD-ROM, built
from the live and the persistent definition separately. A CD-ROM
missing from one definition, an unknown ISO, an ISO name present in
more than one pool, or a change that reached only the running VM is
reported to the user. The mount and unmount log messages were swapped.
2026-10-04 01:08:11 +03:00
catborise
08bf39ef61 fix(instances): domain edits start from the persistent definition
Edits that redefine a domain read the wrong XML:
- boot menu/order, vCPU hotplug and QoS read XMLDesc(0), which has no
  secrets, so every such edit, even on a stopped VM, removed the VNC
  password from the definition;
- console, video, CPU, memory and title/description edits read the live
  XML, so on a running VM pending changes (e.g. a new memory size) were
  reverted and runtime state was written into the definition.

All of them now read VIR_DOMAIN_XML_INACTIVE | SECURE (PERSISTENT_XML).
The console setters take the first graphics device of that definition
instead of looking it up by the live type.

Disk resize no longer redefines the domain. A running VM's image files
grow through blockResize, since qemu-img cannot take the lock QEMU
holds; block volumes (e.g. LVM) are grown by the storage pool first and
QEMU is then told the new size.

Live tests reproduce the losses on a RHEL qemu-kvm host; unit tests
check the XML flags, the console setters and the resize paths in CI.
2026-10-04 01:08:11 +03:00
catborise
47c480f059 fix(bootstrap): the compute bootstrap script parses again
The quoting of the tr argument in the /etc/os-release parsing put the
closing parenthesis inside a single-quoted string, so sh and bash
stopped with "unexpected EOF while looking for matching )" before doing
anything: 'curl ... | sudo sh' failed on every distribution. It is now
tr -d "\"'"; bash -n and shellcheck -S error pass.

Also drops the spice_listen edits of qemu.conf; only VNC is used.
2026-10-04 01:08:11 +03:00
catborise
65a9b9e370 docs(readme): say which packages come from the distro
The virtualenv takes libvirt-python and python-ldap from the distro
when they satisfy conf/requirements.txt, and builds them otherwise.
2026-10-04 01:08:11 +03:00
catborise
6f12da4857 ci: measure coverage and fail below 50%
The test step runs under coverage and reports files that are not fully
covered. pyproject.toml holds the configuration and the floor: 51%
measured without the live libvirt suite on 2026-10-03, floor 50% to
absorb differences between Python versions. With the live suite against
a real host the same code is 69% covered.
2026-10-04 01:08:11 +03:00
catborise
63b9f86436 build: drop unused dependencies
ldap3, markdown, zipp, qrcode and tzdata are not imported anywhere and
no installed package needs them on Linux with Python 3.10-3.13: LDAP
login uses python-ldap through django-auth-ldap, QR codes come from
django-qr-code (segno), qrcode is only an optional django-otp extra,
tzdata is a Windows-only Django dependency and TIME_ZONE is UTC.
python-engineio stays pinned next to python-socketio, which requires it.

The test suite passes with the five packages uninstalled.
2026-10-04 01:08:11 +03:00
catborise
c789406edf fix(instances): add_network escapes posted values
add_network built the interface XML with f-strings, so a posted model
or nwfilter could add arbitrary devices to the domain. It is built with
lxml now; a missing MAC is left to libvirt instead of address='None'.
2026-10-04 01:08:11 +03:00
catborise
e94d69a524 feat(accounts): Django's LoginRequiredMiddleware replaces the package
django-login-required-middleware is dropped; Django 5.1+ ships the same
middleware. The login page is already marked login_not_required by
Django; email_otp gets the decorator, which replaces
LOGIN_REQUIRED_IGNORE_VIEW_NAMES. The datasource still requires login.

Existing settings.py files still name the old middleware. A system
check (accounts.E002) stops migrate and the server until MIDDLEWARE
uses Django's middleware, instead of an ImportError at startup; the
README has the steps.

A new matrix test walks every URL anonymously: everything outside a
short allowlist (login pages, the OpenAPI docs) sends the visitor to
the login page, and API views deny with 401/403. It passed with the old
middleware first and passes unchanged now.
2026-10-04 01:08:11 +03:00
catborise
920f318601 build: Django 5.2.17 LTS
Django 4.2 reached end of life on 2026-04-07.

- Django==5.2.17; django-login-required-middleware 0.9.0 keeps working
  (the authorization matrix passes).
- USE_L10N removed from the settings template.
- Log out is a POST form in the navbar; Django 5 removed GET logout.
- pip-audit in CI no longer ignores the Django 4.2.30 advisories.
- README: Django 5.2, backup step and upgrade notes for existing
  settings and GET logout callers.

python -Wd manage.py test shows no Django deprecation warnings. The live
libvirt suite passes on 5.2 and on 4.2 against a RHEL qemu-kvm host.
2026-10-04 01:08:11 +03:00
catborise
21f0777e76 fix(instances): internal snapshots no longer rewrite UEFI loaders
Taking or reverting an internal snapshot redefined a pflash VM with a
rom loader around the operation. libvirt 10.10 rejects that definition,
so the request ended in a 500; where it was accepted, the snapshot
recorded the fake loader, and a failure between rewrite and restore
left the VM unable to boot.

The definition is no longer touched: libvirt decides, and its error is
shown (for a running VM with raw NVRAM: 'internal snapshots of a VM
with pflash based firmware require QCOW2 nvram format'). A stopped VM's
disk-only internal snapshot is taken, as libvirt allows it. Revert
errors are shown too. The external snapshot path keeps its rewrite for
now.

Live tests cover running and stopped UEFI VMs; both failed with a 500
on the previous code.
2026-10-04 01:08:11 +03:00
catborise
3e427a41b8 fix(instances): find known VMs by UUID, not by name
Instance.proxy looked the domain up by its stored name. After a rename
or a name swap outside WebVirtCloud, an operation on one VM changed
another; after a VM was replaced under the same name, operations hit
the new VM.

wvmInstance takes the UUID and uses lookupByUUIDString, with no
fallback to the name: a missing UUID fails instead of picking another
domain. Every caller that has an Instance passes it: proxy, console,
quota, post-migration autostart and the VDI URL. Migration uses the
domain the proxy already resolved. getvvfile and the gstfsd requests
use the host's current name of that UUID.

Live tests cover a name swap and a VM replaced under the same name;
both fail on the previous code.
2026-10-04 01:08:11 +03:00
catborise
ffb2ce6bfa fix(instances): destroy and volume delete no longer lose disks
Destroy:
- Stops every active state, not only running; a paused VM used to keep
  running as a transient domain after the undefine.
- Undefines with MANAGED_SAVE and SNAPSHOTS_METADATA, before any disk is
  deleted. A managed-save image used to make the undefine fail after the
  disks were already gone; now a failed undefine deletes nothing.
- Keeps disks that another domain references, directly or as a backing
  file anywhere in its chain (running domains' backingStore, and the
  backingStore of the volumes in the active pools for stopped ones), and
  says so; they used to be deleted with the VM.

Volume delete:
- Detaches from the live domain in every active state, including paused.
- Deletes the volume only after neither the live nor the persistent
  definition has the disk (bounded wait for the guest to release it),
  and refuses a volume another VM references. Otherwise the volume is
  kept and the user is told why.

Both hold the compute exclusively for the usage scan and the deletion,
so another VM cannot attach the disk in between; the destroy
confirmation page (GET) takes no lock.

Live tests reproduce each case on a RHEL qemu-kvm host and fail on the
previous code; unit tests cover the destroy ordering and the locks.
2026-10-04 01:08:11 +03:00
catborise
dbbf2ce2e8 test(instances): isolate the live libvirt suite on shared hosts
- Runs only when TEST_LIBVIRT_HOST is set, so manage.py test on a
  developer machine no longer creates VMs on the local libvirtd.
- Every VM and volume is named wvc-test-*, and volumes live in a
  temporary wvc-test pool (/var/lib/libvirt/wvc-test) instead of the
  default pool next to real images.
- Cleanup removes only prefixed domains and the test pool's volumes;
  it also clears leftovers of an interrupted run before starting, and
  tolerates domains that turned transient.
- tearDownClass fails if any domain or volume outside the test
  namespace appeared, disappeared or was replaced.
- livetest gains create_volume/define_vm helpers and
  test_live_dataloss.py, whose setUpClass opens the class transaction
  last, so a failed libvirt connection cannot lock the test database
  for later test classes.

The first live tests reproduce two known external-snapshot data losses
and are marked expectedFailure: reverting deletes a disk added after
the snapshot, and deleting the older of two snapshots deletes the
newer one's overlay.
2026-10-04 01:08:11 +03:00
catborise
fbc6c4976f fix(instances): change the edited NIC instead of adding a new one
The edit form posts net-*-<n> fields for NIC n and carries only the
new MAC. The backend read index 0 and looked the NIC up by the new MAC,
so editing a MAC detached nothing and attached a second NIC, and edits
of any NIC but the first were applied with empty values.

The form now also posts the current MAC (net-old-mac-<n>). The view
reads the submitted NIC's fields, validates the new MAC and reports
errors. The backend replaces the NIC with the old MAC, refuses an
unknown one, and on a failed attach restores the old NIC and raises
instead of reporting success.

The view crashed with a TypeError without a Referer header; it now
returns to the instance page, and a Referer from another site is
ignored.
2026-10-04 01:08:11 +03:00
catborise
9e92d208db fix(instances): flavor redirects and create-from-XML crashed
- Flavor create/update/delete fell back to the nonexistent
  instances:flavors URL; reverse() ran on every request, so each of
  them raised NoReverseMatch. Fall back to instances:index.
- Creating a VM from XML called instances.utils.refresh_instance_database,
  which does not exist; use utils.refr.

Both found by running the live libvirt suite.
2026-10-04 01:08:11 +03:00
catborise
88ec3c5f6a test(instances): make the live libvirt suite runnable on real hosts
- Pick the test VM by name; .first() could select any VM already on
  the compute, and tearDownClass deletes the chosen VM's disks.
- Delete the test volume when setUpClass fails after creating it.
- test-vm.xml: no hard-coded emulator, machine alias q35 instead of
  pc-q35-5.1, so it defines on RHEL qemu-kvm hosts too.
- Follow current behavior: power actions are POST-only, the create
  form needs add_cdrom/add_input and VNC, cloning needs change
  permission on the source VM.
- README: the tests read TEST_LIBVIRT_HOST/TYPE/LOGIN/PASSWORD;
  WEBVIRTCLOUD_TEST_LIBVIRT_URI never existed.
2026-10-04 01:08:11 +03:00
catborise
964b58778d ci: test Python 3.13, check migrations, schema, deploy and audit
- Matrix 3.10-3.13; pip cache keyed on the requirements.
- New steps: makemigrations --check, spectacular --validate
  --fail-on-warn, check --deploy --fail-level ERROR, pip-audit of the
  installed environment (the Django 4.2.30 advisories are ignored until
  the Django 5.2 upgrade), and a Docker image build.
- The separate vrtManager unittest run is dropped; manage.py test
  already runs those tests.
- CodeQL also runs on pull requests to master.
- Docker installs into a plain venv instead of using the distro's
  python3-lxml and python3-libvirt; lxml>=6.1, since the distro's
  lxml 5.2.1 has CVE-2026-41066.
- dev requirements add ruff and pip-audit.
2026-10-04 01:08:11 +03:00
catborise
b04f656d2b ci: replace super-linter with ruff
Super-linter ran from the unpinned :latest image and, because of
branches-ignore: [master], never on pull requests to master. Ruff now
checks syntax errors and pyflakes rules (E9, F) on every push and PR.

Fixes the existing findings: unused imports and variables, an unused
duplicate import in instances/tests.py, and the star import in
accounts/views.py. settings-dev.py keeps its star import of the base
settings.
2026-10-04 01:08:11 +03:00
catborise
a8725ee325 test(auth): reject unexpected failures in the authorization matrix
The matrix client swallowed exceptions, so a view that crashed counted
as "denied". Requests now raise; a 500 is accepted only when it is the
fixture's deliberate libvirt connection failure, which the exception
middleware marks in the template context. Requests carry a Referer, and
add_public_key gets a real key, so the views run their normal path.
2026-10-04 01:08:11 +03:00
catborise
51dedb7b8f fix(instances): disk edits keep their XML; changes lock per VM
Disk edits:
- edit_disk built a new <disk> from the form and replaced the old one,
  losing what the form does not show (e.g. encryption and virtio
  driver options); a bus change detached and re-attached the disk.
  The disk of the persistent definition is now edited in place and the
  domain defined once; a bus change picks a free device name and drops
  the old address and virtio-only options. Pending persistent changes of
  a running VM are kept.
- Clone omits empty disk devices, refuses a disk without a supported
  source and destination, and checks the name before allocating storage.

Locks:
- Every VM change held its compute's exclusive lock, so one slow
  operation blocked all VMs on that compute. A VM change now holds its
  own VM lock plus a shared compute lock; reconciliation, migration and
  cloning hold the compute exclusively (two clones of different VMs
  must not pass the name check together).
- Quota-checked changes of one user (CPU, memory and disk resize,
  clone) run one at a time under a per-user lock, also across computes,
  so concurrent requests cannot all pass the check against the old
  allocation. It is the outermost lock; superusers, who have no quota,
  skip it.
- A busy lock answers 409 to the API and a message in the web UI. API
  power and destroy actions return the view's error instead of success.
2026-10-04 01:08:11 +03:00
catborise
158d4db6c4 fix(storages): allow retry after an interrupted first upload chunk
The upload session was recorded only after the first chunk had been
written, and a retried first chunk was refused when its part file was
missing. The session (with its upload id) is now written before the
part file is created, the part file is created by whichever chunk finds
none, and a missing part file is an error only after the first chunk.
An interrupted first chunk can be sent again without adopting an
unrelated file.
2026-10-04 01:08:11 +03:00
catborise
6dabd69445 fix(accounts): first-login password change is stored, not in the session
The forced change of the generated admin password was a session flag
set at login when the password still matched data/admin_password. A
new session after the file was gone bypassed it.

- UserAttributes.must_change_password (migration 0007) holds it; the
  admin created with a generated password gets it, and a password
  change clears it. ForcePasswordChangeMiddleware reads it.
- A system check (accounts.E001) stops the server unless the middleware
  follows AuthenticationMiddleware; the README has the settings change.
- If the generated password cannot be stored privately, provisioning
  stops without printing it and without creating the admin (one
  transaction for the user and its attributes).
- The admin is provisioned on any real migrate run while no user
  exists, so running migrate again after a failure creates it; flush
  (plan=None) and rollbacks of accounts are skipped.
- The post_migrate receivers check the migration state for the models
  they use, so a targeted migrate on a fresh database (e.g. migrate
  appsettings) no longer fails with 'no such table'.
2026-10-04 01:08:11 +03:00
catborise
3b8869c72f fix(api): typed URLs and accurate response schemas
- Routers use path converters (use_regex_path=False): computes, instances
  and flavors take <int:...> ids, so non-numeric ids get 404 and
  drf-spectacular derives the parameter types itself.
- Each method declares what it really returns: detail serializers,
  a shared StatusSerializer for status-only replies, no request body for
  actions. Instance properties used by the details serializer get return
  type hints.
- The create-instance API drops the firmware field. Its only accepted
  value was "" (BIOS), which produced an empty, invalid enum, and the
  UEFI branch behind it was unreachable. Clients that still send it are
  unaffected; DRF ignores unknown fields.

The schema now passes spectacular --validate --fail-on-warn; a test
keeps it that way.
2026-10-04 01:08:11 +03:00
catborise
61208606f7 fix: render the 404 page for logged-in users
app_active and view_active read request.resolver_match, which is None
when no URL matched, so the navbar raised AttributeError and the 404
became a 500.
2026-10-04 01:08:11 +03:00
catborise
fd5aa069be fix: track migrations; installs apply them instead of generating them
The *migrations/* ignore rule is removed. appsettings, computes and
logs pin default_auto_field to AutoField, so makemigrations --check
reports no pending BigAutoField migrations and existing installs need
no schema change.

The Vagrant provisioning no longer runs makemigrations, which created
install-specific migration files that conflict with the committed
history; it only runs migrate.
2026-10-04 01:08:11 +03:00
catborise
a6364f5d9b fix(gstfsd): append SSH keys, IPv6 loopback bind, server-side timeout
- Add the public key to authorized_keys instead of replacing the file.
- Use AF_INET6 when GSTFSD_BIND_HOST is an IPv6 address (binding ::1
  crashed with the AF_INET server).
- Close idle client connections after 30 s.

Tests run the real daemon against a small fake guestfs module over a
socket; this also exercises the earlier sendall() fix.
2026-10-03 00:02:01 +03:00
catborise
32d7f279e7 test: inventory HTML sinks in templates, realistic datasource params in the matrix
- instances/test_xss.py: scan all templates for .html(, innerHTML = and
  append('<...') and require each line to be in a reviewed allowlist, so
  new sinks fail the test (the exact-string tests only caught the lines
  they named).
- Authorization matrix: fill the datasource "version" parameter with
  "latest" instead of "x". The datasource views then answer 200 to any
  logged-in user; they are listed in the allowlist as known F-14.
2026-10-03 00:00:27 +03:00
catborise
479faaa4ca fix(instances): allow editing disks that have no driver format
A disk without driver/@type could not be edited: the empty format was
rejected, and edit_disk would have written type=''. An empty format is
now accepted and edit_disk leaves the type attribute out.
2026-10-02 23:58:55 +03:00