The CPU bar showed the average since the host booted, read for every
compute on every load of the VM list. The dialog now reads each compute's
graph endpoint (load over one second) when it opens, two computes at a
time so the reads hold at most two workers, and ignores answers of an
earlier opening. A host that cannot be read shows a dash. The since-boot
reading had no other caller and is removed.
Each poll of a host's overview graph held a worker for one second between
two CPU samples. The answer now carries its sample, signed with the time
and the compute; the next poll sends it back and the usage is taken since
then, so a point is the average since the previous one (normally 5 s, at
most 15 s). Without a sample, or with one that is altered, older or from
another host, or when the host restarted, it is measured over one second
as before.
A sample is one reading; the busy share was mixed from separate readings,
counted guest time twice (it is part of user) and added half a percent.
A host that cannot be read leaves a gap in both graphs instead of a zero
CPU point and a broken memory axis.
The first appsettings migrations insert the default rows with fixed ids,
which a PostgreSQL id sequence does not see; the login lockout migration,
the first to insert without an id, then collided with id 1. It now moves
the sequence past the existing rows first (a no-op on sqlite), which also
covers a database upgraded from any earlier migration. A test compute
password exceeded its 14-character column, which only PostgreSQL enforces.
A compute's hostname and login go unquoted into the libvirt connection URI
(qemu+ssh://login@host/system, qemu+tcp://host/system, qemu+tls://...) and,
for an SSH compute, into the ssh command line of the DRBD status view. They
were barely validated: validate_hostname used re.match with a negated class,
so it caught only a leading special character, and login was never checked.
A hostname like "192.0.2.1/system?command=/path" injected a libvirt URI
parameter (verified: connecting ran the named command through the ssh
wrapper), and TLS/SSH parameters such as no_verify, pkipath, socket or
netcat could be added the same way; a login like "-oProxyCommand=..." became
an ssh option. The DRF ComputeSerializer applied neither check, and
validate_name tested a string literal instead of its argument, so it never
raised.
validate_hostname now accepts only an IPv4 address or a host name with
fullmatch (no scheme, port, path, query, whitespace or IPv6 literal; IPv6 is
left out until the URI building brackets it). validate_login accepts an empty
value (TCP falls back to the server's authentication, the socket transport
ignores the login) or a plain name with no leading "-". Both sit on the
Compute model fields, so the forms and the DRF serializer (which copies model
validators onto its fields) reject the values; a migration records the
metadata. validate_name checks its argument. drbd_status validates the login
and hostname before building login@host and ends ssh option parsing with
"--"; an existing or bypassed bad row is skipped rather than run.
Tests: hostname/login validators accept real values and reject URI and
option injection; the SSH form and the serializer (create, and an empty TCP
login) enforce them; drbd_status skips an injecting target. Checked the real
test computes still validate. The new tests fail on the previous code.
What migrate_instance did around the migration itself:
- It checked the destination with a 1 s TCP probe of the libvirt port,
which passes while libvirt is down or refuses the login. It now uses
the compute's connection status, as the rest of the app does.
- Two requests for the same VM waited for the compute locks one after
the other; the second then migrated with the VM's old location. The
VM's record is read again under the locks, and the request is
refused when the VM is no longer on the source.
- Restoring autostart on the destination came after the migration but
before the database move; when it failed, the VM ran on the
destination while the app recorded it on the source. Autostart is
now read from libvirt and a failure to restore it is only logged.
- A database failure after a successful migration was logged as
nothing; the log now says the VM migrated and recording it failed.
- When the destination's list had already made a record for the VM,
the merge kept the owners but dropped the template mark.
moveto read the VM state for each flag and dropped options that did
not fit it, so a request could run as something else than asked; the
log then named the mode from the form ("live" or "offline"), not the
one used. The API did not log at all and answered "instance migrate is
started" after the migration had finished.
- moveto reads the state once. A running VM migrates live, or without
live paused for the transfer (non-live); a paused VM stays paused; a
shut-off VM moves offline. Live for a shut-off VM, offline for a
running or paused one, and any other state are refused before
anything changes. unsafe, auto converge (live only) and compressed
apply to a running VM.
- migrate_instance logs the outcome for the VM page and the API: the
mode and options used, or the error. The API answers
"instance is migrated (<mode>)" and accepts requests that leave out
an optional option.
- Post copy is gone from the form, the view and the API: only the
flag was set, and without migrateStartPostCopy, which the app never
called, the migration never switched to post copy.
Migration is client-driven: libvirt sends the VM's memory to the
destination's own host name unless a migration URI is given, and the
app never gave one. Where the hosts cannot resolve each other's names
(or the names point at a management network) every migration failed
with "Unable to resolve address '<host name>' service '49152'".
A compute now has an optional migration address (an IP address or a
host name, no scheme or port), editable in the compute form and the
API. A VM migrating to that compute is sent to tcp://<address>
(bracketed for IPv6). Empty, the default for existing computes, keeps
the previous behavior. The compute's host name is not reused, because
for SSH computes it can be an ssh_config alias.
The migration log entry names the address when one is set.
The TCP and TLS compute forms never show the stored password, so the
field is empty on every edit. Saving the form without typing the
password again replaced it with nothing, and the next connection to the
host failed to authenticate.
An empty password field on an edit now keeps the stored password; a
new value replaces it.
Nothing limited failed logins: a password could be guessed at the speed
the server answers. django-axes (8.3.1, Django 5.2, Python 3.10-3.13)
now counts them.
- A user is locked for the address the failed logins come from
(username and IP together), so an attacker cannot lock someone out
from elsewhere and users behind one NAT address do not share a lock.
A successful login resets the count. axes sits in front of every
authentication: the login page with or without OTP, LDAP and the API.
- The number of failed logins (default 5) and the lock time in minutes
(default 15) are on the Settings page (LOGIN_FAILURE_LIMIT,
LOGIN_LOCKOUT_MINUTES, added by migration appsettings.0014) and apply
at once; an invalid value falls back to the default. Unlock with
"manage.py axes_reset_username <username>".
- Behind the bundled nginx every request comes from 127.0.0.1; the
client address is read from X-Real-IP, which nginx sets, but only for
requests from a loopback address or from LOGIN_TRUSTED_PROXIES, so a
forged header does not escape a lock.
settings.py.template gets the app, middleware, backend and AXES_*
settings; README tells existing installations what to copy. Tests that
call client.login now pass a request, which the axes backend needs.
The new tests fail with axes disabled.
The instance lists, the VM page header and the power tab knew three
libvirt states: running, paused and shut off. A VM that was blocked,
shutting down, crashed, suspended by its guest or without a state got
an empty status cell and no actions, so a crashed VM could not even be
forced off from the list. The lists also offered actions users could
not use: Suspend and Resume (views for superusers) to everyone, and
power actions to users who see every VM through the read-only
view_instances permission.
instances/states.py holds the rules in one place: the label and color
of each state, the actions each state allows, and which of them a user
may use (superusers all; a VM's owners all but suspend and resume, and
nothing on a VM an administrator suspended, as the VM page already
showed; nobody else any). Blocked is a running VM; shutting down,
crashed and suspended by the guest get Force Off, after which the VM
can be started. A shut-off template offers Clone to whoever has the
clone permission, as the clone view allows.
The templates read them through two tags (vm_states): the lists and the
VM page header show the state, instance_actions.html has one slot per
button, each enabled or disabled from the allowed actions. The power
tab offers Force Off in the states that need it.
The new tests check the rules directly and render every state in the
lists and on the VM page for a superuser, an owner and a
view_instances user; they fail on the previous code.
The VM page polls the VM state every 5 s and, while its Stats tab is
open, the statistics every 10 s; the compute overview polls the host
graph every 5 s. All used setInterval:
- A new request went out on schedule while the previous one was still
open. When a host goes silent, a stats request waits up to 30-45 s,
so each open page held several sync workers at once.
- Each time the Stats tab was shown, the page made new charts and a
new interval without stopping the old ones, and polling went on on
the other tabs of the page.
- Hidden pages kept polling at full rate.
Measured in a browser against a test server (old code / new): Stats tab
shown 4 times, 31 s on it: 12 / 2 requests; 31 s on another tab of the
page: 12 / 0; page hidden for 31 s: 15 / 0, and the new code asks once
right after the page is shown again; with 25 s answers, at most 15 / 1
requests open at once.
static/js/poll.js repeats a request while the page is visible: the
next one 5 or 10 s after the previous answer, none while hidden, one at
once when shown again. The Stats tab makes its charts once and polls
only while shown. The answer handler runs after the poller's own
bookkeeping, so an error in it (a chart for a disk added meanwhile)
does not stop the polling. A Node test of poll.js runs with the test
suite when node is installed.
check_user_quota added up the vCPUs, memory and disks of the user's
VMs, but skipped a VM whose host failed a TCP probe. A host that was
down therefore let users exceed their quota, and the probe always failed
for ssh computes named by an ssh_config alias or reached through
ProxyJump, so their VMs were never counted. A VM that could not be read
turned the page into the error page instead.
- Hosts are checked with Compute.status, the connection WebVirtCloud
uses, once per host. A VM that cannot be read (host down, VM gone)
makes the usage unknown: clone and resize up are refused with their
own message (fail closed).
- A request that adds nothing is not checked, so a user over quota can
still shrink a VM. Users without limits, and an exceeded instance
count (from the database), need no host either.
- Memory resize of a running VM changes only the current memory, but
the quota was charged with the submitted maximum. It is now charged
with no change, and resize_mem gets the current maximum, so a VM that
shuts down meanwhile cannot take an unchecked one.
- The VM page no longer computes the quota on every load (it read every
VM of the user on every host for a "quota reached" banner). The limit
is reported when a change is refused.
The test that expected a down host to pass the quota now expects it to
be refused. Checked against the test hosts: with a VM behind an
ssh_config alias the previous code counted 29 vCPUs, the new one 31;
with a VM on an unreachable host the previous code allowed +1 vCPU,
the new one refuses it after 5.1 s and allows -1 at once. The new
tests fail on the previous code, except the ones guarding unchanged
behaviour.
The overview page decided whether the host is up with a TCP probe of
the compute's hostname. For an ssh compute named by an ssh_config alias
or reached through ProxyJump, that name and port cannot be reached from
WebVirtCloud directly, so the probe failed while the connection worked:
the page showed the host details and never started the CPU and memory
graphs. The overview now uses Compute.status, the connection
WebVirtCloud uses for everything else.
The compute list shows why a host is not connected (Compute.connection_error)
under "Not Connected", for example a host key verification failure.
Checked against the ssh compute of the test setup through an
ssh_config alias: the previous code renders the overview with the
graphs off, the new one with them on. Both new tests fail on the
previous code.
The instance lists read each VM from libvirt while rendering. When that
failed, the libvirt error replaced the whole page with the error page:
one VM removed out of band, or a host that is down, took the index or
a compute's instances page with it. To avoid that, both index views
left out every VM of a host that was not connected, so users saw their
VMs disappear.
- Instance.info is None when the VM cannot be read. The row then shows
the name and owners from the database, the state "Unknown", and no
title, vCPU, memory or actions.
- Both index views list the VMs of a host that is down. The grouped
view marks the host "Not Connected" and leaves out its CPU and memory.
- A compute's instances page shows the connection error, and no longer
says the hypervisor has no instances when it cannot be reached.
Checked against the test hosts with an unreachable compute added: both
index views list its VM as unknown next to the real VMs. The first
load takes 5.5 s (one bounded connection attempt), the next ones
0.1 s. The new tests fail on the previous code (VMs left out, error
pages).
refresh_instance_database opened the transaction first and then asked
libvirt for the domain list. A host that goes silent on an established
connection keeps that call waiting until keepalive gives up (30 s). On
SQLite, the default database, the open transaction holds a read lock,
so every other write of the app failed meanwhile with "database is
locked", logins (session writes) included. Reproduced in the test
setup: with a relay to a TCP compute frozen after connecting, three
session writes during the 30 s refresh each failed after 5 s.
- The connection check and listAllDomains run before the transaction,
still under the compute lock, so create, delete and migration of the
compute stay excluded as before.
- The transaction only applies the changes. It checks the compute row
first: a compute deleted while libvirt was asked gets no instances
(this ended in a foreign key error before).
A failed listing still leaves the database untouched. With the change,
the same reproduction writes the sessions at once and keeps the
instances. Both new tests fail on the previous code.
The VM rows of the instance lists used instance.proxy.instance.info in
every state check, the vCPU column and the action buttons. A template
calls the method on each lookup, so every row made 7 virDomainGetInfo
RPCs (8 on a compute's instances page), and the memory column read the
domain XML once more.
- Instance.info reads virDomainGetInfo once per request; the lists and
instance_actions.html use it.
- The memory column of both index views comes from the same info
(current memory) instead of the domain XML. The value is the same:
checked against currentMemory for every running and shut-off VM of
the test hosts.
A row now makes one info call and one XML read (the title). The new
test counts the calls in the grouped and non-grouped index and on the
compute instances page; it fails on the previous code with 7 and 8
calls.
libvirt has no timeout for opening a connection. Measured in the test
setup: an unreachable host blocked qemu+tcp for 135 s and qemu+ssh for
133 s; a daemon that accepts and never answers blocks forever. Gunicorn
runs sync workers and the instance list connects to every compute, so
one unreachable compute held a worker for minutes per page load.
- A connection is opened in a helper thread, at most one per
connection at a time. The request waits LIBVIRT_CONNECT_TIMEOUT
(default 5 s). An open nobody waits for any more keeps running; until
it ends, requests get its error at once, and if it succeeds later the
connection is used. A failed open is not repeated for
LIBVIRT_RETRY_AFTER (default 30 s). A stuck host holds one thread and
does not keep other hosts from connecting.
- qemu+ssh runs ssh through vrtManager/libvirt-ssh (the URI's command
parameter), which adds ConnectTimeout=10 and BatchMode=yes, so an
abandoned ssh process ends and no prompt can hang. The user's ssh
configuration (aliases, ProxyJump, keys) still applies.
- The close callback acts only for the connection it was registered
for: a late callback of a replaced connection dropped its successor.
The reason is reported as text. A connection that closed before it
was published is an error.
- Compute.status is a bool (it returned the exception object, so the
"host is down" check in refresh_instance_database never fired); the
message is Compute.connection_error.
Unit tests cover the timeout, retry delay, late success, single open,
a stuck host next to a healthy one and the close callback. Live tests
(vrtManager/test_live_connection.py), for TCP and SSH computes of the
test setup: an unreachable address (no ssh left after 10 s), an endpoint
that never answers, a slow open completing later, a host that goes
silent on an established connection (keepalive ends calls within 45 s,
the next request reconnects), an ssh_config alias and a ProxyJump host.
computes.views.get_compute_video_models had no URL and no caller, and
ComputeManager.get_video_models was used only by it. The video models
of the create page and the VM settings come from wvmConnect directly.
Comments, docstrings and live test names cited item IDs of a roadmap
that is not part of the repository. They now say what is meant, or
nothing where the sentence stands without them. Test assertions and
behavior are unchanged.
- host_is_up returned the exception object for a down host, which is
truthy: check_user_quota then connected to it and failed, so a user
with a VM on a down host could not open its page, clone or resize,
and migrate_instance's "target is down" check never fired. It returns
True or False now, closes its socket, and accepts IPv6 and host:port.
- The connection manager no longer holds its global lock while a new
connection is opened, so one slow host does not delay new connections
to the others.
Bounding the connection time itself (including SSH aliases and jump
hosts) is left for a separate change.
The sync deleted the Instance of every VM missing from its compute, and
its owners and rights went with it. A VM moved with virsh, or undefined
and defined again, came back with no owner.
Now, when a VM disappears from a compute:
- if the same UUID is recorded on another compute, it moved: its owners
and template flag go there;
- otherwise they are kept in an InstanceTombstone and restored when the
UUID appears again within INSTANCE_OWNERSHIP_RETENTION_DAYS (30).
A UUID recorded on two computes at the same time is a copy and gets no
owners. VMs deleted through WebVirtCloud leave no tombstone.
Super-linter ran from the unpinned :latest image and, because of
branches-ignore: [master], never on pull requests to master. Ruff now
checks syntax errors and pyflakes rules (E9, F) on every push and PR.
Fixes the existing findings: unused imports and variables, an unused
duplicate import in instances/tests.py, and the star import in
accounts/views.py. settings-dev.py keeps its star import of the base
settings.
Disk edits:
- edit_disk built a new <disk> from the form and replaced the old one,
losing what the form does not show (e.g. encryption and virtio
driver options); a bus change detached and re-attached the disk.
The disk of the persistent definition is now edited in place and the
domain defined once; a bus change picks a free device name and drops
the old address and virtio-only options. Pending persistent changes of
a running VM are kept.
- Clone omits empty disk devices, refuses a disk without a supported
source and destination, and checks the name before allocating storage.
Locks:
- Every VM change held its compute's exclusive lock, so one slow
operation blocked all VMs on that compute. A VM change now holds its
own VM lock plus a shared compute lock; reconciliation, migration and
cloning hold the compute exclusively (two clones of different VMs
must not pass the name check together).
- Quota-checked changes of one user (CPU, memory and disk resize,
clone) run one at a time under a per-user lock, also across computes,
so concurrent requests cannot all pass the check against the old
allocation. It is the outermost lock; superusers, who have no quota,
skip it.
- A busy lock answers 409 to the API and a message in the web UI. API
power and destroy actions return the view's error instead of success.
- Routers use path converters (use_regex_path=False): computes, instances
and flavors take <int:...> ids, so non-numeric ids get 404 and
drf-spectacular derives the parameter types itself.
- Each method declares what it really returns: detail serializers,
a shared StatusSerializer for status-only replies, no request body for
actions. Instance properties used by the details serializer get return
type hints.
- The create-instance API drops the firmware field. Its only accepted
value was "" (BIOS), which produced an empty, invalid enum, and the
UEFI branch behind it was unreachable. Clients that still send it are
unaffected; DRF ignores unknown fields.
The schema now passes spectacular --validate --fail-on-warn; a test
keeps it that way.
The *migrations/* ignore rule is removed. appsettings, computes and
logs pin default_auto_field to AutoField, so makemigrations --check
reports no pending BigAutoField migrations and existing installs need
no schema change.
The Vagrant provisioning no longer runs makemigrations, which created
install-specific migration files that conflict with the committed
history; it only runs migrate.
compute_graph (text/javascript) and the machine type, disk bus, video
model and domain capability views (text/html) return JSON; send it as
application/json. create_instance_w1.html builds the chipset <option>
elements with .val().text() instead of string concatenation.
Add webvirtcloud/test_authorization_matrix.py, which walks every named URL
and checks that a user without any VM/compute relation only reaches an
explicit allowlist, and that the global view_instances permission does not
open consoles. It found:
- /computes/<id>/... views without any check (compute_graph,
get_compute_*, get_dom_capabilities, storages.get_volumes): the whole
computes URL namespace is now wrapped with superuser_only, so a view
added there later cannot be exposed by a missing decorator.
- guess_mac_address (leaks dhcpd.conf host/MAC pairs), random_mac_address
and guess_clone_name: superuser-only, like the buttons that use them.
- check_instance: cross-tenant VM name lookup with no caller; removed.
- getvvfile and get_vdi_url: reachable with only view_instances, and the
.vv file contains the VNC password.
Console access now follows one rule, instances.utils.can_open_console:
an active superuser or an active owner of the VM. The global
view_instances permission is read-only. It is used by the console page,
novncd, getvvfile and get_vdi_url (getvvfile and the VDI tab are kept for
virt-viewer users). The console page answers 403 instead of 500 when
access is denied, with the same answer for a missing VM.
- Computes API: mark password as write_only in ComputeSerializer, restrict ComputeViewSet and architectures to IsAdminUser.
- Instances API: enforce user ownership tenant isolation in InstanceViewSet.list, restrict MigrateViewSet and CreateInstanceViewSet to IsAdminUser, require IsAdminUser for mutating FlavorViewSet.
- Storage, Network, and Interface APIs: restrict StorageViewSet, VolumeViewSet, NetworkViewSet, InterfaceViewSet to IsAdminUser.
- AppSettings & NWFilters: protect appsettings and nwfilter views with @superuser_only (403 Forbidden for non-superusers).
- noVNC Console: sign tokens with TimestampSigner (salt='console.novnc', max_age=600), verify signature and user authorization in novncd with backward-compatible fallback for debug/test mode.
- Accounts: avoid hardcoded default admin password; generate cryptographically secure random password unless ADMIN_PASSWORD is set (preserve 'admin' in test runner).
- gstfsd daemon: bind to 127.0.0.1 by default instead of 0.0.0.0.
- Tests: added comprehensive test coverage for API security, token expiration/rejection, and view permissions (132 tests passing).
* Add rest framework for API: First Commit
* modify some shell scripts to make variable references safer; modify some python scripts to reduce the code complexity and cyclomatic complexity of functions.
* Add REST API for some webvirtcloud functions. Instance list/delete/create, compute list/delete/create, storages-network list/retrieve. Add swagger and redoc for API interface
* update requirements
Co-authored-by: herengui <herengui@uniontech.com>
