CI once found a test worker still running 15 s after SIGTERM, and the
failure said nothing more. The worker children now dump their threads'
stacks on SIGUSR1, and tearDown sends it to a child that is still alive
before killing it, so the next such failure names the place.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pool workers were made non-daemonic so the operations process
tests could start workers, but a terminated pool (Ctrl-C, --failfast)
left those workers running. The test runner now keeps the test cases
tagged "processes" out of the pool and runs them in the main process
after it. A WAL that cannot be written back before cloning now stops
the run instead of giving clones without tables. One CI leg runs the
suite with --parallel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
libvirt's backup copies the disks of a running or paused VM from one
instant into new volumes while it keeps running; the clone is defined
shut off as before. The VM is held for the copy, the copy can be
cancelled, and recovery reads the job's records and libvirt.
Request jobs (an upload's chunks) run in gunicorn: their kind is admitted
running with the request as its worker, never claimed or recovered by the
worker, cancelled in the request, and asked by every housekeeping pass to
end its jobs that went idle. The panel can follow a job the page just
started. A job a request runs does not wait for the worker, so it does not
raise the worker-unavailable warning on the Jobs page or in the panel.
Only the request middleware loaded the Settings page values, so in the
worker process app_settings stayed empty: a VM create job failed reading
the default volume format, and the snapshot free-space check fell back to
its defaults. The worker now loads them before each job.
The list and the detail page were rendered once, so a job that finished
kept showing running until the page was reloaded by hand. While a shown
job is queued, running or cancelling, the page now reloads every five
seconds (not while hidden or while a form field has focus).
