The build/tests/functional subdirectories are consuming huge amounts
of disk space.
Split the location for scratch files into a 'scratch' sub-directory,
separate from log files, and delete it upon completion of each test.
The new env variable QEMU_TEST_KEEP_SCRATCH can be set to preserve
this scratch dir for debugging access if required.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20241121154218.1423005-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20241121165806.476008-3-alex.bennee@linaro.org>
* Various fixes and improvements for the functional tests
* Refresh CI container files with the latest changes from libvirt-ci
* Clean up keyboard code of the next-cube machine to get rid of a legacy API
# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmct48QRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbUAzw//fOgAe+rNX10KIEWaHzTlw6EPRUHbKEo+
# yvy9mZ4CYiK3PfojuW6u9Mt32RIGFjLRUPC+ljZggjbxY2DUjzAUGx/kSwf0bzwE
# 8eRKRvd+l3ZFKklwDd7YQh+/O38P/uAq+T0AZQv/9h7rr0wPD6O+sVpKjVN4akuh
# 66Ekz974bDFaVJi8iIix0WCPCUFMZwWQqxaQrnoBH3wN8Ibc6Ermchs7aa1e6KNh
# DoX519JVNofXbugONNnqEjp43tbx/hzPZFyWf9tyX8ehp6+8po0zcZZwqdRkWhuv
# BkESKPz8rHuQOGMjm8rAnjmDEsbpJ+vOfn/ShdyAERv78mTNWwSvYVwq5zfpcTCt
# 9gpCNhhFzFLKlbDeIfgy9ky4R8Gzrww3icLzCsLlat6YCK547YkSr57BsYXJzLX4
# /NUYsaJ1RPdutwgNnmSzhBhObtUik3rFYH5aD0ETWZBdVY8ig5OqGpJ29P5zo860
# ROVFn4RX/XpmODOpYez4dwKVetR5SLHNG1aqduWKOwhAxPzLxK5OghjYkGf0bsel
# gA9t7A1TafgneYxJARKi7esD39xeUzhfzdzCOIOLXJga5kD2/bvZnQ4tkoLPl8QR
# iQEp1z758XmSFTYfzGUDA7SgIOzi9JG5dqtzcqJklr9wYurxwMYmqicJqA4ImoQt
# fZV/axe8Em0=
# =YRCB
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 08 Nov 2024 10:11:16 GMT
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* tag 'pull-request-2024-11-08' of https://gitlab.com/thuth/qemu:
ui/input-legacy.c: remove unused legacy qemu_add_kbd_event_handler() function
next-kbd: convert to use qemu_input_handler_register()
tests: refresh package lists with latest libvirt-ci
tests/functional: Split the test_aarch64_sbsaref test
tests/functional: Bump timeouts of functional tests
tests/functional: Provide the user with hints where to find more log files
tests/functional: Fix the ppc64_hv and the ppc_40p test for read-only assets
test/functional: Fix Aspeed buildroot tests
tests/functional: Convert the riscv_opensbi avocado test into a standalone test
scripts/checkpatch.pl: Ignore ObjC #import lines for operator spacing
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The test_aarch64_sbsaref test is the test with the longest runtime
in our functional test suite. Split it into parts so that it can
be run on multiple CPUs in parallel.
For this we have to move the fetch_firmware() function out of the
class definition to be able to reuse it easily from the other tests
(deriving the Aarch64SbsarefAlpine and Aarch64SbsarefFreeBSD directly
from Aarch64SbsarefMachine does not work, unfortunately, since we'd
inherit the test_sbsaref_edk2_firmware() function that way, causing
it to be run multiple times - and keeping the fetch_firmware() in
a separate class without the test_sbsaref_edk2_firmware() function
also does not work since the "make precache-functional" won't work
in that case ==> turning fetch_firmware() into a static function is
the best option).
Message-ID: <20241106175029.1000589-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When building QEMU with "--enable-debug" and running the tests
in parallel with "make -j$(nproc) check-functional", many tests are
still timing out due to our conservative timeout settings. Bump
the timeouts of the problematic tests and also increase the default
timeout to 90 seconds (from 60 seconds) to be on the safe side.
Message-ID: <20241106170946.990731-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since the base.log and console.log files are not referenced from the
meson test logs yet, they might be hard to find for the casual users.
Thus let's print some hints in case a test case failed. For this we
have to run unittest.main() with exit=False to get the results of the
testing. Then we can iterate through the failed test cases to print
out the information accordingly.
Message-ID: <20241105123849.359391-1-thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Tested-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The avocado test defined test functions for both, riscv32 and riscv64.
Since we can run the whole file with multiple targets in the new
framework, we can now consolidate the functions so we have to only
define one function per machine now.
Message-ID: <20240821082748.65853-23-thuth@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Merge test 2024/11/07 v2
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEuBi5yt+QicLVzsZrda1lgCoLQhEFAmcs1YAACgkQda1lgCoL
# QhFPbAgA71gnxNjOEIlDwTbFPkS8mj4mtRryp96UN6v+XYEB0RL2uSEwcCJfQ+Cd
# Xbo0/g1GKNFtf9fsn16M2uxPh8VlcwBF7diboN5sGNVRJGgshD5ynedW/9T3Vyds
# 1elahNl/FjqIgcBo9GbAVumcC0TY/8w+BwKG1HQaP0TWSVQQzUfxz3wiVFLJh7Oe
# oxUTvyz0gWFKAqX5rTptWRitfew/xA7LimP0gdl1pVp3mkBS8KiPUdqx+LQ/OsFe
# Gn4JybDhNYdyf8VbcQK5LQKAGVcLLGDtxp5DNdu3fk+LFkqYy4V0AyxxyyU70/OY
# j9+H7nJTlf8pf5s5UrhYADyIvlW+LQ==
# =lGlS
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 07 Nov 2024 14:58:08 GMT
# gpg: using RSA key B818B9CADF9089C2D5CEC66B75AD65802A0B4211
# gpg: Good signature from "Stefan Berger <stefanb@linux.vnet.ibm.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: B818 B9CA DF90 89C2 D5CE C66B 75AD 6580 2A0B 4211
* tag 'pull-tpm-2024-11-07-2' of https://github.com/stefanberger/qemu-tpm:
tests: Adjust path for swtpm state to use path under /var/tmp/
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Move the OrangePi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for OrangePi tests in the functional framework
and update the hash sums of the assets to sha256 along the way.
For the buildroot image and the Armbian image, we've got to switch to
a newer version since the old images have been removed from the server,
and the NetBSD image has been moved to the archive, so we need to update
this URL as well.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-3-thuth@redhat.com>
Move the BananaPi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for Banana Pi tests in the functional framework.
Update the hash sums of the assets to sha256 along the way and fix the
broken link for the buildroot image from storage.kernelci.org.
(Note: The test_arm_bpim2u_openwrt_22_03_3 test is currently broken
due to a regression in commit 4c2c047469 ("target/arm: Fix usage of MMU
indexes when EL3 is AArch32") - it works if that commit gets reverted)
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-2-thuth@redhat.com>
A straight forward conversion, only the usual changes were required
here (i.e. adjustment for asset downloading, machine selection).
Message-ID: <20241023051754.813412-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
I forgot to add the tests to the meson.build file and looks
like I even managed to somehow mix up the hashsums in the
ppc64 test!
Message-ID: <20241023141919.930689-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Now that we are aware of binaries that are available for sh4eb,
we should make sure that there are no regressions with this
target and test it regularly in our CI.
Message-ID: <20241024082735.42324-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This ensures that if a functional test runs QEMU with a writable
disk pointing to a cached asset, an error will be reported, rather
than silently modifying the cache file.
As an example, tweaking test_sbsaref.py to set snapshot=off,
results in a clear error:
Command: ./build/qemu-system-aarch64 ...snip... -drive file=/var/home/berrange/.cache/qemu/download/44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461,format=raw,snapshot=off -cpu max,pauth=off
Output: qemu-system-aarch64: Could not open '/var/home/berrange/.cache/qemu/download/44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461': Permission denied
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241025092659.2312118-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The zstd command will preserve the input archive permissions on the
output file. So when we decompress the readonly cached image, the
resulting per-test run private disk image will also be readonly.
We need it to be writable, so make it so.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241025092659.2312118-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a functional test for the sx1 board that uses the kernel and
rootfs provided by Guenter Roeck in the linux-test-downloads repo:
https://github.com/groeck/linux-test-downloads/
We have three variants of the test for this board:
* just boot initrd
* boot with filesystem on SD card
* boot from flash
In all cases these images have a userspace that is configured to
immediately reboot the system on successful boot, and the board
itself supports telling QEMU to do the reboot, so we only need to
wait for QEMU to exit (via -no-reboot).
Since there are three subtests, the test as a whole takes about
80s on my local machine. That's about the same as the aarch64_virt
test, so give it the same overall test timeout as that one.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20241017163247.711244-3-peter.maydell@linaro.org
This is a simple conversion of the tests with some cleanups and
adjustments to match the new test framework. Replace the zephyr image
MD5 hashes with SHA256 hashes while at it.
The SDK tests depend on a ssh class from avocado.utils which is
difficult to replace. To be addressed separately.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Add a base class for the TuxRun tests, based on the code from
tests/avocado/tuxrun_baselines.py (the test have to be put into
separate file in the following commits, depending on the target
architecture that gets tested).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-2-thuth@redhat.com>
These tests occasionally time out when the host system is under heavy
load. Increase the timeout setting to allow for more headroom here.
Message-ID: <20241002140212.350467-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The _check() function is supposed to check whether the hash of the
downloaded file matches the expected one. Unfortunately, during the
last rework of this function, the check was accidentally turned into
returning the hash value itself instead of a True/False value,
effectively accepting each hash as valid. Let's do a proper check
again now.
Fixes:05e303210d ("tests/functional/qemu_test: Use Python hashlib ...")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
