Block patches:
- Some changes for qcow2's refcount repair algorithm to make it work for
qcow2 images stored on block devices
- Skip test cases that require zstd when support for it is missing
- Some refactoring in the iotests' meson.build
# -----BEGIN PGP SIGNATURE-----
#
# iQJGBAABCAAwFiEEy2LXoO44KeRfAE00ofpA0JgBnN8FAmJf/asSHGhyZWl0ekBy
# ZWRoYXQuY29tAAoJEKH6QNCYAZzfYXUQAKQv5qKQBjU4MTwlS8A4h6B6OJgC1Sik
# 9BB7LO/QFjuuF4vNKpcUlf6i0epxPP8B5pmCjaAolMh6u6wZwL7hHq+SOYXvejTo
# vINW+r097U0qYPkSV+cS6tbW92rYJDD7VxF+34udiWXGjozsBTw/k9DfJaa9Ht66
# 2dw3AxUa4lxN1/ejFzDLx3DNaff+HctLhgVpHeBb0eN2zr2Ug5+ZFgMoiWwU6r6J
# EzTORLAzATerlQVYUkhh4Y/UdVLLw1SzTWOQv5b/NqvaLfKmYsQobSfjC2ajO8XJ
# P2REigcOAij5uWVRf4EY7xoqmADP8pXxuOTzw0hyGNLOLNcXoFbfW45WSPoY+YgH
# EH1TtC4vMsg/MlO/A3PJr9v+SNqxz32cul3MVrY3PuG4Dzz0riy9GhtFUU37igbj
# mR6pP3nSa/f2X4+9B6/UrPjLzusRvc8bvzYqVEnSLABav11npphkYaR9QT1fQUVD
# Zw26igXtmLKUcfop/EqShbhblk0ZLYDTj/Lx7X+thC9OCrK1QgF6qAsIUqiS1iHz
# vwdktRTCofo4ZIT/OCz5QeriJqDz0B7VJ8/4i/uvm2eq8BUsn2mJuyAGD2XtaONV
# rmASrV9VbajdxX5VptjKOOHG6aHtqQlKbyBFog8I4nqVFdjdSMalb++gBMCrPu1A
# 1iZPsOOyz/8+
# =BF0c
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 20 Apr 2022 05:33:47 AM PDT
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF
* tag 'pull-block-2022-04-20' of https://gitlab.com/hreitz/qemu:
qcow2: Add errp to rebuild_refcount_structure()
iotests/108: Test new refcount rebuild algorithm
qcow2: Improve refcount structure rebuilding
iotests/303: Check for zstd support
iotests/065: Check for zstd support
iotests.py: Add supports_qcow2_zstd_compression()
tests/qemu-iotests: Move the bash and sanitizer checks to meson.build
tests/qemu-iotests/meson.build: Improve the indentation
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
There are a number of GDB's on various distros which fail fairly hard
when attempting to talk to a cross-arch guest. The previous attempt to
catch this was incorrect as the shell will deliver signals as 128+n.
Fix the detection and while we are it improve the logging we dump into
the test output.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Gautam Agrawal <gautamnagrawal@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-26-alex.bennee@linaro.org>
Build the "docker.py cc" invocation directly in tests/tcg/configure.sh, and
remove the Makefile.qemu wrapper around Makefile.target. The config-*.mak
files now include the actual variables used when building the tests, rather
than the CROSS_* variables that Makefile.qemu used to "translate".
This is a first step towards generalizing the cross-compilation infrastructure
so that it can be used for firmware as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220401141326.1244422-15-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-18-alex.bennee@linaro.org>
The Fedora 29 kernel is quite old and importantly fails when running
in LPA2 scenarios. As it's not really exercising much of the CPU space
replace it with a custom 5.16.12 kernel with all the architecture
options turned on. There is a minimal buildroot initramfs included in
the kernel which has a few tools for stress testing the memory
subsystem. The userspace also targets the Neoverse N1 processor so
would fail with a v8.0 cpu like cortex-a53.
While we are at it move the test into its own file so it can have an
assigned maintainer.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-2-alex.bennee@linaro.org>
One clear problem with how qcow2's refcount structure rebuild algorithm
used to be before "qcow2: Improve refcount structure rebuilding" was
that it is prone to failure for qcow2 images on block devices: There is
generally unused space after the actual image, and if that exceeds what
one refblock covers, the old algorithm would invariably write the
reftable past the block device's end, which cannot work. The new
algorithm does not have this problem.
Test it with three tests:
(1) Create an image with more empty space at the end than what one
refblock covers, see whether rebuilding the refcount structures
results in a change in the image file length. (It should not.)
(2) Leave precisely enough space somewhere at the beginning of the image
for the new reftable (and the refblock for that place), see whether
the new algorithm puts the reftable there. (It should.)
(3) Test the original problem: Create (something like) a block device
with a fixed size, then create a qcow2 image in there, write some
data, and then have qemu-img check rebuild the refcount structures.
Before HEAD^, the reftable would have been written past the image
file end, i.e. outside of what the block device provides, which
cannot work. HEAD^ should have fixed that.
("Something like a block device" means a loop device if we can use
one ("sudo -n losetup" works), or a FUSE block export with
growable=false otherwise.)
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220405134652.19278-3-hreitz@redhat.com>
303 runs two test cases, one of which requires zstd support.
Unfortunately, given that this is not a unittest-style test, we cannot
easily skip that single case, and instead can only skip the whole test.
(Alternatively, we could split this test into a zlib and a zstd part,
but that seems excessive, given that this test is not in auto and thus
likely only run by developers who have zstd support compiled in.)
Fixes: 677e0bae68 ("iotest 303: explicit compression type")
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
Message-Id: <20220323105522.53660-4-hreitz@redhat.com>
Some test cases run in iotest 065 want to run with zstd compression just
for added coverage. Run them with zlib if there is no zstd support
compiled in.
Reported-by: Thomas Huth <thuth@redhat.com>
Fixes: 12a936171d ("iotest 065: explicit compression type")
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220323105522.53660-3-hreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
We want to get rid of check-block.sh in the long run, so let's move
the checks for the bash version and sanitizers from check-block.sh
into the meson.build file instead.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220223093840.2515281-4-thuth@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Eric noticed while attempting to enable the vhost-user-blk-test for
Aarch64 that that things didn't work unless he put in a dummy
guest_malloc() at the start of the test. Without it
qvirtio_wait_used_elem() would assert when it reads a junk value for
idx resulting in:
qvirtqueue_get_buf: idx:2401 last_idx:0
qvirtqueue_get_buf: 0x7ffcb6d3fe74, (nil)
qvirtio_wait_used_elem: 3000000/0
ERROR:../../tests/qtest/libqos/virtio.c:226:qvirtio_wait_used_elem: assertion failed (got_desc_idx == desc_idx): (50331648 == 0)
Bail out! ERROR:../../tests/qtest/libqos/virtio.c:226:qvirtio_wait_used_elem: assertion failed (got_desc_idx == desc_idx): (50331648 == 0)
What was actually happening is the guest_malloc() effectively pushed
the allocation of the vring into the next page which just happened to
have clear memory. After much tedious tracing of the code I could see
that qvring_init() does attempt initialise a bunch of the vring
structures but skips the vring->used.idx value. It is probably not
wise to assume guest memory is zeroed anyway. Once the ring is
properly initialised the hack is no longer needed to get things
working.
Thanks-to: John Snow <jsnow@redhat.com> for helping debug
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220406173356.1891500-1-alex.bennee@linaro.org>
Tested-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The fuzz tests are currently scheduled for all targets, but their setup
code limits the run to "i386", so that these tests always show "SKIP"
on other targets. Move it to the right x86 list in meson.build, then
we can drop the architecture check during runtime, too.
Message-Id: <20220414130127.719528-1-thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Allow the same set of tests for all MIPS targets, so that "mipsel"
now gets some additional test coverage, too. While we're at it,
simplify the definitions for qtests_mips64 and qtests_mips64el.
Message-Id: <20220414114655.604391-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The perl test harness is not necessary anymore since commit 3d2f73ef75
("build: use "meson test" as the test harness"). Thus remove it from
tests/lcitool/projects/qemu.yml, run "make lcitool-refresh" and manually
clean the remaining docker / vm files that are not managed by lcitool yet.
Message-Id: <20220329102808.423681-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace the global variables with inlined helper functions. getpagesize() is very
likely annotated with a "const" function attribute (at least with glibc), and thus
optimization should apply even better.
This avoids the need for a constructor initialization too.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220323155743.1585078-12-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Replace a config-time define with a compile time condition
define (compatible with clang and gcc) that must be declared prior to
its usage. This avoids having a global configure time define, but also
prevents from bad usage, if the config header wasn't included before.
This can help to make some code independent from qemu too.
gcc supports __BYTE_ORDER__ from about 4.6 and clang from 3.2.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[ For the s390x parts I'm involved in ]
Acked-by: Halil Pasic <pasic@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220323155743.1585078-7-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
When running "make lcitool-refresh", this currently uses the hard-coded
/usr/bin/python3 from the script's shebang line for running Python.
That's bad, since neither /usr/bin/python3 is guaranteed to exist, nor
does it honor the python interpreter that the user might have chosen
while running the "configure" script. Thus let's rather use $(PYTHON)
in the Makefile, and improve the shebang line in the script in case
someone runs this directly.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220329063958.262669-1-thuth@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Block patches for 7.0-rc2:
- Disable GLOBAL_STATE_CODE() assertion for the 7.0 release: We got
another bug report for this, and we do not have the time to
investigate before 7.0, so disable the assertion for the release, to
re-enable and continue investigation in the 7.1 cycle
- stream job fix (regarding interaction with concurrent block jobs)
- iotests fixes
# gpg: Signature made Tue 29 Mar 2022 15:55:33 BST
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "hreitz@redhat.com"
# gpg: Good signature from "Hanna Reitz <hreitz@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF
* tag 'pull-block-2022-03-29' of https://gitlab.com/hreitz/qemu:
iotests: Fix status checks
block/stream: Drain subtree around graph change
main-loop: Disable GLOBAL_STATE_CODE() assertions
iotests: update test owner contact information
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
If the migration is over before we cancel it, we are
waiting in a loop a state that never comes because the state
is already "completed".
To avoid an infinite loop, skip the test if the migration
is "completed" before we were able to cancel it.
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-id: 20220329124259.355995-1-lvivier@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
An iotest's 'paused' condition is fickle; it will be reported as true
whenever the job is drained, for example, or when it is in the process
of completing.
030 and 041 contain such checks, we should replace them by checking the
job status instead. (As was done for 129 in commit f9a6256b48
for the 'busy' condition.)
Additionally, when we want to test that a job is paused on error, we
might want to give it some time to actually switch to the paused state.
Do that by waiting on the corresponding JOB_STATUS_CHANGE event. (But
only if they are not already paused; the loops these places are in fetch
all VM events, so they may have already fetched that event from the
queue.)
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220324180221.24508-1-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
When the stream block job cuts out the nodes between top and base in
stream_prepare(), it does not drain the subtree manually; it fetches the
base node, and tries to insert it as the top node's backing node with
bdrv_set_backing_hd(). bdrv_set_backing_hd() however will drain, and so
the actual base node might change (because the base node is actually not
part of the stream job) before the old base node passed to
bdrv_set_backing_hd() is installed.
This has two implications:
First, the stream job does not keep a strong reference to the base node.
Therefore, if it is deleted in bdrv_set_backing_hd()'s drain (e.g.
because some other block job is drained to finish), we will get a
use-after-free. We should keep a strong reference to that node.
Second, even with such a strong reference, the problem remains that the
base node might change before bdrv_set_backing_hd() actually runs and as
a result the wrong base node is installed.
Both effects can be seen in 030's TestParallelOps.test_overlapping_5()
case, which has five nodes, and simultaneously streams from the middle
node to the top node, and commits the middle node down to the base node.
As it is, this will sometimes crash, namely when we encounter the
above-described use-after-free.
Taking a strong reference to the base node, we no longer get a crash,
but the resuling block graph is less than ideal: The expected result is
obviously that all middle nodes are cut out and the base node is the
immediate backing child of the top node. However, if stream_prepare()
takes a strong reference to its base node (the middle node), and then
the commit job finishes in bdrv_set_backing_hd(), supposedly dropping
that middle node, the stream job will just reinstall it again.
Therefore, we need to keep the whole subtree drained in
stream_prepare(), so that the graph modification it performs is
effectively atomic, i.e. that the base node it fetches is still the base
node when bdrv_set_backing_hd() sets it as the top node's backing node.
Verify this by asserting in said 030's test case that the base node is
always the top node's immediate backing child when both jobs are done.
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220324140907.17192-1-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
This was attempted in commit 533b0a1a41 ("tests/tcg: Fix target-specific
Makefile variables path for user-mode", 2022-01-12) but it also used the
wrong path; default.mak is used for config/devices, not config/targets.
While at it, explain what the inclusion is about.
Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
