Fixes: f5d4361cda
Fixes: 52a474180a
Fixes: 46f49468c6
Remove the try/except block that handles file-opening errors in
QAPISchemaParser.__init__() and add one each to
QAPISchemaParser._include() and QAPISchema.__init__() respectively.
This simultaneously fixes the typing of info.fname (f5d4361cda), A
static typing violation in test-qapi (46f49468c6), and a regression of
an error message (52a474180a).
The short-ish version of what motivates this patch is:
- It's hard to write a good error message in the init method,
because we need to determine the context of our caller to do so.
It's easier to just let the caller write the message.
- We don't want to allow QAPISourceInfo(None, None, None) to exist. The
typing introduced by commit f5d4361cda types the 'fname' field as
(non-optional) str, which was premature until the removal of this
construct.
- Errors made using such an object are currently incorrect (since
52a474180a)
- It's not technically a semantic error if we cannot open the schema.
- There are various typing constraints that make mixing these two cases
undesirable for a single special case.
- test-qapi's code handling an fname of 'None' is now dead, drop it.
Additionally, Not all QAPIError objects have an 'info' field (since
46f49468), so deleting this stanza corrects a typing oversight in
test-qapi introduced by that commit.
Other considerations:
- open() is moved to a 'with' block to ensure file pointers are
cleaned up deterministically.
- Python 3.3 deprecated IOError and made it a synonym for OSError.
Avoid the misleading perception these exception handlers are
narrower than they really are.
The long version:
The error message here is incorrect (since commit 52a474180a):
> python3 qapi-gen.py 'fake.json'
qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory
In pursuing it, we find that QAPISourceInfo has a special accommodation
for when there's no filename. Meanwhile, the intent when QAPISourceInfo
was typed (f5d4361cda) was non-optional 'str'. This usage was
overlooked.
To remove this, I'd want to avoid having a "fake" QAPISourceInfo
object. I also don't want to explicitly begin accommodating
QAPISourceInfo itself being None, because we actually want to eventually
prove that this can never happen -- We don't want to confuse "The file
isn't open yet" with "This error stems from a definition that wasn't
defined in any file".
(An earlier series tried to create a dummy info object, but it was tough
to prove in review that it worked correctly without creating new
regressions. This patch avoids that distraction. We would like to first
prove that we never raise QAPISemError for any built-in object before we
add "special" info objects. We aren't ready to do that yet.)
So, which way out of the labyrinth?
Here's one way: Don't try to handle errors at a level with "mixed"
semantic contexts; i.e. don't mix inclusion errors (should report a
source line where the include was triggered) and command line errors
(where we specified a file we couldn't read).
Remove the error handling from the initializer of the parser. Pythonic!
Now it's the caller's job to figure out what to do about it. Handle the
error in QAPISchemaParser._include() instead, where we can write a
targeted error message where we are guaranteed to have an 'info' context
to report with.
The root level error can similarly move to QAPISchema.__init__(), where
we know we'll never have an info context to report with, so we use a
more abstract error type.
Now the error looks sensible again:
> python3 qapi-gen.py 'fake.json'
qapi-gen.py: can't read schema file 'fake.json': No such file or directory
With these error cases separated, QAPISourceInfo can be solidified as
never having placeholder arguments that violate our desired types. Clean
up test-qapi along similar lines.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
testing and plugin updates:
- various fixes for binfmt_misc docker images
- add hexagon check-tcg support docker image
- add tricore check-tcg support
- refactor ppc docker images
- add missing ppc64le tests
- don't use host_cc for test fallback
- check-tcg configure.sh tweaks for cross compile/clang
- fix some memory leaks in plugins
# gpg: Signature made Tue 18 May 2021 09:37:21 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-and-plugin-updates-180521-2: (29 commits)
configure: use cc, not host_cc to set cross_cc for build arch
tests/tcg: don't allow clang as a cross compiler
tests/tcg: fix missing return
tests/tcg/ppc64le: tests for brh/brw/brd
tests/docker: gcc-10 based images for ppc64{,le} tests
tests/tcg/tricore: Add muls test
tests/tcg/tricore: Add msub test
tests/tcg/tricore: Add madd test
tests/tcg/tricore: Add ftoi test
tests/tcg/tricore: Add fmul test
tests/tcg/tricore: Add fadd test
tests/tcg/tricore: Add dvstep test
tests/tcg/tricore: Add clz test
tests/tcg/tricore: Add bmerge test
tests/tcg/tricore: Add macros to create tests and first test 'abs'
configure: Emit HOST_CC to config-host.mak
tests/tcg/tricore: Add build infrastructure
hw/tricore: Add testdevice for tests in tests/tcg/
tests/tcg: Run timeout cmds using --foreground
tests/tcg: Add docker_as and docker_ld cmds
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Block layer patches
- vhost-user-blk: Fix error handling during initialisation
- Add test cases for the vhost-user-blk export
- Fix leaked Transaction objects
- qcow2: Expose dirty bit in 'qemu-img info'
# gpg: Signature made Tue 18 May 2021 11:57:46 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
vhost-user-blk: Check that num-queues is supported by backend
virtio: Fail if iommu_platform is requested, but unsupported
vhost-user-blk: Get more feature flags from vhost device
vhost-user-blk: Improve error reporting in realize
vhost-user-blk: Don't reconnect during initialisation
vhost-user-blk: Make sure to set Error on realize failure
vhost-user-blk-test: test discard/write zeroes invalid inputs
tests/qtest: add multi-queue test case to vhost-user-blk-test
test: new qTest case to test the vhost-user-blk-server
block/export: improve vu_blk_sect_range_ok()
block: Fix Transaction leak in bdrv_reopen_multiple()
block: Fix Transaction leak in bdrv_root_attach_child()
qcow2: set bdi->is_dirty
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Currently there are two problems.
The first is clang generates a preamble (that is always executed) to
stack xmm registers. This causes a ILLOP on the x86_64 softmmu tests
as SSE isn't enabled.
The second is the inline assembler in test-i386.c breaks clangs
compiler and I don't know how to fix it. Even with Theodore's patch
series (D5741445-7EFD-4AF1-8DB2-E4AFA93CBB1A@icloud.com) I still get
compiler failures.
For now lets just skip clang and allow it to fall back to the
containers which we know have compilers which work.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-31-alex.bennee@linaro.org>
This kind of tests is inspired by the riscv-tests repository. This adds
macros that makes it easy to create single instruction self containing
tests.
It is achieved by macros that create a test sequence for an
instruction and check for a supplied correct value. If the value is correct the
next instruction is tested. Otherwise we jump to fail handler that writes is
test number as a status code back to qemu that then exits on that status code.
If all tests pass we write back 0 as a status code and exit.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20210305170045.869437-7-kbastian@mail.uni-paderborn.de>
[AJB: add container_hosts]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-17-alex.bennee@linaro.org>
At least for the TriCore target no easily available c compiler exists.
Thus we need to rely on "as" and "ld". This allows us to run them
through the docker image. We don't test the generation capabilities of
docker images as they are assumed to work.
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
[AJB: fix quoting, only handle docker & clear, test -n, tweak commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-12-alex.bennee@linaro.org>
There should only be one compiler per architecture. Those cases where
the same compiler can deal with a different architecture should be
explicitly set for both cross_cc and docker configurations. Otherwise
you get strangeness like:
--cross-cc-aarch64=/bin/false
causing the logic to attempt to use a locally available
arm-linux-gnueabihf-gcc rather than forcing the use of the docker
image which is what is implied by the command line option.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-10-alex.bennee@linaro.org>
This image is a little special because it takes a long time to build.
As such most users don't want to be doing that and just pull random
binaries from the ether as intended by the container gods. This
involves someone with credentials and a beefy machine running:
make docker-image-debian-hexagon-cross V=1 NOCACHE=1 J=30
docker tag qemu/debian-hexagon-cross registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross
docker push registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross
With a suitable binary in the "cloud" a normal user will run:
make docker-image-debian-hexagon-cross
or have it run for them through the dependency mechanism of our
over-engineered makefiles and get the binary they wanted. There are a
few wrinkles of course including needing to tweak the final image to
have the credentials of the user so we can actually do our cross
compiles.
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Alessandro Di Federico <ale@rev.ng>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Brian Cain <bcain@quicinc.com>
Message-Id: <20210512102051.12134-7-alex.bennee@linaro.org>
The current user functionality is used for cross compiling to avoid
complications with permissions when building test programs. However
for images that come from the registry we still need the ability to
add the user after the fact.
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-5-alex.bennee@linaro.org>
Reorg FloatParts to use QEMU_GENERIC.
Begin replacing the Berkeley float128 routines with FloatParts128.
- includes a new implementation of float128_muladd
- includes the snan silencing that was missing from
float{32,64}_to_float128 and float128_to_float{32,64}.
- does not include float128_min/max* (written but not yet reviewed).
# gpg: Signature made Sun 16 May 2021 13:27:10 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* remotes/rth-gitlab/tags/pull-fp-20210516: (46 commits)
softfloat: Move round_to_int_and_pack to softfloat-parts.c.inc
softfloat: Move round_to_int to softfloat-parts.c.inc
softfloat: Convert float-to-float conversions with float128
softfloat: Split float_to_float
softfloat: Move div_floats to softfloat-parts.c.inc
softfloat: Introduce sh[lr]_double primitives
softfloat: Tidy mul128By64To192
softfloat: Use add192 in mul128To256
softfloat: Use mulu64 for mul64To128
softfloat: Move muladd_floats to softfloat-parts.c.inc
softfloat: Move mul_floats to softfloat-parts.c.inc
softfloat: Implement float128_add/sub via parts
softfloat: Move addsub_floats to softfloat-parts.c.inc
softfloat: Use uadd64_carry, usub64_borrow in softfloat-macros.h
softfloat: Move round_canonical to softfloat-parts.c.inc
softfloat: Move sf_canonicalize to softfloat-parts.c.inc
softfloat: Move pick_nan_muladd to softfloat-parts.c.inc
softfloat: Move pick_nan to softfloat-parts.c.inc
softfloat: Move return_nan to softfloat-parts.c.inc
softfloat: Convert float128_default_nan to parts
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Block patches:
- drop block/io write notifiers
- qemu-iotests enhancements to make debugging easier
- rbd parsing fix
- HMP qemu-io fix (for iothreads)
- mirror job cancel relaxation (do not cancel in-flight requests when a
READY mirror job is canceled with force=false)
- document qcow2's data_file and data_file_raw features
- fix iotest 297 for pylint 2.8
- block/copy-on-read refactoring
# gpg: Signature made Fri 14 May 2021 17:43:40 BST
# gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg: issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40
* remotes/maxreitz/tags/pull-block-2021-05-14:
write-threshold: deal with includes
test-write-threshold: drop extra TestStruct structure
test-write-threshold: drop extra tests
block/write-threshold: drop extra APIs
test-write-threshold: rewrite test_threshold_(not_)trigger tests
block: drop write notifiers
block/write-threshold: don't use write notifiers
qemu-iotests: fix pylint 2.8 consider-using-with error
block/copy-on-read: use bdrv_drop_filter() and drop s->active
Document qemu-img options data_file and data_file_raw
qemu-iotests: fix case of SOCK_DIR already in the environment
qemu-iotests: let "check" spawn an arbitrary test command
qemu-iotests: move command line and environment handling from TestRunner to TestEnv
qemu-iotests: allow passing unittest.main arguments to the test scripts
qemu-iotests: do not buffer the test output
mirror: stop cancelling in-flight requests on non-force cancel in READY
monitor: hmp_qemu_io: acquire aio contex, fix crash
block/rbd: Add an escape-aware strchr helper
iotests/231: Update expected deprecation message
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* Replace YAML anchors by extends in the gitlab-CI yaml files
* Many small qtest fixes (e.g. to fix issues discovered by Coverity)
* Poison more config switches in common code
* Fix the failing Travis-CI and Cirrus-CI tasks
# gpg: Signature made Fri 14 May 2021 12:17:39 BST
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* remotes/thuth-gitlab/tags/pull-request-2021-05-14:
cirrus.yml: Fix the MSYS2 task
pc-bios/s390-ccw: Fix inline assembly for older versions of Clang
tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
configure: Poison all current target-specific #defines
migration: Move populate_vfio_info() into a separate file
include/sysemu: Poison all accelerator CONFIG switches in common code
tests: Avoid side effects inside g_assert() arguments
tests/qtest/rtc-test: Remove pointless NULL check
tests/qtest/tpm-util.c: Free memory with correct free function
tests/migration-test: Fix "true" vs true
tests/qtest/npcm7xx_pwm-test.c: Avoid g_assert_true() for non-test assertions
tests/qtest/ahci-test.c: Calculate iso_size with 64-bit arithmetic
util/compatfd.c: Replaced a malloc call with g_malloc.
libqtest: refuse QTEST_QEMU_BINARY=qemu-kvm
docs/devel/qgraph: add troubleshooting information
libqos/qgraph: fix "UNAVAILBLE" typo
gitlab-ci: Replace YAML anchors by extends (native_test_job)
gitlab-ci: Replace YAML anchors by extends (native_build_job)
gitlab-ci: Replace YAML anchors by extends (container_job)
tests/docker/dockerfiles: Add ccache to containers where it was missing
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
"qemu/typedefs.h" is enough for include/block/write-threshold.h header
with forward declaration of BlockDriverState. Also drop extra includes
from block/write-threshold.c and tests/unit/test-write-threshold.c
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210506090621.11848-9-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
bdrv_write_threshold_exceeded() is unused.
bdrv_write_threshold_is_set() is used only to double check the value of
bs->write_threshold_offset in tests. No real sense in it (both tests do
check real value with help of bdrv_write_threshold_get())
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210506090621.11848-5-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[mreitz: Adjusted commit message as per Eric's suggestion]
Signed-off-by: Max Reitz <mreitz@redhat.com>
These tests use bdrv_write_threshold_exceeded() API, which is used only
for test (since pre-previous commit). Better is testing real API, which
is used in block.c as well.
So, let's call bdrv_write_threshold_check_write(), and check is
bs->write_threshold_offset cleared or not (it's cleared iff threshold
triggered).
Also we get rid of BdrvTrackedRequest use here. Note, that paranoiac
bdrv_check_request() calls were added in 8b1170012b to protect
BdrvTrackedRequest. Drop them now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210506090621.11848-4-vsementsov@virtuozzo.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
pylint 2.8 introduces consider-using-with error, suggesting
to use the 'with' block statement when possible.
Modify all subprocess.Popen call to use the 'with' statement,
except the one in __init__ of QemuIoInteractive class, since
it is assigned to a class field and used in other methods.
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20210510190449.65948-1-eesposit@redhat.com>
[mreitz: Disable bad-option-value warning in the iotests' pylintrc, so
that disabling consider-using-with in QemuIoInteractive will
not produce a warning in pre-2.8 pylint versions]
Signed-off-by: Max Reitz <mreitz@redhat.com>
Right now there is no easy way for "check" to print a reproducer command.
Because such a reproducer command line would be huge, we can instead teach
check to start a command of our choice. This can be for example a Python
unit test with arguments to only run a specific subtest.
Move the trailing empty line to print_env(), since it always looks better
and one caller was not adding it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20210323181928.311862-5-pbonzini@redhat.com>
Message-Id: <20210503110110.476887-5-pbonzini@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Python test scripts that use unittest consist of multiple tests.
unittest.main allows selecting which tests to run, but currently this
is not possible because the iotests wrapper ignores sys.argv.
unittest.main command line options also allow the user to pick the
desired options for verbosity, failfast mode, etc. While "-d" is
currently translated to "-v", it also enables extra debug output,
and other options are not available at all.
These command line options only work if the unittest.main testRunner
argument is a type, rather than a TestRunner instance. Therefore, pass
the class name and "verbosity" argument to unittest.main, and adjust for
the different default warnings between TextTestRunner and unittest.main.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20210323181928.311862-3-pbonzini@redhat.com>
Message-Id: <20210503110110.476887-3-pbonzini@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
