move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and
update import directives across the tree.
This is done to create a PEP420 namespace package, in which we may
create subpackages. To do this, the namespace directory ("qemu") should
not have any modules in it. Those files will go into new 'machine',
'qmp' and 'utils' subpackages instead.
Implement machine/__init__.py making the top-level classes and functions
from its various modules available directly inside the package. Change
qmp.py to qmp/__init__.py similarly, such that all of the useful QMP
library classes are available directly from "qemu.qmp" instead of
"qemu.qmp.qmp".
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
mypy is kind of weird about how it handles imports. For legacy reasons,
it won't load PEP 420 namespaces, because of logic implemented prior to
that becoming a standard.
So, if you plan on using any, you have to pass
--namespace-packages. Alright, fine.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Even though there are qtest based tests for hotplugging CPUs (from
which this test took some inspiration from), this one adds checks
from a Linux guest point of view.
It should also serve as an example for tests that follow a similar
pattern and need to interact with QEMU (via qmp) and with the Linux
guest via SSH.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210412044644.55083-11-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Both the virtiofs submounts and the linux ssh mips malta tests
contains useful methods related to ssh that deserve to be made
available to other tests. Let's move them to an auxiliary, mix-in
class that will be used on the base LinuxTest class.
The method that helps with setting up an ssh connection will now
support both key and password based authentication, defaulting to key
based.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210412044644.55083-5-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
If the vmlinuz variable is set to anything that evaluates to True,
then the respective arguments should be set. If the variable contains
an empty string, than it will evaluate to False, and the extra
arguments will not be set.
This keeps the same logic, but improves readability a bit.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-3-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Each instance of qemu.machine.QEMUMachine currently has a "test
directory", which may not have any relation to a "test", and it's
really a temporary directory.
Users instantiating the QEMUMachine class will be able to set the
location of the directory that will *contain* the QEMUMachine unique
temporary directory, so that parameter name has been changed from
test_dir to base_temp_dir.
A property has been added to allow users to access it without using
private attributes, and with that, the directory is created on first
use of the property.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210211220146.2525771-3-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
* submodule cleanups (Philippe, myself)
* tiny step towards a usable preconfig mode (myself)
* Kconfig and LOCK_GUARD cleanups (philippe)
* new x86 CPUID feature (Yang Zhong)
* "-object qtest" support (myself)
* Dirty ring support for KVM (Peter)
* Fixes for 6.0 command line parsing breakage (myself)
* Fix for macOS 11.3 SDK (Katsuhiro)
# gpg: Signature made Wed 26 May 2021 13:50:12 BST
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* remotes/bonzini-gitlab/tags/for-upstream: (28 commits)
gitlab-ci: use --meson=git for CFI jobs
hw/scsi: Fix sector translation bug in scsi_unmap_complete_noio
configure: Avoid error messages about missing *-config-*.h files
doc: Add notes about -mon option mode=control argument.
qemu-config: load modules when instantiating option groups
vl: allow not specifying size in -m when using -M memory-backend
replication: move include out of root directory
remove qemu-options* from root directory
meson: Set implicit_include_directories to false
tests/qtest/fuzz: Fix build failure
KVM: Dirty ring support
KVM: Disable manual dirty log when dirty ring enabled
KVM: Add dirty-ring-size property
KVM: Cache kvm slot dirty bitmap size
KVM: Simplify dirty log sync in kvm_set_phys_mem
KVM: Provide helper to sync dirty bitmap from slot to ramblock
KVM: Provide helper to get kvm dirty log
KVM: Create the KVMSlot dirty bitmap on flag changes
KVM: Use a big lock to replace per-kml slots_lock
memory: Introduce log_sync_global() to memory listener
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The replication.h file is included from migration/colo.c and tests/unit/test-replication.c,
so it should be in include/.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
On Fedora 32, using clang (version 10.0.1-3.fc32) we get:
tests/qtest/fuzz/fuzz.c:237:5: error: implicit declaration of function 'qemu_init' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
qemu_init(result.we_wordc, result.we_wordv, NULL);
^
qemu_init() is declared in "sysemu/sysemu.h", include this
header to fix.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210513162008.3922223-1-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Xtensa cores may or may not have hardware support for unaligned memory
access. Remove TARGET_ALIGNED_ONLY=y from all xtensa configurations and
pass MO_ALIGN in memory access flags for all operations that would raise
an exception.
Simplify use of gen_load_store_alignment by passing access size and
alignment requirements in single parameter.
Drop condition from xtensa_cpu_do_unaligned_access and replace it with
assertion.
Add a test.
Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
QAPI patches patches for 2021-05-20
# gpg: Signature made Thu 20 May 2021 16:10:21 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653
* remotes/armbru/tags/pull-qapi-2021-05-20:
qapi/parser: add docstrings
qapi/parser: allow 'ch' variable name
qapi/parser: Remove superfluous list comprehension
qapi/parser: add type hint annotations
qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard
qapi/parser: Fix token membership tests when token can be None
qapi: add must_match helper
qapi/parser: Use @staticmethod where appropriate
qapi/parser: assert object keys are strings
qapi/parser: enforce all top-level expressions must be dict in _parse()
qapi/parser: Assert lexer value is a string
qapi/parser: factor parsing routine into method
qapi/source: Remove line number from QAPISourceInfo initializer
qapi: Add test for nonexistent schema file
qapi/parser: Don't try to handle file errors
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
When the token can be None (EOF), we can't use 'x in "abc"' style
membership tests to group types of tokens together, because 'None in
"abc"' is a TypeError.
Easy enough to fix. (Use a tuple: It's neither a static typing error nor
a runtime error to check for None in Tuple[str, ...])
Add tests to prevent a regression. (Note: they cannot be added prior to
this fix, as the unhandled stack trace will not match test output in the
CI system.)
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Instead of using get_expr nested=False, allow get_expr to always return
any expression. In exchange, add a new error message to the top-level
parser that explains the semantic error: Top-level expressions must
always be JSON objects.
This helps mypy understand the rest of this function which assumes that
get_expr did indeed return a dict.
The exception type changes from QAPIParseError to QAPISemError as a
result, and the error message in two tests now changes.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Fixes: f5d4361cda
Fixes: 52a474180a
Fixes: 46f49468c6
Remove the try/except block that handles file-opening errors in
QAPISchemaParser.__init__() and add one each to
QAPISchemaParser._include() and QAPISchema.__init__() respectively.
This simultaneously fixes the typing of info.fname (f5d4361cda), A
static typing violation in test-qapi (46f49468c6), and a regression of
an error message (52a474180a).
The short-ish version of what motivates this patch is:
- It's hard to write a good error message in the init method,
because we need to determine the context of our caller to do so.
It's easier to just let the caller write the message.
- We don't want to allow QAPISourceInfo(None, None, None) to exist. The
typing introduced by commit f5d4361cda types the 'fname' field as
(non-optional) str, which was premature until the removal of this
construct.
- Errors made using such an object are currently incorrect (since
52a474180a)
- It's not technically a semantic error if we cannot open the schema.
- There are various typing constraints that make mixing these two cases
undesirable for a single special case.
- test-qapi's code handling an fname of 'None' is now dead, drop it.
Additionally, Not all QAPIError objects have an 'info' field (since
46f49468), so deleting this stanza corrects a typing oversight in
test-qapi introduced by that commit.
Other considerations:
- open() is moved to a 'with' block to ensure file pointers are
cleaned up deterministically.
- Python 3.3 deprecated IOError and made it a synonym for OSError.
Avoid the misleading perception these exception handlers are
narrower than they really are.
The long version:
The error message here is incorrect (since commit 52a474180a):
> python3 qapi-gen.py 'fake.json'
qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory
In pursuing it, we find that QAPISourceInfo has a special accommodation
for when there's no filename. Meanwhile, the intent when QAPISourceInfo
was typed (f5d4361cda) was non-optional 'str'. This usage was
overlooked.
To remove this, I'd want to avoid having a "fake" QAPISourceInfo
object. I also don't want to explicitly begin accommodating
QAPISourceInfo itself being None, because we actually want to eventually
prove that this can never happen -- We don't want to confuse "The file
isn't open yet" with "This error stems from a definition that wasn't
defined in any file".
(An earlier series tried to create a dummy info object, but it was tough
to prove in review that it worked correctly without creating new
regressions. This patch avoids that distraction. We would like to first
prove that we never raise QAPISemError for any built-in object before we
add "special" info objects. We aren't ready to do that yet.)
So, which way out of the labyrinth?
Here's one way: Don't try to handle errors at a level with "mixed"
semantic contexts; i.e. don't mix inclusion errors (should report a
source line where the include was triggered) and command line errors
(where we specified a file we couldn't read).
Remove the error handling from the initializer of the parser. Pythonic!
Now it's the caller's job to figure out what to do about it. Handle the
error in QAPISchemaParser._include() instead, where we can write a
targeted error message where we are guaranteed to have an 'info' context
to report with.
The root level error can similarly move to QAPISchema.__init__(), where
we know we'll never have an info context to report with, so we use a
more abstract error type.
Now the error looks sensible again:
> python3 qapi-gen.py 'fake.json'
qapi-gen.py: can't read schema file 'fake.json': No such file or directory
With these error cases separated, QAPISourceInfo can be solidified as
never having placeholder arguments that violate our desired types. Clean
up test-qapi along similar lines.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
testing and plugin updates:
- various fixes for binfmt_misc docker images
- add hexagon check-tcg support docker image
- add tricore check-tcg support
- refactor ppc docker images
- add missing ppc64le tests
- don't use host_cc for test fallback
- check-tcg configure.sh tweaks for cross compile/clang
- fix some memory leaks in plugins
# gpg: Signature made Tue 18 May 2021 09:37:21 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-and-plugin-updates-180521-2: (29 commits)
configure: use cc, not host_cc to set cross_cc for build arch
tests/tcg: don't allow clang as a cross compiler
tests/tcg: fix missing return
tests/tcg/ppc64le: tests for brh/brw/brd
tests/docker: gcc-10 based images for ppc64{,le} tests
tests/tcg/tricore: Add muls test
tests/tcg/tricore: Add msub test
tests/tcg/tricore: Add madd test
tests/tcg/tricore: Add ftoi test
tests/tcg/tricore: Add fmul test
tests/tcg/tricore: Add fadd test
tests/tcg/tricore: Add dvstep test
tests/tcg/tricore: Add clz test
tests/tcg/tricore: Add bmerge test
tests/tcg/tricore: Add macros to create tests and first test 'abs'
configure: Emit HOST_CC to config-host.mak
tests/tcg/tricore: Add build infrastructure
hw/tricore: Add testdevice for tests in tests/tcg/
tests/tcg: Run timeout cmds using --foreground
tests/tcg: Add docker_as and docker_ld cmds
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Block layer patches
- vhost-user-blk: Fix error handling during initialisation
- Add test cases for the vhost-user-blk export
- Fix leaked Transaction objects
- qcow2: Expose dirty bit in 'qemu-img info'
# gpg: Signature made Tue 18 May 2021 11:57:46 BST
# gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6
# gpg: issuer "kwolf@redhat.com"
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
vhost-user-blk: Check that num-queues is supported by backend
virtio: Fail if iommu_platform is requested, but unsupported
vhost-user-blk: Get more feature flags from vhost device
vhost-user-blk: Improve error reporting in realize
vhost-user-blk: Don't reconnect during initialisation
vhost-user-blk: Make sure to set Error on realize failure
vhost-user-blk-test: test discard/write zeroes invalid inputs
tests/qtest: add multi-queue test case to vhost-user-blk-test
test: new qTest case to test the vhost-user-blk-server
block/export: improve vu_blk_sect_range_ok()
block: Fix Transaction leak in bdrv_reopen_multiple()
block: Fix Transaction leak in bdrv_root_attach_child()
qcow2: set bdi->is_dirty
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Currently there are two problems.
The first is clang generates a preamble (that is always executed) to
stack xmm registers. This causes a ILLOP on the x86_64 softmmu tests
as SSE isn't enabled.
The second is the inline assembler in test-i386.c breaks clangs
compiler and I don't know how to fix it. Even with Theodore's patch
series (D5741445-7EFD-4AF1-8DB2-E4AFA93CBB1A@icloud.com) I still get
compiler failures.
For now lets just skip clang and allow it to fall back to the
containers which we know have compilers which work.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-31-alex.bennee@linaro.org>
This kind of tests is inspired by the riscv-tests repository. This adds
macros that makes it easy to create single instruction self containing
tests.
It is achieved by macros that create a test sequence for an
instruction and check for a supplied correct value. If the value is correct the
next instruction is tested. Otherwise we jump to fail handler that writes is
test number as a status code back to qemu that then exits on that status code.
If all tests pass we write back 0 as a status code and exit.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Message-Id: <20210305170045.869437-7-kbastian@mail.uni-paderborn.de>
[AJB: add container_hosts]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-17-alex.bennee@linaro.org>
At least for the TriCore target no easily available c compiler exists.
Thus we need to rely on "as" and "ld". This allows us to run them
through the docker image. We don't test the generation capabilities of
docker images as they are assumed to work.
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
[AJB: fix quoting, only handle docker & clear, test -n, tweak commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-12-alex.bennee@linaro.org>
