21ab34c954
config: split PVRDMA from RDMA
...
In some BSD systems RDMA migration is possible while
the pvrdma device can't be used because the mremap system call
is missing.
Reported-by: Rebecca Cran <rebecca@bluestop.org >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180816151637.24553-1-marcel.apfelbaum@gmail.com >
Reviewed-by: Thomas Huth <thuth@redhat.com >
2018-08-18 18:01:34 +03:00
72221d03b8
hw/pvrdma: remove not needed include
...
No need to include linux/types.h, is empty anyway.
Suggested-by: Thomas Huth <thuth@redhat.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180811171534.11917-1-marcel.apfelbaum@gmail.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
2018-08-18 18:01:27 +03:00
430e440c59
hw/rdma: Add reference to pci_dev in backend_dev
...
The field backend_dev->dev is not initialized, fix it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180805153518.2983-14-yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:01:20 +03:00
7d2ce4b016
hw/rdma: Bugfix - Support non-aligned buffers
...
RDMA application can provide non-aligned buffers to be registered. In
such case the DMA address passed by driver is pointing to the beginning
of the physical address of the mapped page so we can't distinguish
between two addresses from the same page.
Fix it by keeping the offset of the virtual address in mr->virt.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-13-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:55 +03:00
abc665aae3
hw/rdma: Print backend QP number in hex format
...
To be consistent with other prints throughout the code fix places that
print it as decimal number.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-12-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:48 +03:00
292dce627b
hw/rdma: Cosmetic change - move to generic function
...
To ease maintenance of struct comp_thread move all related code to
dedicated function.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-11-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:39 +03:00
67b32fe250
hw/pvrdma: Cosmetic change - indent right
...
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-10-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:30 +03:00
7131c4b028
hw/rdma: Reorder resource cleanup
...
To be consistence with allocation do the reverse order in deallocation
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-9-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:22 +03:00
4c408c9d5b
hw/rdma: Do not allocate memory for non-dma MR
...
There is no use in the memory allocated for non-dma MR.
Delete the code that allocates it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-8-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:15 +03:00
7f99daadbd
hw/rdma: Delete useless structure RdmaRmUserMR
...
The structure RdmaRmUserMR has no benefits, remove it an move all its
fields to struct RdmaRmMR.
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180805153518.2983-7-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 18:00:06 +03:00
6e7dba23af
hw/pvrdma: Make default pkey 0xFFFF
...
0x7FFF is not the default pkey - fix it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-6-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 17:59:58 +03:00
eca0f2a6be
hw/pvrdma: Clean CQE before use
...
Next CQE is fetched from CQ ring, clean it before usage as it still
carries old CQE values.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-5-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 17:59:50 +03:00
ef846e026c
hw/rdma: Modify debug macros
...
- Add line counter to ease navigation in log
- Print rdma instead of pvrdma
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-4-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 17:59:42 +03:00
b462a13480
hw/pvrdma: Bugfix - provide the correct attr_mask to query_qp
...
Calling rdma_rm_query_qp with attr_mask equals to -1 leads to error
where backend query_qp fails to retrieve the needed QP attributes.
Fix it by providing the attr_mask we got from driver.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180805153518.2983-3-yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 17:59:33 +03:00
751522275a
hw/rdma: Make distinction between device init and start modes
...
There are certain operations that are well considered as part of device
configuration while others are needed only when "start" command is
triggered by the guest driver. An example of device initialization step
is msix_init and example of "device start" stage is the creation of a CQ
completion handler thread.
Driver expects such distinction - implement it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180805153518.2983-2-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
2018-08-18 17:59:23 +03:00
78109066e6
hw/rdma: Use the IEC binary prefix definitions
...
It eases code review, unit is explicit.
Patch generated using:
$ git grep -E '(1024|2048|4096|8192|(<<|>>).?(10|20|30))' hw/ include/hw/
and modified manually.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org >
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180625124238.25339-40-f4bug@amsat.org >
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com >
2018-07-02 15:41:17 +02:00
d18a530d85
hw/rdma: Fix possible out of bounds access to port GID index
...
Make sure the backend GID index is less then port's
gid table length.
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180430200223.4119-8-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
6c080b9ea6
hw/rdma: Delete duplicate definition of MAX_RM_TBL_NAME
...
By a mistake this constant was defined twice - remove the duplication.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180430200223.4119-7-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
350929172b
hw/rdma: Fix possible out of bounds access to regs array
...
Coverity (CID1390589, CID1390608).
Array size is RDMA_BAR1_REGS_SIZE, let's make sure the given address is
in range.
While there also:
1. Adjust the size of this bar to reasonable size
2. Report the size of the array with sizeof(array)
Reported-by: Peter Maydell <peter.maydell@linaro.org >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180430200223.4119-6-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
c387e8a4ec
hw/rdma: Fix possible out of bounds access to GID table
...
Array size is MAX_PORT_GIDS, let's make sure the given index is in
range.
While there limit device table size to 1.
Reported-by: Peter Maydell <peter.maydell@linaro.org >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180430200223.4119-5-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
b9e34872b9
hw/rdma: Delete port's pkey table
...
Support for PKEY is not yet implemented. Removing the unneeded table
until a support will be added.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Message-Id: <20180430200223.4119-4-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
b0197cf80a
hw/rdma: Fix possible usage of a NULL pointer
...
Coverity CID 1390586; The cq handle is provided by the guest
and cannot be trusted to be previuosly allocated.
Fix it by exiting the completion flow.
Reported-by: Peter Maydell <peter.maydell@linaro.org >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180430200223.4119-3-marcel.apfelbaum@gmail.com >
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org >
2018-05-03 20:52:29 +03:00
1bad4957c8
hw/rdma: Fix possible munmap call on a NULL pointer
...
Coverity CID 1390620: we call munmap() on a NULL pointer.
Reported-by: Peter Maydell <peter.maydell@linaro.org >
Signed-off-by: Marcel Apfelbaum <marcel.apfelbaum@gmail.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
Message-Id: <20180430200223.4119-2-marcel.apfelbaum@gmail.com >
2018-05-03 20:52:29 +03:00
6f559013c8
hw/rdma: Fix 32-bit compilation
...
Use the correct printf formats, so that a 32-bit compile doesn't spit
out lots of warnings about %lx being incompatible with uint64_t.
Suggested-by: Eric Blake <eblake@redhat.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Eric Blake <eblake@redhat.com >
Tested-by: Eric Blake <eblake@redhat.com >
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org >
Message-Id: <20180322095220.9976-4-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
94f480b8db
hw/rdma: Use correct print format in CHK_ATTR macro
...
Macro should not cast the given variable to u64 instead it should use
the supplied format argument (fmt).
Reported-by: Eric Blake <eblake@redhat.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Eric Blake <eblake@redhat.com >
Tested-by: Eric Blake <eblake@redhat.com >
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org >
Message-Id: <20180322095220.9976-3-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
9bbb8d3577
hw/rdma: Change host_virt to void *
...
To avoid compilation warnings on 32-bit machines:
rdma_backend.c: In function 'rdma_backend_create_mr':
rdma_backend.c:409:37: error: cast to pointer from integer of different
size [-Werror=int-to-pointer-cast]
mr->ibmr = ibv_reg_mr(pd->ibpd, (void *)addr, length, access);
Reported-by: Eric Blake <eblake@redhat.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Eric Blake <eblake@redhat.com >
Tested-by: Eric Blake <eblake@redhat.com >
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org >
Message-Id: <20180322095220.9976-2-yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
197053e212
hw/rdma: fix clang compilation errors
...
Fix some enum castings and extra parentheses.
Reported-by: Michael S. Tsirkin <mst@redhat.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
Message-Id: <20180321140316.96045-1-marcel@redhat.com >
Reviewed-by: Yuval Shaia <yuval.shaia@oracle.com >
2018-03-23 18:38:55 +03:00
0efc9511aa
rdma: fix up include directives
...
Our rule right now is to use <> for external headers only.
RDMA code violates that, fix it up.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com >
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com >
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
79cfdca7aa
hw/rdma: Add support for Query QP verb to pvrdma device
...
This IB verb is needed by some applications - implement it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
c99f217431
hw/rdma: Add Query QP operation
...
This operation is needed by rdma devices - implement it.
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-03-23 18:38:55 +03:00
919ae3dd11
hw/rdma: Implementation of PVRDMA device
...
PVRDMA is the QEMU implementation of VMware's paravirtualized RDMA device.
It works with its Linux Kernel driver AS IS, no need for any special
guest modifications.
While it complies with the VMware device, it can also communicate with
bare metal RDMA-enabled machines and does not require an RDMA HCA in the
host, it can work with Soft-RoCE (rxe).
It does not require the whole guest RAM to be pinned allowing memory
over-commit and, even if not implemented yet, migration support will be
possible with some HW assistance.
Implementation is divided into 2 components, rdma general and pvRDMA
specific functions and structures.
The second PVRDMA sub-module - interaction with PCI layer.
- Device configuration and setup (MSIX, BARs etc).
- Setup of DSR (Device Shared Resources)
- Setup of device ring.
- Device management.
Reviewed-by: Dotan Barak <dotanb@mellanox.com >
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-02-19 13:03:24 +02:00
98d176f8e5
hw/rdma: PVRDMA commands and data-path ops
...
First PVRDMA sub-module - implementation of the PVRDMA device.
- PVRDMA commands such as create CQ and create MR.
- Data path QP operations - post_send and post_recv.
- Completion handler.
Reviewed-by: Dotan Barak <dotanb@mellanox.com >
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-02-19 13:03:24 +02:00
ef6d4ccdc9
hw/rdma: Implementation of generic rdma device layers
...
This layer is composed of two sub-modules, backend and resource manager.
Backend sub-module is responsible for all the interaction with IB layers
such as ibverbs and umad (external libraries).
Resource manager is a collection of functions and structures to manage
RDMA resources such as QPs, CQs and MRs.
Reviewed-by: Dotan Barak <dotanb@mellanox.com >
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-02-19 13:03:24 +02:00
b3a9227769
hw/rdma: Definitions for rdma device and rdma resource manager
...
Definition of various structures and constants used in backend and
resource manager modules.
Reviewed-by: Dotan Barak <dotanb@mellanox.com >
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-02-19 13:03:24 +02:00
dcbf469aef
hw/rdma: Add wrappers and macros
...
As all mapping for this device are from driver to device,
declare wrappers on top of pci_dma_*map functions.
In addition, declare macros to be used for debug messages.
Reviewed-by: Dotan Barak <dotanb@mellanox.com >
Reviewed-by: Zhu Yanjun <yanjun.zhu@oracle.com >
Signed-off-by: Yuval Shaia <yuval.shaia@oracle.com >
Signed-off-by: Marcel Apfelbaum <marcel@redhat.com >
2018-02-19 13:03:24 +02:00