Hi Alex, Arnd, Jean-Philippe, and all,
During the OpenAMP App-services call this week WindRiver gave a couple of extremely interesting presentations, see attached slides.
Dan Milea discussed the usage of virtio between heterogeneous clusters, i.e. virtio frontends on the Cortex-R cluster and virtio backends on the Cortex-A cluster. They used a setup based on pre-shared memory to make it work, which I believe is similar to the swiotlb approach we discussed in Stratos. They noted that the last version of the virtio spec on github had something in regards to virtio and pre-shared memory regions that might help us cover this use-case from a spec perspective.
Dan, would you be able to share a pointer to it for clarity? So far, we acted on the assumption that the virtio spec doesn't allow for this architecture today; it would be fantastic if it turns out that it already does.
The other very interesting presentation by Joshua Pincus was about virtio-mmio and MSIs. They did excellent measurements on the performance of virtio-mmio and they found out that the single source of notifications (one interrupt) is the bottleneck. Adding MSIs vastly improved performance. You can see the detailed breakdown on slide #8 of "OpenAMP Virt I/O MMIO w/ MSI". This analysis really points in the direction of adding MSIs to virtio-mmio.
Cheers,
Stefano
---------- Forwarded message ---------- Date: Wed, 28 Oct 2020 18:41:09 +0000 From: Nathalie Chan King Choy via App-services app-services@lists.openampproject.org Reply-To: Nathalie Chan King Choy nathalie@xilinx.com To: "app-services@lists.openampproject.org" app-services@lists.openampproject.org Subject: [App-services] 2020-10-27 OpenAMP App-services call recording, notes, slides, and action items
Hi all,
The notes from yesterday’s OpenAMP App-services call can be found at:
https://github.com/OpenAMP/open-amp/wiki/OpenAMP-Application-Services-Subgro...
The link to the Webex recording is in the notes. I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks.
Please find attached the slides from Dan & Josh.
Action items:
* Dan & Josh to send slides (DONE)
* Stefano to start a thread w/ the folks who are working on shared memory & VirtIO
Best regards,
Nathalie C. Chan King Choy
Program Manager focused on Open Source and Community
Hi,
The virtio spec generated from this repository https://github.com/oasis-tcs/virtio-spec https://github.com/oasis-tcs/virtio-spec includes a section dedicated to shared memory regions, described as "an additional facility available to devices that need a region of memory that’s continuously shared between the device and the driver, rather than passed between them in the way virtqueue elements are".
This version of the spec does not seem to have been released yet, although some time ago there was an attempt to use shared memory regions with virtio-fs (https://lwn.net/Articles/774495/ https://lwn.net/Articles/774495/).
Regards, Dan
On 10/29/20 1:55 AM, Stefano Stabellini wrote:
[Please note this e-mail is from an EXTERNAL e-mail address]
Hi Alex, Arnd, Jean-Philippe, and all,
During the OpenAMP App-services call this week WindRiver gave a couple of extremely interesting presentations, see attached slides.
Dan Milea discussed the usage of virtio between heterogeneous clusters, i.e. virtio frontends on the Cortex-R cluster and virtio backends on the Cortex-A cluster. They used a setup based on pre-shared memory to make it work, which I believe is similar to the swiotlb approach we discussed in Stratos. They noted that the last version of the virtio spec on github had something in regards to virtio and pre-shared memory regions that might help us cover this use-case from a spec perspective.
Dan, would you be able to share a pointer to it for clarity? So far, we acted on the assumption that the virtio spec doesn't allow for this architecture today; it would be fantastic if it turns out that it already does.
The other very interesting presentation by Joshua Pincus was about virtio-mmio and MSIs. They did excellent measurements on the performance of virtio-mmio and they found out that the single source of notifications (one interrupt) is the bottleneck. Adding MSIs vastly improved performance. You can see the detailed breakdown on slide #8 of "OpenAMP Virt I/O MMIO w/ MSI". This analysis really points in the direction of adding MSIs to virtio-mmio.
Cheers,
Stefano
---------- Forwarded message ---------- Date: Wed, 28 Oct 2020 18:41:09 +0000 From: Nathalie Chan King Choy via App-services app-services@lists.openampproject.org Reply-To: Nathalie Chan King Choy nathalie@xilinx.com To: "app-services@lists.openampproject.org" app-services@lists.openampproject.org Subject: [App-services] 2020-10-27 OpenAMP App-services call recording, notes, slides, and action items
Hi all,
The notes from yesterday’s OpenAMP App-services call can be found at:
https://github.com/OpenAMP/open-amp/wiki/OpenAMP-Application-Services-Subgro...
The link to the Webex recording is in the notes. I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks.
Please find attached the slides from Dan & Josh.
Action items:
Dan & Josh to send slides (DONE)
Stefano to start a thread w/ the folks who are working on shared memory & VirtIO
Best regards,
Nathalie C. Chan King Choy
Program Manager focused on Open Source and Community
Hi,
On Thu, Oct 29, 2020 at 10:26:32AM +0200, Dan Milea wrote:
Hi,
The virtio spec generated from this repository https://github.com/oasis-tcs/ virtio-spec includes a section dedicated to shared memory regions, described as "an additional facility available to devices that need a region of memory that’s continuously shared between the device and the driver, rather than passed between them in the way virtqueue elements are".
This version of the spec does not seem to have been released yet, although some time ago there was an attempt to use shared memory regions with virtio-fs ( https://lwn.net/Articles/774495/).
The most recent spec does use this shared memory extension in virtio-fs (commit ef5a7f405b95 ("virtio-fs: add DAX window") in the above virtio-spec repository). In particular it mandates that virtio-fs uses shared memory region 0 if present, so I guess an extension to share virtqueue and buffers memory would need a read-only register specifying which shared region is used for that, to avoid conflicts?
On 10/29/20 1:55 AM, Stefano Stabellini wrote: Dan Milea discussed the usage of virtio between heterogeneous clusters, i.e. virtio frontends on the Cortex-R cluster and virtio backends on the Cortex-A cluster. They used a setup based on pre-shared memory to make it work, which I believe is similar to the swiotlb approach we discussed in Stratos. They noted that the last version of the virtio spec on github had something in regards to virtio and pre-shared memory regions that might help us cover this use-case from a spec perspective.
I've considered having such an extension described externally [1] but reusing the virtio shared mem feature is interesting. Did you publish any driver or specification update yet? [1] https://op-lists.linaro.org/pipermail/stratos-dev/2020-October/000094.html
Thanks, Jean
app-services@lists.openampproject.org
-
Dan Milea -
Jean-Philippe Brucker -
Stefano Stabellini