Thanks for framing the problem clearly.
I’m top-posting this because I have a few high-level comments.
First, the clean restart process would be a lot simpler if our virtio transport layer supported Virtio v1.2’s vring reset mechanism. There has been a recent effort to provide this for vring PCI in the Linux kernel ([PATCH v7 00/26] virtio pci support VIRTIO_F_RING_RESET https://lore.kernel.org/all/20220308123518.33800-1-xuanzhuo@linux.alibaba.com/#r), but I don’t know how far along the path to merging it is. The Virtio 1.2 spec appears to have been in the very late stages of approval for some time.
Second, RPMsg over virtio is a different kind of device from any of the other devices in the virtio spec. It’s like MMIO, except that config space writes are not automatically caught, and the remoteproc framework adds some wrinkles. If someone is VERY patient, it might be nice to get an RPMsg device section into the next iteration of the spec.
Third, this needs to be “reversible”. The driver/device connection is inherently asymmetric, and the Virtio spec, and the Linux implementation tend to assume that they are the driver side. Some of our use cases need the MCU to be the driver and the rich OS to be the device. It appears there is some attention being paid to this in the system reference work, but I’m not following that very closely right now.
Fourth, the RPMsg protocol itself needs to be enhanced to support things like informing the sender that the receiver has been reset. Doing this in a backwards-compatible fashion will be interesting.
We need to think about all of these things to construct a more robust communications infrastructure.
Regards, Ed M
On Apr 20, 2022, at 3:28 PM, ben levinsky via Openamp-rp openamp-rp@lists.openampproject.org wrote:
Hi All,
As stated in a previous OpenAMP meeting, there was the use case of either the remote or host attempting to re-establish connection if one side goes down.
A proposal for this is as follows:
Changes to existing structures:
1.Add a new feature to the bitmap for VirtIO RPMsg with a name like "VIRTIO_RPMSG_F_RECOVERY" and an extra bit here
- Add a 2 bit field to struct rpmsg_device and struct fw_rsc_vdev to
denote if recovery functionality is supported and/or active that can have the following 3 values:
0 - recovery not supported
1 - recovery supported but RPMsg endpoint reconnection is not yet
allowed.
- 2 - recovery supported and RPMsg endpoint reconnection is allowed.
Changes to host's resource table parsing:
When a host parses the resource table of the remote processor, check if recovery feature is set and update the vdev's features accordingly.
Changes to host's endpoint creation:
- First setup, set struct fw_rsc_vdev reovery flag to "recovery
supported but reconnection is not yet allowed"
- After initial endpoint creation is successful, set recovery flag in
resource table's instance of the struct fw_rsc_vdev to "reconnection allowed" and set resource table's copy of the rpmsg_vdev so that the recovery field denotes reconnection is allowed
- If recovery is fully setup then upon subsequent host endpoint
creation
a. host will not re-initialize virtqueues b. if rpmsg address is already set, do not error out c. unlock rdev virtqueue lock so that messages can sendagain
Changes to remote's endpoint creation:
If recovery is fully setup then the following changes apply:
a. remote will not wait for virtqueues' initializationby host
b. if rpmsg address is already set, do not error out c. unlock rdev virtqueue lock so that messages can sendagain
Changes to rpmsg-send:
None. The virtqueue lock is opened upon endpoint re-connection.
Kind Regards,
Ben
Openamp-rp mailing list -- openamp-rp@lists.openampproject.org To unsubscribe send an email to openamp-rp-leave@lists.openampproject.org
Thanks for framing the problem clearly.
I’m top-posting this because I have a few high-level comments.
First, the clean restart process would be a lot simpler if our virtio transport layer supported Virtio v1.2’s vring reset mechanism. There has been a recent effort to provide this for vring PCI in the Linux kernel ([1][PATCH v7 00/26] virtio pci support VIRTIO_F_RING_RESET), but I don’t know how far along the path to merging it is. The Virtio 1.2 spec appears to have been in the very late stages of approval for some time.
Second, RPMsg over virtio is a different kind of device from any of the other devices in the virtio spec. It’s like MMIO, except that config space writes are not automatically caught, and the remoteproc framework adds some wrinkles. If someone is VERY patient, it might be nice to get an RPMsg device section into the next iteration of the spec.
Third, this needs to be “reversible”. The driver/device connection is inherently asymmetric, and the Virtio spec, and the Linux implementation tend to assume that they are the driver side. Some of our use cases need the MCU to be the driver and the rich OS to be the device. It appears there is some attention being paid to this in the system reference work, but I’m not following that very closely right now.
Fourth, the RPMsg protocol itself needs to be enhanced to support things like informing the sender that the receiver has been reset. Doing this in a backwards-compatible fashion will be interesting.
We need to think about all of these things to construct a more robust communications infrastructure.
Regards,
Ed M
On Apr 20, 2022, at 3:28 PM, ben levinsky via Openamp-rp <[2]openamp-rp@lists.openampproject.org> wrote:
Hi All, As stated in a previous OpenAMP meeting, there was the use case of either the remote or host attempting to re-establish connection if one side goes down. A proposal for this is as follows: Changes to existing structures: 1.Add a new feature to the bitmap for VirtIO RPMsg with a name like "VIRTIO_RPMSG_F_RECOVERY" and an extra bit here 2. Add a 2 bit field to struct rpmsg_device and struct fw_rsc_vdev to denote if recovery functionality is supported and/or active that can have the following 3 values: - 0 - recovery not supported - 1 - recovery supported but RPMsg endpoint reconnection is not yet allowed. - 2 - recovery supported and RPMsg endpoint reconnection is allowed. Changes to host's resource table parsing: When a host parses the resource table of the remote processor, check if recovery feature is set and update the vdev's features accordingly. Changes to host's endpoint creation: 1. First setup, set struct fw_rsc_vdev reovery flag to "recovery supported but reconnection is not yet allowed" 2. After initial endpoint creation is successful, set recovery flag in resource table's instance of the struct fw_rsc_vdev to "reconnection allowed" and set resource table's copy of the rpmsg_vdev so that the recovery field denotes reconnection is allowed 3. If recovery is fully setup then upon subsequent host endpoint creation a. host will not re-initialize virtqueues b. if rpmsg address is already set, do not error out c. unlock rdev virtqueue lock so that messages can send again Changes to remote's endpoint creation: 2. If recovery is fully setup then the following changes apply: a. remote will not wait for virtqueues' initialization by host b. if rpmsg address is already set, do not error out c. unlock rdev virtqueue lock so that messages can send again Changes to rpmsg-send: None. The virtqueue lock is opened upon endpoint re-connection. Kind Regards, Ben -- Openamp-rp mailing list -- [3]openamp-rp@lists.openampproject.org To unsubscribe send an email to [4]openamp-rp-leave@lists.openampproject.org
References
1. https://lore.kernel.org/all/20220308123518.33800-1-xuanzhuo@linux.alibaba.co... 2. mailto:openamp-rp@lists.openampproject.org 3. mailto:openamp-rp@lists.openampproject.org 4. mailto:openamp-rp-leave@lists.openampproject.org