TLDR: virtio-remoteproc device reset
To request a device reset (w/ acknowledgement), the driver should
val = Status field // read current status field val |= 0x82 // set bits 7 and 1 (FAILED and DRIVER) val &= 0xfe // clear bit 0 (ACKNOWLEDGE) // leave other bits unchanged Status field = val // write the new status field value
The driver should then poll the status field waiting for it to be == 0.
Note: writing the status field to 0 when it is already non-zero, should continue to work as a device reset but the driver will not know when it is complete.
Background:
From the virtio spec [1] with extra clarity added by me.
Status Field bits: (Spec section 2.1, html [2]) bit 0, 0x01, dec 1 ACKNOWLEDGE bit 1, 0x02, dec 2 DRIVER bit 2, 0x04, dec 4 DRIVER_OK (also used in suspend protocol) bit 3, 0x08, dec 8 FEATURES_OK bit 4, 0x10, dec 16 SUSPEND only used if negotiated bit 5, 0x20, dec 32 (currently unused) bit 6, 0x40, dec 64 DEVICE_NEEDS_RESET bit 7, 0x80, dec 128 FAILED
All bits except DEVICE_NEEDS_RESET are set (and sometimes cleared) by the device. DEVICE_NEEDS_RESET is set by the device and should only be cleared by a reset.
Logically the driver sets ACKNOWLEDGE, then DRIVER, then DRIVER_OK, then FEATURES_OK. Some of these could be set at the same time but never higher bits before lower bits. Once these bits are set, the driver in not allowed to clear them except by issuing a device reset. The one exception to this is that the DRIVER_OK bit is cleared and set as part of the SUSPEND protocol but only if it has been negotiated.
The driver sets the FAILED bit when it has given up on the device. It is valid for FAILED with any of the following settings: 0b1xxx_0000 Fail before ack 0b1xxx_0001 Fail before driver found 0b1xxx_0011 Fail before driver OK 0b1xxx_0111 Fail before features OK 0b1xxx_1111 Fail after features OK
And if SUSPEND has been negotiated 0b1xx1_0011 Fail while suspend active before features OK 0b1xx1_1011 Fail while Suspend active after features OK
FAILED(0x80) and DRIVER (0x02) active with ACKNOWLEDGE(0x01) cleared is not a value that is possible in the base spec. Therefore we use this setting to request a reset w/ acknowledgement. 0b1xxx_xx10 Reset Request w/ acknowledgement
TBD:
Does/Should the driver send a notification on Status field change? Does/Should the device send a notification on Status field change?
Sending the notice eliminates the need for a time on the other side to poll. Do we already support this? If not, probibly not worth it just for this protocol.
Recent virtio-mmio enhancement:
The patch in [3] is now in the virtio spec master branch but is not part of any released version yet. The patch adds a new V3 version of virtio-mmio where the driver can issue a non-blocking reset request and poll the status field to know when it is done.
So we are not the only ones to notice this issue. However the solution used for virtio-mmio still requires trap and emulate to work and wont work if the Status field in located in normal memory of any type.
In the patch the driver still writes 0 to the status field but the device is allowed to return from the MMR write with the status filed non-zero. The driver then waits for the status field to become zero to know when this operation is complete.
This solution will not work for us as we use a real memory to store the status field and the driver writing that to 0 will take immediate effect and the driver then polling for zero would succeed immediately.
Bill
[1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html [2] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-110001 [3] https://github.com/Linaro/virtio-msg-spec/commit/a1883517ee44cc03d1b621a331d...
Thank You Bill.
Following is the original patch in the linux kernel that triggered this discussion:
https://lore.kernel.org/all/20260317201251.3920841-1-tanmay.shah@amd.com/
Two solutions were proposed during the community call:
1) Decide the unique value for the status field that indicates reset - and then after the actual device reset, the remote will set it to 0. (This is suggested in this email.)
2) Introduce new RSC_VDEV_V2 (new type of resource) that has transport specific method (such as new flag for the device reset). The firmware will poll on this new flag, and reset device when set and set the status to 0.
In either case, the remote is responsible to restore the original (default) resource table during device reset.
I will implement both approaches and post on the mailing list for further reviews. We can decide one way or another after reviewing the implementation.
Thanks, Tanmay
On 5/28/2026 6:03 PM, Bill Mills wrote:
TLDR: virtio-remoteproc device reset
To request a device reset (w/ acknowledgement), the driver should
val = Status field // read current status field val |= 0x82 // set bits 7 and 1 (FAILED and DRIVER) val &= 0xfe // clear bit 0 (ACKNOWLEDGE) // leave other bits unchanged Status field = val // write the new status field value
The driver should then poll the status field waiting for it to be == 0.
Note: writing the status field to 0 when it is already non-zero, should continue to work as a device reset but the driver will not know when it is complete.
Background:
From the virtio spec [1] with extra clarity added by me.
Status Field bits: (Spec section 2.1, html [2]) bit 0, 0x01, dec 1 ACKNOWLEDGE bit 1, 0x02, dec 2 DRIVER bit 2, 0x04, dec 4 DRIVER_OK (also used in suspend protocol) bit 3, 0x08, dec 8 FEATURES_OK bit 4, 0x10, dec 16 SUSPEND only used if negotiated bit 5, 0x20, dec 32 (currently unused) bit 6, 0x40, dec 64 DEVICE_NEEDS_RESET bit 7, 0x80, dec 128 FAILED
All bits except DEVICE_NEEDS_RESET are set (and sometimes cleared) by the device. DEVICE_NEEDS_RESET is set by the device and should only be cleared by a reset.
Logically the driver sets ACKNOWLEDGE, then DRIVER, then DRIVER_OK, then FEATURES_OK. Some of these could be set at the same time but never higher bits before lower bits. Once these bits are set, the driver in not allowed to clear them except by issuing a device reset. The one exception to this is that the DRIVER_OK bit is cleared and set as part of the SUSPEND protocol but only if it has been negotiated.
The driver sets the FAILED bit when it has given up on the device. It is valid for FAILED with any of the following settings: 0b1xxx_0000 Fail before ack 0b1xxx_0001 Fail before driver found 0b1xxx_0011 Fail before driver OK 0b1xxx_0111 Fail before features OK 0b1xxx_1111 Fail after features OK
And if SUSPEND has been negotiated 0b1xx1_0011 Fail while suspend active before features OK 0b1xx1_1011 Fail while Suspend active after features OK
FAILED(0x80) and DRIVER (0x02) active with ACKNOWLEDGE(0x01) cleared is not a value that is possible in the base spec. Therefore we use this setting to request a reset w/ acknowledgement. 0b1xxx_xx10 Reset Request w/ acknowledgement
TBD:
Does/Should the driver send a notification on Status field change? Does/Should the device send a notification on Status field change?
Sending the notice eliminates the need for a time on the other side to poll. Do we already support this? If not, probibly not worth it just for this protocol.
Recent virtio-mmio enhancement:
The patch in [3] is now in the virtio spec master branch but is not part of any released version yet. The patch adds a new V3 version of virtio-mmio where the driver can issue a non-blocking reset request and poll the status field to know when it is done.
So we are not the only ones to notice this issue. However the solution used for virtio-mmio still requires trap and emulate to work and wont work if the Status field in located in normal memory of any type.
In the patch the driver still writes 0 to the status field but the device is allowed to return from the MMR write with the status filed non-zero. The driver then waits for the status field to become zero to know when this operation is complete.
This solution will not work for us as we use a real memory to store the status field and the driver writing that to 0 will take immediate effect and the driver then polling for zero would succeed immediately.
Bill
[1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html [2] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-110001 [3] https://github.com/Linaro/virtio-msg-spec/commit/a1883517ee44cc03d1b621a331d...
All,
I have implemented solution for this problem which combines both following proposals.
It introduces RSC_VDEV_V2 new resource, which notifies status via setting unique value as proposed by Bill.
Here is the kernel patch for review: https://lore.kernel.org/all/20260902214453.634339-2-tanmay.shah@amd.com/
patchwork link for the same: https://patchwork.kernel.org/project/linux-remoteproc/patch/20260902214453.6...
TBD:
Does/Should the driver send a notification on Status field change? Does/Should the device send a notification on Status field change?
I have implemented notification from Driver -> Device in above patch, as it reduces need to polling on device side. However, Device -> Driver side notification is not needed. Because, Driver is polling with 3 second timeout for Device to set status to 0. We can reduce that timeout value to 1 second.
Feel free to review. If you prefer to see open-amp side changes as well, please let me know and I will open PR there as well.
Thank You, Tanmay
On 6/9/2026 4:35 PM, Shah, Tanmay wrote:
Thank You Bill.
Following is the original patch in the linux kernel that triggered this discussion:
https://lore.kernel.org/all/20260317201251.3920841-1-tanmay.shah@amd.com/
Two solutions were proposed during the community call:
- Decide the unique value for the status field that indicates reset -
and then after the actual device reset, the remote will set it to 0. (This is suggested in this email.)
- Introduce new RSC_VDEV_V2 (new type of resource) that has transport
specific method (such as new flag for the device reset). The firmware will poll on this new flag, and reset device when set and set the status to 0.
In either case, the remote is responsible to restore the original (default) resource table during device reset.
I will implement both approaches and post on the mailing list for further reviews. We can decide one way or another after reviewing the implementation.
Thanks, Tanmay
On 5/28/2026 6:03 PM, Bill Mills wrote:
TLDR: virtio-remoteproc device reset
To request a device reset (w/ acknowledgement), the driver should
val = Status field // read current status field val |= 0x82 // set bits 7 and 1 (FAILED and DRIVER) val &= 0xfe // clear bit 0 (ACKNOWLEDGE) // leave other bits unchanged Status field = val // write the new status field value
The driver should then poll the status field waiting for it to be == 0.
Note: writing the status field to 0 when it is already non-zero, should continue to work as a device reset but the driver will not know when it is complete.
Background:
From the virtio spec [1] with extra clarity added by me.
Status Field bits: (Spec section 2.1, html [2]) bit 0, 0x01, dec 1 ACKNOWLEDGE bit 1, 0x02, dec 2 DRIVER bit 2, 0x04, dec 4 DRIVER_OK (also used in suspend protocol) bit 3, 0x08, dec 8 FEATURES_OK bit 4, 0x10, dec 16 SUSPEND only used if negotiated bit 5, 0x20, dec 32 (currently unused) bit 6, 0x40, dec 64 DEVICE_NEEDS_RESET bit 7, 0x80, dec 128 FAILED
All bits except DEVICE_NEEDS_RESET are set (and sometimes cleared) by the device. DEVICE_NEEDS_RESET is set by the device and should only be cleared by a reset.
Logically the driver sets ACKNOWLEDGE, then DRIVER, then DRIVER_OK, then FEATURES_OK. Some of these could be set at the same time but never higher bits before lower bits. Once these bits are set, the driver in not allowed to clear them except by issuing a device reset. The one exception to this is that the DRIVER_OK bit is cleared and set as part of the SUSPEND protocol but only if it has been negotiated.
The driver sets the FAILED bit when it has given up on the device. It is valid for FAILED with any of the following settings: 0b1xxx_0000 Fail before ack 0b1xxx_0001 Fail before driver found 0b1xxx_0011 Fail before driver OK 0b1xxx_0111 Fail before features OK 0b1xxx_1111 Fail after features OK
And if SUSPEND has been negotiated 0b1xx1_0011 Fail while suspend active before features OK 0b1xx1_1011 Fail while Suspend active after features OK
FAILED(0x80) and DRIVER (0x02) active with ACKNOWLEDGE(0x01) cleared is not a value that is possible in the base spec. Therefore we use this setting to request a reset w/ acknowledgement. 0b1xxx_xx10 Reset Request w/ acknowledgement
TBD:
Does/Should the driver send a notification on Status field change? Does/Should the device send a notification on Status field change?
Sending the notice eliminates the need for a time on the other side to poll. Do we already support this? If not, probibly not worth it just for this protocol.
Recent virtio-mmio enhancement:
The patch in [3] is now in the virtio spec master branch but is not part of any released version yet. The patch adds a new V3 version of virtio-mmio where the driver can issue a non-blocking reset request and poll the status field to know when it is done.
So we are not the only ones to notice this issue. However the solution used for virtio-mmio still requires trap and emulate to work and wont work if the Status field in located in normal memory of any type.
In the patch the driver still writes 0 to the status field but the device is allowed to return from the MMR write with the status filed non-zero. The driver then waits for the status field to become zero to know when this operation is complete.
This solution will not work for us as we use a real memory to store the status field and the driver writing that to 0 will take immediate effect and the driver then polling for zero would succeed immediately.
Bill
[1] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html [2] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-110001 [3] https://github.com/Linaro/virtio-msg-spec/commit/a1883517ee44cc03d1b621a331d...
openamp-rp@lists.openampproject.org
-
Bill Mills -
Shah, Tanmay