Sorry - left off the mailing list.

-----Original Message----- From: Driscoll, Dan Sent: Wednesday, January 22, 2020 8:32 AM To: 'Tomas Evensen' tomase@xilinx.com; Bjorn Andersson bjorn.andersson@linaro.org; Rob Herring robh@kernel.org Cc: Stefano Stabellini stefanos@xilinx.com; Rob Herring Rob.Herring@arm.com; Raghuraman, Arvind Arvind_Raghuraman@mentor.com; Anjum, Etsam Etsam_Anjum@mentor.com; Humayun, Waqar Waqar_Humayun@mentor.com Subject: RE: [System-dt] software domains and top level nodes

Not sure if this will help and I know this is quite verbose, but, from what I see, we are converging on things that make sense to us at Mentor given our focus in this area.

We have been using a device tree based approach to system partitioning for the last 3-4 years and here are some points we have learned:

* To separate what we call the "system definition" (ie resource partitioning) from the hardware description, we have what we called a "System Definition Tree" or SDT file (found it kind of funny that SDT was also chosen for System Device Tree) * The SDT is a separate file that uses device tree syntax, but does NOT describe hardware, but rather sub-systems / partitioning using the hardware definition found in the DTS * The SDT file #includes the hardware description (ie top-level DTS file) and references nodes from this DT, so this keeps the 2 clearly separated (system definition versus hardware definition) * Related to the previous point, we don't use the chosen node to encompass the system definition and have our own bindings for "machine" nodes in the SDT (equivalent to "domain" nodes in current discussion) * I think putting all of this info in the chosen node doesn't really solve any real problems and just makes things confusing - having new bindings for system definition / domains / etc, that live outside the chosen node are, to me, much cleaner as the chosen node already has uses for different software that use DT * Each machine (domain) node has similar attributes as discussed in the thread below - memory, cpus, devices, chosen, etc - in addition to some other attributes we use to help determine how our tooling processes these machine nodes. * For instance, machines have a "role" attribute that indicates if a machine is a "virtual" machine (hypervisor uses), a "remote" machine (OpenAMP remote), a "master" machine (OpenAMP master), etc. * Obviously there are lots of permutations and combinations that can occur and need to be accommodated such as running a hypervisor on a Cortex-A SMP cluster and ALSO running an OpenAMP master / remote configuration between the Cortex-A cluster (could be a guest OS or could be the hypervisor) and a Cortex-R cluster * We also have other attributes we use to help package all of the deployable content (ie remote images, guest OS images, etc), but this is outside the scope of these discussions

So, there are 2 problems that need to be solved (as, I think, this group has been considering): 1. Adding necessary hardware description to current device trees so they FULLY describe the hardware (ie heterogeneous SoCs with different subsystems / clusters, device access, interrupt routing, sharing of memory / devices, etc) 2. How to define the partitioning of #1 so a tool can create multiple usable device trees for each software context in a system

I am not enough of an expert in #1 to help extensively here, but for #2, we have been doing this for 3-4 years now and have released commercial products that use device trees for this same purpose so hopefully we can help guide things here.

Our biggest problems right now are that #1 doesn't exist (ie we have been extending existing device trees for SoCs to fully describe them and we are doing this in a way that isn't "clean") and there are a few other areas in our machine definition / bindings that are flimsy as well.

I guess I would like to see us get #1 fully defined before talking too much about #2 as I think the hardware description should stand on its own (ie doesn't depend on any new bindings defined for #2).

Dan

-----Original Message----- From: System-dt [mailto:system-dt-bounces@lists.openampproject.org] On Behalf Of Tomas Evensen via System-dt Sent: Tuesday, January 21, 2020 7:43 PM To: Bjorn Andersson bjorn.andersson@linaro.org; Rob Herring robh@kernel.org Cc: Stefano Stabellini stefanos@xilinx.com; system-dt@lists.openampproject.org; Rob Herring Rob.Herring@arm.com Subject: Re: [System-dt] software domains and top level nodes

One of the things we have tried to achieve with System Device Trees is to make sure we separate the HW description from the domain configuration that typically is done by a different person. That is, you don't want to have to edit or rewrite the parts that describes the HW in order to describe what memory, devices, cpus goes where.

Take an example where 2 cpus can either be configured to a) work together and see the same memory/devices (SMP for example), or b) be separated into two different domains running different OSes with different meory/devices.

So you have either one or two domains for those two cpus.

In this case I don't know that you want the "configurer" to have to go in and rewrite the file to use a different number of domains depending on the situation.

FWIW,

Tomas

On 1/21/20, 3:57 PM, "System-dt on behalf of Bjorn Andersson via System-dt" <system-dt-bounces@lists.openampproject.org on behalf of system-dt@lists.openampproject.org> wrote:

EXTERNAL EMAIL

On Tue 21 Jan 13:18 PST 2020, Rob Herring via System-dt wrote: [..] > To flip all this around, what if domains become the top-level structure: > > domain@0 { > chosen {}; > cpus {}; > memory@0 {}; > reserved-memory {}; > }; > > domain@1 { > chosen {}; > cpus {}; > memory@800000 {}; > reserved-memory {}; > }; >

I like this suggestion, as this both creates a natural grouping and could allow for describing domain-specific hardware as subtrees in each domain.

Regards, Bjorn

> The content of all the currently top-level nodes don't need to change. > The OS's would be modified to treat a domain node as the root node > which shouldn't be very invasive. Then everything else just works as > is. > > This could still have other nodes at the (real) root or links from one > domain to another. I haven't thought thru that part, but I think this > structure can only help because it removes the notion that the root > has a specific cpu view. > > Rob > -- > System-dt mailing list > System-dt@lists.openampproject.org > https://lists.openampproject.org/mailman/listinfo/system-dt -- System-dt mailing list System-dt@lists.openampproject.org https://lists.openampproject.org/mailman/listinfo/system-dt