[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] For [[DDE]]/[[Rump_kernel]]/X.org/... PCI stands for Peripheral Component Interconnect, and it is a hardware standard for allowing multiple devices to communicate together and with the rest of the system smoothly. This is a little bit like multiple people speaking over one telephone: only one person can talk at a time. In the same way, a software PCI Arbiter, ensures that every devices gets a chance to communicate and ensure maximum performance. A current example is Xorg, rump sound drivers, and netdde wifi drivers, etc. All need to access PCI config space at bootup. Currently, at bootup PCI access is sequential: gnumach, netdde, and then Xorg. The PCI Arbiter will eventually allow concurrent access to the PCI config space. The Hurd now has a PCI Arbiter, but it could use some more polishing. You can find its TODO file [[here|https://git.savannah.gnu.org/cgit/hurd/hurd.git/tree/pci-arbiter/TODO]]. Samuel also gave a presentation explaining some of the awesome possibilities that the PCI Arbiter can provide. You can watch his fosdem talk [[here|https://archive.fosdem.org/2018/schedule/event/microkernel_hurd_pci_arbiter/]]. The end goal is to allow different userland drivers to access PCI devices concurrently, while paving the way for fine-grain per-user, per-session, etc. permission management over PCI access, and IOMMUs allow us to do that very safely. Imagine a user being able to access a PCI card as a user! A IO-MMU can control that safely, because it's just like PCI passthrough with a hypervisor. # Documentation ## Accessing PCI Cards - `/servers/pci////` - provides `pci_conf_read/write`, `get_dev_regions`, `get_dev_rom` - The translator provides libpciaccess & pciutils backends ## Accessing PCI Cards as user This is a list of some things that we would like to do, but may not be able to currently. - Give PCI card access on the fly with - `fsysopts /servers/pci --uid 1234 --p 00:1f.3` - (or configure it permantly with settrans) - User app can then - Read/Write config - TODO: map resources / ROM - TODO: get i/o port access token ## An awesome example [[!img images/pci_possibilities.png]] In the above image, a user has complete control over his software stack. The PCI Arbiter on the far left has to run as root, because it needs to be able to access devices. Everything else has the exact permissions that the user wants. This user is running Xorg, and the TCP/IP stack as the user "nobody". Nobody is a uid that runs with essentially no permissions. It only has the permissions granted to it. This means one can run Xorg (which is not the most secure display server) in a confined way. Xorg won't even have permission to open a file, unless you allow it. In the above picture, the same is true for pfinit, and eth0. Things get more interesting, as we move along further to the right of the image. The user has decided that he wants to run pcm, the rump sound daemon, as a normal user, so he can better configure it, but there is a problem. The user doesn't trust the pcm daemon. The pcm daemon could open and read the user's gpg keys. The same is true for Firefox, since it may have flash or other non-safe code. Luckily the Hurd has a solution! Just start a [[subhurd|hurd/subhurd]]that contains firefox and pcm. Now the user can configure which files Firefox and pcm can access. There is one final detail. The graphic has two PCI arbiters. There is the system provided one on the far left, which the user is allowed to access. The user can then configure another PCI arbiter (on the far right) that allows pcm to only access the sound board. Of course, Firefox can still use pcm to play sound. # IRC, freenode, #hurd, 2012-02-19 antrik: we should probably add a gsoc idea on pci bus arbitration DDE is still experimental for now so it's ok that you have to configure it by hand, but it should be automatic at some ponit ## IRC, freenode, #hurd, 2012-02-21 i'm not familiar with the new gnumach interface for userspace drivers, but can this pci enumerator be written with it as it is ? (i'm not asking for a precise answer, just yes - even probably - or no) (idk or utsl will do as well) I'd say yes since all drivers need is interrupts, io ports and iomem the latter was already available through /dev/mem io ports through the i386 rpcs the changes provide both interrupts, and physical-contiguous allocation it should be way enough youpi: ok youpi: thanks for the details :) braunr: this was mentioned in the context of the interrupt forwarding interface... the original one implemented by zhengda isn't suitable for a PCI server; but the ones proposed by youpi and tschwinge would work same for the physical memory interface: the current implementation doesn't allow delegation; but I already said that it's wrong # IRC, freenode, #hurd, 2012-07-15 youpi: Oh, BTW, I keep meaning to ask you. Could sound be done with dde or would there still need to be some kernel work? bddebian: we'd need a PCI arbitrer for that for now just one userland poking with PCI is fine but two can produce bonks They can't use the same? that's precisely the matter they have to use the same and not poke with it themselves that's what an arbiter is for OK, so if we don't have a PCI arbiter now, how do things like netdde and video not collide currently? s/netdde/network/ or disk for that matter bddebian: ah currently, well currently, the network is the only thing using the pci bus How is that possible when I have a PCI video card and disk controller? they are accessed through compatible means I suppose one of the hardest parts is prioritization? i don't think it matters much, no bddebian: netdde and Xorg don't collide essentially because they are not started at the same time (hopefully) braunr: What do you mean it doesn't matter? bddebian: well the point is rather serializing access, we don't need more do other systems actually schedule access to the pci bus ? From what I am reading, yes ok # IRC, freenode, #hurd, 2012-07-16 youpi: the lack of a PCI arbiter is a problem, but I wounldn't consider it a precondition for adding another userspace driver class... it's up to the user to make sure he has only one class active, or take the risk of not doing so... (plus, I suspect writing the arbiter is a smaller task than implementing another DDE class anyways...) Where would the arbiter need to reside, in gnumach? bddebian: kernel would be one possible place (with the advantage of running both userspace and kernel drivers without the potential for conflicts) but I think I would prefer a userspace server antrik: we'd rather have PCI devices automatically set up just like /dev/netdde is already set up for the user so you can't count on the user for the arbitrer, it could as well be userland, while still interacting with the kernel for some devices we however "just" need to get disk drivers in userland to drop PCI drivers from kernel, actually # IRC, freenode, #hurd, 2012-07-17 youpi: So this PCI arbiter should be a hurd server? that'd be better youpi: Is there anything existing to look at as a basis? no idea off-hand I mean you couldn't take what netdde does and generalize it? netdde doesn't do any arbitration # IRC, OFTC, #debian-hurd, 2012-07-19 youpi: Well at some point if you ever have time I'd like to understand better how you see the PCI architecture working in Hurd. I.E. would you expect the server to do enumeration and arbitration? I'd expect both, yes, but that's probably to be discussed rather with antrik, he's the one who took some time to think about it netdde uses libpciaccess currently, right? yes libpciaccess would have to be fixed into using the arbitrer (that'd fix xorg as well) Man, I am still a bit unclear on how this all interacting currently.. :( currently it's not and it's just by luck that it doesn't break Long term xxxdde would use the new server, correct? (well, we are also sure that the gnumach enumeration comes always before the netdde enumeration, and xorg is currently not started automatically, so its enumeration is also always after that) yes the server would essentially provide an interface equivalent to libpciaccess Right In general, where does the pci map get "stored"? In GNU/Linux, is it all /proc based? what do you mean by "pci map" ? Once I have enumerated all of the buses and devices, does it stay stored or is it just redone for every call to a pci device? in linux it's stored in the kernel the abritrator would store it itself # IRC, freenode, #hurd, 2012-07-20 antrik: BTW, youpi says you are the one to talk to for design of a PCI server :) oh, am I? * antrik feels honoured :-) I guess it's true though: I *did* spent a little thought on it... even mentioned something in my thesis IIRC there is one tricky aspect to it though, which I'm not sure how to handle best: we need two different instances of libpciaccess Why two instances of libpciaccess? one used by the PCI server to access the hardware directly (using the existing port poking backend), and one using a new backend to access our PCI server... bddebian: hum, both i guess ? antrik: Why wouldn't the server access the hardware directly? I thought libpciaccess was supposed to be generic on purpose? hm... guess I wasn't clear the point is that the PCI server should use the direct hardware access backend of libpciaccess however, *clients* should use the PCI server backend of libpciaccess I'm not sure backends can be selected at runtime... which might mean that we actually have to compile two different versions of the library. erk. So you are saying the pci server should itself use libpci access rather than having it's own? admittedly, that's not the most fundamental design decision to make ;-) bddebian: yes. no need to rewrite (or copy) this code... Hmm actually that was the plan all along when I first suggested implementing the register poking backend for libpciaccess Hmm, not sure I like it but I am certainly in no position to question it right now :) why don't you like it ? I shouldn't need an Xorg specific library to access PCI on my OS :) oh Though I don't disagree that reinventing the wheel is a bit tedious. :) bddebian: although it originates from X.Org, I don't think there is anything about the library technically making it X-specific... yes that's my opinion too (well, there are some X-specific functions IIRC, but these do not hurt the other functionality) But what is there is api/abi breakage? :) s/is/if/ BTW according to rdepends there appear to be a number of non-X things using the library now like, uhm, hurd yeah, that too... we are already using it for DDE if you have deb-src lines in your sources.list, use the grep-dctrl power: grep-dctrl -sPackage -FBuild-Depends libpciaccess-dev /var/lib/apt/lists/*_source_Sources | sort -u I know we are using it for netdde. nice thing about it is that once we have the PCI server and an appropriate backend for libpciaccess, the same netdde and X binaries should work either with or without the PCI server Then why have the server at all? it's the arbiter you can use the library directly only if you're the only user and what antrik means is that the interface should be the same for both modes Ugh, that is where I am getting confused In that case shouldn't everything use libpciaccess and the PCI server has to arbitrate the requests? bd ? bddebian: yes bddebian: but they use the indirect version of the library whereas the server uses the raw version OK, I gotcha (I think) (but they both provide the same interface, so if you don't have a pci server and you know you're the only user, the direct version can be used) But I am not sure I see the difference between creating a second library or just moving the raw access to the PCI server :) uh, there is no difference in that and you shouldn't do it (if that's what antrik meant at least) if you can select the backend (raw or pci server) easily, then stick to the same code base That's where I struggle. In my worthless opinion, raw access should be the OS job while indirect access would be the libraries responsibility that's true but as an optimization, if an application is the only user, it can directly use raw access How would you know that? I'm sorry if these are dumb questions hum, don't try to make this behaviour automatic it would be selected by the user through command line switches But the OS itself uses PCI for things like disk access and video, no? (it could be automatic but it makes things more complicated) you don't need an arbiter all the time i can't tell you more, wait for antrik to return i realize i might already have said some bullshit bddebian: well, you have a point there that once we have the arbiter and use it for everthing, it isn't strictly useful to still have the register poking in the library however, the code will remain in the library anyways, so we better continue using it rather than introducing redundancy... but again, that's rather a side issue concerning the design of the PCI server antrik: Fair enough. :) So how would I even start on this? bddebian: actually, libpciaccess is a good starting point: checking the API should give you a fairly good idea what functionality the server needs to implement (+1 on library (re)use) antrik: KK sorry, I'm a bit busy right now...