[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] [[!tag open_issue_hurd open_issue_gnumach]] [[General Information|/dde]]. Still waiting for interface finalization and proper integration. [[!toc]] See [[user-space_device_drivers]] for generic discussion related to user-space device drivers. # Disk Drivers Not yet supported. The plan is to use [[libstore_parted]] for accessing partitions. # Upstream Status ## IRC, freenode, #hurd, 2012-02-08 After the microkernel devroom at [[community/meetings/FOSDEM_2012]]: there was quite some talk about DDE. I learnt that there are newer versions in Genode and in Minix (as opposed to the DROPS one we are using) but apparently none of the guys involved is interested in creating a proper upstream project with central repository and communication channels :-( the original DDE creator was also there, but said he isn't working on it anymore OK, and the other two projects basically have their own forks. Or are they actively cooperating? (If you know about it.) well, Genode is also part of the Dresden L4 group; but apart from that, I'd rather call it a fork... hm... actually, I'm not sure anymore whether the guy I talked to was from Genode or Nova... (both from the Dresdem L4 group) ### IRC, freenode, #hurd, 2012-08-12 http://genode.org/documentation/release-notes/12.05#Re-approaching_the_Linux_device-driver_environment I wonder whether the very detailed explanation was prompted by our DDE discussions at FOSDEM... antrik: one could think about approaching them to develop the common dde libs + dde_linux together pinotree: that's what I did at FOSDEM -- they weren't interested antrik: this year's one? why weren't they? maybe at that time dde was not integrated properly yet (netdde is just few months "old") do you really consider it integrated properly ? no, but a bit better than last year I don't see what our integration has to do with anything... they just prefer hacking thing ad-hoc than having some central usptream the helenos people? err... how did helenos come into the picture?... we are talking about genode sorry, confused wrong microkernel OS actually, I don't remember exactly who said what; there were people from genode there and from one or more other DDE projects... but none of them seemed interested in a common DDE err... one or two other L4 projects ## IRC, freenode, #hurd, 2012-02-19 antrik: do we know exactly which DDE version Zheng Da took as a base ? (so as to be able to merge new changes easily) youpi: not sure... but from what I gathered at FOSDEM, the version he based on (from DROPS) is not really actively developed right now; if we want to go for newer versions, we probably have to look at other projects (like Genode or Nova or Minix) there's no central project for dde ? that sucks no... and nobody seemed interested in having one :-( pff which makes me seriously question the original decision to build on DDE... .. if we have to basically maintain it ourselfs anyways, we could just as well have gone with custom glue well, the advantage of DDE is that it already exists now on the positive side, one of the projcets (not sure which) apparently have both USB and SATA working with some variant of DDE ### IRC, freenode, #hurd, 2012-11-03 DrChaos: there is DDEUSB framework for L4. You could port it, if you want. It uses Linux 2.6.26 usb subsystem. ## IRC, freenode, #hurd, 2013-02-15 After the microkernel devroom at [[community/meetings/FOSDEM_2013]]. youpi: speaking of dde, was there any will among other microkernel os developers to eventually develop one single dde (with every team handling the custom glue of the own kernel)? well, there is still upstream dde actually in dresden nothing was really decided or anything (it was a round table, not a workgroup) but conversation converged into sharing the DDE maintenance, yes and dresden would be the logical central place pb is that they don't have the habit of being very open http://svn.tudos.org/repos/oc/tudos/trunk/l4/pkg/dde has a recent enough version which macsim confirmed having all the latest commits from the internal repository i see so it seems a viable solution on the medium term the long term might need a real visible open source project but we should probably still keep basing on dresden work (better take work being done anywhere) well, if the upstream is not really open, microkernel teams could just fork it and all work on it that's what I mean should still be a win than everybody maintaining their own dde sure ah yes, i was writing and i'm slow at it :) but at least we can try to work with dresden see how open they could become by just asking :) right # IRC, OFTC, #debian-hurd, 2012-02-15 i have no idea how the dde system works gnumach patch to provide access to physical memory and interrupts then userland accesses i/o ports by hand to drive things but that assumes that no kernel driver is interfering so one has to disable kernel drivers how are dde drivers used? can they be loaded on their own automatically, or you have to settrans yourself to setup a device? there's no autoloader for now we'd need a bus arbitrer that'd do autoprobing [[PCI_arbiter]]. i see (you see i'm not really that low level, so pardon the flood of posssibly-noobish questions ;) ) I haven't set it up yet, but IIRC you need to specify which driver to be used well, I mostly have the same questions actually :) I just have some guesswork here :) i wonder whether the following could be feasible: I'm wondering how we'll manage to make it work in d-i a) you create a package which would b-d on linux-source, build a selection of (network only for now) drivers and install them in /hurd/dde/ probably through a choice at the boot menu I wouldn't dare depending on linux-source dde is usually not up-to-date b) add a small utility over the actual fsys_settrans() which would pick the driver from /hurd/dde/ ... so you could do `set-dde-driver b43 ` (or something like that) we can provide something like b) yes although documenting the settrans should be fine enough ;) the a) would help/ease with the fact that you need to compile on your own the drivers otherwise we would need to create a new linux-dde-sources-X.Y.Z only with the sources of the drivers we want from linux X.Y.Z (or hurd-dde-linux-X.Y.Z) samuel.thibault * raccdec3 gnumach/debian/ (changelog patches/70_dde.patch patches/series): Add DDE experimental support * debian/patches/70_dde.patch: Add experimental support for irq passing and physical memory allocation for DDE. Also adds nonetdev boot parameter to disable network device drivers. ok, boots fine with the additional nonetdev option now I need to try that dde hurd branch :) samuel.thibault * rf8b9426 gnumach/debian/patches/70_dde.patch: Add experimental.defs to gnuamch-dev # IRC, freenode, #hurd, 2012-02-19 * youpi got dde almost working it's able to send packets, but apparently not receive them (e1000) ok, rtl8139 works antrik: the wiki instructions are correct with e1000 I haven't investigated (Archhurd guys also reported problems with e1000 IIRC... the one I built a while back works fine though on my T40p with real e1000 NIC) maybe I should try with current versions... something might got broken by later changes :-( at least testing could tell the changeset which breaks it Mmm, it's very odd with the debian package, pfinet's call to device_set_filter returns D_INVALID_OPERATION and indeed devnode.c returns that ah but it's libmachdev which is supposed to answer here youpi: so, regarding the failing device_set_filter... I guess you are using some wrong combination of gnumach and pfinet no it's actually that my pfinet was not using bpf I've now fixed it the DDE drivers rely on zhengda's modified pfinet, which uses devnode, but also switched to using proper BPF filters. so you also need his BPF additions/fixes in gnumach OK that's the latter I had already fixed the devnode part but hadn't seen that the filter was different err... did I say gnumach? that of course doesn't come into play here so yes, you just need a pfinet using BPF libmachdev does ;) I'm just using pfinet from zhengda's DDE branch... I think devnode and BPF are the only modifications there's also a libpcap modification in the incubator probably for tcpdump etc. libpcap is used by the modified pfinet to compile the filter rule why does pfinet need to compile the rule ? it's libbpf which is used in the dde driver it doesn't strictly need to... but I guess zhengda considered it more elegant to put the source rule in pfinet on compile it live, rather than the compiled blob I probably discussed this with him myself a few years back... but my memory on this is rather hazy ;-) err... and compile it live ah, right, it's only used when asking pfinet to change its filter but it does not need it for the default filter which is hardcoded I see when would pfinet change its filter? * youpi now completely converting his hurd box to debian packages with dde support on SIOCSIFADDR apparently to set "arp or (ip host %s)", well, that sounds like the default filter... the default filter does not choose an IP oh, right... pfinet has to readjust the filter when setting the IP arg we lack support for kernel options for gnumach in update-grub again, I have a vague recollection of discussing this * youpi crosses fingers yay, works so we *do* need libpcap in pfinet to set proper rules... though I guess it can also work with a static catchall rule (like it did before zhengda's changes), only less efficient well in the past we were already catching everything anyway, so at least it's not a regression :) right # [[PCI_Arbiter]] ## IRC, freenode, #hurd, 2012-02-21 since all drivers need is interrupts, io ports and iomem the latter was already available through /dev/mem io ports through the i386 rpcs the changes provide both interrupts, and physical-contiguous allocation it should be way enough youpi: ok youpi: thanks for the details :) braunr: this was mentioned in the context of the interrupt forwarding interface... the original one implemented by zhengda isn't suitable for a PCI server; but the ones proposed by youpi and tschwinge would work same for the physical memory interface: the current implementation doesn't allow delegation; but I already said that it's wrong # IRC, freenode, #hurd, 2012-02-20 I was a bit wary of including the ton of dde headers in hurd-dev maybe adding another package for that but that would have delayed introducing the dde binaries probably we can do that for next upload i can try to work on it, if is feasible (ie if the dde drivers can currently be built from outside the hurd source tree) it should be, it's a matter of pointing its makefile to a place where the make scripts and include headers are (and the libraries) ok youpi: you mean DDEKit headers? pinotree: actually it doesn't matter where the dde-ified Linux drivers are built -- libdde_linux26 and the actual drivers use a completetly different build system anyways in fact we concluded at some point that they should live in a separate repository -- but that change never happened only the base stuff (ddekit, libmachdev etc.) belong in the Hurd source tree antrik: yes antrik: err, not really completely different the actual drivers' Makefile include the libdde_linux26 mk files the build itself is separate, though youpi: yes, I mean both libdde_linux26 and the drivers use a build system that is completely distinct from the Hurd one ah, yes libdde_linux26 should however be shipped in the system ideally libdde_linux26 and all the drivers should be built in one go I'd say... it should be easily feasible to also have a separate driver too e.g. to quickly try a 2.6 driver youpi: I'm not sure about that. it's not even dynamically linked IIRC?... with scripts to build it it's not but that doesn't mean it can't be separate .a files are usually shipped in -dev packages youpi: ideally we should try to come with a build system that reuses the original Linux makefile snippets to build all the drivers automatically without any manual per-driver work there's usually no modification of the drivers themselves? but yeah "ideally", when somebody takes the time to do it unfortunately, it's necessary to include one particular Hurd/DDE-specific header file in each driver :-( can't it be done through gcc's -include option? zhengda didn't find a way to avoid this... though I still hope that it must be possible somehow I think the problem is that it has to be included *after* the other Linux headers. don't remember the details though uh well, a good script can add a line after the last occurrence of #include yeah... rather hacky, but might work even with a bit of grep, tail, cut, and sed it should work :) note that this is Hurd-specific; the L4 guys didn't need that what is it? don't remember off-hand # IRC, freenode, #hurd, 2012-02-22 antrik: AIUI, it should be possible to include all network drivers in just one binary? that'd permit to use it in d-i and completely replace the mach drivers we just need to make sure to include at least what the mach drivers cover (all DDE network drivers, I mean) of course that doesn't hinder from people to carefully separate drivers in several binaries if they wish antrik: it does link at least, I'll give a try later yes it works! that looks like a plan throw all network drivers in a /hurd/dde_net settrans it on /dev/dde_net, and settrans devnode on /dev/eth[0-9] I'm also uploading a version of hurd which includes headers & libraries, so you just need a make in dde_{e100,e1000,etc,net} (uploading it with the dde driver itself :) ) btw, a nice thing is that we don't really care that all drivers are stuffed into a single binary, since it's a normal process only the useful pages are mapped and actually take memory :) is that really a nice thing though? compared to other systems I mean I know on linux it only loads the modules I need, for example. It's definitely a step up for hurd though :D that's actually precisely what I mean you don't need to load only the modules you need you just load them all and paging eliminates automatically what's not useful even parts of the driver that your device will not need ooh awesome (actually, it's not even loaded, but the pci tables of the drivers are loaded, then paged out) # IRC, freenode, #hurd, 2012-02-24 antrik_: about the #include , I see the issue, it's about jiffies it wouldn't be a very good thing to have a jiffies variable which we'd have to update 100times per second so ZhengDa preferred to make jiffies a macro which calls a function which reads the mapped time [[Mapped-time_interface|microkernel/mach/gnumach/interface/device/time]]. however, that break any use of the work "jiffies", e.g. structure members & such actually it's not only after headers that the #include has to be done, but after any code what uses the word "jiffies" for something else than the variable pb is: it has to be done *before* any code that uses the word "jiffies" for the variable, e.g. inline functions in headers in l4dde, there's already the jiffies variable so it's not a problem # IRC, OFTC, #debian-hurd, 2012-02-27 I plan to do some light performance testing w.r.t. DDE Ethernet. That is DDE vs. Mach, etc. that'd be good, indeed I'm getting 4MiB/s with dde I don't remember with mach Yes. It just shouldn't regress too much. Aha, OK. ## IRC, freenode, #hurd, 2012-02-27 tschwinge: nttcp tells me ~80Mbps for mach-rtl8139, ~72Mbps for dde-rtl8139, ~72Mbps for dde-e1000 civodul: ↑ btw youpi: so the dde network device is not much slower than the kernel-one? youpi: yes, looks good rather almost the same speed apparently that’s quite a deal. what speed should it have as maximum? (means: does the mach version get out all that’s possible?) differently put: What speed would GNU/Linux get? I'm checking that right now cool! we need those numbers for the moth after the next Mmm, I'm not sure you really want the linux number :) 1.6Gbps :) oh… let me check with udp rather than tcp so the Hurd is a “tiny bit” worse at using the network… it might simply be an issue with tcp tuning in pfinet hm, yes tcp is not that cheap and has some pretty advanced stuff for getting to high speeds well, I'm not thinking about being cheap but using more recent tuning that does not believe only 1Mbps network cards exist :) like adaptive windows and such? :) yes * ArneBab remembers that TCP was invented when the connections went over phone lines - by audio :) yep what’s the system load while doing the test? yes, udp seems not so bad ah, cool! it's very variable (300-3000Mbps), but like on linux that pushing it into user space has so low cost is pretty nice. * ArneBab thinks that that’s a point where Hurd pays off that's actually what AST said to fosdem he doesn't care about putting an RPC for each and every port i/o because hardware is slow anyway jupp but it is important to see that in real life # IRC, freenode, #hurd, 2012-04-01 antrik: I wonder whether you could actually not route the IRQs to a non-zero ring, AIUI you can in the x86 IDT table youpi: you mean having a userspace server for each (non-timer) interrupt? youpi: how would a userspace IRQ handler interact with the scheduler? antrik: it doesn't necessarily have to provided that it's trusted youpi: how would you do CPU time accounting if there is no interaction with the scheduler?... antrik: you don't necessarily want to care about it youpi: well, that would mean that all drivers handling interrupts would have to be trusted to not use more than a very small part of CPU time... yes which is usually needed for interrupt handlers anyway youpi: nah, the bottom handler only has to do very basic stuff; afterwards, we can pass off to "normal" driver processes, scheduled just like other processes... but that requires some interaction between the IRQ handler and the scheduler I think the IRQ handler can wake up a thread, yes no need for anything special there so the userspace IRQ server would just decide what process to wake up, and then call the scheduler to do a normal task switch? I guess that's possible; but I'm not sure it would buy much... it would permit userland to quickly react to the IRQ such as acknowledge it to the hardware etc. yeah, but my point is that I don't see much benefit in having this part of the code isolated in a userspace process... it has to be trusted anyways, and it's pretty trivial too I never said it was a good idea # IRC, freenode, #hurd, 2012-04-06 oh i forgot about my work on pcap is devnode (or devopen or whatever) in the upstream repository now ? can't say for sure, but I'd be surprised... don't remember seeing any movement in that regard :-( wasn't it needed for dde ? hm... good point ## IRC, freenode, #hurd, 2013-09-20 i should take some time to integrate my pcap changes into the libpcap debian package at least braunr: if upstream is active, i'd say to go there directly the problem with that approach is that netdde is still not part of our upstream code don't understand the relation i don't want to send the pcap guys code for an interface that is still not considered upstream ... # IRC, freenode, #hurd, 2012-08-14 it's amazing how much code just gets reimplemented needlessly ... libddekit has its own mutex, condition, semaphore etc.. objects with the *exact* same comment about the dequeueing-on-timeout problem found in libpthread *sigh* ## IRC, freenode, #hurd, 2012-08-18 hum, leaks and potential deadlocks in libddekit/thread.c :/ ## IRC, freenode, #hurd, 2012-08-18 nice, dde relies on a race to start .. ## IRC, freenode, #hurd, 2012-08-21 In context of [[libpthread]]. hm, i thought my pthreads patches introduced a deadlock, but actually this one is present in the current upstream/debian code :/ (the deadlock occurs when receiving data fast with sftp) either in netdde or pfinet ## IRC, freenode, #hurd, 2013-02-28 (which needs the same kinds of fixes that libpthread got) actually i'm not sure why he didn't simply reuse the pthread functions :/ which kind of fixes? cancellation? timeouts cancellation too but that's less an issue I'm not sure it really needs timeout work on what RPC? pfinet is just using the mach interface i don't know but it clearly copies some of the previous pthread code from pthread_cond_timedwait see libddekit/thread.c:_sem_timedwait_internal I recognize the comment indeed :) I guess he thought he might need some particular semantic that libpthread may not provide also, now that i think about it, he couldn't have used libpthread, could he ? and there was no condition_timedwait in cthreads there is a deadlock in netdde it occurs sometimes, at high network speeds (well high, 4 MiB/s or more) # IRC, freenode, #hurd, 2012-08-18 hm looks like if netdde crashes, the kernel doesn't handle it cleanly, and we can't attach another netdde instance [[!message-id "877gu8klq3.fsf@kepler.schwinge.homeip.net"]] # DDE for Filesystems ## IRC, freenode, #hurd, 2012-10-07 * pinotree wonders whether the dde layer could aldo theorically support also file systems pinotree: yeah, I also brought up the idea of creating a DDE extension or DDE-like wrapper for Linux filesystems a while back... don't know enough about it though to decide whether it's doable OTOH, I'm not sure it would be worthwhile. we still should probably have a native (not GPLv2-only) implementation for the main FS at least; so the wrapper would only be for accessing external partitions/media... # virtio ## IRC, freenode, #hurd, 2012-07-01 hm, i haven't looked but, does someone know if virtio is included in netdde ? braunr: nope, there's an underlying virtio layer needed before ## IRC, freenode, #hurd, 2013-07-24 btw, I'd love to see libvirt support in hurd I tried to hack up a dde based net translator afaics they are very much like any other pci device, so the infrastructure should be there if anything I expect the libvirt stuff to be more easily portable what do you mean by "a dde based net translator" ? ah, you mean virtio support in netdde ? yes virtio net is present in the kernel version we use for the dde drivers so I just copied the dde driver over, but I had no luck compiling it ok, but what would be the benefice over e1000 & co? any of the dde drivers btw youpi: less overhead e1000 is already low overhead actually there are less and less differences in strategies for driving a real board, and a virtual one we are seeing shared memory request buffer, dma, etc. in real boards which ends up being almost exactly what virtio does :) ahci, for instance, really looks extremely like a virtio interface (I know, it's a disk, but that's the same idea, and I do know what I'm talking about here :) ) that would actually be my next wish, a virtio disk driver, and virt9p ;) on the other hand, i wouldn't spend much time on a virtio disk driver for now the hurd as it is can't boot on a device that isn't managed by the kernel we'd need to change the boot protocol ok, I wasn't planning to, just wanted to see if I can easily hack up the virtio-net translator well, as youpi pointed, there is little benefit to that as well but if that's what you find fun, help yourself :) I didn't know that, I assumed there was some value to the virtio stuff there is but relatively to other improvements, it's low ## IRC, freenode, #hurd, 2013-09-14 I'm slowly beginning to understand the virtio driver framework after reading Rusty's virtio paper and the Linux sources of a few virtio drivers. Has anyone started working on virtio drivers yet? rekado: nobody has worked on virtio drivers, as I know of youpi: I'm still having a hard time figuring out where virtio would fit in in the hurd. I'm afraid I don't understand how drivers in the hurd work at all. Will part of this have to be implemented in Mach? rekado: it could be implemented either as a Mach driver, or as a userland driver better try the second alternative i.e. as a translator sitting on e.g. /dev/eth0 or /dev/hd0 ## IRC, freenode, #hurd, 2013-09-18 To get started with virtio I'd like to write a simple driver for the entropy device which appears as a PCI device when running qemu with -device virtio-rng-pci . why entropy ? because it's the easiest. is it ? the driver itself may be, but integrating it within the system probably isn't It uses the virtio framework but only really consists of a read-only buffer virtqueue you're likely to want something that can be part of an already existing subsystem like networking All the driver has to do is push empty buffers onto the queue and pass the data it receives back from the host device to the client The thing about existing subsystems is: I don't really understand them enough. I understand virtio, though. but isn't your goal understanding at least one ? yes. then i suggest working on virtio-net and making it work in netdde But to write a virtio driver for network I must first understand how to actually talk to the host virtio driver/device. rekado: why ? There is still a knowledge gap between what I know about virtio and what I have learned about the Hurd/Mach. are you trying to learn about virtio or the hurd ? both, because I'd like to write virtio drivers for the hurd. hm no with virtio drivers pass buffers to queues and then notify the host. you may want it, but it's not what's best for the project oh. what's best is reusing existing drivers we're much too far from having enough manpower to maintain our own you mean porting the linux virtio drivers? there already is a virtio-net driver in linux 2.6 so yes, reuse it the only thing which might be worth it is a gnumach in-kernel driver for virtio block devices because currently, we need our boot devices to be supported by the kernel itself ... when I boot the hurd with qemu and the entropy device I see it as an unknown PCI device in the output of lspci. that's just the lspci database which doesn't know it Well, does this mean that I could actually talk to the device already? E.g., through libpciaccess? I'm asking because I don't understand how exactly devices "appear" on the Hurd. it's one of the most difficult topic currently you probably can talk to the device, yes but there are issues with pci arbitration * rekado takes notes: "pci arbitration" so, this is about coordinating bus access, right? yes i'm not a pci expert so i can't tell you much more heh, okay. what kind of "issues with pci arbitration" are you referring to, though? Is this due to something that Mach isn't doing? ideally, mach doesn't know about pci the fact we still need in-kernel drivers for pci devices is a big problem we may need something like a pci server in userspace on l4 system it's called an io server How do in-kernel drivers avoid these issues? they don't Or rather: why is it they don't have these issues? they do oh. we had it when youpi added the sata driver so currently, all drivers need to avoid sharing common interrupts for example again, since i'm not an expert about pci, i don't know more about the details pci arbitrations are made by hardware ... no ? Hooligan0: i don't know i'm not merely talking about bus mastering here simply preventing drivers from mapping the same physical memory should be enforced somewhere i'm not sure it is same for irq sharing braunr : is the support for boot devices into the kernel is really needed if a loader put servers into the memory before starting mach ? Hooligan0: there is a chicken-and-egg problem during boot, whatever the solution obviously, we can preload from memory, but then you really want your root file system to use a disk Hooligan0: the problem with preloading from memory is that you want the root file system to use a real device the same way / refers to one on unix so you have an actual, persistent hierarchy from which the system can be initialized and translators started you also want to share as much as possible between the early programs and the others so for example, both the disk driver and the root file system should be able to use the same libc instance this requires a "switch root" mechanism that needs to be well defined and robust otherwise we'd just build our drivers and root fs statically (which is currently done with rootfs actually) and this isn't something we're comfortable with so for now, in-kernel drivers humm ... disk driver and libc ... i see in other way ... disk drivers can use only a little number of lib* functions ; so with a static version, a bit of memory is lots s/lots/lost and maybe the driver can be hot-replaced after boot (ok ok, it's more simple to say than to write)