path: root/open_issues/dde.mdwn

[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation,
Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]

[[!tag open_issue_hurd open_issue_gnumach]]

[[General Information|/dde]].

Still waiting for interface finalization and proper integration.

[[!toc]]

See [[user-space_device_drivers]] for generic discussion related to user-space
device drivers.


# Disk Drivers

Not yet supported.

The plan is to use [[libstore_parted]] for accessing partitions.


# Upstream Status


## IRC, freenode, #hurd, 2012-02-08

After the microkernel devroom at [[community/meetings/FOSDEM_2012]]:

    <antrik> there was quite some talk about DDE. I learnt that there are newer
      versions in Genode and in Minix (as opposed to the DROPS one we are
      using)
    <antrik> but apparently none of the guys involved is interested in creating
      a proper upstream project with central repository and communication
      channels :-(
    <antrik> the original DDE creator was also there, but said he isn't working
      on it anymore
    <tschwinge> OK, and the other two projects basically have their own forks.
    <tschwinge> Or are they actively cooperating?
    <tschwinge> (If you know about it.)
    <antrik> well, Genode is also part of the Dresden L4 group; but apart from
      that, I'd rather call it a fork...
    <antrik> hm... actually, I'm not sure anymore whether the guy I talked to
      was from Genode or Nova...
    <antrik> (both from the Dresdem L4 group)


### IRC, freenode, #hurd, 2012-08-12

    <antrik>
      http://genode.org/documentation/release-notes/12.05#Re-approaching_the_Linux_device-driver_environment
    <antrik> I wonder whether the very detailed explanation was prompted by our
      DDE discussions at FOSDEM...
    <pinotree> antrik: one could think about approaching them to develop the
      common dde libs + dde_linux together
    <antrik> pinotree: that's what I did at FOSDEM -- they weren't interested
    <pinotree> antrik: this year's one? why weren't they?
    <pinotree> maybe at that time dde was not integrated properly yet (netdde
      is just few months "old")
    <braunr> do you really consider it integrated properly ?
    <pinotree> no, but a bit better than last year
    <antrik> I don't see what our integration has to do with anything...
    <antrik> they just prefer hacking thing ad-hoc than having some central
      usptream
    <pinotree> the helenos people?
    <antrik> err... how did helenos come into the picture?...
    <antrik> we are talking about genode
    <pinotree> sorry, confused wrong microkernel OS
    <antrik> actually, I don't remember exactly who said what; there were
      people from genode there and from one or more other DDE projects... but
      none of them seemed interested in a common DDE
    <antrik> err... one or two other L4 projects


## IRC, freenode, #hurd, 2012-02-19

    <youpi> antrik: do we know exactly which DDE version Zheng Da took as a
      base ?
    <youpi> (so as to be able to merge new changes easily)
    <antrik> youpi: not sure... but from what I gathered at FOSDEM, the version
      he based on (from DROPS) is not really actively developed right now; if
      we want to go for newer versions, we probably have to look at other
      projects (like Genode or Nova or Minix)
    <youpi> there's no central project for dde ?
    <youpi> that sucks
    <antrik> no... and nobody seemed interested in having one :-(
    <youpi> pff
    <antrik> which makes me seriously question the original decision to build
      on DDE...
    <braunr> ..
    <antrik> if we have to basically maintain it ourselfs anyways, we could
      just as well have gone with custom glue
    <youpi> well, the advantage of DDE is that it already exists now
    <antrik> on the positive side, one of the projcets (not sure which)
      apparently have both USB and SATA working with some variant of DDE


### IRC, freenode, #hurd, 2012-11-03

    <mcsim> DrChaos: there is DDEUSB framework for L4. You could port it, if
      you want. It uses Linux 2.6.26 usb subsystem.


## IRC, freenode, #hurd, 2013-02-15

After the microkernel devroom at [[community/meetings/FOSDEM_2013]].

    <pinotree> youpi: speaking of dde, was there any will among other
      microkernel os developers to eventually develop one single dde (with
      every team handling the custom glue of the own kernel)?
    <youpi> well, there is still upstream dde actually
    <youpi> in dresden
    <youpi> nothing was really decided or anything (it was a round table, not a
      workgroup)
    <youpi> but conversation converged into sharing the DDE maintenance, yes
    <youpi> and dresden would be the logical central place
    <youpi> pb is that they don't have the habit of being very open
    <youpi> http://svn.tudos.org/repos/oc/tudos/trunk/l4/pkg/dde  has a recent
      enough version
    <youpi> which macsim confirmed having all the latest commits from the
      internal repository
    <pinotree> i see
    <youpi> so it seems a viable solution on the medium term
    <youpi> the long term might need a real visible open source project
    <youpi> but we should probably still keep basing on dresden work
    <youpi> (better take work being done anywhere)
    <pinotree> well, if the upstream is not really open, microkernel teams
      could just fork it and all work on it
    <youpi> that's what I mean
    <pinotree> should still be a win than everybody maintaining their own dde
    <youpi> sure
    <pinotree> ah yes, i was writing and i'm slow at it :)
    <youpi> but at least we can try to work with dresden
    <youpi> see how open they could become by just asking :)
    <pinotree> right


# IRC, OFTC, #debian-hurd, 2012-02-15

    <pinotree> i have no idea how the dde system works
    <youpi> gnumach patch to provide access to physical memory and interrupts
    <youpi> then userland accesses i/o ports by hand to drive things
    <youpi> but  that assumes that no kernel driver is interfering
    <youpi> so one has to disable kernel drivers
    <pinotree> how are dde drivers used? can they be loaded on their own
      automatically, or you have to settrans yourself to setup a device?
    <youpi> there's no autoloader for now
    <youpi> we'd need a bus arbitrer that'd do autoprobing

[[PCI_arbiter]].

    <pinotree> i see
    <pinotree> (you see i'm not really that low level, so pardon the flood of
      posssibly-noobish questions ;) )
    <youpi> I haven't set it up yet, but IIRC you need to specify which driver
      to be used
    <youpi> well, I mostly have the same questions actually :)
    <youpi> I just have some guesswork here :)
    <pinotree> i wonder whether the following could be feasible:
    <youpi> I'm wondering how we'll manage to make it work in d-i
    <pinotree> a) you create a package which would b-d on linux-source, build a
      selection of (network only for now) drivers and install them in
      /hurd/dde/
    <youpi> probably through a choice at the boot menu
    <youpi> I wouldn't dare depending on linux-source
    <youpi> dde is usually not up-to-date
    <pinotree> b) add a small utility over the actual fsys_settrans() which
      would pick the driver from /hurd/dde/
    <pinotree> ... so you could do `set-dde-driver b43 <device>` (or something
      like that)
    <youpi> we can provide something like b) yes
    <youpi> although documenting the settrans should be fine enough ;)
    <pinotree> the a) would help/ease with the fact that you need to compile on
      your own the drivers
    <pinotree> otherwise we would need to create a new linux-dde-sources-X.Y.Z
      only with the sources of the drivers we want from linux X.Y.Z
    <pinotree> (or hurd-dde-linux-X.Y.Z)
    <CIA-4> samuel.thibault * raccdec3 gnumach/debian/ (changelog
      patches/70_dde.patch patches/series): 
    <CIA-4> Add DDE experimental support
    <CIA-4> * debian/patches/70_dde.patch: Add experimental support for irq
      passing and
    <CIA-4> physical memory allocation for DDE. Also adds nonetdev boot
      parameter to
    <CIA-4> disable network device drivers.
    <youpi> ok, boots fine with the additional nonetdev option
    <youpi> now I need to try that dde hurd branch :)
    <CIA-4> samuel.thibault * rf8b9426 gnumach/debian/patches/70_dde.patch: Add
      experimental.defs to gnuamch-dev


# IRC, freenode, #hurd, 2012-02-19

    * youpi got dde almost working
    <youpi> it's able to send packets, but apparently not receive them
    <youpi> (e1000)
    <youpi> ok, rtl8139 works
    <youpi> antrik: the wiki instructions are correct
    <youpi> with e1000 I haven't investigated
    <antrik> (Archhurd guys also reported problems with e1000 IIRC... the one I
      built a while back works fine though on my T40p with real e1000 NIC)
    <antrik> maybe I should try with current versions... something might got
      broken by later changes :-(
    <youpi> at least testing could tell the changeset which breaks it
    <youpi> Mmm, it's very odd
    <youpi> with the debian package, pfinet's call to device_set_filter returns
      D_INVALID_OPERATION
    <youpi> and indeed devnode.c returns that
    <youpi> ah but it's libmachdev which is supposed to answer here
    <antrik> youpi: so, regarding the failing device_set_filter... I guess you
      are using some wrong combination of gnumach and pfinet
    <youpi> no it's actually that my pfinet was not using bpf
    <youpi> I've now fixed it
    <antrik> the DDE drivers rely on zhengda's modified pfinet, which uses
      devnode, but also switched to using proper BPF filters. so you also need
      his BPF additions/fixes in gnumach
    <antrik> OK
    <youpi> that's the latter
    <youpi> I had already fixed the devnode part
    <youpi> but hadn't seen that the filter was different
    <antrik> err... did I say gnumach? that of course doesn't come into play
      here
    <antrik> so yes, you just need a pfinet using BPF
    <youpi> libmachdev does ;)
    <antrik> I'm just using pfinet from zhengda's DDE branch... I think devnode
      and BPF are the only modifications
    <youpi> there's also a libpcap modification in the incubator
    <youpi> probably for tcpdump etc.
    <antrik> libpcap is used by the modified pfinet to compile the filter rule
    <youpi> why does pfinet need to compile the rule ?
    <youpi> it's libbpf which is used in the dde driver
    <antrik> it doesn't strictly need to... but I guess zhengda considered it
      more elegant to put the source rule in pfinet on compile it live, rather
      than the compiled blob
    <antrik> I probably discussed this with him myself a few years back... but
      my memory on this is rather hazy ;-)
    <antrik> err... and compile it live
    <youpi> ah, right, it's only used when asking pfinet to change its filter
    <youpi> but it does not need it for the default filter
    <youpi> which is hardcoded
    <antrik> I see
    <antrik> when would pfinet change its filter?
    * youpi now completely converting his hurd box to debian packages with dde
        support
    <youpi> on SIOCSIFADDR apparently
    <youpi> to set "arp or (ip host %s)",
    <antrik> well, that sounds like the default filter...
    <youpi> the default filter does not choose an IP
    <antrik> oh, right... pfinet has to readjust the filter when setting the IP
    <youpi> arg we lack support for kernel options for gnumach in update-grub
    <antrik> again, I have a vague recollection of discussing this
    * youpi crosses fingers
    <youpi> yay, works
    <antrik> so we *do* need libpcap in pfinet to set proper rules... though I
      guess it can also work with a static catchall rule (like it did before
      zhengda's changes), only less efficient
    <youpi> well in the past we were already catching everything anyway, so at
      least it's not a regression :)
    <antrik> right


# [[PCI_Arbiter]]

## IRC, freenode, #hurd, 2012-02-21

    <youpi> since all drivers need is interrupts, io ports and iomem
    <youpi> the latter was already available through /dev/mem
    <youpi> io ports through the i386 rpcs
    <youpi> the changes provide both interrupts, and physical-contiguous
      allocation
    <youpi> it should be way enough
    <braunr> youpi: ok
    <braunr> youpi: thanks for the details :)
    <antrik> braunr: this was mentioned in the context of the interrupt
      forwarding interface... the original one implemented by zhengda isn't
      suitable for a PCI server; but the ones proposed by youpi and tschwinge
      would work
    <antrik> same for the physical memory interface: the current implementation
      doesn't allow delegation; but I already said that it's wrong


# IRC, freenode, #hurd, 2012-02-20

    <youpi> I was a bit wary of including the ton of dde headers in hurd-dev
    <youpi> maybe adding another package for that
    <youpi> but that would have delayed introducing the dde binaries
    <youpi> probably we can do that for next upload
    <pinotree> i can try to work on it, if is feasible (ie if the dde drivers
      can currently be built from outside the hurd source tree)
    <youpi> it should be, it's a matter of pointing its makefile to a place
      where the make scripts and include headers are
    <youpi> (and the libraries)
    <pinotree> ok
    <antrik> youpi: you mean DDEKit headers?
    <antrik> pinotree: actually it doesn't matter where the dde-ified Linux
      drivers are built -- libdde_linux26 and the actual drivers use a
      completetly different build system anyways
    <antrik> in fact we concluded at some point that they should live in a
      separate repository -- but that change never happened
    <antrik> only the base stuff (ddekit, libmachdev etc.) belong in the Hurd
      source tree
    <youpi> antrik: yes
    <youpi> antrik: err, not really completely different
    <youpi> the actual drivers' Makefile include the libdde_linux26 mk files
    <youpi> the build itself is separate, though
    <antrik> youpi: yes, I mean both libdde_linux26 and the drivers use a build
      system that is completely distinct from the Hurd one
    <youpi> ah, yes
    <youpi> libdde_linux26 should however be shipped in the system
    <antrik> ideally libdde_linux26 and all the drivers should be built in one
      go I'd say...
    <youpi> it should be easily feasible to also have a separate driver too
    <youpi> e.g. to quickly try a 2.6 driver
    <antrik> youpi: I'm not sure about that. it's not even dynamically linked
      IIRC?...
    <youpi> with scripts to build it
    <youpi> it's not
    <youpi> but that doesn't mean it can't be separate
    <youpi> .a files are usually shipped in -dev packages
    <antrik> youpi: ideally we should try to come with a build system that
      reuses the original Linux makefile snippets to build all the drivers
      automatically without any manual per-driver work
    <youpi> there's usually no modification of the drivers themselves?
    <youpi> but yeah
    <youpi> "ideally", when somebody takes the time to do it
    <antrik> unfortunately, it's necessary to include one particular
      Hurd/DDE-specific header file in each driver :-(
    <youpi> can't it be done through gcc's -include option?
    <antrik> zhengda didn't find a way to avoid this... though I still hope
      that it must be possible somehow
    <antrik> I think the problem is that it has to be included *after* the
      other Linux headers. don't remember the details though
    <youpi> uh
    <youpi> well, a good script can add a  line after the last occurrence of
      #include
    <antrik> yeah... rather hacky, but might work
    <youpi> even with a bit of grep, tail, cut, and sed it should work :)
    <antrik> note that this is Hurd-specific; the L4 guys didn't need that
    <youpi> what is it?
    <antrik> don't remember off-hand


# IRC, freenode, #hurd, 2012-02-22

    <youpi> antrik: AIUI, it should be possible to include all network drivers
      in just one binary?
    <youpi> that'd permit to use it in d-i
    <youpi> and completely replace the mach drivers
    <youpi> we just need to make sure to include at least what the mach drivers
      cover
    <youpi> (all DDE network drivers, I mean)
    <youpi> of course that doesn't hinder from people to carefully separate
      drivers in several binaries if they wish
    <youpi> antrik: it does link at least, I'll give a try later
    <youpi> yes it works!
    <youpi> that looks like a plan
    <youpi> throw all network drivers in a /hurd/dde_net
    <youpi> settrans it on /dev/dde_net, and settrans devnode on /dev/eth[0-9]
    <youpi> I'm also uploading a version of hurd which includes headers &
      libraries, so you just need a make in dde_{e100,e1000,etc,net}
    <youpi> (uploading it with the dde driver itself :) )
    <youpi> btw, a nice thing is that we don't really care that all drivers are
      stuffed into a single binary, since it's a normal process only the useful
      pages are mapped and actually take memory :)
    <Tekk_> is that really a nice thing though? compared to other systems I
      mean
    <Tekk_> I know on linux it only loads the modules I need, for example. It's
      definitely a step up for hurd though :D
    <youpi> that's actually precisely what I mean
    <youpi> you don't need to load only the modules you need
    <youpi> you just load them all
    <youpi> and paging eliminates automatically what's not useful
    <youpi> even parts of the driver that your device will not need
    <Tekk_> ooh
    <Tekk_> awesome
    <youpi> (actually, it's not even loaded, but the pci tables of the drivers
      are loaded, then paged out)


# IRC, freenode, #hurd, 2012-02-24

    <youpi> antrik_: about the #include <ddekit/timer.h>, I see the issue, it's
      about jiffies
    <youpi> it wouldn't be a very good thing to have a jiffies variable which
      we'd have to update 100times per second
    <youpi> so ZhengDa preferred to make jiffies a macro which calls a function
      which reads the mapped time

[[Mapped-time_interface|master/microkernel/mach/gnumach/interface/device/time]].

    <youpi> however, that break any use of the work "jiffies", e.g. structure
      members & such
    <youpi> actually it's not only after headers that the #include has to be
      done, but after any code what uses the word "jiffies" for something else
      than the variable
    <youpi> pb is: it has to be done *before* any code that uses the word
      "jiffies" for the variable, e.g. inline functions in headers
    <youpi> in l4dde, there's already the jiffies variable so it's not a
      problem


# IRC, OFTC, #debian-hurd, 2012-02-27

    <tschwinge> I plan to do some light performance testing w.r.t. DDE
      Ethernet.  That is DDE vs. Mach, etc.
    <youpi> that'd be good, indeed
    <youpi> I'm getting 4MiB/s with dde
    <youpi> I don't remember with mach
    <tschwinge> Yes.  It just shouldn't regress too much.
    <tschwinge> Aha, OK.


## IRC, freenode, #hurd, 2012-02-27

    <youpi> tschwinge: nttcp tells me ~80Mbps for mach-rtl8139, ~72Mbps for
      dde-rtl8139, ~72Mbps for dde-e1000
    <youpi> civodul: ↑ btw
    <ArneBab> youpi: so the dde network device is not much slower than the
      kernel-one?
    <civodul> youpi: yes, looks good
    <ArneBab> rather almost the same speed
    <youpi> apparently
    <ArneBab> that’s quite a deal. 
    <ArneBab> what speed should it have as maximum?
    <ArneBab> (means: does the mach version get out all that’s possible?)
    <ArneBab> differently put: What speed would GNU/Linux get?
    <youpi> I'm checking that right now
    <ArneBab> cool!
    <ArneBab> we need those numbers for the moth after the next
    <youpi> Mmm, I'm not sure you really want the linux number :)
    <youpi> 1.6Gbps :)
    <ArneBab> oh…
    <youpi> let me check with udp rather than tcp
    <ArneBab> so the Hurd is a “tiny bit” worse at using the network… 
    <youpi> it might simply be an issue with tcp tuning in pfinet
    <ArneBab> hm, yes
    <ArneBab> tcp is not that cheap
    <ArneBab> and has some pretty advanced stuff for getting to high speeds
    <youpi> well, I'm not thinking about being cheap
    <youpi> but using more recent tuning
    <youpi> that does not believe only 1Mbps network cards exist :)
    <ArneBab> like adaptive windows and such?
    <ArneBab> :)
    <youpi> yes
    * ArneBab remembers that TCP was invented when the connections went over
        phone lines - by audio :)
    <youpi> yep
    <ArneBab> what’s the system load while doing the test?
    <youpi> yes, udp seems not so bad
    <ArneBab> ah, cool!
    <youpi> it's very variable (300-3000Mbps), but like on linux
    <ArneBab> that pushing it into user space has so low cost is pretty nice.
    * ArneBab thinks that that’s a point where Hurd pays off
    <youpi> that's actually what AST said to fosdem
    <youpi> he doesn't care about putting an RPC for each and every port i/o
    <youpi> because hardware is slow anyway
    <ArneBab> jupp
    <ArneBab> but it is important to see that in real life


# IRC, freenode, #hurd, 2012-04-01

    <youpi> antrik: I wonder whether you could actually not route the IRQs to a
      non-zero ring, AIUI you can in the x86 IDT table
    <antrik> youpi: you mean having a userspace server for each (non-timer)
      interrupt?
    <antrik> youpi: how would a userspace IRQ handler interact with the
      scheduler?
    <youpi> antrik: it doesn't necessarily have to
    <youpi> provided that it's trusted
    <antrik> youpi: how would you do CPU time accounting if there is no
      interaction with the scheduler?...
    <youpi> antrik: you don't necessarily want to care about it
    <antrik> youpi: well, that would mean that all drivers handling interrupts
      would have to be trusted to not use more than a very small part of CPU
      time...
    <youpi> yes
    <youpi> which is usually needed for interrupt handlers anyway
    <antrik> youpi: nah, the bottom handler only has to do very basic stuff;
      afterwards, we can pass off to "normal" driver processes, scheduled just
      like other processes... but that requires some interaction between the
      IRQ handler and the scheduler I think
    <youpi> the IRQ handler can wake up a thread, yes
    <youpi> no need for anything special there
    <antrik> so the userspace IRQ server would just decide what process to wake
      up, and then call the scheduler to do a normal task switch? I guess
      that's possible; but I'm not sure it would buy much...
    <youpi> it would permit userland to quickly react to the IRQ
    <youpi> such as acknowledge it to the hardware etc.
    <antrik> yeah, but my point is that I don't see much benefit in having this
      part of the code isolated in a userspace process... it has to be trusted
      anyways, and it's pretty trivial too
    <youpi> I never said it was a good idea


# IRC, freenode, #hurd, 2012-04-06

    <braunr> oh i forgot about my work on pcap
    <braunr> is devnode (or devopen or whatever) in the upstream repository now
      ?
    <antrik> can't say for sure, but I'd be surprised... don't remember seeing
      any movement in that regard :-(
    <braunr> wasn't it needed for dde ?
    <antrik> hm... good point


# IRC, freenode, #hurd, 2012-08-14

    <braunr> it's amazing how much code just gets reimplemented needlessly ...
    <braunr> libddekit has its own mutex, condition, semaphore etc.. objects
    <braunr> with the *exact* same comment about the dequeueing-on-timeout
      problem found in libpthread
    <braunr> *sigh*


## IRC, freenode, #hurd, 2012-08-18

    <braunr> hum, leaks and potential deadlocks in libddekit/thread.c :/


## IRC, freenode, #hurd, 2012-08-18

    <braunr> nice, dde relies on a race to start ..


## IRC, freenode, #hurd, 2012-08-21

In context of [[libpthread]].

    <braunr> hm, i thought my pthreads patches introduced a deadlock, but
      actually this one is present in the current upstream/debian code :/
    <braunr> (the deadlock occurs when receiving data fast with sftp)
    <braunr> either in netdde or pfinet


## IRC, freenode, #hurd, 2013-02-28

    <braunr> (which needs the same kinds of fixes that libpthread got)
    <braunr> actually i'm not sure why he didn't simply reuse the pthread
      functions :/
    <youpi> which kind of fixes?
    <youpi> cancellation?
    <braunr> timeouts
    <braunr> cancellation too but that's less an issue
    <youpi> I'm not sure it really needs timeout work
    <youpi> on what RPC?
    <youpi> pfinet is just using the mach interface
    <braunr> i don't know but it clearly copies some of the previous pthread
      code from pthread_cond_timedwait
    <braunr> see libddekit/thread.c:_sem_timedwait_internal
    <youpi> I recognize the comment indeed :)
    <youpi> I guess he thought he might need some particular semantic that
      libpthread may not provide
    <braunr> also, now that i think about it, he couldn't have used libpthread,
      could he ?
    <braunr> and there was no condition_timedwait in cthreads
    <braunr> there is a deadlock in netdde
    <braunr> it occurs sometimes, at high network speeds
    <braunr> (well high, 4 MiB/s or more)


# IRC, freenode, #hurd, 2012-08-18

    <braunr> hm looks like if netdde crashes, the kernel doesn't handle it
      cleanly, and we can't attach another netdde instance

[[!message-id "877gu8klq3.fsf@kepler.schwinge.homeip.net"]]


# DDE for Filesystems

## IRC, freenode, #hurd, 2012-10-07

    * pinotree wonders whether the dde layer could aldo theorically support
        also file systems
    <antrik> pinotree: yeah, I also brought up the idea of creating a DDE
      extension or DDE-like wrapper for Linux filesystems a while back... don't
      know enough about it though to decide whether it's doable
    <antrik> OTOH, I'm not sure it would be worthwhile. we still should
      probably have a native (not GPLv2-only) implementation for the main FS at
      least; so the wrapper would only be for accessing external
      partitions/media...


# virtio


## IRC, freenode, #hurd, 2012-07-01

    <braunr> hm, i haven't looked but, does someone know if virtio is included
      in netdde ?
    <youpi> braunr: nope, there's an underlying virtio layer needed before