Revert "rename open_issues.mdwn to service_solahart_jakarta_selatan__082122541663.mdwn"

This reverts commit 95878586ec7611791f4001a4ee17abf943fae3c1.

5 files changed, 1621 insertions, 0 deletions

diff --git a/open_issues/virtualization/capsicum.mdwn b/open_issues/virtualization/capsicum.mdwn
new file mode 100644
index 00000000..44503e34
--- /dev/null
+++ b/open_issues/virtualization/capsicum.mdwn
@@ -0,0 +1,22 @@
+[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+*Capsicum - practical capabilities for UNIX*
+
+<http://www.cl.cam.ac.uk/research/security/capsicum/>
+
+<http://www.lightbluetouchpaper.org/2010/08/12/capsicum-practical-capabilities-for-unix/>
+(server disappeared; [Google
+cache](http://webcache.googleusercontent.com/search?q=cache:cCAqjWOhhksJ:www.lightbluetouchpaper.org/2010/08/12/capsicum-practical-capabilities-for-unix/))
+
+<http://lackingrhoticity.blogspot.com/2010/10/process-descriptors-in-freebsd-capsicum.html>
+
+<http://www.cl.cam.ac.uk/research/security/capsicum/slides/20100811-usenix-capsicum.pdf>
+/ <http://www.youtube.com/watch?v=raNx9L4VH2k>
diff --git a/open_issues/virtualization/fakeroot.mdwn b/open_issues/virtualization/fakeroot.mdwn
new file mode 100644
index 00000000..441d5c13
--- /dev/null
+++ b/open_issues/virtualization/fakeroot.mdwn
@@ -0,0 +1,1330 @@
+[[!meta copyright="Copyright © 2010, 2013, 2014 Free Software Foundation,
+Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+
+# IRC, freenode, #hurd, 2013-02-26
+
+    <youpi> btw, about fakeroot-hurd
+    <youpi> the remaining issue I see is with argv[0] (yes, again...)
+
+
+## IRC, freenode, #hurd, 2013-04-03
+
+    <youpi> btw, I believe our fakeroot-hurd is close to working actually
+    <youpi> it's just a argv[0] issue supposed to be fixed by exec_file_name
+      but apparently not fixed in that case, for some reason
+
+[[glibc#execve_relative_paths]].
+
+
+## IRC, freenode, #hurd, 2013-08-26
+
+    < teythoon> also I looked into the fakeroot issue, aiui the problem is that
+      scripts are not handled correctly, right?
+    < teythoon> the exec server fails to locate the scripts file name, and so
+      it hands the file_t to the interpreter process and passes /dev/fds/3 as
+      script name
+    < teythoon> afaics that breaks e.g. python
+    < youpi> yes
+    < youpi> pinotree's exec_file_name is supposed to fix that, but for some
+      reason it doesn't work here
+
+[[glibc#execve_relative_paths]].
+
+    < pinotree> it was pochu's, not mine
+    < youpi> ah, right
+    < teythoon> ah I see, I was wondering about that
+    < pochu> it was working for a long time, wasn't it?
+    < pochu> and only stopped working recently
+    < youpi> did it completely stop?
+    < youpi> I have indeed seen odd issues
+    < youpi> I haven't actually  checked whether it has completely stopped
+      working
+    < youpi> probably worth looking there first
+    < pinotree> gtk+3.0 fails, but other stuff like glib2.0 and gtester-using
+      stuff works
+    < teythoon> huh? I created tests like "#!/bin/sh\necho $0" and that says
+      /dev/fd..., and a python script doing the same doesn't even run, so how
+      can it work for a package build?
+    < youpi> it works for me in plain bash
+    < youpi> #!/bin/sh
+    < youpi> echo $0
+    < youpi> € $PWD/test.sh
+    < youpi> /home/samy/test.sh
+    < teythoon> it does !?
+    < youpi> yes
+    < youpi> not in fakeroot-hurd however, as we said
+    < teythoon> well, obviously it works when not being run under
+      fakeroot-hurd, yes
+    < youpi> ok, so we weren't talking about the same thing
+    < youpi> a mere shell script doesn't work in fakeroot-hurd indeed
+    < youpi> that's why we still use fakeroot-sysv
+    < teythoon> right
+    < youpi> err, -tcp
+
+
+## IRC, freenode, #hurd, 2013-11-18
+
+    <teythoon> I believe I figured out the argv[0] issue with fakeroot-hurd
+    <teythoon> but I'm not sure how to fix this
+    <teythoon> first of all, Emilios file_exec_file_name patch set works fine
+
+[[glibc#execve_relative_paths]].
+
+    <teythoon> but not with fakeroot
+    <teythoon>
+      http://git.sceen.net/hurd/hurd.git/blob/HEAD:/exec/hashexec.c#l300
+    <teythoon> check_hashexec tries to locate the script file using a heuristic
+    <teythoon> Emilios patch improves the situation with just providing this
+      information
+    <teythoon> but then, the identity port of the "discovered" file is compared
+      with the id port of the script file
+    <teythoon> to verify if the heuristic found the right file
+    <teythoon> but when using fakeroot-hurd, /hurd/fakeroot proxies all
+      requests
+    <teythoon> but the exec server is outside of the /hurd/fakeroot
+      environment, so it gets the id port from the real filesystem
+    <teythoon> we could skip that test if the script name is explicitly
+      provided though
+    <teythoon> that test was meant to see whether a search through $PATH turned
+      up the right file
+    <braunr> teythoon: nice
+    <teythoon> braunr: thanks :)
+    <teythoon> unfortunately, dpkg-buildpackaging hurd with it still fails for
+      some reason
+    <teythoon> but it is faster than fakeroot-tcp :)
+    <braunr> even chown ?
+    <braunr> or chmod ?
+    <teythoon> dunno in detail, but the whole build is faster
+    <braunr> if you can try it, i'm interested
+    <braunr> because chown/chmod is also slow on linux with fakeroot-tcp
+    <teythoon> i can try...
+    <braunr> so it's probably not a hurd bug
+    <teythoon> braunr: yes, it really is
+    <braunr> no i mean
+    <braunr> chown/chmod being slow with fakeroot-tcp is probably not a hurd
+      bug
+    <braunr> but a fakeroot-tcp bug
+    <teythoon> chowning all files in /usr/bin takes 5.930s with fakeroot-hurd
+      (6.09 with startup overhead) vs 26.42s (26.59s) with fakeroot-tcp
+    <braunr> but try it on linux (fakeroot-tcp i mean)
+    <braunr> although you may want to do it on something you don't care much
+      about :p)
+
+
+## IRC, freenode, #hurd, 2013-12-03
+
+    * teythoon is gonna hunt a fakeroot bug ...
+    <teythoon> % fakeroot-hurd /bin/sh -c ":> /tmp/some_file"
+    <teythoon> /bin/sh: 1: cannot create /tmp/some_file: Is a directory
+    <braunr> ah fakeroot-hurd
+    <teythoon> prevents installing stuff with /bin/install
+    <teythoon> sure fakeroot-hurd, why would i work on the slow one ?
+    <braunr> i don't know
+    <braunr> because it makes chmod/chown/maybe others horrenddously slow
+    <braunr> ?
+    <teythoon> yes, fixing this involves fixing fakeroot-hurd
+    <braunr> are you sure ?
+    <braunr> i prefer repeating just in case: i saw that problem on linux as
+      well
+    <braunr> with fakeroot-sysv
+    <teythoon> so ?
+    <braunr> i'm almost certain it's a pure fakeroot bug, not a hurd bug
+    <braunr> so
+    <teythoon> even if this is fixed, it still has to pay the socket
+      communication overhead
+    <braunr> fixing fakeroot-hurd so that i can be used instead of fakeroot-tcp
+      is a very good thing to do, obviously
+    <braunr> it*
+    <braunr> but it won't solve the chown/chmod speed
+    <braunr> (or, probably won't)
+    <teythoon> huh, why not ?
+    <braunr> 15:53 < braunr> i'm almost certain it's a pure fakeroot bug, not a
+      hurd bug
+    <braunr> when i say it's slow, i should be more precise
+    <braunr> it doesn't show up in top
+    <teythoon> yes, but why would fakeroot-hurd suffer from the same issue ?
+    <braunr> the cpu is almost idle
+    <braunr> oh right, it's a completely different tool
+    <braunr> my bad
+    <braunr> right, right, the proper way to implement fakeroot actually :)
+    <teythoon> yes
+    <teythoon> this will bring near-native speed
+
+
+## IRC, freenode, #hurd, 2013-12-05
+
+    <teythoon> fakeroot-hurd just successfully built mig :)
+    <teythoon> hangs in dh_gencontrol when building gnumach or hurd though
+    <teythoon> i believe it hangs waiting for a lock
+    <teythoon> lock like in file lock that is
+    <teythoon> braunr: no more room for vm_map_find_entry in 80220a40
+    <teythoon> 80220a40 <- is that a task ?
+    <braunr> or a vm_map, not sure
+    <braunr> probably a vm_map
+
+
+## IRC, freenode, #hurd, 2013-12-06
+
+    <teythoon> well, aren't threads a source of endless entertainment ... ?
+    <teythoon> well, I found three more bugs in fakeroot-hurd
+    <teythoon> one of them requires fixing the locking used in fakeroot
+    <braunr> ouch
+    <teythoon> the current code does some lock cycling to aquire a lock out of
+      order
+    <braunr> cycling ?
+    <teythoon> in the netfs_node_norefs function
+    <teythoon> release and reaquire
+    <braunr> i see
+    <teythoon> which imho should be better solved with a weak reference
+    <teythoon> working on it, it no longer deadlocks but i broke something else
+      ...
+    <teythoon> endless fun ;)
+    <braunr> such things could have been done right in the beginning
+    <braunr> ...
+    <teythoon> yes, I wonder
+    <teythoon> libports has weak references
+    <teythoon> but pflocal is the only user
+    <braunr> hm
+    <teythoon> none of the lib*fs support that
+    <braunr> didn't i add one in libdiskfs too ?
+    <braunr> anyway, irrelevant
+    <braunr> weak references are a nice feature
+    <braunr> teythoon: i don't see the cycling you mentioned
+    <braunr> only netfs_node_refcnt_lock being dropped temporarily
+    <teythoon> yep, that one
+    <teythoon> line 145
+    <teythoon> note that due to another bug this code is currently never run
+    <braunr> how surprising ..
+    <braunr> the note about some leak actually gave a hint about that
+    <teythoon> yeah, that leak
+    <teythoon> I think i'm actually very close
+    <teythoon> it's just so frustrating, i thought i got it last night
+    <braunr> good luck then
+    <teythoon> thanks :)
+
+
+## IRC, freenode, #hurd, 2013-12-09
+
+    <teythoon> sweet, i fixed fakeroot-hurd :)
+    <braunr>  /clap
+    <braunr> what was the problem ?
+    <teythoon> lots
+    <braunr> i see
+    <teythoon> it's amazing it actually run as well as it did
+    <braunr> mess strikes again
+    <braunr> i hate messy code ..
+    * teythoon is building half a hurd package using this ... stay tuned ;)
+    <azeem> teythoon: is this going to make building faster as well?
+    <teythoon> most likely, yes
+    <teythoon> fakeroot-tcp is known to be slow, even on linux
+    <braunr> teythoon: are you sure about the transparent retry patch ?
+    <teythoon> pretty sure, why ?
+    <braunr> it's about a more general issue that we didn't fix yet
+    <braunr> our last discussions about it lead us to agree that clients should
+      check the identity of a server before interacting with it
+    <teythoon> braunr: i don't understand, what's the problem here ?
+    <braunr> teythoon: fakeroot does the lookup itself, doesn't it ?
+    <teythoon> yes
+    <braunr> teythoon: but was that also the case before your patch ?
+    <teythoon> braunr: yes
+    <braunr> teythoon: then ok
+    <braunr> teythoon: i guess fakeroot handles requests only for a specific
+      set of calls right ?
+    <braunr> and for others, requests are directly relayed
+    <teythoon> braunr: yes
+    <braunr> and that still is the case, right ?
+    <teythoon> yes
+    <braunr> ok
+    <braunr> looks right since it only affects lookups
+    <braunr> ok then
+    <teythoon> well, fakeroot-hurd built half a hurd package in less than 70
+      minutes
+    <teythoon> a new record for my box
+    <braunr> compared to how much before ?
+    <braunr> (and why half of it ?)
+    <teythoon> unfortunately it hung after signing the packages... some perl
+      process with a /usr/bin/tee child
+    <teythoon> killing tee made it succeed though
+    <teythoon> braunr: i don't build the udeb package
+    <braunr> oh ok
+    <teythoon> braunr: compared with ~75 with fakeroot-tcp and my demuxer
+      rework, ~80 before
+    <braunr> teythoon: nice
+
+
+## IRC, freenode, #hurd, 2013-12-18
+
+    <teythoon> there, i fixed the last fakeroot-hurd bug
+    <teythoon> *whee* :)
+    <teythoon> i thought so many times that i got the last fakeroot bug ...
+    <teythoon> last as in it's in a good enough shape to compile the hurd
+      package that is
+    <teythoon> but now it is
+    <braunr> :)
+    <braunr> this will make glibc and others so much faster to build
+
+
+## IRC, freenode, #hurd, 2013-12-19
+
+    <braunr> teythoon_: hum, you should make the behaviour of fakeroot-hurd on
+      the last client exiting optional
+    <teythoon_> y?
+    <teythoon_> fakeroot-tcp does the very same thing
+    <braunr> fakeroot-hurd is different
+    <braunr> it's part of the file system
+    <teythoon_> yes
+    <braunr> users may want it to stay around
+    <braunr> and reuse it without checking it's actually there
+    <teythoon_> but once the last client is gone, who is ever getting another
+      port to it ?
+    <teythoon_> no
+    <teythoon_> that cannot happen
+    <braunr> really ?
+    <teythoon_> yes
+    <braunr> i thought it was like remap
+    <braunr> since remap is based on it
+    <teythoon_> the same thing applies to remap
+    <teythoon_> only settrans has the control port
+    <braunr> hum
+    <teythoon_> and uses it once to get a protid for the working dir of the
+      initial process started inside the chrooted environment
+    <braunr> you may not want to chroot inside
+    <teythoon_> so ?
+    <teythoon_> then, you get another protid
+    <braunr> i'll make an example
+    <braunr> i create a myroot directory implemented by fakeroot
+    <braunr> populate it
+    <braunr> leave and do something else,
+    <braunr> i might want to return to it later
+    <teythoon_> ah
+    <teythoon_> ok, so you are not using settrans --chroot
+
+[[hurd/settrans/discussion#chroot]].
+
+    <braunr> or maybe i'm confusing the fakeroot translator and fakeroot-hurd
+    <braunr> 10:48 < braunr> you may not want to chroot inside
+    <braunr> yes
+    <teythoon_> hm
+    <teythoon_> ok, so the patch could be changed to check whether the last
+      control port is gone too
+    <braunr> i have no idea of any practical use, but i don't see a valid
+      reason to make a translator go away just because it has no client
+    <braunr> except for resource usage
+    <braunr> and if it's installed as a passive translator
+    <braunr> although that would make fakeroot loose its state
+    <braunr> though remap state is on the command line so it would be fine for
+      it
+    <braunr> see what i mean ?
+    <teythoon_> yes i do
+    <braunr> fakeroot state could be saved in some db one day so it may apply,
+      if anyone feels the need
+    <teythoon_> so what about checking for control ports too ?
+    <braunr> i'm not too familiar with those
+    <braunr> who has the control port of a passive translator ? the parent ?
+    <teythoon_> that should cover the use case you described
+    <teythoon_> for the parent translator
+    <teythoon_> for fsys_getroot requests it has to keep it around
+    <teythoon_> and for more fsys stuff too
+    <braunr> and if active ? settrans ? who just looses it ?
+    <teythoon_> if settrans is used to start an active translator, the parent
+      fs still gets a right to the control port
+    <braunr> ok
+    <braunr> i don't have a clear view of what this implies for fakeroot-hurd
+    <braunr> we'd want fakeroot-hurd to clean all resources including the
+      fakeroot translator on exit
+    <teythoon_> for fakeroot-hurd (or any child translator) this means that a
+      port from the control port class will still exists
+    <teythoon_> so we do not exit
+    <teythoon_> oh, you're speaking of fakeroot.sh ? the wrapper script ?
+    <braunr> probably
+    <braunr> for me, fakeroot-hurd is the command line too, similar to
+      fakeroot-sysv and fakeroot-tcp
+    <braunr> and fakeroot is the translator
+    <teythoon_> yes, agreed
+    <teythoon_> fakeroot-hurd could use settrans --force --chroot ... to force
+      fakeroot to exit if the main chrooted process dies
+    <teythoon_> but that'd kill anything that outlives that process
+    <teythoon_> that might be legitimate, say a process daemonized
+    <teythoon_> so detecting that noone uses fakeroot is the much cleaner
+      solution
+    <braunr> ok
+    <teythoon_> also, that's what fakeroot-tcp does
+    <braunr> which is why i suggested an option for that
+    <teythoon_> why add an option if we can do the right thing without
+      troubling the user ?
+    <braunr> ah, if we can, good
+    <teythoon_> i think we can
+    <teythoon_> I'll rework the patch, thanks for the hint
+    <braunr> so
+    <braunr> just to be clear
+    <braunr> the way you intend it to work is
+    <braunr> wait for all clients and the control port to drop before shutting
+      down
+    <braunr> the control port is dropped when dettaching the translator, right
+      ?
+    <teythoon_> yes
+    <braunr> but hm
+    <braunr> what if clients spawn other processes ?
+    <braunr> they won't find the translator any more
+    <teythoon_> then, that client get's a port to fakeroot at least for it's
+      working dir
+    <teythoon_> so another protid is created
+    <braunr> ah yes, it's usually choorted for such uses
+    <braunr> chrooted
+    <teythoon_> so fakeroot will stick around
+    <braunr> but clients, even from fakeroot, might simply use absolute paths
+    <teythoon_> so ?
+    <braunr> in which case they won't find fakeroot
+    <teythoon_> it will hit fakeroots dir_lookup
+    <teythoon_> sure
+    <braunr> how so ?
+    <teythoon_> if the path is absolute, it will trigger a magic retry of some
+      kind
+    <teythoon_> so the client uses it's root dir port
+    <braunr> i thought the lookup would be done straight from the root fs port
+      ..
+    <teythoon_> which points to fakeroot of course
+    <braunr> ah, chrooted again
+    <teythoon_> that's the whole point
+    <braunr> so this implies clients are chrooted
+    <teythoon_> they are
+    <teythoon_> even if you do another chroot
+    <braunr> what i mean is
+    <teythoon_> that root port also points to a fakeroot port
+    <braunr> if we detach the translator, and clients outside the chroot spawn
+      processes, say shell scripts, they won't find the fakeroot tree
+    <braunr> now, i wonder if we want to actually handle that
+    <braunr> i'm just uncomfortable with a translator silently shutting down
+      because it has no client
+    <teythoon_> if fakeroot is detached, how are clients outside the chroot
+      ever supposed to get a handle to files inside the fakerooted env ?
+    <braunr> it makes sense for fakeroot, so the expected behaviours here aer
+      conflicting
+    <braunr> they had those before fakeroot being detached
+    <teythoon_> then fakeroot wouldn't go away
+    <braunr> right
+    <braunr> unless there is a race but i don't think there is
+    <teythoon_> there isn't
+    <teythoon_> i call netfs_shutdown
+    <braunr> clients get the rights before the parent has a chance to terminate
+    <teythoon_> and only shutdown if it doesn't return ebusy
+    <braunr> makes sense
+    <braunr> ok go ahead :)
+    <teythoon_> cool, thanks for the walk-through ;)
+    <braunr> on the other hand ..
+    <braunr> that's a complicated topic left unfinished by the original authors
+    <teythoon_> one of many
+    <braunr> having translators automatically go away when there is no client
+      may be a good feature
+    <braunr> but it only makes sense for passive translators
+    <braunr> and this should be automated
+    <braunr> the lib*fs libraries should be able to handle it
+    <teythoon_> or, we could go for proper persistence instead
+    <braunr> stay around if active, leave after a while when no more clients if
+      passive
+    <braunr> why ?
+    <teythoon_> clean solution
+    <braunr> persistence looks much more expensive to me
+    <teythoon_> other benefits
+    <braunr> i mean
+    <braunr> persistence looks so expensive it doesn't make sense in a general
+      purpose system
+    <teythoon_> sure, we could make our *fs libs handle this smarter at a much
+      lower cost
+    <teythoon_> don't we get a handle to the underlying file ?
+    <braunr> i think we do yes
+    <teythoon_> if that's actually a file and not a directory, we could store
+      data into it
+    <braunr> many translators are read-only
+    <teythoon_> so ?
+    <braunr> well, when we can write, we can use passive translators instead
+    <braunr> normally
+    <teythoon_> yes
+    <braunr> depends on the fs type actually but you're right, we could use
+      regular files
+    <braunr> or a special type of file, i don't know
+    <antrik> braunr: BTW, I agree that active translators should only go away
+      when no ports are open anymore, while passive ones can exit when control
+      ports are still open but no protids
+    <teythoon> antrik: you mean as a general rule ?
+    <teythoon> that leaves the question how the translator distinguishes
+      between having a passive translator record and not having one
+    <antrik> I believe I already arrived at that conclusion in some design
+      discussion, probaly regarding namespace-based translator selection
+    <antrik> teythoon: yeah, as a general rule
+    <teythoon> interesting
+    <antrik> currently there are command line arguments controling timeouts,
+      but they don't consider control ports IIRC
+    <teythoon> i thought there are problems with shutting down translators in
+      general
+    <antrik> (also, command line arguments seem inconvenient to distinguish the
+      passive vs. active case...)
+    <teythoon> yeah, but we disregard the timeouts in the debian flavor of hurd
+    <antrik> teythoon: err... no we don't. at least not last time I knew. are
+      you confusing this with thread timeouts?
+    <antrik> simple test: do ls -l on /dev, wait a few minutes, compare
+    <teythoon> what do you expect will happen ?
+    <antrik> the unused translators should go away
+    <teythoon> no
+    <antrik> that must be new then
+    <teythoon> might be, yes
+    <teythoon>
+      http://darnassus.sceen.net/gitweb/teythoon/packaging/hurd.git/blame/HEAD:/debian/patches/libports_stability.patch
+    <braunr> antrik: debian currently disables both the global and thread
+      timeouts in libports
+    <braunr> my work on thread destruction consists in part in reenabling
+      thread timeouts, and my binary packages do that well so far :)
+    <antrik> braunr: any idea why the global timeouts were disabled?
+
+
+## IRC, freenode, #hurd, 2013-12-20
+
+    <braunr> antrik: not sure
+    <braunr> but i suspect there could be races
+    <braunr> if a message arrives while the server is going away, i'm not sure
+      the client can determine this and retry transparently
+    <antrik> good point... not sure how that is supposed to work exactly
+
+
+## IRC, freenode, #hurd, 2013-12-31
+
+    <braunr> btw, we should remove the libports_stability patch and directly
+      change the upstream code
+    <braunr> if you agree, i can force the global timeout to 0 (because we're
+      still not sure what can go wrong when a translator goes away while a
+      message is being delivered to it)
+    <braunr> i didn't experience any slowdown with thread destruction however
+    <braunr> so i'm tempted to set that to an actual reasonable timeout value
+      of 30-300 seconds
+    <teythoon> braunr: if you do, please introduce a macro for the default
+      value so it can be changed easily
+    <braunr> teythoon: yes
+    <braunr> i don't understand why these are left as parameters tbh
+    <teythoon> true
+    <braunr> 30 seconds seems to be plenty enough
+
+
+## IRC, freenode, #hurd, 2014-01-17
+
+    <braunr> time to give fakeroot-hurd a shot
+    <braunr> http://darnassus.sceen.net/~rbraun/darnassus_fakeroot_hurd_assert
+    <teythoon> braunr: (wrt fakeroot-hurd) well in my book that shouldn't
+      happen
+    <teythoon> that's why i put the assertion there ;)
+    <braunr> i assumed so :)
+    <teythoon> then again, /me does not agree with "threads" as concurrency
+      model >,<, and that feeling seems to be mutual :p
+    <braunr> ?
+    <teythoon> well, obviously, the threads do not agree with me wrt to that
+      assertion
+    <braunr> the threads ?
+    <teythoon> well, fakeroot is a multithreaded server
+    <braunr> teythoon: i'm not sure i get the point, are you saying you're not
+      comfortable with threads ?
+    <teythoon> that's exactly what i'm saying
+    <braunr> ok
+    <braunr> coroutines/functional i guess ?
+    <teythoon> csp
+    <teythoon> functional not so much
+
+
+## IRC, freenode, #hurd, 2014-01-20
+
+[[open_issues/libpthread]],
+[[open_issues/libpthread/t/fix_have_kernel_resources]].
+
+    <braunr> teythoon: it's perfectly possible that the bug i had with
+      fakeroot-hurd have been caused by my own glibc thread related patches
+    <braunr> has*
+    <teythoon> ok
+    <teythoon> *phew* :p
+    <braunr> :)
+    <teythoon> i wonder if youpi could reproduce his issue on his machine
+    <braunr> what issue ?
+    <braunr> i must have missed something
+    <teythoon> some package failed
+    <teythoon> but he didn't gave any details
+    <teythoon> he wanted to try it on his vm first
+    <braunr> ok
+
+
+## IRC, freenode, #hurd, 2014-01-21
+
+    <braunr> teythoon: i still get the same assertion failure with
+      fakeroot-hurd
+    <braunr> will take a look at that sometimes too
+    <teythoon> braunr: hrm :/
+    <braunr> teythoon: don't worry, i'm sure it's nothing big
+    <braunr> in the mean time, there are updated hurd and glibc packages on my
+      repository with fixed tls and thread destruction
+    <teythoon> cool :)
+
+
+## IRC, freenode, #hurd, 2014-01-23
+
+    <braunr> teythoon: can you briefly explain this fake reference thing in
+      fakeroot when you have some time please ?
+    <teythoon> braunr: fakeroot creates ports to hand out to clients
+    <teythoon> every port represents a node and references a real node
+    <teythoon> fakeroot allows one to set attributes, e.g. file permissions on
+      any node as if the client was root
+    <teythoon> those faked attributes are stored in the node objects
+    <braunr> let's focus on fake_reference please
+    <teythoon> once some attribute is faked, that node has to be kept alive
+    <teythoon> otherwise, that faked information is lost
+    <teythoon> so if the last peropen object is closed and some information is
+      faked, a fake reference is kept
+    <teythoon> as indicated by a flag
+    <braunr> hm
+    <teythoon> in dir lookup, if a node is looked-up that has a fake reference,
+      it is recycled, i.e. the flag cleared and the referecne count is not
+      incremented
+    <teythoon> so every time fakeroot_netfs_release_protid is called b/c, the
+      node in question should not have the fake reference flag set
+    <braunr> what's the relation between the number of hard links and this fake
+      reference ?
+    <teythoon> i don'
+    <teythoon> i don't think fakeroot has a notion of 'hard links'
+    <braunr> it does
+    <braunr> the fake reference is added on nodes with a hard link count
+      greater than 0
+    <braunr> but i guess that just means the underlying node still exists
+    <teythoon> ah yes
+    <teythoon> right
+    <teythoon> currently, if the real node is deleted, the fake node is still
+      kept around
+    <braunr> let's say it's ok for now
+    <teythoon> that's what the comment is talking about, the one that indicates
+      that garbage collection could help here
+    <teythoon> yes
+    <teythoon> properly fixing this is difficult
+    <braunr> agreed
+    <braunr> it would require something like inotify anyway
+    <teythoon> b/c of the way file deletion works
+    <braunr> let's just ignore the issue, that's not what i'm hunting
+    <teythoon> agreed
+    <braunr> the assertion i have is telling us that we're dropping a fake
+      reference
+    <braunr> are we certain this isn't possible ?
+    <teythoon> that function is called if a client dereferences a port
+    <teythoon> in order to have a port in the first place, it has to get it
+      from a dir_lookup
+    <teythoon> the dir lookup turns a fake reference into a real one
+    <teythoon> so i'm certain of that (barring a race condition somewhere)
+    <braunr> ok
+    <braunr> netfs_S_dir_lookup grabs idport_ihash_lock (line 354) but doesn't
+      release it if nn == NULL (lines 388-392)
+    <teythoon> hm, my file numbers are slightly different o_O
+    <braunr> i have printfs around
+    <braunr> sorry :)
+    <teythoon> ok
+    <teythoon> new node unlocks it
+    <teythoon> new_node
+    <braunr> oh
+    <braunr> how unintuitive ..
+    <teythoon> yes, don't blame me ;) that's how it was
+    <braunr> :)
+    <braunr> worse, the description says "if successful" ..
+    <braunr> ah no, the node lock
+    <braunr> ok
+    <teythoon> yes, badly worded description
+    <braunr> i strongly doubt it's a race
+    <teythoon> how do you trigger that assertion failure ?
+    <braunr> dpkg-buildpackage -rfakeroot-hurd -uc -us
+    <braunr> for the hurd package
+    <braunr> very similar to one of your test cases i think
+    <teythoon> umm :-/
+    <braunr> one thing that i find confusing is that fake_reference seems to
+      apply to nodes, whereas release_protid is about, well, protids
+    <braunr> is there a 1:1 relationship ?
+    <braunr> since there is a peropen in the protid, i assume not
+    <braunr> it may be a race actually
+    <braunr> np->references must be accessed with netfs_node_refcnt_lock locked
+    <braunr> hm no, that's not it
+    <teythoon> no, it's not a 1:1 relationship
+    <teythoon> note that the lock idport_ihash_lock serializes most operations,
+      despite it's name indicating that it's only for the hash table
+    <teythoon> the "interesting" operations being dir_lookup and release_protid
+    <braunr> yes
+    <braunr> again, that's another issue
+    <teythoon> why ? that's a pretty strong guarantee already
+    <braunr> ah yes, i was referring to scalability
+    <teythoon> sure
+    <braunr> the assertion is triggered from ports_port_deref in
+      ports_manage_port_operations_multithread
+    <teythoon> but i found it hard to reason about fakeroot, there are multiple
+      locks involved, two kinds of reference counting across different libs
+    <braunr> yes
+    <teythoon> yes, that's to be expected
+    <braunr> teythoon: do we agree that the fake reference is reused by a
+      protid ?
+    <teythoon> braunr: yes
+    <braunr> why is there a ref counter for the protid as well as the peropen
+      then ? :/
+    <teythoon> funny... i thought there was no refcnt for the peropen objects,
+      but there is
+    <teythoon> but for fakeroot-hurd that shouldn't matter, right ?
+    <braunr> i don't know
+    <teythoon> here, one protid object is associated with one peropen object
+    <braunr> yes
+    <teythoon> and the other way around, i.e. it's 1:1
+    <teythoon> so the refcount for those should be identical
+    <braunr> but i get a case where protid has a refcnt of 0 while the peropen
+      has 2 ..
+    <teythoon> umm, that doesn't sound right
+    <braunr> teythoon: ok, it does look like a race on np->references
+    <braunr> node references are protected by a global lock in lib*fs libs
+    <teythoon> yes
+    <braunr> you check it without holding it
+    <braunr> which means another protid can be closed at the same time, setting
+      the flag on the underlying node
+    <braunr> i'll make a proper patch soon
+    <teythoon> they cannot both hold the hash lock
+    <braunr> hm
+    <braunr> teythoon: actually, i don't see why that's relevant
+    <braunr> one thread closes its protid, sets the fakeref flag
+    <braunr> the other does the same, chokes on the assertion
+    <braunr> serially
+    <teythoon> i'm always a little fuzzy when exactly the references get
+      decremented
+    <teythoon> but shouldn't only the second thread set the fakeref flag ?
+    <braunr> well, that's not what i see
+    <braunr> i'll check what happens to this ref counter
+    <teythoon> see how my release_protid function calls netfs_release_protid
+      just after the out label
+    <teythoon> *while holding the big hash lock
+    <teythoon> so, any refcounting should happen while the lock is being held,
+      no ?
+    <braunr> perhaps
+    <braunr> now, my logs show something new
+    <braunr> a case where the protid being released was never printed before
+    <braunr> i.e. not obtained from dir_lookup
+    <braunr> or at least, not fakeroot dir_lookup
+    <teythoon> huh, where did it came from then ?
+    <braunr> no idea
+    <teythoon> only dir_lookup hands out those
+    <braunr> check_openmodes calls dir_lookup too
+    <teythoon> yes, but that's not our dir_lookup
+    <braunr> that's what i mean
+    <braunr> it bypasses fakeroot's custom dir_lookup
+    <braunr> but i guess the reference already exists at this point
+    <teythoon> bypass ? i wouldn't call it that
+    <braunr> you're right, wrong wording
+    <teythoon> it accesses files on other translators
+    <braunr> yes
+    <braunr> the netnode is already present
+    <teythoon> yes
+    <braunr> could it be the root node ?
+    <teythoon> i do not believe so
+    <teythoon> the root node is always faked
+    <teythoon> and is handed out to the first process in the fakeroot env for
+      it's current directory port
+    <teythoon> so you could try something that chdirs away to test that
+      hypothesis
+    <braunr> the assertion looks triggered by a chdir
+    <teythoon> how do you know that ?
+    <braunr> dh_auto_install: error: unable to chdir to build-deb
+    <teythoon> ah
+    <teythoon> well, or that is just the operation after fakeroot died and
+      completely unrelated
+    <braunr> maybe
+    <teythoon> can you trigger this reliably ?
+    <braunr> yes
+    <braunr> i'm trying to write a shell script for that
+    <teythoon> so for you, fakeroot-hurd never succeeded in building a hurd
+      package ?
+    <braunr> no
+    <teythoon> on darnassus ?
+    <braunr> yes
+    <teythoon> b/c i stopped working on fakeroot-hurd when it was in a
+      good-enough shape to build the hurd package
+    <teythoon> >,<
+    <teythoon> maybe my system is not fast enough to hit this race (if it turns
+      out to be one)
+    <braunr> some calls seems to decrease the refcount of the root node
+    <braunr> call*
+    <teythoon> have you confirmed that it's the root node ?
+    <braunr> almost
+    <braunr> i could say yes
+    <braunr> teythoon: actually no, it's not ..
+    <braunr> could be ..
+    <braunr> teythoon: on what node does fakeroot-hurd install the fakeroot
+      translator when used to build debian packages ?
+    <braunr> hum
+    <braunr> could it simply be that the check on np->references should be
+      moved above the assertion ?
+    <teythoon> braunr: it is not bound to any node, check settrans --chroot
+
+[[hurd/settrans/discussion#chroot]].
+
+    <braunr> oh right
+    <braunr> teythoon: ok i mean
+    <braunr> does it shadow / ?
+    <braunr> looks very likely, otherwise the chroot wouldn't work
+    <teythoon> i'm not sure what you mean by shadow
+    <braunr> settrans --chroot cmd -- / /hurd/fakeroot ?
+    <teythoon> but yes, for any process in the chroot-like env every real node
+      is replaced, including /
+    <braunr> makes sense
+    <braunr> teythoon: moving the assertion seems to fix the issue
+    <braunr> intuitively, it seems reasonable to assume the fakeref flag can
+      only be set when there is only one reference, namely the fake reference
+    <braunr> (well, the fake ref, recycled by the last open)
+    <teythoon> no, i don't follow
+    <teythoon> i'd still say, that if ...release_protid is called, then there
+      is no way for the fake flag to be set in the first place
+    <teythoon> that's why i put the assertion in ;)
+    <braunr> on the other hand, you check the refcnt precisely because other
+      threads may have reacquired the node
+    <teythoon> but why would moving the assertion change anything ?
+    <teythoon> if we would do that, we'd "lose" all threads that see
+      np->reference being >1
+    <teythoon> but for those objects the fake_reference flag should never be
+      set anyways
+    <teythoon> i cannot see why this would help
+    <teythoon> (does it help ?)
+    <teythoon> (and if it does, it points to a serious problem imho)
+    <braunr> i'm recreating the traces that made me think that
+    <braunr> to get a clearer view of what's happening
+    <braunr> the problem i have with the current code is this
+    <braunr> there can be multiple protid referring to the same node occurring
+      at the same time
+    <braunr> they are serialized by the hash table lock, ok
+    <braunr> but there apparently are cases where the first (of two) protids
+      being closed sets the fakeref flag
+    <braunr> and the following chokes because the flag is set
+    <braunr> i assume you put this refcount check because you assumed only the
+      last protid being closed can set the flag, right ?
+    <braunr> but then, why > 1 ? why not > 0 ?
+    <teythoon> yes, that's what i was trying to assert
+    <teythoon> b/c the 1 is our reference
+    <braunr> which one exactly ?
+    <teythoon> >1 is anyone *beside* us
+    <teythoon> ?
+    <braunr> hm
+    <braunr> you mean the reference held by the protid being destroyed
+    <teythoon> yes
+    <braunr> isn't that reference already dropped before calling the cleanup
+      function ?
+    <braunr> ah no, it's the node ref
+    <teythoon> yes
+    <braunr> released by netfs_release_protid
+    <teythoon> exactly
+    <braunr> which is called without the hash table lock held
+    <braunr> hm no
+    <braunr> it's locked
+    <braunr> damn my brain is slow today
+    <teythoon> i actually think that it's the combination of manual reference
+      counting and the primitive concurrency model that makes it hard to reason
+      about this
+    <braunr> well
+    <braunr> the model is quite simple too
+    <braunr> accesses to refcounters must be protected by the appropriate lock
+    <braunr> this isn't done here, on the assumption that all referencing
+      operations are protected by another global lock all the time
+    <teythoon> even if a model is simple, this does not mean that it is a good
+      model for human beings to comprehend and reason about
+    <braunr> i don't know
+    <braunr> note that netfs_drop_node is designed to be called with
+      netfs_node_refcnt_lock locked
+    <braunr> implying the refcount must remain stable between checking it and
+      dropping the node
+    <braunr> netfs_make_peropen is called without the hash table lock held in
+      dir_lookup
+    <braunr> and this increases the refcount
+    <braunr> although the problem is rather that something decreases it without
+      the lock held
+    <teythoon> we should port libtsan and just ask gcc -fsanitize=thread
+    <braunr> what about the netfs_nput call at the end of dir_lookup ?
+    <braunr> the fake ref should be set by the norefs function
+    <teythoon> that should not decrease the count to 0 b/c the caller holds a
+      reference too
+    <braunr> yes that's ugly
+    <braunr> ugh
+    <braunr> i'm unable to think clearly right now
+    <teythoon> as mentioned in the commit message, you cannot do something like
+      this in the norefs function
+    <teythoon> bbl ;)
+    <braunr> bye teythoon 
+    <braunr> thanks for your time
+    <braunr> for when you come back :
+    <braunr> instead of maintaining this "fake" reference, why not assumeing
+      the hash table holds a reference, and simply count it
+    <braunr> the same way a cache does
+    <braunr> and drop that reference when removing a node, either to reflect
+      the current state of the underlying node, or because the translator is
+      being shut down ?
+    <braunr> why not assume*
+    <braunr> bbl too
+    <teythoon> sure, refactoring is definitively an option
+
+
+## IRC, freenode, #hurd, 2014-01-24
+
+    <braunr> teythoon: ok, i'll take care of fakeroot
+    <teythoon> braunr: thanks. tbh i was a little fed up with that little
+      bugger >,<
+    <braunr> i can imagine
+    <braunr> considering the number of patches you've sent already
+
+    <braunr> teythoon: are you sure about your call to fshelp_lock_init ?
+    <teythoon> yes, why do you ask ?
+    <teythoon> (the test case is given in the commit message)
+    <braunr> it doesn't look right to me to call "init" while the node is
+      potentially locked
+    <braunr> i noticed libdiskfs peropen release function takes care of
+      releasing locks
+    <braunr> it looks better to me
+    <teythoon> it's not about releasing the lock
+    <teythoon> it's about faking the file being closed which implicitly
+      releases the lock
+    <braunr> the file is being close
+    <braunr> closed
+    <braunr> since it's in the cleanup function
+    <teythoon> yes, but we keep it b/c the file has faked attributes
+    <teythoon> did you look at the problem description in the commit message ?
+    <braunr> we keep the node
+    <braunr> not the peropen
+    <teythoon> so ?
+    <teythoon> the lock is in the node
+    <braunr> why would libdiskfs do it in the peropen release then ?
+    <braunr> there is an inconsistency somwhere
+    <braunr> actually, the lock looks to be per open
+    <braunr> or rather, the lock is per node, but its status is recorded per
+      open
+    <braunr> allowing the implementation to track if a file descriptor was used
+      to install a lock and release it when that file descriptor goes away
+    <teythoon> why would the node be locked ?
+    <teythoon> locked in what way, file-locking locked ?
+    <braunr> yes
+    <braunr> posix explicitely says that file locks must be implicitely removed
+      when closing the file descriptor used to install them, so that makes
+      sense
+    <teythoon> isn't hat exactly what i'm doing ?
+    <braunr> no
+    <braunr> you're initializing the file lock
+    <braunr> init != unlock
+    <braunr> and it's specific to fakeroot, while it looks like libnetfs should
+      be doing it
+    <teythoon> libnetfs would do it
+    <teythoon> but we prevent that by keeping the node alive
+    <braunr> again, it's a per open thing
+    <braunr> and no, libnetfs doesn't release locks implicitely in the current
+      version
+    <teythoon> didn't we agree that for fakeroot one peropen object is
+      associated with one protid object ?
+    <braunr> yes
+    <braunr> and don't keep those alive
+    <braunr> so let them die peacefully, and fix libnetfs so it releases the
+      lock as it's supposed to
+    <braunr> and we* don't
+    <teythoon> we don't keep those alive
+    <teythoon> why would we ?
+    <braunr> yes that's what i wanted to say
+    <braunr> what i mean is
+    <braunr> since letting peropens die is already what is being done
+    <braunr> there is no need for a special handling of locks in fakeroot
+    <teythoon> oh
+    <braunr> on the other hand, libnetfs must be fixed
+    <teythoon> ok, that might very well be true
+    <teythoon> (we need to bring libnetfs and diskfs closer so that they can be
+      diff'ed easily)
+    <braunr> i just wanted to check your reason for using lock_init in the
+      first place
+    <braunr> yes ..
+    <braunr> teythoon: also, i think we actually do have what's necessary to
+      deal with garbage collection
+    <braunr> namely, dead-name notifications
+    <braunr> i'll see if i can cook something simple enough
+    <braunr> otherwise, merely keeping every node around is also acceptable
+      considering the use cases
+    <teythoon> dead-name notifications won't help if the real node disappears,
+      no ?
+    <braunr> teythoon: dead name notifications on the real node port :)
+    <braunr> teythoon: at least i can reliably build the hurd package using
+      fakeroot-hurd now
+    <braunr> let's try glibc :)
+
+## IRC, freenode, #hurd, 2014-01-25
+
+    <teythoon> braunr: awesome :)
+    <braunr> teythoon: hm not sure :/
+    <braunr> darnassus got oom
+    <braunr> teythoon: could be unrelated though
+    <braunr> teythoon: something has apprently made /home unresponsive :(
+    <braunr> teythoon: i suspect bots hitting apache and in particular the git
+      repositories to have increased memory usage
+
+
+## IRC, freenode, #hurd, 2014-01-26
+
+    <braunr> teythoon: btw, fakeroot interacts very very badly with other netfs
+      file systems
+    <braunr> e.g., listing /proc through it creates lots of nodes
+    <braunr> i'm not yet sure how to fix that
+    <braunr> using a dead name notification doesn't seem appropriate (at least
+      not directly) because fakeroot holds a true reference that prevents the
+      deallocation of the target node
+
+
+## IRC, freenode, #hurd, 2014-01-27
+
+    <braunr> teythoon: good news (more or less): fakeroot is actually leaking a
+      lot when crossing file systems
+    <braunr> which means if i fix that, there is a good chance we can use it to
+      build all packages with it
+    <braunr> -with it
+    <teythoon> what do you mean exactly ?
+    <braunr> if target nodes are from /, there is no such leak
+    <braunr> as soon as the target nodes are from another file system, ports
+      rights are leaked
+    <braunr> that's what fills the kernel allocator actually
+    <teythoon> oh, so dir_lookup leaks ports when crossing translator
+      boundaries ?
+    <braunr> seems so
+    <teythoon> yeah, that might very well be it
+    <teythoon> the dir_lookup logic in lib*fs is quite involved :/
+    <braunr> yes, my simple attempts were unsuccessful
+    <braunr> but i'm confident i can fix it soon
+    <teythoon> that sounds good :)
+    <braunr> i also remove the fake_ref flag and replace it with "accounting
+      the reference in the hash table" as soon as a node is faked
+    <teythoon> fine with me
+    <braunr> these will be the expected leak
+    <braunr> but they're far less in numbers than what i observe
+    <braunr> and garbage collection can be implemented later
+    <braunr> although i would prefer notifications a lot more
+    <braunr> end of the news, bbl :)
+    <braunr> found it :>
+    <teythoon> braunr: -v ;)
+    <braunr> err = dir_lookup (...);
+    <braunr> if (dir != dnp->nn->file) mach_port_deallocate (mach_task_self (),
+      dir);
+    <braunr> in other words, deallocate ports for intermediate file system root
+      directories .. :)
+    <braunr> teythoon: currently building hurd and glibc packages
+    <braunr> but i intend to improve some more with the addition of a default
+      faked state
+    <braunr> so that only nodes with modified faked states are retained
+    <teythoon> how do you mark nodes as having the default faked state ?
+    <braunr> i don't
+    <teythoon> ok, right, makes sense :)
+    <teythoon> this sounds awesome, thanks for following up on this
+    <braunr> i'm quite busy with other stuff so, with proper testing, it should
+      take me the week to get merged
+    <braunr> teythoon: well thanks for all the fixes you've done
+    <braunr> fakeroot was completely unusable before that
+    <teythoon> if you push your changes somewhere i'll integrate them into my
+      packages and test them
+    <braunr> ok
+    <braunr> implementing fakeroot -u could also be a good thing
+    <braunr> and this should work easily with that default faked state strategy
+
+
+## IRC, freenode, #hurd, 2014-01-28
+
+    <braunr> teythoon: i should be able to test fakeroot-hurd with the default
+      faked attributes strategy today on glibc
+    <teythoon> braunr: very nice :)
+    <braunr> azeem_: do you happen to know if fakeroot -u is used by debian ?
+    <braunr> i mean when building packages
+    <teythoon> braunr: how does fakeroot-hurd perform on darnassus ?
+    <teythoon> i mean, does it yield a noticeable improvement over fakeroot-tcp
+      just like on my slow box ?
+    <braunr> i'm not measuring that :/
+    <teythoon> ok, no problem
+    <braunr> and since nodes are removed from the hash table, performance might
+      decrease slightly
+    <braunr> but the number of rights is kept very low, as expected
+    <teythoon> that's good
+    <braunr> i keep seeing leaks though
+    <braunr> when switching cwd between file systems
+    <teythoon> humm
+    <braunr> so i assume something is wrong with the identity of . or ..
+    <braunr> it's so insignificant compared to the previous problems that i
+      won't waste time on that
+    <braunr> teythoon: the problem with measuring on darnassus is that it's a
+      public machine
+    <teythoon> right
+    <braunr> often scanned by ssh worms or http bots
+
+[[cannot_create__dev_null__interrupted_system_call]].
+
+    <braunr> but it makes complete sense to get better performance with
+      fakeroot-hurd
+    <braunr> that's actually one of the reasons i'm working on it
+    <braunr> if not the main one
+    <teythoon> :)
+    <teythoon> that was my motivation too
+    <braunr> it shows how you can get an interchangeable unix tool that
+      directly plugs well with the low level system
+    <braunr> and make it work better
+    <teythoon> nicely put :)
+
+    <braunr> teythoon: i still can't manage to build glibc with fakeroot-hurd
+    <braunr> but i'm not sure why :/
+    <braunr> there was no kernel memory exhaustion this time
+    <teythoon> :/
+    <braunr> cp: cannot create regular file `debian/libc-bin.dirs': Permission
+      denied
+    <braunr> hum
+    <braunr> youpi: do you know if building debian packages requires fakeroot
+      -u option ?
+    <youpi> I don't know
+    <gg0> braunr: man dpkg-buildpackage says it just runs "fakeroot
+      debian/rules <target>"
+    <gg0> sources confirm that
+      http://sources.debian.net/src/dpkg/1.17.6/scripts/dpkg-buildpackage.pl#L465
+    <braunr> gg0: ok
+
+
+## IRC, freenode, #hurd, 2014-01-29
+
+    <braunr> it seems that something sets the permissions of this
+      debian/libc-bin.dirs file to 000 ...
+    <teythoon> i've seen this too
+    <braunr> oh
+    <braunr> do you think it's a fakeroot-hurd bug ?
+    <teythoon> have i mentioned something like this in a commit message ?
+    <teythoon> yes
+    <teythoon> it is
+    <braunr> ok
+    <braunr> i didn't see any mention of it
+    <braunr> but i could have missed it
+    <teythoon> hm, i cannot recall it either
+    <teythoon> but i've seen this issue with fakeroot-hurd
+    <braunr> ok
+    <braunr> it's probably the last issue to fix to get it to work for our
+      packages
+    <braunr> teythoon: i think i have a solution for that last mode bug
+    <braunr> fakeroot doesn't relay chmod requests, unless they change an
+      executable bit
+    <braunr> i don't see the point, and simply removed that condition to relay
+      any chmod request
+    <teythoon> braunr: did it work ?
+    <braunr> no
+    <braunr> fakeroot still consumes too many ports
+    <braunr> and for each file, there are at least two ports, the faked one,
+      and the real one
+    <braunr> it should be completely reworked
+    <braunr> but i don't have time to do that
+    <braunr> i'll see if it works when building from scratch
+    <braunr> actually, it's not even a quantity problem but a fragmentation
+      problem
+    <braunr> the function that fails is kmem_realloc ..
+    <braunr> ipc spaces are arrays in kernel space ....
+    <teythoon> it's more like three ports per file, you forgot the identity
+      port
+    <braunr> ah yes
+
+
+## IRC, freenode, #hurd, 2014-02-03
+
+    <braunr> teythoon: i'll commit my changes on fakeroot tonight
+    <braunr> they do improve the tool, but not enough to build glibc with it
+    <teythoon> braunr: cool :), so how do we make it fully usable ?
+    <braunr> teythoon: i don't know ..
+    <braunr> i'll try re adding detection of nodes with no hard links for one
+    <braunr> but imho, it needs a rework based on what the real fakeroot does
+    <braunr> i won't work on it though
+
+    <braunr> teythoon: also, it looks like i've tested building glibc with a
+      wrong test binary of my fakeroot version :/
+    <braunr> so consider all test results irrelevant so far
+
+
+## IRC, freenode, #hurd, 2014-02-04
+
+    <braunr> fakeroot-hurd might turn out to be easily usable for our debian
+      packages with the fixed binary :)
+
+    <braunr> teythoon: hum, can you explain
+      672005782e57e049c7c8f4d6d0b2a80c0df512b4 (trans: fix locking issue in
+      fakeroot) when you have time please ?
+    <braunr> it looks like it introduces a deadlock by calling new_node (which
+      acquires the hash table lock) while dir is locked, violating the hash
+      table -> node locking order
+
+    <teythoon> braunr: awesome, then there still is hope for fakeroot-hurd :)
+
+    <braunr> teythoon: i've been able to build glibc packages several times
+      this night
+    <braunr> so except for this deadlock i've seen once, it looks good
+    <teythoon> right
+    <teythoon> that deadlock
+    <teythoon> right, it does indeed violate the partial order of the locks :-/
+
+    <braunr> teythoon: can you explain why you moved the lock in attempt_mkfile
+      please ?
+
+    <braunr> teythoon: i've just tested a fakeroot binary without the patch
+      introducing the deadlock, and glibc built without any problem
+    <teythoon> braunr: well, this is very good news :)
+    <braunr> teythoon: but i still wonder why you made this patch in the first
+      place, i don't want to revert it blindly and reintroduce a potential
+      regression
+    <teythoon> braunr: i thought i was fixing the order in which locks were
+      taken. if the commit message does not specify that it fixes an issue,
+      then i was probably just wrong and you can revert it
+    <braunr> oh ok
+    <braunr> good
+
+    <braunr> teythoon: another successful build :)
+    <braunr> i'll commit my changes
+    <teythoon> awesome :)
+    <braunr> there might still be concurrency issues but it's much better
+    <teythoon> i'm curious what you did :)
+    <braunr> so little :)
+    <braunr> i was sick all week heh
+    <braunr> you'll se
+    <braunr> see
+    <teythoon> well, that's good actually ;)
+    <braunr> yes
+
+    <braunr> teythoon: actually there was another debugging line left over, and
+      again, my test results are irrelevant @#!
+
+
+## IRC, freenode, #hurd, 2014-02-05
+
+    <braunr> teythoon: i got an assertion about nn->np->nn not being equal to
+      nn atfer the hash table lookup is dir_lookup
+    <braunr> +failure
+    <teythoon> that's bad
+    <braunr> not over yet
+    <teythoon> i had a couple of those too
+    <teythoon> i guess it's a use after free
+    <braunr> yes
+    <teythoon> i used to poison the pointers and comment out the frees to track
+      them down iirc
+    <braunr> teythoon: one of your patches stores netnodes instead of nodes in
+      the hash table, citing some overwriting issue
+    <braunr> teythoon: i don't understand why using netnodes fixes this
+    <teythoon> braunr: libihash has this cookie for fast deletes
+    <teythoon> that has to be stored somewhere
+    <teythoon> the node structure has no room for it
+    <braunr> uh
+    <teythoon> yes
+    <teythoon> it was that bad
+    <braunr> ...
+    <teythoon> hence the uglyish back pointers
+    <braunr> i see
+    <teythoon> looking back i cannot even say why it worked at all
+    <braunr> well, it didn't
+    <teythoon> i believe libihash must have destroyed a linked list in the node
+      struct
+    <braunr> possibly
+    <teythoon> no, it did not >,<, but for simple tests it kind of did
+    <braunr> yes fakeroot sometimes corrupts memory badly ....
+    <braunr> and yes, turns out the assertion is triggered on nodes with 0 refs
+      ..
+    <braunr> teythoon: it looks like even the current version makes wrong usage
+      of the ihash interface
+    <braunr> locp_offset is defined as "The offset of the location pointer from
+      the hash value"
+    <braunr> and indeed, it's an intptr_t
+    <braunr> teythoon: hm no, it looks ok actually, forget what i said :)
+    <teythoon> *phew
+    <teythoon> :p
+
+    <braunr> hmm, still occasional double frees in fakeroot, but it looks in
+      good shape for single threaded tasks like package building
+
+    <braunr> teythoon: i've just sent my fakeroot patches
+    <teythoon> braunr: sweet, i'll have a closer look tomorrow :)
+    <braunr> teythoon: i couldn't debug the double frees though :/
+
+
+## IRC, freenode, #hurd, 2014-02-06
+
+    <braunr> btw, i'm able to successfully use fakeroot-hurd to build glibc
+      packages, but is there a way to make sure the resulting archives contain
+      the right privileges and ownerships ?
+    <youpi> I don't remember whether debdiff checks permissions
+
+    <youpi> braunr: I've just got  fakeroot-hurd debian/rules clean
+    <youpi> dh_clean 
+    <youpi> fakeroot: ../../trans/fakeroot.c:161: netfs_node_norefs: Assertion
+      `np->nn->np == np' failed.
+    <youpi> while building eglibc
+    <teythoon> youpi: yes, that lockup is most annoying... :/
+    <braunr> youpi: with the new version ?
+    <youpi> yes
+    <braunr> hum
+    <braunr> i only had rare double frees, not that any more :/
+    <braunr> youpi: ok i got the error too
+    <braunr> still not good enough
+    <youpi> ok
+
+
+## IRC, freenode, #hurd, 2014-02-07
+
+    <braunr> youpi: debdiff seems to handle permissions
+    <braunr> i've found the cause of the assertions
+    <youpi> braunr: groovie :)
+
+
+## IRC, freenode, #hurd, 2014-02-08
+
+    <teythoon> braunr: nice :)
+    <braunr> http://darnassus.sceen.net/~rbraun/debdiff_report
+
+
+## IRC, freenode, #hurd, 2014-02-10
+
+    <braunr> and, on a completely different topic, here is a crash i can
+      reproduce when using fakeroot:
+      http://darnassus.sceen.net/~rbraun/fakeroot_hurd_rpctrace_o_var_tmp_out_rm_rf_dir.png
+
+
+## IRC, freenode, #hurd, 2014-02-11
+
+    <braunr> still working on fakeroot
+    <braunr> there are still races (not disturbing for package building but
+      still ..)
+    <braunr> there may be wrong right handling
+    <teythoon> i believe i have witnessed a fakeroot deadlock :/
+    <braunr> aw
+    <teythoon> not sure though, buildbot killed the build process before i
+      could investigate
+    <braunr> teythoon: was it a big package ?
+    <teythoon> half of the hurd package
+    <braunr> that's not a port right overflow then
+
+
+## IRC, freenode, #hurd, 2014-03-05
+
+    <teythoon> youpi: what about the exec_filename patch series? even though
+      fakeroot still has some issues (has it?), i consider it worthy for
+      inclusion
+
+[[glibc#execve_relative_paths]].
+
+    <youpi> Roland was disagreeing with it
+    <youpi> iirc the fakeroot issue was solved
+    <teythoon> braunr: ^
+    <braunr> fakeroot goot a lot more robust than it used to be
+    <braunr> but i haven't checked that it actually behaves exactly like the
+      library for corner cases
+    <braunr> there are minor differences
+    <braunr> also, it seems to trigger concurrency bugs in ext2fs
+    <braunr> e.g. git reporting that files either "already exist" or "can't be
+      found"
+    <braunr> it happens (rarely) when directly using ext2
+    <braunr> and more often through fakeroot
+    <braunr> i didn't take the time to investigate
+
+## youpi
+
+    the daily script of debian-installer uses the -s / -i options of fakeroot. How could we manage to implement them?
diff --git a/open_issues/virtualization/file_systems.mdwn b/open_issues/virtualization/file_systems.mdwn
new file mode 100644
index 00000000..a12ea10d
--- /dev/null
+++ b/open_issues/virtualization/file_systems.mdwn
@@ -0,0 +1,24 @@
+[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+Of course, it is possible to use commodity file systems on [[virtualized
+systems|virtualization]], like [[hurd/translator/ext2fs]] or
+[[hurd/translator/nfs]], but there are also other possibilities which ought to
+be explored.
+
+  * [[network_file_system_by_just_forwarding_RPCs]]
+
+  * Linux saw a patch for [*generic name to handle and open by handle
+    syscalls*](http://thread.gmane.org/gmane.linux.file-systems/46648) posted,
+    which in turn can be beneficial for a [[QEMU]] emulation of a 9P file
+    system.  LWN's Jonathan Corbet covered this [*open by
+    handle*](http://lwn.net/Articles/375888/) functionality on 2010-02-23.
diff --git a/open_issues/virtualization/networking.mdwn b/open_issues/virtualization/networking.mdwn
new file mode 100644
index 00000000..122f21ab
--- /dev/null
+++ b/open_issues/virtualization/networking.mdwn
@@ -0,0 +1,88 @@
+[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+Collection about stuff that is relevant for *virtualization* and *networking*.
+
+  * [[Virtual_Square_View-OS]]
+
+  * [*Virtual Networks*](http://virtualsquare.org/vn.html)
+
+  * [User Level Networking](http://uln.sourceforge.net/)
+
+  * [Virtual Distributed Ethernet](http://vde.sourceforge.net/)
+
+  * [Application Level
+    Environment4Networking](http://sourceforge.net/projects/ale4net/)
+
+    *Ale4NET used dyn library call diversion to define networking at process
+    level.* -- what we're doing with our approach for overriding the default
+    [[hurd/translator/pfinet]] by setting environment variables.
+
+    Project is now part of [[Virtual_Square_View-OS]].
+
+
+# OpenVPN
+
+[[community/meetings/GHM2013]].
+
+OpenVPN can use pfinet's tun as it is, and be configured completely as joe user,
+as shown below. Note that the tun0 node name has to begin with "tun", so pfinet
+knows it's a tun.
+
+    $ mkdir -p $HOME/servers/socket
+    $ touch $HOME/servers/tun0
+    $ settrans -ca $HOME/servers/socket/2 /hurd/pfinet $HOME/servers/tun0 -a 10.0.0.1 -p 10.0.0.2
+    $ cat vpn.conf
+    client
+    dev tun
+    dev-node /home/samy/servers/tun0
+    ...
+    $ openvpn --config vpn.conf --verb 5
+    ...
+
+So we let openvpn running, and now we can make applications use the pfinet
+TCP/IP stack started above: the remap command below starts a new shell, where
+it redirects /servers/socket/2 (where the system TCP/IP stack resides) onto
+$HOME/servers/socket/2 (where the new TCP/IP stack resides).
+
+    $ remap /servers/socket/2 $HOME/servers/socket/2
+    $ wget www.gnu.org
+
+Ideally openvpn would be made to setup pfinet itself, but at least for now it
+can be configured by hand like that.
+
+It would probably be possible to make pfinet able to produce a tap too, would
+need some code.
+
+## IRC, freenode, #hurd, 2013-09-07
+
+    <d3f> anyone here knows how /dev/net is handled on the hurd? Programs using
+      it say it's not a directory. I tried creating one and setting a netdde
+      translator for a tun device in it, but this may be wrong as it doesn't
+      work
+    <teythoon> d3f: what does /dev/net do?
+    <teythoon> ah, its tun/tap stuff...
+    <d3f> on my gnu/linux it includes a tun device
+    <teythoon> right
+    <d3f> I am still reading about the Hurd and try to understand /hurd/netdde
+      and devnode but by now I am quite sure I will need those to set a tun
+      networktranslator on /dev/net/tun?
+    <teythoon> hm, I don't think netdde or devnode will be of any help
+    <teythoon> afaiui devnode makes mach devices available in the hurdish way,
+      i.e. available for lookup in the filesystem
+    <teythoon> d3f: ping youpi if he shows up, he hacked up openvpn to work on
+      the hurd
+    <d3f> yeah I know, I talked to him as I am tring to get tinc working on the
+      Hurd (tinc builds by now). I will give him a shot about creating the
+      "tun" device
+
+tun has indeed nothing to do with devnode and netdde, it's pfinet which creates it, as documented above.
diff --git a/open_issues/virtualization/remap_root_translator.mdwn b/open_issues/virtualization/remap_root_translator.mdwn
new file mode 100644
index 00000000..8f8668fe
--- /dev/null
+++ b/open_issues/virtualization/remap_root_translator.mdwn
@@ -0,0 +1,157 @@
+[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+/!\ [[!tag open_issue_documentation]] Does this completely resolve
+[[community/gsoc/project_ideas/server_overriding]]?
+
+
+# IRC, freenode, #hurd, 2013-01-05
+
+    <youpi> so we have a "remap" root translator?
+    <youpi> I  mean this:
+    <youpi> I'd run my shell in a subhurd whose only difference is that the
+      root is not the system's root, but my own
+    <youpi> which catches accesses to /servers/socket/2 for instance
+    <youpi> but leaves the rest flow through the system's root
+    <braunr> there is just boot, i don't think it can do that
+    <youpi> it'd be useful to have that
+    <youpi> it'd be a very useful feature
+    <youpi> to use another tcp/ip stack etc.
+    <braunr> what happens when translators need to locate other translators
+      used by the client ?
+    <youpi> can't it tell the client to ask the real system's root?
+    <youpi> (with the same path)
+    <youpi> I don't remember the exact reply name
+    <braunr> hum, it's getting too fuzzy for my head :p
+    <youpi> well, I mean it's just like translator entries in an ext2fs
+    <youpi> ext2fs replies "not me, this one"
+    <braunr> but what if e.g. a user has its own pflocal, and when calling
+      another translator, that one wants to contact the pflocal used by the
+      client ?
+    <youpi> ah, that won't work of course
+    <braunr> do we actually have such cases btw ?
+    <braunr> procfs perhaps
+    <youpi> I don't think we'd want it actually
+    <braunr> but isn't that required sometimes ?
+    <youpi> inside a shell script, yes
+    <braunr> for example, a storeio translator could ask about the priority
+      properties of the client to proc
+    <youpi> but I don't remember a case where an external translator would need
+      the access
+    <youpi> well, that's actually what we want
+    <youpi> we don't want to fool the storeio with user-provided data :)
+    <braunr> yes
+    <youpi> unless the user starts the storeio himself, in which case he will
+      have to re-root it
+    <braunr> so it has to locate the right translator, despite not using the
+      remap root translator
+    <youpi> err, it will already
+    <youpi> by just using the system's path
+    <braunr> ?
+    <youpi> maybe you need to say exactly what "it" and "right" are :)
+    <braunr> ok, let's imagine your previous example with a subhurd and pfinet
+    <braunr> the remap translator would imply that users from the subhurd
+      *directly* access all services from the main hurd, except when routed
+      otherwise by the remap translator to pfinet
+    <youpi> by "directly", I mean asking the remap translator, which gives as
+      answer "not me, the root"
+    <braunr> now, what if a translator in the main hurd wants e.g. network
+      stats from pfinet, it will ask the main one, not the one obtained through
+      remap
+    <braunr> yes
+    <youpi> that's completely fine
+    <braunr> ah
+    <braunr> that's fine if the results don't matter
+    <youpi> to get network stats from the user pfinet you'd have to be inside
+      the shell using the remap translator
+    <braunr> otherwise they're inconsistent
+    <braunr> yes
+    <youpi> I don't see why you'd want to get the pfinet stats from outside
+    <youpi> you mean ethernet board usage?
+    <braunr> service interactions
+    <braunr> i can't think of anything relevant with pfinet
+    <braunr> but imagine pflocal and credentials
+    <youpi> I believe that'd still be ok
+    <youpi> i.e. things outside the remap want to know the actual system things
+    <youpi> while things inside want to know the remapped things
+    <youpi> and you need that to avoid getting fooled by the user remapping
+    <braunr> for credentials, i think it works because the client provides
+      rights, so it would provide rights to the remapped translators in this
+      case
+    <braunr> this would need to be generalized
+    <youpi> I believe it's already general
+    <braunr> well no
+    <braunr> procfs for example will always talk to the "true" proc server
+    <youpi> sure
+    <youpi> that's what I want from the outside
+    <youpi> if the user, from the inside, wants another view, he'll have to
+      start another procfs
+    <youpi> his own one
+    <braunr> ok
+    <youpi> attached to the remapping
+
+
+## IRC, freenode, #hurd, 2013-01-29
+
+    <youpi> ok, the remap translator was too easy
+    <youpi> just took fakeroot.c
+    <youpi> added if (!strcmp("bin/foo", filename))         filename =
+      "bin/bash"; in 
+    <youpi> netfs_S_dir_lookup
+    <youpi> and it just works
+
+[[hurd/interface/dir_lookup]].
+
+    <youpi> ok, remap does indeed take my own pfinet
+    <youpi> good :)
+    <youpi> pfinet's tun seems to be working too
+    <youpi> it's however not really flexible, it has to show up in /dev/tunx
+    <youpi> I'll have a look at fixing that
+    <youpi> yep, works fine
+
+
+## IRC, freenode, #hurd, 2013-02-01
+
+    <youpi> braunr: as I expected, simply passing FS_RETRY_REAUTH does the
+      remapping trick
+
+
+# IRC, freenode, #hurd, 2013-02-12
+
+    <braunr>
+      http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/server_overriding/
+    <braunr> youpi: isn't that your remap translator ?
+    <youpi> completely
+    <youpi> remap being (5)
+
+
+# IRC, freenode, #hurd, 2013-02-25
+
+    <youpi> I'm just having an issue with getcwd getting in the sky
+    <youpi> I wonder whether libc might need patching to understand it's in
+      some sort of chroot
+    <youpi> or perhaps remap fixed into avoiding .. of / being odd
+    <youpi> erf, it's actually an explicit error
+    <youpi> libc just doesn't want to have a ".." / being different from CRDIR
+    <youpi> let me just comment out that :)
+    <youpi> way better :)
+    <youpi> yep, just works fine
+
+
+# IRC, freenode, #hurd, 2013-03-16
+
+    <braunr> youpi: is the /bin/remap --help output correct ?
+
+
+# [[hurd/fsysopts]]
+
+Doesn't support [[hurd/fsysopts]].