Merge remote-tracking branch 'fp/master'

15 files changed, 815 insertions, 4 deletions

diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn
index f7136056..fd24f081 100644
--- a/hurd/debugging/rpctrace.mdwn
+++ b/hurd/debugging/rpctrace.mdwn
@@ -52,6 +52,43 @@ See `rpctrace --help` about how to use it.
         <antrik> note that there is a number of known bugs in rpctrace, for which zhengda has sent patches... though I haven't reviewed all of them I think
         <antrik> there are some nasty Mach operations that are really hard to proxy -- but I don't think the auth mechanism needs any of these...
 
+* IRC, freenode, #hurd, 2011-11-04
+
+    [[!taglink open_issue_documentation]]
+
+        <mcsim> hello. Are there any documentation about understanding output
+          of rpctrace?
+        <braunr> no
+        <braunr> you should read the source code, best doc available
+        <braunr> if you have too many numbers and almost no symbolc names,
+          you're lacking rpc definition lists
+        <braunr> check that the gnumach-common package is installed, as it
+          provides the gnumach definitions
+        <braunr> (the glibc ones are almost always available)
+        <braunr> with those two, you should be fine for the beginning
+        <mcsim> gnumach-common is installed. And what is the name for glibc
+          package for gnumach definitions.
+        <mcsim> Also I'm using libraries specified by LD_LIBRARY_PATH. Does it
+          make influence on absence of symbolic names?
+        <braunr> no
+        <braunr> rpctrace --help
+        <braunr> see the --rpc-list=FILE option
+        <braunr> the default lists are in /usr/share/msgids/, with the .msgids
+          extension
+        <braunr> $ dpkg -S msgids
+        <braunr> gnumach-common: /usr/share/msgids/gnumach.msgids
+        <braunr> hurd: /usr/share/msgids/hurd.msgids
+        <braunr> ok, glibc has none, it's the hurd
+        <braunr> for more details about the output, read the source code
+        <braunr> it shouldn't be that hard to grasp
+        <mcsim> -I /usr/share/msgids helped
+        <mcsim> thank you
+        <braunr> it shouldn't have, it's the default path
+        <mcsim> but symbolic names appeared
+        <braunr> well, that's weird :)
+        <pinotree> braunr: the output of rpctrace --help should tell the
+          default dir for msgids
+
 
 # See Also
 
diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
index a9317c21..5228515f 100644
--- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
+++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
@@ -233,3 +233,40 @@ See also: [[open_issues/resource_management_problems/pagers]].
       I have never looked at it.
 
 [[open_issues/mach-defpager_vs_defpager]].
+
+
+# IRC, freenode, #hurd, 2011-11-08
+
+    <mcsim> who else uses defpager besides tmpfs and kernel?
+    <braunr> normally, nothing directly
+    <mcsim> than why tmpfs should use defpager?
+    <braunr> it's its backend
+    <braunr> backign store rather
+    <braunr> the backing store of most file systems are partitions
+    <braunr> tmpfs has none, it uses the swap space
+    <mcsim> if we allocate memory for tmpfs using vm_allocate, will it be able
+      to use swap partition?
+    <braunr> it should
+    <braunr> vm_allocate just maps anonymous memory
+    <braunr> anonymous memory uses swap space as its backing store too
+    <braunr> but be aware that this part of the vm system is known to have
+      deficiencies
+    <braunr> which is why all mach based implementations have rewritten their
+      default pager
+    <mcsim> what kind of deficiencies?
+    <braunr> bugs
+    <braunr> and design issues, making anonymous memory fragmentation horrible
+    <antrik> mcsim: vm_allocate doesn't return a memory object; so it can't be
+      passed to clients for mmap()
+    <mcsim> antrik: I use vm_allocate in pager_read_page
+    <antrik> mcsim: well, that means that you have to actually implement a
+      pager yourself
+    <antrik> also, when the kernel asks the pager to write back some pages, it
+      expects the memory to become free. if you are "paging" to ordinary
+      anonymous memory, this doesn't happen; so I expect it to have a very bad
+      effect on system performance
+    <antrik> both can be avoided by just passing a real anonymous memory
+      object, i.e. one provided by the defpager
+    <antrik> only problem is that the current defpager implementation can't
+      really handle that...
+    <antrik> at least that's my understanding of the situation
diff --git a/hurd/virtual_file_system/discussion.mdwn b/hurd/virtual_file_system/discussion.mdwn
new file mode 100644
index 00000000..9e12d01e
--- /dev/null
+++ b/hurd/virtual_file_system/discussion.mdwn
@@ -0,0 +1,39 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation]]
+
+IRC, freenode, #hurd, 2011-11-12:
+
+    <sea4ever> So hurd implements a 'transparent translator' somewhere which
+      just passes all IO calls to the posix IO I'm used to? (i.e. read, write,
+      open, close, etc.?)
+    <youpi> it's the normal way of operation
+    <youpi> glibc's read() doesn't do a system call, it always does an RPC to
+      the underlying translator
+    <youpi> be it ext2fs for /, or your foobarfs for your node
+    <sea4ever> Ok that makes sense. How does one program know which translator
+      it should refer to though?
+    <sea4ever> the read() call magically knows which process to invoke?
+    <youpi> the / translator is always known
+    <youpi> and then you ask /'s translator about /home, then /home/you, then
+      /home/you/foobar
+    <youpi> it tells you which other translator tyou have to contact
+    <youpi> that's on open
+    <sea4ever> It's a tree! Ok.
+    <youpi> the notion of fd is then simply knowing the translator
+    <sea4ever> Right. 'file descriptor' is now 'translator address descriptor'
+      maybe.
+    <youpi> it's glibc which knows about FDs, nothing else knows
+    <youpi> yes
+    <youpi> actually an RPC port, simply
+    <sea4ever> I want to try out the new RPC mechanism that mach implements
+    <youpi> err, which "new" RPC ?
+    <youpi> mach's RPCs are very old actually :)
diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn
index 43b99d83..ca2f42c4 100644
--- a/microkernel/mach/gnumach/memory_management.mdwn
+++ b/microkernel/mach/gnumach/memory_management.mdwn
@@ -80,3 +80,25 @@ IRC, freenode, #hurd, 2011-06-09
     <braunr> wow
     <braunr> i remember the linux support for 4G/4G split when there was enough
       RAM to fill the kernel space with struct page entries
+
+
+IRC, freenode, #hurd, 2011-11-12
+
+    <youpi> well, the Hurd doesn't "artificially" limits itself to 1.5GiB
+      memory
+    <youpi> i386 has only 4GiB addressing space
+    <youpi> we currently chose 2GiB for the kernel and 2GiB for the userspace
+    <youpi> since kernel needs some mappings, that leaves only 1.5GiB usable
+      physical memory
+    <sea4ever`> Hm? 2GiB for kernel, 2GiB for userspace, 500MiB are used for
+      what?
+    <youpi> for mappings
+    <youpi> such as device iomap
+    <youpi> contiguous buffer allocation
+    <youpi> and such things
+    <sea4ever`> Ah, ok. You map things in kernel space into user space then.
+    <youpi> linux does the same without the "bigmem" support
+    <youpi> no, just in kernel space
+    <youpi> kernel space is what determines how much physical memory you can
+      address
+    <youpi> unless using the linux-said-awful "bigmem" support
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn
index 46526641..13599e19 100644
--- a/open_issues/anatomy_of_a_hurd_system.mdwn
+++ b/open_issues/anatomy_of_a_hurd_system.mdwn
@@ -87,7 +87,7 @@ RPC stubs.
 
 More stuff like [[hurd/IO_path]].
 
---
+---
 
 IRC, freenode, #hurd, 2011-10-18:
 
@@ -96,3 +96,29 @@ IRC, freenode, #hurd, 2011-10-18:
     <antrik> short version: grub loads mach, ext2, and ld.so/exec; mach starts
       ext2; ext2 starts exec; ext2 execs a few other servers; ext2 execs
       init. from there on, it's just standard UNIX stuff
+
+---
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+    <sekon_> is __dir_lookup a RPC ??
+    <sekon_> where can i find the source of __dir_lookup ??
+    <sekon_> grepping most gives out rvalue assignments 
+    <sekon_> -assignments 
+    <sekon_> but in hurs/fs.h it is used as a function ??
+    <pinotree> it should be the mig-generated function for that rpc
+    <sekon_> how do i know how its implemented ??
+    <sekon_> is there any way to delve deeprer into mig-generated functions 
+    <tschwinge> sekon_: The MIG-generated stuff will either be found in the
+      package's build directory (if it's building it for themselves), or in the
+      glibc build directory (libhurduser, libmachuser; which are all the
+      available user RPC stubs).
+    <tschwinge> sekon_: The implementation can be found in the various Hurd
+      servers/libraries.
+    <tschwinge> sekon_: For example, [hurd]/libdiskfs/dir-lookup.c.
+    <tschwinge> sekon_: What MIG does is provide a function call interface for
+      these ``functions'', and the Mach microkernel then dispatches the
+      invocation to the corresponding server, for example a /hurd/ext2fs file
+      system (via libdiskfs).
+    <tschwinge> sekon_: This may help a bit:
+      http://www.gnu.org/software/hurd/hurd/hurd_hacking_guide.html
diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
index c0d0867b..075533e7 100644
--- a/open_issues/ext2fs_page_cache_swapping_leak.mdwn
+++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
@@ -12,7 +12,10 @@ License|/fdl]]."]]"""]]
 
 There is a [[!FF_project 272]][[!tag bounty]] on this task.
 
-IRC, OFTC, #debian-hurd, 2011-03-24
+[[!toc]]
+
+
+# IRC, OFTC, #debian-hurd, 2011-03-24
 
     <youpi> I still believe we have an ext2fs page cache swapping leak, however
     <youpi> as the 1.8GiB swap was full, yet the ld process was only 1.5GiB big
@@ -24,7 +27,7 @@ IRC, OFTC, #debian-hurd, 2011-03-24
     <youpi> yes
     <youpi> the disk content, basicallyt :)
 
-IRC, freenode, #hurd, 2011-04-18
+# IRC, freenode, #hurd, 2011-04-18
 
     <antrik> damn, a cp -a simply gobbles down swap space...
     <braunr> really ?
@@ -173,3 +176,84 @@ IRC, freenode, #hurd, 2011-04-18
       backing store of memory objects created from its pager
     <braunr> so you can view swap as the file system for everything that isn't
       an external memory object
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+    <braunr> hm, now my system got unstable
+    <braunr> swap is increasing, without any apparent reason
+    <antrik> you mean without any load?
+    <braunr> with load, yes
+    <braunr> :)
+    <antrik> well, with load is "normal"...
+    <antrik> at least for some loads
+    <braunr> i can't create memory pressure to stress reclaiming without any
+      load
+    <antrik> what load are you using?
+    <braunr> ftp mirrorring
+    <antrik> hm... never tried that; but I guess it's similar to apt-get
+    <antrik> so yes, that's "normal". I talked about it several times, and also
+      wrote to the ML
+    <braunr> antrik: ok
+    <antrik> if you find out how to fix this, you are my hero ;-)
+    <braunr> arg :)
+    <antrik> I suspect it's the infamous double swapping problem; but that's
+      just a guess
+    <braunr> looks like this
+    <antrik> BTW, if you give me the exact command, I could check if I see it
+      too
+    <braunr> i use lftp (mirror -Re) from a linux git repository
+    <braunr> through sftp
+    <braunr> (lots of small files, big content)
+    <antrik> can't you just give me the exact command? I don't feel like
+      figuring it out myself
+    <braunr> antrik: cd linux-stable; lftp sftp://hurd_addr/
+    <braunr> inside lftp: mkdir linux-stable; cd linux-stable; mirror -Re
+    <braunr> hm, half of physical memory just got freed
+    <braunr> our page cache is really weird :/
+    <braunr> (i didn't delete any file when that happened)
+    <antrik> hurd_addr?
+    <braunr> ssh server ip address
+    <braunr> or name
+    <braunr> of your hurd :)
+    <antrik> I'm confused. you are mirroring *from* the Hurd box?
+    <braunr> no, to it
+    <antrik> ah, so you login via sftp and then push to it?
+    <braunr> yes
+    <braunr> fragmentation looks very fine
+    <braunr> even for the huge pv_entry cache and its 60k+ entries
+    <braunr> (and i'm running a kernel with the cpu layer enabled)
+    <braunr> git reset/status/diff/log/grep all work correctly
+    <braunr> anyway, mcsim's branch looks quite stable to me
+    <antrik> braunr: I can't reproduce the swap leak with ftp. free memory
+      idles around 6.5 k (seems to be the threshold where paging starts), and
+      swap use is constant
+    <antrik> might be because everything swappable is already present in swap
+      from previous load I guess...
+    <antrik> err... scratch that. was connected to the wrong host, silly me
+    <antrik> indeed swap gets eaten away, as expected
+    <antrik> but only if free memory actually falls below the
+      threshold. otherwise it just oscillates around a constant value, and
+      never touches swap
+    <antrik> so this seems to confirm the double swapping theory
+    <youpi> antrik: is that "double swap" theory written somewhere?
+    <youpi> (no, a quick google didn't tell me)
+
+
+## IRC, freenode, #hurd, 2011-11-16
+
+    <antrik> youpi:
+      http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html talks
+      about "double paging". probably it's also the term others used for it;
+      however, the term is generally used in a completely different meaning, so
+      I guess it's not really suitable for googling either ;-)
+    <antrik> IIRC slpz (or perhaps someone else?) proposed a solution to this,
+      but I don't remember any details
+    <youpi> ok so it's the same thing I was thinking about with swap getting
+      filled
+    <youpi> my question was: is there something to release the double swap,
+      once the ext2fs pager managed to recover?
+    <antrik> apparently not
+    <antrik> the only way to free the memory seems to be terminating the FS
+      server
+    <youpi> uh :/
diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn
index 9a4418c1..c9c3e64f 100644
--- a/open_issues/gnumach_memory_management.mdwn
+++ b/open_issues/gnumach_memory_management.mdwn
@@ -1810,3 +1810,205 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task.
 
     <braunr> etenil: but mcsim's work is, for one, useful because the allocator
       code is much clearer, adds some debugging support, and is smp-ready
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+    <braunr> i've just realized that replacing the zone allocator removes most
+      (if not all) static limit on allocated objects
+    <braunr> as we have nothing similar to rlimits, this means kernel resources
+      are actually exhaustible
+    <braunr> and i'm not sure every allocation is cleanly handled in case of
+      memory shortage
+    <braunr> youpi: antrik: tschwinge: is this acceptable anyway ?
+    <braunr> (although IMO, it's also a good thing to get rid of those limits
+      that made the kernel panic for no valid reason)
+    <youpi> there are actually not many static limits on allocated objects
+    <youpi> only a few have one
+    <braunr> those defined in kern/mach_param.h
+    <youpi> most of them are not actually enforced
+    <braunr> ah ?
+    <braunr> they are used at zinit() time
+    <braunr> i thought they were
+    <youpi> yes,  but most zones are actually fine with overcoming the max
+    <braunr> ok
+    <youpi> see zone->max_size += (zone->max_size >> 1);
+    <youpi> you need both !EXHAUSTIBLE and FIXED
+    <braunr> ok
+    <pinotree> making having rlimits enforced would be nice...
+    <pinotree> s/making//
+    <braunr> pinotree: the kernel wouldn't handle many standard rlimits anyway
+
+    <braunr> i've just committed my final patch on mcsim's branch, which will
+      serve as the starting point for integration
+    <braunr> which means code in this branch won't change (or only last minute
+      changes)
+    <braunr> you're invited to test it
+    <braunr> there shouldn't be any noticeable difference with the master
+      branch
+    <braunr> a bit less fragmentation
+    <braunr> more memory can be reclaimed by the VM system
+    <braunr> there are debugging features
+    <braunr> it's SMP ready
+    <braunr> and overall cleaner than the zone allocator
+    <braunr> although a bit slower on the free path (because of what's
+      performed to reduce fragmentation)
+    <braunr> but even "slower" here is completely negligible
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+    <mcsim> I enabled cpu_pool layer and kentry cache exhausted at "apt-get
+      source gnumach && (cd gnumach-* && dpkg-buildpackage)"
+    <mcsim> I mean kernel with your last commit
+    <mcsim> braunr: I'll make patch how I've done it in a few minutes, ok? It
+      will be more specific.
+    <braunr> mcsim: did you just remove the #if NCPUS > 1 directives ?
+    <mcsim> no. I replaced macro NCPUS > 1 with SLAB_LAYER, which equals NCPUS
+      > 1, than I redefined macro SLAB_LAYER
+    <braunr> ah, you want to make the layer optional, even on UP machines
+    <braunr> mcsim: can you give me the commands you used to trigger the
+      problem ?
+    <mcsim> apt-get source gnumach && (cd gnumach-* && dpkg-buildpackage)
+    <braunr> mcsim: how much ram & swap ?
+    <braunr> let's see if it can handle a quite large aptitude upgrade
+    <mcsim> how can I check swap size?
+    <braunr> free
+    <braunr> cat /proc/meminfo
+    <braunr> top
+    <braunr> whatever
+    <mcsim>              total       used       free     shared    buffers
+      cached
+    <mcsim> Mem:        786368     332296     454072          0          0
+      0
+    <mcsim> -/+ buffers/cache:     332296     454072
+    <mcsim> Swap:      1533948          0    1533948
+    <braunr> ok, i got the problem too
+    <mcsim> braunr: do you run hurd in qemu?
+    <braunr> yes
+    <braunr> i guess the cpu layer increases fragmentation a bit
+    <braunr> which means more map entries are needed
+    <braunr> hm, something's not right
+    <braunr> there are only 26 kernel map entries when i get the panic
+    <braunr> i wonder why the cache gets that stressed
+    <braunr> hm, reproducing the kentry exhaustion problem takes quite some
+      time
+    <mcsim> braunr: what do you mean?
+    <braunr> sometimes, dpkg-buildpackage finishes without triggering the
+      problem
+    <mcsim> the problem is in apt-get source gnumach
+    <braunr> i guess the problem happens because of drains/fills, which
+      allocate/free much more object than actually preallocated at boot time
+    <braunr> ah ?
+    <braunr> ok
+    <braunr> i've never had it at that point, only later
+    <braunr> i'm unable to trigger it currently, eh
+    <mcsim> do you use *-dbg kernel?
+    <braunr> yes
+    <braunr> well, i use the compiled kernel, with the slab allocator, built
+      with the in kernel debugger
+    <mcsim> when you run apt-get source gnumach, you run it in clean directory?
+      Or there are already present downloaded archives?
+    <braunr> completely empty
+    <braunr> ah just got it
+    <braunr> ok the limit is reached, as expected
+    <braunr> i'll just bump it
+    <braunr> the cpu layer drains/fills allocate several objects at once (64 if
+      the size is small enough)
+    <braunr> the limit of 256 (actually 252 since the slab descriptor is
+      embedded in its slab) is then easily reached
+    <antrik> mcsim: most direct way to check swap usage is vmstat
+    <braunr> damn, i can't live without slabtop and the amount of
+      active/inactive cache memory any more
+    <braunr> hm, weird, we have active/inactive memory in procfs, but not
+      buffers/cached memory
+    <braunr> we could set buffers to 0 and everything as cached memory, since
+      we're currently unable to communicate the purpose of cached memory
+      (whether it's used by disk servers or file system servers)
+    <braunr> mcsim: looks like there are about 240 kernel map entries (i forgot
+      about the ones used in kernel submaps)
+    <braunr> so yes, addin the cpu layer is what makes the kernel reach the
+      limit more easily
+    <mcsim> braunr: so just increasing limit will solve the problem?
+    <braunr> mcsim: yes
+    <braunr> slab reclaiming looks very stable
+    <braunr> and unfrequent
+    <braunr> (which is surprising)
+    <pinotree> braunr: "unfrequent"?
+    <braunr> pinotree: there isn't much memory pressure
+    <braunr> slab_collect() gets called once a minute on my hurd
+    <braunr> or is it infrequent ?
+    <braunr> :)
+    <pinotree> i have no idea :)
+    <braunr> infrequent, yes
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+    <braunr> for those who want to play with the slab branch of gnumach, the
+      slabinfo tool is available at http://git.sceen.net/rbraun/slabinfo.git/
+    <braunr> for those merely interested in numbers, here is the output of
+      slabinfo, for a hurd running in kvm with 512 MiB of RAM, an unused swap,
+      and a short usage history (gnumach debian packages built, aptitude
+      upgrade for a dozen of packages, a few git commands)
+    <braunr> http://www.sceen.net/~rbraun/slabinfo.out
+    <antrik> braunr: numbers for a long usage history would be much more
+      interesting :-)
+
+
+## IRC, freenode, #hurd, 2011-11-17
+
+    <braunr> antrik: they'll come :)
+    <etenil> is something going on on darnassus? it's mighty slow
+    <braunr> yes
+    <braunr> i've rebooted it to run a modified kernel (with the slab
+      allocator) and i'm building stuff on it to stress it
+    <braunr> (i don't have any other available machine with that amount of
+      available physical memory)
+    <etenil> ok
+    <antrik> braunr: probably would be actually more interesting to test under
+      memory pressure...
+    <antrik> guess that doesn't make much of a difference for the kernel object
+      allocator though
+    <braunr> antrik: if ram is larger, there can be more objects stored in
+      kernel space, then, by building something large such as eglibc, memory
+      pressure is created, causing caches to be reaped
+    <braunr> our page cache is useless because of vm_object_cached_max
+    <braunr> it's a stupid arbitrary limit masking the inability of the vm to
+      handle pressure correctly 
+    <braunr> if removing it, the kernel freezes soon after ram is filled
+    <braunr> antrik: it may help trigger the "double swap" issue you mentioned
+    <antrik> what may help trigger it?
+    <braunr> not checking this limit
+    <antrik> hm... indeed I wonder whether the freezes I see might have the
+      same cause
+
+
+## IRC, freenode, #hurd, 2011-11-19
+
+    <braunr> http://www.sceen.net/~rbraun/slabinfo.out <= state of the slab
+      allocator after building the debian libc packages and removing all files
+      once done
+    <braunr> it's mostly the same as on any other machine, because of the
+      various arbitrary limits in mach (most importantly, the max number of
+      objects in the page cache)
+    <braunr> fragmentation is still quite low
+    <antrik> braunr: actually fragmentation seems to be lower than on the other
+      run...
+    <braunr> antrik: what makes you think that ?
+    <antrik> the numbers of currently unused objects seem to be in a similar
+      range IIRC, but more of them are reclaimable I think
+    <antrik> maybe I'm misremembering the other numbers
+    <braunr> there had been more reclaims on the other run
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+    <braunr> mcsim: i've just updated the slab branch, please review my last
+      commit when you have time
+    <mcsim> braunr: Do you mean compilation/tests?
+    <braunr> no, just a quick glance at the code, see if it matches what you
+      intended with your original patch
+    <mcsim> braunr: everything is ok
+    <braunr> good
+    <braunr> i think the branch is ready for integration
diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
index 93055b77..80fc9fcd 100644
--- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
+++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
@@ -54,3 +54,53 @@ License|/fdl]]."]]"""]]
       compatibility with eventual 3rd party users is not broken
     <pinotree> but those using them, other than hurd itself, won't compile
       anymore, so you fix them progressively
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+    <braunr> is the mach_debug interface packaged in debian ?
+    <antrik> what mach_debug interface?
+    <braunr> include/include/mach_debug/mach_debug.defs in gnumach
+    <braunr> include/mach_debug/mach_debug.defs in gnumach
+    <antrik> what exactly is supposed to be packaged there?
+    <braunr> i'm talking about the host_*_info client code
+    <antrik> braunr: you mean MIG-generated stubs?
+    <braunr> antrik: yes
+    <braunr> i wrote a tiny slabinfo tool, and rpctrace doesn't show the
+      host_slab_info call
+    <braunr> do you happen to know why ?
+    <antrik> braunr: doesn't show it at all, or just doesn't translate?
+    <braunr> antrik: doesn't at all, the msgids file contains what's needed to
+      translate
+    <braunr> btw, i was able to build the libc0.3 packages with a kernel using
+      the slab allocator today, while monitoring it with the aforementioned
+      slabinfo tool, everything went smoothly
+    <antrik> great :-)
+    <braunr> i'll probably add a /proc/slabinfo entry some day
+    <braunr> and considering the current state of our beloved kernel, i'm
+      wondering why host_*_info rpcs are considered debugging calls
+    <braunr> imo, they should always be included by default
+    <braunr> and part of the standard mach interface
+    <braunr> (if the rpc is missing, an error is simply returned)
+    <antrik> I guess that's been inherited from original Mach
+    <antrik> so you think the stubs should be provided by libmachuser?
+    <braunr> i'm not sure
+    <antrik> actually, it's a bit arguable. if interfaces are not needed by
+      libc itself, it isn't really necessary to build them as part of the libc
+      build...
+    <braunr> i don't know the complete list of potential places for such calls
+    <antrik> OTOH, as any updates will happen in sync with other Mach updates,
+      it makes sense to keep them in one place, to reduce transition pain
+    <braunr> and i didn't want to imply they should be part of libc
+    <braunr> on the contrary, libmachuser seems right
+    <antrik> libmachuser is part of libc
+    <braunr> ah
+    <braunr> :)
+    <braunr> why so ?
+    <antrik> well, for one, libc needs the Mach (and Hurd) stubs itself
+    <antrik> also, it's traditionally the role of libc to provide the call
+      wrappers for syscalls... so it makes some sense
+    <braunr> sure, but why doesn't it depend on an external libmachuser instead
+      of embedding it ?
+    <braunr> right
+    <antrik> now that's a good question... no idea TBH :-)
diff --git a/open_issues/mig_portable_rpc_declarations.mdwn b/open_issues/mig_portable_rpc_declarations.mdwn
new file mode 100644
index 00000000..084d7454
--- /dev/null
+++ b/open_issues/mig_portable_rpc_declarations.mdwn
@@ -0,0 +1,58 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_mig]]
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+    <braunr> also, what's the best way to deal with types such as
+    <braunr> type cache_info_t = struct[23] of integer_t;
+    <braunr> whereas cache_info_t contains longs, which are obviously not
+      integer-wide on 64-bits processors
+    <braunr> ?
+    <youpi> you mean, to port mach to 64bit?
+    <braunr> no, to make the RPC declaration portable
+    <braunr> just in case :)
+    <youpi> refine integer_t into something more precise
+    <youpi> such as size_t, off_t, etc.
+    <braunr> i can't use a single line then
+    <braunr> struct cache_info contains ints, vm_size_t, longs
+    <braunr> should i just use the maximum size it can get ?
+    <braunr> or declare two sizes depending on the word size ?
+    <youpi> well, I'd say three
+    <braunr> youpi: three ?
+    <youpi> the ints, the vm_size_ts, and the longs
+    <braunr> youpi: i don't get it
+    <braunr> youpi: how would i write it in mig language ?
+    <youpi> I don't know the mig language
+    <braunr> me neither :)
+    <youpi> but I'd say don't lie
+    <braunr> i just see struct[23] of smething
+    <braunr> the original zone_info struct includes both integer_t and
+      vm_size_t, and declares it as
+    <braunr> type zone_info_t = struct[9] of integer_t;
+    <braunr> in its mig defs file
+    <braunr> i don't have a good example to reuse
+    <youpi> which is lying
+    <braunr> yes
+    <braunr> which is why i was wondering if mach architects themselves
+      actually solved that problem :)
+    <braunr> "There is no way to specify the fields of  a
+    <braunr> C  structure  to  MIG. The size and type-desc are just used to
+      give the size of
+    <braunr> the structure.
+    <braunr> "
+    <braunr> well, this sucks :/
+    <braunr> well, i'll do what the rest of the code seems to do, and let it
+      rot until a viable solution is available
+    <antrik> braunr: we discussed the problem of expressing structs with MIG in
+      the libburn thread
+    <antrik> (which I still need to follow up on... [sigh])
diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn
index 212d65e7..d136e3a8 100644
--- a/open_issues/mission_statement.mdwn
+++ b/open_issues/mission_statement.mdwn
@@ -10,7 +10,10 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_documentation]]
 
-IRC, freenode, #hurd, 2011-10-12:
+[[!toc]]
+
+
+# IRC, freenode, #hurd, 2011-10-12
 
     <ArneBab> we have a mission statement: http://hurd.gnu.org
     <Gorodish> yes
@@ -37,3 +40,10 @@ IRC, freenode, #hurd, 2011-10-12:
       ceases to amaze me
     <Gorodish> I agree that the informational, factual, results oriented
       documentation is the primary objective of documenting
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+    <antrik> heh, nice: http://telepathy.freedesktop.org/wiki/Rationale
+    <antrik> most of this could be read as a rationale for the Hurd just as
+      well ;-)
diff --git a/open_issues/page_cache.mdwn b/open_issues/page_cache.mdwn
new file mode 100644
index 00000000..062fb8d6
--- /dev/null
+++ b/open_issues/page_cache.mdwn
@@ -0,0 +1,73 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach]]
+
+IRC, freenode, #hurd, 2011-11-28:
+
+    <braunr> youpi: would you find it reasonable to completely disable the page
+      cache in gnumach ?
+    <braunr> i'm wondering if it wouldn't help make the system more stable
+      under memory pressure
+    <youpi> assuming cache=writeback in gnumach?
+    <youpi> because disabling the page cache will horribly hit performance
+    <braunr> no, it doesn't have anything to do with the host
+    <braunr> i'm not so sure
+    <braunr> while observing the slab allocator, i noticed our page cache is
+      not used that often
+    <youpi> eeh?
+    <youpi> apart from the damn 4000 limitation, I've seen it used
+    <youpi> (and I don't  why it wouldn't be used)
+    <youpi> (e.g. for all parts of libc)
+    <youpi> ah, no, libc would be kept open by ext2fs
+    <braunr> taht's precisely because of the 4k limit
+    <youpi> but e.g. .o file emitted during make
+    <braunr> well, no
+    <youpi> well, see the summary I had posted some time ago, the 4k limit
+      makes it completely randomized
+    <youpi> and thus you lose locality
+    <braunr> yes
+    <youpi> but dropping the limit would just fix it
+    <braunr> that's my point
+    <youpi> which I had tried to do, and there were issues, you mentioned why
+    <youpi> and (as usual), I haven't had anyu time to have a look at the issue
+      again
+    <braunr> i'm just trying to figure out the pros and cons for having teh
+      current page cache implementation
+    <braunr> but are you saying you tried with a strict limit of 0 ?
+    <youpi> non, I'm saying I tried with no limit
+    <youpi> but then memory fills up
+    <braunr> yes
+    <youpi> so trying to garbage collect
+    <braunr> i tried that too, the system became unstable very quickly
+    <youpi> but refs don't falldown to 0, you said
+    <braunr> did i ?
+    <youpi> or maybe somebody else
+    <youpi> see the list archives
+    <braunr> that's possible
+    <braunr> i'd imagine someone like sergio lopez
+    <youpi> possibly
+    <youpi> somebody that knows memory stuff way better than me in any case
+    <braunr> youpi: i'm just wondering how much we'd loose by disabling the
+      page cache, and if we actually gain more stability (and ofc, if it's
+      worth it)
+    <youpi> no idea, measures will tell
+    <youpi> fixing the page cache shouldn't be too hard I believe, however
+    <youpi> you just need to know what you are doing, which I don't
+    <youpi> I do believe the cache is still at least a bit useful
+    <youpi> even if dumb because of randomness
+    <youpi> e.g. running make lib in the glibc tree gets faster on second time
+    <youpi> because the cache wouldbe filled at least randomly with glibc tree
+      stuff
+    <braunr> yes, i agree on that
+    <youpi> braunr: btw, the current stability is fine for the buildds
+    <youpi> restarting them every few days is ok
+    <youpi> so I'd rather keep the performance :)
+    <braunr> ok
diff --git a/open_issues/perl.mdwn b/open_issues/perl.mdwn
index 45680328..48343e3e 100644
--- a/open_issues/perl.mdwn
+++ b/open_issues/perl.mdwn
@@ -36,6 +36,56 @@ First, make the language functional, have its test suite pass without errors.
 
 [[!inline pages=community/gsoc/project_ideas/perl_python feeds=no]]
 
+
+## IRC, OFTC, #debian-hurd, 2011-11-08
+
+    <Dom> pinotree: so, with your three fixes applied to 5.14.2, there are
+      still 9 tests failing. They don't seem to be regressions in perl, since
+      they also fail when I build 5.14.0 (even though the buildd managed it).
+    <Dom> What do you suggest as the way forward?
+    <Dom> (incidentally I'm trying on strauss's sid chroot to see how that
+      compares)
+    <pinotree> Dom: samuel makes buildds build perl with nocheck (otherwise
+      we'd have no perl at all)
+    <pinotree> which tests still fail?
+    <Dom> ../cpan/Sys-Syslog/t/syslog.t ../cpan/Time-HiRes/t/HiRes.t
+      ../cpan/autodie/t/recv.t ../dist/IO/t/io_pipe.t ../dist/threads/t/libc.t
+      ../dist/threads/t/stack.t ../ext/Socket/t/socketpair.t io/pipe.t
+      op/sigdispatch.t
+    <Dom> buildds> I see
+    <pinotree> ah ok, those that are failing for me even with my patches
+    <Dom> I hadn't spotted that the builds were done with nocheck.
+    <Dom> (but only sometimes...)
+    <Dom> Explains a lot
+    <pinotree> syslog is kind of non-working on hurd, and syslog.t succeeds in
+      buildds (as opposted to crash the machine...) because there's no /var/log
+      in chroots
+    <pinotree> libc.t appears to succeed too in buildds
+    * Dom notices how little memory strauss has, and cancels the build, now
+        that he *knows* that running out of memory caused the crahses
+    <pinotree> iirc HiRes.t, io_pipe.t , pipe.t and sigdispatch.t fails because
+      of trobules we have with posix signals
+    <pinotree> socketpair.t is kind of curious, it seems to block on
+      socketpair()...
+    * Dom wonders if a wiki page tracking this would be worthwhile
+    <pinotree> stack.t fails because we cannot set a different size for pthread
+      stacks, yet (similar failing test cases are also in the python test
+      suite)
+    <Dom> if there are problems which aren't going to get resolved any time
+      soon, it may be worth a few SKIPs conditional on architecture, depending
+      on how serious the issue is
+    <Dom> then we'd get better visibility of other/new issues
+    <Dom> (problems which aren't bugs in perl, that is)
+    <pinotree> understandable, yes
+    <pinotree> i think nobody digged much deeper in the failing ones yet, to
+      actually classify them as due to glibc/hurd/mach
+    <pinotree> (eg unlike the pipe behaviour in sysconf.t i reported)
+
+### 2011-11-26
+
+    <pinotree> cool, my recvfrom() fix also makes the perl test recv.t pass
+
+
 ---
 
 
diff --git a/open_issues/robustness.mdwn b/open_issues/robustness.mdwn
new file mode 100644
index 00000000..d32bd509
--- /dev/null
+++ b/open_issues/robustness.mdwn
@@ -0,0 +1,64 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation open_issue_hurd]]
+
+
+# IRC, freenode, #hurd, 2011-11-18
+
+    <nocturnal> I'm learning about GNU Hurd and was speculating with a friend
+      who is also a computer enthusiast. I would like to know if Hurds
+      microkernel can recover services should they crash? and if it can, does
+      that recovery code exist in multiple services or just one core kernel
+      service? 
+    <braunr> nocturnal: you should read about passive translators
+    <braunr> basically, there is no dedicated service to restore crashed
+      servers
+    <etenil> Hi everyone!
+    <braunr> services can crash and be restarted, but persistence support is
+      limited, and rather per serivce
+    <braunr> actually persistence is more a side effect than a designed thing
+    <braunr> etenil: hello
+    <etenil> braunr: translators can also be spawned on an ad-hoc basis, for
+      instance when accessing a particular file, no?
+    <braunr> that's what being passive, for a translator, means
+    <etenil> ah yeah I thought so :)
+
+
+# IRC, freenode, #hurd, 2011-11-19
+
+    <chromaticwt> will hurd ever have the equivalent of a rs server?, is that
+      even possible with hurd?
+    <youpi> chromaticwt: what is an rs server ?
+    <chromaticwt> a reincarnation server
+    <youpi> ah, like minix. Well, the main ground issue is restoring existing
+      information, such as pids of processes, etc.
+    <youpi> I don't know how minix manages it
+    <antrik> chromaticwt: I have a vision of a session manager that could also
+      take care of reincarnation... but then, knowing myself, I'll probably
+      never imlement it
+    <youpi> we do get proc  crashes from times to times
+    <youpi> it'd be cool to see the system heal itself :)
+    <braunr> i need a better description of reincarnation
+    <braunr> i didn't think it would make core servers like proc able to get
+      resurrected in a safe way
+    <antrik> depends on how it is implemented
+    <antrik> I don't know much about Minix, but I suspect they can recover most
+      core servers
+    <antrik> essentially, the condition is to make all precious state be
+      constantly serialised, and held by some third party, so the reincarnated
+      server could restore it
+    <braunr> should it work across reboots ?
+    <antrik> I haven't thought about the details of implementing it for each
+      core server; but proc should be doable I guess... it's not necessary for
+      the system to operate, just for various UNIX mechanisms
+    <antrik> well, I'm not aware of the Minix implementation working across
+      reboots. the one I have in mind based on a generic session management
+      infrastructure should though :-)
diff --git a/open_issues/syslog.mdwn b/open_issues/syslog.mdwn
index 5fec38b1..2e902698 100644
--- a/open_issues/syslog.mdwn
+++ b/open_issues/syslog.mdwn
@@ -43,3 +43,30 @@ IRC, freenode, #hurd, 2011-08-08
     < youpi> shm should work with the latest libc
     < youpi> what won't is sysv sem
     < youpi> (i.e. semget)
+
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+    * pinotree sighs at #645790 :/
+    <tschwinge> pinotree: W.r.t. 645790 -- yeah, ``someone'' should finally
+      figure out what's going on with syslog.
+      http://lists.gnu.org/archive/html/bug-hurd/2008-07/msg00152.html
+    <tschwinge> pinotree: And this...
+      http://lists.gnu.org/archive/html/bug-hurd/2007-02/msg00042.html
+    <pinotree> tschwinge: i did that 20 invocations tests recently, and
+      basically none of them has been logged
+    <pinotree> tschwinge: when i started playing with logger more, as result i
+      had some server that started taking all the cpu, followed by other
+      servers and in the end my ssh connection were dropped and i had nothing
+      to do (not even login from console)
+    <tschwinge> pinotree: Sounds like ``fun''.  Hopefully we can manage to
+      understand (and fix the underlying issue) why a simple syslog()
+      invocation can make the whole system instable.
+    <pinotree> tschwinge: to be honest, i got havoc in the system when i told
+      syslog to manually look for /dev/log (-u /dev/log), possibly alao when
+      telling to use a datagram socket (-d)
+    <pinotree> but even if a normal syslog() invocation does not cause havoc,
+      there's still the "lost messages" issue
+    <tschwinge> Yep.  What I've been doing ever since, is deinstall all
+      *syslog* packages.
+    <tschwinge> This ``fixed'' all syslog() hangs.
diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn
index 11793582..14ea1c6d 100644
--- a/open_issues/translator_stdout_stderr.mdwn
+++ b/open_issues/translator_stdout_stderr.mdwn
@@ -11,11 +11,43 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_hurd]]
 
+There have been several discussions and proposals already, about adding a
+suitable logging mechanism to translators, for example.
+
+
 Decide / implement / fix that (all?) running (passive?) translators' output
 should show up on the (Mach / Hurd) console / syslog.
 
+
 [[!taglink open_issue_documentation]]: [[!message-id
 "87oepj1wql.fsf@becket.becket.net"]]
 
+
 [[!taglink open_issue_documentation]]: Neal once had written an email on this
 topic.
+
+
+IRC, freenode, #hurd, 2011-11-06
+
+    <youpi> about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) {
+      fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); }
+    <tschwinge> Isn't stderr in auto-flush mode by default?
+    <tschwinge> man setbuf: The standard error stream stderr is always
+      unbuffered by default.
+    <youpi> tschwinge: "by default" is the important thing here
+    <youpi> in the case of translators iirc stderr is buffered
+    <tschwinge> youpi: Oh!
+    <tschwinge> That sounds wrong.
+
+
+IRC, freenode, #hurd, 2011-11-23
+
+    <braunr> we'd need a special logging task for hurd servers
+    <pinotree> if syslog would work, that could be used optionally
+    <braunr> no, it relies on services provided by the hurd
+    <braunr> i'm thinking of something using merely the mach interface
+    <braunr> e.g. using mach_msg to send log messages to a special port used to
+      reference the logging service
+    <braunr> which would then store the messages in a circular buffer in ram
+    <braunr> maybe sending to syslog if the service is available
+    <braunr> the hurd system buffer if you want