Merge remote-tracking branch 'fp/master'

7 files changed, 333 insertions, 10 deletions

diff --git a/community/gsoc/project_ideas/mtab.mdwn b/community/gsoc/project_ideas/mtab.mdwn
index a60a8038..694effca 100644
--- a/community/gsoc/project_ideas/mtab.mdwn
+++ b/community/gsoc/project_ideas/mtab.mdwn
@@ -1,12 +1,13 @@
-[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2008, 2009, 2013 Free Software Foundation,
+Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
 document under the terms of the GNU Free Documentation License, Version 1.2 or
 any later version published by the Free Software Foundation; with no Invariant
 Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
 
 [[!meta title="mtab"]]
 
@@ -72,3 +73,89 @@ Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar)
 Exercise: Make some improvement to any of the existing Hurd translators.
 Especially those in [hurdextras](http://www.nongnu.org/hurdextras/) are often
 quite rudimentary, and it shouldn't be hard to find something to improve.
+
+
+# Related Discussion
+
+## IRC, freenode, #hurd, 2013-04-17
+
+    <kuldeepdhaka> thinking how to get the listing. traversing would be
+      ineffecient,   trying to come up with something better
+    <braunr> what listing ?
+    <braunr> and traversing what ?
+    <kuldeepdhaka> mtab
+    <braunr> well i assumed so
+    <braunr> be more precise please
+    <kuldeepdhaka> when the translator is done initalized      <translation
+      info> are written to /etc/mtab      <translation info> will be provided
+      by the translator, and when some one want to read the info just read it
+      this way if their is some credentials like ftp sites pass username can be
+      masked by the translator
+    <kuldeepdhaka> if some trans dont want to list them, no need to write to
+      file    | while unmounting (sorry i couldnt find the right word) , it
+      will pass the mount node address | <translation info> will have special
+      structure to remove/add mounts example "a /mount-to /mount-from" = add
+      , "r /mount-to" = remove    here "/mount-to" will be unique for every
+      mount
+    <kuldeepdhaka> this have a draw back , we would have to trust trans for the
+      listed data  | also "/mount-to" + "/mount-from" could be used a
+      combination for making sure that other trans unable remove others trans
+      mount data
+    <kuldeepdhaka> sorry but "also "/mount-to" + "/mount-from" could be used a
+      combination for making sure that other trans unable remove others trans
+      mount data"  this is a bad idea if we had to print the whole thing
+    <kuldeepdhaka> braunr, whats ur opinion?
+    <pinotree> you don't need a mtab to "unmount" things on hurd
+    <braunr> kuldeepdhaka: hum, have you read the project idea ?
+    <braunr>
+      http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/
+    <braunr> A more promising approach is to have mtab exported by a special
+      translator, which gathers the necessary information on demand. This could
+      work by traversing the tree of translators, asking each one for mount
+      points attached to it.
+    <kuldeepdhaka> pinotree, not to unmount, i mean is to remove the
+      <translation data>
+    <braunr> for a first implementation, i'd suggest a recursive traversal of
+      root-owned translators
+    <kuldeepdhaka> braunr, hum, but it did stated it as inefficient
+    <braunr> where ?
+    <kuldeepdhaka> para 5 , line 3
+    <kuldeepdhaka> and line 6
+    <braunr> no
+    <braunr> traversing "all" nodes would be inefficient
+    <braunr> translators which host the nodes of other translators could
+      maintain a simple list of active translators
+    <braunr> ext2fs, e.g. (if that's not already the case) could keep the list
+      of the translators it started
+    <braunr> we can already see that list with pstree for example
+    <braunr> but this new list would only retain those relevant for mtab
+    <braunr> i.e. root-owned ones
+    <pinotree> i would not limit to those though
+    <braunr> and then filter on their type (e.g. file system ones)
+    <braunr> pinotree: why ?
+    <pinotree> this way you could have proper per-user /proc/$pid/mounts info
+    <braunr> we could also very easily have a denial of service
+    <kuldeepdhaka> but how will the mount point and source point will be
+      listed?
+    <braunr> they're returned by the translator
+    <kuldeepdhaka> k
+    <braunr> you ask /, it returns its store and its options, and asks its
+      children recursively 
+    <braunr> a /home translator would return its store and its options
+    <braunr> etc..
+    <braunr> each translator would build the complete path before returning it
+    <braunr> sort of, it's very basic
+    <braunr> but that would be a very hurdish way to do it
+    <kuldeepdhaka> shall /etc/mtab should be made seek-able and what should be
+      the filesize?  content are generated on demand so, it could arise problem
+      (fsize:0 , seek-able:no), ur opinions?
+    <braunr> kuldeepdhaka: it should have all the properties of a regular file
+    <braunr> the filesize would be determined after it's generated
+    <braunr> being empty doesn't imply it's not seekable
+    <kuldeepdhaka> content is generated on demand so, could cause problem while
+      seeking and filesize, shall i still program as regular file?
+    <kuldeepdhaka> in two different read, it could generate different content,
+      though same seek pos is used...
+    <braunr> what ?
+    <braunr> the content is generated on open
+    <kuldeepdhaka> ooh, ok
diff --git a/hurd/translator/ext2fs.mdwn b/hurd/translator/ext2fs.mdwn
index a89b40b1..20faed5e 100644
--- a/hurd/translator/ext2fs.mdwn
+++ b/hurd/translator/ext2fs.mdwn
@@ -137,6 +137,13 @@ small backend stores, like floppy devices.
     <youpi> ah, there is the same parameter on diskfs_start_disk_pager
 
 
+##### IRC, freenode, #hurd, 2013-04-23
+
+    <braunr> and i'm working again on the ext2fs large store patch
+    <braunr> i finished separating the libpager interface change from the rest,
+      as Thomas suggested, so a new version should be ready soon
+
+
 #### `EXT2FS_DEBUG`
 
 ##### IRC, freenode, #hurd, 2013-03-04
diff --git a/microkernel/mach/rpc/discussion.mdwn b/microkernel/mach/rpc/discussion.mdwn
index 00e4a012..ee9f059a 100644
--- a/microkernel/mach/rpc/discussion.mdwn
+++ b/microkernel/mach/rpc/discussion.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,8 +10,12 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_documentation]]
 
+[[!toc]]
 
-# IRC, freenode, #hurd, 2011-06-11
+
+# Mach initiating RPCs to userspace tasks
+
+## IRC, freenode, #hurd, 2011-06-11
 
     <antrik> I don't think we have a precendence case of Mach initiating RPCs
       to userspace tasks
@@ -115,3 +119,32 @@ License|/fdl]]."]]"""]]
     <braunr> right, i really need to read about mig again
     <antrik> it's pretty normal for a translator to both implement and use an
       interface
+
+
+# `MACH_SEND_INTERRUPT`/`MACH_RCV_INTERRUPT`
+
+[[!tag open_issue_glibc open_issue_gnumach]]
+
+
+## IRC, freenode, #hurd, 2013-03-22
+
+    <braunr> i'm also testing glibc packages on darnassus with a patch that
+      removes the MACH_{SEND,RCV}_INTERRUPT options from mach_msg calls to
+      avoid taking the slow path because of them
+    <braunr> if i got it right, almost every mach_msg call doesn't use any of
+      these options, except for select
+    <braunr> it could slightly improve that, i'm not sure
+    <youpi> but don't we need that to get EINTR ?
+    <braunr> the options are purely userspace
+    <braunr> i'll upload the patch
+    <braunr>
+      http://www.sceen.net/~rbraun/0001-Mask-options-implemented-by-the-userspace-side-of-ma.patch
+    <youpi> ah, ok, you mean userspace already implements what we need
+
+
+## IRC, freenode, #hurd, 2013-04-23
+
+    <braunr> i couldn't measure any difference with the glibc patch that
+      removes the mach_msg interrupt related flags
+    <braunr> which isn't very surprising as it only concerns select as far as i
+      can tell
diff --git a/open_issues/gdb_attach.mdwn b/open_issues/gdb_attach.mdwn
index 8f1b7b48..b9114b69 100644
--- a/open_issues/gdb_attach.mdwn
+++ b/open_issues/gdb_attach.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -54,3 +54,30 @@ License|/fdl]]."]]"""]]
       process
     <tschwinge> That is supposed to work.
     <tschwinge> If the permission match.
+
+
+# `gdb --pid [PID]`
+
+That is, not explicitly specifying an `executable-file`.
+
+
+## IRC, OFTC, debian-hurd, 2013-04-15
+
+    <paravoid> I don't seem to be able to run gdb
+    <paravoid> that happened on the buildd and happens on exodar too
+    <paravoid> #0  0x010c07cc in ?? ()
+    <paravoid> #1  0x010c1078 in ?? ()
+    <paravoid> #2  0x0109eabf in ?? ()
+    <paravoid> [...]
+    <paravoid> Backtrace stopped: previous frame inner to this frame (corrupt
+      stack?)
+    <paravoid> that's pid 24235 on exodar
+    <paravoid> I did gdb -p 24235 and then bt
+    <paravoid> just the output above
+    <youpi> I don't know about gdb -p
+    <youpi> I usually run gdb
+      /home/paravoid/gdnsd-1.8.1/plugins/meta/libgdmaps/t/.libs/lt-t17_extn_empty.bin
+      24235
+    <paravoid> okay, that indeed works 
+    <youpi> I guess the support for finding out the binary automatically wasn't
+      done yet on hurd
diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn
index 511d3f2b..60ec7357 100644
--- a/open_issues/gnumach_memory_management.mdwn
+++ b/open_issues/gnumach_memory_management.mdwn
@@ -2229,3 +2229,37 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task.
     <braunr> (i mean, when i made my tests, it looked like there were few
       kernel map entries, but i may have missed corner cases that could cause
       more of them to be needed)
+
+
+# IRC, freenode, #hurd, 2013-04-18
+
+    <braunr> oh nice, i've found a big scalability issue with my slab allocator
+    <braunr> it shouldn't affect gnumach much though
+
+
+## IRC, freenode, #hurd, 2013-04-19
+
+    <ArneBab> braunr: is it fixable?
+    <braunr> yes
+    <braunr> well, i'll do it in x15 for a start
+    <braunr> again, i don't think gnumach is much affected
+    <braunr> it's a scalability issue
+    <braunr> when millions of objects are in use
+    <braunr> gnumach rarely has more than a few hundred thousands
+    <braunr> it's also related to heavy multithreading/smp
+    <braunr> and by multithreading, i also mean preemption
+    <braunr> gnumach isn't preemptible and uniprocessor
+    <braunr> if the resulting diff is clean enough, i'll push it to gnumach
+      though :)
+
+
+### IRC, freenode, #hurd, 2013-04-21
+
+    <braunr> ArneBab_: i fixed the scalability problems btw
+
+
+## IRC, freenode, #hurd, 2013-04-20
+
+    <braunr> well, there is also a locking error in the slab allocator,
+      although not a problem for a non preemptible kernel like gnumach
+    <braunr> non preemptible / uniprocessor
diff --git a/open_issues/libpthread_assertion_thread_prevp.mdwn b/open_issues/libpthread_assertion_thread_prevp.mdwn
index 1418bea1..e8160528 100644
--- a/open_issues/libpthread_assertion_thread_prevp.mdwn
+++ b/open_issues/libpthread_assertion_thread_prevp.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -13,7 +13,8 @@ failed"]]
 
 [[!tag open_issue_libpthread]]
 
-IRC, OFTC, #debian-hurd, 2011-10-21:
+
+# IRC, OFTC, #debian-hurd, 2011-10-21
 
     [Python testsuite]
     <pinotree> [169/340/1] test_logging
@@ -22,7 +23,8 @@ IRC, OFTC, #debian-hurd, 2011-10-21:
       __pthread_enqueue: Assertion `thread->prevp == 0' failed.
     <pinotree> sigh
 
-IRC, freenode, #hurd, 2011-10-21:
+
+## IRC, freenode, #hurd, 2011-10-21
 
     <pinotree> am i missing anything, or in libpthread the __pthread_threads
       list does not ever has elements removed from it?
@@ -33,7 +35,8 @@ IRC, freenode, #hurd, 2011-10-21:
     <pinotree> maybe reusing the same next+prevp pointers in the __pthread
       struct for more than one list at the same time isn't a good idea...
 
-2011-10-23:
+
+## IRC, freenode, #hurd, 2011-10-23
 
     <youpi> pinotree: I don't understand the relation between thread->prevp !=
       0 and the __pthread_threads list
@@ -50,3 +53,37 @@ IRC, freenode, #hurd, 2011-10-21:
     <pinotree> yeah
     <pinotree> apparently prevp/next are used for lists of held
       waitcond/mutex/rwlock and free threads
+
+
+# IRC, freenode, #hurd, 2013-03-20
+
+    <braunr> aw
+    <braunr> i hit the ext2fs.static: ./pthread/pt-internal.h:122:
+      __pthread_enqueue: Assertion `thread->prevp == 0' failed.
+    <braunr> assertion
+    <braunr> looks like there is a deadlock on assert
+    <braunr> which might explain why i never saw progress when i tested that in
+      the past
+
+
+## IRC, freenode, #hurd, 2013-04-21
+
+    <braunr> damn, there still bugs in libpthread
+    <braunr> (about prevp not being null when it should i mean)
+    <pinotree> braunr: found another trigger for that?
+    <braunr> no
+    <braunr> it's so random i wonder if it's not a completely unrelated
+      corruption
+    <braunr> pinotree: also, i'm having more of these issues with my custom
+      hurd packages that let threads exit after some time from managing ports
+    <braunr> (i removed the libports_stability patch)
+    <braunr> i once had this : http://www.sceen.net/~rbraun/darnassus_crash.png
+
+[The assertion failure.]
+
+
+## IRC, freenode, #hurd, 2013-04-23
+
+    <braunr> removing the libports_stability patch exposed bugs in libpthread,
+      triggering assertions when queueing/dequeue threads from a queue (but i
+      don't know which one / in which function)
diff --git a/open_issues/libpthread_cancellation_points.mdwn b/open_issues/libpthread_cancellation_points.mdwn
index af0efa9d..48f1acf5 100644
--- a/open_issues/libpthread_cancellation_points.mdwn
+++ b/open_issues/libpthread_cancellation_points.mdwn
@@ -39,3 +39,101 @@ type to asynchronous permits this testcase to terminate. We do have the
 pthread_setcanceltype glibc/libpthread hook in the forward structure, but we are
 not using it: the LIBC_CANCEL_ASYNC macros are void, and we're not using them in
 the mig msg call either.
+
+
+# Provenance
+
+## IRC, OFTC, #debian-hurd, 2013-04-15
+
+    <paravoid> so, let me say a few things about the bug in the first place
+    <paravoid> the package builds and runs a test suite
+    <paravoid> the second test in the test suite blocks forever
+    <paravoid> a blocked pthread_join is what I see
+    <paravoid> I'm unsure why
+    <paravoid> have you seen anything like it before?
+    <youpi> whenever the thread doesn't actually terminate, sure
+    <youpi> what is the thread usually blocked on when you cancel it?
+    <paravoid> this is a hurd-specific issue
+    <paravoid> works on all other arches
+    <youpi> could be just that all other archs have more relaxed behavior
+    <youpi> thus the question of what exactly is supposed to be happening
+    <youpi> apparently it is inside a select?
+    <youpi> it seems select is not cancellable here
+    <pinotree> wasn't the patch you sent?
+    <youpi> no, my patch was about signals
+    <youpi> not cancellation
+    <pinotree> k
+    <youpi> (even if that could be related, of course)
+    <paravoid> how did you see that?
+    <paravoid> what's the equivalent of strace?
+    <youpi> thread 3 is inside _hurd_select
+    <paravoid> thread 1 is blocked on join
+    <paravoid> but the code is
+    <paravoid>     if(gdmaps->reload_thread_spawned) {
+    <paravoid>         pthread_cancel(gdmaps->reload_tid);
+    <paravoid>         pthread_join(gdmaps->reload_tid, NULL);
+    <paravoid>     }
+    <paravoid> so cancel should have killed the thread
+    <youpi> cancelling a thread is a complex matter
+    <youpi> there are cancellation points
+    <youpi> e.g. a thread performing while(1); can't be cancelled
+    <paravoid> thread 3 is just a libev event loop
+    <youpi> yes, "just" calling poll, the most complex system call of unix :)
+    <youpi> paravoid: anyway, don't look for a bug in your program, it's most
+      likely a bug in glibc, thanks for the report
+    <paravoid> I think it all boils down to a problem cancelling a thread in
+      poll()
+    <youpi> yes
+    <youpi> paravoid: ok, actually with the latest libc it does work
+    <paravoid> oh?
+    <youpi> where latest = not uploaded yet :/
+    <paravoid> did you test this on exodar?
+    <youpi> pinotree: that's the libpthread_cancellation.diff I guess
+    <paravoid> because I commented out the join :)
+    <youpi> paravoid:  in the root, yes
+    <youpi> well, I tried my own program
+    <paravoid> oh, okay
+    <youpi> which is indeed hanging inside select (or just read) in the chroot
+    <youpi> but not in the root
+    <pinotree> ah, richard's patch
+    <paravoid> url?
+    <youpi> I've installed the build-dep in the root, if you want to try
+    <paravoid> strange that root is newer than the chroot :)
+    <youpi> paravoid: it's the usual eglibc debian source
+    <paravoid> tried in root, still fails
+    <youpi> could you keep the process running?
+    <paravoid> done
+    <youpi> Mmm, but the thread running gdmaps_reload_thread never set the
+      cancel type to async?
+    <youpi> that said I guess read and select are supposed to be cancellation
+      points
+    <youpi> thus cancel_deferred should be working, but they are not
+    <youpi> it seems it's cancellation points which have just not been
+      implemented
+    <youpi> (they happen to be one of the most obscure things in posix)
+
+
+## IRC, freenode, #hurd, 2013-04-15
+
+    <youpi> but yes, there is still an issue, with PTHREAD_CANCEL_DEFERRED
+    <youpi> how calls like read() or select() are supposed to test
+      cancellation?
+    <pinotree> iirc there are the LIBC_CANCEL_* macros in glibc
+    <pinotree> eg sysdeps/unix/sysv/linux/pread.c
+    <youpi> yes
+    <youpi> but in our libpthredaD?
+    <pinotree> could it be we lack the libpthread → glibc bridge of
+      cancellation stuff?
+    <youpi> we do have pthread_setcancelstate/type forwards
+    <youpi> but it seems the default LIBC_CANCEL_ASYNC is void
+    <pinotree> i mean, so when you cancel a thread, you can get that cancel
+      status in libc proper, just like it seems done with LIBC_CANCEL_* macros
+      and nptl
+    <youpi> as I said, the bridge is there
+    <youpi> we're just not using it in glibc
+    <youpi> I'm writing an open_issues page
+
+
+### IRC, freenode, #hurd, 2013-04-16
+
+    <braunr> youpi: yes, we said some time ago that it was lacking