diff options
Diffstat (limited to 'open_issues')
36 files changed, 2533 insertions, 94 deletions
diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn index 65a82d92..8a2a037f 100644 --- a/open_issues/arm_port.mdwn +++ b/open_issues/arm_port.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -260,3 +260,17 @@ architecture. <matty3269> Well, I have a ARM gnumach kernel compiled. It just doesn't run! :) <braunr> matty3269: good luck :) + + +# IRC, freenode, #hurd, 2013-01-30 + + <slpz> Hi, i've read there's an ongoing effort to port GNU Mach to ARM. How + is it going? + <braunr> not sure where you read that + <braunr> but i'm pretty sure it's not started if it exists + <slpz> braunr: http://www.gnu.org/software/hurd/open_issues/arm_port.html + <braunr> i confirm what i said + <slpz> braunr: OK, thanks. I'm interested on it, and didn't want to + duplicate efforts. + <braunr> little addition: it may have started, but we don't know about it + diff --git a/open_issues/bash.mdwn b/open_issues/bash.mdwn index 47598071..f6b14a08 100644 --- a/open_issues/bash.mdwn +++ b/open_issues/bash.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,7 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_porting]] +[[!tag open_issue_glibc open_issue_porting]] + # *bash* 4.0 vs. typing `C-c` (*SIGINT*) @@ -45,3 +46,14 @@ After having noticed that this error doesn't occur if starting *bash* with bash: start_pipeline: pgrp pipe: (ipc/mig) wrong reply message ID So, there's something different with stdout in / after the SIGINT handler. + + +# IRC, freenode, #hurd, 2013-01-13 + +Perhaps completely unrelated to the issue above, perhaps not. + + <tschwinge> bash: xmalloc: ../../../bash/lib/sh/strtrans.c:60: cannot + allocate 261 bytes (323584 bytes allocated) + <tschwinge> 1.5 GiB RAM were free. + <tschwinge> This happened when I did a rever history search (C-r [...]), + and then pressed C-c. diff --git a/open_issues/clock_gettime.mdwn b/open_issues/clock_gettime.mdwn index 5345ed6b..83ad81e8 100644 --- a/open_issues/clock_gettime.mdwn +++ b/open_issues/clock_gettime.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -23,7 +23,8 @@ applications assume that it is. What about adding a nanosecond-precision clock, too? --[[tschwinge]] -IRC, freenode, #hurd, 2011-08-26: + +# IRC, freenode, #hurd, 2011-08-26 < pinotree> youpi: thing is: apparently i found a simple way to have a monotonic clock as mmap-able device inside gnumach @@ -40,7 +41,8 @@ IRC, freenode, #hurd, 2011-08-26: < braunr> sure < youpi> and that's the way I was considering implementing it -IRC, freenode, #hurd, 2011-09-06: + +# IRC, freenode, #hurd, 2011-09-06 <pinotree> yeah, i had a draft of improved idea for also handling nanoseconds @@ -69,3 +71,116 @@ IRC, freenode, #hurd, 2011-09-06: <youpi> yes <tschwinge> And it will forever be a witness of the evolving of this map_time interface. :-) + + +# IRC, freenode, #hurd, 2013-02-11 + +In context of [[select]]. + + <pinotree> braunr: would you send for review (and inclusion) your + time_data_t addition? + <pinotree> this way we could add nanosecs-based utime rpc (and then their + implementation in libc) + <braunr> pinotree: it's part of the hurd branch + <braunr> do you want it sent separately ? + <pinotree> yeah + <braunr> ok + <braunr> let me get it right first :) + <pinotree> sure :) + + +## IRC, freenode, #hurd, 2013-02-12 + + <braunr> pinotree: + http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout_pthread_v2&id=6ec50e62d9792c803d00cbff1cab2c0b3675690a + <pinotree> uh nice + <pinotree> will need two small inline functions to convert time_data_t <-> + timespec, but that's it + <braunr> hm right + <braunr> i could have thought about it + <braunr> but i'll leave it for another patch :p + <pinotree> oh sure, no hurry + + +## IRC, freenode, #hurd, 2013-02-19 + + <youpi> braunr: about time_data_t, I get it's needed that it be an array + <youpi> so it can be passed by reference, not by value? + <braunr> by address, yes + <braunr> that's the difference between array and struct + + +## IRC, freenode, #hurd, 2013-02-25 + + <youpi> braunr: why did you want to see time_data passed as pointer, not as + struct? + <braunr> to microoptimize + <braunr> the struct is 2 64-bit integers + <youpi> well, we already pass structs along in a few cases, + e.g. io_statbuf_t, rusage_t, etc. + <youpi> be it written t[0].sec or t->sec, it seems odd + <youpi> copying 2 64bit integers is not much compared to the potential for + bugs here + <braunr> bugs ? + <youpi> yes, as in trying to access t[1], passing a wrong pointer, etc. + <youpi> or the reader frowning on "why is this case different than the + others?" + <braunr> well, i'm already usually frowning when i see what mig does .. + <youpi> right + <youpi> on the plus side, it's only the client side, i.e. mostly glibc, + which sees the t[0] + <braunr> and the practice established by my patch is to convert to struct + timespec as soon as possible + <braunr> the direct use of this type is therefore limited + <youpi> could we define time_data_t as a struct time_data * instead of + struct time_data[1] ? + <youpi> (in the.h) + <youpi> that would make more sense to define a struct time_data, and pass a + pointer to it + <braunr> i'm not sure + <braunr> the mach server writing guide was very clear about array implying + a C array too + <braunr> and i remember having compilation problems before doing that + <braunr> but i don't remember their nature exactly + <youpi> I'm not sure to understand what you said about converting to struct + timespec + <youpi> what makes it not possible now? + <youpi> and what is the relation with being an array or a pointer? + <braunr> concerning struct timespec, what i mean is that the functions + called by the mig stub code directly convert time_data_t to a struct + timespec (which is the real type used throughout the hurd code) + <braunr> about the rest, i'm not sure, i'd have to try again + <braunr> mig just assumes it's an array + <youpi> and why not just using struct timespec? + <youpi> (for the mig type too) + <braunr> my brain can't correctly compute variable sized types in mig + definition files + <braunr> i wanted something that would remain correct for the 64-bit port + <youpi> ah, you mean because tv_nsec is a long, which will not be the same + type? + <braunr> and tv_sec being a time_t (thus a long too) + <youpi> but we have the same issue e.g. for the rusage structure, don't we? + <braunr> yes + <youpi> so we'll have to fix things for that too anyway + <braunr> sure + <youpi> making a special case will not necessarily help + <braunr> but it doesn't mean new interfaces have to be buggy too + <youpi> well, using the proper type in the server itself is nicer + <youpi> instead of having to convert + <braunr> yes + <braunr> i'm not exactly sure where to declare struct timespec then + <braunr> should it be declared in hurd_types.h, and simply reused by the + libc headers ? + <youpi> ? AIUI, it's the converse, hurd_types.h uses the struct timespec + from libc headers, and defines timespec_t + <braunr> ok + <youpi> timespec_t being the internal type whose definition gets done right + for mig to do the right thing + <braunr> yes + <braunr> i see + <braunr> so, you'd like a struct of integer_t instead of an array of + signed64 + <youpi> for our current 32bit userland yes + <braunr> do you want to make the changes yourself or should i add a new + branch ? + <youpi> and we'll make that a 64bit struct when we have a64bit userland diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index 5f6fcf6a..b25e53d7 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -33,7 +33,7 @@ The plan is to use [[libstore_parted]] for accessing partitions. ## IRC, freenode, #hurd, 2012-02-08 -At the microkernel davroom at [[community/meetings/FOSDEM_2012]]: +After the microkernel devroom at [[community/meetings/FOSDEM_2012]]: <antrik> there was quite some talk about DDE. I learnt that there are newer versions in Genode and in Minix (as opposed to the DROPS one we are @@ -109,6 +109,40 @@ At the microkernel davroom at [[community/meetings/FOSDEM_2012]]: you want. It uses Linux 2.6.26 usb subsystem. +## IRC, freenode, #hurd, 2013-02-15 + +After the microkernel devroom at [[community/meetings/FOSDEM_2013]]. + + <pinotree> youpi: speaking of dde, was there any will among other + microkernel os developers to eventually develop one single dde (with + every team handling the custom glue of the own kernel)? + <youpi> well, there is still upstream dde actually + <youpi> in dresden + <youpi> nothing was really decided or anything (it was a round table, not a + workgroup) + <youpi> but conversation converged into sharing the DDE maintenance, yes + <youpi> and dresden would be the logical central place + <youpi> pb is that they don't have the habit of being very open + <youpi> http://svn.tudos.org/repos/oc/tudos/trunk/l4/pkg/dde has a recent + enough version + <youpi> which macsim confirmed having all the latest commits from the + internal repository + <pinotree> i see + <youpi> so it seems a viable solution on the medium term + <youpi> the long term might need a real visible open source project + <youpi> but we should probably still keep basing on dresden work + <youpi> (better take work being done anywhere) + <pinotree> well, if the upstream is not really open, microkernel teams + could just fork it and all work on it + <youpi> that's what I mean + <pinotree> should still be a win than everybody maintaining their own dde + <youpi> sure + <pinotree> ah yes, i was writing and i'm slow at it :) + <youpi> but at least we can try to work with dresden + <youpi> see how open they could become by just asking :) + <pinotree> right + + # IRC, OFTC, #debian-hurd, 2012-02-15 <pinotree> i have no idea how the dde system works @@ -484,26 +518,17 @@ At the microkernel davroom at [[community/meetings/FOSDEM_2012]]: <braunr> *sigh* - -# IRC, freenode, #hurd, 2012-08-18 +## IRC, freenode, #hurd, 2012-08-18 <braunr> hum, leaks and potential deadlocks in libddekit/thread.c :/ -# IRC, freenode, #hurd, 2012-08-18 +## IRC, freenode, #hurd, 2012-08-18 <braunr> nice, dde relies on a race to start .. -# IRC, freenode, #hurd, 2012-08-18 - - <braunr> hm looks like if netdde crashes, the kernel doesn't handle it - cleanly, and we can't attach another netdde instance - -[[!message-id "877gu8klq3.fsf@kepler.schwinge.homeip.net"]] - - -# IRC, freenode, #hurd, 2012-08-21 +## IRC, freenode, #hurd, 2012-08-21 In context of [[libpthread]]. @@ -513,6 +538,40 @@ In context of [[libpthread]]. <braunr> either in netdde or pfinet +## IRC, freenode, #hurd, 2013-02-28 + + <braunr> (which needs the same kinds of fixes that libpthread got) + <braunr> actually i'm not sure why he didn't simply reuse the pthread + functions :/ + <youpi> which kind of fixes? + <youpi> cancellation? + <braunr> timeouts + <braunr> cancellation too but that's less an issue + <youpi> I'm not sure it really needs timeout work + <youpi> on what RPC? + <youpi> pfinet is just using the mach interface + <braunr> i don't know but it clearly copies some of the previous pthread + code from pthread_cond_timedwait + <braunr> see libddekit/thread.c:_sem_timedwait_internal + <youpi> I recognize the comment indeed :) + <youpi> I guess he thought he might need some particular semantic that + libpthread may not provide + <braunr> also, now that i think about it, he couldn't have used libpthread, + could he ? + <braunr> and there was no condition_timedwait in cthreads + <braunr> there is a deadlock in netdde + <braunr> it occurs sometimes, at high network speeds + <braunr> (well high, 4 MiB/s or more) + + +# IRC, freenode, #hurd, 2012-08-18 + + <braunr> hm looks like if netdde crashes, the kernel doesn't handle it + cleanly, and we can't attach another netdde instance + +[[!message-id "877gu8klq3.fsf@kepler.schwinge.homeip.net"]] + + # DDE for Filesystems ## IRC, freenode, #hurd, 2012-10-07 diff --git a/open_issues/gcc/pie.mdwn b/open_issues/gcc/pie.mdwn index a4598d1e..da951001 100644 --- a/open_issues/gcc/pie.mdwn +++ b/open_issues/gcc/pie.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -13,7 +13,7 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gcc]] -# IRC, freenode, #debian-hurd, 2012-11-08 +# IRC, freenode, #hurd, 2012-11-08 <pinotree> tschwinge: i'm not totally sure, but it seems the pie options for gcc/ld are causing issues @@ -38,3 +38,9 @@ License|/fdl]]."]]"""]] <youpi> uh <pinotree> this causes the w3m build failure and (indirectly, due to elinks built with -pie) aptitude + + +## IRC, freenode, #hurd, 2013-01-19 + + <gnu_srs> pinotree: I can confirm that -fPIE -pie fails and only -fPIE + works for mktable in w3m. Still have to check with elinks. What's up doc? diff --git a/open_issues/git-core-2.mdwn b/open_issues/git-core-2.mdwn index 2d8ad96b..cbf47bd2 100644 --- a/open_issues/git-core-2.mdwn +++ b/open_issues/git-core-2.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,9 +16,7 @@ License|/fdl]]."]]"""]] [[!toc]] -# Log - -December, 2008. +# 2008-12 On the otherwise-idle flubber: @@ -57,11 +55,15 @@ Fixing this situation is easy enough: # On branch master nothing to commit (working directory clean) -Still seen on 2010-03-16. ---- +## 2010-03-16 + +Still seen. + -A very similar issue, seen on 2010-11-17. The working tree had a lot of +# 2010-11-17 + +A very similar issue. The working tree had a lot of differences to HEAD. tschwinge@grubber:~/tmp/gcc/hurd $ git reset --hard HEAD @@ -92,9 +94,10 @@ differences to HEAD. Checking out files: 100% (1149/1149), done. HEAD is now at fe3e43c Merge commit 'refs/top-bases/hurd/master' into hurd/master ---- -2010-12-22, grubber: +## 2010-12-22 + +grubber: $ git remote update Fetching savannah @@ -106,9 +109,10 @@ differences to HEAD. fatal: index-pack failed error: Could not fetch savannah ---- -2011-06-10, coulomb.SCHWINGE, checking out [[binutils]]' master branch, +## 2011-06-10 + +coulomb.SCHWINGE, checking out [[binutils]]' master branch, starting from an empty working directory (after an external `git push`): $ git checkout -f @@ -144,7 +148,7 @@ starting from an empty working directory (after an external `git push`): # Analysis -2011-06-13 +## 2011-06-13 Running `git checkout -f` under GDB: @@ -188,3 +192,25 @@ there are cases where `unlink` apparently returns EINTR, which is not kosher either. Etc. Do we have problems with `SA_RESTART` vs. the atomicity of our syscall-alikes? + + +## IRC, freenode, #hurd, 2013-01-30 + + <braunr> hm, let's try to clone a huge repository + <braunr> hm, cloning a whole linux repo, and still no problem :) + <pinotree> weren't most/all the issues at unpack time? + <braunr> i don't remember + <braunr> we'll see when it gets there + <braunr> the longest part is "resolving deltas", for which ext2fs is + clearly the big bottleneck (no I/O, page-cache only, but still) + <braunr> pinotree: well, slow, but no error + + +### IRC, freenode, #hurd, 2013-01-31 + + <braunr> fyi, i've tried several checkouts of big repositories, and never + got a single error + <braunr> youpi: looks like the recent fixes also solved some git issues we + had + <braunr> i could clone big repositories without any problem + <youpi> cool :) diff --git a/open_issues/git_duplicated_content.mdwn b/open_issues/git_duplicated_content.mdwn index cbc171a7..f0ffad77 100644 --- a/open_issues/git_duplicated_content.mdwn +++ b/open_issues/git_duplicated_content.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -116,16 +116,13 @@ Some more copying: No further difference. ---- - $ git-new-workdir git master master $ diff -x .git -ur tar_master/ master/ > master.diff $ rm -rf ar_master* && (cd git/ && git archive master) | (mkdir ar_master && cd ar_master/ && tar -x) && diff -x .git -ru tar_master/ ar_master/ > ar_master.diff; ls -l ar_master.diff $ (cd git/ && git archive master) | md5sum ---- -2011-06-13 +# 2011-06-13 -> [[git-core-2]] diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index 4111700b..425ce827 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012 Free Software +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -385,6 +385,51 @@ Last reviewed up to the [[Git mirror's d3bd58cf0a027016544949ffd27300ac5fb01bb8 <Tekk_> pinotree: undefined <pinotree> expected, given the output above + * `getsockopt`, `setsockopt` + + IRC, freenode, #hurd, 2013-02-14 + + <gnu_srs> Hi, {get,set}sockopt is not supported on Hurd. This shows + e.g. in the gnulib's test-{poll,select} code. + <gnu_srs> Reading + http://hea-www.harvard.edu/~fine/Tech/addrinuse.html there might + be reasons _not_ to implement them, comments? + <pinotree> uh? they are supported on hurd + <gnu_srs> not SO_REUSEPORT for setsockopt() + <pinotree> that isn't the same as claiming "get/setsockopt is not + supported on hurd" + <pinotree> most probably that option is not implemented by the + socket family you are using + <gnu_srs> OK, some options like SO_REUSEPORT then, more info in + the link. + <pinotree> note also SO_REUSEPORT is not posix + <pinotree> and i don't see SO_REUSEPORT mentioned in the page you + linked + <gnu_srs> No, but SO_REUSEADDR + + IRC, freenode, #hurd, 2013-02-23 + + <gnu_srs> as an example, the poll test code from gnulib fails due + to that problem (and I've told you before) + <pinotree> gnu_srs: what's the actual failure? + <pinotree> can you provide a minimal test case showing the issue? + <gnu_srs> pinotree: A smaller test program: + http://paste.debian.net/237495/ + <pinotree> gnu_srs: setting SO_REUSEADDR before binding the socket + works... + <pinotree> and it seems it was a bug in the gnulib tests, see + http://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commit;h=6ed6dffbe79bcf95e2ed5593eee94ab32fcde3f4 + <gnu_srs> pinotree: You are right, still the code I pasted pass on + Linux, not on Hurd. + <pinotree> so? + <pinotree> the code is wrong + <pinotree> you cannot change what bind does after you have called + it + * pinotree → out + <gnu_srs> so linux is buggy? + <braunr> no, linux is more permissive + <braunr> (at least, on this matter) + For specific packages: * [[octave]] @@ -669,6 +714,198 @@ Last reviewed up to the [[Git mirror's d3bd58cf0a027016544949ffd27300ac5fb01bb8 [[!message-id "201211172058.21035.toscano.pino@tiscali.it"]]. + In context of [[libpthread]]. + + IRC, freenode, #hurd, 2013-01-21 + + <braunr> ah, found something interesting + <braunr> tschwinge: there seems to be a race on our file descriptors + <braunr> the content written by one thread seems to be retained + somewhere and another thread writing data to the file descriptor will + resend what the first already did + <braunr> it could be a FILE race instead of fd one though + <braunr> yes, it's not at the fd level, it's above + <braunr> so good news, seems like the low level message/signalling code + isn't faulty here + <braunr> all right, simple explanation: our IO_lockfile functions are + no-ops + <pinotree> braunr: i found that out days ago, and samuel said they were + okay + <braunr> well, they're not no-ops in libpthreads + <braunr> so i suppose they replace the default libc stubs, yes + <pinotree> so the issue happens in cthreads-using apps? + <braunr> no + <braunr> we don't have cthreads apps any more + <braunr> and aiui, libpthreads provides cthreads compatibility calls to + libc, so everything is actually using pthreads + <braunr> more buffer management debugging needed :/ + <pinotree> hm, so how can it be that there's a multithread app with no + libpthread-provided file locking? + <braunr> ? + <braunr> file locking looks fine + <braunr> hm, the recursive locking might be wrong though + <braunr> ./sysdeps/mach/hurd/bits/libc-lock.h:#define + __libc_lock_owner_self() ((void *) __hurd_threadvar_location (0)) + <braunr> nop, looks fine too + <braunr> indeed, without stream buffering, the problem seems to go away + <braunr> pinotree: it really looks like the stub IO_flockfile is used + <braunr> i'll try to make sure it's the root of the problem + <pinotree> braunr: you earlier said that there's some race with + different threads, no? + <braunr> yes + <braunr> either a race or an error in the iostream management code + <braunr> but i highly doubt the latter + <pinotree> if the stub locks are used, then libpthread is not + loaded... so which different threads are running? + <braunr> that's the thing + <braunr> the libpthread versions should be used + <pinotree> so the application is linked to pthread? + <braunr> yes + <pinotree> i see, that was the detail i was missing earlier + <braunr> the common code looks fine, but i can see wrong values even + there + <braunr> e.g. when vfprintf calls write, the buffer is already wrong + <braunr> i've made similar tests on linux sid, and it behaves as it + should + <pinotree> hm + <braunr> i even used load to "slow down" my test program so that + preemption is much more likely to happen + <pinotree> note we have slightly different behaviour in glibc's libio, + ie different memory allocation ways (mmap on linux, malloc for us) + <braunr> the problem gets systematic on the hurd while it never occurs + on linux + <braunr> that shouldn't matter either + <pinotree> ok + <braunr> but i'll make sure it doesn't anyway + <braunr> this mach_print system call is proving very handy :) + <braunr> and also, with load, unbuffered output is always correct too + <pinotree> braunr: you could try the following hack + http://paste.debian.net/227106/ + <braunr> what does it do ? + <pinotree> (yes, ugly as f**k) + <braunr> does it force libio to use mmap ? + <braunr> or rather, enable ? + <pinotree> provides a EXEC_PAGESIZE define in libio, so it makes it use + mmap (like on linux) instead of malloc + + `t/pagesize`. + + <braunr> yes, the stub is used instead of the libpthreads code + <braunr> tschwinge: ^ + <braunr> i'll override those to check that it fixes the problem + <braunr> hm, not that easy actually + <pinotree> copy their files from libpthreads to sysdeps/mach/hurd + <pinotree> hm right, in libpthread they are not that split as in glibc + <braunr> let's check symbol declaration to understand why the stubs + aren't overriden by ld + <braunr> _IO_vfprintf correctly calls @plt versions + <braunr> i don't know enough about dynamic linking to see what causes + the problem :/ + <braunr> youpi: it seems our stdio functions use the stub IO_flockfile + functions + <youpi> really? I thought we were going through cthreads-compat.c + <braunr> yes really + <braunr> i don't know why, but that's the origin of the "duplicated" + messages issue + <braunr> messages aren't duplicated, there is a race that makes on + thread reuse the content of the stream buffer + <braunr> one* + <youpi> k, quite bad + <braunr> at least we know where the problem comes from now + <braunr> youpi: what would be the most likely reason why weak symbols + in libc wouldn't be overriden by global ones from libpthread ? + <youpi> being loaded after libc + <braunr> i tried preloading it + <braunr> i'll compare with what is done on wheezy + <youpi> you have the local-dl-dynamic-weak.diff patch, right? + <braunr> (on squeeze, the _IO_flockfile function in libc seems to do + real work unlike our noop stub) + <braunr> it's the debian package, i have all patches provided there + <braunr> indeed, on linux, libc provides valid IO_flock functions + <braunr> ./sysdeps/pthread/flockfile.c:strong_alias (__flockfile, + _IO_flockfile) + <braunr> that's how ntpl exports it + <braunr> nptl* + <pinotree> imho we should restructure libpthread to be more close to + nptl + <braunr> i wish i knew what it involves + <pinotree> file structing for sources and tests, for example + <braunr> well yes obviously :) + <braunr> i've just found a patch that does exactly that for linuxthreads + <pinotree> that = fix the file locking? + <braunr> in addition to linuxthreads/lockfile.c (which we also + equivalently provide), there is + linuxthreads/sysdeps/pthread/flockfile.c + <braunr> no, restructiring + <braunr> restructuring* + <braunr> i still have only a very limited idea of how the glibc sources + are organized + <pinotree> the latter is used as source file when compiling flockfile.c + in stdio-common + <braunr> shouldn't we provide one too ? + <pinotree> that would mean it would be compiled as part of libc proper, + not libpthread + <braunr> yes + <braunr> that's what both linuxthreads and nptl seem to do + <braunr> and the code is strictly the same, i.e. a call to the internal + _IO_lock_xxx functions + <youpi> I guess that's for the hot-dlopen case + <youpi> you need to have locks properly taken at dlopen time + <braunr> youpi: do you mean adding an flockfile.c file to our sysdeps + will only solve the problem by side effect ? + <braunr> and that the real problem is that the libpthread versions + aren't used ? + <youpi> yes + <braunr> ok + <braunr> youpi: could it simply be a versioning issue ? + <youpi> could be + <braunr> it seems so + <braunr> i've rebuilt with the flockfile functions versioned to 2.2.6 + (same as in libc) and the cthreads_compat functions are now used + <braunr> and the problem doesn't occur any more with my test code + <braunr> :) + <youpi> could you post a patch? + <braunr> i need a few info before + <youpi> it'd be good to check which such functions are hooked + <braunr> i suppose the version for functions declared in libpthreads + shouldn't change, right ? + <youpi> yes + <braunr> ok + <youpi> they didn't have a vresion before + <braunr> shall i commit directly ? + <youpi> so it should be fine + <braunr> well, they did + <braunr> 2.12 + <youpi> yes, but please tell me when it's done + <braunr> sure + <youpi> so I can commit that to debian's eglibc + <youpi> I mean, before we integrated libpthread build into glibc + <youpi> so they never had any version before 2.12 + <braunr> ok + <youpi> basically we need to check the symbols which are both in + libpthread and referenced in libc + <youpi> to make sure they have the same version in the reference + <braunr> ok + <youpi> only weak references need to be checked, others would have + produced a runtime error + <braunr> youpi: done + <braunr> arg, the version i mention in the comment is wrong + <braunr> i suppose people understand nonetheless + <youpi> probably, yes + <braunr> ah, i can now appreciate the headache this bug hunting gave me + these last days :) + + IRC, freenode, #hurd, 2013-01-22 + + <youpi> braunr: commited to debian glibc + <youpi> btw, it's normal that the program doesn't terminate, right? + <youpi> (i.e. it's the original bug you were chasing) + <braunr> youpi: about your earlier question (yesterday) about my test + code, it's expected to block, which is the problem i was initially + working on + <youpi> ok, so all god + <youpi> +o + * `t/pagesize` IRC, freenode, #hurd, 2012-11-16 @@ -677,6 +914,37 @@ Last reviewed up to the [[Git mirror's d3bd58cf0a027016544949ffd27300ac5fb01bb8 the fact that EXEC_PAGESIZE is not defined on hurd, libio/libioP.h switches the allocation modes from mmap to malloc + IRC, freenode, #hurd, 2013-01-21 + + <braunr> why is it a hack ? + <pinotree> because most probably glibc shouldn't rely on EXEC_PAGESIZE + like that + <braunr> ah + <pinotree> there's a mail from roland, replying to thomas about this + issue, that this use of EXEC_PAGESIZE to enable mmap or not is just + wrong + <braunr> ok + <pinotree> (the above is + http://thread.gmane.org/87mxd9hl2n.fsf@kepler.schwinge.homeip.net ) + <braunr> thanks + <pinotree> (just added the reference to that in the wiki) + <braunr> pinotree: btw, what's wrong with using malloc instead of mmap + in libio ? + <pinotree> braunr: i'm still not totally sure, most probably it should + be slightly slower currently + <braunr> locking contention ? + <braunr> pinotree: + http://www.sourceware.org/ml/libc-alpha/2006-11/msg00061.html + <braunr> pinotree: it looks to me there is now no valid reason not to + use malloc + <braunr> the best argument for mmap is that libio requires zeroed + memory, but as the OP says, zeroing a page is usually more expensive + than a small calloc (even on kernel that keep a list of zeroed pages + for quick allocations, frequent mmaps() often make this list empty) + <pinotree> braunr: mmap allocations in libio are rounded to the page + size + <braunr> well they have to + * `LD_DEBUG` IRC, freenode, #hurd, 2012-11-22 diff --git a/open_issues/gnumach_page_cache_policy.mdwn b/open_issues/gnumach_page_cache_policy.mdwn index 22b05953..5e93887e 100644 --- a/open_issues/gnumach_page_cache_policy.mdwn +++ b/open_issues/gnumach_page_cache_policy.mdwn @@ -771,12 +771,12 @@ License|/fdl]]."]]"""]] <tschwinge> And set precedence. -## IRC, freenode, #hurd, 2012-07-26 +# IRC, freenode, #hurd, 2012-07-26 <braunr> hm i killed darnassus, probably the page cache patch again -## IRC, freenode, #hurd, 2012-09-19 +# IRC, freenode, #hurd, 2012-09-19 <youpi> I was wondering about the page cache information structure <youpi> I guess the idea is that if we need to add a field, we'll just @@ -786,3 +786,28 @@ License|/fdl]]."]]"""]] <braunr> youpi: have a look at the rbraun/page_cache gnumach branch <youpi> that's what I was referring to <braunr> ok + + +# IRC, freenode, #hurd, 2013-01-15 + + <braunr> hm, no wonder the page cache patch reduced performance so much + <braunr> the page cache when building even moderately large packages is + about a few dozens MiB (around 50) + <braunr> the patch enlarged it to several hundreds :/ + <ArneBab> braunr: so the big page cache essentially killed memory locality? + <braunr> ArneBab: no, it made ext2fs crazy (disk translators - used as + pagers - scan their cached pages every 5 seconds to flush the dirty ones) + <braunr> you can imagine what happens if scanning and flushing a lot of + pages takes more than 5 seconds + <ArneBab> ouch… that’s heavy, yes + <ArneBab> I already see it pile up in my mindb + <braunr> and it's completely linear, using a lock to protect the whole list + <braunr> darnassus is currently showing such a behaviour, because tschwinge + is linking huge files (one object with lots of pages) + <braunr> 446 MB of swap used, between 200 and 1850 MiB of RAM used, and i + can still use vim and build stuff without being too disturbed + <braunr> the system does feel laggy, but there has been great stability + improvements + <braunr> have* + <braunr> and even if laggy, it doesn't feel much more than the usual lag of + a network (ssh) based session diff --git a/open_issues/gnumach_panic_thread_dispatch.mdwn b/open_issues/gnumach_panic_thread_dispatch.mdwn new file mode 100644 index 00000000..db094f2f --- /dev/null +++ b/open_issues/gnumach_panic_thread_dispatch.mdwn @@ -0,0 +1,20 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] + + +# IRC, freenode, #hurd, 2013-02-10 + + <youpi> panic: thread_dispatch: thread a958c950 has unexpected state 114 + <youpi> hum ): + <braunr> ouch + <youpi> during a perl build + <braunr> TH_SWAPPED | TH_HALTED | TH_RUN diff --git a/open_issues/hurd_101.mdwn b/open_issues/hurd_101.mdwn index 6146885d..574a03ec 100644 --- a/open_issues/hurd_101.mdwn +++ b/open_issues/hurd_101.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,7 +12,8 @@ License|/fdl]]."]]"""]] Not the first time that something like this is proposed... -IRC, freenode, #hurd, 2011-07-25 + +# IRC, freenode, #hurd, 2011-07-25 [failed GNU/Hurd project] < antrik> gnu_srs1: I wouldn't say he was on track. just one of the many @@ -39,3 +40,23 @@ IRC, freenode, #hurd, 2011-07-25 < Tekk_`> under the right conditions < cluck> antrik: jokes aside some sort of triage system/training ground for newcomers could be helpful + + +# IRC, freenode, #hurd, 2013-01-20 + + <zacts> so once I have written my first translators, and really understand + that, what kinds of projects would you recommend to an operating + systems/hurd newbie. + <zacts> I am reading the minix book now as I have it, but I'm waiting on + getting the modern operating systems book by the same author. + <zacts> I was initially going to start working on minix, but their focus + seems to be on embedded, and I want to work on a system that is more + general purpose, and I like the philosophy of freedom surrounding the + hurd. + <zacts> I like how the hurd design allows more freedom for users of the + operating system, but I would also like to incorporate ideas from minix + on the hurd. mainly, rebootless updates of servers/translators. + <neal> then you should study how translators work + <neal> how ipc works + <neal> and understand exactly what state is stored where + <zacts> ok diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn index 80fe36f8..670c82cb 100644 --- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn +++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -121,3 +121,15 @@ License|/fdl]]."]]"""]] libmachuser and libhurduser? should they be linked to explicitly, or assume libc brings them? <braunr> pinotree: libc should bring them + + +# IRC, freenode, #hurd, 2013-02-25 + + <braunr> we should also discuss the mach_debug interface some day + <braunr> it's not exported by libc, but the kernel provides it + <braunr> slabinfo depends on it, and i'd like to include it in the hurd + <braunr> but i don't know what kind of security problems giving access to + mach_debug RPCs would create + <braunr> (imo, the mach_debug interface should be adjusted to be used with + privileged ports only) + <braunr> (well, maybe not all mach_debug RPCs) diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index 05aab85f..f0c0db58 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -1170,6 +1170,12 @@ There is a [[!FF_project 275]][[!tag bounty]] on this task. <braunr> haven't tested +### IRC, freenode, #hurd, 2013-01-26 + + <braunr> ah great, one of the recent fixes (probably select-eintr or + setitimer) fixed exim4 :) + + ## IRC, freenode, #hurd, 2012-09-23 <braunr> tschwinge: i committed the last hurd pthread change, @@ -1270,6 +1276,17 @@ There is a [[!FF_project 275]][[!tag bounty]] on this task. <youpi> that's it, yes +### IRC, freenode, #hurd, 2013-03-01 + + <youpi> braunr: btw, "unable to adjust libports thread priority: (ipc/send) + invalid destination port" is actually not a sign of fatality + <youpi> bach recovered from it + <braunr> youpi: well, it never was a sign of fatality + <braunr> but it means that, for some reason, a process looses a right for a + very obscure reason :/ + <braunr> weird sentence, agreed :p + + ## IRC, freenode, #hurd, 2012-12-05 <braunr> tschwinge: i'm currently working on a few easy bugs and i have @@ -1459,3 +1476,332 @@ Same issue as [[term_blocking]] perhaps? <braunr> we have a similar problem with the hurd-specific cancellation code, it's in my todo list with io_select <youpi> ah, no, the condvar is not global + + +## IRC, freenode, #hurd, 2013-01-14 + + <braunr> *sigh* thread cancellable is totally broken :( + <braunr> cancellation* + <braunr> it looks like playing with thread cancellability can make some + functions completely restart + <braunr> (e.g. one call to printf to write twice its output) + +[[git_duplicated_content]], [[git-core-2]]. + + * braunr is cooking a patch to fix pthread cancellation in + pthread_cond_{,timed}wait, smells good + <braunr> youpi: ever heard of something that would make libc functions + "restart" ? + <youpi> you mean as a feature, or as a bug ? + <braunr> when changing the pthread cancellation state of a thread, i + sometimes see printf print its output twice + <youpi> or perhaps after a signal dispatch? + <braunr> i'll post my test code + <youpi> that could be a duplicate write + <youpi> due to restarting after signal + <braunr> http://www.sceen.net/~rbraun/pthreads_test_cancel.c + #include <stdio.h> + #include <stdarg.h> + #include <stdlib.h> + #include <pthread.h> + #include <unistd.h> + + static pthread_cond_t cond = PTHREAD_COND_INITIALIZER; + static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; + static int predicate; + static int ready; + static int cancelled; + + static void + uncancellable_printf(const char *format, ...) + { + int oldstate; + va_list ap; + + va_start(ap, format); + pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &oldstate); + vprintf(format, ap); + pthread_setcancelstate(oldstate, &oldstate); + va_end(ap); + } + + static void * + run(void *arg) + { + uncancellable_printf("thread: setting ready\n"); + ready = 1; + uncancellable_printf("thread: spin until cancellation is sent\n"); + + while (!cancelled) + sched_yield(); + + uncancellable_printf("thread: locking mutex\n"); + pthread_mutex_lock(&mutex); + uncancellable_printf("thread: waiting for predicate\n"); + + while (!predicate) + pthread_cond_wait(&cond, &mutex); + + uncancellable_printf("thread: unlocking mutex\n"); + pthread_mutex_unlock(&mutex); + uncancellable_printf("thread: exit\n"); + return NULL; + } + + int + main(int argc, char *argv[]) + { + pthread_t thread; + + uncancellable_printf("main: create thread\n"); + pthread_create(&thread, NULL, run, NULL); + uncancellable_printf("main: spin until thread is ready\n"); + + while (!ready) + sched_yield(); + + uncancellable_printf("main: sending cancellation\n"); + pthread_cancel(thread); + uncancellable_printf("main: setting cancelled\n"); + cancelled = 1; + uncancellable_printf("main: joining thread\n"); + pthread_join(thread, NULL); + uncancellable_printf("main: exit\n"); + return EXIT_SUCCESS; + } + <braunr> youpi: i'd see two calls to write, the second because of a signal, + as normal, as long as the second call resumes, but not restarts after + finishing :/ + <braunr> or restarts because nothing was done (or everything was entirely + rolled back) + <youpi> well, with an RPC you may not be sure whether it's finished or not + <braunr> ah + <youpi> we don't really have rollback + <braunr> i don't really see the difference with a syscall there + <youpi> the kernel controls the interruption in the case of the syscall + <braunr> except that write is normally atomic if i'm right + <youpi> it can't happen on the way back to userland + <braunr> but that could be exactly the same with RPCs + <youpi> while perhaps it can happen on the mach_msg back to userland + <braunr> back to userland ok, back to the application, no + <braunr> anyway, that's a side issue + <braunr> i'm fixing a few bugs in libpthread + <braunr> and noticed that + <braunr> (i should soon have patches to fix - at least partially - thread + cancellation and timed blocking) + <braunr> i was just wondering how cancellation how handled in glibc wrt + libpthread + <youpi> I don't know + <braunr> (because the non standard hurd cancellation has nothing to do with + pthread cancellation)à + <braunr> ok + <braunr> s/how h/is h/ + + +### IRC, freenode, #hurd, 2013-01-15 + + <tschwinge> braunr: Re »one call to printf to write twice its output«: + sounds familiar: + http://www.gnu.org/software/hurd/open_issues/git_duplicated_content.html + and http://www.gnu.org/software/hurd/open_issues/git-core-2.html + <braunr> tschwinge: what i find strange with the duplicated operations i've + seen is that i merely use pthreads and printf, nothing else + <braunr> no setitimer, no alarm, no select + <braunr> so i wonder how cancellation/syscall restart is actually handled + in our glibc + <braunr> but i agree with you on the analysis + + +### IRC, freenode, #hurd, 2013-01-16 + + <braunr> neal: do you (by any chance) remember if there could possibly be + spurious wakeups in your libpthread implementation ? + <neal> braunr: There probably are. + <neal> but I don't recall + + <braunr> i think the duplicated content issue is due to the libmach/glibc + mach_msg wrapper + <braunr> which restarts a message send if interrupted + <tschwinge> Hrm, depending on which point it has been interrupted you mean? + <braunr> yes + <braunr> not sure yet and i could be wrong + <braunr> but i suspect that if interrupted after send and during receive, + the restart might be wrongfully done + <braunr> i'm currently reworking the timed* pthreads functions, doing the + same kind of changes i did last summer when working on select (since + implement the timeout at the server side requires pthread_cond_timedwait) + <braunr> and i limit the message queue size of the port used to wake up + threads to 1 + <braunr> and it seems i have the same kind of problems, i.e. blocking + because of a second, unexpected send + <braunr> i'll try using __mach_msg_trap directly and see how it goes + <tschwinge> Hrm, mach/msg.c:__mach_msg does look correct to me, but yeah, + won't hurd to confirm this by looking what direct usage of + __mach_msg_trap is doing. + <braunr> tschwinge: can i ask if you still have a cthreads based hurd + around ? + <braunr> tschwinge: and if so, to send me libthreads.so.0.3 ... :) + <tschwinge> braunr: darnassus:~tschwinge/libthreads.so.0.3 + <braunr> call 19c0 <mach_msg@plt> + <braunr> so, cthreads were also using the glibc wrapper + <braunr> and i never had a single MACH_SEND_INTERRUPTED + <braunr> or a busy queue :/ + <braunr> (IOW, no duplicated messages, and the wrapper indeed looks + correct, so it's something else) + <tschwinge> (Assuming Mach is doing the correct thing re interruptions, of + course...) + <braunr> mach doesn't implement it + <braunr> it's explicitely meant to be done in userspace + <braunr> mach merely reports the error + <braunr> i checked the osfmach code of libmach, it's almost exactly the + same as ours + <tschwinge> Yeah, I meant Mach returns the interurption code but anyway + completed the RPC. + <braunr> ok + <braunr> i don't expect mach wouldn't do it right + <braunr> the only difference in osf libmach is that, when retrying, + MACH_SEND_INTERRUPT|MACH_RCV_INTERRUPT are both masked (for both the + send/send+receive and receive cases) + <tschwinge> Hrm. + <braunr> but they say it's for performance, i.e. mach won't take the slow + path because of unexpected bits in the options + <braunr> we probably should do the same anyway + + +### IRC, freenode, #hurd, 2013-01-17 + + <braunr> tschwinge: i think our duplicated RPCs come from + hurd/intr-msg.c:148 (err == MACH_SEND_INTERRUPTED but !(option & + MACH_SEND_MSG)) + <braunr> a thread is interrupted by a signal meant for a different thread + <braunr> hum no, still not that .. + <braunr> or maybe .. :) + <tschwinge> Hrm. Why would it matter for for the current thread for which + reason (different thread) mach_msg_trap returns *_INTERRUPTED? + <braunr> mach_msg wouldn't return it, as explained in the comment + <braunr> the signal thread would, to indicate the send was completed but + the receive must be retried + <braunr> however, when retrying, the original user_options are used again, + which contain MACH_SEND_MSG + <braunr> i'll test with a modified version that masks it + <braunr> tschwinge: hm no, doesn't fix anything :( + + +### IRC, freenode, #hurd, 2013-01-18 + + <braunr> the duplicated rpc calls is one i find very very frustrating :/ + <youpi> you mean the dup writes we've seen lately? + <braunr> yes + <youpi> k + + +### IRC, freenode, #hurd, 2013-01-19 + + <braunr> all right, i think the duplicated message sends are due to thread + creation + <braunr> the duplicated message seems to be sent by the newly created + thread + <braunr> arg no, misread + + +### IRC, freenode, #hurd, 2013-01-20 + + <braunr> tschwinge: youpi: about the diplucated messages issue, it seems to + be caused by two threads (with pthreads) doing an rpc concurrently + <braunr> duplicated* + + +### IRC, freenode, #hurd, 2013-01-21 + + <braunr> ah, found something interesting + <braunr> tschwinge: there seems to be a race on our file descriptors + <braunr> the content written by one thread seems to be retained somewhere + and another thread writing data to the file descriptor will resend what + the first already did + <braunr> it could be a FILE race instead of fd one though + <braunr> yes, it's not at the fd level, it's above + <braunr> so good news, seems like the low level message/signalling code + isn't faulty here + <braunr> all right, simple explanation: our IO_lockfile functions are + no-ops + <pinotree> braunr: i found that out days ago, and samuel said they were + okay + +[[glibc]], `flockfile`/`ftrylockfile`/`funlockfile`. + + +## IRC, freenode, #hurd, 2013-01-15 + + <braunr> hmm, looks like subhurds have been broken by the pthreads patch :/ + <braunr> arg, we really do have broken subhurds :(( + <braunr> time for an immersion in the early hurd bootstrapping stuff + <tschwinge> Hrm. Narrowed down to cthreads -> pthread you say. + <braunr> i think so + <braunr> but i think the problem is only exposed + <braunr> it was already present before + <braunr> even for the main hurd, i sometimes have systems blocking on exec + <braunr> there must be a race there that showed far less frequently with + cthreads + <braunr> youpi: we broke subhurds :/ + <youpi> ? + <braunr> i can't start one + <braunr> exec seems to die and prevent the root file system from + progressing + <braunr> there must be a race, exposed by the switch to pthreads + <braunr> arg, looks like exec doesn't even reach main :( + <braunr> now, i'm wondering if it could be the tls support that stops exec + <braunr> although i wonder why exec would start correctly on a main hurd, + and not on a subhurd :( + <braunr> i even wonder how much progress ld.so.1 is able to make, and don't + have much idea on how to debug that + + +### IRC, freenode, #hurd, 2013-01-22 + + <braunr> hm, subhurds seem to be broken because of select + <braunr> damn select ! + <braunr> hm i see, we can't boot a subhurd that still uses libthreads from + a main hurd that doesn't + <braunr> the linker can't find it and doesn't start exec + <braunr> pinotree: do you understand what the fmh function does in + sysdeps/mach/hurd/dl-sysdep.c ? + <braunr> i think we broke subhurds by fixing vm_map with size 0 + <pinotree> braunr: no idea, but i remember thomas talking about this code + +[[vm_map_kernel_bug]] + + <braunr> it checks for KERN_INVALID_ADDRESS and KERN_NO_SPACE + <braunr> and calls assert_perror(err); to make sure it's one of them + <braunr> but now, KERN_INVALID_ARGUMENT can be returned + <braunr> ok i understand what it does + <braunr> and youpi has changed the code, so he does too + <braunr> (now i'm wondering why he didn't think of it when we fixed vm_map + size with 0 but his head must already be filled with other things so ..) + <braunr> anyway, once this is dealt with, we get subhurds back :) + <braunr> yes, with a slight change, my subhurd starts again \o/ + <braunr> youpi: i found the bug that prevents subhurds from booting + <braunr> it's caused by our fixing of vm_map with size 0 + <braunr> when ld.so.1 starts exec, the code in + sysdeps/mach/hurd/dl-sysdep.c fails because it doesn't expect the new + error code we introduced + <braunr> (the fmh functions) + <youpi> ah :) + <youpi> good :) + <braunr> adding KERN_INVALID_ARGUMENT to the list should do the job, but if + i understand the code correctly, checking if fmhs isn't 0 before calling + vm_map should do the work too + <braunr> s/do the work/work/ + <braunr> i'm not sure which is the preferred way + <youpi> otherwise I believe fmh could be just fixed to avoid calling vm_map + in the !fmhs case + <braunr> yes that's what i currently do + <braunr> at the start of the loop, just after computing it + <braunr> seems to work so far + + +## IRC, freenode, #hurd, 2013-01-22 + + <braunr> i have almost completed fixing both cancellation and timeout + handling, but there are still a few bugs remaining + <braunr> fyi, the related discussion was + https://lists.gnu.org/archive/html/bug-hurd/2012-08/msg00057.html diff --git a/open_issues/mach_tasks_memory_usage.mdwn b/open_issues/mach_tasks_memory_usage.mdwn index 9abb7639..7a7a77ce 100644 --- a/open_issues/mach_tasks_memory_usage.mdwn +++ b/open_issues/mach_tasks_memory_usage.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,9 +8,10 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_documentation]] +[[!tag open_issue_documentation open_issue_gnumach]] -IRC, freenode, #hurd, 2011-01-06 + +# IRC, freenode, #hurd, 2011-01-06 <antrik> hm, odd... vmstat tells me that ~500 MiB of RAM are in use; but the sum of all RSS is <300 MiB... what's the rest? @@ -100,7 +101,7 @@ IRC, freenode, #hurd, 2011-01-06 libraries -IRC, freenode, #hurd, 2011-07-24 +# IRC, freenode, #hurd, 2011-07-24 < braunr> the panic is probably due to memory shortage < braunr> so as antrik suggested, use more swap @@ -145,3 +146,30 @@ IRC, freenode, #hurd, 2011-07-24 looks like it is on both seqnos_memory_object_data_initialize and seqnos_memory_object_data_write < braunr> antrik: so i guess reserved memory is accounted for + + +# IRC, freenode, #hurd, 2013-01-12 + + <tschwinge> darnassus linking clang: 600 MiB swap in use and 22 MiB RAM + free, of 2 GiB. But ps shows a RSS of just 100 MiB, huh? + <tschwinge> Getting "better": near the end of the link, nearly 1 GiB swap + in use, and 200 KiB (!) RAM free. + <sobhan> can hurd have more than 1GB of ram ? + <tschwinge> And then it completed; 75 MiB swap in use, and 1.2 GiB RAM + free. + <braunr> tschwinge: unless i'm mistaken, mach uses the legacy "swapping" + bsd mechanism + <braunr> tschwinge: i.e. when it swaps a process, it swaps all of it + <braunr> tschwinge: the rest is probably one big anonymous vm object + containing the process space + <braunr> cached objects aren't currently well accounted + <braunr> (well, since youpi got my page cache patches in, they are, but + procfs isn't yet modified to report them) + <braunr> tschwinge: right, i'm currently looking at the machine and it + doesn't add up, i suppoe there are some big files still in the cache + <braunr> ah, git packed objects :p + <braunr> and a few llvm .a/.so/executable files too + <braunr> and since they're probably targets, they're built last, which + explains why they're retained in the cache for a while + +[[microkernel/mach/message/msgh_id]] (why on *that* page?). diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn index b32d6ba6..a1c8f235 100644 --- a/open_issues/mission_statement.mdwn +++ b/open_issues/mission_statement.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -697,3 +698,11 @@ License|/fdl]]."]]"""]] <braunr> nowhere_man: it can be used that way too <braunr> functional programming is getting more and more attention <braunr> so it's fine if you're a lisp fan really + + +# IRC, freenode, #hurd, 2013-02-04 + + <civodul> BTW, it's weird that the mission statement linked from + hurd.gnu.org is in weblog/ and written in the first person + <braunr> yes + <braunr> very :) diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn index f631a80b..d7804864 100644 --- a/open_issues/multithreading.mdwn +++ b/open_issues/multithreading.mdwn @@ -266,6 +266,94 @@ Tom Van Cutsem, 2009. async by nature, will create messages floods anyway +### IRC, freenode, #hurd, 2013-02-23 + + <braunr> hmm let's try something + <braunr> iirc, we cannot limit the max number of threads in libports + <braunr> but did someone try limiting the number of threads used by + libpager ? + <braunr> (the only source of system stability problems i currently have are + the unthrottled writeback requests) + <youpi> braunr: perhaps we can limit the amount of requests batched by the + ext2fs sync? + <braunr> youpi: that's another approach, yes + <youpi> (I'm not sure to understand what threads libpager create) + <braunr> youpi: one for each writeback request + <youpi> ew + <braunr> but it makes its own call to + ports_manage_port_operations_multithread + <braunr> i'll write a new ports_manage_port_operations_multithread_n + function that takes a mx threads parameter + <braunr> and see if it helps + <braunr> i thought replacing spin locks with mutexes would help, but it's + not enough, the true problem is simply far too much contention + <braunr> youpi: i still think we should increase the page dirty timeout to + 30 seconds + <youpi> wouldn't that actually increase the amount of request done in one + go? + <braunr> it would + <braunr> but other systems (including linux) do that + <youpi> but they group requests + <braunr> what linux does is scan pages every 5 seconds, and writeback those + who have been dirty for more than 30 secs + <braunr> hum yes but that's just a performance issue + <braunr> i mean, a separate one + <braunr> a great source of fs performance degradation is due to this + regular scan happenning at the same time regular I/O calls are made + <braunr> e.G. aptitude update + <braunr> so, as a first step, until the sync scan is truley optimized, we + could increase that interval + <youpi> I'm afraid of the resulting stability regression + <youpi> having 6 times as much writebacks to do + <braunr> i see + <braunr> my current patch seems to work fine for now + <braunr> i'll stress it some more + <braunr> (it limits the number of paging threads to 10 currently) + <braunr> but iirc, you fixed a deadlock with a debian patch there + <braunr> i think the case was a pager thread sending a request to the + kernel, and waiting for the kernel to call another RPC that would unblock + the pager thread + <braunr> ah yes it was merged upstream + <braunr> which means a thread calling memory_object_lock_request with sync + == 1 must wait for a memory_object_lock_completed + <braunr> so it can deadlock, whatever the number of threads + <braunr> i'll try creating two separate pools with a limited number of + threads then + <braunr> we probably have the same deadlock issue in + pager_change_attributes btw + <braunr> hm no, i can still bring a hurd down easily with a large i/o + request :( + <braunr> and now it just recovered after 20 seconds without any visible cpu + or i/o usage .. + <braunr> i'm giving up on this libpager issue + <braunr> it simply requires a redesign + + +### IRC, freenode, #hurd, 2013-02-28 + + <smindinvern> so what causes the stability issues? or is that not really + known yet? + <braunr> the basic idea is that the kernel handles the page cache + <braunr> and writebacks aren't correctly throttled + <braunr> so a huge number of threads (several hundreds, sometimes + thousands) are created + <braunr> when this pathological state is reached, it's very hard to recover + because of the various sources of (low) I/O in the system + <braunr> a simple line sent to syslog increases the load average + <braunr> the solution requires reworking the libpager library, and probably + the libdiskfs one too, perhaps others, certainly also the pagers + <braunr> maybe the kernel too, i'm not sure + <braunr> i'd say so because it manages a big part of the paging policy + + +### IRC, freenode, #hurd, 2013-03-02 + + <braunr> i think i have a simple-enough solution for the writeback + instability + +[[hurd/libpager]]. + + ## Alternative approaches: * <http://www.concurrencykit.org/> @@ -273,7 +361,7 @@ Tom Van Cutsem, 2009. * Continuation-passing style * [[microkernel/Mach]] internally [[uses - continuations|microkernel/mach/continuation]], too. + continuations|microkernel/mach/gnumach/continuation]], too. * [[Erlang-style_parallelism]] diff --git a/open_issues/nice_vs_mach_thread_priorities.mdwn b/open_issues/nice_vs_mach_thread_priorities.mdwn index 76788a53..e27d3018 100644 --- a/open_issues/nice_vs_mach_thread_priorities.mdwn +++ b/open_issues/nice_vs_mach_thread_priorities.mdwn @@ -373,3 +373,17 @@ here. <pochu> braunr: can't remember right now, either that or to fix a ftbfs in debian <youpi> iirc it's coreutils which wants proper nice levels + + +# IRC, OFTC, #debian-hurd, 2013-03-04 + + <Steap> Is it not possible to set the priority of a process to 1 ? + <Steap> these macros: + <Steap> #define MACH_PRIORITY_TO_NICE(prio) (2 * ((prio) - 12)) + <Steap> #define NICE_TO_MACH_PRIORITY(nice) (12 + ((nice) / 2)) + <Steap> are used in the setpriority() implementation of Hurd + <Steap> so setting a process' priority to 1 is just like setting it to 0 + <youpi> Steap: that has already been discussed to drop the *2 + <youpi> the issue is mach not supporting enough sched levels + <youpi> can be fixed, of course + <youpi> just nobody did yet diff --git a/open_issues/ogi.mdwn b/open_issues/ogi.mdwn index e4372dc0..c58d2ee1 100644 --- a/open_issues/ogi.mdwn +++ b/open_issues/ogi.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -23,3 +23,32 @@ interesting. checking copyright situation, also for thesis / w.r.t. university project + + IRC, freenode, #hurd, 2013-02-15: + + <tschwinge> ogi: The question was rather (IIRC) whether your + university has the copyright of this project, given it was done + on their time. + <ogi> tschwinge: no problems with my university + + +# IRC, freenode, #hurd, 2013-02-15 + + <ogi> braunr: i want to update my ext3fs server to ext4 actually + <braunr> you have an ext3 server ? + <ogi> braunr: this was my M.Sc. thesis and the 2G patch was a side effect + <ogi> braunr: but it easily crashes under stress, so not usable + <braunr> it does ? + <ogi> braunr: it's not available for download ATM + <braunr> are you sure it's not a thread storm issue caused by the + unthrottled mach writebacks ? + <ogi> braunr: i don't know, haven't looked at it since 2004 + <braunr> oh :) + <braunr> ok + <ogi> i have all ext3fs stuff archived, just haven't put it on + http://fire.tower.3.bg/ yet + <tschwinge> ogi: If the copyright situation is clear, we can put it into + upstream Git repositories, no matter how dirty it is. + <tschwinge> "dirty" in the sense of that it needs cleanup, has bugs, etc. + <ogi> so at some point i want to audit libdiskfs and then continue with + ext4fs: https://savannah.gnu.org/patch/?1839 diff --git a/open_issues/packaging_libpthread.mdwn b/open_issues/packaging_libpthread.mdwn index 18f124b4..171dc7a0 100644 --- a/open_issues/packaging_libpthread.mdwn +++ b/open_issues/packaging_libpthread.mdwn @@ -155,6 +155,30 @@ cherry-picked. upstream +## IRC, OFTC, #debian-hurd, 2013-02-08 + + <tschwinge> I also have it on my (never-ending) agenda to add libpthread to + the tschwinge/Roger_Whittaker branch and/or propose it be added upstream + (as a Git submodule?). + <pinotree> imho a git submodule could be a solution, if glibc people would + accept it + <pinotree> if so, libpthread.git would need proper glibc/x.y branches to + follow glibc + <tschwinge> Yep. + <tschwinge> I though that would be the least invasive approach for glibc + upstream -- and quite convenient for us, too. + <pinotree> after all, git submodules don't track branches, but point to + specific commits, no? + <tschwinge> Correct. + <tschwinge> So we can do locally/in Debian whatever we want, and every once + in a while update the upstream glibc commit ID for libpthread. + <pinotree> so we could update the git submodule references in glibc when + we've tested enough libpthread changes + <tschwinge> Just like when committing patches upstream, just without + pestering them with all the patches/commits. + <tschwinge> Yep. + + # IRC, freenode, #hurd, 2012-11-16 <pinotree> *** $(common-objpfx)resolv/gai_suspend.o: uses diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index 706e1632..be582e8a 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -2556,3 +2557,429 @@ License|/fdl]]."]]"""]] <braunr> you're asking if you can include the large store patch in your work, and by extension, in the main branch <braunr> i would say yes, but this must be discussed with others + + +## IRC, freenode, #hurd, 2013-02-18 + + <braunr> mcsim: so, currently reviewing gnumach + <mcsim> braunr: hello + <braunr> mcsim: the review branch, right ? + <mcsim> braunr: yes + <mcsim> braunr: What do you start with? + <braunr> memory refreshing + <braunr> i see you added the advice twice, to vm_object and vm_map_entry + <braunr> iirc, we agreed to only add it to map entries + <braunr> am i wrong ? + <mcsim> let me see + <braunr> the real question being: what do you use the object advice for ? + <mcsim> >iirc, we agreed to only add it to map entries + <mcsim> braunr: TBH, do not remember that. At some point we came to + conclusion that there should be only one advice. But I'm not sure if it + was final point. + <braunr> maybe it wasn't, yes + <braunr> that's why i've just reformulated the question + <mcsim> if (map_entry && (map_entry->advice != VM_ADVICE_DEFAULT)) + <mcsim> advice = map_entry->advice; + <mcsim> else + <mcsim> advice = object->advice; + <braunr> ok + <mcsim> It just participates in determining actual advice + <braunr> ok that's not a bad thing + <braunr> let's keep it + <braunr> please document VM_ADVICE_KEEP + <braunr> and rephrase "How to handle page faults" in vm_object.h to + something like 'How to tune page fault handling" + <braunr> mcsim: what's the point of VM_ADVICE_KEEP btw ? + <mcsim> braunr: Probably it is better to remove it? + <braunr> well if it doesn't do anything, probably + <mcsim> braunr: advising was part of mo_set_attributes before + <mcsim> no it is redudant + <braunr> i see + <braunr> so yes, remove it + <mcsim> s/no/now + <braunr> (don't waste time on a gcs-like changelog format for now) + <braunr> i also suggest creating _vX branches + <braunr> so we can compare the changes between each of your review branches + <braunr> hm, minor coding style issues like switch(...) instead of switch + (...) + <braunr> why does syscall_vm_advise return MACH_SEND_INTERRUPTED if the + target map is NULL ? + <braunr> is it modelled after an existing behaviour ? + <braunr> ah, it's the syscall version + <mcsim> braunr: every syscall does so + <braunr> and the error is supposed to be used by user stubs to switch to + the rpc version + <braunr> ok + <braunr> hm + <braunr> you've replaced obsolete port_set_select and port_set_backup calls + with your own + <braunr> don't do that + <braunr> instead, add your calls to the new gnumach interface + <braunr> mcsim: out of curiosity, have you actually tried the syscall + version ? + <mcsim> braunr: Isn't it called by default? + <braunr> i don't think so, no + <mcsim> than no + <braunr> ok + <braunr> you could name vm_get_advice_info vm_advice_info + <mcsim> regarding obsolete calls, did you say that only in regard of + port_set_* or all other calls too? + <braunr> all of the + <braunr> m + <braunr> i missed one, yes + <braunr> the idea is: don't change the existing interface + <mcsim> >you could name vm_get_advice_info vm_advice_info + <mcsim> could or should? i.e. rename? + <braunr> i'd say should, to remain consistent with the existing similar + calls + <mcsim> ok + <braunr> can you explain KERN_NO_DATA a bit more ? + <braunr> i suppose it's what servers should answer for neighbour pages that + don't exist in the backend, right ? + <mcsim> kernel can ask server for some data to read them beforehand, but + server can be in situation when it does not know what data should be + prefetched + <mcsim> yes + <braunr> ok + <mcsim> it is used by ext2 server + <mcsim> with large store patch + <braunr> so its purpose is to allow the kernel to free the preallocated + pages that won't be used + <braunr> do i get it right ? + <mcsim> no. + <mcsim> ext2 server has a buffer for pages and when kernel asks to read + pages ahead it specifies region of that buffer + <braunr> ah ok + <mcsim> but consecutive pages in buffer does not correspond to consecutive + pages on disk + <braunr> so, the kernel can only prefetch pages that were already read by + the server ? + <mcsim> no, it can ask a server to prefetch pages that were not read by + server + <braunr> hum + <braunr> ok + <mcsim> but in case with buffer, if buffer page is empty, server does not + know what to prefetch + <braunr> i'm not sure i'm following + <braunr> well, i'm sure i'm not following + <braunr> what happens when the kernel requests data from a server, right + after a page fault ? + <braunr> what does the message afk for ? + <mcsim> kernel is unaware regarding actual size of file where was page + fault because of buffer indirection, right? + <braunr> i don't know what "buffer" refers to here + <mcsim> this is buffer in memory where ext2 server reads pages + <mcsim> with large store patch ext2 server does not map the whole disk, but + some of its pages + <mcsim> and it maps these pages in special buffer + <mcsim> that means that constructiveness of pages in memory does not mean + that they are consecutive on disk or logically (belong to the same file) + <braunr> ok so it's a page pool + <braunr> with unordered pages + <braunr> but what do you mean when you say "server does not know what to + prefetch" + <braunr> it normally has everything to determine that + <mcsim> For instance, page fault occurs that leads to reading of + 4k-file. But kernel does not know actual size of file and asks to + prefetch 16K bytes + <braunr> yes + <mcsim> There is no sense to prefetch something that does not belong to + this file + <braunr> yes but the server *knows* that + <mcsim> and server answers with KERN_NO_DATA + <mcsim> server should always say something about every page that was asked + <braunr> then, again, isn't the purpose of KERN_NO_DATA to notify the + kernel it can release the preallocated pages meant for the non existing + data ? + <braunr> (non existing or more generally non prefetchable) + <mcsim> yes + <braunr> then + <braunr> why did you answer no to + <braunr> 15:46 < braunr> so its purpose is to allow the kernel to free the + preallocated pages that won't be used + <braunr> is there something missing ? + <braunr> (well obviously, notify the kernel it can go on with page fault + handling) + <mcsim> braunr: sorry, misunderstoo/misread + <braunr> ok + <braunr> so good, i got this right :) + <braunr> i wonder if KERN_NO_DATA may be a bit too vague + <braunr> people might confuse it with ENODATA + <mcsim> Actually, this is transformation of ENODATA + <mcsim> I was looking among POSIX error codes and thought that this is the + most appropriate + <braunr> i'm not sure it is + <braunr> first, it's about STREAMS, a commonly unused feature + <braunr> and second, the code is obsolete + <mcsim> braunr: AFAIR purpose of KERN_NO_DATA is not only free + pages. Without this call something should hang + <braunr> 15:59 < braunr> (well obviously, notify the kernel it can go on + with page fault handling) + <mcsim> yes + <braunr> hm + <mcsim> sorry again + <braunr> i don't see anything better for the error name for now + <braunr> and it's really minor so let's keep it as it is + <braunr> actually, ENODATA being obsolete helps here + <braunr> ok, done for now, work calling + <braunr> we'll continue later or tomorrow + <mcsim> braunr: ok + <braunr> other than that, this looks ok on the kernel side for now + <braunr> the next change is a bit larger so i'd like to take the time to + read it + <mcsim> braunr: ok + <mcsim> regarding moving calls in mach.defs, can I put them elsewhere? + <braunr> gnumach.defs + <braunr> you'll probably need to rebase your changes to get it + <mcsim> braunr: I'll rebase this later, when we finish with review + <braunr> ok + <braunr> keep the comments in a list then, not to forget + <braunr> (logging irc is also useful) + + +## IRC, freenode, #hurd, 2013-02-20 + + <braunr> mcsim: why does VM_ADVICE_DEFAULT have its own entry ? + <mcsim> braunr: this kind of fallback mode + <mcsim> i suppose that even random strategy could even read several pages + at once + <braunr> yes + <braunr> but then, why did you name it "default" ? + <mcsim> because it is assigned by default + <braunr> ah + <braunr> so you expect pagers to set something else + <braunr> for all objects they create + <mcsim> yes + <braunr> ok + <braunr> why not, but add a comment please + <mcsim> at least until all pagers will support clustered reading + <mcsim> ok + <braunr> even after that, it's ok + <braunr> just say it's there to keep the previous behaviour by default + <braunr> so people don't get the idea of changing it too easily + <mcsim> comment in vm_advice.h? + <braunr> no, in vm_fault.C + <braunr> right above the array + <braunr> why does vm_calculate_clusters return two ranges ? + <braunr> also, "Function PAGE_IS_NOT_ELIGIBLE is used to determine if", + PAGE_IS_NOT_ELIGIBLE doesn't look like a function + <mcsim> I thought make it possible not only prefetch range, but also free + some memory that is not used already + <mcsim> braunr: ^ + <mcsim> but didn't implement it :/ + <braunr> don't overengineer it + <braunr> reduce to what's needed + <mcsim> braunr: ok + <mcsim> braunr: do you think it's worth to implement? + <braunr> no + <mcsim> braunr: it could be useful for sequential policy + <braunr> describe what you have in mind a bit more please, i think i don't + have the complete picture + <mcsim> with sequential policy user supposed to read strictly in sequential + order, so pages that user is not supposed to read could be put in unused + list + <braunr> what pages the user isn't supposed to read ? + <mcsim> if user read pages in increasing order than it is not supposed to + read pages that are right before the page where page fault occured + <braunr> right ? + <braunr> do you mean higher ? + <mcsim> that are before + <braunr> before would be lower then + <braunr> oh + <braunr> "right before" + <mcsim> yes :) + <braunr> why not ? + <braunr> the initial assumption, that MADV_SEQUENTIAL expects *strict* + sequential access, looks wrong + <braunr> remember it's just a hint + <braunr> a user could just acces pages that are closer to one another and + still use MADV_SEQUENTIAL, expecting a speedup because pages are close + <braunr> well ok, this wouldn't be wise + <braunr> MADV_SEQUENTIAL should be optimized for true sequential access, + agreed + <braunr> but i'm not sure i'm following you + <mcsim> but I'm not going to page these pages out. Just put in unused + list, and if they will be used later they will be move to active list + <braunr> your optimization seem to be about freeing pages that were + prefetched and not actually accessed + <braunr> what's the unused list ? + <mcsim> inactive list + <braunr> ok + <braunr> so that they're freed sooner + <mcsim> yes + <braunr> well, i guess all neighbour pages should first be put in the + inactive list + <braunr> iirc, pages in the inactive list aren't mapped + <braunr> this would force another page fault, with a quick resolution, to + tell the vm system the page was actually used, and must become active, + and paged out later than other inactive pages + <braunr> but i really think it's not worth doing it now + <braunr> clustered pagins is about improving I/O + <braunr> page faults without I/O are orders of magnitude faster than I/O + <braunr> it wouldn't bring much right now + <mcsim> ok, I remove this, but put in TODO + <mcsim> I'm not sure that right list is inactive list, but the list that is + scanned to pageout pages to swap partition. There should be such list + <braunr> both the active and inactive are + <braunr> the active one is scanned when the inactive isn't large enough + <braunr> (the current ratio of active pages is limited to 1/3) + <braunr> (btw, we could try increasing it to 1/2) + <braunr> iirc, linux uses 1/2 + <braunr> your comment about unlock_request isn't obvious, i'll have to + reread again + <braunr> i mean, the problem isn't obvious + <braunr> ew, functions with so many indentation levels :/ + <braunr> i forgot how ugly some parts of the mach vm were + <braunr> mcsim: basically it's ok, i'll wait for the simplified version for + another pass + <mcsim> simplified? + <braunr> 22:11 < braunr> reduce to what's needed + <mcsim> ok + <mcsim> and what comment? + <braunr> your XXX in vm_fault.c + <braunr> when calling vm_calculate_clusters + <mcsim> is m->unlock_request the same for all cluster or I should + recalculate it for every page? + <mcsim> s/all/whole + <braunr> that's what i say, i'll have to come back to that later + <braunr> after i have reviewed the userspace code i think + <braunr> so i understand the interactions better + <mcsim> braunr: pushed v1 branch + <mcsim> braunr: "Move new calls to gnumach.defs file" and "Implement + putting pages in inactive list with sequential policy" are in my TODO + <braunr> mcsim: ok + + +## IRC, freenode, #hurd, 2013-02-24 + + <braunr> mcsim: where does the commit from neal (reworking libpager) come + from ? + <braunr> (ok the question looks a little weird semantically but i think you + get my point) + <mcsim> braunr: you want me to give you a link to mail with this commit? + <braunr> why not, yes + <mcsim> http://permalink.gmane.org/gmane.os.hurd.bugs/446 + <braunr> ok so + http://lists.gnu.org/archive/html/bug-hurd/2012-06/msg00001.html + <braunr> ok so, we actually have three things to review here + <braunr> that libpager patch, the ext2fs large store one, and your work + <braunr> mcsim: i suppose something in your work depends on neal's patch, + right ? + <braunr> i mean, why did you work on top of it ? + <mcsim> Yes + <mcsim> All user level code + <braunr> i see it adds some notifications + <mcsim> no + <mcsim> notifacations are for large store + <braunr> ok + <mcsim> but the rest is for my work + <braunr> but what does it do that you require ? + <mcsim> braunr: this patch adds support for multipage work. There were just + stubs that returned errors for chunks longer than one page before. + <braunr> ok + <braunr> for now, i'll just consider that it's ok, as well as the large + store patch + <braunr> ok i've skipped all patches up to "Make mach-defpager process + multipage requests in m_o_data_request." since they're obvious + <braunr> but this one isn't + <braunr> mcsim: why is the offset member a vm_size_t in struct block ? + <braunr> (these things matter for large file support on 32-bit systems) + <mcsim> braunr: It should be vm_offset_t, right? + <braunr> yes + <braunr> well + <braunr> it seems so but + <braunr> im not sure what offset is here + <braunr> vm_offset is normally the offset inside a vm_object + <braunr> and if we want large file support, it could become a 64-bit + integer + <braunr> while vm_size_t is a size inside an address space, so it's either + 32 or 64-bit, depending on the address space size + <braunr> but here, if offset is an offset inside an address space, + vm_size_t is fine + <braunr> same question for send_range_parameters + <mcsim> braunr: TBH, I do not differ vm_size_t and vm_offset_t well + <braunr> they can be easily confused yes + <braunr> they're both offsets and sizes actually + <braunr> they're integers + <mcsim> so here I used vm_offset_t because field name is offset + <braunr> but vm_size_t is an offset/size inside an address space (a + vm_map), while vm_offset_t is an offset/size inside an object + <mcsim> braunr: I didn't know that + <braunr> it's not clear at all + <braunr> and it may not have been that clear in mach either + <braunr> but i think it's best to consider them this way from now on + <braunr> well, it's not that important anyway since we don't have large + file support, but we should some day :/ + <braunr> i'm afraid we'll have it as a side effect of the 64-bit port + <braunr> mcsim: just name them vm_offset_t when they're offsets for + consistency + <mcsim> but seems that I guessed, because I use vm_offset_t variables in + mo_ functions + <braunr> well ok, but my question was about struct block + <braunr> where you use vm_size_t + <mcsim> braunr: I consider this like a mistake + <braunr> ok + <braunr> moving on + <braunr> in upload_range, there are two XXX comments + <braunr> i'm not sure to understand + <mcsim> Second XXX I put because at the moment when I wrote this not all + hurd libraries and servers supported size different from vm_page_size + <mcsim> But then I fixed this and replaced vm_page_size with size in + page_read_file_direct + <braunr> ok then update the comment accordingly + <mcsim> When I was adding third XXX, I tried to check everything. But I + still had felling that I forgot something. + <mcsim> No it is better to remove second and third XXX, since I didn't find + what I missed + <braunr> well, that's what i mean by "update" :) + <mcsim> ok + <mcsim> and first XXX just an optimisation. Its idea is that there is no + case when the whole structure is used in one function. + <braunr> ok + <mcsim> But I was not sure if was worth to do, because if there will appear + some bug in future it could be hard to find it. + <mcsim> I mean that maintainability decreases because of using union + <mcsim> So, I'd rather keep it like it is + <braunr> how is struct send_range_parameters used ? + <braunr> it doesn't looked to be something stored long + <braunr> also, you're allowed to use GNU extensions + <mcsim> It is used to pass parameters from one function to another + <mcsim> which of them? + <braunr> see + http://gcc.gnu.org/onlinedocs/gcc-4.4.7/gcc/Unnamed-Fields.html#Unnamed-Fields + <braunr> mcsim: if it's used to pass parameters, it's likely always on the + stack + <mcsim> braunr: I use it when necessary + <braunr> we really don't care much about a few extra words on the stack + <braunr> the difference in size would + <mcsim> agree + <braunr> matter + <braunr> oops + <braunr> the difference in size would matter if a lot of those were stored + in memory for long durations + <braunr> that's not the case, so the size isn't a problem, and you should + remove the comment + <mcsim> ok + <braunr> mcsim: if i get it right, the libpager rework patch changes some + parameters from byte offset to page frame numbers + <mcsim> braunr: yes + <braunr> why don't you check errors in send_range ? + <mcsim> braunr: it was absent in original code, but you're right, I should + do this + <braunr> i'm not sure how to handle any error there, but at least an assert + <mcsim> I found a place where pager just panics + <braunr> for now it's ok + <braunr> your work isn't about avoiding panics, but there must be a check, + so if we can debug it and reach that point, we'll know what went wrong + <braunr> i don't understand the prototype change of default_read :/ + <braunr> it looks like it doesn't return anything any more + <braunr> has it become asynchronous ? + <mcsim> It was returning some status before, but now it handles this status + on its own + <braunr> hum + <braunr> how ? + <braunr> how do you deal with errors ? + <mcsim> in old code default_read returned kr and this kr was used to + determine what m_o_ function will be used + <mcsim> now default_read calls m_o_ on its own + <braunr> ok diff --git a/open_issues/pfinet_timers.mdwn b/open_issues/pfinet_timers.mdwn new file mode 100644 index 00000000..387ad4fe --- /dev/null +++ b/open_issues/pfinet_timers.mdwn @@ -0,0 +1,17 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-02-11 + + <braunr> now that there is a pthread_hurd_cond_timedwait_np function + available, we could replace the ulgy timers in pfinet diff --git a/open_issues/pflocal_socket_credentials_for_local_sockets.mdwn b/open_issues/pflocal_socket_credentials_for_local_sockets.mdwn index dfdc213c..d252eb54 100644 --- a/open_issues/pflocal_socket_credentials_for_local_sockets.mdwn +++ b/open_issues/pflocal_socket_credentials_for_local_sockets.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,10 +8,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -IRC, freenode, #hurd, 2011-03-28 - [[!tag open_issue_hurd]] + +# IRC, freenode, #hurd, 2011-03-28 + <pinotree> basically, i'm trying to implement socket credentials for local sockets, and i guessed doing it in pflocal would be the appropriate place <pinotree> what i thought was filling the cmsg data for MSG_CRED at @@ -41,6 +42,25 @@ IRC, freenode, #hurd, 2011-03-28 <youpi> yes <pinotree> nice thanks, i will try that change first + +# IRC, OFTC, #debian-hurd, 2013-02-20 + + <pinotree> youpi: while debugging #700530, it seems that xorg does not have + working socket credentials on kfreebsd (and hurd too) + <pinotree> julien provided sune with + http://people.debian.org/~jcristau/kbsd-peercred.diff to test, but of + course that won't work for us (even if we would have working socket + credentials with cmsg) + <pinotree> (that patch is not tested yet) + <pinotree> at least, we're aware there's another place in need for working + socket credentials now + <youpi> k + <pinotree> youpi: (the patch above has been confirmed to work, with + s/SOL_SOCKET/0/ ) + <youpi> 0 ?! + <pinotree> yeah + + --- See also [[pflocal_reauth]] and [[sendmsg_scm_creds]]. diff --git a/open_issues/ps_SIGSEGV.mdwn b/open_issues/ps_SIGSEGV.mdwn new file mode 100644 index 00000000..24d5cb4f --- /dev/null +++ b/open_issues/ps_SIGSEGV.mdwn @@ -0,0 +1,17 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-02-05 + + <braunr> ps -l segfaults + <braunr> ah, perhaps because of the subhurd diff --git a/open_issues/rpc_stub_generator.mdwn b/open_issues/rpc_stub_generator.mdwn index 05eb53b8..d4622d67 100644 --- a/open_issues/rpc_stub_generator.mdwn +++ b/open_issues/rpc_stub_generator.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -97,3 +97,50 @@ License|/fdl]]."]]"""]] scatter-gather to be used with x15 <braunr> once more, i fell back on omg idl <braunr> oh, there is also flick that looks interesting + + +# IRC, freenode, #hurd, 2013-13-16 + + <tschwinge> braunr: By the way, regarding your recent IDL considerations + (and I too suggest using some kind of RPC generator basone on whichever + IDL) -- are you aware that for Viengoos, Neal has written a RPC stub + generator entirely in C Preprocessor macros? No idea whather that's + suitable for your case, but may be worth having a look at. + <neal> it probably isn't easy to port to Mach + <neal> genode has an ipc generator as well + <neal> which is written in a real langugage + <neal> that might be worth checking out as well + <neal> (note: I haven't followed the conversation at all.) + <braunr> i was considering using macros only too actually + <braunr> (i thought genode had switched to complex c++ templates) + <neal> dunno + <neal> I'm not up to date + <neal> macros are nice, but marshalling complicated data structures is hard + <sekon_> why implement it with just macros ?? + <neal> no lexer, no parser + <neal> no special special tools + <neal> the first are a burden + <neal> the latter is a pain + <neal> + http://git.savannah.gnu.org/gitweb/?p=hurd/viengoos.git;a=blob;f=libviengoos/viengoos/rpc.h;h=721768358a0299637fb79f226aea6a304571da85;hb=refs/heads/viengoos-on-bare-metal + <neal> in the same directory, you there are headers that use it + <braunr> neal: cf. http://genode.org/documentation/release-notes/11.05 + <braunr> tschwinge: why do you recommend an IDL ? + <neal> braunr: What about it? + <braunr> neal: it shows the difference between the earlier ipc/rpc + interface, and the new one based only on templates and dynamic + marshalling using c++ streams + <neal> ok + <tschwinge> braunr: In my book, the definition of RPC interfaces is just + "data" in the sense that it describes data structures (exchanged + messages) and as such should be expressed as data (by means of an IDL), + instead of directly codifying it in a specific programming language. + <tschwinge> Of course, there may be other reasons for doing the latter + anyway, such as performance/optimization reasons. + <braunr> tschwinge: well, from my pov, you're justifying the use of an idl + from the definition of an rpc + <braunr> i'm not sure it makes much sense for me + <braunr> in addition, the idl becomes the "specific programming language" + <tschwinge> Well, I see it as data that has to be translated into several + formats: different programming languages' stub code. + <braunr> you could consider c the "common" language :) diff --git a/open_issues/select.mdwn b/open_issues/select.mdwn index 391509a9..caecc437 100644 --- a/open_issues/select.mdwn +++ b/open_issues/select.mdwn @@ -215,7 +215,7 @@ IRC, unknown channel, unknown date: <youpi> it's better than nothing yes -# IRC, freenode, #hurd, 2012-07-21 +### IRC, freenode, #hurd, 2012-07-21 <braunr> damn, select is actually completely misdesigned :/ <braunr> iiuc, it makes servers *block*, in turn :/ @@ -315,7 +315,7 @@ IRC, unknown channel, unknown date: really easy and nice :-) -## IRC, freenode, #hurd, 2012-07-22 +#### IRC, freenode, #hurd, 2012-07-22 <braunr> antrik: you can't block in servers with sync ipc <braunr> so in this case, "select" becomes a request for notifications @@ -379,7 +379,7 @@ IRC, unknown channel, unknown date: his reasoning (as does braunr) -## IRC, freenode, #hurd, 2012-07-23 +#### IRC, freenode, #hurd, 2012-07-23 <braunr> antrik: i was meaning sync in the most common meaning, yes, the client blocking on the reply @@ -650,6 +650,9 @@ IRC, unknown channel, unknown date: <braunr> which is why i could choose time_value_t (a struct of 2 integer_t) <youpi> well, I'd say gnumach could grow a nanosecond-precision time value <youpi> e.g. for clock_gettime precision and such + +[[clock_gettime]]. + <braunr> so you would prefer me adding the time_spec_t time to gnumach rather than the hurd ? <youpi> well, if hurd RPCs are using mach types and there's no mach type @@ -782,7 +785,7 @@ IRC, unknown channel, unknown date: API definition at RPC level too -## IRC, freenode, #hurd, 2012-07-24 +#### IRC, freenode, #hurd, 2012-07-24 <braunr> youpi: antrik: is vm_size_t an appropriate type for a c long ? <braunr> (appropriate mig type) @@ -809,7 +812,7 @@ IRC, unknown channel, unknown date: continue -## IRC, freenode, #hurd, 2012-07-25 +#### IRC, freenode, #hurd, 2012-07-25 <antrik_> braunr: well, for actual kernel calls, machine-specific types are probably hard to avoid... the problem is when they are used in other RPCs @@ -900,7 +903,7 @@ IRC, unknown channel, unknown date: <braunr> antrik: ah about that, ok -## IRC, freenode, #hurd, 2012-07-26 +#### IRC, freenode, #hurd, 2012-07-26 <pinotree> braunr: wrt your select_timeout branch, why not push only the time_data stuff to master? @@ -914,7 +917,7 @@ IRC, unknown channel, unknown date: <braunr> i "only" have to adjust the client side select implementation now -## IRC, freenode, #hurd, 2012-07-27 +#### IRC, freenode, #hurd, 2012-07-27 <braunr> io_select should remain a routine (i.e. synchronous) for server side stub code @@ -922,7 +925,14 @@ IRC, unknown channel, unknown date: <braunr> (since _hurs_select manually handles replies through a port set) -## IRC, freenode, #hurd, 2012-07-28 +##### IRC, freenode, #hurd, 2013-02-09 + + <braunr> io_select becomes a simpleroutine, except inside the hurd, where + it's a routine to keep the receive and reply mig stub code + <braunr> (the server side) + + +#### IRC, freenode, #hurd, 2012-07-28 <braunr> why are there both REPLY_PORTS and IO_SELECT_REPLY_PORT macros in the hurd .. @@ -941,7 +951,7 @@ IRC, unknown channel, unknown date: <braunr> i did something a bit ugly but it seems to do what i wanted -## IRC, freenode, #hurd, 2012-07-29 +#### IRC, freenode, #hurd, 2012-07-29 <braunr> good, i have a working client-side select <braunr> now i need to fix the servers a bit :x @@ -998,7 +1008,7 @@ IRC, unknown channel, unknown date: queue ... -## IRC, freenode, #hurd, 2012-07-30 +#### IRC, freenode, #hurd, 2012-07-30 <braunr> hm nice, the problem i have with my hurd_condition_timedwait seems to also exist in libpthread @@ -1096,7 +1106,7 @@ IRC, unknown channel, unknown date: (http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout&id=40fe717ba9093c0c893d9ea44673e46a6f9e0c7d) -## IRC, freenode, #hurd, 2012-08-01 +#### IRC, freenode, #hurd, 2012-08-01 <braunr> damn, i can't manage to make threads calling condition_wait to dequeue themselves from the condition queue :( @@ -1132,7 +1142,7 @@ IRC, unknown channel, unknown date: <braunr> it frightens me because i don't see any flaw in the logic :( -## IRC, freenode, #hurd, 2012-08-02 +#### IRC, freenode, #hurd, 2012-08-02 <braunr> ah, seems i found a reliable workaround to my deadlock issue, and more than a workaround, it should increase efficiency by reducing @@ -1150,7 +1160,7 @@ IRC, unknown channel, unknown date: (/etc/hurd/runsystem i assume) -## IRC, freenode, #hurd, 2012-08-03 +#### IRC, freenode, #hurd, 2012-08-03 <braunr> glibc actually makes some direct use of cthreads condition variables @@ -1302,7 +1312,7 @@ IRC, unknown channel, unknown date: <braunr> tests, reviews, more tests, polishing, commits, packaging -## IRC, freenode, #hurd, 2012-08-04 +#### IRC, freenode, #hurd, 2012-08-04 <braunr> grmbl, apt-get fails on select in my subhurd with the updated glibc @@ -1324,7 +1334,7 @@ IRC, unknown channel, unknown date: and thomas d -## IRC, freenode, #hurd, 2012-08-05 +#### IRC, freenode, #hurd, 2012-08-05 <braunr> eh, i made dpkg-buildpackage use the patched c library, and it finished the build oO @@ -1352,7 +1362,7 @@ IRC, unknown channel, unknown date: extremely large -## IRC, freenode, #hurd, 2012-08-06 +#### IRC, freenode, #hurd, 2012-08-06 <braunr> i have bad news :( it seems there can be memory corruptions with my io_select patch @@ -1395,7 +1405,7 @@ IRC, unknown channel, unknown date: [[libpthread]]. -## IRC, freenode, #hurd, 2012-08-07 +#### IRC, freenode, #hurd, 2012-08-07 <rbraun_hurd> anyone knows of applications extensively using non-blocking networking functions ? @@ -1470,7 +1480,7 @@ IRC, unknown channel, unknown date: other for the servers, eh) -## IRC, freenode, #hurd, 2012-08-07 +#### IRC, freenode, #hurd, 2012-08-07 <braunr> when running gitk on [darnassus], yesterday, i could push the CPU to 100% by simply moving the mouse in the window :p @@ -1490,7 +1500,7 @@ IRC, unknown channel, unknown date: <rbraunh> this linear search on dequeue is a real pain :/ -## IRC, freenode, #hurd, 2012-08-09 +#### IRC, freenode, #hurd, 2012-08-09 `screen` doesn't close a window/hangs after exiting the shell. @@ -1503,7 +1513,7 @@ IRC, unknown channel, unknown date: [[Term_blocking]]. -# IRC, freenode, #hurd, 2012-12-05 +### IRC, freenode, #hurd, 2012-12-05 <braunr> well if i'm unable to build my own packages, i'll send you the one line patch i wrote that fixes select/poll for the case where there is @@ -1512,7 +1522,7 @@ IRC, unknown channel, unknown date: timeout, doubling the total wait time when there is no event) -## IRC, freenode, #hurd, 2012-12-06 +#### IRC, freenode, #hurd, 2012-12-06 <braunr> damn, my eglibc patch breaks select :x <braunr> i guess i'll just simplify the code by using the same path for @@ -1546,12 +1556,12 @@ IRC, unknown channel, unknown date: <braunr> this can account for the slowness of a bunch of select/poll users -## IRC, freenode, #hurd, 2012-12-07 +#### IRC, freenode, #hurd, 2012-12-07 <braunr> finally, my select patch works :) -## IRC, freenode, #hurd, 2012-12-08 +#### IRC, freenode, #hurd, 2012-12-08 <braunr> for those interested, i pushed my eglibc packages that include this little select/poll timeout fix on my debian repository @@ -1560,7 +1570,7 @@ IRC, unknown channel, unknown date: regressions -## IRC, freenode, #hurd, 2012-12-10 +#### IRC, freenode, #hurd, 2012-12-10 <gnu_srs> I have verified your double timeout bug in hurdselect.c. <gnu_srs> Since I'm also working on hurdselect I have a few questions @@ -1631,7 +1641,13 @@ IRC, unknown channel, unknown date: <braunr> i'll try the non intrusive mode -## IRC, freenode, #hurd, 2012-12-11 +##### IRC, freenode, #hurd, 2013-01-26 + + <braunr> ah great, one of the recent fixes (probably select-eintr or + setitimer) fixed exim4 :) + + +#### IRC, freenode, #hurd, 2012-12-11 <gnu_srs1> braunr: What is the technical difference of having the delay at io_select compared to mach_msg for one FD? @@ -1641,7 +1657,7 @@ IRC, unknown channel, unknown date: <braunr> (for L4 guys it wouldn't be considered a slight optimization :)) -## IRC, freenode, #hurd, 2012-12-17 +#### IRC, freenode, #hurd, 2012-12-17 <braunr> tschwinge: http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/select_timeout_for_one_fd @@ -1668,20 +1684,20 @@ IRC, unknown channel, unknown date: notifications resulting from the way io_select works -## IRC, freenode, #hurd, 2012-12-19 +#### IRC, freenode, #hurd, 2012-12-19 <braunr> tschwinge: i've tested the glibc rbraun/select_timeout_for_one_fd branch for a few days on darnassus now, and nothing wrong to report -## IRC, freenode, #hurd, 2012-12-20 +#### IRC, freenode, #hurd, 2012-12-20 <youpi> braunr: so, shall I commit the single hurd select timeout fix to the debian package? <braunr> youpi: i'd say so yes -## IRC, freenode, #hurd, 2013-01-03 +#### IRC, freenode, #hurd, 2013-01-03 <braunr> gnu_srs: sorry, i don't understand your poll_timeout patch <braunr> it basically reverts mine for poll only @@ -1848,7 +1864,7 @@ IRC, unknown channel, unknown date: to the poll stuff: Have to check further with my poll patch... -## IRC, freenode, #hurd, 2013-01-04 +#### IRC, freenode, #hurd, 2013-01-04 <gnu_srs> Summary of the eglibc-2.13-38 issues: without the unsubmitted-setitimer_fix.diff patch and with @@ -2037,6 +2053,388 @@ IRC, unknown channel, unknown date: See also [[alarm_setitimer]]. +#### IRC, freenode, #hurd, 2013-01-22 + + <gnu_srs> youpi: Maybe it's overkill to have a separate case for DELAY; but + it enhances readability (and simplifies a lot too) + <youpi> but it reduces factorization + <youpi> if select is already supposed to behave the same way as delay, + there is no need for a separate code + <gnu_srs> OK; I'll make a two-way split then. What about POLL and nfds=0, + timeout !=0? + <braunr> gnu_srs: handle nfds=0 as a pure timeout as the linux man page + describes + <braunr> it makes sense, and as other popular systems do it, it's better to + do it the same way + <braunr> and i disagree with you, factorization doesn't imply less + readability + <gnu_srs> So you agree with me to have a special case for DELAY? + <gnu_srs> Coding style is a matter of taste: for me case a: case b: etc is + more readable than "if then elseif then else ..." + <braunr> it's not coding style + <braunr> avoiding duplication is almost always best + <braunr> whatever the style + <braunr> i don't see the need for a special delay case + <braunr> it's the same mach_msg call + <braunr> (for now) + <braunr> gnu_srs: i'd say the only reason to duplicate is when you can't do + otherwise + <gnu_srs> ways of coding then... And I agree with the idea of avoiding code + duplication, ever heard of Literate Programming + <braunr> we'll need a "special case" when the timeout is handled at the + server side, but it's like two lines .. + + +#### IRC, freenode, #hurd, 2013-02-11 + + <youpi> braunr: the libpthread hurd_cond_timedwait_np looks good to me + + +##### IRC, freenode, #hurd, 2013-02-15 + + <youpi> braunr: does cond_timedwait_np depend on the cancellation fix? + <braunr> yes + <youpi> ok + <braunr> the timeout fix + <youpi> so I also have to pull that into my glibc build + <braunr> (i fixed cancellation too because the cleanup routine had to be + adjusted anyway + <braunr> ) + <youpi> ah, and I need the patches hurd package too + <braunr> if unsure, you can check my packages + <youpi> ok, not for tonight then + <braunr> i listed the additional patches in the changelog + <youpi> yep, I'll probably use them + + +#### IRC, freenode, #hurd, 2013-02-11 + + <youpi> braunr: I don't understand one change in glibc: + <youpi> - err = __io_select (d[i].io_port, d[i].reply_port, 0, &type); + <youpi> + err = __io_select (d[i].io_port, d[i].reply_port, type); + <braunr> youpi: the waittime parameter ahs been removed + <braunr> has* + <youpi> where? when? + <braunr> in the hurd branch + <youpi> in the defs? + <braunr> yes + <youpi> I don't see this change + <youpi> only the addition of io_select_timeout + <braunr> hum + <youpi> also, io_select_timeout should be documented along io_select in + hurd.texi + <braunr> be6e5b86bdb9055b01ab929cb6b6eec49521ef93 + <braunr> Selectively compile io_select{,_timeout} as a routine + <braunr> * hurd/io.defs (io_select_timeout): Declare as a routine if + <braunr> _HURD_IO_SELECT_ROUTINE is defined, or a simpleroutine + otherwise. + <braunr> (io_select): Likewise. In addition, remove the waittime + timeout parameter. + <youpi> ah, it's in another commit + <braunr> yes, perhaps misplaced + <braunr> that's the kind of thing i want to polish + <braunr> my main issue currently is that time_data_t is passed by value + <braunr> i'm trying to pass it by address + <youpi> I don't know the details of routine vs simpleroutine + <braunr> it made sense for me to remove the waittime parameter at the same + time as adding the _HURD_IO_SELECT_ROUTINE macro, since waittime is what + allows glibc to use a synchronous RPC in an asynchronous way + <youpi> is it only a matter of timeout parameter? + <braunr> simpleroutine sends a message + <braunr> routine sends and receives + <braunr> by having a waittime parameter, _hurd_select could make io_select + send a message and return before having a reply + <youpi> ah, that's why in glibc you replaced MACH_RCV_TIMED_OUT by 0 + <braunr> yes + <youpi> it seems a bit odd to have a two-face call + <braunr> it is + <youpi> can't we just keep it as such? + <braunr> no + <youpi> damn + <braunr> well we could, but it really wouldn't make any sense + <youpi> why not? + <braunr> because the way select is implemented implies io_select doesn't + expect a reply + <braunr> (except for the single df case but that's an optimization) + <braunr> fd* + <youpi> that's how it is already, yes? + <braunr> yes + <braunr> well yes and no + <braunr> that's complicated :) + <braunr> there are two passes + <braunr> let me check before saying anything ;p + <youpi> :) + <youpi> in the io_select(timeout=0) case, can it ever happen that we + receive an answer? + <braunr> i don't think it is + <braunr> you mean non blocking right ? + <braunr> not infinite timeout + <youpi> I mean calling io_select with the timeout parameter being set to 0 + <braunr> so yes, non blocking + <braunr> no, i think we always get MACH_RCV_TIMED_OUT + <youpi> for me non-blocking can mean a lot of things :) + <braunr> ok + <braunr> i was thinking mach_msg here + <braunr> ok so, let's not consider the single fd case + <braunr> the first pass simply calls io_select with a timeout 0 to send + messages + <youpi> I don't think it's useful to try to optimize it + <youpi> it'd only lead to bugs :) + <braunr> me neither + <braunr> yes + <youpi> (as was shown :) ) + <braunr> what seems useful to me however is to optimize the io_select call + <braunr> with a waittime parameter, the generated code is an RPC (send | + receive) + <braunr> whereas, as a simpleroutine, it becomes a simple send + <youpi> ok + <youpi> my concern is that, as you change it, you change the API of the + __io_select() function + <youpi> (from libhurduser) + <braunr> yes but glibc is the only user + <braunr> and actually no + <braunr> i mean + <braunr> i change the api at the client side only + <youpi> that's what I mean + <braunr> remember that io.Defs is almost full + <youpi> "full" ? + <braunr> i'm almost certain it becomes full with io_select_timeout + <braunr> there is a practical limit of 100 calls per interface iirc + <braunr> since the reply identifiers are request + 100 + <youpi> are we at it already? + <braunr> i remember i had problems with it so probably + <youpi> but anyway, I'm not thinking about introducing yet another RPC + <youpi> but get a reasonable state of io_select + <braunr> i'l have to check that limit + <braunr> it looks wrong now + <braunr> or was it 50 + <braunr> i don't remember :/ + <braunr> i understand + <braunr> but what i can guarantee is that, while the api changes at the + client side, it doesn't at the server side + <youpi> ideally, the client api of io_select could be left as it is, and + libc use it as a simpleroutine + <youpi> sure, I understand that + <braunr> which means glibc, whether patched or not, still works fine with + that call + <braunr> yes it could + <braunr> that's merely a performance optimization + <youpi> my concern is that an API depends on the presence of + _HURD_IO_SELECT_ROUTINE, and backward compatibility being brought by + defining it! :) + <braunr> yes + <braunr> i personally don't mind much + <youpi> I'd rather avoid the clutter + <braunr> what do you mean ? + <youpi> anything that avoids this situation + <youpi> like just using timeout = 0 + <braunr> well, in that case, we'll have both a useless timeout at the + client side + <braunr> and another call for truely passing a timeout + <braunr> that's also weird + <youpi> how so a useless timeout at the client side? + <braunr> 22:39 < youpi> - err = __io_select (d[i].io_port, d[i].reply_port, + 0, &type); + <braunr> 0 here is the waittime parameter + <youpi> that's a 0-timeout + <braunr> and it will have to be 0 + <youpi> yes + <braunr> that's confusing + <youpi> ah, you mean the two io_select calls? + <braunr> yes + <youpi> but isn't that necessary for the several-fd case, anyway? + <braunr> ? + <braunr> if the io_select calls are simple routines, this useless waittime + parameter can just be omitted like i did + <youpi> don't we *have* to make several calls when we select on several + fds? + <braunr> suure but i don't see how it's related + <youpi> well then I don't see what optimization you are doing then + <youpi> except dropping a parameter + <youpi> which does not bring much to my standard :) + <braunr> a simpleroutine makes mach_msg take a much shorter path + <youpi> that the 0-timeout doesn't take? + <braunr> yes + <braunr> it's a send | receive + <youpi> ok, but that's why I asked before + <braunr> so there are a bunch of additional checks until the timeout is + handled + <youpi> whether timeout=0 means we can't get a receive + <youpi> and thus the kernel could optimize + <braunr> that's not the same thing :) + <youpi> ok + <braunr> it's a longer path to the same result + <youpi> I'd really rather see glibc building its own private simpleroutine + version of io_select + <youpi> iirc we already have such kind of thing + <braunr> ok + <braunr> well there are io_request and io_reply defs + <braunr> but i haven't seen them used anywhere + <braunr> but agreed, we should do that + <youpi> braunr: the prototype for io_select seems bogus in the io_request, + id_tag is no more since ages :) + <braunr> youpi: yes + <braunr> youpi: i'll recreate my hurd branch with only one commit + <braunr> without the routine/simpleroutine hack + <braunr> and with time_data_t passed by address + <braunr> and perhaps other very minor changes + <youpi> braunr: the firstfd == -1 test needs a comment + <braunr> or better, i'll create a v2 branch to make it easy to compare them + <braunr> ok + <youpi> braunr: actually it's also the other branch of the if which needs a + comment: "we rely on servers implementing the timeout" + <braunr> youpi: ok + <youpi> - (msg.success.result & SELECT_ALL) == 0) + <youpi> why removing that test? + <youpi> you also need to document the difference between got and ready + <braunr> hm i'll have to remember + <braunr> i wrote this code like a year ago :) + <braunr> almost + <youpi> AIUI, got is the number of replies + <braunr> but i think it has to do with error handling + <braunr> and + <braunr> + if (d[i].type) + <braunr> + ++ready; + <youpi> while ready is the number of successful replies + <braunr> is what replaces it + <braunr> youpi: yes + <braunr> the poll wrapper already normalizes the timeout parameter to + _hurd_select + <braunr> no you probably don't + <braunr> the whole point of the patch is to remove this ugly hack + <braunr> youpi: ok so + <braunr> 23:24 < youpi> - (msg.success.result & SELECT_ALL) + == 0) + <braunr> when a request times out + <youpi> ah, right + <braunr> we could get a result with no event + <braunr> and no error + <braunr> and this is what makes got != ready + <youpi> tell that to the source, not me :) + <braunr> sure :) + <braunr> i'm also saying it to myself + <braunr> ... :) + <youpi> right, using io_select_request() is only an optimization, which we + can do later + <braunr> what i currently do is remove the waittime parameter from + io_select + <braunr> what we'll do instead (soon) is let the parameter there to keep + the API unchancged + <braunr> but always use a waittime of 0 + <braunr> to make the mach_msg call non blocking + <braunr> then we'll try to get the io_request/io_reply definitions back so + we can have simpleroutines (send only) version of the io RPCs + <braunr> and we'll use io_select_request (without a waittime) + <braunr> youpi: is that what you understood too ? + <youpi> yes + <youpi> (and we can do that later) + <braunr> gnu_srs: does it make more sense for you ? + <braunr> this change is quite sparsed so it's not easy to get the big + picture + <braunr> sparse* + <braunr> it requires changes in libpthread, the hurd, and glibc + <youpi> the libpthread change can be almost forgotten + <youpi> it's just yet another cond_foo function :) + <braunr> well not if he's building his own packages + <youpi> right + <youpi> ok, apart from the io_select_request() and documenting the newer + io_select_timeout(), the changes seem good to me + <braunr> youpi: actually, a send | timeout takes the slow path in mach_msg + <braunr> and i actually wonder if send | receive | timeout = 0 can get a + valid reply from the server + <braunr> but the select code already handles that so it shouldn't be much + of a problem + <youpi> k + + +##### IRC, freenode, #hurd, 2013-02-12 + + <braunr> hum + <braunr> io_select_timeout actually has to be a simpleroutine at the client + side :/ + <braunr> grmbl + <youpi> ah? + <braunr> otherwise it blocks + <youpi> how so? + <braunr> routines wait for replies + <youpi> even with timeout 0? + <braunr> there is no waittime for io_select_timeout + <braunr> adding one would be really weird + <youpi> oh, sorry, I thought you were talking about io_select + <braunr> it would be more interesting to directly use + io_select_timeout_request + <braunr> but this means additional and separate work to make the + request/reply defs up to date + <braunr> and used + <braunr> personally i don't mind, but it matters for wheezy + <braunr> youpi: i suppose it's not difficult to add .defs to glibc, is it ? + <braunr> i mean, make glibc build the stub code + <youpi> it's probably not difficult indeed + <braunr> ok then it's better to do that first + <youpi> yes + <youpi> there's faultexec for instance in hurd/Makefile + <braunr> ok + <youpi> or rather, apparently it'd be simply user-interfaces + <youpi> it'll probably be linked into libhurduser + <youpi> but with an odd-enough name it shouldn't matter + <braunr> youpi: adding io_request to the list does indeed build the RPCs :) + <braunr> i'll write a patch to sync io/io_reply/io_request + <braunr> youpi: oh by the way, i'm having a small issue with the + io_{reply,request} interfaces + <braunr> the generated headers both share the same enclosing macro + (_io_user) + <braunr> so i'm getting compiler warning + <braunr> s + <youpi> we could fix that quickly in mig, couldn't we? + <braunr> youpi: i suppose, yes, just mentioning + + +##### IRC, freenode, #hurd, 2013-02-19 + + <youpi> in the hurdselect.c code, I'd rather see it td[0]. rather than + td-> + <braunr> ok + <youpi> otherwise it's frownprone + <youpi> (it has just made me frown :) ) + <braunr> yes, that looked odd to me too, but at the same time, i didn't + want it to seem to contain several elements + <youpi> I prefer it to look like there could be several elements (and then + the reader has to find out how many, i.e. 1), rather than it to look like + the pointer is not initialized + <braunr> right + <youpi> I'll also rather move that code further + <youpi> so the preparation can set timeout to 0 + <youpi> (needed for poll) + <youpi> how about turning your branch into a tg branch? + <braunr> feel free to add your modifications on top of it + <braunr> sure + <youpi> ok + <youpi> I'll handle these then + <braunr> youpi: i made an updated changelog entry in the + io_select_timeout_v3 branch + <youpi> could you rather commit that to the t/io_select_timeout branch I've + just created? + <braunr> i mean, i did that a few days ago + <youpi> (in the .topmsg file) + <youpi> ah + <youpi> k + + +##### IRC, freenode, #hurd, 2013-02-26 + + <braunr> youpi: i've just pushed a rbraun/select_timeout_pthread_v4 branch + in the hurd repository that includes the changes we discussed yesterday + <braunr> untested, but easy to compare with the previous version + + +##### IRC, freenode, #hurd, 2013-02-27 + + <youpi> braunr: io_select_timeout seems to be working fine here + <youpi> braunr: I feel like uploading them to debian-ports, what do you + think? + <braunr> youpi: the packages i rebuild last night work fine too + + # See Also See also [[select_bogus_fd]] and [[select_vs_signals]]. diff --git a/open_issues/select_vs_signals.mdwn b/open_issues/select_vs_signals.mdwn index 9e9699b8..db616acb 100644 --- a/open_issues/select_vs_signals.mdwn +++ b/open_issues/select_vs_signals.mdwn @@ -42,6 +42,21 @@ In context of [[alarm_setitimer]]. Proposed patch: [[!message-id "20130105162817.GW5965@type.youpi.perso.aquilenet.fr"]]. + +## IRC, freenode, #hurd, 2013-01-15 + + <_d3f> Hello, any one else having problems with git? + <braunr> _d3f: yes + <braunr> _d3f: it will be fixed in the next glibc release + <_d3f> oh thx. what was the problem? + <braunr> http://lists.gnu.org/archive/html/bug-hurd/2013-01/msg00005.html + <WhiteKIBA> exactly this problem is preventing us building glibc + <braunr> it's indeed very annoying + <braunr> and this fix will probably have a visible and positive effect on + other issues + <_d3f> let's hope so. + <braunr> well, i'm already using it and see the difference + --- See also [[select]] and [[select_bogus_fd]]. diff --git a/open_issues/some_todo_list.mdwn b/open_issues/some_todo_list.mdwn index 82822a29..80592abf 100644 --- a/open_issues/some_todo_list.mdwn +++ b/open_issues/some_todo_list.mdwn @@ -27,7 +27,6 @@ From Marcus, 2002: * xkb driver for console (for international users) * kbd leds in console (well, in general, Roland's new driver in oskit for that crap) -* fixing fakeroot (it's buggy) * fixing tmpfs (it's buggy, Neal says it's Mach's fault) * adding posix shared memory (requires the io\_close call to be implemented) * adding posix file locking (requires the io\_close call to be implemented) diff --git a/open_issues/subhurd_vs_proc_server.mdwn b/open_issues/subhurd_vs_proc_server.mdwn new file mode 100644 index 00000000..36d150f8 --- /dev/null +++ b/open_issues/subhurd_vs_proc_server.mdwn @@ -0,0 +1,54 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-02-09 + + <youpi> also, can you actually gdb a process of another subhurd? + <braunr> yes + <youpi> but you need to talk to its proc server, don't you? + <braunr> i don't know + <braunr> but i did it several times + <youpi> how? + <braunr> the usual way + <braunr> gdb /path/to/bin pid + <youpi> but which pid? + <braunr> the hard part was finding the right pid + <youpi> well, gdb still needs to talk with the right proc too + <braunr> i don't think it does + <youpi> btw about the "unable to adjust libports thread priority" errors + I'm seeing on the buildd consoles + <braunr> from what i've seen, proc "creates" tasks when it first sees them + too + <youpi> it's about the destination port + <braunr> yes + <braunr> i have those when starting a subhurd too + <youpi> so it would mean that proc somehow got bogus + <youpi> ah + <youpi> so you can actually use your own proc + <braunr> yes + <braunr> and it feels bogus to me + <youpi> and I guess mach lets that proc access the task because your proc + is privileged + <braunr> probably + <braunr> it feels bogus because, you can't rely on pids being allocated per + task + <braunr> what i mean is that, if some tasks spawn and die quickly + <braunr> and you start another application running long enough to see it in + ps + <braunr> it's pid will be +1, not +the number of created tasks + <braunr> which means the proc server will never have seen those previous + tasks + <braunr> it's minor but a bit confusing + <braunr> i personally don't like seeing the tasks of other systems in ps :/ + <braunr> and despite the ability to use gdb from another hurd, i think we + should improve the intra system debugging tools diff --git a/open_issues/syslog.mdwn b/open_issues/syslog.mdwn index 19cba82e..ab32b2e1 100644 --- a/open_issues/syslog.mdwn +++ b/open_issues/syslog.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -13,6 +13,7 @@ License|/fdl]]."]]"""]] [[!toc]] + # IRC, unknown channel, unknown date <tschwinge> scolobb: In wiki edit 60accafa79f645ae61b578403f7fc0c11914b725 @@ -105,3 +106,11 @@ IRC, OFTC, #debian-hurd, 2011-11-02: <antrik> it depends on how many things are actually logged. IIRC the hang happens when some client sends 128 messages to syslog or something like that + + +# IRC, freenode, #hurd, 2013-02-09 + + <pinotree> tschwinge: looks like now you could disable syslog no + <pinotree> ... more + <tschwinge> It that working now? + <pinotree> should be yes, samuel fixed its issue many months ago diff --git a/open_issues/system_stats.mdwn b/open_issues/system_stats.mdwn index 9a13b29a..ce34ec09 100644 --- a/open_issues/system_stats.mdwn +++ b/open_issues/system_stats.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -37,3 +37,9 @@ system statistics, how to interpret them, and some example/expected values. <mcsim> yes <braunr> (a test that fails with the 2G/2G split of the debian kernel, but not on your vanilla version btw) + + +## IRC, frenode, #hurd, 2013-01-26 + + <braunr> ah great, one of the recent fixes (probably select-eintr or + setitimer) fixed exim4 :) diff --git a/open_issues/systemd.mdwn b/open_issues/systemd.mdwn index 1d774307..c23f887f 100644 --- a/open_issues/systemd.mdwn +++ b/open_issues/systemd.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -92,7 +93,16 @@ Likely there's also some other porting needed. <pochu> agreed -# Requires Interfaces +## IRC, freenode, #hurd, 2013-01-18 + + <braunr> systemd relies on linux specific stuff that is difficult to + implement + <braunr> notably cgroups to isolate the deamons it starts so it knows when + they stopped regardless of their pid + <braunr> just assume you can't use systemd on anything else than linux + + +# Required Interfaces In the thread starting [here](http://lists.debian.org/debian-devel/2011/07/threads.html#00269), a diff --git a/open_issues/translators_set_up_by_untrusted_users.mdwn b/open_issues/translators_set_up_by_untrusted_users.mdwn index 1dac130c..521331e9 100644 --- a/open_issues/translators_set_up_by_untrusted_users.mdwn +++ b/open_issues/translators_set_up_by_untrusted_users.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -350,3 +350,230 @@ IRC, freenode, #hurd, 2011-09-14: <antrik> cjuner: either glibc or the parent translators Continued discussion about [[resource_management_problems/pagers]]. + + +# IRC, freenode, #hurd, 2013-02-24 + + <braunr> on a more general topic, i've been thinking about client and + server trust + <braunr> there have been many talkbs about it regarding l4/coyotos/hurdng + <youpi> I generally think the client can trust the server + <braunr> and passing the select timeout to servers corroborates this + <youpi> because it's either root, or it's the same user + <braunr> hum yes, but that's not exactly my question, you'll see + <braunr> there is one feature the hurd has, and i'm not sure we should have + it considering what it requires + <braunr> the feature is that clients can, at any time, "break" from a + server + <youpi> "break" ? + <braunr> the current implementation is to cancel the current RPC after 3 + seconds without a reply when the user sends SIGINT + <braunr> the problem is that, moving to a complete migrating thread model + would make that impossible (or very complicated to do right) + +[[mach_migrating_threads]]. + + <braunr> would it be ok to remove this feature ? + <youpi> well, we need to have SIGINT working, don't we? + <braunr> obviously + <braunr> but that's not what the feature is meant to do + <braunr> it allows clients to recover from a server that misbehaves and + doesn't return + <youpi> then I don't understand in enough details what you mean :) + <braunr> imagine a buggy driver in linux that gets into an uninterruptible + sleep + <braunr> you can't even kill your process + <braunr> that's what the feature is meant to solve + <youpi> that's a quite useful thing + <youpi> e.g. stuck nfs etc., it's useful to be able to recover from that + <braunr> forbidding uninterruptible sleeps would also be a solution, but + then it means relying on servers acting right + <braunr> which is why i mention we usually trust servers + <youpi> well, there is "trust" and "trust" :) + <youpi> i.e. security-wise and robustness-wise + <youpi> I meant clients can usually trust servers security-wise + <braunr> my current idea for x15 is to forbid this kind of breaking, but + also forbid uninterruptible sleeps + <youpi> robustness-wise, I'd say no + <braunr> this way, sending a signal directly reaches the server, which is + trusted to return right away (well, conforming to the handling behaviour) + <braunr> so yes, buggy servers would prevent that, but on the other hand, + stuck nfs wouldn't + <youpi> provided the nfs implementation is not bogus + <braunr> yes + <youpi> I'd tend to agree, but would rather see this discussed on the list + <braunr> yes + <braunr> actually, it wouldn't be that hard to keep the current behaviour, + since i won't implement true migrating threads + <braunr> but it means retaining some issues we have (most importantely, + denial of service) + <braunr> -e + <braunr> what i want to avoid is + http://www.gnu.org/software/hurd/hurd/ng/cancellationforwarding.html + <youpi> for non-trusted servers, we could have a safety wrapper + <youpi> which we trust and does things carefully when talking with the + non-trusted server + <braunr> what would a non trusted server be ? + <youpi> whatever is neither root nor me + <youpi> e.g. nobody-provided /ftp:, or $HOME of another user, etc. + <braunr> i'd argue we don't talk to non trusted servers at all, period + <youpi> users won't like it :) + <braunr> and i'd extend root to a system provided list + <youpi> actually the nobody /ftp: case is important + <braunr> users should be able to create their own list of trusted users + <youpi> it's also the nobody /dev/null case + <youpi> atm it's root + <braunr> yes + <braunr> i see the point + <braunr> i'm just saying the idea of "using non-trusted server" doesn't + make sense + <youpi> actually running /ftp: under nobody is dangerous + <youpi> since if you're running as nobody (because you broke into the + system or whatever), then you can poke with nobody-provided servers + <braunr> yes + <youpi> so we'd rather have really-nobody processes + <braunr> taht's an already existing problem + <youpi> which can't be poked into + <youpi> (and thus can't poke into each other) + <braunr> or a separate user for each + <youpi> that'd be difficult + <braunr> or separate tokens, it's not important + <youpi> for /ftp:/ftp.debian.org used by someone, and /ftp:/ftp.foo.org + used by someone else + <braunr> what i mean is that, by talking to a server, a client implicitely + trusts it + <braunr> youpi: wouldn't that just be the same "ftp" user ? + <youpi> ideally, a carefully-crafted client could avoid having to trust it + <braunr> really ? + <youpi> braunr: that's the problem: then each ftpfs can poke on each other + <braunr> well, each global one + <youpi> there's the daemon-sharing issue too, yes + <braunr> i wasn't thinking about ftpfs, but rather the "system" pfinet for + example + <youpi> like /dev/null is shared + <braunr> when you say root or me, it's "system" or me + <braunr> by default, users trust their system + <braunr> they don't trust other users + <youpi> avoid having to trust it: yes, by using timeouts etc. + <braunr> that's clearly not enough + <youpi> why? + <braunr> shapiro described this in a mail but i can't find it right now + <youpi> I wouldn't like to have to trust ftpfs + <braunr> well time is one thing, data provided for example is another + <braunr> well, you do + <youpi> who knows what bug ftpfs has + <youpi> ideally I would be able not to have to + <youpi> braunr: you can check data + <braunr> i don't think that ideal is possible + <braunr> it you set a ftp translator with a user account, you give it the + password + <youpi> which password? + <braunr> the account password + <youpi> which account? + <braunr> "a user account" + <braunr> i.e. not anonymoius + <youpi> ah + <youpi> well, sure, you have to give that to ftpfs + <youpi> I mean the ftp server might be malicious or whatever + <youpi> and trigger a bug in ftpfs + <braunr> yes + <youpi> so I don't want to have to trust ftpfs + <braunr> what would that mean in practice ? + <youpi> have a trusted translation layer which papers over it, checking + timeouts & data + <braunr> how do you check data ? + <youpi> by knowing the protocol + <braunr> ? + <braunr> can you give a quick example ? + <youpi> well, which data check do you need? + <youpi> (it's you who mentioned data issues :) ) + <braunr> i don't know what you mean by that so, choose as you see fit + <braunr> well the password one for example + <braunr> i was merely saying that, buy using an application, be it a + regular one or a translator, you automatically trust it + <youpi> you mean the ftp user password ? + <braunr> it becomes part of your tcb + <youpi> of course you have to provide it to ftpfs + <youpi> that's not a problem + <youpi> yes, but it's not because you connect to an http website that you + trust the webserver on the other end + <youpi> your web browser does checking for you + <braunr> when the protocol allows it + <braunr> (in this case, i'm thinking assymmetrical cryptography) + <youpi> in which case example doesn't it ? + <youpi> it seems we're not talking about the same kind of issue, thus not + understanding each other + <braunr> indeed + <youpi> by "trusting", I don't mean "be sure that it's the right server at + the other end" + <braunr> my point is that not trusting a server is impossible + <youpi> I mean "it behaves correectly" + <braunr> yes + <braunr> it may not behave correctly, and we might not know it + <youpi> as long as it doesn't make the program crash, that's fine + <youpi> that's what I mean + <braunr> that's where the difference is + <youpi> but giving the password is not my concern here + <youpi> and giving the password is a matter of cryptography, etc. yes, but + that's completely not my point + <braunr> i'm concerned about absolute correct behaviour + <braunr> hm + <braunr> no actually i was + <braunr> but not any more, the discussion has shifted back to the timeout + issue + <braunr> ah no, i remember + <braunr> we talked about which servers to trust + <braunr> and how to alter communication accordingly + <braunr> and my point was that altering communication shouldn't be done, we + either trust the server, and talk to it, or we don't, and we stay away + <braunr> the wrapper would help for this specific blocking issue, yes + <youpi> I don't agree on this + <youpi> let me take a way more simple example + <youpi> a server that provides data through io_read + <youpi> I don't want to trust it because it's provided by some joe user + <youpi> but I still want to have a look at the data that it produces + <youpi> I'm fine that the data may be completely non-sense, that's not a + problem + <youpi> what is a problem, however, is if the hexdump program I've run over + it can't be ^C-ed + <braunr> yes, that's the specific issue i mentioned + <youpi> and for that, a mere trusted intermediate could be enough + <braunr> iirc, there is a firmlink-related issue + <youpi> ? + <braunr> + http://www.gnu.org/software/hurd/open_issues/translators_set_up_by_untrusted_users.html + <youpi> I'm not able to guess what conclusion you are drawing here :) + <braunr> don't talk to untrusted servers + <youpi> or be careful + <youpi> the rm -fr /tmp being aabout being careful actually + <braunr> right + <braunr> i have a very unix-centric view for my system actually + <braunr> i think posix compatibility is very important for me + <braunr> more than it seems to have been in the past when the hurd was + designed + <braunr> to* me + <braunr> so i don't trust tools to be careful + <youpi> that's why a wrapping translator could make it back to posix + compatibility + <braunr> but i see what you mean + <youpi> being careful for the tools + <braunr> hum, a wrapping _translator_ ? + <youpi> yes, similar to remap and fakeroot + <braunr> ok + <youpi> you'd tell it "for this path, please be careful for my tools" + <braunr> ok so + <braunr> it would basically still end up trusting system or me + <braunr> but you'd add this wrapper to the system + <youpi> "it" ? + <braunr> the situation + <braunr> i don't know :) + <braunr> the implementation, whatever + <youpi> the shell I'm running, you mean + <braunr> and it would be the job of this translator to shield the user + <youpi> yes + <braunr> that's a good idea, yes + <youpi> it could reduce the allowed RPC set to what it knows to check + <braunr> how would the shell use it ? + <braunr> would it "shadow" / ? + <youpi> yes + <braunr> ok diff --git a/open_issues/virtualization.mdwn b/open_issues/virtualization.mdwn index 10cf73db..34074c18 100644 --- a/open_issues/virtualization.mdwn +++ b/open_issues/virtualization.mdwn @@ -46,3 +46,5 @@ An index of things to work on w.r.t. virtualization. * [[Networking]] * [[remap_root_translator]] + + * [[fakeroot]] diff --git a/open_issues/virtualization/fakeroot.mdwn b/open_issues/virtualization/fakeroot.mdwn new file mode 100644 index 00000000..ec762b59 --- /dev/null +++ b/open_issues/virtualization/fakeroot.mdwn @@ -0,0 +1,17 @@ +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-02-26 + + <youpi> btw, about fakeroot-hurd + <youpi> the remaining issue I see is with argv[0] (yes, again...) diff --git a/open_issues/virtualization/remap_root_translator.mdwn b/open_issues/virtualization/remap_root_translator.mdwn index 3cb574ae..67d64ae0 100644 --- a/open_issues/virtualization/remap_root_translator.mdwn +++ b/open_issues/virtualization/remap_root_translator.mdwn @@ -95,3 +95,47 @@ License|/fdl]]."]]"""]] <youpi> his own one <braunr> ok <youpi> attached to the remapping + + +## IRC, freenode, #hurd, 2013-01-29 + + <youpi> ok, the remap translator was too easy + <youpi> just took fakeroot.c + <youpi> added if (!strcmp("bin/foo", filename)) filename = + "bin/bash"; in + <youpi> netfs_S_dir_lookup + <youpi> and it just works + <youpi> ok, remap does indeed take my own pfinet + <youpi> good :) + <youpi> pfinet's tun seems to be working too + <youpi> it's however not really flexible, it has to show up in /dev/tunx + <youpi> I'll have a look at fixing that + <youpi> yep, works fine + + +## IRC, freenode, #hurd, 2013-02-01 + + <youpi> braunr: as I expected, simply passing FS_RETRY_REAUTH does the + remapping trick + + +# IRC, freenode, #hurd, 2013-02-12 + + <braunr> + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/server_overriding/ + <braunr> youpi: isn't that your remap translator ? + <youpi> completely + <youpi> remap being (5) + + +# IRC, freenode, #hurd, 2013-02-25 + + <youpi> I'm just having an issue with getcwd getting in the sky + <youpi> I wonder whether libc might need patching to understand it's in + some sort of chroot + <youpi> or perhaps remap fixed into avoiding .. of / being odd + <youpi> erf, it's actually an explicit error + <youpi> libc just doesn't want to have a ".." / being different from CRDIR + <youpi> let me just comment out that :) + <youpi> way better :) + <youpi> yep, just works fine diff --git a/open_issues/vm_map_kernel_bug.mdwn b/open_issues/vm_map_kernel_bug.mdwn index 613c1317..159e9d04 100644 --- a/open_issues/vm_map_kernel_bug.mdwn +++ b/open_issues/vm_map_kernel_bug.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -52,3 +52,20 @@ License|/fdl]]."]]"""]] <braunr> it could be that gnumach isn't good at aligning to large values [[!message-id "87fw4pb4c7.fsf@kepler.schwinge.homeip.net"]] + + +# IRC, frenode, #hurd, 2013-01-22 + +In context of [[libpthread]]. + + <braunr> pinotree: do you understand what the fmh function does in + sysdeps/mach/hurd/dl-sysdep.c ? + <braunr> ok i understand what it does + <braunr> and youpi has changed the code, so he does too + <braunr> youpi: do you have a suggestion about how to solve this issue in + the fmh function ? + <youpi> do we remember which bug it's after? + <braunr> what do you mean ? + <braunr> ah + <braunr> no :/ + <braunr> it could be a good occasion to get rid of it, yes |