From 12c341b917921eb631026ec44a284c4d884e5de6 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 6 Mar 2013 21:52:20 +0100 Subject: IRC. --- open_issues/select.mdwn | 458 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 428 insertions(+), 30 deletions(-) (limited to 'open_issues/select.mdwn') diff --git a/open_issues/select.mdwn b/open_issues/select.mdwn index 391509a9..caecc437 100644 --- a/open_issues/select.mdwn +++ b/open_issues/select.mdwn @@ -215,7 +215,7 @@ IRC, unknown channel, unknown date: it's better than nothing yes -# IRC, freenode, #hurd, 2012-07-21 +### IRC, freenode, #hurd, 2012-07-21 damn, select is actually completely misdesigned :/ iiuc, it makes servers *block*, in turn :/ @@ -315,7 +315,7 @@ IRC, unknown channel, unknown date: really easy and nice :-) -## IRC, freenode, #hurd, 2012-07-22 +#### IRC, freenode, #hurd, 2012-07-22 antrik: you can't block in servers with sync ipc so in this case, "select" becomes a request for notifications @@ -379,7 +379,7 @@ IRC, unknown channel, unknown date: his reasoning (as does braunr) -## IRC, freenode, #hurd, 2012-07-23 +#### IRC, freenode, #hurd, 2012-07-23 antrik: i was meaning sync in the most common meaning, yes, the client blocking on the reply @@ -650,6 +650,9 @@ IRC, unknown channel, unknown date: which is why i could choose time_value_t (a struct of 2 integer_t) well, I'd say gnumach could grow a nanosecond-precision time value e.g. for clock_gettime precision and such + +[[clock_gettime]]. + so you would prefer me adding the time_spec_t time to gnumach rather than the hurd ? well, if hurd RPCs are using mach types and there's no mach type @@ -782,7 +785,7 @@ IRC, unknown channel, unknown date: API definition at RPC level too -## IRC, freenode, #hurd, 2012-07-24 +#### IRC, freenode, #hurd, 2012-07-24 youpi: antrik: is vm_size_t an appropriate type for a c long ? (appropriate mig type) @@ -809,7 +812,7 @@ IRC, unknown channel, unknown date: continue -## IRC, freenode, #hurd, 2012-07-25 +#### IRC, freenode, #hurd, 2012-07-25 braunr: well, for actual kernel calls, machine-specific types are probably hard to avoid... the problem is when they are used in other RPCs @@ -900,7 +903,7 @@ IRC, unknown channel, unknown date: antrik: ah about that, ok -## IRC, freenode, #hurd, 2012-07-26 +#### IRC, freenode, #hurd, 2012-07-26 braunr: wrt your select_timeout branch, why not push only the time_data stuff to master? @@ -914,7 +917,7 @@ IRC, unknown channel, unknown date: i "only" have to adjust the client side select implementation now -## IRC, freenode, #hurd, 2012-07-27 +#### IRC, freenode, #hurd, 2012-07-27 io_select should remain a routine (i.e. synchronous) for server side stub code @@ -922,7 +925,14 @@ IRC, unknown channel, unknown date: (since _hurs_select manually handles replies through a port set) -## IRC, freenode, #hurd, 2012-07-28 +##### IRC, freenode, #hurd, 2013-02-09 + + io_select becomes a simpleroutine, except inside the hurd, where + it's a routine to keep the receive and reply mig stub code + (the server side) + + +#### IRC, freenode, #hurd, 2012-07-28 why are there both REPLY_PORTS and IO_SELECT_REPLY_PORT macros in the hurd .. @@ -941,7 +951,7 @@ IRC, unknown channel, unknown date: i did something a bit ugly but it seems to do what i wanted -## IRC, freenode, #hurd, 2012-07-29 +#### IRC, freenode, #hurd, 2012-07-29 good, i have a working client-side select now i need to fix the servers a bit :x @@ -998,7 +1008,7 @@ IRC, unknown channel, unknown date: queue ... -## IRC, freenode, #hurd, 2012-07-30 +#### IRC, freenode, #hurd, 2012-07-30 hm nice, the problem i have with my hurd_condition_timedwait seems to also exist in libpthread @@ -1096,7 +1106,7 @@ IRC, unknown channel, unknown date: (http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?h=rbraun/select_timeout&id=40fe717ba9093c0c893d9ea44673e46a6f9e0c7d) -## IRC, freenode, #hurd, 2012-08-01 +#### IRC, freenode, #hurd, 2012-08-01 damn, i can't manage to make threads calling condition_wait to dequeue themselves from the condition queue :( @@ -1132,7 +1142,7 @@ IRC, unknown channel, unknown date: it frightens me because i don't see any flaw in the logic :( -## IRC, freenode, #hurd, 2012-08-02 +#### IRC, freenode, #hurd, 2012-08-02 ah, seems i found a reliable workaround to my deadlock issue, and more than a workaround, it should increase efficiency by reducing @@ -1150,7 +1160,7 @@ IRC, unknown channel, unknown date: (/etc/hurd/runsystem i assume) -## IRC, freenode, #hurd, 2012-08-03 +#### IRC, freenode, #hurd, 2012-08-03 glibc actually makes some direct use of cthreads condition variables @@ -1302,7 +1312,7 @@ IRC, unknown channel, unknown date: tests, reviews, more tests, polishing, commits, packaging -## IRC, freenode, #hurd, 2012-08-04 +#### IRC, freenode, #hurd, 2012-08-04 grmbl, apt-get fails on select in my subhurd with the updated glibc @@ -1324,7 +1334,7 @@ IRC, unknown channel, unknown date: and thomas d -## IRC, freenode, #hurd, 2012-08-05 +#### IRC, freenode, #hurd, 2012-08-05 eh, i made dpkg-buildpackage use the patched c library, and it finished the build oO @@ -1352,7 +1362,7 @@ IRC, unknown channel, unknown date: extremely large -## IRC, freenode, #hurd, 2012-08-06 +#### IRC, freenode, #hurd, 2012-08-06 i have bad news :( it seems there can be memory corruptions with my io_select patch @@ -1395,7 +1405,7 @@ IRC, unknown channel, unknown date: [[libpthread]]. -## IRC, freenode, #hurd, 2012-08-07 +#### IRC, freenode, #hurd, 2012-08-07 anyone knows of applications extensively using non-blocking networking functions ? @@ -1470,7 +1480,7 @@ IRC, unknown channel, unknown date: other for the servers, eh) -## IRC, freenode, #hurd, 2012-08-07 +#### IRC, freenode, #hurd, 2012-08-07 when running gitk on [darnassus], yesterday, i could push the CPU to 100% by simply moving the mouse in the window :p @@ -1490,7 +1500,7 @@ IRC, unknown channel, unknown date: this linear search on dequeue is a real pain :/ -## IRC, freenode, #hurd, 2012-08-09 +#### IRC, freenode, #hurd, 2012-08-09 `screen` doesn't close a window/hangs after exiting the shell. @@ -1503,7 +1513,7 @@ IRC, unknown channel, unknown date: [[Term_blocking]]. -# IRC, freenode, #hurd, 2012-12-05 +### IRC, freenode, #hurd, 2012-12-05 well if i'm unable to build my own packages, i'll send you the one line patch i wrote that fixes select/poll for the case where there is @@ -1512,7 +1522,7 @@ IRC, unknown channel, unknown date: timeout, doubling the total wait time when there is no event) -## IRC, freenode, #hurd, 2012-12-06 +#### IRC, freenode, #hurd, 2012-12-06 damn, my eglibc patch breaks select :x i guess i'll just simplify the code by using the same path for @@ -1546,12 +1556,12 @@ IRC, unknown channel, unknown date: this can account for the slowness of a bunch of select/poll users -## IRC, freenode, #hurd, 2012-12-07 +#### IRC, freenode, #hurd, 2012-12-07 finally, my select patch works :) -## IRC, freenode, #hurd, 2012-12-08 +#### IRC, freenode, #hurd, 2012-12-08 for those interested, i pushed my eglibc packages that include this little select/poll timeout fix on my debian repository @@ -1560,7 +1570,7 @@ IRC, unknown channel, unknown date: regressions -## IRC, freenode, #hurd, 2012-12-10 +#### IRC, freenode, #hurd, 2012-12-10 I have verified your double timeout bug in hurdselect.c. Since I'm also working on hurdselect I have a few questions @@ -1631,7 +1641,13 @@ IRC, unknown channel, unknown date: i'll try the non intrusive mode -## IRC, freenode, #hurd, 2012-12-11 +##### IRC, freenode, #hurd, 2013-01-26 + + ah great, one of the recent fixes (probably select-eintr or + setitimer) fixed exim4 :) + + +#### IRC, freenode, #hurd, 2012-12-11 braunr: What is the technical difference of having the delay at io_select compared to mach_msg for one FD? @@ -1641,7 +1657,7 @@ IRC, unknown channel, unknown date: (for L4 guys it wouldn't be considered a slight optimization :)) -## IRC, freenode, #hurd, 2012-12-17 +#### IRC, freenode, #hurd, 2012-12-17 tschwinge: http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/select_timeout_for_one_fd @@ -1668,20 +1684,20 @@ IRC, unknown channel, unknown date: notifications resulting from the way io_select works -## IRC, freenode, #hurd, 2012-12-19 +#### IRC, freenode, #hurd, 2012-12-19 tschwinge: i've tested the glibc rbraun/select_timeout_for_one_fd branch for a few days on darnassus now, and nothing wrong to report -## IRC, freenode, #hurd, 2012-12-20 +#### IRC, freenode, #hurd, 2012-12-20 braunr: so, shall I commit the single hurd select timeout fix to the debian package? youpi: i'd say so yes -## IRC, freenode, #hurd, 2013-01-03 +#### IRC, freenode, #hurd, 2013-01-03 gnu_srs: sorry, i don't understand your poll_timeout patch it basically reverts mine for poll only @@ -1848,7 +1864,7 @@ IRC, unknown channel, unknown date: to the poll stuff: Have to check further with my poll patch... -## IRC, freenode, #hurd, 2013-01-04 +#### IRC, freenode, #hurd, 2013-01-04 Summary of the eglibc-2.13-38 issues: without the unsubmitted-setitimer_fix.diff patch and with @@ -2037,6 +2053,388 @@ IRC, unknown channel, unknown date: See also [[alarm_setitimer]]. +#### IRC, freenode, #hurd, 2013-01-22 + + youpi: Maybe it's overkill to have a separate case for DELAY; but + it enhances readability (and simplifies a lot too) + but it reduces factorization + if select is already supposed to behave the same way as delay, + there is no need for a separate code + OK; I'll make a two-way split then. What about POLL and nfds=0, + timeout !=0? + gnu_srs: handle nfds=0 as a pure timeout as the linux man page + describes + it makes sense, and as other popular systems do it, it's better to + do it the same way + and i disagree with you, factorization doesn't imply less + readability + So you agree with me to have a special case for DELAY? + Coding style is a matter of taste: for me case a: case b: etc is + more readable than "if then elseif then else ..." + it's not coding style + avoiding duplication is almost always best + whatever the style + i don't see the need for a special delay case + it's the same mach_msg call + (for now) + gnu_srs: i'd say the only reason to duplicate is when you can't do + otherwise + ways of coding then... And I agree with the idea of avoiding code + duplication, ever heard of Literate Programming + we'll need a "special case" when the timeout is handled at the + server side, but it's like two lines .. + + +#### IRC, freenode, #hurd, 2013-02-11 + + braunr: the libpthread hurd_cond_timedwait_np looks good to me + + +##### IRC, freenode, #hurd, 2013-02-15 + + braunr: does cond_timedwait_np depend on the cancellation fix? + yes + ok + the timeout fix + so I also have to pull that into my glibc build + (i fixed cancellation too because the cleanup routine had to be + adjusted anyway + ) + ah, and I need the patches hurd package too + if unsure, you can check my packages + ok, not for tonight then + i listed the additional patches in the changelog + yep, I'll probably use them + + +#### IRC, freenode, #hurd, 2013-02-11 + + braunr: I don't understand one change in glibc: + - err = __io_select (d[i].io_port, d[i].reply_port, 0, &type); + + err = __io_select (d[i].io_port, d[i].reply_port, type); + youpi: the waittime parameter ahs been removed + has* + where? when? + in the hurd branch + in the defs? + yes + I don't see this change + only the addition of io_select_timeout + hum + also, io_select_timeout should be documented along io_select in + hurd.texi + be6e5b86bdb9055b01ab929cb6b6eec49521ef93 + Selectively compile io_select{,_timeout} as a routine + * hurd/io.defs (io_select_timeout): Declare as a routine if + _HURD_IO_SELECT_ROUTINE is defined, or a simpleroutine + otherwise. + (io_select): Likewise. In addition, remove the waittime + timeout parameter. + ah, it's in another commit + yes, perhaps misplaced + that's the kind of thing i want to polish + my main issue currently is that time_data_t is passed by value + i'm trying to pass it by address + I don't know the details of routine vs simpleroutine + it made sense for me to remove the waittime parameter at the same + time as adding the _HURD_IO_SELECT_ROUTINE macro, since waittime is what + allows glibc to use a synchronous RPC in an asynchronous way + is it only a matter of timeout parameter? + simpleroutine sends a message + routine sends and receives + by having a waittime parameter, _hurd_select could make io_select + send a message and return before having a reply + ah, that's why in glibc you replaced MACH_RCV_TIMED_OUT by 0 + yes + it seems a bit odd to have a two-face call + it is + can't we just keep it as such? + no + damn + well we could, but it really wouldn't make any sense + why not? + because the way select is implemented implies io_select doesn't + expect a reply + (except for the single df case but that's an optimization) + fd* + that's how it is already, yes? + yes + well yes and no + that's complicated :) + there are two passes + let me check before saying anything ;p + :) + in the io_select(timeout=0) case, can it ever happen that we + receive an answer? + i don't think it is + you mean non blocking right ? + not infinite timeout + I mean calling io_select with the timeout parameter being set to 0 + so yes, non blocking + no, i think we always get MACH_RCV_TIMED_OUT + for me non-blocking can mean a lot of things :) + ok + i was thinking mach_msg here + ok so, let's not consider the single fd case + the first pass simply calls io_select with a timeout 0 to send + messages + I don't think it's useful to try to optimize it + it'd only lead to bugs :) + me neither + yes + (as was shown :) ) + what seems useful to me however is to optimize the io_select call + with a waittime parameter, the generated code is an RPC (send | + receive) + whereas, as a simpleroutine, it becomes a simple send + ok + my concern is that, as you change it, you change the API of the + __io_select() function + (from libhurduser) + yes but glibc is the only user + and actually no + i mean + i change the api at the client side only + that's what I mean + remember that io.Defs is almost full + "full" ? + i'm almost certain it becomes full with io_select_timeout + there is a practical limit of 100 calls per interface iirc + since the reply identifiers are request + 100 + are we at it already? + i remember i had problems with it so probably + but anyway, I'm not thinking about introducing yet another RPC + but get a reasonable state of io_select + i'l have to check that limit + it looks wrong now + or was it 50 + i don't remember :/ + i understand + but what i can guarantee is that, while the api changes at the + client side, it doesn't at the server side + ideally, the client api of io_select could be left as it is, and + libc use it as a simpleroutine + sure, I understand that + which means glibc, whether patched or not, still works fine with + that call + yes it could + that's merely a performance optimization + my concern is that an API depends on the presence of + _HURD_IO_SELECT_ROUTINE, and backward compatibility being brought by + defining it! :) + yes + i personally don't mind much + I'd rather avoid the clutter + what do you mean ? + anything that avoids this situation + like just using timeout = 0 + well, in that case, we'll have both a useless timeout at the + client side + and another call for truely passing a timeout + that's also weird + how so a useless timeout at the client side? + 22:39 < youpi> - err = __io_select (d[i].io_port, d[i].reply_port, + 0, &type); + 0 here is the waittime parameter + that's a 0-timeout + and it will have to be 0 + yes + that's confusing + ah, you mean the two io_select calls? + yes + but isn't that necessary for the several-fd case, anyway? + ? + if the io_select calls are simple routines, this useless waittime + parameter can just be omitted like i did + don't we *have* to make several calls when we select on several + fds? + suure but i don't see how it's related + well then I don't see what optimization you are doing then + except dropping a parameter + which does not bring much to my standard :) + a simpleroutine makes mach_msg take a much shorter path + that the 0-timeout doesn't take? + yes + it's a send | receive + ok, but that's why I asked before + so there are a bunch of additional checks until the timeout is + handled + whether timeout=0 means we can't get a receive + and thus the kernel could optimize + that's not the same thing :) + ok + it's a longer path to the same result + I'd really rather see glibc building its own private simpleroutine + version of io_select + iirc we already have such kind of thing + ok + well there are io_request and io_reply defs + but i haven't seen them used anywhere + but agreed, we should do that + braunr: the prototype for io_select seems bogus in the io_request, + id_tag is no more since ages :) + youpi: yes + youpi: i'll recreate my hurd branch with only one commit + without the routine/simpleroutine hack + and with time_data_t passed by address + and perhaps other very minor changes + braunr: the firstfd == -1 test needs a comment + or better, i'll create a v2 branch to make it easy to compare them + ok + braunr: actually it's also the other branch of the if which needs a + comment: "we rely on servers implementing the timeout" + youpi: ok + - (msg.success.result & SELECT_ALL) == 0) + why removing that test? + you also need to document the difference between got and ready + hm i'll have to remember + i wrote this code like a year ago :) + almost + AIUI, got is the number of replies + but i think it has to do with error handling + and + + if (d[i].type) + + ++ready; + while ready is the number of successful replies + is what replaces it + youpi: yes + the poll wrapper already normalizes the timeout parameter to + _hurd_select + no you probably don't + the whole point of the patch is to remove this ugly hack + youpi: ok so + 23:24 < youpi> - (msg.success.result & SELECT_ALL) + == 0) + when a request times out + ah, right + we could get a result with no event + and no error + and this is what makes got != ready + tell that to the source, not me :) + sure :) + i'm also saying it to myself + ... :) + right, using io_select_request() is only an optimization, which we + can do later + what i currently do is remove the waittime parameter from + io_select + what we'll do instead (soon) is let the parameter there to keep + the API unchancged + but always use a waittime of 0 + to make the mach_msg call non blocking + then we'll try to get the io_request/io_reply definitions back so + we can have simpleroutines (send only) version of the io RPCs + and we'll use io_select_request (without a waittime) + youpi: is that what you understood too ? + yes + (and we can do that later) + gnu_srs: does it make more sense for you ? + this change is quite sparsed so it's not easy to get the big + picture + sparse* + it requires changes in libpthread, the hurd, and glibc + the libpthread change can be almost forgotten + it's just yet another cond_foo function :) + well not if he's building his own packages + right + ok, apart from the io_select_request() and documenting the newer + io_select_timeout(), the changes seem good to me + youpi: actually, a send | timeout takes the slow path in mach_msg + and i actually wonder if send | receive | timeout = 0 can get a + valid reply from the server + but the select code already handles that so it shouldn't be much + of a problem + k + + +##### IRC, freenode, #hurd, 2013-02-12 + + hum + io_select_timeout actually has to be a simpleroutine at the client + side :/ + grmbl + ah? + otherwise it blocks + how so? + routines wait for replies + even with timeout 0? + there is no waittime for io_select_timeout + adding one would be really weird + oh, sorry, I thought you were talking about io_select + it would be more interesting to directly use + io_select_timeout_request + but this means additional and separate work to make the + request/reply defs up to date + and used + personally i don't mind, but it matters for wheezy + youpi: i suppose it's not difficult to add .defs to glibc, is it ? + i mean, make glibc build the stub code + it's probably not difficult indeed + ok then it's better to do that first + yes + there's faultexec for instance in hurd/Makefile + ok + or rather, apparently it'd be simply user-interfaces + it'll probably be linked into libhurduser + but with an odd-enough name it shouldn't matter + youpi: adding io_request to the list does indeed build the RPCs :) + i'll write a patch to sync io/io_reply/io_request + youpi: oh by the way, i'm having a small issue with the + io_{reply,request} interfaces + the generated headers both share the same enclosing macro + (_io_user) + so i'm getting compiler warning + s + we could fix that quickly in mig, couldn't we? + youpi: i suppose, yes, just mentioning + + +##### IRC, freenode, #hurd, 2013-02-19 + + in the hurdselect.c code, I'd rather see it td[0]. rather than + td-> + ok + otherwise it's frownprone + (it has just made me frown :) ) + yes, that looked odd to me too, but at the same time, i didn't + want it to seem to contain several elements + I prefer it to look like there could be several elements (and then + the reader has to find out how many, i.e. 1), rather than it to look like + the pointer is not initialized + right + I'll also rather move that code further + so the preparation can set timeout to 0 + (needed for poll) + how about turning your branch into a tg branch? + feel free to add your modifications on top of it + sure + ok + I'll handle these then + youpi: i made an updated changelog entry in the + io_select_timeout_v3 branch + could you rather commit that to the t/io_select_timeout branch I've + just created? + i mean, i did that a few days ago + (in the .topmsg file) + ah + k + + +##### IRC, freenode, #hurd, 2013-02-26 + + youpi: i've just pushed a rbraun/select_timeout_pthread_v4 branch + in the hurd repository that includes the changes we discussed yesterday + untested, but easy to compare with the previous version + + +##### IRC, freenode, #hurd, 2013-02-27 + + braunr: io_select_timeout seems to be working fine here + braunr: I feel like uploading them to debian-ports, what do you + think? + youpi: the packages i rebuild last night work fine too + + # See Also See also [[select_bogus_fd]] and [[select_vs_signals]]. -- cgit v1.2.3