From 6c7d45e4631784d0e077e806521a736da6b0266e Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 7 Apr 2013 18:18:44 +0200 Subject: IRC. --- community/gsoc/project_ideas/smp.mdwn | 11 ++ faq/smp.mdwn | 5 +- hurd.mdwn | 1 + hurd/debugging/rpctrace.mdwn | 8 +- hurd/libihash.mdwn | 17 ++- hurd/libps.mdwn | 27 ++++ hurd/translator/ext2fs.mdwn | 31 +++++ hurd/translator/pfinet/implementation.mdwn | 51 +++++++- hurd/translator/pfinet/ipv6.mdwn | 14 +++ hurd/translator/tarfs.mdwn | 17 ++- microkernel/mach/deficiencies.mdwn | 136 +++++++++++++++++++++ open_issues/anatomy_of_a_hurd_system.mdwn | 32 ++++- open_issues/clock_gettime.mdwn | 9 ++ open_issues/glibc/debian/experimental.mdwn | 115 +++++++++++++++++ open_issues/libpthread.mdwn | 13 ++ open_issues/linux_vmsig.mdwn | 17 ++- open_issues/managed_runtime_initiative.mdwn | 72 +++++++++++ open_issues/multithreading.mdwn | 27 ++++ open_issues/nice_vs_mach_thread_priorities.mdwn | 40 ++++++ open_issues/performance/io_system/read-ahead.mdwn | 42 +++++++ open_issues/pfinet_timers.mdwn | 102 ++++++++++++++++ open_issues/time.mdwn | 75 +++++++++++- open_issues/translate_fd_or_port_to_file_name.mdwn | 70 +++++++++-- open_issues/virtualization/fakeroot.mdwn | 7 ++ .../virtualization/remap_root_translator.mdwn | 5 + 25 files changed, 918 insertions(+), 26 deletions(-) create mode 100644 hurd/libps.mdwn create mode 100644 open_issues/glibc/debian/experimental.mdwn create mode 100644 open_issues/managed_runtime_initiative.mdwn diff --git a/community/gsoc/project_ideas/smp.mdwn b/community/gsoc/project_ideas/smp.mdwn index d9788e56..c6da2c41 100644 --- a/community/gsoc/project_ideas/smp.mdwn +++ b/community/gsoc/project_ideas/smp.mdwn @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!meta title="SMP"]] +See also the [[FAQ entry|faq/smp]]. + # IRC, freenode, #hurd, 2012-09-30 @@ -19,3 +21,12 @@ License|/fdl]]."]]"""]] ## IRC, freenode, #hurd, 2013-01-02 i'd like to mentor someone for adding smp to gnumach + + +## IRC, freenode, #hurd, 2013-03-14 + + but i'm afraid we'll have to fight obscur smp-safety issues + for one, drivers are much probably not smp safe and would require + a big kernel lock + userspace (such as signal delivery in libc) might be affected too + smp isn't that easy diff --git a/faq/smp.mdwn b/faq/smp.mdwn index e95edcd2..04396401 100644 --- a/faq/smp.mdwn +++ b/faq/smp.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -26,3 +27,5 @@ That is why for now GNU/Hurd will only use one logical processor (i.e. one core Once this issue is solved, there are follow-up issues about [[open_issues/multiprocessing]] and [[open_issues/multithreading]]. + +[[GSoC project idea|gsoc/project_ideas/smp]]. diff --git a/hurd.mdwn b/hurd.mdwn index d5dd7e4f..10276325 100644 --- a/hurd.mdwn +++ b/hurd.mdwn @@ -94,6 +94,7 @@ in the *unstable* branch of the Debian archive. * [[libihash]] * [[libthreads]] * [[libpthread]] + * [[libps]] * [[IO_Path]] * [[Porting]] * [[Debugging]] diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn index c506861a..de72ae4c 100644 --- a/hurd/debugging/rpctrace.mdwn +++ b/hurd/debugging/rpctrace.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2010, 2011, 2012 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2010, 2011, 2012, 2013 Free +Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -175,6 +175,10 @@ See `rpctrace --help` about how to use it. /home/rbraun/hd0s7/hurd/hurd-20120710/./utils/rpctrace.c:1287: trace_and_forward: Assertion `reply_type == 18' failed. + * IRC, OFTC, #debian-hurd, 2013-03-14 + + uhu, there's a TODO just above that assertion :) + # See Also diff --git a/hurd/libihash.mdwn b/hurd/libihash.mdwn index d6b8e8b6..be20fa58 100644 --- a/hurd/libihash.mdwn +++ b/hurd/libihash.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2010, 2011, 2012 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2009, 2010, 2011, 2012, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -19,19 +19,26 @@ License|/fdl]]."]]"""]] * hurd-l4 libhurd-ihash + +# Open Issues + * [[viengoos libhurd-ihash|microkernel/viengoos/projects/new_hash_function]] - IRC, unknown channel, unknown date + IRC, freenode, #hurd, 2008/2009 so, we need a new ihash implementation marcusb: When 80% full, the collision rate is very high. marcusb: I tested using 512mb / 4096 entries - marcusb: Changing the load factor to 30% resulted in my program running more than an order of magnitude faster. + marcusb: Changing the load factor to 30% resulted in my program + running more than an order of magnitude faster. yeah, it shouldn't get so full don't we do an exponential back-off in the array size? of course it's clear we can do much better the ihash algo is very simple - I'm not even sure it makes much sense to have a generic library + I'm not even sure it makes much sense to have a generic + library + + * [[community/gsoc/project_ideas/object_lookups]] # Alternatives? diff --git a/hurd/libps.mdwn b/hurd/libps.mdwn new file mode 100644 index 00000000..05ab3ad1 --- /dev/null +++ b/hurd/libps.mdwn @@ -0,0 +1,27 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +`libps` provides an API for accessing process information. An alternative is a +[[Linux-compatible `/proc` filesystem|translator/procfs]]. + + +# IRC, OFTC, #debian-hurd, 2013-03-09 + +[[!tag open_issue_documentation]] + + what's the difference between PSTAT_STATE_P_STOP and + PSTAT_STATE_T_HALT in libps/ps.h ? + Steap: from reading the gnumach source, it seems halted threads are + thread which have finished they work but not finished terminating yet + while STOP is the standard preempted state + i.e. not sleeping, but have handed cpu to another thread + youpi: thanks. Is this "halted" state similar to anything in Linux + ? + TASK_DEAD, probably diff --git a/hurd/translator/ext2fs.mdwn b/hurd/translator/ext2fs.mdwn index 65361ff4..b276ff0c 100644 --- a/hurd/translator/ext2fs.mdwn +++ b/hurd/translator/ext2fs.mdwn @@ -89,6 +89,37 @@ small backend stores, like floppy devices. which can be quite probable +##### IRC, freenode, #hurd, 2013-03-19 + + youpi: i'm back on polishing the large store patch + did you remember seeing something else than the bzero/memset + out-of-scope changes ? + (i mean, readily noticeable) + I don't remember + ok + the original code already assumes mmap succeeds + is it ok to consider the patch can do the same ? + I'd say so + ok + youpi: actually, it looks like a good part of the patch isn't + related to large stores + for example, in ext2fs/inode.c, there are calls to + dino_ref/dino_deref + hum + i'm not sure at all these have anything to do with handling large + stores + but dino_ref is introduced by this patch, isn't it? + it replaces dino + yes, it replaces it because the dino() approach can't work beyond + 2GiB + i see + youpi: i'd like to replace the recursive call to + disk_cache_block_ref with a goto, is that fine with you ? + looks ok to me + better than relying on tail recursion + that's the idea :) + + #### [[libpager]] API change ##### IRC, freenode, #hurd, 2013-03-04 diff --git a/hurd/translator/pfinet/implementation.mdwn b/hurd/translator/pfinet/implementation.mdwn index 50b5dfc2..3232e0cc 100644 --- a/hurd/translator/pfinet/implementation.mdwn +++ b/hurd/translator/pfinet/implementation.mdwn @@ -1,13 +1,58 @@ -[[!meta copyright="Copyright © 2000, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2000, 2008, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] The `pfinet` server is a hacked Linux internet implementation with a glue layer translating between the Hurd [[RPC]]s and the middle layer of the Linux implementation. + + +# Bugs + +## Those Listed on [[Open_Issues]] + +## [[IPv6]] + +## IRC, freenode, #hurd, 2013-04-03 + + youpi: there are indeed historical bugs with small packets and + tcp_nodelay in linux 2.0/2.2 tcp/ip + oh + http://jl-icase.home.comcast.net/~jl-icase/LinuxTCP2.html + + +# Reimplementation, [[!GNU_Savannah_task 5469]] + +## IRC, freenode, #hurd, 2013-04-03 + +[[!tag open_issue_hurd]] + + I was thinking about just using liblwip this afternoon, btw + what is it ? + hm, why not + i would still prefer using code from netbsd + especially now with the rump kernel project making it even easier + well, whatever is easy to maintain up to date actually + netbsd's focus on general portability normally makes it easy to + maintain + the author of the rump project was able to make netbsd code run in + browsers :) + and he actually showed clients using the networking stack on + windows, remotely (not in the same process) + so that's very close to what we want + indeed + though liblwip is exactly the same portability focus :) + apparently, for embedded systems + but bsd's code is probably better + yes + i think so, more general purpose, larger user base + I used it for the stubdomains in Xen + (it = lwip) + ok diff --git a/hurd/translator/pfinet/ipv6.mdwn b/hurd/translator/pfinet/ipv6.mdwn index 42ee3c55..824cc733 100644 --- a/hurd/translator/pfinet/ipv6.mdwn +++ b/hurd/translator/pfinet/ipv6.mdwn @@ -158,3 +158,17 @@ Amongst other things, support for [[IOCTL]]s is missing. I don't remember its url git://git.kernel.org/pub/scm/linux/kernel/git/tglx/history.git might be even older + + +## IRC, freenode, #hurd, 2013-04-01 + + hi, the ipv6 route works or not? + I can use ipv6 to connect vms, but the vm can't access other + host except the router + the ipv6 route -G seems not work for me, anyone make it work? + I'm using SLAAC + youpi: you mean static config not work? + I mean automatic config does work + I don't use static config myself + youpi: My ipv6 works now, just because not fully enable ipv6 + forwarding and firewall, thank you:) diff --git a/hurd/translator/tarfs.mdwn b/hurd/translator/tarfs.mdwn index e25e3255..2e37d9cf 100644 --- a/hurd/translator/tarfs.mdwn +++ b/hurd/translator/tarfs.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -20,6 +20,21 @@ Works fine on most cases, occasional corruptions when writing using bzip2/gzip stores. +## IRC, freenode, #hurd, 2013-03-16 + +[[!tag open_issue_hurd]] + + hm is tarfs working? its simple test suite fails here + (translator dies with gz and bz2 compression) + no idea + have you tried without compression ? + that succeeds + it might rely on storeio for compression + no, it does gz/bz2 on its own + but it uses libstore for i/o of the file + ok + + # Source incubator, tarfs/master diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index b3e1758c..dcabb56e 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -615,3 +615,139 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. way it is in mach and netbsd but the arch-specific interfaces aren't well defined yet because there is only one (and incomplete) arch + + +### IRC, freenode, #hurd, 2013-03-08 + + BTW, what is your current direction? did you follow through with + abandonning Mach resemblance?... + no + it's very similar to mach in many ways + unless mach is defined by its ipc in which case it's not mach at + all + the ipc interface will be similar to the qnx one + well, Mach is pretty much defined by it's IPC and VM interface... + the vm interface remains + its + although vm maps will be first class objects + so that it will be possible to move parts of the vm server outside + the kernel some day if it feels like a good thing to do + i.e. vm maps won't be inferred from tasks + not implicitely + the kernel will be restricted to scheduling, memory management, + and ipc, much as mach is (notwithstanding drivers) + hm... going with QNX IPC still seems risky to me... it's designed + for simple embedded environments, not for general-purpose operating + systems in my understanding + no, the qnx ipc interface is very generic + they can already call remote services + the system can scale well on multiprocessor machines + that's not risky at all, on the contrary + yeah, I'm sure it's generic... but I don't think anybody tried to + build a Hurd-like system on top of it; so it's not at all clear whether + it will work out at all... + clueless question: does x15 have any inspiration from + helenos? + absolutely none + i'd say x15 is almost an opposite to helenos + it's meant as a foundation for unix systems, like mach + some unix interfaces considered insane by helenos people (such as + fork and signals) will be implemented (although not completely in the + kernel) + ipc will be mostly synchronous + they're very different + well, helenos is very different + cool + x15 and actually propel (the current name i have for the final + system), are meant to create a hurd clone + another clueless question: any similarities of x15 to minix? + and since we're few, implementing posix efficiently is a priority + goal for me + again, absolutely none + for the same reasons + minix targets resilience in embedded environments + propel is a hurd clone + propel aims at being a very scalable and performant hurd clone + that's all + neato + unfortunately, i couldn't find a name retaining all the cool + properties of the hurd + feel free to suggest ideas :) + propel? as in to launch forward? + push forward, yes + that's very likely a better name than anything i could + conjure up + x15 is named after mach (the first aircraft to break mach 4, + reaching a bit less than mach 7) + servers will be engines, and together to push the system forward + ..... :) + nice + thrust might be a bit too generic i guess + oh i'm looking for something like "hurd" + doubly recursive acronym, related to gnu + and short, so it can be used as a c namespace + antrik: i've thought about it a lot, and i'm convinced this kind + of interface is fine for a hurd like system + the various discussions i found about the hurd requirements + (remember roland talking about notifications) all went in this direction + note however the interface isn't completely synchronous + and that's very important + well, I'm certainly curious. but if you are serious about this, + you'd better start building a prototype as soon as possible, rather than + perfecting SMP ;-) + i'm not perfecting smp + but i consider it very important to have migrations and preemption + actually working before starting the prototype + so that tricky mistakes about concurrency can be catched early + my current hunch is that you are trying to do too much at the same + time... improving both the implementation details and redoing the system + design + so, for example, there is (or will be soon, actually) thread + migratio, but the scheduler doesn't take processor topology into account + that's why i'm starting from scratch + i don't delve too deep into the details + just the ones i consider very important + what do you mean by thread migration here? didn't you say you + don't even have IPC?... + i mean migration between cpus + OK + the other is too confusing + and far too unused and unknown to be used + and i won't actually implement it the way it was done in mach + again, it will be similar to qnx + oh? now that's news for me :-) + you seemed pretty hooked on thread migration when we talked about + these things last time... + i still am + i'm just saying it won't be implemented the same way + instead of upcalls from the kernel into userspace, i'll "simply" + make server threads inherit from the caller's scheduling context + the ideas i had about stack management are impossible to apply in + practice + which make the benefit i imagined unrealistic + and the whole idea was very confusing when compared and integrated + into a unix like view + so stack usage will be increased + that's ok + but thread migration is more or less equivalent with first-class + RPCs AIUI. does that work with the QNX IPC model?... + the very important property that threads don't block and wake a + server when sending, and the server again blocks and wake the client on + reply, is preserved + (in fact I find the term "first-class RPC" much clearer...) + i dont + there are two benefits in practice: since the scheduling context + is inherited, the client is charged for the cpu time consumed + and since there are no wakeups and blockings, but a direct hand + off in the scheduler, the cost of crossing task space is closer to the + system call + which can be problematic too... but still it's the solution chosen + by EROS for example AIUI + (inheriting scheduling contexts I mean) + by practically all modern microkernel based systems actually, as + noted by shapiro + braunr: well, both benefits can be achieved by other means as + well... scheduler activations like in Viengoos should handle the hand-off + part AIUI, and scheduling contexts can be inherited explicitly too, like + in EROS (and in a way in Viengoos) + i don't understand viengoos well enough to do it that way diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 3e585876..8119cdd5 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -347,3 +348,32 @@ Need more stuff like that. http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Operating_systems should be a pretty good list :) + + +# IRC, freenode, #hurd, 2013-03-12 + + i have a question regarding ipc in hurd. if a task is created, does + it contain any default port rights in its space? i am trying to deduce + how one calls dir_lookup() on the root translator in glibc's open(). + mjjc: yes, there are some default port rights, but I don't + remember the details :/ + kilobug: do you know where i should search for details? + mjjc: hum either in the Hurd's hacking guide + https://www.gnu.org/software/hurd/hacking-guide/ or directly in the + source code of exec server/libc I would say, or just ask again the + question here later on to see if someone else has more information + ok, thanks + there's also rpctrace to, as the name says, trace all the rpc's + executed + some ports are introduced in new tasks, yes + see + http://www.gnu.org/software/hurd/hacking-guide/hhg.html#The-main-function + and + + http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports + yes, the second link was just what i was looking for, thanks + the second is very general + also, the first applies to translators only + if you're looking for how to do it for a non-translator + application, the answer is probably somewhere in glibc + _hurd_startup i'd guess diff --git a/open_issues/clock_gettime.mdwn b/open_issues/clock_gettime.mdwn index 83ad81e8..b2fbf3e0 100644 --- a/open_issues/clock_gettime.mdwn +++ b/open_issues/clock_gettime.mdwn @@ -184,3 +184,12 @@ In context of [[select]]. do you want to make the changes yourself or should i add a new branch ? and we'll make that a 64bit struct when we have a64bit userland + + +# IRC, freenode, #hurd, 2013-04-06 + + pinotree: You had once been working on adding nsec-procision + timestamps to GNU Mach's maptime interface (or what the name is). Is + that blocked on something or just waiting to be continued? + blocked on me needing to learn more the proper way to do + "atomic" update of the struct with time :) diff --git a/open_issues/glibc/debian/experimental.mdwn b/open_issues/glibc/debian/experimental.mdwn new file mode 100644 index 00000000..8d117e99 --- /dev/null +++ b/open_issues/glibc/debian/experimental.mdwn @@ -0,0 +1,115 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_glibc]] + +Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental. + + +# IRC, OFTC, #debian-hurd, 2013-03-14 + + I have a strange tcp via localhost question: + The other side closes the connection, but I haven't read + all data, yet. I should still be able to read the pending data, no? + At least it seems to work that way on Linux, but not on + Hurd. + Got a simple repro with nc, if you're interested... + markus_wanner: yes, we're interested + youpi: okay, here we go: + session 1: nc -l -p 7777 localhost + session 2: nc 127.0.0.1 7777 + session 2: a b c + session 1: [ pause with Ctrl-Z ] + session 2: [ send more data ] d e f + session 2: [ quit with Ctrl-C ] + session 1: [ resume with 'fg' ] + The server on session 1 doesn't get the data sent after it + paused and before the client closed the connection. + I'm not sure if that's a valid TCP thing. However, on + Linux, the server still gets the data. On hurd it doesn't. + I'm working on a C-code test case, ATM. + markus_wanner: on which box are you seeing this behavior? + exodar does not have it + i.e. I do get the d e f + a private VM (I'm not a DD) + ..updated to latest experimental stuff. + GNU lematur 0.3 GNU-Mach 1.3.99-486/Hurd-0.3 i686-AT386 GNU + ok, I can't reproduce it on my vm either + maybe the C program will help + Hm.. cannot corrently reproduce that in C. (Netcat still + shows the issue, though). + I'll try to strace netcat... + ..Meh. strace not available on Hurd? + no, but there is rpctrace to show the various rpc + Cool, looks helpful. + Thx + Uh.. that introduces another error: + rpctrace: ../../utils/rpctrace.c:1287: trace_and_forward: + Assertion `reply_type == 18' failed. + +[[hurd/debugging/rpctrace]]. + + I'm checking on a box without ipv6 configuration + maybe that's the difference between you and me + I guess your /etc/alternatives/nc is /bin/nc.traditional ? + Yup, nc.traditional. + Looks like that box only has IPv4 configured. + Something very strange is going on here. No matter how hard + I try, I cannot reproduce this with netcat, anymore. + not even after a reboot? + Woo.. here, it happened, again! This is driving me crazy! + Now, nc seemingly connects, but is unable to send data + between the two. Netcat would somehow complain, if it failed to connect, + no? + No it worked. + So this seems to be an intermittent issue. So far, I could + only ever repro it as a normal user, not as root. May be coincidental, + though. + Now, 'a' and 'b' made it through, but not the 'c' sent + manually just after that. Something with that TCP/IP stack is definitely + fishy. + Anything I can try to investigate? Or shall I simply + restart and see if the problem persists? + maybe restart, yes + did you restart since the upgrade ? + Yes, I restarted after that. + Hm.. okay, restarted. Some problem persists. + I currently have two netcat processes connected, the + listening one got some first two messages and seems stuck now. + With the client, I tried to send more data, but the server + doesn't get it, anymore. + Any idea on what I can do to analyze the situation? + for the netcat issue, I haven't experienced this + are you running in kvm or virtualbox or something else? + I'm currently puzzled about what "experimental" actually + ships. + On kvm. + My libc0.3 used to be 2.13-39+hurd.3. + But packages.d.o already shows 2.17.0experimental2. + experimental ships experimental versions, which you aren't supposed + to use + unless you know what you are doing + iirc 2.17 is known to be quite broken for now + Okay. So I guess I'll try to "downgrade" to unstable, then. + Phew, okay, successfully downgraded to unstable. + Hopefully monotone's test suite runs through fine, now. + Yup, WORKING! Looks like some experimental packages caused + the problem. The netcat test as well as that one failing monotone test + work fine, now. + + +## IRC, OFTC, #debian-hurd, 2013-03-19 + + pinotree, youpi: Is there anything from that markus_wanner + discussion about pfinet/netcat/signals that needs to be filed? I guess + we don't know what exactly he changed so that everything workedd fine + eventually? (Some experimental package(s), but which?) + that was libc0.3 packages + which are indeed known to break the network diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index f0c0db58..acc03efd 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -1287,6 +1287,19 @@ There is a [[!FF_project 275]][[!tag bounty]] on this task. weird sentence, agreed :p +### IRC, freenode, #hurd, 2013-03-11 + + youpi: oh btw, i noticed a problem with the priority adjustement + code + a thread created by a privileged server (e.g. an ext2fs + translator) can then spawn a server from a node owned by an unprivileged + user + which inherits the priority + easy to fix but worth saying to keep in mind + uh + indeed + + ## IRC, freenode, #hurd, 2012-12-05 tschwinge: i'm currently working on a few easy bugs and i have diff --git a/open_issues/linux_vmsig.mdwn b/open_issues/linux_vmsig.mdwn index a4311d3e..182fb6f9 100644 --- a/open_issues/linux_vmsig.mdwn +++ b/open_issues/linux_vmsig.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -27,3 +27,18 @@ From a quick glance, this sounds to [[me|tschwinge]] quite a bit like mechanisms also found in (originating in?) Mach's [[microkernel/mach/external_pager_mechanism]]. May be worth having a look at it. + + +# IRC, freenode, #hurd, 2013-03-06 + + tschwinge: from a quick look, this isn't similar to the external + pager mechanism + it's an additional tool to help userspace application manage + internal caches + it's similar to what is done to reclaim memory from the slab + allocator for example + and it would indeed be a very good thing to have so that e.g. the + hurd can build a distributed but completely dynamic dentry-like cache + i'm actually glad to see someone else thought of using real time + signals for this + i didn't do any research on that subject yet diff --git a/open_issues/managed_runtime_initiative.mdwn b/open_issues/managed_runtime_initiative.mdwn new file mode 100644 index 00000000..7a880beb --- /dev/null +++ b/open_issues/managed_runtime_initiative.mdwn @@ -0,0 +1,72 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] + + +# IRC, freenode, #hurd, 2013-04-02 + + hi again, maybe someone has some metrics + is mprotect / munprotect faster in hurd then in linux ? + ? + can i protected a memory page against write access in hurd + and if so, is it a fast operation ? + you can, I never measured, but it's probably the same cost as in + linux + I don't see why it would be different, as it boils down to the same + x86 trick + but i suppose it is part of the mach kernel doing the protection + and not part of the unix layer ? + it is + the unix layer doesn't have mm control + it has to ask the kernel + it's slower on mach, as it's less optimized because of historical + reasons + but that's about it + less optimized, how so? + well, more entry fragmentation + in the end you mark the page table and flush the tlb + yes + the high level virtual memory layer is a bit slower + but fragmentation doesn't come into play it you just have one + memory object, does it? + it does, as it's about memory areas, not objects + the object is merely a backing store + protection, inheritance, copy on write are all area (vm_map_entry) + attributes + also, some operations affect all the address spaces where a + physical page is mapped + although i think linux does the same thing as mach/bsd now + but mprotect/munprotect doesn't, does it? + no + or perhaps by side effect, in some situations, i'm not sure + i think it depends if the memory is shared between processes, but + i don't remember the details and can't think of a proper example right + now + but anyway, "slower" here is negligible unless address spaces are + really huge and filled with lots of map entries + psockali: why do you ask ? + can i post a link here ? + about what ? + it's regarding azul / managed runtime initiative + a GC for java + why not + although i don't see the point for now :) + they have a custom MM management module for their GC as linux + kernel modul + and i was wondering if mach would be any faster then linux in + that aspect + + http://stackoverflow.com/questions/3358545/whats-actually-in-the-managed-runtime-initiatives-kernel-patches-and-jvm + psockali: generally speaking, mach is slower than linux because of + its age and the fact it didn't receive as much attention and + microoptimization as linux did + psockali: about this article, there is no reason mach would be + faster diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn index d7804864..03614fae 100644 --- a/open_issues/multithreading.mdwn +++ b/open_issues/multithreading.mdwn @@ -354,6 +354,33 @@ Tom Van Cutsem, 2009. [[hurd/libpager]]. +### IRC, freenode, #hurd, 2013-03-29 + + some day i'd like to add a system call that threads can use to + terminate themselves, passing their stack as a parameter for deallocation + then, we should review the timeouts used with libports management + having servers go away when unneeded is a valuable and visible + feature of modularity + + +### IRC, freenode, #hurd, 2013-04-03 + + youpi: i also run without the libports_stability patch + and i'd like it to be removed unless you still have a good reason + to keep it around + well, the reason I know is mentioned in the patch + i.e. the box becomes unresponsive when all these threads wake up at + the same time + maybe we could just introduce some randomness in the sleep time + youpi: i didn't experience the problem + well, I did :) + or if i did, it was very short + for the libports stability, I'd really say take a random value + between timeout/2 and timeout + and that should just nicely fix the issue + ok + + ## Alternative approaches: * diff --git a/open_issues/nice_vs_mach_thread_priorities.mdwn b/open_issues/nice_vs_mach_thread_priorities.mdwn index e27d3018..1f4c6ab8 100644 --- a/open_issues/nice_vs_mach_thread_priorities.mdwn +++ b/open_issues/nice_vs_mach_thread_priorities.mdwn @@ -387,3 +387,43 @@ here. the issue is mach not supporting enough sched levels can be fixed, of course just nobody did yet + +GNU Mach commit 6a234201081156e6d5742e7eeabb68418b518fad (and commit +6fe36b76f7983ec9a2e8a70420e431d54252442e). + + +## IRC, OFTC, #debian-hurd, 2013-03-07 + + youpi: btw, why did you increase the number of priorites to 50 ? + for the nice levels + and probably something more, there are only 40 nice levels + yes, the current computation leaves some margin + so I preferred to keep a margin too + ok + e.g. for the idle thread, etc. + or interrupt threads + yep + i see the point, thanks + Is the number of 40 specified by POSIX (or whatever) or is that + a Linuxism? + good question + posix is weird when it comes to such old unixisms + there is a NZERO value, but i don't remember how it's specified + it's at least 20 + (I don't object to the change; just wondered. And if practice, + you probably wouldn't really need more than a handful. But if that + change (plus some follow-up in glibc (?) improves something while not + adding a lot of overhead, then that's entirely fine, of course.) + "A maximum nice value of 2*{NZERO}-1 and a minimum nice value of 0 + shall be imposed by the system" + NZERO being 20 by default + and 20 is the minimum for NZERO too + hm, not the default, the minimul + minimum + yes that's it + ok so it's actually well specified + Aha, I see (just read it, too). So before that change we + simply couldn't satisfy the POSIX requirement of (minimum) NZERO = 20, + and allowing for step-1 increments. Alright. + yep + thus failing in coreutils testsuite diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index be582e8a..4034e940 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -2983,3 +2983,45 @@ License|/fdl]]."]]"""]] determine what m_o_ function will be used now default_read calls m_o_ on its own ok + + +## IRC, freenode, #hurd, 2013-03-06 + + braunr: hi, regarding memory policies. Should I create separate + policy that will do pageout or VM_ADVICE_SEQUENTIAL is good enough? + braunr: at the moment it is exactly like NORMAL + mcsim: i thought you only did pageins + braunr: yes, but I'm doing pageouts now + oh + i'd prefer you didn't :/ + if you want to improve paging, i have a suggestion i believe is a + lot better + and we have 3 patches concerning libpager that we need to review, + polish, and merge in + braunr: That's not hard, and I think I know what to do + yes i understand that + but it may change the interface and conflict with the pending + changes + braunr: What changes? + the large store patch, neal's libpager rework patch on top of + which you made your changes, and your changes + the idea i have in mind was writeback throttling + +[[hurd/translator/ext2fs]], [[hurd/libpager]]. + + i was planning on doing it myself but if you want to work on it, + feel free to + it would be a much better improvement at this time than clustered + pageouts + (which can then immediately follow + ) + braunr: ok + braunr: but this looks much more bigger task for me + we'll talk about the strategy i had in mind tomorrow + i hope you find it simple enough + on the other hand, clustered pageouts are very similar to pageins + and we have enough paging related changes to review that adding + another wouldn't be such a problem actually + so, add? + if that's what you want to do, ok + i'll think about your initial question tomorrow diff --git a/open_issues/pfinet_timers.mdwn b/open_issues/pfinet_timers.mdwn index 387ad4fe..5db192e3 100644 --- a/open_issues/pfinet_timers.mdwn +++ b/open_issues/pfinet_timers.mdwn @@ -15,3 +15,105 @@ License|/fdl]]."]]"""]] now that there is a pthread_hurd_cond_timedwait_np function available, we could replace the ulgy timers in pfinet + + +# IRC, freenode, #hurd, 2013-04-02 + + youpi: also, i think i know why fakeroot is slow on the hurd + well, pfinet actually + tcp flow control? + i think it's because of our timer resolution + ah + and i think i spotted a small mistake in the timer emulation code + btw + it's so obvious i even doubt it's actually true :) + see timer_emul.c:timer_function + the code checks for timers->expires < jiffies + shouldn't it be "<=" + ? + well it'd be equivalent + if ==, then wait becomes 0 in the else + see the second scheck + the one performed right before running the callback + ah, the while? + yes + I'd say <= could do it yes + ok + i have hurd packages ready to be tested + although I'm unsure it'd make a difference + do you notice some? + just waiting for my current glibc to finish building + and i'll test right after + i think it would actually + in which case? + the timer resolution is 10ms + it's huge + well, i hope it fixes fakeroot slowness :) + so timers below that would trigger immediately? + or equal + taht's the problem + for 10ms, timers that have expired must wait for the next tick to + fire + I include "equal" in below + actually :) + then yes :) + soryy i never know when equal is implicit + because they boil down to expires = jiffies + in french it's implicit + but anyway here equal is not really important + right + why ? + it's the fact that 1ms would be rounded up to 10ms, not down to 0ms + well, doing that seems reasonable + or rather, rounded down to 0ms, but waited for 10ms because of the + < + we do want timers not to fire before the time event + I'm however wondering which part of the code would have timer below + 10ms + well, a select with low timeout from a client perhaps ? + but then we have to round up + as you say we don't want to fire before some time + well yes + that's fine + all that being said, linux has been having 10ms clock for a long + time + but when the timer fires, i.e. when expires == jiffie, we do want + it to fire + highres timers are only relativeley recent + not at jiffie + 1 + braunr: ah, right, so we don't wait for 20ms instead of 10ms + yes + ok, so it's not a miracle fix, but could bring 2times fix + well, pfinet eats 40% of my processor when this problem occurs + which i'm seeing right now + yes, I've seen that too + so it may be a visible win + build finished :) let's see + Mmm, thinking again + indeed + when expires become jiffies + we mach_msg (0) + but don't do anything + and then restart + so just eaten cpu for nothing + is there a small package that builds fast and uses fakeroot a lot + ? + something like a package which has a lot of data to install at make + install + or i can rebuild glibc with -nc + ok, I've checked in linux + libarchive's testsuite used to intermittently failing under + fakeroot + it's definitely <= which is meant + it looks better + but still not 100% cpu + at any rate, as long as it doesn't seem to break anything, please + push the fix + it definitely makes sense + (and I don't see why it could break anything) + ok + i'll look into this timer problem a bit more, there may be other + code involved + yes, schedule_timeout could need a review + actually, fakeroot rm -rf * is a good test + and it's still damn slow diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn index ab239aef..cc3951c3 100644 --- a/open_issues/time.mdwn +++ b/open_issues/time.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -67,3 +68,75 @@ While testing some [[performance/IPC_virtual_copy]] performance issues: And I can confirm that with dd if=/dev/zero of=/dev/null bs=4k running, a parallel sleep 10 takes about 20 s (on strauss). + + +# IRC, OFTC, #debian-hurd, 2013-03-30 + + /usr/bin/time seems broken on hurd. It reports weird things. Ex: + # /usr/bin/time sleep 1 + 0.00user 0.00system 4:37:46elapsed 0%CPU (0avgtext+0avgdata + 0maxresident)k + 0inputs+0outputs (0major+0minor)pagefaults 0swaps + o_O + indeed, let's see what that time does + seems like only the elapsed time, %E + not only the time, but also the other variables (pagefaults, cpu + used, etc) are wrong. For example compare the output of + /usr/bin/time openssl speed ecdhp521 + on linux and hurd + most probably they are not implemented yet + they are all 0 + yes + should i report a bug to pkg time? + not sure + at least, there's this difference between eg amd64 and hurd-i386 + in configure's output: + -checking for wait3 that fills in rusage... yes + +checking for wait3 that fills in rusage... no + found this: + https://www.gnu.org/software/hurd/open_issues/time.html + seems related, yes + clopez: apparently all the ways to get the HZ define, either + directly or with CLOCKS_PER_SEC or CLK_TCK, so it gets defined as HZ + ... as 60, i mean (instead of 1000000) + $ ./time sleep 1 + 0.00user 0.00system 0:01.01elapsed 0PU (0avgtext+0avgdata + 0maxresident)k + :) + what it was? + i added the check for time.h, and included in the no-wait3 case + in resuse.c + (omg, the last release of gnu time was in 1997) + lol + hm not yet fixed + oh minor typo + clopez: http://paste.debian.net/246004/ + i will update the wiki page (on the hurd site) and send the + patch tomorrow + nice + yw, thanks again + i dropped the patch on debian/patches of pkg time.. rebuilt it + both on linux and hurd + and works as expected in both cases + i think you should forward the patch to the mantainer of pkg time + is there anyone maintaining gnu time? + http://packages.qa.debian.org/t/time.html + Maintainers for time are Bob Proulx . + that's the debian maintainer, yes, which is what i implied + earlier with "send the patch" + i guess that filling a bug against time with this patch attached + should be enough + yeah + wow... not only you fixed the elapsed time but also the other + variables :) + /usr/bin/time openssl speed ecdhp521 + now it reports cpu used and pagefaults :) + does it? + 10.00user 0.01system 0:10.11elapsed 99%CPU (0avgtext+0avgdata + 0maxresident)k + 0inputs+0outputs (67major+656minor)pagefaults 0swaps + + +# IRC, OFTC, #debian-hurd, 2013-03-31 + + clopez: #704283 diff --git a/open_issues/translate_fd_or_port_to_file_name.mdwn b/open_issues/translate_fd_or_port_to_file_name.mdwn index bd9abcf9..0d786d2a 100644 --- a/open_issues/translate_fd_or_port_to_file_name.mdwn +++ b/open_issues/translate_fd_or_port_to_file_name.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,14 +16,17 @@ License|/fdl]]."]]"""]] # IRC, freenode, #hurd, June (?) 2010 - is there a way (POSIX or Hurdish) to get the corresponding file name for a fd or a hurd port? + is there a way (POSIX or Hurdish) to get the corresponding file + name for a fd or a hurd port? there is a way marcusb: which one would that be? I forgot there is an implementation in libc realpath has a similar job but that's not what I mean - pochu: maybe I am misremembering. But it was something where you keep looking up .. and list that directory, looking for the node with the ID of the node you had .. for + pochu: maybe I am misremembering. But it was something where you + keep looking up .. and list that directory, looking for the node with the + ID of the node you had .. for maybe it works only for directories yeah pochu: check the getcwd() implementation of libc @@ -30,20 +34,25 @@ License|/fdl]]."]]"""]] _hurd_canonicalize_directory_name_internal  * pochu looks marcusb: interesting - though that is for dirs, and doesn't seem to be extensible to files, as you cannot lookup for ".." under a file + though that is for dirs, and doesn't seem to be extensible to + files, as you cannot lookup for ".." under a file right oh you already said that :) actually, I am not sure that's correct - it's probably correct, but there is no reason why looking .. up on a file couldn't return the directory it's contianed in - I don't know the interfaces or the Hurd internals very well yet, but it would look strange to me if you could do that + it's probably correct, but there is no reason why looking .. up + on a file couldn't return the directory it's contianed in + I don't know the interfaces or the Hurd internals very well yet, + but it would look strange to me if you could do that the hurd is strange - it sounds like if you could `ls getcwd.c/..` to get sysdeps/mach/hurd/ :-) + it sounds like if you could `ls getcwd.c/..` to get + sysdeps/mach/hurd/ :-) yep ok. interesting you wouldn't find "ls foo.zip/.." very strange, wouldn't you? I guess not if `ls foo.zip` listed the contents of foo.zip there you go - or the other way round: would you be surprised if "cat somedir" would work? + or the other way round: would you be surprised if "cat somedir" + would work? I think so. if it did, what would it do? originally, cat dir would list the directory content! in the old unix times @@ -51,10 +60,49 @@ License|/fdl]]."]]"""]] and some early BSDs * pochu feels young :-) he don't worry, I didn't see those times either - technically, files and directories are implemented in the same way in the hurd, they both are objects implementing the fs.defs interface + technically, files and directories are implemented in the same + way in the hurd, they both are objects implementing the fs.defs interface which combines file and directory operations - of course, files and directories implement those functions differently - marcusb: do you know why this behavior (cat on directories) was changed? + of course, files and directories implement those functions + differently + marcusb: do you know why this behavior (cat on directories) was + changed? + + +## IRC, freenode, #hurd, 2013-03-07 + + * pinotree ponders about sending as RFC his patch for /proc/$pid/maps + Including a scheme for providing the names of mapped files? + ;-D + that would be really great indeed + I have not yet researched how Linux does this. Perhaps store + the filename used for first opening a file as a string somewhere? + tschwinge: eh, indeed that's lacking in my patch + i'm not sure we should aim at doing it the same way + I was wondering about having interfaces for naming tasks, threads, + objects + that'd be useful for debugging in general + yes + i don't think we need to take namespaces into account + a simple name or path should be quite enough + Agreed. "Just something!" + So, a Java toString() method for ports. + ;-) + yes + Oh, and could this also work recursively? The ext2fs instance + on /home asks its parent fs about its own path -- can it do that? (And + then cache that, most likely?) Would one get rooted filesnames that way? + i really don't think we should link it to the VFS + it should merely be a name for debugging + yep, same for me + I'd say it's the linker's task of just setting a sane name + first, keeping it isolated prevents increasing complexity + next, it doesn't reduce performance + youpi: Linker? + braunr: Ack. + yes, ld is the one creating the mappings + tschwinge: the one that loads libraries + Ah, for /proc/*/maps, right. I've been thinking more globally. # IRC, freenode, #hurd, 2011-07-13 diff --git a/open_issues/virtualization/fakeroot.mdwn b/open_issues/virtualization/fakeroot.mdwn index ec762b59..f4739776 100644 --- a/open_issues/virtualization/fakeroot.mdwn +++ b/open_issues/virtualization/fakeroot.mdwn @@ -15,3 +15,10 @@ License|/fdl]]."]]"""]] btw, about fakeroot-hurd the remaining issue I see is with argv[0] (yes, again...) + + +## IRC, freenode, #hurd, 2013-04-03 + + btw, I believe our fakeroot-hurd is close to working actually + it's just a argv[0] issue supposed to be fixed by exec_file_name + but apparently not fixed in that case, for some reason diff --git a/open_issues/virtualization/remap_root_translator.mdwn b/open_issues/virtualization/remap_root_translator.mdwn index 67d64ae0..dcef310d 100644 --- a/open_issues/virtualization/remap_root_translator.mdwn +++ b/open_issues/virtualization/remap_root_translator.mdwn @@ -139,3 +139,8 @@ License|/fdl]]."]]"""]] let me just comment out that :) way better :) yep, just works fine + + +# IRC, freenode, #hurd, 2013-03-16 + + youpi: is the /bin/remap --help output correct ? -- cgit v1.2.3