From ff8af7a8958e412a03445e71340aedab2dc5fb05 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 6 Mar 2011 00:25:17 +0100 Subject: open_issues/unit_testing: Add 2011-03-05 IRC discussion. --- open_issues/unit_testing.mdwn | 163 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 163 insertions(+) (limited to 'open_issues') diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index d7821fd3..8749e650 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -70,3 +70,166 @@ abandoned). * [LaBrea](https://github.com/dustin/labrea/wiki), or similar tools can be used for modelling certain aspects of system behavior (long response times, for example). + + +# Discussion + +freenode, #hurd channel, 2011-03-05: + + what about testing though? + like sort of "what's missing? lets write tests for it so that + when someone gets to implementing it, he knows what to do. Repeat" + project + you mean creating an automated testing framework? + this is actually a task I want to add for this year, if I get + around to it :-) + yeah I'd very much want to apply for that one + cuz I've never done Kernel hacking before, but I know that with + the right tasks like "test VM functionality", I would be able to write up + the automated tests and hopefully learn more about what breaks/makes the + kernel + (and it would make implementing the feature much less hand-wavy + and more correct) + antrik, I believe the framework(CUnit right?) exists, but we just + need to write the tests. + do you have prior experience implementing automated tests? + lots of tests! + yes, in Java mostly, but I've played around with CUnit + ah, great :-) + here's what I know from experience: 1) write basic tests. 2) + write ones that test multiple features 3) stress test [option 4) + benchmark and record to DB] + well, what we'd rather need is to fix the issues we already know + from the existing testsuites :) + +[[GSoC project propsal|community/gsoc/project_ideas/testsuites]]. + + youpi, true, and I think I should check what's available in way + of tests, but if the tests are "all or nothing" then it becomes really + hard to fix your mistakes + they're not all or nothing + youpi: the existing testsuites don't test specific system features + libc ones do + we could also check posixtestsuite which does too + +[[open_issues/open_posix_test_suite]]. + + AFAIK libc has very few failing tests + +[[open_issues/glibc_testsuite]]. + + err, like twenty? + € grep -v '^#' expected-results-i486-gnu-libc | wc -l + 67 + nope, even more + oh, sorry, I confused it with coreutils + plus the binutils ones, i guess + yes + +[[open_issues/binutils#weak]]. + + anyways, I don't think relying on external testsuites for + regression testing is a good plan + also, that doesn't cover unit testing at all + why ? + sure there can be unit testing at the translator etc. level + if we want to implement test-driven development, and/or do serious + refactoring without too much risk, we need a test harness where we can + add specific tests as needed + but more than often, the issues are at the libc / etc. level + because of a combination o fthings at the translator level, which unit + testing won't find out + * nixness yewzzz! + sure unit testing can find them out. if they're good "unit" tests + the problem is that you don't necessarily know what "good" means + e.g. for posix correctness + since it's not posix + but if they're composite clever tests, then you lose that + granularity + youpi, is that a blackbox test intended to be run at the very end + to check if you're posix compliant? + also, if we have our own test harness, we can run tests + automatically as part of the build process, which is a great plus IMHO + nixness: "that" = ? + oh nvm, I thought there was a test stuie called "posix + correctness" + there's the posixtestsuite yes + it's an external one however + antrik: being external doesn't mean we can't invoke it + automatically as part of the build process when it's available + youpi, but being internal means it can test the inner workings of + certain modules that you are unsure of, and not just the interface + sure, that's why I said it'd be useful too + but as I said too, most bugs I've seen were not easy to find out at + the unit level + but rather at the libc level + of course we can integrate external tests if they exist and are + suitable. but that that doesn't preclude adding our own ones too. in + either case, that integration work has to be done too + again, I've never said I was against internal testsuite + also, the major purpose of a test suite is checking known + behaviour. a low-level test won't directly point to a POSIX violation; + but it will catch any changes in behaviour that would lead to one + what I said is that it will be hard to write them tight enough to + find bugs + again, the problem is knowing what will lead to a POSIX violation + it's long work + while libc / posixtestsuite / etc. already do that + *any* unexpected change in behaviour is likely to cause bugs + somewher + but WHAT is "expected" ? + THAT is the issue + and libc/posixtessuite do know that + at the translator level we don't really + see the recent post about link() + +[link(dir,name) should fail with +EPERM](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00007.html) + + in my memory jkoenig pointed it out for a lot of such calls + and that issue is clearly not known at the translator level + so you're saying that the tests have to be really really + low-level, and work our way up? + I'm saying that the translator level tests will be difficult to + write + why isn't it known at the translator level? if it's a translator + (not libc) bug, than obviously the translator must return something wrong + at some point, and that's something we can check + because you'll have to know all the details of the combinations + used in libc, to know whether they'll lead to posix issues + antrik: sure, but how did we detect that that was unexpected + behavior? + because of a glib test + at the translator level we didn't know it was an unexpected + behavior + gnulib actually + and if you had asked me, I wouldn't have known + again, we do *not* write a test suite to find existing bugs + right, took one for the other + doesn't really matter actually + antrik: ok, I don't care then + we write a test suite to prevent future bugs, or track status of + known bugs + (don't care *enough* for now, I mean) + hmm, so write something up antrik for GSoC :) and I'll be sure to + apply + now that we know some translators return a wrong error status in a + particular situation, we can write a test that checks exactly this error + status. that way we know when it is fixed, and also when it breaks again + nixness: great :-) + sweet. that kind of thing would also need a db backend + nixness: BTW, if you have a good idea, you can send an application + for it even if it's not listed among the proposed tasks... + so you don't strictly need a writeup from me to apply for this :-) + antrik, I'll keep that in mind, but I'll also be checking your + draft page + oh cool :) + (and it's a well known fact that those projects which students + proposed themselfs tend to be the most successful ones :-) ) + * nixness draft initiated + youpi: BTW, I'm surprised that you didn't mention libc testsuite + before... working up from there is probably a more effective plan than + starting with high-level test suites like Python etc... + wasn't it already in the gsoc proposal? + bummer + nope -- cgit v1.2.3 From afe6f5029552c8c6320e0074dd6b96f76db41a8d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 6 Mar 2011 00:29:22 +0100 Subject: open_issues/unit_testing: Might be a GSoC task. --- community/gsoc/project_ideas.mdwn | 3 ++- open_issues/unit_testing.mdwn | 4 ++++ 2 files changed, 6 insertions(+), 1 deletion(-) (limited to 'open_issues') diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index 649e05c1..e03dbede 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -104,6 +104,7 @@ See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/H [[!inline pages="community/gsoc/project_ideas/cdparanoia" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/perl_python" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/testsuites" show=0 feeds=no actions=yes]] +[[!inline pages="open_issues/unit_testing" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/libcap" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/xattr" show=0 feeds=no actions=yes]] [[!inline pages="open_issues/valgrind" show=0 feeds=no actions=yes]] diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index 8749e650..95aa0b5f 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -8,6 +8,10 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +This task may be suitable for [[community/GSoC]]. + +--- + A collection of thoughts with respect to unit testing. We definitely want to add unit test suites to our code base. -- cgit v1.2.3 From c318adc3b829acd0e1ce0fb6dd055a1c6d165e98 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 6 Mar 2011 00:54:54 +0100 Subject: community/gsoc/project_ideas: Add a listing of the tasks. --- community/gsoc/project_ideas.mdwn | 7 +++++++ open_issues/locking.mdwn | 4 ++-- open_issues/performance/io_system.mdwn | 2 +- open_issues/unit_testing.mdwn | 2 +- open_issues/valgrind.mdwn | 5 ++++- 5 files changed, 15 insertions(+), 5 deletions(-) (limited to 'open_issues') diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index e03dbede..4aa9b547 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -76,6 +76,13 @@ will assist you as well as we can. See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/Hurd_Porting). +Here is a list of what we consider suitable tasks: + +[[!map show=titlepages="(community/gsoc/project_ideas/* and + !community/gsoc/project_ideas/*/*) or tagged(gsoc-task)"]] + +Here are these pages' content: + [[!inline pages="community/gsoc/project_ideas/language_bindings" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/virtualization" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/file_locking" show=0 feeds=no actions=yes]] diff --git a/open_issues/locking.mdwn b/open_issues/locking.mdwn index 11a10524..6e22f887 100644 --- a/open_issues/locking.mdwn +++ b/open_issues/locking.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -9,7 +9,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_hurd]] +[[!tag open_issue_hurd gsoc-task]] Every now and then, new locking issues are discovered in [[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays diff --git a/open_issues/performance/io_system.mdwn b/open_issues/performance/io_system.mdwn index 4af093ba..8535eae3 100644 --- a/open_issues/performance/io_system.mdwn +++ b/open_issues/performance/io_system.mdwn @@ -11,7 +11,7 @@ License|/fdl]]."]]"""]] [[!meta title="I/O System"]] -[[!tag open_issue_hurd]] +[[!tag open_issue_hurd gsoc-task]] The most obvious reason for the Hurd feeling slow compared to mainstream systems like GNU/Linux, is a low I/O system performance, in particular very diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index 95aa0b5f..0ff7ecca 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -8,7 +8,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -This task may be suitable for [[community/GSoC]]. +This task may be suitable for [[community/GSoC]][[!tag gsoc-task]]. --- diff --git a/open_issues/valgrind.mdwn b/open_issues/valgrind.mdwn index 2b0624d7..bd45829c 100644 --- a/open_issues/valgrind.mdwn +++ b/open_issues/valgrind.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="Porting Valgrind to the Hurd"]] +[[!tag gsoc-task]] + [Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. (And some other kinds of hard-to-find errors too.) Aside from being useful for program development in general, -- cgit v1.2.3 From 6ef71999bc49af76b201f36f11747148d32f5863 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 6 Mar 2011 10:52:01 +0100 Subject: Some more IRC discussion. --- hurd/subhurd.mdwn | 48 ++++++++++++++++++++++++- open_issues/perlmagick.mdwn | 45 ++++++++++++++++++++++- open_issues/unit_testing.mdwn | 83 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 174 insertions(+), 2 deletions(-) (limited to 'open_issues') diff --git a/hurd/subhurd.mdwn b/hurd/subhurd.mdwn index cb4a40a8..b8e595d3 100644 --- a/hurd/subhurd.mdwn +++ b/hurd/subhurd.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -21,6 +21,8 @@ possible to use [[debugging/gdb/noninvasive_debugging]], but this is less flexible.) Vice versa, it is also possible to use a subhurd to debug the *main* Hurd system, for example, the latter's root file system. +[[!toc levels=2]] + # Howto @@ -139,3 +141,47 @@ translator|translator/ext2fs]]'s [[PID]], but this is no problem, as currently [[PID]]s are visible across subhurd boundaries. (It is a [[!taglink open_issue_hurd]] whether this is the right thing to do in [[open_issues/virtualization]] contexts, but that's how it currently is.) + + + +## Unit Testing + +freenode, #hurd channel, 2011-03-06: + +From [[open_issues/unit_testing]]. + + it could be interesting to have scripts that automatically start a + sub-hurd to do the tests + though that'd catch sub-hurd issues :) + so a sub-hurd is a hurd that I can run on something that I know + works(like linux)? + Virtual machine I would think + and over a network connection it would submit results back to + the host :p + * foocraft brain damage + sub-hurd is a bit like chroot + except that it's more complete + oh okay + i.e. almost everything gets replaced with what you want, except the + micro-kernel + that way you can even test the exec server for instance, without + risks of damaging the host OS + and we know the micro-kernel works correctly, right youpi? + well, at least it's small enough that most bugs are not there + 1) all tests run in subhurd 2) output results in a place in the + subhurd 3) tester in the host checks the result and pretty-prints it 4) + rinse & repeat + the output can actually be redirected iirc + since you give the sub-hurd a "console" + youpi, yup yeah, so now it's more like chroot if that's the case + it really looks like chroot, yes + but again, there's this subset of tests that we need to have + that ensures that even the tester running on the subhurd is valid, and it + didn't break because of a bug in the subhurd + As long as you do in-system testing, you'll always (have to) + rely on some functionality provided by the host system. + the worst thing that could happen with unit testing is false + results that lead someone to try to fix something that isn't broken :p + Yes. + usually one tries to repeat the test by hand in a normal + environment diff --git a/open_issues/perlmagick.mdwn b/open_issues/perlmagick.mdwn index 1daac62b..8a57a8fd 100644 --- a/open_issues/perlmagick.mdwn +++ b/open_issues/perlmagick.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -57,6 +57,49 @@ Etc. +/usr/lib/gcc/i486-gnu/4.4.2/include/omp.h: +# State as of 2011-03-06 + +freenode, #hurd channel, 2011-03-06: + + tschwinge: (speaking on working perl, how did it en with that + "(glibc) double free" crash with perl?) + *end + I think I remember I suspected it's a libgomp (!) issue in the + end. I have not yet continued working on that. + libogmp? looks like you know more than me, then :) + tschwinge: oh, I'm interested + I know a bit about libgomp :) + I bisected this down to where Imagemagick added -fgomp (or + whatever it is). And then the perl library (Imagemagick.pm?) which loads + the imagemagick.so segfaulted. + ImageMagick did this change in the middle of a x.x.x.something + release.. + My next step would have been to test whether libgomp works at + all for us. + ./usr/sbin/debootstrap:DEBOOTSTRAP_CHECKSUM_FIELD="SHA$SHA_SIZE" + erf + so they switched to another checksum + but we don't have that one on all of our packages :) + tschwinge: + buildd@bach:~$ OMP_NUM_THREADS=2 ./test + I'm 0x1 + I'm 0x3 + libgomp works at least a bit + OK. + i guess we should hope the working bits don't stop at that point + ;) + If open_issues/perlmagick is to be believed a diff of 6.4.1-1 + and 6.4.1-2 should tell what exactly was changed. + Oh! + I even have it on the page already! ;-) + -fopenmp + I've tried the pragmas that imagemagick uses + they work + Might be the issue fixed itself? + I don't know, it's the latest libc here + (and latest hurd, to be uploaded) + + # Other [[!debbug 551017]] diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index 0ff7ecca..a5ffe19d 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -58,6 +58,8 @@ abandoned). Developers*](http://lwn.net/Articles/412302/) by Steven Rostedt, 2010-10-28. [v2](http://lwn.net/Articles/414064/), 2010-11-08. + * + # Related @@ -237,3 +239,84 @@ EPERM](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00007.html) wasn't it already in the gsoc proposal? bummer nope + +freenode, #hurd channel, 2011-03-06: + + how's the hurd coding workflow, typically? + +*nixness* -> *foocraft*. + + we're discussing how TDD can work with Hurd (or general kernel + development) on #osdev + so what I wanted to know, since I haven't dealt much with GNU + Hurd, is how do you guys go about coding, in this case + Our current workflow scheme is... well... is... + Someone wants to work on something, or spots a bug, then works + on it, submits a patch, and 0 to 10 years later it is applied. + Roughly. + hmm so there's no indicator of whether things broke with that + patch + and how low do you think we can get with tests? A friend of mine + was telling me that with kernel dev, you really don't know whether, for + instance, the stack even exists, and a lot of things that I, as a + programmer, can assume while writing code break when it comes to writing + kernel code + Autotest seems promising + +See autotest link given above. + + but in any case, coming up with the testing framework that + doesn't break when the OS itself breaks is hard, it seems + not sure if autotest isolates the mistakes in the os from + finding their way in the validity of the tests themselves + it could be interesting to have scripts that automatically start a + sub-hurd to do the tests + +[[hurd/subhurd#unit_testing]]. + + foocraft: To answer one of your earlier questions: you can do + really low-level testing. Like testing Mach's message passing. A + million times. The questions is whether that makes sense. And / or if + it makes sense to do that as part of a unit testing framework. Or rather + do such things manually once you suspect an error somewhere. + The reason for the latter may be that Mach IPC is already + heavily tested during normal system operation. + And yet, there still may be (there are, surely) bugs. + But I guess that you have to stop at some (arbitrary?) level. + so we'd assume it works, and test from there + Otherwise you'd be implementing the exact counter-part of what + you're testing. + Which may be what you want, or may be not. Or it may just not + be feasible. + maybe the testing framework should have dependencies + which we can automate using make, and phony targets that run + tests + so everyone writes a test suite and says that it depends on A + and B working correctly + then it'd go try to run the tests for A etc. + Hmm, isn't that -- on a high level -- have you have by + different packages? For example, the perl testsuite depends (inherently) + on glibc working properly. A perl program's testsuite depends on perl + working properly. + yeah, but afaik, the ordering is done by hand + +freenode, #hurd channel, 2011-03-07: + + actually, I think for most tests it would be better not to use a + subhurd... that leads precisely to the problem that if something is + broken, you might have a hard time running the tests at all :-) + foocraft: most of the Hurd code isn't really low-level. you can + use normal debugging and testing methods + gnumach of course *does* have some low-level stuff, so if you add + unit tests to gnumach too, you might run into issues :-) + tschwinge: I think testing IPC is a good thing. as I already said, + automated testing is *not* to discover existing but unknown bugs, but to + prevent new ones creeping in, and tracking progress on known bugs + tschwinge: I think you are confusing unit testing and regression + testing. http://www.bddebian.com/~hurd-web/open_issues/unit_testing/ + talks about unit testing, but a lot (most?) of it is actually about + regression tests... + antrik: That may certainly be -- I'm not at all an expert in + this, and just generally though that some sort of automated testing is + needed, and thus started collecting ideas. + antrik: You're of course invited to fix that. -- cgit v1.2.3 From 817df620bedae9c1daa0497f64a901d51e5bd2dd Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 00:52:08 +0100 Subject: Some more IRC discussions. --- open_issues/anatomy_of_a_hurd_system.mdwn | 73 +++++++++ open_issues/ext2fs_page_cache_swapping_leak.mdwn | 23 +++ open_issues/pfinet_vs_system_time_changes.mdwn | 42 ++++++ ...dez-vous_leading_to_duplicate_port_destroy.mdwn | 163 +++++++++++++++++++++ open_issues/sudo_date_crash.mdwn | 16 -- open_issues/unit_testing.mdwn | 20 +++ 6 files changed, 321 insertions(+), 16 deletions(-) create mode 100644 open_issues/anatomy_of_a_hurd_system.mdwn create mode 100644 open_issues/ext2fs_page_cache_swapping_leak.mdwn create mode 100644 open_issues/pfinet_vs_system_time_changes.mdwn create mode 100644 open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn delete mode 100644 open_issues/sudo_date_crash.mdwn (limited to 'open_issues') diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn new file mode 100644 index 00000000..e1d5c9d8 --- /dev/null +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -0,0 +1,73 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!taglink open_issue_documentation]] + +A bunch of this should also be covered in other (introductionary) material, +like Bushnell's Hurd paper. All this should be unfied and streamlined. + +IRC, freenode, #hurd, 2011-03-08 + + I've a question on what are the "units" in the hurd project, if + you were to divide them into units if they aren't, and what are the + dependency relations between those units(roughly, nothing too pedantic + for now) + there is GNU Mach (the microkernel); there are the server + libraries in the Hurd package; there are the actual servers in the same; + and there is the POSIX implementation layer in glibc + relations are a bit tricky + Mach is the base layer which implements IPC and memory management + hmm I'll probably allocate time for dependency graph generation, + in the worst case + on top of this, the Hurd servers, using the server libraries, + implement various aspects of the system functionality + client programs use libc calls to use the servers + (servers also use libc to communicate with other servers and/or + Mach though) + so every server depends solely on mach, and no other server? + s/mach/mach and/or libc/ + I think these things should be pretty clear one you are somewhat + familiar with the Hurd architecture... nothing really tricky there + no + servers often depend on other servers for certain functionality + +--- + +IRC, freenode, #hurd, 2011-03-12 + + when mach first starts up, does it have some basic i/o or fs + functionality built into it to start up the initial hurd translators? + I/O is presently completely in Mach + filesystems are in userspace + the root filesystem and exec server are loaded by grub + o I see + so in order to start hurd, you would have to start mach and + simultaneously start the root filesystem and exec server? + not exactly + GRUB loads all three, and then starts Mach. Mach in turn starts + the servers according to the multiboot information passed from GRUB + ok, so does GRUB load them into ram? + I'm trying to figure out in my mind how hurd is initially started + up from a low-level pov + yes, as I said, GRUB loads them + ok, thanks antrik...I'm new to the idea of microkernels, but a + veteran of monolithic kernels + although I just learned that windows nt is a hybrid kernel which I + never knew! + note there's a /hurd/ext2fs.static + I belive that's what is used initially... right? + yes + loading the shared libraries in addition to the actual server + would be unweildy + so the root FS server is linked statically instead + what does the root FS server do? + well, it serves the root FS ;-) + it also does some bootstrapping work during startup, to bring the + rest of the system up diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn new file mode 100644 index 00000000..0ace5cd3 --- /dev/null +++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn @@ -0,0 +1,23 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +IRC, OFTC, #debian-hurd, 2011-03-24 + + I still believe we have an ext2fs page cache swapping leak, however + as the 1.8GiB swap was full, yet the ld process was only 1.5GiB big + a leak at swapping time, you mean? + I mean the ext2fs page cache being swapped out instead of simply + dropped + ah + so the swap tends to accumulate unuseful stuff, i see + yes + the disk content, basicallyt :) diff --git a/open_issues/pfinet_vs_system_time_changes.mdwn b/open_issues/pfinet_vs_system_time_changes.mdwn new file mode 100644 index 00000000..a9e1e242 --- /dev/null +++ b/open_issues/pfinet_vs_system_time_changes.mdwn @@ -0,0 +1,42 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +IRC, unknown channel, unknown date. + + I did a sudo date... + and the machine hangs + +This was very likely as misdiagnosis: + +IRC, freenode, #hurd, 2011-03-25 + + antrik: I suspect it'S some timing stuff in pfinet that perhaps + uses absolute time, and somehow wildely gets confused? + tschwinge: BTW, pfinet doesn't actually die I think -- it just + drops open connections... + perhaps it thinks they timed out + antrik: Isn't the translator restarted instead? + don't think so + when pfinet actually dies, I also loose the NFS mounts, which + doesn't happen in this case + hehe "... and the machine hangs" + he didn't bother to check that the machine is perfectly fine, only + the SSH connection got dropped + Ah, I see. So it'S perhaps indeed simply closes TCP + connections that have been without data for ``too long''? + yeah, that's my guess + my clock is speeding, so ntpdate sets it in the past + perhaps there is some math that concludes the connection have been + inactive for -200 seconds, which (unsigned) is more than any timeout :-) + (The other way round, you might likely get some integer + wrap-around, and thus the same result.) + Yes. diff --git a/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn b/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn new file mode 100644 index 00000000..9db92250 --- /dev/null +++ b/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn @@ -0,0 +1,163 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[RPC to self with rendez-vous leading to duplicate port +destroy](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00045.html) + +IRC, freenode, #hurd, 2011-03-14 + + youpi: I wonder, why does the root FS call diskfs_S_dir_lookup() + at all?... + errr, because a client asked for it? + (problem with RPCs is you can't easily know where they come from :) + ) + (especially when it's the root fs...) + ah, it's about a client request... didn't see that + well, I just said "is called", yes + I do not really understand though why it tries to reauthenticate + against itself... + I fear my memory of the lookup mechanism grew a bit dim + see the source + it's about a translated entry + (and I never fully understood some aspects anyways...) + it needs to start the translated entry as another user, possibly + yes, but a translated entry normally would be served by *another* + process?... + sure, but ext2fs has to prepare it + thus reauthenticate to prepare the correct set of rights + prepare what? + rights + so the process is not root, doesn't have / opened as root, etc. + rights for what? + err, about everything + IIRC the reauthentication is done by the parent FS on the port to + the *translated* node + and the translated node should be a different process?... + that's not what I read in the source + fshelp_fetch_root + ports[INIT_PORT_CRDIR] = reauth (getcrdir ()); + here, getcrdir() returns ext2fs itself + well, perhaps the issue is that I have no idea what + fshelp_fetch_root() does, nor why it is called here... + it notably starts the translator that dir_lookup is looking at, if + needed + possibly as a different user, thus reauthentication of CRDIR + so this is about a port that is passed to the translator being + started? + no + well, depends on what you mean by "port" + it's about reauthenticating a port to be passed to the translator + being started + and for that a rendez-vous port is needed for the reauthentication + and that's the one at stake + yeah, I meant the port that is reauthenticated + what is CRDIR? + current root dir ... + so the parent translator passes it's own root dir to the child + translator; and the issue is that for the root FS the root dir points to + the root FS itself... + yes + OK, that makes sense + (but that's only one example, rgrep mach_port_destroy hurd/ show + other potential issues) + well, that's actually what I wanted to mention next... why is the + rendez-vous port destroyed, instead of just deallocating the port right + and letting reference counting to it's thing?... + do its thing + "just to make sure" I guess + it's pretty obvious that this will cause trouble for any RPC + referencing itself... + well, follow-up with that on the list + with roland/tb in CC + only they would know any real reason for destroy + btw, if you knew how we could make _hurd_select()'s raw __mach_msg + call be interruptible by signals, that'll permit to fix sudo + (damn, I need sleep, my tenses are all wrong) + BTW, does this cause any actual trouble?... + I don't know much about interruption... cfhammer might have a + better idea, he look into that stuff quite a bit AIUI + looked + (hehe, it's not only your tenses... guess there's something in the + ether ;-) ) + it makes sudo, mailq, etc. fail sometimes + I mean the rendez-vous thing + that's it, yes + sudo etc. fail at least due to this + so these are two different problems that both affect sudo? + (rendez-vous and interruption I mean) + yes + with my patch the buildds have much fewer issues, but still some + (my interrupt-related patch) + I'm installing a s/destroy/deallocate/ version of ext2fs on the + buildds, we'll see how it behaves + (it fixes my testcase at least) + interrupt-related patch? + only thing interrupt-related I remember was the reauthentication + race... + that's what I mean + well, cfhammer investigated this is quite some depth, explaining + quite well why the race is only mitigated but still exists... problem is + that we didn't know how to fix it properly + because nobody seems to understand the cancellation code, except + perhaps for Roland and Thomas + (and I'm not even entirely sure about them :-) ) + I think his findings and our conclusions are documented on the + ML... + by "much fewer issues", I mean that some of the symptoms have + disappeared, others haven't + BTW, couldn't the rendez-vous thing be worked around by simply + ignoring the errors from the failing deallocate?... + no, failing deallocate are actually dangerous + why? + since the name might have been reused for something else in the + meanwhile + that's the whole point of the warning I had added in the kernel + itself + I see + such things really deserve tracking, since they can have any kind + of consequence + does Mach try to reuse names quickly, rather than only after + wrapping around?... + it seems to + OK, then this is a serious problem indeed + (note: I rarely divine issues when there aren't actual frequent + symptoms :) ) + well, the problem with the warning is that it only shows in the + cases that do *not* cause a problem... so it's hard to associate them + with any specific issues + well, most of the time the port is not reused quickly enough + so in most case it shows up more often than causing problem + +IRC, freenode, #hurd, 2011-03-14 + + ok, mach_port_deallocate actually can't be used + since mach_reply_port() returns a receive right, not a send right + * youpi guesses he will really have to manage to understand all that port + stuff completely + oh, right + youpi: hm... now I'm confused though. if one client holds a + receive right, the other client (or in this case the same process) should + have a send or send-once right -- these should *not* share the same name + in my understanding + destroying the receive right should turn the send right into a + dead name + so unless I'm missing something, the destroy shouldn't be a + problem, and there must be something else going wrong + hm... actually I'm probably wrong + yeah, definitely wrong. receive rights and "ordinary" send rights + share the name. only send-once rights are special + I wonder whether the problem could be worked around by using a + send-once right... + mach_port_mod_refs(mach_task_self(), name, + MACH_PORT_RIGHT_RECEIVE, -1) can be used to deallocate only the receive + right + oh, you already figured that out :-) diff --git a/open_issues/sudo_date_crash.mdwn b/open_issues/sudo_date_crash.mdwn deleted file mode 100644 index 53303abc..00000000 --- a/open_issues/sudo_date_crash.mdwn +++ /dev/null @@ -1,16 +0,0 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_gnumach]] - -IRC, unknown channel, unknown date. - - I did a sudo date... - and the machine hangs diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index a5ffe19d..feda3be4 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -320,3 +320,23 @@ freenode, #hurd channel, 2011-03-07: this, and just generally though that some sort of automated testing is needed, and thus started collecting ideas. antrik: You're of course invited to fix that. + +IRC, freenode, #hurd, 2011-03-08 + +(After discussing the [[anatomy_of_a_hurd_system]].) + + so that's what your question is actually about? + so what I would imagine is a set of only-this-server tests for + each server, and then we can have fun adding composite tests + thus making debugging the composite scenarios a bit less tricky + indeed + and if you were trying to pass a composite test, it would also + help knowing that you still didn't break the server-only test + there are so many different things that can be tested... the + summer will only suffice to dip into this really :-) + yeah, I'm designing my proposal to focus on 1) make/use a + testing framework that fits the Hurd case very well 2) write some tests + and docs on how to write good tests + well, doesn't have to be *one* framework... unit testing and + regression testing are quite different things, which can be covered by + different frameworks -- cgit v1.2.3 From c82754dbb4bf6efd9ba50cbf5e5d0586ffaf816c Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 00:52:52 +0100 Subject: open_issues/gdb_noninvasive_mode_new_threads: New. --- open_issues/gdb_noninvasive_mode_new_threads.mdwn | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 open_issues/gdb_noninvasive_mode_new_threads.mdwn (limited to 'open_issues') diff --git a/open_issues/gdb_noninvasive_mode_new_threads.mdwn b/open_issues/gdb_noninvasive_mode_new_threads.mdwn new file mode 100644 index 00000000..9b3992f4 --- /dev/null +++ b/open_issues/gdb_noninvasive_mode_new_threads.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb]] + +Debugging a translator. `gdb binary`. `set noninvasive on`. `attach [PID]`. +Translator does some work. GDB doesn't notice new threads. `detach`. `attach +[PID]` -- now new threads are visible. -- cgit v1.2.3 From 64f60ab301c296aa6abe6a2dbc4ad27df3763034 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 00:53:40 +0100 Subject: open_issues/gdb_thread_ids: Another issue. --- open_issues/gdb_thread_ids.mdwn | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) (limited to 'open_issues') diff --git a/open_issues/gdb_thread_ids.mdwn b/open_issues/gdb_thread_ids.mdwn index c31a9967..c04a10ee 100644 --- a/open_issues/gdb_thread_ids.mdwn +++ b/open_issues/gdb_thread_ids.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="GDB: thread ids"]] @@ -23,3 +23,9 @@ GNU GDB's Pedro Alves: Also see [[thread numbering of ps and GDB]]. + +--- + +`attach` to a multi-threaded process. See threads 1 to 5. `detach`. `attach` +again -- thread numbers continue where they stopped last time: now they're +threads 6 to 10. -- cgit v1.2.3 From 2ad64644b9290ed1b63108b01ef63939adba5a93 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 01:29:05 +0100 Subject: Re-integrate GSoC pages with the non-GSoC world. Remove duplicates, apart from procfs, which should rather be removed from the GSoC items. --- .../gsoc/project_ideas/disk_io_performance.mdwn | 10 +-- community/gsoc/project_ideas/dtrace.mdwn | 8 ++- .../gsoc/project_ideas/libdiskfs_locking.mdwn | 10 +-- community/gsoc/project_ideas/procfs.mdwn | 8 ++- community/gsoc/project_ideas/valgrind.mdwn | 2 + lttng.mdwn | 2 +- open_issues/code_analysis.mdwn | 4 +- open_issues/debugging.mdwn | 4 +- open_issues/dtrace.mdwn | 47 ------------ open_issues/locking.mdwn | 40 ----------- open_issues/performance.mdwn | 4 +- open_issues/performance/io_system.mdwn | 50 ------------- .../performance/io_system/binutils_ld_64ksec.mdwn | 5 +- .../io_system/clustered_page_faults.mdwn | 2 + open_issues/performance/io_system/read-ahead.mdwn | 2 + open_issues/profiling.mdwn | 2 +- open_issues/valgrind.mdwn | 83 ---------------------- systemtap.mdwn | 2 +- topgit.mdwn | 5 +- 19 files changed, 45 insertions(+), 245 deletions(-) delete mode 100644 open_issues/dtrace.mdwn delete mode 100644 open_issues/locking.mdwn delete mode 100644 open_issues/performance/io_system.mdwn delete mode 100644 open_issues/valgrind.mdwn (limited to 'open_issues') diff --git a/community/gsoc/project_ideas/disk_io_performance.mdwn b/community/gsoc/project_ideas/disk_io_performance.mdwn index b6f223c8..ae634709 100644 --- a/community/gsoc/project_ideas/disk_io_performance.mdwn +++ b/community/gsoc/project_ideas/disk_io_performance.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="Disk I/O Performance Tuning"]] +[[!tag open_issue_hurd]] + The most obvious reason for the Hurd feeling slow compared to mainstream systems like GNU/Linux, is a low I/O system performance, in particular very slow hard disk access. @@ -18,8 +20,8 @@ slow hard disk access. The reason for this slowness is lack and/or bad implementation of common optimization techniques, like scheduling reads and writes to minimize head movement; effective block caching; effective reads/writes to partial blocks; -[[reading/writing multiple blocks at once|clustered_page_faults]]; and -[[read-ahead]]. The +[[reading/writing multiple blocks at once|open_issues/performance/io_system/clustered_page_faults]]; and +[[open_issues/performance/io_system/read-ahead]]. The [[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some optimizations at a higher logical level. @@ -29,12 +31,12 @@ requires understanding the data flow through the various layers involved in disk access on the Hurd ([[filesystem|hurd/virtual_file_system]], [[pager|hurd/libpager]], driver), and general experience with optimizing complex systems. That said, the killing feature we are definitely -missing is the [[read-ahead]], and even a very simple implementation would bring +missing is the [[open_issues/performance/io_system/read-ahead]], and even a very simple implementation would bring very big performance speedups. Here are some real testcases: - * [[binutils_ld_64ksec]]; + * [[open_issues/performance/io_system/binutils_ld_64ksec]]; * running the Git testsuite which is mostly I/O bound; diff --git a/community/gsoc/project_ideas/dtrace.mdwn b/community/gsoc/project_ideas/dtrace.mdwn index 252c303a..8fd3231f 100644 --- a/community/gsoc/project_ideas/dtrace.mdwn +++ b/community/gsoc/project_ideas/dtrace.mdwn @@ -11,13 +11,15 @@ License|/fdl]]."]]"""]] [[!meta title="dtrace Support"]] +[[!tag open_issue_gnumach]] + One of the main problems of the current Hurd implementation is very poor -[[performance]]. While we have a bunch of ideas what could cause the performance +[[open_issues/performance]]. While we have a bunch of ideas what could cause the performance problems, these are mostly just guesses. Better understanding what really causes bad performance is necessary to improve the situation. For that, we need tools for performance measurements. While all kinds of more -or less specific [[profiling]] tools could be conceived, the most promising and +or less specific [[open_issues/profiling]] tools could be conceived, the most promising and generic approach seems to be a framework for logging certain events in the running system (both in the microkernel and in the Hurd servers). This would allow checking how much time is spent in certain modules, how often certain @@ -35,7 +37,7 @@ with integrating existing components as well as low-level programming. Possible mentors: Samuel Thibault (youpi) -Related: [[profiling]], [[LTTng]], [[SystemTap]] +Related: [[open_issues/profiling]], [[LTTng]], [[SystemTap]] Exercise: In lack of a good exercise directly related to this task, just pick one of the kernel-related or generally low-level tasks from the bug/task diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn index 0e38b6a0..f26efb7f 100644 --- a/community/gsoc/project_ideas/libdiskfs_locking.mdwn +++ b/community/gsoc/project_ideas/libdiskfs_locking.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="Fix libdiskfs Locking Issues"]] +[[!tag open_issue_hurd]] + Every now and then, new locking issues are discovered in [[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays these in fact seem to be the most often encountered cause of Hurd crashes @@ -24,19 +26,19 @@ faulty paths causing these lockups. The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking issues. To achieve this, some kind of test harness has to be implemented: For example instrumenting the code to check locking correctness constantly at -runtime. Or implementing a [[unit testing]] framework that explicitly checks +runtime. Or implementing a [[open_issues/unit_testing]] framework that explicitly checks locking in various code paths. (The latter could serve as a template for implementing unit tests in other parts of the Hurd codebase...) -(A [[systematic code review|security]] would probably suffice to find the +(A [[systematic code review|open_issues/security]] would probably suffice to find the existing locking issues; but it wouldn't document the work in terms of actual code produced, and thus it's not suitable for a GSoC project...) This task requires experience with debugging locking issues in -[[multithreaded|multithreading]] applications. +[[multithreaded|open_issues/multithreading]] applications. -Tools have been written for automated [[code analysis]]; these can help to +Tools have been written for automated [[open_issues/code_analysis]]; these can help to locate and fix such errors. Possible mentors: Samuel Thibault (youpi) diff --git a/community/gsoc/project_ideas/procfs.mdwn b/community/gsoc/project_ideas/procfs.mdwn index d4760aae..b4d1dc5f 100644 --- a/community/gsoc/project_ideas/procfs.mdwn +++ b/community/gsoc/project_ideas/procfs.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -43,3 +44,8 @@ Exercise: Add or fix one piece in the existing procfs translator. *Status*: Madhusudan.C.S has implemented a new, fully functional [[procfs|madhusudancs]] for GSoC 2008. He is still working on some outstanding issues. + +--- + +Note that [[jkoenig's `procfs` re-write|hurd/translator/procfs/jkoenig]] should +address all these issues already. diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn index 79c8bd1d..16cd1373 100644 --- a/community/gsoc/project_ideas/valgrind.mdwn +++ b/community/gsoc/project_ideas/valgrind.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="Porting Valgrind to the Hurd"]] +[[!tag open_issue_gnumach open_issues_hurd]] + [Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. (And some other kinds of hard-to-find errors too.) Aside from being useful for program development in general, diff --git a/lttng.mdwn b/lttng.mdwn index 5776baa8..840312c4 100644 --- a/lttng.mdwn +++ b/lttng.mdwn @@ -19,7 +19,7 @@ License|/fdl]]."]]"""]] # Related - * [[open_issues/dtrace]] + * [[community/gsoc/project_ideas/dtrace]] * [[SystemTap]] diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn index ad59f962..21e09089 100644 --- a/open_issues/code_analysis.mdwn +++ b/open_issues/code_analysis.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -40,7 +40,7 @@ analysis|performance]], [[formal_verification]], as well as general * - * [[Valgrind]] + * [[community/gsoc/project_ideas/Valgrind]] * [Smatch](http://smatch.sourceforge.net/) diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn index e66a086f..e087f484 100644 --- a/open_issues/debugging.mdwn +++ b/open_issues/debugging.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -42,7 +42,7 @@ We have debugging infrastructure. For example: Continues: , which introduces . - * [[locking]] + * [[community/gsoc/project_ideas/libdiskfs_locking]] * , or -- just two examples; there's a lot of such stuff for Linux. diff --git a/open_issues/dtrace.mdwn b/open_issues/dtrace.mdwn deleted file mode 100644 index cbac28fb..00000000 --- a/open_issues/dtrace.mdwn +++ /dev/null @@ -1,47 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -One of the main problems of the current Hurd implementation is very poor -[[performance]]. While we have a bunch of ideas what could cause the performance -problems, these are mostly just guesses. Better understanding what really -causes bad performance is necessary to improve the situation. - -For that, we need tools for performance measurements. While all kinds of more -or less specific [[profiling]] tools could be conceived, the most promising and -generic approach seems to be a framework for logging certain events in the -running system (both in the microkernel and in the Hurd servers). This would -allow checking how much time is spent in certain modules, how often certain -situations occur, how things interact, etc. It could also prove helpful in -debugging some issues that are otherwise hard to find because of complex -interactions. - -The most popular framework for that is Sun's dtrace; but there might be others. -The student has to evaluate the existing options, deciding which makes most -sense for the Hurd; and implement that one. (Apple's implementation of dtrace -in their Mach-based kernel might be helpful here...) - -This project requires ability to evaluate possible solutions, and experience -with integrating existing components as well as low-level programming. - -Possible mentors: Samuel Thibault (youpi) - -Related: [[profiling]], [[LTTng]], [[SystemTap]] - -Exercise: In lack of a good exercise directly related to this task, just pick -one of the kernel-related or generally low-level tasks from the bug/task -trackers on savannah, and make a go at it. You might not be able to finish the -task in a limited amount of time, but you should at least be able to make a -detailed analysis of the issue. - -*Status*: Andei Barbu was working on -[SystemTap](http://csclub.uwaterloo.ca/~abarbu/hurd/) for GSoC 2008, but it -turned out too Linux-specific. He implemented kernel probes, but there is no -nice frontend yet. diff --git a/open_issues/locking.mdwn b/open_issues/locking.mdwn deleted file mode 100644 index 6e22f887..00000000 --- a/open_issues/locking.mdwn +++ /dev/null @@ -1,40 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_hurd gsoc-task]] - -Every now and then, new locking issues are discovered in -[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays -these in fact seem to be the most often encountered cause of Hurd crashes -/ lockups. - -One of these could be traced -recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken -and not released in some cases. There is reason to believe that there are more -faulty paths causing these lockups. - -The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking -issues. To achieve this, some kind of test harness has to be implemented: For -example instrumenting the code to check locking correctness constantly at -runtime. Or implementing a [[unit testing]] framework that explicitly checks -locking in various code paths. (The latter could serve as a template for -implementing unit tests in other parts of the Hurd codebase...) - -(A [[systematic code review|security]] would probably suffice to find the -existing locking -issues; but it wouldn't document the work in terms of actual code produced, and -thus it's not suitable for a GSoC project...) - -This task requires experience with debugging locking issues in -[[multithreaded|multithreading]] applications. - -Tools have been written for automated [[code analysis]]; these can help to -locate and fix such errors. diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn index 9b3701b3..eb9f3f8a 100644 --- a/open_issues/performance.mdwn +++ b/open_issues/performance.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -18,7 +18,7 @@ In [[microkernel]]-based systems, there is generally a considerable [[RPC]] overhead. In a multi-server system, it is non-trivial to implement a high-performance -[[I/O System|io_system]]. +[[I/O System|community/gsoc/project_ideas/disk_io_performance]]. When providing [[faq/POSIX_compatibility]] (and similar interfaces) in an environemnt that doesn't natively implement these interfaces, there may be a diff --git a/open_issues/performance/io_system.mdwn b/open_issues/performance/io_system.mdwn deleted file mode 100644 index 8535eae3..00000000 --- a/open_issues/performance/io_system.mdwn +++ /dev/null @@ -1,50 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!meta title="I/O System"]] - -[[!tag open_issue_hurd gsoc-task]] - -The most obvious reason for the Hurd feeling slow compared to mainstream -systems like GNU/Linux, is a low I/O system performance, in particular very -slow hard disk access. - -The reason for this slowness is lack and/or bad implementation of common -optimization techniques, like scheduling reads and writes to minimize head -movement; effective block caching; effective reads/writes to partial blocks; -[[reading/writing multiple blocks at once|clustered_page_faults]]; and -[[read-ahead]]. The -[[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some -optimizations at a higher logical level. - -The goal of this project is to analyze the current situation, and implement/fix -various optimizations, to achieve significantly better disk performance. It -requires understanding the data flow through the various layers involved in -disk access on the Hurd ([[filesystem|hurd/virtual_file_system]], -[[pager|hurd/libpager]], driver), and general experience with -optimizing complex systems. That said, the killing feature we are definitely -missing is the [[read-ahead]], and even a very simple implementation would bring -very big performance speedups. - -Here are some real testcases: - - * [[binutils_ld_64ksec]]; - - * running the Git testsuite which is mostly I/O bound; - - * use [[TopGit]] on a non-toy repository. - - -Possible mentors: Samuel Thibault (youpi) - -Exercise: Look through all the code involved in disk I/O, and try something -easy to improve. It's quite likely though that you will find nothing obvious -- -in this case, please contact us about a different exercise task. diff --git a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn index b59a87a7..79c2300f 100644 --- a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn +++ b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -This one may be considered as a testcase for I/O system optimization. +This one may be considered as a testcase for [[I/O system +optimization|community/gsoc/project_ideas/disk_io_performance]]. It is taken from the [[binutils testsuite|binutils]], `ld/ld-elf/sec64k.exp`, where this diff --git a/open_issues/performance/io_system/clustered_page_faults.mdwn b/open_issues/performance/io_system/clustered_page_faults.mdwn index 3a187523..37433e06 100644 --- a/open_issues/performance/io_system/clustered_page_faults.mdwn +++ b/open_issues/performance/io_system/clustered_page_faults.mdwn @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gnumach open_issue_hurd]] +[[community/gsoc/project_ideas/disk_io_performance]]. + IRC, freenode, #hurd, 2011-02-16 exceptfor the kernel, everything in an address space is diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index 3ee30b5d..b6851edd 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gnumach open_issue_hurd]] +[[community/gsoc/project_ideas/disk_io_performance]] + IRC, #hurd, freenode, 2011-02-13: youpi: Would libdiskfs/diskfs.h be in the right place to make diff --git a/open_issues/profiling.mdwn b/open_issues/profiling.mdwn index e04fb08a..7e3c7350 100644 --- a/open_issues/profiling.mdwn +++ b/open_issues/profiling.mdwn @@ -17,7 +17,7 @@ done for [[performance analysis|performance]] reasons. Should be working, but some issues have been reported, regarding GCC spec files. Should be possible to fix (if not yet done) easily. - * [[dtrace]] + * [[community/gsoc/project_ideas/dtrace]] Have a look at this, integrate it into the main trees. diff --git a/open_issues/valgrind.mdwn b/open_issues/valgrind.mdwn deleted file mode 100644 index bd45829c..00000000 --- a/open_issues/valgrind.mdwn +++ /dev/null @@ -1,83 +0,0 @@ -[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!meta title="Porting Valgrind to the Hurd"]] - -[[!tag gsoc-task]] - -[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. -(And some other kinds of hard-to-find errors too.) -Aside from being useful for program development in general, -a Hurd port will help finding out why certain programs segfault on the Hurd, -although they work on Linux. -Even more importantly, it will help finding bugs in the Hurd servers themselfs. - -To keep track of memory use, -Valgrind however needs to know how each [[system call]] affects the validity of memory regions. -This knowledge is highly kernel-specific, -and thus Valgrind needs to be explicitely ported for every system. - -Such a port involves two major steps: -making Valgrind understand how kernel traps work in general on the system in question; -and how all the individual kernel calls affect memory. -The latter step is where most of the work is, -as the behaviour of each single [[system call]] needs to be described. - -Compared to Linux, -[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps. -Almost all [[system call]]s are implemented as [[RPC]]s instead -- -either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]]. -All RPCs use a pair of `mach_msg` invocations: -one to send a request message, and one to receive a reply. -However, while all RPCs use the same `mach_msg` trap, -the actual effect of the call varies greatly depending on which RPC is invoked -- -similar to the `ioctl` call on Linux. -Each request thus must be handled individually. - -Unlike `ioctl`, -the RPC invocations have explicit type information for the parameters though, -which can be retrieved from the message header. -By analyzing the parameters of the RPC reply message, -Valgrind can know exactly which memory regions are affected by that call, -even without specific knowledge of the RPC in question. -Thus implementing a general parser for the reply messages -will already give Valgrind a fairly good approximation of memory validity -- -without having to specify the exact semantic of each RPC by hand. - -While this should make Valgrind quite usable on the Hurd already, it's not perfect: -some RPCs might return a buffer that is only partially filled with valid data; -or some reply parameters might be optional, -and only contain valid data under certain conditions. -Such specific semantics can't be deduced from the message headers alone. -Thus for a complete port, -it will still be necessary to go through the list of all known RPCs, -and implement special handling in Valgrind for those RPCs that need it. - -The goal of this task is at minimum to make Valgrind grok Mach traps, -and to implement the generic RPC handler. -Ideally, specific handling for RPCs needing it should also be implemented. - -Completing this project will require digging into Valgrind's handling of [[system call]]s, -and into Hurd RPCs. -It is not an easy task, but a fairly predictable one -- -there shouldn't be any unexpected difficulties, -and no major design work is necessary. -It doesn't require any specific previous knowledge: -only good programming skills in general. -On the other hand, -the student will obtain a good understanding of Hurd RPCs while working on this task, -and thus perfect qualifications for Hurd development in general :-) - -Possible mentors: Samuel Thibault (youpi) - -Exercise: As a starter, -students can try to teach valgrind a couple of Linux ioctls, -as this will make them learn how to use the read/write primitives of valgrind. diff --git a/systemtap.mdwn b/systemtap.mdwn index abd1961e..ba64c0d4 100644 --- a/systemtap.mdwn +++ b/systemtap.mdwn @@ -19,7 +19,7 @@ License|/fdl]]."]]"""]] # Related - * [[open_issues/dtrace]] + * [[community/gsoc/project_ideas/dtrace]] * [[LTTng]] diff --git a/topgit.mdwn b/topgit.mdwn index b71038ec..fb374337 100644 --- a/topgit.mdwn +++ b/topgit.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -30,7 +30,8 @@ development branches, for example [[source_repositories/binutils]] or # Running it on GNU/Hurd Nothing special to that, technically, *only* that our [[I/O system's (non-) -performance|open_issues/performance/io_system]] will render this unbearably +performance|community/gsoc/project_ideas/disk_io_performance]] will render this +unbearably slow for anything but simple test cases. So don't try to run it on the [[GCC]] or [[glibc]] repositories. Talk to [[tschwinge]] about how he's using it on a GNU/Linux machine and push the resulting trees to GNU/Hurd systems. -- cgit v1.2.3 From a2a4176bcd74dc6c607d48131f26cb5fa4affb3d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 01:42:25 +0100 Subject: open_issues/unit_testing: Move discussion to community/gsoc/project_ideas/testing_framework/discussion. --- .../gsoc/project_ideas/testing_framework.mdwn | 2 +- .../testing_framework/discussion.mdwn | 270 +++++++++++++++++++++ open_issues/unit_testing.mdwn | 266 +------------------- 3 files changed, 276 insertions(+), 262 deletions(-) create mode 100644 community/gsoc/project_ideas/testing_framework/discussion.mdwn (limited to 'open_issues') diff --git a/community/gsoc/project_ideas/testing_framework.mdwn b/community/gsoc/project_ideas/testing_framework.mdwn index 0448bc6b..ff9899d9 100644 --- a/community/gsoc/project_ideas/testing_framework.mdwn +++ b/community/gsoc/project_ideas/testing_framework.mdwn @@ -36,7 +36,7 @@ before the end of the summer. (As a bonus, in addition to these explicit tests, it would be helpful to integrate some methods -for testing [[locking validity|open_issues/locking]], +for testing [[locking validity|libdiskfs_locking]], performing static code analysis etc.) This task probably requires some previous experience diff --git a/community/gsoc/project_ideas/testing_framework/discussion.mdwn b/community/gsoc/project_ideas/testing_framework/discussion.mdwn new file mode 100644 index 00000000..872d0eb7 --- /dev/null +++ b/community/gsoc/project_ideas/testing_framework/discussion.mdwn @@ -0,0 +1,270 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +freenode, #hurd channel, 2011-03-05: + + what about testing though? + like sort of "what's missing? lets write tests for it so that + when someone gets to implementing it, he knows what to do. Repeat" + project + you mean creating an automated testing framework? + this is actually a task I want to add for this year, if I get + around to it :-) + yeah I'd very much want to apply for that one + cuz I've never done Kernel hacking before, but I know that with + the right tasks like "test VM functionality", I would be able to write up + the automated tests and hopefully learn more about what breaks/makes the + kernel + (and it would make implementing the feature much less hand-wavy + and more correct) + antrik, I believe the framework(CUnit right?) exists, but we just + need to write the tests. + do you have prior experience implementing automated tests? + lots of tests! + yes, in Java mostly, but I've played around with CUnit + ah, great :-) + here's what I know from experience: 1) write basic tests. 2) + write ones that test multiple features 3) stress test [option 4) + benchmark and record to DB] + well, what we'd rather need is to fix the issues we already know + from the existing testsuites :) + +[[GSoC project propsal|community/gsoc/project_ideas/testsuites]]. + + youpi, true, and I think I should check what's available in way + of tests, but if the tests are "all or nothing" then it becomes really + hard to fix your mistakes + they're not all or nothing + youpi: the existing testsuites don't test specific system features + libc ones do + we could also check posixtestsuite which does too + +[[open_issues/open_posix_test_suite]]. + + AFAIK libc has very few failing tests + +[[open_issues/glibc_testsuite]]. + + err, like twenty? + € grep -v '^#' expected-results-i486-gnu-libc | wc -l + 67 + nope, even more + oh, sorry, I confused it with coreutils + plus the binutils ones, i guess + yes + +[[open_issues/binutils#weak]]. + + anyways, I don't think relying on external testsuites for + regression testing is a good plan + also, that doesn't cover unit testing at all + why ? + sure there can be unit testing at the translator etc. level + if we want to implement test-driven development, and/or do serious + refactoring without too much risk, we need a test harness where we can + add specific tests as needed + but more than often, the issues are at the libc / etc. level + because of a combination o fthings at the translator level, which unit + testing won't find out + * nixness yewzzz! + sure unit testing can find them out. if they're good "unit" tests + the problem is that you don't necessarily know what "good" means + e.g. for posix correctness + since it's not posix + but if they're composite clever tests, then you lose that + granularity + youpi, is that a blackbox test intended to be run at the very end + to check if you're posix compliant? + also, if we have our own test harness, we can run tests + automatically as part of the build process, which is a great plus IMHO + nixness: "that" = ? + oh nvm, I thought there was a test stuie called "posix + correctness" + there's the posixtestsuite yes + it's an external one however + antrik: being external doesn't mean we can't invoke it + automatically as part of the build process when it's available + youpi, but being internal means it can test the inner workings of + certain modules that you are unsure of, and not just the interface + sure, that's why I said it'd be useful too + but as I said too, most bugs I've seen were not easy to find out at + the unit level + but rather at the libc level + of course we can integrate external tests if they exist and are + suitable. but that that doesn't preclude adding our own ones too. in + either case, that integration work has to be done too + again, I've never said I was against internal testsuite + also, the major purpose of a test suite is checking known + behaviour. a low-level test won't directly point to a POSIX violation; + but it will catch any changes in behaviour that would lead to one + what I said is that it will be hard to write them tight enough to + find bugs + again, the problem is knowing what will lead to a POSIX violation + it's long work + while libc / posixtestsuite / etc. already do that + *any* unexpected change in behaviour is likely to cause bugs + somewher + but WHAT is "expected" ? + THAT is the issue + and libc/posixtessuite do know that + at the translator level we don't really + see the recent post about link() + +[link(dir,name) should fail with +EPERM](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00007.html) + + in my memory jkoenig pointed it out for a lot of such calls + and that issue is clearly not known at the translator level + so you're saying that the tests have to be really really + low-level, and work our way up? + I'm saying that the translator level tests will be difficult to + write + why isn't it known at the translator level? if it's a translator + (not libc) bug, than obviously the translator must return something wrong + at some point, and that's something we can check + because you'll have to know all the details of the combinations + used in libc, to know whether they'll lead to posix issues + antrik: sure, but how did we detect that that was unexpected + behavior? + because of a glib test + at the translator level we didn't know it was an unexpected + behavior + gnulib actually + and if you had asked me, I wouldn't have known + again, we do *not* write a test suite to find existing bugs + right, took one for the other + doesn't really matter actually + antrik: ok, I don't care then + we write a test suite to prevent future bugs, or track status of + known bugs + (don't care *enough* for now, I mean) + hmm, so write something up antrik for GSoC :) and I'll be sure to + apply + now that we know some translators return a wrong error status in a + particular situation, we can write a test that checks exactly this error + status. that way we know when it is fixed, and also when it breaks again + nixness: great :-) + sweet. that kind of thing would also need a db backend + nixness: BTW, if you have a good idea, you can send an application + for it even if it's not listed among the proposed tasks... + so you don't strictly need a writeup from me to apply for this :-) + antrik, I'll keep that in mind, but I'll also be checking your + draft page + oh cool :) + (and it's a well known fact that those projects which students + proposed themselfs tend to be the most successful ones :-) ) + * nixness draft initiated + youpi: BTW, I'm surprised that you didn't mention libc testsuite + before... working up from there is probably a more effective plan than + starting with high-level test suites like Python etc... + wasn't it already in the gsoc proposal? + bummer + nope + +freenode, #hurd channel, 2011-03-06: + + how's the hurd coding workflow, typically? + +*nixness* -> *foocraft*. + + we're discussing how TDD can work with Hurd (or general kernel + development) on #osdev + so what I wanted to know, since I haven't dealt much with GNU + Hurd, is how do you guys go about coding, in this case + Our current workflow scheme is... well... is... + Someone wants to work on something, or spots a bug, then works + on it, submits a patch, and 0 to 10 years later it is applied. + Roughly. + hmm so there's no indicator of whether things broke with that + patch + and how low do you think we can get with tests? A friend of mine + was telling me that with kernel dev, you really don't know whether, for + instance, the stack even exists, and a lot of things that I, as a + programmer, can assume while writing code break when it comes to writing + kernel code + Autotest seems promising + +See autotest link given above. + + but in any case, coming up with the testing framework that + doesn't break when the OS itself breaks is hard, it seems + not sure if autotest isolates the mistakes in the os from + finding their way in the validity of the tests themselves + it could be interesting to have scripts that automatically start a + sub-hurd to do the tests + +[[hurd/subhurd#unit_testing]]. + + foocraft: To answer one of your earlier questions: you can do + really low-level testing. Like testing Mach's message passing. A + million times. The questions is whether that makes sense. And / or if + it makes sense to do that as part of a unit testing framework. Or rather + do such things manually once you suspect an error somewhere. + The reason for the latter may be that Mach IPC is already + heavily tested during normal system operation. + And yet, there still may be (there are, surely) bugs. + But I guess that you have to stop at some (arbitrary?) level. + so we'd assume it works, and test from there + Otherwise you'd be implementing the exact counter-part of what + you're testing. + Which may be what you want, or may be not. Or it may just not + be feasible. + maybe the testing framework should have dependencies + which we can automate using make, and phony targets that run + tests + so everyone writes a test suite and says that it depends on A + and B working correctly + then it'd go try to run the tests for A etc. + Hmm, isn't that -- on a high level -- have you have by + different packages? For example, the perl testsuite depends (inherently) + on glibc working properly. A perl program's testsuite depends on perl + working properly. + yeah, but afaik, the ordering is done by hand + +freenode, #hurd channel, 2011-03-07: + + actually, I think for most tests it would be better not to use a + subhurd... that leads precisely to the problem that if something is + broken, you might have a hard time running the tests at all :-) + foocraft: most of the Hurd code isn't really low-level. you can + use normal debugging and testing methods + gnumach of course *does* have some low-level stuff, so if you add + unit tests to gnumach too, you might run into issues :-) + tschwinge: I think testing IPC is a good thing. as I already said, + automated testing is *not* to discover existing but unknown bugs, but to + prevent new ones creeping in, and tracking progress on known bugs + tschwinge: I think you are confusing unit testing and regression + testing. http://www.bddebian.com/~hurd-web/open_issues/unit_testing/ + talks about unit testing, but a lot (most?) of it is actually about + regression tests... + antrik: That may certainly be -- I'm not at all an expert in + this, and just generally though that some sort of automated testing is + needed, and thus started collecting ideas. + antrik: You're of course invited to fix that. + +IRC, freenode, #hurd, 2011-03-08 + +(After discussing the [[open_issues/anatomy_of_a_hurd_system]].) + + so that's what your question is actually about? + so what I would imagine is a set of only-this-server tests for + each server, and then we can have fun adding composite tests + thus making debugging the composite scenarios a bit less tricky + indeed + and if you were trying to pass a composite test, it would also + help knowing that you still didn't break the server-only test + there are so many different things that can be tested... the + summer will only suffice to dip into this really :-) + yeah, I'm designing my proposal to focus on 1) make/use a + testing framework that fits the Hurd case very well 2) write some tests + and docs on how to write good tests + well, doesn't have to be *one* framework... unit testing and + regression testing are quite different things, which can be covered by + different frameworks diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index feda3be4..1378be85 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -8,7 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -This task may be suitable for [[community/GSoC]][[!tag gsoc-task]]. +This task may be suitable for [[community/GSoC]]: +[[community/gsoc/project_ideas/testing_framework]] --- @@ -80,263 +81,6 @@ abandoned). # Discussion -freenode, #hurd channel, 2011-03-05: - - what about testing though? - like sort of "what's missing? lets write tests for it so that - when someone gets to implementing it, he knows what to do. Repeat" - project - you mean creating an automated testing framework? - this is actually a task I want to add for this year, if I get - around to it :-) - yeah I'd very much want to apply for that one - cuz I've never done Kernel hacking before, but I know that with - the right tasks like "test VM functionality", I would be able to write up - the automated tests and hopefully learn more about what breaks/makes the - kernel - (and it would make implementing the feature much less hand-wavy - and more correct) - antrik, I believe the framework(CUnit right?) exists, but we just - need to write the tests. - do you have prior experience implementing automated tests? - lots of tests! - yes, in Java mostly, but I've played around with CUnit - ah, great :-) - here's what I know from experience: 1) write basic tests. 2) - write ones that test multiple features 3) stress test [option 4) - benchmark and record to DB] - well, what we'd rather need is to fix the issues we already know - from the existing testsuites :) - -[[GSoC project propsal|community/gsoc/project_ideas/testsuites]]. - - youpi, true, and I think I should check what's available in way - of tests, but if the tests are "all or nothing" then it becomes really - hard to fix your mistakes - they're not all or nothing - youpi: the existing testsuites don't test specific system features - libc ones do - we could also check posixtestsuite which does too - -[[open_issues/open_posix_test_suite]]. - - AFAIK libc has very few failing tests - -[[open_issues/glibc_testsuite]]. - - err, like twenty? - € grep -v '^#' expected-results-i486-gnu-libc | wc -l - 67 - nope, even more - oh, sorry, I confused it with coreutils - plus the binutils ones, i guess - yes - -[[open_issues/binutils#weak]]. - - anyways, I don't think relying on external testsuites for - regression testing is a good plan - also, that doesn't cover unit testing at all - why ? - sure there can be unit testing at the translator etc. level - if we want to implement test-driven development, and/or do serious - refactoring without too much risk, we need a test harness where we can - add specific tests as needed - but more than often, the issues are at the libc / etc. level - because of a combination o fthings at the translator level, which unit - testing won't find out - * nixness yewzzz! - sure unit testing can find them out. if they're good "unit" tests - the problem is that you don't necessarily know what "good" means - e.g. for posix correctness - since it's not posix - but if they're composite clever tests, then you lose that - granularity - youpi, is that a blackbox test intended to be run at the very end - to check if you're posix compliant? - also, if we have our own test harness, we can run tests - automatically as part of the build process, which is a great plus IMHO - nixness: "that" = ? - oh nvm, I thought there was a test stuie called "posix - correctness" - there's the posixtestsuite yes - it's an external one however - antrik: being external doesn't mean we can't invoke it - automatically as part of the build process when it's available - youpi, but being internal means it can test the inner workings of - certain modules that you are unsure of, and not just the interface - sure, that's why I said it'd be useful too - but as I said too, most bugs I've seen were not easy to find out at - the unit level - but rather at the libc level - of course we can integrate external tests if they exist and are - suitable. but that that doesn't preclude adding our own ones too. in - either case, that integration work has to be done too - again, I've never said I was against internal testsuite - also, the major purpose of a test suite is checking known - behaviour. a low-level test won't directly point to a POSIX violation; - but it will catch any changes in behaviour that would lead to one - what I said is that it will be hard to write them tight enough to - find bugs - again, the problem is knowing what will lead to a POSIX violation - it's long work - while libc / posixtestsuite / etc. already do that - *any* unexpected change in behaviour is likely to cause bugs - somewher - but WHAT is "expected" ? - THAT is the issue - and libc/posixtessuite do know that - at the translator level we don't really - see the recent post about link() - -[link(dir,name) should fail with -EPERM](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00007.html) - - in my memory jkoenig pointed it out for a lot of such calls - and that issue is clearly not known at the translator level - so you're saying that the tests have to be really really - low-level, and work our way up? - I'm saying that the translator level tests will be difficult to - write - why isn't it known at the translator level? if it's a translator - (not libc) bug, than obviously the translator must return something wrong - at some point, and that's something we can check - because you'll have to know all the details of the combinations - used in libc, to know whether they'll lead to posix issues - antrik: sure, but how did we detect that that was unexpected - behavior? - because of a glib test - at the translator level we didn't know it was an unexpected - behavior - gnulib actually - and if you had asked me, I wouldn't have known - again, we do *not* write a test suite to find existing bugs - right, took one for the other - doesn't really matter actually - antrik: ok, I don't care then - we write a test suite to prevent future bugs, or track status of - known bugs - (don't care *enough* for now, I mean) - hmm, so write something up antrik for GSoC :) and I'll be sure to - apply - now that we know some translators return a wrong error status in a - particular situation, we can write a test that checks exactly this error - status. that way we know when it is fixed, and also when it breaks again - nixness: great :-) - sweet. that kind of thing would also need a db backend - nixness: BTW, if you have a good idea, you can send an application - for it even if it's not listed among the proposed tasks... - so you don't strictly need a writeup from me to apply for this :-) - antrik, I'll keep that in mind, but I'll also be checking your - draft page - oh cool :) - (and it's a well known fact that those projects which students - proposed themselfs tend to be the most successful ones :-) ) - * nixness draft initiated - youpi: BTW, I'm surprised that you didn't mention libc testsuite - before... working up from there is probably a more effective plan than - starting with high-level test suites like Python etc... - wasn't it already in the gsoc proposal? - bummer - nope - -freenode, #hurd channel, 2011-03-06: - - how's the hurd coding workflow, typically? - -*nixness* -> *foocraft*. - - we're discussing how TDD can work with Hurd (or general kernel - development) on #osdev - so what I wanted to know, since I haven't dealt much with GNU - Hurd, is how do you guys go about coding, in this case - Our current workflow scheme is... well... is... - Someone wants to work on something, or spots a bug, then works - on it, submits a patch, and 0 to 10 years later it is applied. - Roughly. - hmm so there's no indicator of whether things broke with that - patch - and how low do you think we can get with tests? A friend of mine - was telling me that with kernel dev, you really don't know whether, for - instance, the stack even exists, and a lot of things that I, as a - programmer, can assume while writing code break when it comes to writing - kernel code - Autotest seems promising - -See autotest link given above. - - but in any case, coming up with the testing framework that - doesn't break when the OS itself breaks is hard, it seems - not sure if autotest isolates the mistakes in the os from - finding their way in the validity of the tests themselves - it could be interesting to have scripts that automatically start a - sub-hurd to do the tests - -[[hurd/subhurd#unit_testing]]. - - foocraft: To answer one of your earlier questions: you can do - really low-level testing. Like testing Mach's message passing. A - million times. The questions is whether that makes sense. And / or if - it makes sense to do that as part of a unit testing framework. Or rather - do such things manually once you suspect an error somewhere. - The reason for the latter may be that Mach IPC is already - heavily tested during normal system operation. - And yet, there still may be (there are, surely) bugs. - But I guess that you have to stop at some (arbitrary?) level. - so we'd assume it works, and test from there - Otherwise you'd be implementing the exact counter-part of what - you're testing. - Which may be what you want, or may be not. Or it may just not - be feasible. - maybe the testing framework should have dependencies - which we can automate using make, and phony targets that run - tests - so everyone writes a test suite and says that it depends on A - and B working correctly - then it'd go try to run the tests for A etc. - Hmm, isn't that -- on a high level -- have you have by - different packages? For example, the perl testsuite depends (inherently) - on glibc working properly. A perl program's testsuite depends on perl - working properly. - yeah, but afaik, the ordering is done by hand - -freenode, #hurd channel, 2011-03-07: - - actually, I think for most tests it would be better not to use a - subhurd... that leads precisely to the problem that if something is - broken, you might have a hard time running the tests at all :-) - foocraft: most of the Hurd code isn't really low-level. you can - use normal debugging and testing methods - gnumach of course *does* have some low-level stuff, so if you add - unit tests to gnumach too, you might run into issues :-) - tschwinge: I think testing IPC is a good thing. as I already said, - automated testing is *not* to discover existing but unknown bugs, but to - prevent new ones creeping in, and tracking progress on known bugs - tschwinge: I think you are confusing unit testing and regression - testing. http://www.bddebian.com/~hurd-web/open_issues/unit_testing/ - talks about unit testing, but a lot (most?) of it is actually about - regression tests... - antrik: That may certainly be -- I'm not at all an expert in - this, and just generally though that some sort of automated testing is - needed, and thus started collecting ideas. - antrik: You're of course invited to fix that. - -IRC, freenode, #hurd, 2011-03-08 - -(After discussing the [[anatomy_of_a_hurd_system]].) - - so that's what your question is actually about? - so what I would imagine is a set of only-this-server tests for - each server, and then we can have fun adding composite tests - thus making debugging the composite scenarios a bit less tricky - indeed - and if you were trying to pass a composite test, it would also - help knowing that you still didn't break the server-only test - there are so many different things that can be tested... the - summer will only suffice to dip into this really :-) - yeah, I'm designing my proposal to focus on 1) make/use a - testing framework that fits the Hurd case very well 2) write some tests - and docs on how to write good tests - well, doesn't have to be *one* framework... unit testing and - regression testing are quite different things, which can be covered by - different frameworks +See the [[GSoC project idea|community/gsoc/project_ideas/testing_framework]]'s +[[discussion +subpage|community/gsoc/project_ideas/testing_framework/discussion]]. -- cgit v1.2.3 From 511bcd2ac7194ca47c33e45131479b8bfdef33a9 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 01:55:07 +0100 Subject: Link to google-coredumper library. --- open_issues/crash_server.mdwn | 8 +++++++- open_issues/debugging.mdwn | 3 +++ open_issues/gdb_gcore.mdwn | 5 ++++- 3 files changed, 14 insertions(+), 2 deletions(-) (limited to 'open_issues') diff --git a/open_issues/crash_server.mdwn b/open_issues/crash_server.mdwn index d97f5458..7ed4afbf 100644 --- a/open_issues/crash_server.mdwn +++ b/open_issues/crash_server.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -187,3 +188,8 @@ one... /home/tschwinge/tmp/gnumach/gnumach-1-branch-Xen-branch.build/../gnumach-1-branch-Xen-branch/kern/ipc_kobject.c:76 mach_msg_trap /home/tschwinge/tmp/gnumach/gnumach-1-branch-Xen-branch.build/../gnumach-1-branch-Xen-branch/ipc/mach_msg.c:1367 + +--- + +If someone is working in this area, they may want to have a look at +[[GDB_gcore]], and port , too. diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn index e087f484..e5fbf7a0 100644 --- a/open_issues/debugging.mdwn +++ b/open_issues/debugging.mdwn @@ -42,6 +42,9 @@ We have debugging infrastructure. For example: Continues: , which introduces . + * [[crash_server}}, [[GDB_gcore]], + + * [[community/gsoc/project_ideas/libdiskfs_locking]] * , or -- diff --git a/open_issues/gdb_gcore.mdwn b/open_issues/gdb_gcore.mdwn index 7d4980f1..69211ac0 100644 --- a/open_issues/gdb_gcore.mdwn +++ b/open_issues/gdb_gcore.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -21,3 +21,6 @@ GDB's `gcore` command doesn't work / needs to be implemented / ported in GDB: /media/data/home/tschwinge/core.cA0ICY:2: Error in sourced command file: Undefined command: "gcore". Try "help". gcore: failed to create core.8371 + +If someone is working in this area, they may want to port +, too. -- cgit v1.2.3 From 2eda0c930873e95212b70fe4ca0d6b14274506a6 Mon Sep 17 00:00:00 2001 From: antrik Date: Sat, 26 Mar 2011 19:10:49 +0100 Subject: open_issues/pfinet_vs_system_time_changes: typo --- open_issues/pfinet_vs_system_time_changes.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'open_issues') diff --git a/open_issues/pfinet_vs_system_time_changes.mdwn b/open_issues/pfinet_vs_system_time_changes.mdwn index a9e1e242..714c8784 100644 --- a/open_issues/pfinet_vs_system_time_changes.mdwn +++ b/open_issues/pfinet_vs_system_time_changes.mdwn @@ -15,7 +15,7 @@ IRC, unknown channel, unknown date. I did a sudo date... and the machine hangs -This was very likely as misdiagnosis: +This was very likely a misdiagnosis: IRC, freenode, #hurd, 2011-03-25 -- cgit v1.2.3 From 88f00eed14d898e233677b6cc73c0f24786a9fde Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 21:02:42 +0100 Subject: open_issues/file_system_exerciser: New. --- community/gsoc/project_ideas/testsuites.mdwn | 3 +++ open_issues/file_system_exerciser.mdwn | 15 +++++++++++++++ 2 files changed, 18 insertions(+) create mode 100644 open_issues/file_system_exerciser.mdwn (limited to 'open_issues') diff --git a/community/gsoc/project_ideas/testsuites.mdwn b/community/gsoc/project_ideas/testsuites.mdwn index 8100bfb7..f5ee2084 100644 --- a/community/gsoc/project_ideas/testsuites.mdwn +++ b/community/gsoc/project_ideas/testsuites.mdwn @@ -19,6 +19,9 @@ some of the tests fail on the Hurd in general. There is also the [[Open POSIX Testsuite|open_issues/open_posix_test_suite]] which is more of a whole system interface testing suite. +Then, there is the [[open_issues/File_System_Exerciser]] which we can use to +test our file system servers for conformity. + While in some cases these might point to wrong usage of system interfaces, most of the time such failures are actually caused by shortcomings in Hurd's implementation of these interfaces. These shortcomings are often not obvious in normal use, diff --git a/open_issues/file_system_exerciser.mdwn b/open_issues/file_system_exerciser.mdwn new file mode 100644 index 00000000..4277e5e7 --- /dev/null +++ b/open_issues/file_system_exerciser.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +Test our file system implementations with the File System Exerciser. + + * -- cgit v1.2.3 From 172b698e79c56234fca2c95ca1bdf6aabd18bdcd Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 21:03:30 +0100 Subject: open_issues/debian_cross_toolchain: New. --- open_issues/debian_cross_toolchain.mdwn | 15 +++++++++++++++ open_issues/nightly_builds.mdwn | 4 +++- open_issues/nightly_builds_deb_packages.mdwn | 6 +++++- 3 files changed, 23 insertions(+), 2 deletions(-) create mode 100644 open_issues/debian_cross_toolchain.mdwn (limited to 'open_issues') diff --git a/open_issues/debian_cross_toolchain.mdwn b/open_issues/debian_cross_toolchain.mdwn new file mode 100644 index 00000000..e0665466 --- /dev/null +++ b/open_issues/debian_cross_toolchain.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +Have a look at the Debian *Cross Toolchain* project, +, +. diff --git a/open_issues/nightly_builds.mdwn b/open_issues/nightly_builds.mdwn index 506697bb..5d1257fb 100644 --- a/open_issues/nightly_builds.mdwn +++ b/open_issues/nightly_builds.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,6 +15,8 @@ Resources: * [[toolchain/cross-gnu]] + * [[Debian_Cross_Toolchain]] + * As reported in the [[news/2010-05-31]] news, there's Hydra doing nightly builds / Nix packages. diff --git a/open_issues/nightly_builds_deb_packages.mdwn b/open_issues/nightly_builds_deb_packages.mdwn index 9f5e2373..11fc4c79 100644 --- a/open_issues/nightly_builds_deb_packages.mdwn +++ b/open_issues/nightly_builds_deb_packages.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -24,4 +24,8 @@ There is infrastructure available to test whole OS installations. --- +[[Debian_Cross_Toolchain]] for cross-building? + +--- + See also [[nightly_builds]]. -- cgit v1.2.3 From 4f90536eeffb05781f7f33c153f17041eeeeb482 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 21:04:06 +0100 Subject: open_issues/multithreading: Concurrency Kit. --- open_issues/multithreading.mdwn | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) (limited to 'open_issues') diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn index 39203333..addc29c3 100644 --- a/open_issues/multithreading.mdwn +++ b/open_issues/multithreading.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,21 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -Hurd servers / VFS libraries are multithreaded, roughly using one thread per +Hurd servers / VFS libraries are multithreaded. + + +# Implementation + + * well-known threading libraries + + * [[hurd/libthreads]] + + * [[hurd/libpthread]] + + +# Design + +Roughly using one thread per incoming request. This is not the best approach: it doesn't really make sense to scale the number of worker threads with the number of incoming requests, but instead they should be scaled according to the backends' characteristics. @@ -18,7 +32,9 @@ instead they should be scaled according to the backends' characteristics. The [[hurd/Critique]] should have some more on this. -Alternative approaches: +# Alternative approaches: + + * * Continuation-passing style -- cgit v1.2.3 From 3268d34951db85a913d227b637759c57e6d13f37 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 26 Mar 2011 21:14:54 +0100 Subject: open_issues/performance/fork: Suggest some reading about Windows / Cygwin issues. --- open_issues/performance/fork.mdwn | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) (limited to 'open_issues') diff --git a/open_issues/performance/fork.mdwn b/open_issues/performance/fork.mdwn index 2748be53..5ceb6455 100644 --- a/open_issues/performance/fork.mdwn +++ b/open_issues/performance/fork.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,3 +14,24 @@ Our [[`fork` implementation|glibc/fork]] is nontrivial. To do: hard numbers. [[Microbenchmarks]]? + + +# Windows / Cygwin + + * + + * + + In particular, *5.6. Process Creation*. + + * + + * + + > Cygwin has recently adopted something called the "cygwin heap". This is + > an internal heap that is inherited by forked/execed children. It + > consists of process specific information that should be inherited. So + > things like the file descriptor table, the current working directory, and + > the chroot value live there. + + * -- cgit v1.2.3 From 29384d59a2503c5a3cc46ce12c95bedc8653986d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 27 Mar 2011 11:22:22 +0200 Subject: open_issues/sync_but_still_unclean_filesystem: Link to Savannah bug report. --- open_issues/sync_but_still_unclean_filesystem.mdwn | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) (limited to 'open_issues') diff --git a/open_issues/sync_but_still_unclean_filesystem.mdwn b/open_issues/sync_but_still_unclean_filesystem.mdwn index f1fbb4e0..83c7951e 100644 --- a/open_issues/sync_but_still_unclean_filesystem.mdwn +++ b/open_issues/sync_but_still_unclean_filesystem.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,9 +10,19 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gnumach open_issue_hurd]] +Also filed as [[!GNU_Savannah_bug 29292]]. + \#hurd, 2010, end of May / beginning of June [runnign sync, but sill unclean filesystem at next boot] - guillem: when libpager syncs an object, it sends an m_o_lock_request and waits (if the synchronous argument was specified) for a m_o_lock_completed. But m_o_lock_completed only means that dirty pages have been sent to the translator, and this one still needs to write them to the backing storage - guillem: there's no problem if sync() returns before actually writting the changes to disk, but this also happens when shutting down the translator - guillem: in theory, locking mechanisms in libpager should prevent this from happening by keeping track of write operations, but this seems to fail in some situations + guillem: when libpager syncs an object, it sends an m_o_lock_request + and waits (if the synchronous argument was specified) for a + m_o_lock_completed. But m_o_lock_completed only means that dirty pages + have been sent to the translator, and this one still needs to write them + to the backing storage + guillem: there's no problem if sync() returns before actually + writting the changes to disk, but this also happens when shutting down + the translator + guillem: in theory, locking mechanisms in libpager should prevent + this from happening by keeping track of write operations, but this seems + to fail in some situations -- cgit v1.2.3