diff options
Diffstat (limited to 'community')
| -rw-r--r-- | community/gsoc/project_ideas.mdwn | 6 | ||||
| -rw-r--r-- | community/gsoc/project_ideas/disk_io_performance.mdwn | 48 | ||||
| -rw-r--r-- | community/gsoc/project_ideas/dtrace.mdwn | 49 | ||||
| -rw-r--r-- | community/gsoc/project_ideas/libdiskfs_locking.mdwn | 47 | ||||
| -rw-r--r-- | community/gsoc/project_ideas/procfs.mdwn | 45 | ||||
| -rw-r--r-- | community/gsoc/project_ideas/valgrind.mdwn | 81 | ||||
| -rw-r--r-- | community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn | 13 |
7 files changed, 282 insertions, 7 deletions
diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index da247847..030c18f3 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -89,10 +89,10 @@ Here are these pages' content: [[!inline pages="community/gsoc/project_ideas/server_overriding" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/tcp_ip_stack" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/nfs" show=0 feeds=no actions=yes]] -[[!inline pages="open_issues/locking" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/pthreads" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/sound" show=0 feeds=no actions=yes]] -[[!inline pages="open_issues/performance/io_system" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/disk_io_performance" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/vm_tuning" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/mtab" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/gnumach_cleanup" show=0 feeds=no actions=yes]] @@ -114,4 +114,4 @@ Here are these pages' content: [[!inline pages="open_issues/unit_testing" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/libcap" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/xattr" show=0 feeds=no actions=yes]] -[[!inline pages="open_issues/valgrind" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/valgrind" show=0 feeds=no actions=yes]] diff --git a/community/gsoc/project_ideas/disk_io_performance.mdwn b/community/gsoc/project_ideas/disk_io_performance.mdwn new file mode 100644 index 00000000..b6f223c8 --- /dev/null +++ b/community/gsoc/project_ideas/disk_io_performance.mdwn @@ -0,0 +1,48 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Disk I/O Performance Tuning"]] + +The most obvious reason for the Hurd feeling slow compared to mainstream +systems like GNU/Linux, is a low I/O system performance, in particular very +slow hard disk access. + +The reason for this slowness is lack and/or bad implementation of common +optimization techniques, like scheduling reads and writes to minimize head +movement; effective block caching; effective reads/writes to partial blocks; +[[reading/writing multiple blocks at once|clustered_page_faults]]; and +[[read-ahead]]. The +[[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some +optimizations at a higher logical level. + +The goal of this project is to analyze the current situation, and implement/fix +various optimizations, to achieve significantly better disk performance. It +requires understanding the data flow through the various layers involved in +disk access on the Hurd ([[filesystem|hurd/virtual_file_system]], +[[pager|hurd/libpager]], driver), and general experience with +optimizing complex systems. That said, the killing feature we are definitely +missing is the [[read-ahead]], and even a very simple implementation would bring +very big performance speedups. + +Here are some real testcases: + + * [[binutils_ld_64ksec]]; + + * running the Git testsuite which is mostly I/O bound; + + * use [[TopGit]] on a non-toy repository. + + +Possible mentors: Samuel Thibault (youpi) + +Exercise: Look through all the code involved in disk I/O, and try something +easy to improve. It's quite likely though that you will find nothing obvious -- +in this case, please contact us about a different exercise task. diff --git a/community/gsoc/project_ideas/dtrace.mdwn b/community/gsoc/project_ideas/dtrace.mdwn new file mode 100644 index 00000000..252c303a --- /dev/null +++ b/community/gsoc/project_ideas/dtrace.mdwn @@ -0,0 +1,49 @@ +[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="dtrace Support"]] + +One of the main problems of the current Hurd implementation is very poor +[[performance]]. While we have a bunch of ideas what could cause the performance +problems, these are mostly just guesses. Better understanding what really +causes bad performance is necessary to improve the situation. + +For that, we need tools for performance measurements. While all kinds of more +or less specific [[profiling]] tools could be conceived, the most promising and +generic approach seems to be a framework for logging certain events in the +running system (both in the microkernel and in the Hurd servers). This would +allow checking how much time is spent in certain modules, how often certain +situations occur, how things interact, etc. It could also prove helpful in +debugging some issues that are otherwise hard to find because of complex +interactions. + +The most popular framework for that is Sun's dtrace; but there might be others. +The student has to evaluate the existing options, deciding which makes most +sense for the Hurd; and implement that one. (Apple's implementation of dtrace +in their Mach-based kernel might be helpful here...) + +This project requires ability to evaluate possible solutions, and experience +with integrating existing components as well as low-level programming. + +Possible mentors: Samuel Thibault (youpi) + +Related: [[profiling]], [[LTTng]], [[SystemTap]] + +Exercise: In lack of a good exercise directly related to this task, just pick +one of the kernel-related or generally low-level tasks from the bug/task +trackers on savannah, and make a go at it. You might not be able to finish the +task in a limited amount of time, but you should at least be able to make a +detailed analysis of the issue. + +*Status*: Andei Barbu was working on +[SystemTap](http://csclub.uwaterloo.ca/~abarbu/hurd/) for GSoC 2008, but it +turned out too Linux-specific. He implemented kernel probes, but there is no +nice frontend yet. diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn new file mode 100644 index 00000000..0e38b6a0 --- /dev/null +++ b/community/gsoc/project_ideas/libdiskfs_locking.mdwn @@ -0,0 +1,47 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Fix libdiskfs Locking Issues"]] + +Every now and then, new locking issues are discovered in +[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays +these in fact seem to be the most often encountered cause of Hurd crashes +/ lockups. + +One of these could be traced +recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken +and not released in some cases. There is reason to believe that there are more +faulty paths causing these lockups. + +The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking +issues. To achieve this, some kind of test harness has to be implemented: For +example instrumenting the code to check locking correctness constantly at +runtime. Or implementing a [[unit testing]] framework that explicitly checks +locking in various code paths. (The latter could serve as a template for +implementing unit tests in other parts of the Hurd codebase...) + +(A [[systematic code review|security]] would probably suffice to find the +existing locking +issues; but it wouldn't document the work in terms of actual code produced, and +thus it's not suitable for a GSoC project...) + +This task requires experience with debugging locking issues in +[[multithreaded|multithreading]] applications. + +Tools have been written for automated [[code analysis]]; these can help to +locate and fix such errors. + +Possible mentors: Samuel Thibault (youpi) + +Exercise: If you could actually track down and fix one of the existing locking +errors before the end of the application process, that would be excellent. This +might be rather tough though, so probably you need to talk to us about an +alternative exercise task... diff --git a/community/gsoc/project_ideas/procfs.mdwn b/community/gsoc/project_ideas/procfs.mdwn new file mode 100644 index 00000000..d4760aae --- /dev/null +++ b/community/gsoc/project_ideas/procfs.mdwn @@ -0,0 +1,45 @@ +[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!meta title="procfs"]] + +Although there is no standard (POSIX or other) for the layout of the `/proc` +pseudo-filesystem, it turned out a very useful facility in GNU/Linux and other +systems, and many tools concerned with process management use it. (`ps`, `top`, +`htop`, `gtop`, `killall`, `pkill`, ...) + +Instead of porting all these tools to use [[hurd/libps]] (Hurd's official method for +accessing process information), they could be made to run out of the box, by +implementing a Linux-compatible `/proc` filesystem for the Hurd. + +The goal is to implement all `/proc` functionality needed for the various process +management tools to work. (On Linux, the `/proc` filesystem is used also for +debugging purposes; but this is highly system-specific anyways, so there is +probably no point in trying to duplicate this functionality as well...) + +The [[existing_partially_working_procfs_implementation|hurd/translator/procfs]] +can serve as a starting point, but needs to be largely rewritten. (It should +use [[hurd/libnetfs]] rather than [[hurd/libtrivfs]]; the data format needs to +change to be more Linux-compatible; and it needs adaptation to newer system +interfaces.) + +This project requires learning [[hurd/translator]] programming, and +understanding some of the internals of process management in the Hurd. It +should not be too hard coding-wise; and the task is very nicely defined by the +existing Linux `/proc` interface -- no design considerations necessary. + +**Note**: We already have several applications for this task. + +Possible mentors: Olaf Buddenhagen (antrik) + +Exercise: Add or fix one piece in the existing procfs translator. + +*Status*: Madhusudan.C.S has implemented a new, fully functional [[procfs|madhusudancs]] for +GSoC 2008. He is still working on some outstanding issues. diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn new file mode 100644 index 00000000..79c8bd1d --- /dev/null +++ b/community/gsoc/project_ideas/valgrind.mdwn @@ -0,0 +1,81 @@ +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Porting Valgrind to the Hurd"]] + +[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. +(And some other kinds of hard-to-find errors too.) +Aside from being useful for program development in general, +a Hurd port will help finding out why certain programs segfault on the Hurd, +although they work on Linux. +Even more importantly, it will help finding bugs in the Hurd servers themselfs. + +To keep track of memory use, +Valgrind however needs to know how each [[system call]] affects the validity of memory regions. +This knowledge is highly kernel-specific, +and thus Valgrind needs to be explicitely ported for every system. + +Such a port involves two major steps: +making Valgrind understand how kernel traps work in general on the system in question; +and how all the individual kernel calls affect memory. +The latter step is where most of the work is, +as the behaviour of each single [[system call]] needs to be described. + +Compared to Linux, +[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps. +Almost all [[system call]]s are implemented as [[RPC]]s instead -- +either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]]. +All RPCs use a pair of `mach_msg` invocations: +one to send a request message, and one to receive a reply. +However, while all RPCs use the same `mach_msg` trap, +the actual effect of the call varies greatly depending on which RPC is invoked -- +similar to the `ioctl` call on Linux. +Each request thus must be handled individually. + +Unlike `ioctl`, +the RPC invocations have explicit type information for the parameters though, +which can be retrieved from the message header. +By analyzing the parameters of the RPC reply message, +Valgrind can know exactly which memory regions are affected by that call, +even without specific knowledge of the RPC in question. +Thus implementing a general parser for the reply messages +will already give Valgrind a fairly good approximation of memory validity -- +without having to specify the exact semantic of each RPC by hand. + +While this should make Valgrind quite usable on the Hurd already, it's not perfect: +some RPCs might return a buffer that is only partially filled with valid data; +or some reply parameters might be optional, +and only contain valid data under certain conditions. +Such specific semantics can't be deduced from the message headers alone. +Thus for a complete port, +it will still be necessary to go through the list of all known RPCs, +and implement special handling in Valgrind for those RPCs that need it. + +The goal of this task is at minimum to make Valgrind grok Mach traps, +and to implement the generic RPC handler. +Ideally, specific handling for RPCs needing it should also be implemented. + +Completing this project will require digging into Valgrind's handling of [[system call]]s, +and into Hurd RPCs. +It is not an easy task, but a fairly predictable one -- +there shouldn't be any unexpected difficulties, +and no major design work is necessary. +It doesn't require any specific previous knowledge: +only good programming skills in general. +On the other hand, +the student will obtain a good understanding of Hurd RPCs while working on this task, +and thus perfect qualifications for Hurd development in general :-) + +Possible mentors: Samuel Thibault (youpi) + +Exercise: As a starter, +students can try to teach valgrind a couple of Linux ioctls, +as this will make them learn how to use the read/write primitives of valgrind. diff --git a/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn b/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn index d55527a7..00d09094 100644 --- a/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn +++ b/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn @@ -56,7 +56,9 @@ For that I want to use: ### Tarfs apt-get install zip libz-dev libbz2-dev - hg clone http://bitbucket.org/ArneBab/hurd-tarfs tarfs + git clone git://git.sv.gnu.org/hurd/incubator.git tarfs + cd tarfs/ + git checkout tarfs/master cd tarfs make make install @@ -86,9 +88,12 @@ For that I want to use: cd ../.. mkdir test - settrans -c test2 /usr/local/bin/nsmux test - tar -cf test/intro.tar repos/hurd_intro - ls test2/intro.tar,,tarfs + touch test/hello + settrans -ca test2 /usr/local/bin/nsmux test + # tar -cvf test/intro.tar repos/hurd_intro + cat test2/hello + cat test2/hello,,hello + # Hello, World! ### clisp |