gsoc/project_ideas: Restore all project ideas here

While there is certainly some overlap with other areas, it is *not*
acceptable to drop mentors and exercises from GSoC tasks, nor to add
random crap, nor do any other changes that make them less useful as GSoC
tasks -- and this is *not* obvious if they do not live in the
project_ideas namespace. It's also confusing in general.

I tried to preserve all valid changes to the task descriptions themself
-- though I might have messed up some things.

I did leave the now redundant entries in open_tasks in place. Not sure
how to deal with them. As the content is virtually identical anyways,
they probably should be just turned into stubs pointing here. Or don't
list them explicitely at all -- we point out in other places that GSoC
ideas are useful in other contexts too...

For the future, please refrain from reorganising things here without
prior discussion.

6 files changed, 273 insertions, 3 deletions

diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn
index da247847..030c18f3 100644
--- a/community/gsoc/project_ideas.mdwn
+++ b/community/gsoc/project_ideas.mdwn
@@ -89,10 +89,10 @@ Here are these pages' content:
 [[!inline pages="community/gsoc/project_ideas/server_overriding" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/tcp_ip_stack" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/nfs" show=0 feeds=no actions=yes]]
-[[!inline pages="open_issues/locking" show=0 feeds=no actions=yes]]
+[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/pthreads" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/sound" show=0 feeds=no actions=yes]]
-[[!inline pages="open_issues/performance/io_system" show=0 feeds=no actions=yes]]
+[[!inline pages="community/gsoc/project_ideas/disk_io_performance" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/vm_tuning" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/mtab" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/gnumach_cleanup" show=0 feeds=no actions=yes]]
@@ -114,4 +114,4 @@ Here are these pages' content:
 [[!inline pages="open_issues/unit_testing" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/libcap" show=0 feeds=no actions=yes]]
 [[!inline pages="community/gsoc/project_ideas/xattr" show=0 feeds=no actions=yes]]
-[[!inline pages="open_issues/valgrind" show=0 feeds=no actions=yes]]
+[[!inline pages="community/gsoc/project_ideas/valgrind" show=0 feeds=no actions=yes]]
diff --git a/community/gsoc/project_ideas/disk_io_performance.mdwn b/community/gsoc/project_ideas/disk_io_performance.mdwn
new file mode 100644
index 00000000..b6f223c8
--- /dev/null
+++ b/community/gsoc/project_ideas/disk_io_performance.mdwn
@@ -0,0 +1,48 @@
+[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
+Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="Disk I/O Performance Tuning"]]
+
+The most obvious reason for the Hurd feeling slow compared to mainstream
+systems like GNU/Linux, is a low I/O system performance, in particular very
+slow hard disk access.
+
+The reason for this slowness is lack and/or bad implementation of common
+optimization techniques, like scheduling reads and writes to minimize head
+movement; effective block caching; effective reads/writes to partial blocks;
+[[reading/writing multiple blocks at once|clustered_page_faults]]; and
+[[read-ahead]].  The
+[[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some
+optimizations at a higher logical level.
+
+The goal of this project is to analyze the current situation, and implement/fix
+various optimizations, to achieve significantly better disk performance. It
+requires understanding the data flow through the various layers involved in
+disk access on the Hurd ([[filesystem|hurd/virtual_file_system]],
+[[pager|hurd/libpager]], driver), and general experience with
+optimizing complex systems.  That said, the killing feature we are definitely
+missing is the [[read-ahead]], and even a very simple implementation would bring
+very big performance speedups.
+
+Here are some real testcases:
+
+  * [[binutils_ld_64ksec]];
+
+  * running the Git testsuite which is mostly I/O bound;
+
+  * use [[TopGit]] on a non-toy repository.
+
+
+Possible mentors: Samuel Thibault (youpi)
+
+Exercise: Look through all the code involved in disk I/O, and try something
+easy to improve. It's quite likely though that you will find nothing obvious --
+in this case, please contact us about a different exercise task.
diff --git a/community/gsoc/project_ideas/dtrace.mdwn b/community/gsoc/project_ideas/dtrace.mdwn
new file mode 100644
index 00000000..252c303a
--- /dev/null
+++ b/community/gsoc/project_ideas/dtrace.mdwn
@@ -0,0 +1,49 @@
+[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation,
+Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="dtrace Support"]]
+
+One of the main problems of the current Hurd implementation is very poor
+[[performance]]. While we have a bunch of ideas what could cause the performance
+problems, these are mostly just guesses. Better understanding what really
+causes bad performance is necessary to improve the situation.
+
+For that, we need tools for performance measurements. While all kinds of more
+or less specific [[profiling]] tools could be conceived, the most promising and
+generic approach seems to be a framework for logging certain events in the
+running system (both in the microkernel and in the Hurd servers). This would
+allow checking how much time is spent in certain modules, how often certain
+situations occur, how things interact, etc. It could also prove helpful in
+debugging some issues that are otherwise hard to find because of complex
+interactions.
+
+The most popular framework for that is Sun's dtrace; but there might be others.
+The student has to evaluate the existing options, deciding which makes most
+sense for the Hurd; and implement that one. (Apple's implementation of dtrace
+in their Mach-based kernel might be helpful here...)
+
+This project requires ability to evaluate possible solutions, and experience
+with integrating existing components as well as low-level programming.
+
+Possible mentors: Samuel Thibault (youpi)
+
+Related: [[profiling]], [[LTTng]], [[SystemTap]]
+
+Exercise: In lack of a good exercise directly related to this task, just pick
+one of the kernel-related or generally low-level tasks from the bug/task
+trackers on savannah, and make a go at it. You might not be able to finish the
+task in a limited amount of time, but you should at least be able to make a
+detailed analysis of the issue.
+
+*Status*: Andei Barbu was working on
+[SystemTap](http://csclub.uwaterloo.ca/~abarbu/hurd/) for GSoC 2008, but it
+turned out too Linux-specific. He implemented kernel probes, but there is no
+nice frontend yet.
diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn
new file mode 100644
index 00000000..0e38b6a0
--- /dev/null
+++ b/community/gsoc/project_ideas/libdiskfs_locking.mdwn
@@ -0,0 +1,47 @@
+[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
+Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="Fix libdiskfs Locking Issues"]]
+
+Every now and then, new locking issues are discovered in
+[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example.  Nowadays
+these in fact seem to be the most often encountered cause of Hurd crashes
+/ lockups.
+
+One of these could be traced
+recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken
+and not released in some cases.  There is reason to believe that there are more
+faulty paths causing these lockups.
+
+The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking
+issues. To achieve this, some kind of test harness has to be implemented: For
+example instrumenting the code to check locking correctness constantly at
+runtime. Or implementing a [[unit testing]] framework that explicitly checks
+locking in various code paths. (The latter could serve as a template for
+implementing unit tests in other parts of the Hurd codebase...)
+
+(A [[systematic code review|security]] would probably suffice to find the
+existing locking
+issues; but it wouldn't document the work in terms of actual code produced, and
+thus it's not suitable for a GSoC project...)
+
+This task requires experience with debugging locking issues in
+[[multithreaded|multithreading]] applications.
+
+Tools have been written for automated [[code analysis]]; these can help to
+locate and fix such errors.
+
+Possible mentors: Samuel Thibault (youpi)
+
+Exercise: If you could actually track down and fix one of the existing locking
+errors before the end of the application process, that would be excellent. This
+might be rather tough though, so probably you need to talk to us about an
+alternative exercise task...
diff --git a/community/gsoc/project_ideas/procfs.mdwn b/community/gsoc/project_ideas/procfs.mdwn
new file mode 100644
index 00000000..d4760aae
--- /dev/null
+++ b/community/gsoc/project_ideas/procfs.mdwn
@@ -0,0 +1,45 @@
+[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled
+[[GNU Free Documentation License|/fdl]]."]]"""]]
+
+[[!meta title="procfs"]]
+
+Although there is no standard (POSIX or other) for the layout of the `/proc`
+pseudo-filesystem, it turned out a very useful facility in GNU/Linux and other
+systems, and many tools concerned with process management use it. (`ps`, `top`,
+`htop`, `gtop`, `killall`, `pkill`, ...)
+
+Instead of porting all these tools to use [[hurd/libps]] (Hurd's official method for
+accessing process information), they could be made to run out of the box, by
+implementing a Linux-compatible `/proc` filesystem for the Hurd.
+
+The goal is to implement all `/proc` functionality needed for the various process
+management tools to work.  (On Linux, the `/proc` filesystem is used also for
+debugging purposes; but this is highly system-specific anyways, so there is
+probably no point in trying to duplicate this functionality as well...)
+
+The [[existing_partially_working_procfs_implementation|hurd/translator/procfs]]
+can serve as a starting point, but needs to be largely rewritten.  (It should
+use [[hurd/libnetfs]] rather than [[hurd/libtrivfs]]; the data format needs to
+change to be more Linux-compatible; and it needs adaptation to newer system
+interfaces.)
+
+This project requires learning [[hurd/translator]] programming, and
+understanding some of the internals of process management in the Hurd.  It
+should not be too hard coding-wise; and the task is very nicely defined by the
+existing Linux `/proc` interface -- no design considerations necessary.
+
+**Note**: We already have several applications for this task.
+
+Possible mentors: Olaf Buddenhagen (antrik)
+
+Exercise: Add or fix one piece in the existing procfs translator.
+
+*Status*: Madhusudan.C.S has implemented a new, fully functional [[procfs|madhusudancs]] for
+GSoC 2008. He is still working on some outstanding issues.
diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn
new file mode 100644
index 00000000..79c8bd1d
--- /dev/null
+++ b/community/gsoc/project_ideas/valgrind.mdwn
@@ -0,0 +1,81 @@
+[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation,
+Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="Porting Valgrind to the Hurd"]]
+
+[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors.
+(And some other kinds of hard-to-find errors too.)
+Aside from being useful for program development in general,
+a Hurd port will help finding out why certain programs segfault on the Hurd,
+although they work on Linux.
+Even more importantly, it will help finding bugs in the Hurd servers themselfs.
+
+To keep track of memory use,
+Valgrind however needs to know how each [[system call]] affects the validity of memory regions.
+This knowledge is highly kernel-specific,
+and thus Valgrind needs to be explicitely ported for every system.
+
+Such a port involves two major steps:
+making Valgrind understand how kernel traps work in general on the system in question;
+and how all the individual kernel calls affect memory.
+The latter step is where most of the work is,
+as the behaviour of each single [[system call]] needs to be described.
+
+Compared to Linux,
+[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps.
+Almost all [[system call]]s are implemented as [[RPC]]s instead --
+either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]].
+All RPCs use a pair of `mach_msg` invocations:
+one to send a request message, and one to receive a reply.
+However, while all RPCs use the same `mach_msg` trap,
+the actual effect of the call varies greatly depending on which RPC is invoked --
+similar to the `ioctl` call on Linux.
+Each request thus must be handled individually.
+
+Unlike `ioctl`,
+the RPC invocations have explicit type information for the parameters though,
+which can be retrieved from the message header.
+By analyzing the parameters of the RPC reply message,
+Valgrind can know exactly which memory regions are affected by that call,
+even without specific knowledge of the RPC in question.
+Thus implementing a general parser for the reply messages
+will already give Valgrind a fairly good approximation of memory validity --
+without having to specify the exact semantic of each RPC by hand.
+
+While this should make Valgrind quite usable on the Hurd already, it's not perfect:
+some RPCs might return a buffer that is only partially filled with valid data;
+or some reply parameters might be optional,
+and only contain valid data under certain conditions.
+Such specific semantics can't be deduced from the message headers alone.
+Thus for a complete port,
+it will still be necessary to go through the list of all known RPCs,
+and implement special handling in Valgrind for those RPCs that need it.
+
+The goal of this task is at minimum to make Valgrind grok Mach traps,
+and to implement the generic RPC handler.
+Ideally, specific handling for RPCs needing it should also be implemented.
+
+Completing this project will require digging into Valgrind's handling of [[system call]]s,
+and into Hurd RPCs.
+It is not an easy task, but a fairly predictable one --
+there shouldn't be any unexpected difficulties,
+and no major design work is necessary.
+It doesn't require any specific previous knowledge:
+only good programming skills in general.
+On the other hand,
+the student will obtain a good understanding of Hurd RPCs while working on this task,
+and thus perfect qualifications for Hurd development in general :-)
+
+Possible mentors: Samuel Thibault (youpi)
+
+Exercise: As a starter,
+students can try to teach valgrind a couple of Linux ioctls,
+as this will make them learn how to use the read/write primitives of valgrind.