Re-integrate GSoC pages with the non-GSoC world.

Remove duplicates, apart from procfs, which should rather be removed from the
GSoC items.

11 files changed, 14 insertions, 229 deletions

diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn
index ad59f962..21e09089 100644
--- a/open_issues/code_analysis.mdwn
+++ b/open_issues/code_analysis.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -40,7 +40,7 @@ analysis|performance]], [[formal_verification]], as well as general
 
       * <http://blog.llvm.org/2010/04/whats-wrong-with-this-code.html>
 
-  * [[Valgrind]]
+  * [[community/gsoc/project_ideas/Valgrind]]
 
   * [Smatch](http://smatch.sourceforge.net/)
 
diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn
index e66a086f..e087f484 100644
--- a/open_issues/debugging.mdwn
+++ b/open_issues/debugging.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -42,7 +42,7 @@ We have debugging infrastructure.  For example:
     Continues: <http://lwn.net/Articles/414264/>, which introduces
     <http://dmtcp.sourceforge.net/>.
 
-  * [[locking]]
+  * [[community/gsoc/project_ideas/libdiskfs_locking]]
 
   * <http://lwn.net/Articles/415728/>, or <http://lwn.net/Articles/415471/> --
     just two examples; there's a lot of such stuff for Linux.
diff --git a/open_issues/dtrace.mdwn b/open_issues/dtrace.mdwn
deleted file mode 100644
index cbac28fb..00000000
--- a/open_issues/dtrace.mdwn
+++ /dev/null
@@ -1,47 +0,0 @@
-[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation,
-Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-One of the main problems of the current Hurd implementation is very poor
-[[performance]]. While we have a bunch of ideas what could cause the performance
-problems, these are mostly just guesses. Better understanding what really
-causes bad performance is necessary to improve the situation.
-
-For that, we need tools for performance measurements. While all kinds of more
-or less specific [[profiling]] tools could be conceived, the most promising and
-generic approach seems to be a framework for logging certain events in the
-running system (both in the microkernel and in the Hurd servers). This would
-allow checking how much time is spent in certain modules, how often certain
-situations occur, how things interact, etc. It could also prove helpful in
-debugging some issues that are otherwise hard to find because of complex
-interactions.
-
-The most popular framework for that is Sun's dtrace; but there might be others.
-The student has to evaluate the existing options, deciding which makes most
-sense for the Hurd; and implement that one. (Apple's implementation of dtrace
-in their Mach-based kernel might be helpful here...)
-
-This project requires ability to evaluate possible solutions, and experience
-with integrating existing components as well as low-level programming.
-
-Possible mentors: Samuel Thibault (youpi)
-
-Related: [[profiling]], [[LTTng]], [[SystemTap]]
-
-Exercise: In lack of a good exercise directly related to this task, just pick
-one of the kernel-related or generally low-level tasks from the bug/task
-trackers on savannah, and make a go at it. You might not be able to finish the
-task in a limited amount of time, but you should at least be able to make a
-detailed analysis of the issue.
-
-*Status*: Andei Barbu was working on
-[SystemTap](http://csclub.uwaterloo.ca/~abarbu/hurd/) for GSoC 2008, but it
-turned out too Linux-specific. He implemented kernel probes, but there is no
-nice frontend yet.
diff --git a/open_issues/locking.mdwn b/open_issues/locking.mdwn
deleted file mode 100644
index 6e22f887..00000000
--- a/open_issues/locking.mdwn
+++ /dev/null
@@ -1,40 +0,0 @@
-[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
-Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-[[!tag open_issue_hurd gsoc-task]]
-
-Every now and then, new locking issues are discovered in
-[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example.  Nowadays
-these in fact seem to be the most often encountered cause of Hurd crashes
-/ lockups.
-
-One of these could be traced
-recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken
-and not released in some cases.  There is reason to believe that there are more
-faulty paths causing these lockups.
-
-The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking
-issues. To achieve this, some kind of test harness has to be implemented: For
-example instrumenting the code to check locking correctness constantly at
-runtime. Or implementing a [[unit testing]] framework that explicitly checks
-locking in various code paths. (The latter could serve as a template for
-implementing unit tests in other parts of the Hurd codebase...)
-
-(A [[systematic code review|security]] would probably suffice to find the
-existing locking
-issues; but it wouldn't document the work in terms of actual code produced, and
-thus it's not suitable for a GSoC project...)
-
-This task requires experience with debugging locking issues in
-[[multithreaded|multithreading]] applications.
-
-Tools have been written for automated [[code analysis]]; these can help to
-locate and fix such errors.
diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn
index 9b3701b3..eb9f3f8a 100644
--- a/open_issues/performance.mdwn
+++ b/open_issues/performance.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -18,7 +18,7 @@ In [[microkernel]]-based systems, there is generally a considerable [[RPC]]
 overhead.
 
 In a multi-server system, it is non-trivial to implement a high-performance
-[[I/O System|io_system]].
+[[I/O System|community/gsoc/project_ideas/disk_io_performance]].
 
 When providing [[faq/POSIX_compatibility]] (and similar interfaces) in an
 environemnt that doesn't natively implement these interfaces, there may be a
diff --git a/open_issues/performance/io_system.mdwn b/open_issues/performance/io_system.mdwn
deleted file mode 100644
index 8535eae3..00000000
--- a/open_issues/performance/io_system.mdwn
+++ /dev/null
@@ -1,50 +0,0 @@
-[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
-Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-[[!meta title="I/O System"]]
-
-[[!tag open_issue_hurd gsoc-task]]
-
-The most obvious reason for the Hurd feeling slow compared to mainstream
-systems like GNU/Linux, is a low I/O system performance, in particular very
-slow hard disk access.
-
-The reason for this slowness is lack and/or bad implementation of common
-optimization techniques, like scheduling reads and writes to minimize head
-movement; effective block caching; effective reads/writes to partial blocks;
-[[reading/writing multiple blocks at once|clustered_page_faults]]; and
-[[read-ahead]].  The
-[[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some
-optimizations at a higher logical level.
-
-The goal of this project is to analyze the current situation, and implement/fix
-various optimizations, to achieve significantly better disk performance. It
-requires understanding the data flow through the various layers involved in
-disk access on the Hurd ([[filesystem|hurd/virtual_file_system]],
-[[pager|hurd/libpager]], driver), and general experience with
-optimizing complex systems.  That said, the killing feature we are definitely
-missing is the [[read-ahead]], and even a very simple implementation would bring
-very big performance speedups.
-
-Here are some real testcases:
-
-  * [[binutils_ld_64ksec]];
-
-  * running the Git testsuite which is mostly I/O bound;
-
-  * use [[TopGit]] on a non-toy repository.
-
-
-Possible mentors: Samuel Thibault (youpi)
-
-Exercise: Look through all the code involved in disk I/O, and try something
-easy to improve. It's quite likely though that you will find nothing obvious --
-in this case, please contact us about a different exercise task.
diff --git a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn
index b59a87a7..79c2300f 100644
--- a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn
+++ b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,7 +10,8 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_hurd]]
 
-This one may be considered as a testcase for I/O system optimization.
+This one may be considered as a testcase for [[I/O system
+optimization|community/gsoc/project_ideas/disk_io_performance]].
 
 It is taken from the [[binutils testsuite|binutils]],
 `ld/ld-elf/sec64k.exp`, where this
diff --git a/open_issues/performance/io_system/clustered_page_faults.mdwn b/open_issues/performance/io_system/clustered_page_faults.mdwn
index 3a187523..37433e06 100644
--- a/open_issues/performance/io_system/clustered_page_faults.mdwn
+++ b/open_issues/performance/io_system/clustered_page_faults.mdwn
@@ -10,6 +10,8 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_gnumach open_issue_hurd]]
 
+[[community/gsoc/project_ideas/disk_io_performance]].
+
 IRC, freenode, #hurd, 2011-02-16
 
     <braunr> exceptfor the kernel, everything in an address space is
diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn
index 3ee30b5d..b6851edd 100644
--- a/open_issues/performance/io_system/read-ahead.mdwn
+++ b/open_issues/performance/io_system/read-ahead.mdwn
@@ -10,6 +10,8 @@ License|/fdl]]."]]"""]]
 
 [[!tag open_issue_gnumach open_issue_hurd]]
 
+[[community/gsoc/project_ideas/disk_io_performance]]
+
 IRC, #hurd, freenode, 2011-02-13:
 
     <etenil> youpi: Would libdiskfs/diskfs.h be in the right place to make
diff --git a/open_issues/profiling.mdwn b/open_issues/profiling.mdwn
index e04fb08a..7e3c7350 100644
--- a/open_issues/profiling.mdwn
+++ b/open_issues/profiling.mdwn
@@ -17,7 +17,7 @@ done for [[performance analysis|performance]] reasons.
     Should be working, but some issues have been reported, regarding GCC spec
     files.  Should be possible to fix (if not yet done) easily.
 
-  * [[dtrace]]
+  * [[community/gsoc/project_ideas/dtrace]]
 
     Have a look at this, integrate it into the main trees.
 
diff --git a/open_issues/valgrind.mdwn b/open_issues/valgrind.mdwn
deleted file mode 100644
index bd45829c..00000000
--- a/open_issues/valgrind.mdwn
+++ /dev/null
@@ -1,83 +0,0 @@
-[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation,
-Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-[[!meta title="Porting Valgrind to the Hurd"]]
-
-[[!tag gsoc-task]]
-
-[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors.
-(And some other kinds of hard-to-find errors too.)
-Aside from being useful for program development in general,
-a Hurd port will help finding out why certain programs segfault on the Hurd,
-although they work on Linux.
-Even more importantly, it will help finding bugs in the Hurd servers themselfs.
-
-To keep track of memory use,
-Valgrind however needs to know how each [[system call]] affects the validity of memory regions.
-This knowledge is highly kernel-specific,
-and thus Valgrind needs to be explicitely ported for every system.
-
-Such a port involves two major steps:
-making Valgrind understand how kernel traps work in general on the system in question;
-and how all the individual kernel calls affect memory.
-The latter step is where most of the work is,
-as the behaviour of each single [[system call]] needs to be described.
-
-Compared to Linux,
-[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps.
-Almost all [[system call]]s are implemented as [[RPC]]s instead --
-either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]].
-All RPCs use a pair of `mach_msg` invocations:
-one to send a request message, and one to receive a reply.
-However, while all RPCs use the same `mach_msg` trap,
-the actual effect of the call varies greatly depending on which RPC is invoked --
-similar to the `ioctl` call on Linux.
-Each request thus must be handled individually.
-
-Unlike `ioctl`,
-the RPC invocations have explicit type information for the parameters though,
-which can be retrieved from the message header.
-By analyzing the parameters of the RPC reply message,
-Valgrind can know exactly which memory regions are affected by that call,
-even without specific knowledge of the RPC in question.
-Thus implementing a general parser for the reply messages
-will already give Valgrind a fairly good approximation of memory validity --
-without having to specify the exact semantic of each RPC by hand.
-
-While this should make Valgrind quite usable on the Hurd already, it's not perfect:
-some RPCs might return a buffer that is only partially filled with valid data;
-or some reply parameters might be optional,
-and only contain valid data under certain conditions.
-Such specific semantics can't be deduced from the message headers alone.
-Thus for a complete port,
-it will still be necessary to go through the list of all known RPCs,
-and implement special handling in Valgrind for those RPCs that need it.
-
-The goal of this task is at minimum to make Valgrind grok Mach traps,
-and to implement the generic RPC handler.
-Ideally, specific handling for RPCs needing it should also be implemented.
-
-Completing this project will require digging into Valgrind's handling of [[system call]]s,
-and into Hurd RPCs.
-It is not an easy task, but a fairly predictable one --
-there shouldn't be any unexpected difficulties,
-and no major design work is necessary.
-It doesn't require any specific previous knowledge:
-only good programming skills in general.
-On the other hand,
-the student will obtain a good understanding of Hurd RPCs while working on this task,
-and thus perfect qualifications for Hurd development in general :-)
-
-Possible mentors: Samuel Thibault (youpi)
-
-Exercise: As a starter,
-students can try to teach valgrind a couple of Linux ioctls,
-as this will make them learn how to use the read/write primitives of valgrind.