From eccf2986513cc41c412b1c30aa5dcb88a4c981b5 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Mon, 29 Nov 2010 07:58:51 +0100 Subject: Add links to some LWN articles, and then some. --- community/gsoc/project_ideas.mdwn | 6 +-- .../gsoc/project_ideas/libdiskfs_locking.mdwn | 41 ----------------- documentation.mdwn | 6 ++- glibc.mdwn | 9 ++++ glibc/environment_variables.mdwn | 15 ++++++ glibc/fork.mdwn | 9 ++++ glibc/poll.mdwn | 15 ++++++ hurd/debugging.mdwn | 10 ++-- hurd/translator.mdwn | 6 ++- hurd/translator/libguestfs.mdwn | 15 ++++++ open_issues/debugging.mdwn | 42 +++++++++++++++++ open_issues/gdb-heap.mdwn | 15 ++++++ open_issues/locking.mdwn | 53 ++++++++++++++++++++++ open_issues/performance.mdwn | 2 + open_issues/unit_testing.mdwn | 10 ++++ open_issues/virtualization/file_systems.mdwn | 3 +- unix.mdwn | 16 +++++-- 17 files changed, 214 insertions(+), 59 deletions(-) delete mode 100644 community/gsoc/project_ideas/libdiskfs_locking.mdwn create mode 100644 glibc/environment_variables.mdwn create mode 100644 glibc/poll.mdwn create mode 100644 hurd/translator/libguestfs.mdwn create mode 100644 open_issues/debugging.mdwn create mode 100644 open_issues/gdb-heap.mdwn create mode 100644 open_issues/locking.mdwn diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index 2102e8f7..b039608f 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] We offer a wide range of possible projects to choose from. If you have an idea not listed here, we'd love to hear about it! @@ -82,7 +82,7 @@ See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/H [[!inline pages="community/gsoc/project_ideas/server_overriding" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/tcp_ip_stack" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/nfs" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]] +[[!inline pages="open_issues/locking" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/pthreads" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/sound" show=0 feeds=no actions=yes]] [[!inline pages="open_issues/performance/io_system" show=0 feeds=no actions=yes]] diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn deleted file mode 100644 index 0618bbe6..00000000 --- a/community/gsoc/project_ideas/libdiskfs_locking.mdwn +++ /dev/null @@ -1,41 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] - -[[!meta title="Fix libdiskfs Locking Issues"]] - -Nowadays the most often encountered cause of Hurd crashes seems to be lockups -in the [[hurd/translator/ext2fs]] server. One of these could be traced -recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken -and not released in some cases. There is reason to believe that there are more -faulty paths causing these lockups. - -The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking -issues. To achieve this, some kind of test harness has to be implemented: For -example instrumenting the code to check locking correctness constantly at -runtime. Or implementing a unit testing framework that explicitly checks -locking in various code paths. (The latter could serve as a template for -implementing unit checks in other parts of the Hurd codebase...) - -(A systematic code review would probably suffice to find the existing locking -issues; but it wouldn't document the work in terms of actual code produced, and -thus it's not suitable for a GSoC project...) - -[Linux' *sparse*](https://sparse.wiki.kernel.org/) could be worth looking at. - -This task requires experience with debugging locking issues in multithreaded -applications. - -Possible mentors: Samuel Thibault (youpi) - -Exercise: If you could actually track down and fix one of the existing locking -errors before the end of the application process, that would be excellent. This -might be rather tough though, so probably you need to talk to us about an -alternative exercise task... diff --git a/documentation.mdwn b/documentation.mdwn index 62d96e9c..5c666f3f 100644 --- a/documentation.mdwn +++ b/documentation.mdwn @@ -5,8 +5,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[FAQ]] @@ -18,6 +18,8 @@ Documentation for... * [[MIG|microkernel/mach/mig/documentation]] + * [[UNIX]] + # Presentations diff --git a/glibc.mdwn b/glibc.mdwn index cefbb19c..f47efc03 100644 --- a/glibc.mdwn +++ b/glibc.mdwn @@ -29,6 +29,15 @@ Porting glibc to a specific architecture is non-trivial. ## [[Hurd-specific Port|hurd/glibc]] +# Implementation Details + + * [[environment_variables]] + + * [[fork]] + + * [[poll]] + + # Open Issues [[!inline pages=tag/open_issue_glibc raw=yes feeds=no]] diff --git a/glibc/environment_variables.mdwn b/glibc/environment_variables.mdwn new file mode 100644 index 00000000..76c1371e --- /dev/null +++ b/glibc/environment_variables.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# External + + * [*putenv() and setenv()*](http://www.greenend.org.uk/rjk/2008/putenv.html) + by Richard Kettlewell. diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index 564d9d5b..c9efd1f4 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -49,3 +49,12 @@ they have patches for software packages, to avoid using `fork` followed by * We no longer support `MACH_IPC_COMPAT`, thus we can get rid of the `err = __mach_port_allocate_name ([...]); if (err == KERN_NAME_EXISTS)` code ([[!taglink open_issue_glibc]]). + + +# External + + * [*How fork(2) ought to be*](http://www.greenend.org.uk/rjk/fork.html) by + Richard Kettlewell. + + * [*The self-pipe trick*](http://cr.yp.to/docs/selfpipe.html) by + D. J. Bernstein. diff --git a/glibc/poll.mdwn b/glibc/poll.mdwn new file mode 100644 index 00000000..d96f27a5 --- /dev/null +++ b/glibc/poll.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# External + + * [*poll() and EOF*](http://www.greenend.org.uk/rjk/2001/06/poll.html) by + Richard Kettlewell. diff --git a/hurd/debugging.mdwn b/hurd/debugging.mdwn index d6c5b18f..d6e9c8b5 100644 --- a/hurd/debugging.mdwn +++ b/hurd/debugging.mdwn @@ -6,8 +6,9 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + # Strategies @@ -16,11 +17,6 @@ is included in the section entitled * [[subhurd]] -- running another Hurd system in parallel * [[rpctrace]] -- tracing [[RPC]]s -## To Do - - * [[open_issues/ltrace]] - * [[open_issues/latrace]] - * [[open_issues/profiling]] # About Specific Packages diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index c3ca1278..9e109a28 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] A translator is simply a normal program acting as an object server and participating in the Hurd's @@ -117,6 +117,8 @@ Read about translator [[short-circuiting]]. * [[wishlist_1]] * [[wishlist_2]] * [[open_issues/network_file_system_by_just_forwarding_RPCs]] + * [[libguestfs]] + # Internally diff --git a/hurd/translator/libguestfs.mdwn b/hurd/translator/libguestfs.mdwn new file mode 100644 index 00000000..649b31f5 --- /dev/null +++ b/hurd/translator/libguestfs.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[libguestfs](http://libguestfs.org/) is said to be able to access a lot +of different filesystem types -- can we use it to build GNU Hurd +[[translator]]s? (There is a [[FUSE]] module, too.) diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn new file mode 100644 index 00000000..95b7bf9b --- /dev/null +++ b/open_issues/debugging.mdwn @@ -0,0 +1,42 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# Existing + +We have debugging infrastructure. For example: + + * [[GDB]] + + * [[GNU Mach debugging|microkernel/mach/gnumach/debugging]] + + * [[GNU Hurd debugging|hurd/debugging]], including + [[hurd/debugging/rpctrace]] and more. + + +# To Do + + * [[ltrace]] + + * [[latrace]] + + * [[profiling]] + + * *[Checkpoint/restart](http://lwn.net/Articles/412749/) allows the state of + a set of processes to be saved to persistent storage, then restarted at + some future time* -- quoting from Jonathan Corbet's 2010 Linux Kernel + Summit report. + + This is surely a very useful facility to have for reproducing failures, for + example. But on the other hand it's questionable how it can help with + debugging failures in [[GNU Hurd server|hurd/translator]]s' interactions, + as their state is typically spread between several processes. + + * [[locking]] diff --git a/open_issues/gdb-heap.mdwn b/open_issues/gdb-heap.mdwn new file mode 100644 index 00000000..75c31bbe --- /dev/null +++ b/open_issues/gdb-heap.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb]] + +Might be interesting to have a look at +[*gdb-heap*](https://fedorahosted.org/gdb-heap/) with respect to our +long-lived [[hurd/translator]] processes. diff --git a/open_issues/locking.mdwn b/open_issues/locking.mdwn new file mode 100644 index 00000000..1717133a --- /dev/null +++ b/open_issues/locking.mdwn @@ -0,0 +1,53 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +Every now and then, new locking issues are discovered in +[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays +these in fact seem to be the most often encountered cause of Hurd crashes +/ lockups. + +One of these could be traced +recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken +and not released in some cases. There is reason to believe that there are more +faulty paths causing these lockups. + +The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking +issues. To achieve this, some kind of test harness has to be implemented: For +example instrumenting the code to check locking correctness constantly at +runtime. Or implementing a [[unit testing]] framework that explicitly checks +locking in various code paths. (The latter could serve as a template for +implementing unit tests in other parts of the Hurd codebase...) + +(A systematic code review would probably suffice to find the existing locking +issues; but it wouldn't document the work in terms of actual code produced, and +thus it's not suitable for a GSoC project...) + +This task requires experience with debugging locking issues in multithreaded +applications. + +Tools have been written for static code analysis, than can help to locate +and fix such errors. + + * Coccinelle + + * + + * + + * clang + + * + + * Linux' sparse + + * diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn index a4816680..3d146a72 100644 --- a/open_issues/performance.mdwn +++ b/open_issues/performance.mdwn @@ -11,3 +11,5 @@ License|/fdl]]."]]"""]] * [[I/O System|io_system]] * [[fork]] + + * [[unit testing]] diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index b9fb3700..80a2860a 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -43,3 +43,13 @@ abandoned). * * + + * [*[ANNOUNCE] ktest.pl: Easy and flexible testing script for Linux Kernel + Developers*](http://lwn.net/Articles/412302/) by Steven Rostedt, + 2010-10-28. + + * -- ``comprehensive testing and + benchmarking platform''. This one might be useful for [[performance]] + testing, too? + + * diff --git a/open_issues/virtualization/file_systems.mdwn b/open_issues/virtualization/file_systems.mdwn index 3bf2299d..a12ea10d 100644 --- a/open_issues/virtualization/file_systems.mdwn +++ b/open_issues/virtualization/file_systems.mdwn @@ -20,4 +20,5 @@ be explored. * Linux saw a patch for [*generic name to handle and open by handle syscalls*](http://thread.gmane.org/gmane.linux.file-systems/46648) posted, which in turn can be beneficial for a [[QEMU]] emulation of a 9P file - system. + system. LWN's Jonathan Corbet covered this [*open by + handle*](http://lwn.net/Articles/375888/) functionality on 2010-02-23. diff --git a/unix.mdwn b/unix.mdwn index a927eb64..601b36d1 100644 --- a/unix.mdwn +++ b/unix.mdwn @@ -1,12 +1,15 @@ -[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2010 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="UNIX"]] + # External @@ -15,3 +18,10 @@ is included in the section entitled * [*Standardizing UNIX*](http://www.informit.com/articles/printerfriendly.aspx?p=691503), an article by David Chisnall. + + * [*Ghosts of Unix Past: a historical search for design + patterns*](http://lwn.net/Articles/411845/) (2010-10-27) by Neil Brown, + including file descriptors and the single, hierarchical namespace. + + * [*UNIX File Permissions*](http://www.greenend.org.uk/rjk/2004/perms.html) + (2004) by Richard Kettlewell. -- cgit v1.2.3