From e90db4db98bf65bd354994a7496b6b4e534e3f32 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 26 Nov 2010 09:33:49 +0100 Subject: glibc/fork: New. --- glibc/fork.mdwn | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 glibc/fork.mdwn (limited to 'glibc') diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn new file mode 100644 index 00000000..d1b26906 --- /dev/null +++ b/glibc/fork.mdwn @@ -0,0 +1,41 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +On [[Unix]] systems, `fork` is a rather simple system call. + +Our implementation in [[glibc]] is and needs to be rather bulky. + +For example, it has to duplicate all port rights for the new [[Mach +task|microkernel/mach/task]]. The address space can simply be duplicated by +standard means of the [[microkernel/Mach]], but as [[file descriptor]]s (for +example) are a concept that is implemented inside [[glibc]] (based on [[Mach +port|microkernel/mach/port]]s), these have to be duplicated from userspace, +which requires a small number of [[RPC]] for each of them. + +In sum, [[this affects performance|open_issues/performance/fork]] when new +processes are continuously being spawned from the shell, for example. + +Often, a `fork` call will eventually be followed by an `exec`, which will in +turn close (most of) the duplicated port rights. Unfortunately, this cannot be +known at the time the `fork` executing, so the code calling `fork` has to be +modified, and the `fork`, `exec` combo be replaced by a `posix_spawn` call, for +example, to avoid this work of duplicating each port right, then closing each +again. + +As far as we know, Cygwin has the same problem of `fork` being a nontrivial +operation. Perhaps we can learn from what they're been doing? Also, perhaps +they have patches for software packages, to avoid using `fork` followed by +`exec`, for example. + +--- + +We no longer support `MACH_IPC_COMPAT`, thus we can get rid of the `err = +__mach_port_allocate_name ([...]); if (err == KERN_NAME_EXISTS)` code[[!tag +open_issue_glibc]]. -- cgit v1.2.3 From 1f0e31cfa2c521b3a60d784c66d57922e96b5bd5 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 26 Nov 2010 10:34:00 +0100 Subject: glibc/fork: Add some TODOs. --- glibc/fork.mdwn | 18 ++++++++++++++---- .../fork_mach_port_mod_refs_ekern_urefs_owerflow.mdwn | 5 +++++ 2 files changed, 19 insertions(+), 4 deletions(-) (limited to 'glibc') diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index d1b26906..564d9d5b 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -34,8 +34,18 @@ operation. Perhaps we can learn from what they're been doing? Also, perhaps they have patches for software packages, to avoid using `fork` followed by `exec`, for example. ---- -We no longer support `MACH_IPC_COMPAT`, thus we can get rid of the `err = -__mach_port_allocate_name ([...]); if (err == KERN_NAME_EXISTS)` code[[!tag -open_issue_glibc]]. +# TODO + + * [[fork: mach_port_mod_refs: + EKERN_UREFS_OWERFLOW|open_issues/fork_mach_port_mod_refs_ekern_urefs_owerflow]] + ([[!taglink open_issue_glibc]]). + + * Include de-duplicate information from elsewhere: [[hurd-paper]], + [[hurd-talk]] [[hurd/ng/trivialconfinementvsconstructorvsfork]], + [[open_issues/resource_management_problems/zalloc_panics]] ([[!taglink + open_issue_glibc open_issue_documentation]]). + + * We no longer support `MACH_IPC_COMPAT`, thus we can get rid of the `err = + __mach_port_allocate_name ([...]); if (err == KERN_NAME_EXISTS)` code + ([[!taglink open_issue_glibc]]). diff --git a/open_issues/fork_mach_port_mod_refs_ekern_urefs_owerflow.mdwn b/open_issues/fork_mach_port_mod_refs_ekern_urefs_owerflow.mdwn index b6ecb92a..e80a5661 100644 --- a/open_issues/fork_mach_port_mod_refs_ekern_urefs_owerflow.mdwn +++ b/open_issues/fork_mach_port_mod_refs_ekern_urefs_owerflow.mdwn @@ -159,3 +159,8 @@ The error is 19, `EKERN_UREFS_OVERFLOW`. successfully.) [[!tag open_issue_glibc]] + + +# Discussion + + -- cgit v1.2.3 From eccf2986513cc41c412b1c30aa5dcb88a4c981b5 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Mon, 29 Nov 2010 07:58:51 +0100 Subject: Add links to some LWN articles, and then some. --- community/gsoc/project_ideas.mdwn | 6 +-- .../gsoc/project_ideas/libdiskfs_locking.mdwn | 41 ----------------- documentation.mdwn | 6 ++- glibc.mdwn | 9 ++++ glibc/environment_variables.mdwn | 15 ++++++ glibc/fork.mdwn | 9 ++++ glibc/poll.mdwn | 15 ++++++ hurd/debugging.mdwn | 10 ++-- hurd/translator.mdwn | 6 ++- hurd/translator/libguestfs.mdwn | 15 ++++++ open_issues/debugging.mdwn | 42 +++++++++++++++++ open_issues/gdb-heap.mdwn | 15 ++++++ open_issues/locking.mdwn | 53 ++++++++++++++++++++++ open_issues/performance.mdwn | 2 + open_issues/unit_testing.mdwn | 10 ++++ open_issues/virtualization/file_systems.mdwn | 3 +- unix.mdwn | 16 +++++-- 17 files changed, 214 insertions(+), 59 deletions(-) delete mode 100644 community/gsoc/project_ideas/libdiskfs_locking.mdwn create mode 100644 glibc/environment_variables.mdwn create mode 100644 glibc/poll.mdwn create mode 100644 hurd/translator/libguestfs.mdwn create mode 100644 open_issues/debugging.mdwn create mode 100644 open_issues/gdb-heap.mdwn create mode 100644 open_issues/locking.mdwn (limited to 'glibc') diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index 2102e8f7..b039608f 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] We offer a wide range of possible projects to choose from. If you have an idea not listed here, we'd love to hear about it! @@ -82,7 +82,7 @@ See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/H [[!inline pages="community/gsoc/project_ideas/server_overriding" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/tcp_ip_stack" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/nfs" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]] +[[!inline pages="open_issues/locking" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/pthreads" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/sound" show=0 feeds=no actions=yes]] [[!inline pages="open_issues/performance/io_system" show=0 feeds=no actions=yes]] diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn deleted file mode 100644 index 0618bbe6..00000000 --- a/community/gsoc/project_ideas/libdiskfs_locking.mdwn +++ /dev/null @@ -1,41 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] - -[[!meta title="Fix libdiskfs Locking Issues"]] - -Nowadays the most often encountered cause of Hurd crashes seems to be lockups -in the [[hurd/translator/ext2fs]] server. One of these could be traced -recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken -and not released in some cases. There is reason to believe that there are more -faulty paths causing these lockups. - -The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking -issues. To achieve this, some kind of test harness has to be implemented: For -example instrumenting the code to check locking correctness constantly at -runtime. Or implementing a unit testing framework that explicitly checks -locking in various code paths. (The latter could serve as a template for -implementing unit checks in other parts of the Hurd codebase...) - -(A systematic code review would probably suffice to find the existing locking -issues; but it wouldn't document the work in terms of actual code produced, and -thus it's not suitable for a GSoC project...) - -[Linux' *sparse*](https://sparse.wiki.kernel.org/) could be worth looking at. - -This task requires experience with debugging locking issues in multithreaded -applications. - -Possible mentors: Samuel Thibault (youpi) - -Exercise: If you could actually track down and fix one of the existing locking -errors before the end of the application process, that would be excellent. This -might be rather tough though, so probably you need to talk to us about an -alternative exercise task... diff --git a/documentation.mdwn b/documentation.mdwn index 62d96e9c..5c666f3f 100644 --- a/documentation.mdwn +++ b/documentation.mdwn @@ -5,8 +5,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[FAQ]] @@ -18,6 +18,8 @@ Documentation for... * [[MIG|microkernel/mach/mig/documentation]] + * [[UNIX]] + # Presentations diff --git a/glibc.mdwn b/glibc.mdwn index cefbb19c..f47efc03 100644 --- a/glibc.mdwn +++ b/glibc.mdwn @@ -29,6 +29,15 @@ Porting glibc to a specific architecture is non-trivial. ## [[Hurd-specific Port|hurd/glibc]] +# Implementation Details + + * [[environment_variables]] + + * [[fork]] + + * [[poll]] + + # Open Issues [[!inline pages=tag/open_issue_glibc raw=yes feeds=no]] diff --git a/glibc/environment_variables.mdwn b/glibc/environment_variables.mdwn new file mode 100644 index 00000000..76c1371e --- /dev/null +++ b/glibc/environment_variables.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# External + + * [*putenv() and setenv()*](http://www.greenend.org.uk/rjk/2008/putenv.html) + by Richard Kettlewell. diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index 564d9d5b..c9efd1f4 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -49,3 +49,12 @@ they have patches for software packages, to avoid using `fork` followed by * We no longer support `MACH_IPC_COMPAT`, thus we can get rid of the `err = __mach_port_allocate_name ([...]); if (err == KERN_NAME_EXISTS)` code ([[!taglink open_issue_glibc]]). + + +# External + + * [*How fork(2) ought to be*](http://www.greenend.org.uk/rjk/fork.html) by + Richard Kettlewell. + + * [*The self-pipe trick*](http://cr.yp.to/docs/selfpipe.html) by + D. J. Bernstein. diff --git a/glibc/poll.mdwn b/glibc/poll.mdwn new file mode 100644 index 00000000..d96f27a5 --- /dev/null +++ b/glibc/poll.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# External + + * [*poll() and EOF*](http://www.greenend.org.uk/rjk/2001/06/poll.html) by + Richard Kettlewell. diff --git a/hurd/debugging.mdwn b/hurd/debugging.mdwn index d6c5b18f..d6e9c8b5 100644 --- a/hurd/debugging.mdwn +++ b/hurd/debugging.mdwn @@ -6,8 +6,9 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + # Strategies @@ -16,11 +17,6 @@ is included in the section entitled * [[subhurd]] -- running another Hurd system in parallel * [[rpctrace]] -- tracing [[RPC]]s -## To Do - - * [[open_issues/ltrace]] - * [[open_issues/latrace]] - * [[open_issues/profiling]] # About Specific Packages diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index c3ca1278..9e109a28 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] A translator is simply a normal program acting as an object server and participating in the Hurd's @@ -117,6 +117,8 @@ Read about translator [[short-circuiting]]. * [[wishlist_1]] * [[wishlist_2]] * [[open_issues/network_file_system_by_just_forwarding_RPCs]] + * [[libguestfs]] + # Internally diff --git a/hurd/translator/libguestfs.mdwn b/hurd/translator/libguestfs.mdwn new file mode 100644 index 00000000..649b31f5 --- /dev/null +++ b/hurd/translator/libguestfs.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[libguestfs](http://libguestfs.org/) is said to be able to access a lot +of different filesystem types -- can we use it to build GNU Hurd +[[translator]]s? (There is a [[FUSE]] module, too.) diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn new file mode 100644 index 00000000..95b7bf9b --- /dev/null +++ b/open_issues/debugging.mdwn @@ -0,0 +1,42 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# Existing + +We have debugging infrastructure. For example: + + * [[GDB]] + + * [[GNU Mach debugging|microkernel/mach/gnumach/debugging]] + + * [[GNU Hurd debugging|hurd/debugging]], including + [[hurd/debugging/rpctrace]] and more. + + +# To Do + + * [[ltrace]] + + * [[latrace]] + + * [[profiling]] + + * *[Checkpoint/restart](http://lwn.net/Articles/412749/) allows the state of + a set of processes to be saved to persistent storage, then restarted at + some future time* -- quoting from Jonathan Corbet's 2010 Linux Kernel + Summit report. + + This is surely a very useful facility to have for reproducing failures, for + example. But on the other hand it's questionable how it can help with + debugging failures in [[GNU Hurd server|hurd/translator]]s' interactions, + as their state is typically spread between several processes. + + * [[locking]] diff --git a/open_issues/gdb-heap.mdwn b/open_issues/gdb-heap.mdwn new file mode 100644 index 00000000..75c31bbe --- /dev/null +++ b/open_issues/gdb-heap.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb]] + +Might be interesting to have a look at +[*gdb-heap*](https://fedorahosted.org/gdb-heap/) with respect to our +long-lived [[hurd/translator]] processes. diff --git a/open_issues/locking.mdwn b/open_issues/locking.mdwn new file mode 100644 index 00000000..1717133a --- /dev/null +++ b/open_issues/locking.mdwn @@ -0,0 +1,53 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +Every now and then, new locking issues are discovered in +[[hurd/libdiskfs]] or [[hurd/translator/ext2fs]], for example. Nowadays +these in fact seem to be the most often encountered cause of Hurd crashes +/ lockups. + +One of these could be traced +recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken +and not released in some cases. There is reason to believe that there are more +faulty paths causing these lockups. + +The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking +issues. To achieve this, some kind of test harness has to be implemented: For +example instrumenting the code to check locking correctness constantly at +runtime. Or implementing a [[unit testing]] framework that explicitly checks +locking in various code paths. (The latter could serve as a template for +implementing unit tests in other parts of the Hurd codebase...) + +(A systematic code review would probably suffice to find the existing locking +issues; but it wouldn't document the work in terms of actual code produced, and +thus it's not suitable for a GSoC project...) + +This task requires experience with debugging locking issues in multithreaded +applications. + +Tools have been written for static code analysis, than can help to locate +and fix such errors. + + * Coccinelle + + * + + * + + * clang + + * + + * Linux' sparse + + * diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn index a4816680..3d146a72 100644 --- a/open_issues/performance.mdwn +++ b/open_issues/performance.mdwn @@ -11,3 +11,5 @@ License|/fdl]]."]]"""]] * [[I/O System|io_system]] * [[fork]] + + * [[unit testing]] diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index b9fb3700..80a2860a 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -43,3 +43,13 @@ abandoned). * * + + * [*[ANNOUNCE] ktest.pl: Easy and flexible testing script for Linux Kernel + Developers*](http://lwn.net/Articles/412302/) by Steven Rostedt, + 2010-10-28. + + * -- ``comprehensive testing and + benchmarking platform''. This one might be useful for [[performance]] + testing, too? + + * diff --git a/open_issues/virtualization/file_systems.mdwn b/open_issues/virtualization/file_systems.mdwn index 3bf2299d..a12ea10d 100644 --- a/open_issues/virtualization/file_systems.mdwn +++ b/open_issues/virtualization/file_systems.mdwn @@ -20,4 +20,5 @@ be explored. * Linux saw a patch for [*generic name to handle and open by handle syscalls*](http://thread.gmane.org/gmane.linux.file-systems/46648) posted, which in turn can be beneficial for a [[QEMU]] emulation of a 9P file - system. + system. LWN's Jonathan Corbet covered this [*open by + handle*](http://lwn.net/Articles/375888/) functionality on 2010-02-23. diff --git a/unix.mdwn b/unix.mdwn index a927eb64..601b36d1 100644 --- a/unix.mdwn +++ b/unix.mdwn @@ -1,12 +1,15 @@ -[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2010 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="UNIX"]] + # External @@ -15,3 +18,10 @@ is included in the section entitled * [*Standardizing UNIX*](http://www.informit.com/articles/printerfriendly.aspx?p=691503), an article by David Chisnall. + + * [*Ghosts of Unix Past: a historical search for design + patterns*](http://lwn.net/Articles/411845/) (2010-10-27) by Neil Brown, + including file descriptors and the single, hierarchical namespace. + + * [*UNIX File Permissions*](http://www.greenend.org.uk/rjk/2004/perms.html) + (2004) by Richard Kettlewell. -- cgit v1.2.3 From 38368072b37bf73dda26dac536e4aa6cf13c67e4 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Mon, 29 Nov 2010 13:41:16 +0100 Subject: system_call: New. --- community/gsoc/project_ideas/libcap.mdwn | 8 ++++---- community/gsoc/project_ideas/libcap/details.mdwn | 8 ++++---- community/gsoc/project_ideas/secure_chroot.mdwn | 11 ++++++----- community/gsoc/project_ideas/valgrind.mdwn | 8 ++++---- extensibility.mdwn | 9 +++++---- faq/sharing_the_user_space.mdwn | 2 +- glibc.mdwn | 13 +++++++++++-- glibc/fork.mdwn | 20 +++++++++++++------- hurd/glibc/hurd-specific_api.mdwn | 11 ++++++----- hurd/networking.mdwn | 11 ++++++----- hurd/ng/microkernelcoyotos.mdwn | 4 +++- hurd/ng/trivialconfinementvsconstructorvsfork.mdwn | 18 ++++++++++++++---- hurd/translator/wishlist_2.mdwn | 12 +++++++++++- qemu.mdwn | 12 ++++++------ system_call.mdwn | 19 +++++++++++++++++++ 15 files changed, 113 insertions(+), 53 deletions(-) create mode 100644 system_call.mdwn (limited to 'glibc') diff --git a/community/gsoc/project_ideas/libcap.mdwn b/community/gsoc/project_ideas/libcap.mdwn index 1346203d..18c49c48 100644 --- a/community/gsoc/project_ideas/libcap.mdwn +++ b/community/gsoc/project_ideas/libcap.mdwn @@ -1,12 +1,12 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Implementing libcap"]] @@ -33,7 +33,7 @@ probably doable without previous experience with either, though. David Hedberg applied for this project in 2010, and though he didn't go through with it, -he fleshed out many [[libcap/details]]. +he fleshed out many [[details]]. Possible mentors: Samuel Thibault (youpi) diff --git a/community/gsoc/project_ideas/libcap/details.mdwn b/community/gsoc/project_ideas/libcap/details.mdwn index aa27a84e..85695978 100644 --- a/community/gsoc/project_ideas/libcap/details.mdwn +++ b/community/gsoc/project_ideas/libcap/details.mdwn @@ -5,8 +5,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Details on implementing libcap"]] @@ -59,7 +59,7 @@ Each process has a three bit fields representing each of the three sets (P, E and I). Each bit field is currently built up of two (32 bit) integers to be able to hold the 33 currently defined capabilities (see linux/capability.h). Each process further has a bounding set which -bounds the permitted set. Two syscalls handles the setting and getting +bounds the permitted set. Two [[system call]]s handles the setting and getting of capabilities; *capset* and *capget*. Some related functionality can also be controlled by calling *prctl*: the right to read/drop the bounding capabilities (PR_CAPBSET_READ/PR_CAPBSET_DROP) and whether @@ -428,7 +428,7 @@ the following (also detailed somewhat in the same article): * Execute process as root (or setuid) to gain all capabilities. -* Use the prctl system call to enable keepcaps for the process +* Use the prctl [[system call]] to enable keepcaps for the process (same(?) effect as enabling SECURE_NO_SETUID_FIXUP for the process). keepcaps should be off by default. diff --git a/community/gsoc/project_ideas/secure_chroot.mdwn b/community/gsoc/project_ideas/secure_chroot.mdwn index feb30a7c..57739861 100644 --- a/community/gsoc/project_ideas/secure_chroot.mdwn +++ b/community/gsoc/project_ideas/secure_chroot.mdwn @@ -1,17 +1,18 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Secure chroot Implementation"]] As the Hurd attempts to be (almost) fully [[UNIX]]-compatible, it also implements a -`chroot()` system call. However, the current implementation is not really +`chroot` [[system call]]. However, the current implementation is not really good, as it allows easily escaping the `chroot`, for example by use of [[passive_translators|hurd/translator]]. @@ -20,7 +21,7 @@ workaround changing the behavior of passive translators in a `chroot`; changing the context in which passive translators are executed; changing the interpretation of filenames in a chroot; to reworking the whole passive translator mechanism. Some involving a completely different approach to -`chroot` implementation, using a proxy instead of a special system call in the +`chroot` implementation, using a proxy instead of a special [[system call]] in the filesystem servers. See diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn index c6fc7459..7d68e82d 100644 --- a/community/gsoc/project_ideas/valgrind.mdwn +++ b/community/gsoc/project_ideas/valgrind.mdwn @@ -18,7 +18,7 @@ although they work on Linux. Even more importantly, it will help finding bugs in the Hurd servers themselfs. To keep track of memory use, -Valgrind however needs to know how each system call affects the validity of memory regions. +Valgrind however needs to know how each [[system call]] affects the validity of memory regions. This knowledge is highly kernel-specific, and thus Valgrind needs to be explicitely ported for every system. @@ -26,11 +26,11 @@ Such a port involves two major steps: making Valgrind understand how kernel traps work in general on the system in question; and how all the individual kernel calls affect memory. The latter step is where most of the work is, -as the behaviour of each single system call needs to be described. +as the behaviour of each single [[system call]] needs to be described. Compared to Linux, Mach (the microkernel used by the Hurd) has very few kernel traps. -Almost all system calls are implemented as RPCs instead -- +Almost all [[system call]]s are implemented as RPCs instead -- either handled by Mach itself, or by the various Hurd servers. All RPCs use a pair of mach\_msg() invocations: one to send a request message, and one to receive a reply. @@ -62,7 +62,7 @@ The goal of this task is at minimum to make Valgrind grok Mach traps, and to implement the generic RPC handler. Ideally, specific handling for RPCs needing it should also be implemented. -Completing this project will require digging into Valgrind's handling of system calls, +Completing this project will require digging into Valgrind's handling of [[system call]]s, and into Hurd RPCs. It is not an easy task, but a fairly predictable one -- there shouldn't be any unexpected difficulties, diff --git a/extensibility.mdwn b/extensibility.mdwn index 01b1f3b1..17cd5e51 100644 --- a/extensibility.mdwn +++ b/extensibility.mdwn @@ -1,17 +1,18 @@ -[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] An extensible system is one that enables extensibility. Enabling extensibility means providing non-privileged mechanisms to extend existing objects and to introduce new objects. [[UNIX]] is generally not an extensible system as it does -not generally facilitate the hooking of system calls. For instance, there is +not generally facilitate the hooking of [[system call]]s. For instance, there is no way to hook into the virtual file system. This has motivated the introduction of separate, parallel interfaces by both the GNOME and KDE projects to provide users a more integrated view of their objects. diff --git a/faq/sharing_the_user_space.mdwn b/faq/sharing_the_user_space.mdwn index 7d09ccc0..ec880827 100644 --- a/faq/sharing_the_user_space.mdwn +++ b/faq/sharing_the_user_space.mdwn @@ -15,7 +15,7 @@ everything but the kernel is shared? *Answer:* Given that both Linux and GNU Hurd are using the [[ELF]] binary format, this could indeed be made possible, if all programs agreed to rely on only one abstraction layer, for example the standard C library ([[glibc]]). -(Additionally, for example for system calls that are not covered by glibc +(Additionally, for example for [[system call]]s that are not covered by glibc calls, you'd need to be able to reliably trap and emulate these.) However, Linux' and the GNU Hurd's [[ABI]]'s have sufficiently diverged, so that this is not easy to do. That's why you can't currently install a system in this way, diff --git a/glibc.mdwn b/glibc.mdwn index f47efc03..2eba3667 100644 --- a/glibc.mdwn +++ b/glibc.mdwn @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="GNU C Library"]] @@ -31,6 +31,15 @@ Porting glibc to a specific architecture is non-trivial. # Implementation Details + * [[hurd/glibc/Hurd-specific API]] + + * [[open_issues/secure_file_descriptor_handling]] + + +## Individual functions + +Some of these are well-known as [[UNIX]] [[system call]]s. + * [[environment_variables]] * [[fork]] diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index c9efd1f4..e8556a91 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -8,7 +8,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -On [[Unix]] systems, `fork` is a rather simple system call. +On [[Unix]] systems, `fork` is a rather simple [[system call]]. Our implementation in [[glibc]] is and needs to be rather bulky. @@ -22,12 +22,13 @@ which requires a small number of [[RPC]] for each of them. In sum, [[this affects performance|open_issues/performance/fork]] when new processes are continuously being spawned from the shell, for example. -Often, a `fork` call will eventually be followed by an `exec`, which will in -turn close (most of) the duplicated port rights. Unfortunately, this cannot be -known at the time the `fork` executing, so the code calling `fork` has to be -modified, and the `fork`, `exec` combo be replaced by a `posix_spawn` call, for -example, to avoid this work of duplicating each port right, then closing each -again. +Often, a `fork` call will eventually be followed by an `exec`, which [[may in +turn close|open_issues/secure_file_descriptor_handling]] (most of) the +duplicated port rights. Unfortunately, this cannot be known at the time the +`fork` executing, so in order to optimize this, the code calling `fork` has to +be modified instead, and the `fork`, `exec` combo be replaced by a +`posix_spawn` call, for example, to avoid this work of duplicating each port +right, then closing each again. As far as we know, Cygwin has the same problem of `fork` being a nontrivial operation. Perhaps we can learn from what they're been doing? Also, perhaps @@ -51,6 +52,11 @@ they have patches for software packages, to avoid using `fork` followed by ([[!taglink open_issue_glibc]]). +## Related + + * [[secure file descriptor handling]]. + + # External * [*How fork(2) ought to be*](http://www.greenend.org.uk/rjk/fork.html) by diff --git a/hurd/glibc/hurd-specific_api.mdwn b/hurd/glibc/hurd-specific_api.mdwn index aeb63d91..75220279 100644 --- a/hurd/glibc/hurd-specific_api.mdwn +++ b/hurd/glibc/hurd-specific_api.mdwn @@ -1,17 +1,18 @@ -[[!meta copyright="Copyright © 2002, 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2007, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Hurd-specific glibc API"]] These functions have meaning only under Hurd. They are needed to get port -names that are used in native Hurd API (the RPC calls to servers). The `.defs` +names that are used in native Hurd API (the [[RPC]]s to servers). The `.defs` and `.h` files can be found in `/include/hurd` when all development files are installed (Debian package `hurd-dev`.) Note that `.defs` are not included in C programs -- they are used to produce `.h` files. @@ -157,7 +158,7 @@ programs -- they are used to produce `.h` files.

thread_t
hurd_thread_self (void);
-
Return the current thread's thread port. This is a cheap operation (no system call), but it relies on Hurd signal state being set up.
+
Return the current thread's thread port. This is a cheap operation (no [[system call]]), but it relies on Hurd signal state being set up.

error_t
diff --git a/hurd/networking.mdwn b/hurd/networking.mdwn index ff16eb25..bdf9def2 100644 --- a/hurd/networking.mdwn +++ b/hurd/networking.mdwn @@ -1,12 +1,13 @@ -[[!meta copyright="Copyright © 2000, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2000, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] For each supported `PF_*` protocol family, there is a file `/servers/socket/N` where `N` is the numberic value fo the `PF_*` symbol. Right now @@ -17,10 +18,10 @@ where `N` is the numberic value fo the `PF_*` symbol. Right now User programs open those files, and use the `socket_create` [[RPC]] to make a new socket. With that socket, they can use the other `socket_*` RPCs and also the `io_*` RPCs. The `socket_*` RPCs are essentially clones of the [[Unix]] -syscalls in question. +[[system call]]s in question. The only exception is `sockaddrs`, which are implemented as [[ports|libports]] -instead of the opaque data arrays they are in the syscalls. You manipulate +instead of the opaque data arrays they are in the system calls. You manipulate `sockaddr` ports with the `socket_create_address`, `socket_fabricate_address`, and `socket_whatis_address` calls. The `sockaddr` port is then used in socket calls like `socket_connect` and `socket_accept`. diff --git a/hurd/ng/microkernelcoyotos.mdwn b/hurd/ng/microkernelcoyotos.mdwn index cdf4e1bf..2340901d 100644 --- a/hurd/ng/microkernelcoyotos.mdwn +++ b/hurd/ng/microkernelcoyotos.mdwn @@ -2,7 +2,9 @@ [Coyotos](http://www.coyotos.org/index.html) is a microkernel and OS and the successor of EROS, that itself is the successor of KeyKOS. A more complete history can be found [here](http://www.coyotos.org/history.html). Its main objectives are to correcte some shortcomings of EROS, demonstrate that an atomic kernel design scales well, and (eventually) to completely formally verify both the kernel and critical system components by writing them in a new language called [bitc](http://www.bitc-lang.org/). [See [l4.verified](http://nicta.com.au/research/projects/l4.verified) for work on formally verifying an L4 microkernel.] -Coyotos is an orthogonally persistent pure capability system. It uses continuation based unbuffered asynchronous IPC (actually it's synchronous IPC with asynchronous syscalls). +Coyotos is an orthogonally persistent pure capability system. It uses +continuation based unbuffered asynchronous IPC (actually it's synchronous IPC +with asynchronous [[system calls]]). TODO: explain these terms and (more important) their consequences on system design. diff --git a/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn b/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn index 4eeef6ee..0d91dee7 100644 --- a/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn +++ b/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn @@ -6,10 +6,11 @@ This comparison is about a simple situation: there is a parent process P, which # Trivial Confinement -For trivial confinement, there is a system call to create a process from some memory pages. P performs the following steps: +For trivial confinement, there is a [[system call]] to create a process from +some memory pages. P performs the following steps: * Allocate some memory and put the code image of the child into that memory. This can be done by P, or for example by the file system which then gives the resulting memory (space bank) to P. -* Perform the system call on that memory. The result is a capability to C. +* Perform the [[system call]] on that memory. The result is a capability to C. * Send A to C using the returned capability. Note that it is up to the implementation of the system what happens with P's access to the memory which holds the child. For example, it is probably a good idea if it is at least unmapped, so it cannot accidentily write things in it. It could even be revoked, so that it can't write things in it, even if it wants to. @@ -32,7 +33,16 @@ This mechanism is targeted at a specific use pattern, namely that a process is c # POSIX Fork -POSIX fork, or rather fork+exec, is how things are done on many current systems. It may be insightful to see it included in the comparison, especially for people who are new to the subject. There are two system calls, fork and exec. Fork will create a clone of the current process, including all the capabilities (that is, file descriptors) of the parent (except the ones which have explicitly been excluded). Exec is a system call which really goes to the filesystem, not the kernel (although on systems which use it, the filesystem usually resides in the kernel), and asks it to spawn a new process from the contents of a certain path in place of the caller. This passes all capabilities to the new process. The procedure is: +POSIX fork, or rather fork+exec, is how things are done on many current +systems. It may be insightful to see it included in the comparison, especially +for people who are new to the subject. There are two [[system call]]s, fork and +exec. Fork will create a clone of the current process, including all the +capabilities (that is, file descriptors) of the parent (except the ones which +have explicitly been excluded). Exec is a [[system call]] which really goes to +the filesystem, not the kernel (although on systems which use it, the +filesystem usually resides in the kernel), and asks it to spawn a new process +from the contents of a certain path in place of the caller. This passes all +capabilities to the new process. The procedure is: * P calls fork(), creating P'. * P' drops B. @@ -67,7 +77,7 @@ Except for the control, there is really only one other difference, and that's ad What it doesn't do is protect the code image against bugs in P. In the constructor the trusted and well-tested constructor code is handling the image, for trivial confinement the (very possibly) buggy program P. In particular, when starting a program from a file system, with trivial confinement the operation is: * Ask the file system for the code, receive a capability to a space bank with a copy (on write) of it. -* Make the system call to turn it into a program. +* Make the [[system call]] to turn it into a program. Now this isn't much more complicated than the constructor which does: diff --git a/hurd/translator/wishlist_2.mdwn b/hurd/translator/wishlist_2.mdwn index a927db55..77f39644 100644 --- a/hurd/translator/wishlist_2.mdwn +++ b/hurd/translator/wishlist_2.mdwn @@ -70,7 +70,17 @@ Here's an [idea](http://www.circlemud.org/~jelson/software/fusd/docs/node13.html * "One particularly interesting application of FUSD that we've found very useful is as a way to let regular user-space libraries export device file APIs. For example, imagine you had a library which factored large composite numbers. Typically, it might have a C interface--say, a function called `int *factorize(int bignum)`. With FUSD, it's possible to create a device file interface--say, a device called `/dev/factorize` to which clients can `write(2)` a big number, then `read(2)` back its factors. -* This may sound strange, but device file APIs have at least three advantages over a typical library API. First, it becomes much more language independent--any language that can make system calls can access the factorization library. Second, the factorization code is running in a different address space; if it crashes, it won't crash or corrupt the caller. Third, and most interestingly, it is possible to use `select(2)` to wait for the factorization to complete. `select(2)` would make it easy for a client to factor a large number while remaining responsive to other events that might happen in the meantime. In other words, FUSD allows normal user-space libraries to integrate seamlessly with UNIX's existing, POSIX-standard event notification interface: `select(2)`." +* This may sound strange, but device file APIs have at least three advantages + over a typical library API. First, it becomes much more language + independent--any language that can make [[system call]]s can access the + factorization library. Second, the factorization code is running in a + different address space; if it crashes, it won't crash or corrupt the + caller. Third, and most interestingly, it is possible to use `select(2)` to + wait for the factorization to complete. `select(2)` would make it easy for a + client to factor a large number while remaining responsive to other events + that might happen in the meantime. In other words, FUSD allows normal + user-space libraries to integrate seamlessly with UNIX's existing, + POSIX-standard event notification interface: `select(2)`." ## Mail diff --git a/qemu.mdwn b/qemu.mdwn index 19b5fb9f..d7cea5ad 100644 --- a/qemu.mdwn +++ b/qemu.mdwn @@ -1,13 +1,13 @@ -[[!meta copyright="Copyright © 2005, 2007, 2008, 2009 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2005, 2007, 2008, 2009, 2010 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] QEMU is free software written by Fabrice Bellard that implements a fast processor [[emulator|emulation]], allowing a user to run one operating system @@ -19,8 +19,8 @@ reasonable speed while being easy to port on new host CPUs. QEMU has two operating modes: * User mode emulation: QEMU can launch Linux processes compiled for one CPU - on another CPU. Linux system calls are converted because of endianness and - 32/64 bit mismatches. Wine and Dosemu are the main targets for QEMU. + on another CPU. Linux [[system call]]s are converted because of endianness + and 32/64 bit mismatches. Wine and Dosemu are the main targets for QEMU. * System mode emulation: QEMU emulates a full system, including a processor and various peripherials. It enables easier testing and debugging of diff --git a/system_call.mdwn b/system_call.mdwn new file mode 100644 index 00000000..197889cb --- /dev/null +++ b/system_call.mdwn @@ -0,0 +1,19 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +In an [[UNIX]]-like system, a *system call* (*syscall*) is used to request all +kinds of functionality from the operating system kernel. + +A [[microkernel]]-based system typically won't offer a lot of system calls -- +apart from one central one, and that is *send message* -- but instead [[RPC]]s +will be used instead. + +In the [[GNU Hurd|hurd]], a lot of what is traditionlly considered to be a UNIX +system call is implemented (primarily by means of [[RPC]]) inside [[glibc]]. -- cgit v1.2.3 From cd782d77c1e90976cb6dacf6ba78ba762f145a50 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Tue, 30 Nov 2010 10:39:28 +0100 Subject: Been reading another LWN issue... --- capability.mdwn | 14 ++-- community/gsoc/project_ideas.mdwn | 2 +- community/gsoc/project_ideas/valgrind.mdwn | 80 ---------------------- glibc.mdwn | 9 ++- glibc/fallocate.mdwn | 17 +++++ glibc/fork.mdwn | 18 +++-- glibc/signals.mdwn | 32 +++++++++ hurd/glibc/hurd-specific_api.mdwn | 8 ++- hurd/ng/trivialconfinementvsconstructorvsfork.mdwn | 24 ++++--- hurd/translator/magic.mdwn | 11 +-- open_issues/code_analysis.mdwn | 12 +++- open_issues/debugging.mdwn | 16 +++-- open_issues/multithreading.mdwn | 4 +- open_issues/nightly_builds_deb_packages.mdwn | 6 ++ open_issues/secure_file_descriptor_handling.mdwn | 9 +++ open_issues/unit_testing.mdwn | 9 ++- open_issues/valgrind.mdwn | 80 ++++++++++++++++++++++ persistency.mdwn | 11 +-- unix.mdwn | 48 +++++++++++-- unix/file_descriptor.mdwn | 13 ++++ virtualization.mdwn | 7 +- 21 files changed, 296 insertions(+), 134 deletions(-) delete mode 100644 community/gsoc/project_ideas/valgrind.mdwn create mode 100644 glibc/fallocate.mdwn create mode 100644 glibc/signals.mdwn create mode 100644 open_issues/valgrind.mdwn create mode 100644 unix/file_descriptor.mdwn (limited to 'glibc') diff --git a/capability.mdwn b/capability.mdwn index 367ea163..d78810d5 100644 --- a/capability.mdwn +++ b/capability.mdwn @@ -1,12 +1,13 @@ -[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] A capability is a protected reference. It is a reference in that it designates an object; it is protected in that in cannot be @@ -27,7 +28,6 @@ sent a string to identify the file to B, the identifier lacks a than A intended. Be ensuring that [[designation]] and [[authorization]] are always bound together, these problems are avoided. -[[Unix]] file descriptors can be viewed as capabilities. Unix file -descriptors do not survive reboot, that is, they are not -[[persistent|persistency]]. To work around this, [[ACL]]s are used to -recover authority. +[[UNIX file descriptors|unix/file_descriptor]] can be viewed as capabilities. +They do not survive reboot, that is, they are not [[persistent|persistency]]. +To work around this, [[ACL]]s are used to recover authority. diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index b039608f..649e05c1 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -106,4 +106,4 @@ See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/H [[!inline pages="community/gsoc/project_ideas/testsuites" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/libcap" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/xattr" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/valgrind" show=0 feeds=no actions=yes]] +[[!inline pages="open_issues/valgrind" show=0 feeds=no actions=yes]] diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn deleted file mode 100644 index 7d68e82d..00000000 --- a/community/gsoc/project_ideas/valgrind.mdwn +++ /dev/null @@ -1,80 +0,0 @@ -[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] - -[[!meta title="Porting Valgrind to the Hurd"]] - -[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. -(And some other kinds of hard-to-find errors too.) -Aside from being useful for program development in general, -a Hurd port will help finding out why certain programs segfault on the Hurd, -although they work on Linux. -Even more importantly, it will help finding bugs in the Hurd servers themselfs. - -To keep track of memory use, -Valgrind however needs to know how each [[system call]] affects the validity of memory regions. -This knowledge is highly kernel-specific, -and thus Valgrind needs to be explicitely ported for every system. - -Such a port involves two major steps: -making Valgrind understand how kernel traps work in general on the system in question; -and how all the individual kernel calls affect memory. -The latter step is where most of the work is, -as the behaviour of each single [[system call]] needs to be described. - -Compared to Linux, -Mach (the microkernel used by the Hurd) has very few kernel traps. -Almost all [[system call]]s are implemented as RPCs instead -- -either handled by Mach itself, or by the various Hurd servers. -All RPCs use a pair of mach\_msg() invocations: -one to send a request message, and one to receive a reply. -However, while all RPCs use the same mach\_msg() trap, -the actual effect of the call varies greatly depending on which RPC is invoked -- -similar to the ioctl() call on Linux. -Each request thus must be handled individually. - -Unlike ioctl(), -the RPC invocations have explicit type information for the parameters though, -which can be retrieved from the message header. -By analyzing the parameters of the RPC reply message, -Valgrind can know exactly which memory regions are affected by that call, -even without specific knowledge of the RPC in question. -Thus implementing a general parser for the reply messages -will already give Valgrind a fairly good approximation of memory validity -- -without having to specify the exact semantic of each RPC by hand. - -While this should make Valgrind quite usable on the Hurd already, it's not perfect: -some RPCs might return a buffer that is only partially filled with valid data; -or some reply parameters might be optional, -and only contain valid data under certain conditions. -Such specific semantics can't be deduced from the message headers alone. -Thus for a complete port, -it will still be necessary to go through the list of all known RPCs, -and implement special handling in Valgrind for those RPCs that need it. - -The goal of this task is at minimum to make Valgrind grok Mach traps, -and to implement the generic RPC handler. -Ideally, specific handling for RPCs needing it should also be implemented. - -Completing this project will require digging into Valgrind's handling of [[system call]]s, -and into Hurd RPCs. -It is not an easy task, but a fairly predictable one -- -there shouldn't be any unexpected difficulties, -and no major design work is necessary. -It doesn't require any specific previous knowledge: -only good programming skills in general. -On the other hand, -the student will obtain a good understanding of Hurd RPCs while working on this task, -and thus perfect qualifications for Hurd development in general :-) - -Possible mentors: Samuel Thibault (youpi) - -Exercise: As a starter, -students can try to teach valgrind a couple of Linux ioctls, -as this will make them learn how to use the read/write primitives of valgrind. diff --git a/glibc.mdwn b/glibc.mdwn index 2eba3667..124216d9 100644 --- a/glibc.mdwn +++ b/glibc.mdwn @@ -36,11 +36,18 @@ Porting glibc to a specific architecture is non-trivial. * [[open_issues/secure_file_descriptor_handling]] +## Concepts + + * [[environment_variables]] + + * [[signals]] + + ## Individual functions Some of these are well-known as [[UNIX]] [[system call]]s. - * [[environment_variables]] + * [[fallocate]] * [[fork]] diff --git a/glibc/fallocate.mdwn b/glibc/fallocate.mdwn new file mode 100644 index 00000000..3aecf16b --- /dev/null +++ b/glibc/fallocate.mdwn @@ -0,0 +1,17 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Not yet implemented for the GNU Hurd in [[glibc]]. + + +# External + + * [*Punching holes in files*](http://lwn.net/Articles/415889/), Jonathan + Corbet, 2010-11-17. diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index e8556a91..378fe835 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -14,10 +14,10 @@ Our implementation in [[glibc]] is and needs to be rather bulky. For example, it has to duplicate all port rights for the new [[Mach task|microkernel/mach/task]]. The address space can simply be duplicated by -standard means of the [[microkernel/Mach]], but as [[file descriptor]]s (for -example) are a concept that is implemented inside [[glibc]] (based on [[Mach -port|microkernel/mach/port]]s), these have to be duplicated from userspace, -which requires a small number of [[RPC]] for each of them. +standard means of the [[microkernel/Mach]], but as [[unix/file_descriptor]]s +(for example) are a concept that is implemented inside [[glibc]] (based on +[[Mach port|microkernel/mach/port]]s), these have to be duplicated from +userspace, which requires a small number of [[RPC]] for each of them. In sum, [[this affects performance|open_issues/performance/fork]] when new processes are continuously being spawned from the shell, for example. @@ -43,7 +43,7 @@ they have patches for software packages, to avoid using `fork` followed by ([[!taglink open_issue_glibc]]). * Include de-duplicate information from elsewhere: [[hurd-paper]], - [[hurd-talk]] [[hurd/ng/trivialconfinementvsconstructorvsfork]], + [[hurd-talk]], [[hurd/ng/trivialconfinementvsconstructorvsfork]], [[open_issues/resource_management_problems/zalloc_panics]] ([[!taglink open_issue_glibc open_issue_documentation]]). @@ -54,13 +54,11 @@ they have patches for software packages, to avoid using `fork` followed by ## Related - * [[secure file descriptor handling]]. + * [[open_issues/secure_file_descriptor_handling]]. # External - * [*How fork(2) ought to be*](http://www.greenend.org.uk/rjk/fork.html) by - Richard Kettlewell. + * {{$unix#djb_self-pipe}}. - * [*The self-pipe trick*](http://cr.yp.to/docs/selfpipe.html) by - D. J. Bernstein. + * {{$unix#rjk_fork}}. diff --git a/glibc/signals.mdwn b/glibc/signals.mdwn new file mode 100644 index 00000000..40fdc0e1 --- /dev/null +++ b/glibc/signals.mdwn @@ -0,0 +1,32 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +*[[UNIX]] signals* are a means to asynchronously invoke a specific function +(*signal handler*). This may impact on [[system call]]s that are executing at +the same time in that they may be completely aborted, return incomplete +results, scheduled for restarting, or cause signal delivery to be blocked upon +the system call's completion. + +An explanation can be found in the relevant standards, an overview, including +UNIX signals' deficiencies is given in {{$unix#2010_brown_ghosts_3}}, for +example. + +The UNIX signalling mechanism is implemented for the GNU Hurd by means of a +separate signal-handling thread that is part of every process. This makes +handling of signals a separate thread of control. + + * [[SA_SIGINFO, SA_SIGACTION|open_issues/sa_siginfo_sa_sigaction]] + + +# External + + * {{$unix#djb_self-pipe}}. + + * {{$unix#rjk_fork}}. diff --git a/hurd/glibc/hurd-specific_api.mdwn b/hurd/glibc/hurd-specific_api.mdwn index 75220279..7ead63cd 100644 --- a/hurd/glibc/hurd-specific_api.mdwn +++ b/hurd/glibc/hurd-specific_api.mdwn @@ -82,7 +82,13 @@ programs -- they are used to produce `.h` files.
openport (io_t port, int flags);

-
Open a file descriptor on a port. FLAGS are as for open; flags affected by io_set_openmodes are not changed by this. If successful, this consumes a user reference for PORT (which will be deallocated on close.) See <hurd/io.defs> and <hurd/io.h>.
+
Open a [[unix/file_descriptor]] on a [[microkernel/mach/port]]. FLAGS + are as for open; flags affected by io_set_openmodes are + not changed by this. If successful, this consumes a user reference for + PORT (which will be deallocated on close.) See + <hurd/io.defs> and + <hurd/io.h>. +

task_t
diff --git a/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn b/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn index 0d91dee7..949895e7 100644 --- a/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn +++ b/hurd/ng/trivialconfinementvsconstructorvsfork.mdwn @@ -34,15 +34,15 @@ This mechanism is targeted at a specific use pattern, namely that a process is c # POSIX Fork POSIX fork, or rather fork+exec, is how things are done on many current -systems. It may be insightful to see it included in the comparison, especially -for people who are new to the subject. There are two [[system call]]s, fork and -exec. Fork will create a clone of the current process, including all the -capabilities (that is, file descriptors) of the parent (except the ones which -have explicitly been excluded). Exec is a [[system call]] which really goes to -the filesystem, not the kernel (although on systems which use it, the -filesystem usually resides in the kernel), and asks it to spawn a new process -from the contents of a certain path in place of the caller. This passes all -capabilities to the new process. The procedure is: +systems. It may be insightful to see it included in the comparison, especially +for people who are new to the subject. There are two [[system call]]s, fork +and exec. Fork will create a clone of the current process, including all the +capabilities (that is, [[unix/file_descriptor]]s) of the parent (except the +ones which have explicitly been excluded). Exec is a [[system call]] which +really goes to the filesystem, not the kernel (although on systems which use +it, the filesystem usually resides in the kernel), and asks it to spawn a new +process from the contents of a certain path in place of the caller. This +passes all capabilities to the new process. The procedure is: * P calls fork(), creating P'. * P' drops B. @@ -62,7 +62,11 @@ In contrast, the other two options don't pass anything by default. If there is a The problem of fork+exec can be solved. It is if the default would be to not pass capabilities to the new process, but specify a list of capabilities that it should keep, or (like in the other cases) pass them over a new channel which is implicitly created during the fork. However, in that case the only difference with trivial confinement is that P' dies in the process (and thus must be created to prevent P from dying). Almost any use of exec is in practice preceded by a fork for this purpose. It would be easier to make trivial confinement the default operation and let P die directly after it in the rare case that it should. -The only reason for continuing to use fork+exec would be that it is what existing programs do. However, they break anyway if they need to specify which file descriptors to pass. So they need to be adapted. Therefore, it's better to make the usual spawning method the primitive one, and emulate the other. +The only reason for continuing to use fork+exec would be that it is what +existing programs do. However, they break anyway if they need to specify which +[[unix/file_descriptor]]s to pass. So they need to be adapted. Therefore, it's +better to make the usual spawning method the primitive one, and emulate the +other. # Trivial Confinement vs Constructor diff --git a/hurd/translator/magic.mdwn b/hurd/translator/magic.mdwn index 06ee798b..84bacdfb 100644 --- a/hurd/translator/magic.mdwn +++ b/hurd/translator/magic.mdwn @@ -1,20 +1,21 @@ -[[!meta copyright="Copyright © 2006, 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2006, 2007, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] The magic translator provides `/dev/fd`. $ showtrans /dev/fd /hurd/magic --directory fd -The `/dev/fd` directory holds the open file descriptors for your current -process. You can't see them with `ls -l /dev/fd/` but you can see them +The `/dev/fd` directory holds the open [[unix/file_descriptor]]s for your +current process. You can't see them with `ls -l /dev/fd/` but you can see them individually like this: $ ls -l /dev/fd/0 diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn index 98447e98..ad104e68 100644 --- a/open_issues/code_analysis.mdwn +++ b/open_issues/code_analysis.mdwn @@ -8,7 +8,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -There is static and dynamic code analysis. +There is static and dynamic code analysis. This overlaps with [[debugging]]. * [[GCC]]'s warnings. Yes, really. @@ -29,3 +29,13 @@ There is static and dynamic code analysis. * * + + * [[Valgrind]] + + * + + * + + * + + * diff --git a/open_issues/debugging.mdwn b/open_issues/debugging.mdwn index 95b7bf9b..e66a086f 100644 --- a/open_issues/debugging.mdwn +++ b/open_issues/debugging.mdwn @@ -18,7 +18,7 @@ We have debugging infrastructure. For example: * [[GNU Mach debugging|microkernel/mach/gnumach/debugging]] * [[GNU Hurd debugging|hurd/debugging]], including - [[hurd/debugging/rpctrace]] and more. + [[hurd/debugging/rpctrace]], and more. # To Do @@ -29,14 +29,20 @@ We have debugging infrastructure. For example: * [[profiling]] - * *[Checkpoint/restart](http://lwn.net/Articles/412749/) allows the state of - a set of processes to be saved to persistent storage, then restarted at - some future time* -- quoting from Jonathan Corbet's 2010 Linux Kernel - Summit report. + * *Checkpoint/restart allows the state of a set of processes to be saved to + persistent storage, then restarted at some future time* -- quoting from + Jonathan Corbet's [2010 Linux Kernel Summit + report](http://lwn.net/Articles/412749/). This is surely a very useful facility to have for reproducing failures, for example. But on the other hand it's questionable how it can help with debugging failures in [[GNU Hurd server|hurd/translator]]s' interactions, as their state is typically spread between several processes. + Continues: , which introduces + . + * [[locking]] + + * , or -- + just two examples; there's a lot of such stuff for Linux. diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn index 81b96280..170734fd 100644 --- a/open_issues/multithreading.mdwn +++ b/open_issues/multithreading.mdwn @@ -22,9 +22,11 @@ Alternative approaches: * Continuation-passing style + * [[Erlang-style_parallelism]] + * [libtcr - Threaded Coroutine Library](http://oss.linbit.com/libtcr/) - * [[Erlang-style_parallelism]] + * --- diff --git a/open_issues/nightly_builds_deb_packages.mdwn b/open_issues/nightly_builds_deb_packages.mdwn index 29219c2a..9f5e2373 100644 --- a/open_issues/nightly_builds_deb_packages.mdwn +++ b/open_issues/nightly_builds_deb_packages.mdwn @@ -18,4 +18,10 @@ packages. --- +There is infrastructure available to test whole OS installations. + + * + +--- + See also [[nightly_builds]]. diff --git a/open_issues/secure_file_descriptor_handling.mdwn b/open_issues/secure_file_descriptor_handling.mdwn index c9956ede..1a514e69 100644 --- a/open_issues/secure_file_descriptor_handling.mdwn +++ b/open_issues/secure_file_descriptor_handling.mdwn @@ -8,7 +8,16 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!tag open_issue_glibc]] + `O_CLOEXEC`, `dup3` et al.; see . [[tschwinge]] once worked on this, posted patches to libc-alpha. This works needs to be resumed and finished. + +--- + +In an interesting point is made: *you [may] +want some [[unix/file_descriptor]] to still be open if 'exec' fails, but you +don't want it to be open after the exec succeeds*. [[I|tschwinge]]'m not sure +whether our current `O_CLOEXEC` implementation adheres to that. diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index 80a2860a..d50f5f6d 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -46,7 +46,14 @@ abandoned). * [*[ANNOUNCE] ktest.pl: Easy and flexible testing script for Linux Kernel Developers*](http://lwn.net/Articles/412302/) by Steven Rostedt, - 2010-10-28. + 2010-10-28. [v2](http://lwn.net/Articles/414064/), 2010-11-08. + + +# Related + + * [[nightly_builds]] + + * [[nightly_builds_deb_packages]] * -- ``comprehensive testing and benchmarking platform''. This one might be useful for [[performance]] diff --git a/open_issues/valgrind.mdwn b/open_issues/valgrind.mdwn new file mode 100644 index 00000000..2b0624d7 --- /dev/null +++ b/open_issues/valgrind.mdwn @@ -0,0 +1,80 @@ +[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Porting Valgrind to the Hurd"]] + +[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. +(And some other kinds of hard-to-find errors too.) +Aside from being useful for program development in general, +a Hurd port will help finding out why certain programs segfault on the Hurd, +although they work on Linux. +Even more importantly, it will help finding bugs in the Hurd servers themselfs. + +To keep track of memory use, +Valgrind however needs to know how each [[system call]] affects the validity of memory regions. +This knowledge is highly kernel-specific, +and thus Valgrind needs to be explicitely ported for every system. + +Such a port involves two major steps: +making Valgrind understand how kernel traps work in general on the system in question; +and how all the individual kernel calls affect memory. +The latter step is where most of the work is, +as the behaviour of each single [[system call]] needs to be described. + +Compared to Linux, +[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps. +Almost all [[system call]]s are implemented as [[RPC]]s instead -- +either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]]. +All RPCs use a pair of `mach_msg` invocations: +one to send a request message, and one to receive a reply. +However, while all RPCs use the same `mach_msg` trap, +the actual effect of the call varies greatly depending on which RPC is invoked -- +similar to the `ioctl` call on Linux. +Each request thus must be handled individually. + +Unlike `ioctl`, +the RPC invocations have explicit type information for the parameters though, +which can be retrieved from the message header. +By analyzing the parameters of the RPC reply message, +Valgrind can know exactly which memory regions are affected by that call, +even without specific knowledge of the RPC in question. +Thus implementing a general parser for the reply messages +will already give Valgrind a fairly good approximation of memory validity -- +without having to specify the exact semantic of each RPC by hand. + +While this should make Valgrind quite usable on the Hurd already, it's not perfect: +some RPCs might return a buffer that is only partially filled with valid data; +or some reply parameters might be optional, +and only contain valid data under certain conditions. +Such specific semantics can't be deduced from the message headers alone. +Thus for a complete port, +it will still be necessary to go through the list of all known RPCs, +and implement special handling in Valgrind for those RPCs that need it. + +The goal of this task is at minimum to make Valgrind grok Mach traps, +and to implement the generic RPC handler. +Ideally, specific handling for RPCs needing it should also be implemented. + +Completing this project will require digging into Valgrind's handling of [[system call]]s, +and into Hurd RPCs. +It is not an easy task, but a fairly predictable one -- +there shouldn't be any unexpected difficulties, +and no major design work is necessary. +It doesn't require any specific previous knowledge: +only good programming skills in general. +On the other hand, +the student will obtain a good understanding of Hurd RPCs while working on this task, +and thus perfect qualifications for Hurd development in general :-) + +Possible mentors: Samuel Thibault (youpi) + +Exercise: As a starter, +students can try to teach valgrind a couple of Linux ioctls, +as this will make them learn how to use the read/write primitives of valgrind. diff --git a/persistency.mdwn b/persistency.mdwn index f5347a4e..36f90c8a 100644 --- a/persistency.mdwn +++ b/persistency.mdwn @@ -1,18 +1,19 @@ -[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] A persistent object is an object that survives reboot. On [[Unix]], files and directories are persistent but -processes and file descriptors are not. [[microkernel/EROS]] is +processes and [[unix/file_descriptor]]s are not. [[microkernel/EROS]] is an example of an orthogonally persistent system: -processes and capabilities also survive reboot. To a +processes and [[capabilities|capability]] also survive reboot. To a process, it generally only looks as if it had not been scheduled for a long time; the rest of its environment remains essentially the indistinguishable. diff --git a/unix.mdwn b/unix.mdwn index 601b36d1..bf361e2e 100644 --- a/unix.mdwn +++ b/unix.mdwn @@ -19,9 +19,49 @@ License|/fdl]]."]]"""]] UNIX*](http://www.informit.com/articles/printerfriendly.aspx?p=691503), an article by David Chisnall. - * [*Ghosts of Unix Past: a historical search for design - patterns*](http://lwn.net/Articles/411845/) (2010-10-27) by Neil Brown, - including file descriptors and the single, hierarchical namespace. + * The first in the series, {{$2010_brown_ghosts_1}} introduces the concepts + of [[file_descriptor]]s and the single, hierarchical [[namespace]]. + + Next, {{$2010_brown_ghosts_2}} discusses issues with *conflated designs* + such as the `mount` command (a problem we have partly solved / solved + differently with our [[hurd/translator]] approach and the + [[hurd/virtual_file_system]]), and the plethora of flags that can be passed + to the `open` [[system_call]]. + + In {{$2010_brown_ghosts_3}}, he deals with *unfixable designs*, such as + [[UNIX signals|glibc/signals]] and the *UNIX permission model* (which is + clearly inferior to a [[capability]]-based system). * [*UNIX File Permissions*](http://www.greenend.org.uk/rjk/2004/perms.html) - (2004) by Richard Kettlewell. + (2004) by Richard Kettlewell. ([[!taglink open_issue_documentation]]) + + +[[!ymlfront data=""" + +djb_self-pipe: + + D. J. Bernstein's [*self-pipe trick*](http://cr.yp.to/docs/selfpipe.html) + +rjk_fork: + + Richard Kettlewell's suggestions about [*how fork(2) ought to + be*](http://www.greenend.org.uk/rjk/fork.html) + +2010_brown_ghosts_1: + + "Neil Brown's 2010-10-27 article [*Ghosts of Unix Past: a historical search + for design patterns*](http://lwn.net/Articles/411845/)" + +2010_brown_ghosts_2: + + "Neil Brown's 2010-11-04 article [*Ghosts of Unix past, part 2: Conflated + designs*](http://lwn.net/Articles/412131/)" + +2010_brown_ghosts_3: + + "Neil Brown's 2010-11-16 article [*Ghosts of Unix past, part 3: Unfixable + designs*](http://lwn.net/Articles/414618/)" + +"""]] diff --git a/unix/file_descriptor.mdwn b/unix/file_descriptor.mdwn new file mode 100644 index 00000000..16e03fdf --- /dev/null +++ b/unix/file_descriptor.mdwn @@ -0,0 +1,13 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +A *file descriptor* is a [[concept]] of [[UNIX]], and represents a +non-[[persistent|persistency]] handle to an object (a file, for example). With +respect to specific aspects, it is comparable to a [[capability]]. diff --git a/virtualization.mdwn b/virtualization.mdwn index 3a207ae8..78457eb9 100644 --- a/virtualization.mdwn +++ b/virtualization.mdwn @@ -6,8 +6,11 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + * [[hurd/virtualization]] in the GNU Hurd's context. + # External -- cgit v1.2.3