diff options
66 files changed, 1264 insertions, 413 deletions
diff --git a/.templates/autotag.tmpl b/.templates/autotag.tmpl index 1a963267..5ff710d2 100644 --- a/.templates/autotag.tmpl +++ b/.templates/autotag.tmpl @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this diff --git a/capability.mdwn b/capability.mdwn index ddadf137..7219cdce 100644 --- a/capability.mdwn +++ b/capability.mdwn @@ -11,7 +11,7 @@ License|/fdl]]."]]"""]] A capability is a protected reference. It is a reference in that it designates an object; it is protected in that in cannot be -forged. A capabilities both designates the object it refers to and +forged. A capability both designates the object it refers to and carries the authority to manipulate it. By binding [[designation]] and [[authorization]] together, capabilities @@ -25,7 +25,7 @@ to protect against A hijacking his authority. (This problem is refused to the [[confused_deputy]] problem.) Also, since A likely sent a string to identify the file to B, the identifier lacks a [[naming_context]] and therefore may resolve to a different object -than A intended. Be ensuring that [[designation]] and [[authorization]] are +than A intended. By ensuring that [[designation]] and [[authorization]] are always bound together, these problems are avoided. Capability-based system architectures strive to meet the *principle of least @@ -39,8 +39,8 @@ individually); attenuation design pattern, membranes (wikipedia_object-capability_model)? --> -A capability mechanism is typically implemented in software my the operating -system kernel (typically a [[microkernel]]. The computing cost (as compared to +A capability mechanism is typically implemented in software by the operating +system kernel (typically a [[microkernel]]). The computing cost (as compared to a hardware implementation) is neglectable. diff --git a/community/gsoc/project_ideas/maxpath.mdwn b/community/gsoc/project_ideas/maxpath.mdwn index 5be8917f..4a1314c2 100644 --- a/community/gsoc/project_ideas/maxpath.mdwn +++ b/community/gsoc/project_ideas/maxpath.mdwn @@ -13,7 +13,7 @@ is included in the section entitled POSIX describes some constants (or rather macros) like PATH_MAX/MAXPATHLEN and similar, which may be defined by the system to indicate certain limits. Many people overlook the *may* though: Systems only should define them if they -actually have such fixed limits. The Hurd, following the GNU Coding Standards, +actually have such fixed limits (see [limits.h](http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html)). The Hurd, following the GNU Coding Standards, tries to avoid this kind of arbitrary limits, and consequently doesn't define the macros. diff --git a/community/meetings.mdwn b/community/meetings.mdwn index 6c15d144..4ae52a1a 100644 --- a/community/meetings.mdwn +++ b/community/meetings.mdwn @@ -13,15 +13,17 @@ License|/fdl]]."]]"""]] # Upcoming - * [[GNU Hackers Meeting, 2011, Paris|ghm2011]] ## In the Future + * [[FOSDEM_2012]] * [[Self-organised]] # Past + * [[FrOSCon_2011]] + * [[GNU Hackers Meeting, 2011, Paris|ghm2011]] * [[FOSDEM_2011]] * [[DebConf10]] * [[GNU Hackers Meeting, 2010, Den Haag|ghm2010]] diff --git a/community/meetings/fosdem_2012.mdwn b/community/meetings/fosdem_2012.mdwn new file mode 100644 index 00000000..b6f31efc --- /dev/null +++ b/community/meetings/fosdem_2012.mdwn @@ -0,0 +1,34 @@ +[[!meta copyright="Copyright © 2006, 2007, 2008, 2009, 2010, 2011 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FOSDEM 2012"]] + +<http://fosdem.org/2012> + +FOSDEM will take place on February 4th/5th at the Université Libre de +Bruxelles. + + +# Who and When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return","Share room with us" +"[[Maksym Planeta]]","no" +"Olaf Buddenhagen","most likely","","","yes" +"Richard Braun","no" +"Svante Signell","no" +"[[Thomas Schwinge|tschwinge]]","will try to","","","yes" +"""]] + + +# Multiserver, Microkernel-Based Operating Systems Devroom + +[Announcement](http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00137.html). diff --git a/community/meetings/froscon_2011.mdwn b/community/meetings/froscon_2011.mdwn new file mode 100644 index 00000000..b15140d6 --- /dev/null +++ b/community/meetings/froscon_2011.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FrOSCon, 2011, Sankt Augustin, Germany"]] + +<http://www.froscon.de/> + + * [Arch Hurd booth](http://www.froscon.de/en/exhibitors/projekte.html#c1413) diff --git a/community/meetings/ghm2011.mdwn b/community/meetings/ghm2011.mdwn index 7a2df8a0..8e77d500 100644 --- a/community/meetings/ghm2011.mdwn +++ b/community/meetings/ghm2011.mdwn @@ -11,3 +11,17 @@ License|/fdl]]."]]"""]] [[!meta title="GNU Hackers Meeting, 2011, Paris"]] <http://www.gnu.org/ghm/2011/paris/> + + * {{$thibault_hurd}} + + +[[!ymlfront data=""" + +thibault_hurd: + + "presentation by Samuel Thibault: [*GNU/Hurd, aka. Extensibility from the + Ground*](http://www.gnu.org/ghm/2011/paris/#outline-container-2-5) + ([slides](http://www.gnu.org/ghm/2011/paris/slides/samuel-thibault-hurd.pdf), + [video](http://audio-video.gnu.org/video/ghm2011/Samuel_Thibault-GNU_Hurd.ogv))" + +"""]] diff --git a/config_edittemplate/open_issue_page.mdwn b/config_edittemplate/open_issue_page.mdwn index ede41ad8..172614eb 100644 --- a/config_edittemplate/open_issue_page.mdwn +++ b/config_edittemplate/open_issue_page.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this diff --git a/config_edittemplate/regular_page.mdwn b/config_edittemplate/regular_page.mdwn index 68669f84..3f95f9b5 100644 --- a/config_edittemplate/regular_page.mdwn +++ b/config_edittemplate/regular_page.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this diff --git a/contributing/web_pages.mdwn b/contributing/web_pages.mdwn index d1b3c0fc..b2e96121 100644 --- a/contributing/web_pages.mdwn +++ b/contributing/web_pages.mdwn @@ -130,13 +130,22 @@ is also read-only. $ git clone http://www.bddebian.com:8888/git/hurd-web [dest] -For all cases: if you omit `[dest]` it will default to `hurd-web`. +Or, you can check out the Savannah repository: -Later, you can just `cd` into the `hurd-web` directory and run a `git pull` to -get hold of the latest changes others have been installing in the mean time. -(In most cases, you should use `git pull --rebase`, -to avoid useless *Merge branch ...* messages. See the -Git documentation for details.) + $ git clone git://git.savannah.gnu.org/hurd/web.git [dest] + +See <http://git.savannah.gnu.org/cgit/hurd/web.git>. If you're using the `ssh` +protocol, and you're a member of the Hurd's [[rules/Savannah_group]], you can +also push to this repository. The disadvantage of pushing to the Savannah +repository is that there is no [[ikiwiki]] installation where the pushed +changes are immediatelly rendered and viewable by everyone. + +For all cases: if you omit `[dest]` it will default to `hurd-web` for the +`bddebian.com` repositories, or `web` for a Savannah clone. + +Later, you can just `cd` into the `hurd-web` or `web` directory, and, for +example, run `git pull` to get hold of the latest changes others have been +installing in the mean time. ## Editing the Content diff --git a/contributing/web_pages/news/moth_next.mdwn b/contributing/web_pages/news/moth_next.mdwn index 2555cd73..9b0844bd 100644 --- a/contributing/web_pages/news/moth_next.mdwn +++ b/contributing/web_pages/news/moth_next.mdwn @@ -27,43 +27,96 @@ else=" This month [hurd hacker] [item] -Also … +<http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00079.html> -- Bouju +Alain submitted a patch to suport cpuinfo in the /proc interface -[our hackers] … +rbraun committed the last patch to mplanetas branch of the slab allocator +work, for integration. -Mainly thanks to … +IRC, freenode, #hurd, 2011-11-14: -Additionally … +Features: -And … + (22:30:39) braunr: there shouldn't be any noticeable difference with the + master branch + (22:30:46) braunr: a bit less fragmentation + (22:30:55) braunr: more memory can be reclaimed by the VM system + (22:31:02) braunr: there are debugging features + (22:31:06) braunr: it's SMP ready + (22:31:15) braunr: and overall cleaner than the zone allocator + (22:31:31) braunr: although a bit slower on the free path (because of + what's performed to reduce fragmentation) + (22:32:42) braunr: but even "slower" here is completely negligible -[reason for contibuting to the Hurd] +**New porter box: exodar*** -<!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.--> +I/O Path Documentation [[hurd/io_path/]] + +Debugging: + +- Pino Toscano: recvfrom() with null http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00161.html +- Maksym Planeta: tmpfs http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00125.html http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00118.html +- Samuel Thibault: libtool http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00073.html mknod http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00070.html +- Jim Meyering: gnu tools with user id 4294967295 http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00094.html +- Paul Eggert: Add error-checking on GNU http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00130.html +- + +Bouju Alain: Procfs with cpuinfo http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00084.html + +Social network sites for GNU Hurd: + +- Google+: https://plus.google.com/114942488385711891227#114942488385711891227/posts +- identi.ca: http://identi.ca/group/hurd + +Ludovic Courtès: - * [[toolchain/ELFOSABI_GNU]] +- Continuously-built Nix-based QEMU image: http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00047.html +- modern Autoconf initialization: http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00068.html +- Allow /hurd/init to be a symlink: http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00032.html +- The Hurd now builds with Savannah’s libc (2.14+): http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00025.html - * [Arch Hurd, DDE](http://www.archhurd.org/news/22/) +Pino Toscano and Thomas Schwinge: pthread improvements: http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00027.html http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00020.html http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00013.html - * Arch Hurd will have a booth at [FrOSCon](http://www.froscon.org/). +Sergio Lopez: Better Memory Management and memfs: http://www.bddebian.com/~hurd-web/user/Sergio_Lopez/ - * Bits from the Debian GNU/Hurd porters, - id:"20110721172827.GF4057@const.famille.thibault.fr", - <http://lists.debian.org/debian-devel-announce/2011/07/msg00002.html> +(bug hurd done till 2011-11 (inclusive)) - * [[2011-q2-ps]] +TODO: +- http://lists.gnu.org/archive/html/bug-hurd/2011-12 +- http://lists.debian.org/debian-hurd/2011/10/threads.html - * GSoC end +------ - * mcsim memory allocator project +Add stuff here? (This is already in q3) - * GHM. +Also … + +[our hackers] … + +Mainly thanks to … + +Additionally … + +And … - * <http://audio-video.gnu.org/video/ghm2011/>; Samuel. Slides. Also add - all to media. +So if you want to [reason for contibuting to the Hurd], +please [[get in contact|contact_us]] -- and maybe already grab the [[source +code|source_repositories]]. - * Richard's new buildd. +--- - Is this for Q4? +The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a +collection of servers that run on the Mach microkernel to implement file +systems, network protocols, file access control, and other features that are +implemented by the Unix kernel or similar kernels (such as Linux). [[More +detailed|hurd/documentation]]. + +**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It +provides an Inter Process Communication (IPC) mechanism that the Hurd uses to +define interfaces for implementing in a distributed multi-server fashion the +services a traditional operating system kernel provides. [[More +detailed|microkernel/mach/gnumach]]. + +<!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.--> """]] diff --git a/contributing/web_pages/news/skeleton.mdwn b/contributing/web_pages/news/skeleton.mdwn index 34c7e2f8..17227476 100644 --- a/contributing/web_pages/news/skeleton.mdwn +++ b/contributing/web_pages/news/skeleton.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -37,7 +37,23 @@ Additionally … And … -[reason for contibuting to the Hurd] +So if you want to [reason for contibuting to the Hurd], +please [[get in contact|contact_us]] -- and maybe already grab the [[source +code|source_repositories]]. + +--- + +The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a +collection of servers that run on the Mach microkernel to implement file +systems, network protocols, file access control, and other features that are +implemented by the Unix kernel or similar kernels (such as Linux). [[More +detailed|hurd/documentation]]. + +**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It +provides an Inter Process Communication (IPC) mechanism that the Hurd uses to +define interfaces for implementing in a distributed multi-server fashion the +services a traditional operating system kernel provides. [[More +detailed|microkernel/mach/gnumach]]. <!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.--> diff --git a/copyright.mdwn b/copyright.mdwn index 24dc1725..31f24e32 100644 --- a/copyright.mdwn +++ b/copyright.mdwn @@ -1,2 +1,2 @@ -Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 The -Contributing Authors +Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012 +The Contributing Authors diff --git a/donate.mdwn b/donate.mdwn index 938f950b..6293633e 100644 --- a/donate.mdwn +++ b/donate.mdwn @@ -91,30 +91,6 @@ Please don't hesitate to ask [[Thomas Schwinge|tschwinge]] if you need help. Continue to explore the [[list of open bounties|tag/bounty]]. -# g10 Code Maintenance Points - -If you've got more money on hand than hacking time, you might consider buying -some [maintenance points](http://g10code.com/products.html#maintpoints) (EUR 10 a point) to -help the Hurd along. From the [g10 Code](http://www.g10code.com/products.html) -site: - -> Hurd Maintenance Points are special: Some of our employees are well known -> Hurd hackers in their spare time; collected points for this program will be -> given to them in form of paid time. - -See also this related -[mailing-list](http://mail.gnu.org/archive/html/help-hurd/2003-04/msg00044.html) -thread. - -And for further motivation, some words of wisdom from Marcus Brinkmann: - -> By the way, if you are more on the speculating side, then it can't harm to -> just buy one or two maintenance points. That means that at some time I get -> an incentive to start the hacking, and there is a chance that when I start I -> don't stop for a while, and just continue on my private time (as I did for -> the last five years, if I might add that ;). - - # Hurd Developer Meetings Another possibility is to meet with the Hurd developers at a @@ -23,8 +23,17 @@ repository|source_repositories/glibc]]. # Specifics + +## Ports + Porting glibc to a specific architecture is non-trivial. +The main port is x86, which is somewhat complete and is maintained. There is +an incomplete and unmaintained port for PowerPC. There [[!message-id +desc="have" "20111129161104.25123.qmail@sourceware.org"]] [[!message-id +desc="been" "Pine.LNX.4.64.1111291611170.26895@digraph.polyomino.org.uk"]] +incomplete and largely unmaintained ports for Alpha and MIPS in glibc-ports. + ## [[Hurd-specific Port|hurd/glibc]] diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn index 9417106d..12ca2d19 100644 --- a/glibc/fork.mdwn +++ b/glibc/fork.mdwn @@ -53,6 +53,9 @@ they have patches for software packages, to avoid using `fork` followed by * Can we/why can't we use the concept of *inherited ports array*s/`mach_ports_register` ([[!taglink open_issue_glibc]])? + * GNUnet `vfork` signal race issue: [[!message-id + "87r50ww6m4.fsf@kepler.schwinge.homeip.net"]]. + ## Related @@ -65,7 +65,6 @@ in the *unstable* branch of the Debian archive. ## Common Problems -* [[Console]] * [[Xfree86]] -- [[DebianX]] -- [[DebianXorg]] * [[GNUstep]] * [[XattrHurd]]: Setting translators under GNU/Linux @@ -94,9 +93,11 @@ in the *unstable* branch of the Debian archive. * [[libnetfs]] -- short introductory material * [[libdiskfs]] * [[libihash]] + * [[libthreads]] * [[libpthread]] * [[IO_Path]] * [[Porting]] * [[Debugging]] * [Hurd Sourcecode Reference](http://www.htu.tugraz.at/~past/hurd/global/): Searchable and browsable index of the code. * [[Networking]] +* [[Console]] diff --git a/hurd/console.mdwn b/hurd/console.mdwn index 4f976efd..f7230011 100644 --- a/hurd/console.mdwn +++ b/hurd/console.mdwn @@ -1,3 +1,88 @@ +[[!meta copyright="Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2011 +Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The Hurd console's implementation is broken into two pieces each running on +it's own process, the console client and server. + +The console client is also split into modules (input driver, display driver, +speaker, ...) but they all run in the same process. + +The console server puts itself as a translator on top of `/dev/vcs` folder +presenting the following hierarchy: + + + /dev/vcs + \ + +- 1 + \ + +- console + +- input + +- display + +- .. + +- n + +where the numbered nodes represent virtual consoles and their contents are all +alike. + +As the following graph shows, the console, input and display nodes are the +interfaces used by the terminal server, input driver and display drivers +respectively. + + +------------------+ +-----------------+ + | Input driver | | Terminal Server | + | | | | + | pc-kbd | | | + +------------------+ +-----------------+ + | _cons_vcons_input | + | writes to reads | + | vcs/i/input vcs/i/console | + | +-----------------+ | + | | Console Server | | + | | /hurd/console | | + | input_enqueue | --------------- | input_dequeue | + +--------------->| Input Queue |>---------------+ + | --------------- | + +--------------->| Output Buffer |>---------------+ + | +-----------------+ | + | | + | writes reads | + | vcs/i/console vcs/i/display | + | | + +----------------+ +-----------------+ + | Teminal Server | | Display driver | + | | | | + | /hurd/term | | vga | + +----------------+ +-----------------+ + +The input driver takes scancodes from the in-kernel kbd queue, translates them +into characters and writes them to the input node. Then the terminal server +reads the console node taking the characters out of the console server. + +Each of theese actions is actually an RPC handled by the translator on +`/dev/vcs`. Writes to input nodes are handled by calling `input_enqueue` to +put the character into a queue. And reads from console nodes are handled by +calling `input_dequeue` which takes out charecters from the queue and gives +them to the reader. + +It's important to note here that both `input_enqueue` and `input_dequeue` are +blocking operations and a blocked `input_dequeue` necessarily needs an +`input_enqueue` call to continue. + +[[RPC]]s are handled by the console server with the help of [[hurd/libports]]' +`ports_manage_multithreaded` API. + + +--- + +/!\ old content; [[!taglink open_issue_documentation]]: cleanup needed. + The below is a reworked version of Marcus Brinkmann's [letter to the debian-hurd list](http://lists.debian.org/debian-hurd/2002/debian-hurd-200209/msg00054.html). It describes how to setup the new console server for the Hurd. I am testing this right now, so this document is a work in progress. -- [[Main/JoachimNilsson]] - 21 Jan 2003 diff --git a/hurd/dde/guide.mdwn b/hurd/dde/guide.mdwn index a3c08754..6a83519c 100644 --- a/hurd/dde/guide.mdwn +++ b/hurd/dde/guide.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010,2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this diff --git a/hurd/io_path.mdwn b/hurd/io_path.mdwn index 492edffe..c47b5dca 100644 --- a/hurd/io_path.mdwn +++ b/hurd/io_path.mdwn @@ -46,9 +46,9 @@ Back in `_Xio_read`. If the 2048 byte buffer is not decided to be used (out-of-line case or bigger than 2048 bytes case; server decides to instead provide a new memory region), -the [[`dealloc`|microkernel/mach/mig/dealloc]] flag is being set, which causes -Mach to unmap that memory region from the server's address space, i.e., doing a -memory *move* from the server to the client. +the [[`dealloc`|microkernel/mach/mig/documentation/dealloc]] flag is being set, +which causes Mach to unmap that memory region from the server's address space, +i.e., doing a memory *move* from the server to the client. Leave server-side RPC stub `_Xio_read`. diff --git a/hurd/libpager.mdwn b/hurd/libpager.mdwn index 99f28f2a..d844743d 100644 --- a/hurd/libpager.mdwn +++ b/hurd/libpager.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] Mach's [[microkernel/mach/external_pager_mechanism]]. @@ -16,6 +16,16 @@ Mach [[microkernel/mach/IPC]]'s [[microkernel/mach/ipc/sequence_numbering]]. [GNU Hurd Reference Manual: 4.2 Pager Library](http://www.gnu.org/software/hurd/doc/hurd_5.html#SEC32). + +# Writeback: Writing Out Dirty Pages + + +## Related + + * LWN, Jonathan Corbet, [*No-I/O dirty + throttling*](http://lwn.net/Articles/456904/), 2011-08-31. + + # Open Issues * [[open_issues/linux_vmsig]] diff --git a/hurd/libports.mdwn b/hurd/libports.mdwn index 28274338..6f2cd46d 100644 --- a/hurd/libports.mdwn +++ b/hurd/libports.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -18,3 +18,20 @@ provide an interface independently of the underlying [[microkernel]]. *libports* does not itself depend on *[[libthreads]]*, but the appropriate threading hooks are used if present, that is if *[[libthreads]]* is used by another component. + + +# Message Processing + +## `ports_manage_multithreaded` + +When a message is recieved, the thread acting as receiver checks if any other +thread is also waiting for requests. If there is none, a new thread is +spawned. Thus, the current thread continues processing the message while the +newly created thread starts listening for new ones. ([[!taglink +open_issue_hurd]]: [[open_issues/multithreading]].) + +Also, there are configurable timeouts for [[translator]]s who want to go away +when they are not used. ([[!taglink open_issue_hurd]]: there used to be bugs +in this area, [[!message-id "87hev152we.fsf@becket.becket.net"]], but it may be +fixed as of [[!message-id "20111030210045.GA4983@myhost"]], +[[!GNU_Savannah_Git_hurd_hurd 9b5429e834cde56f73b8ff605e36afc7d9bb6e1b]].) diff --git a/hurd/libthreads.mdwn b/hurd/libthreads.mdwn new file mode 100644 index 00000000..8b1a97e6 --- /dev/null +++ b/hurd/libthreads.mdwn @@ -0,0 +1,28 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +`libthreads` a.k.a. C threads. + + +# Internals + + +## Threads' Death + +C threads death doesn't actually free the thread's stack (and maybe not the +associated Mach ports either). That's because there's no way to free the stack +after the thread dies (because the thread of control is gone); the stack needs +to be freed by something else, and there's nothing convenient to do it. There +are many ways to make it work. + +However, it isn't really a leak, because the unfreed resources do get used for +the next thread. So the issue is that the shrinkage of resource consumption +never happens, but it doesn't grow without bounds; it just stays at the maximum +even if the current number of threads is lower. diff --git a/hurd/porting/guidelines.mdwn b/hurd/porting/guidelines.mdwn index 07807b59..e47d4408 100644 --- a/hurd/porting/guidelines.mdwn +++ b/hurd/porting/guidelines.mdwn @@ -308,6 +308,14 @@ Not implemented, not POSIX. Try to disable the feature in the package. There is no programming interface for the parallel port on GNU/Hurd yet. +## <a name="cdrom"> <linux/cdrom.h> </a> + +Use <sys/cdrom.h> instead. + +## <a name="baud"> CBAUD </a> + +This is not actually standard; cfsetspeed, cfsetispeed, or cfsetospeed should be used instead. + ## <a name="errno"> `errno` values </a> When dealing with `errno`, you should always use the predefined error codes defined with the `E*` constants, instead of manually comparing/assigning/etc with their values. diff --git a/hurd/running/debian/faq/xserver-common.mdwn b/hurd/running/debian/faq/xserver-common.mdwn index 09fbc902..d1b17f31 100644 --- a/hurd/running/debian/faq/xserver-common.mdwn +++ b/hurd/running/debian/faq/xserver-common.mdwn @@ -8,5 +8,5 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -You need to run `dpkg-reconfigure xserver-common` and select `Anybody` for +You need to run `dpkg-reconfigure x11-common` and select `Anybody` for starting X as there is no way to detect console users currently. diff --git a/hurd/running/gnu/universal_package_manager.mdwn b/hurd/running/gnu/universal_package_manager.mdwn index 58841b02..bf1b92e0 100644 --- a/hurd/running/gnu/universal_package_manager.mdwn +++ b/hurd/running/gnu/universal_package_manager.mdwn @@ -155,3 +155,4 @@ To join the project just list your name below. 8. Abhradip Mukherjee 9. Ermenegildo Fiorito 10. Oltion Doda + 11. Russell James diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index bf7af3ce..3527267f 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -80,6 +80,7 @@ The [[concept|concepts]] of translators creates its own problems, too: # Existing Translators +* [[hello]] * [[auth]] * [[exec]] * [[pfinet]] diff --git a/hurd/translator/hello.mdwn b/hurd/translator/hello.mdwn new file mode 100644 index 00000000..bd56cd76 --- /dev/null +++ b/hurd/translator/hello.mdwn @@ -0,0 +1,14 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The *hello* translator is an example of a simple [[libtrivfs]]-based one-node +[[translator]]. It is shipped as part of the [[Hurd source code +repository|source_repositories]], and exists in a single-threaded and a +multi-threaded variant. diff --git a/media_appearances.mdwn b/media_appearances.mdwn index b52b80bf..c005c381 100644 --- a/media_appearances.mdwn +++ b/media_appearances.mdwn @@ -18,6 +18,11 @@ A lot of stuff is missing here. # 2011 +## August + + * GNU Hackers Meeting in Paris: {{$community/meetings/ghm2011#thibault_hurd}} + + ## July diff --git a/microkernel/mach/gnumach.mdwn b/microkernel/mach/gnumach.mdwn index d9ff6535..edd0cfdb 100644 --- a/microkernel/mach/gnumach.mdwn +++ b/microkernel/mach/gnumach.mdwn @@ -9,7 +9,10 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -GNU Mach is the microkernel that the GNU/Hurd system is based on. +GNU Mach is the microkernel upon which a GNU Hurd system is based. It provides +an Inter Process Communication (IPC) mechanism that the Hurd uses to define +interfaces for implementing in a distributed multi-server fashion the services +a traditional operating system kernel provides. It is maintained by the Hurd developers for the GNU project and remains compatible with [[Mach]] 3.0. diff --git a/news/2011-q3.mdwn b/news/2011-q3.mdwn new file mode 100644 index 00000000..c1a78319 --- /dev/null +++ b/news/2011-q3.mdwn @@ -0,0 +1,116 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta date="2011-11-17 14:15 UTC"]] + +A quarter of the Hurd: *Arch Hurd with DDE*, *Debian boxes*, *GHM talk* and +*GSoC: Java*. +[[!if test="included()" then="""[[!toggle id=full_news +text="Details."]][[!toggleable id=full_news text="[[!paste id=full_news]]"]]""" +else=" +[[!paste id=full_news]]"]] + +[[!cut id="full_news" text=""" + +In the third quarter of 2011, the Arch Hurd hackers [packaged DDE (Device +Driver Environment)](http://www.archhurd.org/news/22/), so a subset of the +Linux 2.6 device drivers can now easily be run as user-space processes on Arch +Hurd, replacing GNU Mach's in-kernel device drivers. (This has been possible +before, too, but involved several [[manual steps|hurd/dde/guide]].) At the +time of writing, our DDE port supports several network cards, while for other +driver types we will need to add further generic infrastructure. Also, Arch +Hurd had [a booth at +FrOSCon](http://www.froscon.de/en/exhibitors/projekte.html#c1413) and [released +a new Arch Hurd LiveCD](http://www.archhurd.org/news/24/), so new users can +easily test the current state of the Arch flavor of the Hurd. + +Richard Braun contributed additional GNU Hurd instances: [[a *Debian buildd*, a +*Debian porterbox*, and a *public Hurd box*|public_hurd_boxen]]. Especially +the last one is important for *you*: after requesting an account, you can use +it to test the Hurd without any own setup. + +Samuel Thibault sent a new [Bits from the Debian GNU/Hurd +porters](http://lists.debian.org/debian-devel-announce/2011/07/msg00002.html) +to keep the Debian folks up to date with our progres. And it is quite good: +thanks to the relentless work of our porters, you can now use [70 % of all +Debian packages with the Hurd](https://buildd.debian.org/stats/graph-big.png), +so we're getting closer to [the goal of finishing a Release Canditate in time +for Debian Wheezy](http://wiki.debian.org/Debian_GNU/Hurd). If you can, for +example, port Debian packages and want to help the Hurd, this is the perfect +time to get in contact and [port your favorite missing +package](http://www.debian.org/ports/hurd/hurd-devel-debian) to the Hurd. + +A different kind of status update was delivered by Samuel Thibault on the [[GNU +Hacker Meeting (GHM) in Paris|community/meetings/ghm2011]]. We hope you enjoy +watching the video of the {{$community/meetings/ghm2011#thibault_hurd}}. He +nicely explains how the simple yet powerful concept of a [[hurd/translator]] +gives power to a system's less-priviledged users (that is, without `root` +access), without any security implications, and how [[hurd/subhurd]]s and +[[hurd/neighborhurd]]s compare to Linux containers. *It's all about [freedom +0](http://www.gnu.org/philosophy/free-sw.html)*. + +On the technical side, Thomas Schwinge improved the technical documentation of +the [[I/O path|hurd/io_path]] when translators are involved, to make it easier +for new developers to understand how all the different system components +interact. Amongst others, Guillem Jover, Fridolín Pokorný and Jonathan +Neuschäfer +[sent](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00184.html) +[many](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00093.html) +[patches](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00030.html) for +GNU Mach, improving stability, fixing memory leaks and generally cleaning up +the code. + +Maksym Planeta finished a project he has been doing as a university task: +replace GNU Mach's old zone memory allocator with a new [[!wikipedia +slab_allocation desc="slab allocator"]] written by Richard Braun, who also +mentored Maksym during the project. [This +allocator](http://git.savannah.gnu.org/cgit/hurd/gnumach.git/commit/?h=mplaneta/libbraunr/master&id=59c9da87375ad3c8401890ecd4f7f101093f2463), +apart from being overally cleaner than the zone allocator, is meant to waste +less memory than the zone allocator (less fragmentation and more memory can be +reclaimed by the VM system), there are debugging/inspection features, and it's +SPM-ready, thus readily usable once we get up-do-date SMP support in GNU Mach. +It is now being tested and integrated. + +And last but definitely not least, Jérémie Koenig finished his Google Summer of +Code project to [[improve Java support on GNU Hurd|user/jkoenig/java]]. All in +all, he also [improved the Hurd signalling +code](http://lists.gnu.org/archive/html/bug-hurd/2011-06/msg00073.html), ported +OpenJDK and began designing and creating a [library for Java bindings for Mach +and Hurd](https://github.com/jeremie-koenig/hurd-java) which already allows +writing a [Hello World translator in +Java](https://github.com/jeremie-koenig/hurd-java/blob/master/HelloMach.java). +It is still pretty low-level, but it paves the way for extending the core of +the Hurd with Java, which is one of the benefits of the Hurd's distributed +multi-server architecture: different components of the operating system can be +written in different programming languages; not just +[C](http://www.gnu.org/software/hurd/hacking-guide/hhg.html#An-Example-using-trivfs), +but also C++, [[Common Lisp|user/flaviocruz]], and now Java -- and more to +come. + +So if you want to help getting the Debian GNU/Hurd Release Candidate done, or +want to dig deep into DDE to have more device drivers running as user-space +processes, please [[get in contact|contact_us]] -- and maybe already grab the +[[source code|source_repositories]]. + +--- + +The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a +collection of servers that run on the Mach microkernel to implement file +systems, network protocols, file access control, and other features that are +implemented by the Unix kernel or similar kernels (such as Linux). [[More +detailed|hurd/documentation]]. + +**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It +provides an Inter Process Communication (IPC) mechanism that the Hurd uses to +define interfaces for implementing in a distributed multi-server fashion the +services a traditional operating system kernel provides. [[More +detailed|microkernel/mach/gnumach]]. + +"""]] diff --git a/open_issues/binutils.mdwn b/open_issues/binutils.mdwn index b86ed607..246e819d 100644 --- a/open_issues/binutils.mdwn +++ b/open_issues/binutils.mdwn @@ -30,8 +30,8 @@ though, as explained below. # Configuration -Last reviewed up to the [[Git mirror's 066f3707e988b24f8f39d01f105662a814c0979a -(2011-10-11) sources|source_repositories/binutils]]. +Last reviewed up to the [[Git mirror's e605e0bc9fced128199177fff775851c7e50b4e5 +(2011-10-20) sources|source_repositories/binutils]]. * Globally @@ -108,7 +108,7 @@ Last reviewed up to the [[Git mirror's 066f3707e988b24f8f39d01f105662a814c0979a # Build Here's a log of a binutils build run; this is from our [[Git repository's -11637a710adc7baea102d0ef5975b7a92b07aea1 (2011-10-11) +6e78bdc1d64c55962d36ef29d15f06fb2415261b (2011-10-20) sources|source_repositories/binutils]], run on kepler.SCHWINGE and coulomb.SCHWINGE. diff --git a/open_issues/binutils/log_build.diff b/open_issues/binutils/log_build.diff index 79172cf6..79946f63 100644 --- a/open_issues/binutils/log_build.diff +++ b/open_issues/binutils/log_build.diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2011-10-11 14:23:15.275568065 +0200 -+++ /dev/fd/62 2011-10-11 14:23:15.275568065 +0200 +--- /dev/fd/63 2011-10-22 11:44:16.461445903 +0200 ++++ /dev/fd/62 2011-10-22 11:44:16.461445903 +0200 @@ -276,12 +276,12 @@ checking for sys/sysinfo.h... yes checking for machine/hal_sysinfo.h... no @@ -214,7 +214,7 @@ checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes -@@ -2473,7 +2452,7 @@ +@@ -2475,7 +2454,7 @@ checking for BSD- or MS-compatible name lister (nm)... nm checking the name lister (nm) interface... BSD nm checking whether ln -s works... yes @@ -223,7 +223,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -2494,7 +2473,7 @@ +@@ -2496,7 +2475,7 @@ checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no @@ -232,7 +232,7 @@ checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes -@@ -2510,7 +2489,7 @@ +@@ -2512,7 +2491,7 @@ checking if g++-4.6 supports -c -o file.o... yes checking if g++-4.6 supports -c -o file.o... (cached) yes checking whether the g++-4.6 linker (ld) supports shared libraries... yes @@ -241,7 +241,7 @@ checking how to hardcode library paths into programs... immediate checking whether NLS is requested... yes checking for catalogs to be installed... bg da es fi fr ga id ja sv tr vi zh_CN zh_TW -@@ -2590,13 +2569,13 @@ +@@ -2592,13 +2571,13 @@ /bin/dash ../../master/ld/../ylwrap ../../master/ld/ldgram.y y.tab.c ldgram.c y.tab.h ldgram.h y.output ldgram.output -- bison -y -d updating ldgram.h (echo "/* This file is automatically generated. DO NOT EDIT! */";\ @@ -257,7 +257,7 @@ | sed -e 's/ e/ ld/g' -e 's/ ld/ /g' -e 's/[.]o//g'`; do \ echo " &ld_${f}_emulation, \\"; \ done;\ -@@ -2710,17 +2689,11 @@ +@@ -2712,17 +2691,11 @@ gcc-4.6 -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT plugin.o -MD -MP -MF .deps/plugin.Tpo -c -o plugin.o ../../master/ld/plugin.c mv -f .deps/plugin.Tpo .deps/plugin.Po cp ../../master/ld/emultempl/astring.sed stringify.sed diff --git a/open_issues/binutils/log_install.diff b/open_issues/binutils/log_install.diff index 27dff188..100ebf5c 100644 --- a/open_issues/binutils/log_install.diff +++ b/open_issues/binutils/log_install.diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2011-10-11 14:23:22.899444227 +0200 -+++ /dev/fd/62 2011-10-11 14:23:22.903444162 +0200 +--- /dev/fd/63 2011-10-22 11:44:48.780903894 +0200 ++++ /dev/fd/62 2011-10-22 11:44:48.780903894 +0200 @@ -68,7 +68,6 @@ libtool: install: /usr/bin/install -c .libs/libbfd.a [...]/hurd/master.build.install/lib/libbfd.a libtool: install: chmod 644 [...]/hurd/master.build.install/lib/libbfd.a diff --git a/open_issues/binutils/sum.diff b/open_issues/binutils/sum.diff index a466b82c..0c612411 100644 --- a/open_issues/binutils/sum.diff +++ b/open_issues/binutils/sum.diff @@ -1,9 +1,9 @@ ---- open_issues/binutils/sum_linux 2011-10-11 14:23:39.000000000 +0200 -+++ open_issues/binutils/sum_hurd 2011-10-11 14:23:33.000000000 +0200 +--- open_issues/binutils/sum_linux 2011-10-22 11:49:33.000000000 +0200 ++++ open_issues/binutils/sum_hurd 2011-10-22 11:23:09.000000000 +0200 @@ -1,5 +1,5 @@ --Test Run By thomas on Tue Oct 11 14:12:56 2011 +-Test Run By thomas on Sat Oct 22 11:43:32 2011 -Native configuration is i686-pc-linux-gnu -+Test Run By thomas on Tue Oct 11 13:36:51 2011 ++Test Run By thomas on Fri Oct 21 12:54:58 2011 +Native configuration is i686-unknown-gnu0.3 === binutils tests === @@ -12,9 +12,9 @@ # of expected passes 90 # of unsupported tests 3 --Test Run By thomas on Tue Oct 11 14:13:19 2011 +-Test Run By thomas on Sat Oct 22 11:43:59 2011 -Native configuration is i686-pc-linux-gnu -+Test Run By thomas on Tue Oct 11 13:38:54 2011 ++Test Run By thomas on Fri Oct 21 12:56:51 2011 +Native configuration is i686-unknown-gnu0.3 === ld tests === @@ -60,20 +60,20 @@ PASS: ELF DSO small bar (size) PASS: ELF DSO foo with small bar (size) PASS: ELF DSO big bar (size) -@@ -939,13 +939,13 @@ Running [...]/hurd/master/ld/testsuite/l +@@ -940,13 +940,13 @@ Running [...]/hurd/master/ld/testsuite/l === ld Summary === --# of expected passes 671 +-# of expected passes 672 -# of expected failures 8 -+# of expected passes 661 ++# of expected passes 662 +# of expected failures 18 # of untested testcases 6 - [...]/hurd/master.build/ld/ld-new 2.22.51.20111011 + [...]/hurd/master.build/ld/ld-new 2.22.51.20111021 --Test Run By thomas on Tue Oct 11 14:13:02 2011 +-Test Run By thomas on Sat Oct 22 11:43:38 2011 -Native configuration is i686-pc-linux-gnu -+Test Run By thomas on Tue Oct 11 13:37:18 2011 ++Test Run By thomas on Fri Oct 21 12:55:21 2011 +Native configuration is i686-unknown-gnu0.3 === gas tests === diff --git a/open_issues/binutils/sum_hurd b/open_issues/binutils/sum_hurd index 7d12b17f..31f8f997 100644 --- a/open_issues/binutils/sum_hurd +++ b/open_issues/binutils/sum_hurd @@ -1,4 +1,4 @@ -Test Run By thomas on Tue Oct 11 13:36:51 2011 +Test Run By thomas on Fri Oct 21 12:54:58 2011 Native configuration is i686-unknown-gnu0.3 === binutils tests === @@ -122,7 +122,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... # of expected passes 90 # of unsupported tests 3 -Test Run By thomas on Tue Oct 11 13:38:54 2011 +Test Run By thomas on Fri Oct 21 12:56:51 2011 Native configuration is i686-unknown-gnu0.3 === ld tests === @@ -625,6 +625,7 @@ PASS: Check --gc-section/-r/-u PASS: --gc-sections -r without -e PASS: --gc-sections with note section PASS: --gc-sections with __start_ +PASS: --gc-sections with __gxx_personality PASS: --gc-sections with shared library Running [...]/hurd/master/ld/testsuite/ld-h8300/h8300.exp ... Running [...]/hurd/master/ld/testsuite/ld-i386/i386.exp ... @@ -939,12 +940,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 661 +# of expected passes 662 # of expected failures 18 # of untested testcases 6 -[...]/hurd/master.build/ld/ld-new 2.22.51.20111011 +[...]/hurd/master.build/ld/ld-new 2.22.51.20111021 -Test Run By thomas on Tue Oct 11 13:37:18 2011 +Test Run By thomas on Fri Oct 21 12:55:21 2011 Native configuration is i686-unknown-gnu0.3 === gas tests === @@ -1416,5 +1417,5 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === # of expected passes 346 -../as-new 2.22.51.20111011 +../as-new 2.22.51.20111021 diff --git a/open_issues/binutils/sum_linux b/open_issues/binutils/sum_linux index f765b983..09c11722 100644 --- a/open_issues/binutils/sum_linux +++ b/open_issues/binutils/sum_linux @@ -1,4 +1,4 @@ -Test Run By thomas on Tue Oct 11 14:12:56 2011 +Test Run By thomas on Sat Oct 22 11:43:32 2011 Native configuration is i686-pc-linux-gnu === binutils tests === @@ -122,7 +122,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... # of expected passes 90 # of unsupported tests 3 -Test Run By thomas on Tue Oct 11 14:13:19 2011 +Test Run By thomas on Sat Oct 22 11:43:59 2011 Native configuration is i686-pc-linux-gnu === ld tests === @@ -625,6 +625,7 @@ PASS: Check --gc-section/-r/-u PASS: --gc-sections -r without -e PASS: --gc-sections with note section PASS: --gc-sections with __start_ +PASS: --gc-sections with __gxx_personality PASS: --gc-sections with shared library Running [...]/hurd/master/ld/testsuite/ld-h8300/h8300.exp ... Running [...]/hurd/master/ld/testsuite/ld-i386/i386.exp ... @@ -939,12 +940,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 671 +# of expected passes 672 # of expected failures 8 # of untested testcases 6 -[...]/hurd/master.build/ld/ld-new 2.22.51.20111011 +[...]/hurd/master.build/ld/ld-new 2.22.51.20111021 -Test Run By thomas on Tue Oct 11 14:13:02 2011 +Test Run By thomas on Sat Oct 22 11:43:38 2011 Native configuration is i686-pc-linux-gnu === gas tests === @@ -1416,5 +1417,5 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === # of expected passes 346 -../as-new 2.22.51.20111011 +../as-new 2.22.51.20111021 diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn index bb74d958..d776d81a 100644 --- a/open_issues/code_analysis.mdwn +++ b/open_issues/code_analysis.mdwn @@ -36,6 +36,11 @@ There is a [[!FF_project 276]][[!tag bounty]] on some of these tasks. specifiers, and have it emit useful warnings in case these are pointing to uninitialized data (for *in* only). + * [[Port Sequence Numbers|microkernel/mach/ipc/sequence_numbering]]. If + these are used, care must be taken to update them reliably, [[!message-id + "1123688017.3905.22.camel@buko.sinrega.org"]]. This could be checked by a + static analysis tool. + * [Static Source Code Analysis Tools for C](http://spinroot.com/static/) * [[!wikipedia List_of_tools_for_static_code_analysis]] diff --git a/open_issues/dbus.mdwn b/open_issues/dbus.mdwn index ec39e063..2f02579e 100644 --- a/open_issues/dbus.mdwn +++ b/open_issues/dbus.mdwn @@ -8,6 +8,14 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!tag open_issue_hurd]] + +The dbus problems are due to missing scm credentials [[sendmsg_scm_creds]] and socket credentials +[[pflocal_socket_credentials_for_local_sockets]]. There was also a problem with short timeout in +[[select]], but that has been solved in Debian by setting a minimum timeout of 1ms. + +--- + IRC, freenode, #hurd, 2011-11-26: <antrik> BTW, how much effort is necessary to fix dbus? diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index 9e2ec742..e2cff94f 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -8,25 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[General Information|/dde]]. - - -# IRC, freenode, #hurd, 2011-10-18 +[[!tag open_issue_hurd open_issue_gnumach]] -[[!tag open_issue_hurd]] - - [DDE crash, or similar] - <youpi> it's fake_local_irq_disable_flags(), then raw_local_irq_disable(), - then raw_local_irq_restore(), which *does not* release the cli_lock - <youpi> the "prove it" comment is (as I expected) completely wrong - <youpi> npnth: http://paste.debian.net/137784/ - -[[137784]] +[[General Information|/dde]]. - <youpi> could you try this patch ? - <youpi> (I've not even tried to build it) - <npnth> youpi: speaking of which, it still seems to hang :/ I'll 1) double - check it applied correctly and 2) get a gdb output if it did - <youpi> npnth: could you add printing the value of unlock_refcnt? - <youpi> so we can check what's happening - <npnth> unlock_refcnt is at 0, interesting +Still waiting for interface finalization and proper integration. diff --git a/open_issues/dde/137784 b/open_issues/dde/137784 deleted file mode 100644 index 1529465b..00000000 --- a/open_issues/dde/137784 +++ /dev/null @@ -1,43 +0,0 @@ -diff --git a/libdde_linux26/lib/src/arch/l4/cli_sti.c b/libdde_linux26/lib/src/arch/l4/cli_sti.c -index 051f259..6a8c460 100644 ---- a/libdde_linux26/lib/src/arch/l4/cli_sti.c -+++ b/libdde_linux26/lib/src/arch/l4/cli_sti.c -@@ -4,6 +4,8 @@ - - /* IRQ lock reference counter */ - static atomic_t _refcnt = ATOMIC_INIT(0); -+/* Refcnt value at which unlocking the cli_lock (it's not always 0) */ -+static int unlock_refcnt; - static ddekit_lock_t cli_lock; - - /* Check whether IRQs are currently disabled. -@@ -57,9 +59,6 @@ void fake_local_irq_restore(unsigned long flags) - /* Store the current flags state. - * - * This is done by returning the current refcnt. -- * -- * XXX: Up to now, flags was always 0 at this point and -- * I assume that this is always the case. Prove? - */ - unsigned long __raw_local_save_flags(void) - { -@@ -82,7 +81,7 @@ void raw_local_irq_restore(unsigned long flags) - { - Assert(cli_lock != NULL); - atomic_set(&_refcnt, flags); -- if (flags == 0) -+ if (flags == unlock_refcnt) - ddekit_lock_unlock(&cli_lock); - } - -@@ -95,7 +94,9 @@ void raw_local_irq_disable(void) - if (cli_lock == NULL) - ddekit_lock_init_unlocked(&cli_lock); - -- nested_lock(cli_lock); -+ if (nested_lock(cli_lock)) -+ /* Tell the corresponding restorer to release cli_lock */ -+ unlock_refcnt = atomic_read(&_refcnt); - atomic_inc(&_refcnt); - } - diff --git a/open_issues/fakeroot-tcp_vs_eintr.mdwn b/open_issues/fakeroot_exit_0.mdwn index 36707cd2..c6075b17 100644 --- a/open_issues/fakeroot-tcp_vs_eintr.mdwn +++ b/open_issues/fakeroot_exit_0.mdwn @@ -8,23 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!toc]] - - -# [[!debbug 641200]] - -[[!message-id "87litvz9me.fsf@kepler.schwinge.homeip.net"]] - + $ fakeroot ./scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include [...] - if test -z "$*"; then - FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$LIB" ${SHELL:-/bin/sh} - RESULT=$? - else - FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$LIB" "$@" - RESULT=$? - fi - + test -z './scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include' - + FAKEROOTKEY=2040 + LD_LIBRARY_PATH=/usr/lib/libfakeroot:/usr/lib64/libfakeroot:/usr/lib32/libfakeroot + LD_PRELOAD=libfakeroot-tcp.so + ./scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include @@ -36,8 +21,7 @@ License|/fdl]]."]]"""]] kill -s HUP 23612 + kill -s HUP 23612 - -## `exit(1)` +(The `EINTR` issue has been [[!debbug desc="fixed" 641200]].) `connect() < 0` invokes `fail()` which invokes `exit(1)`. Not yet figured out why the process exits 0 dispite that. `LD_PRELOAD` issue? ([[!taglink diff --git a/open_issues/gdb.mdwn b/open_issues/gdb.mdwn index 8821762e..0aec12e3 100644 --- a/open_issues/gdb.mdwn +++ b/open_issues/gdb.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -49,6 +49,8 @@ Last reviewed up to the [[Git mirror's 09ddc54333cdbc2f695fd83cbf091a7d5a1c3604 * glibc-tdep et al. also for GNU/Hurd? + * [[gdbserver]] + # Build diff --git a/open_issues/gdb/gdbserver.mdwn b/open_issues/gdb/gdbserver.mdwn new file mode 100644 index 00000000..fe239bbc --- /dev/null +++ b/open_issues/gdb/gdbserver.mdwn @@ -0,0 +1,20 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb]] + +Needs porting. Can probably use/share most code from GDB proper. + +# Miscellaneous Issues + + * ``m addr,length'` + + What is `errno`? Is is *the* `errno`? If yes, of the system GDB is + running on, or the system gdbserver is running on? Clarify documentation. diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index d3f88673..e8279139 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -27,8 +27,17 @@ Here's what's to be done for maintaining glibc. # Configuration -Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 -(2011-10-08) sources|source_repositories/glibc]]. +<!-- + +git checkout reviewed +git log --reverse --pretty=fuller --stat=$COLUMNS,$COLUMNS -p -C --cc ..sourceware/master +-i +/^commit |^---$|hurd|linux|gs:|__ASSUME + +--> + +Last reviewed up to the [[Git mirror's 9d65ea3a9b83ac3961229ba296a7caf90abce68d +(2011-11-17) sources|source_repositories/glibc]]. * t/dup3 @@ -53,13 +62,9 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 $ git log --reverse --pretty=fuller --stat=$COLUMNS,$COLUMNS -p -C --cc -S__ASSUME_ top-bases/t/kernel-features.h_includes..baseline - * t/tls - - * Discuss d2431f633e6139a62e1575ec18830f7e81160cf0 with Samuel. + * [[`t/tls`|t/tls]] - * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed. - - * [[t/tls-threadvar]] + * [[`t/tls-threadvar`|t/tls-threadvar]] * t/verify.h @@ -156,7 +161,8 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 `PTRACE_*` (for example, cbff0d9689c4d68578b6a4f0a17807232506ea27), `RLIMIT_RTTIME`, `SEEK_DATA` (`unistd.h`), `SEEK_HOLE` (`unistd.h`) `clock_adjtime`, `fallocate`, `fallocate64`, `getcontext` (and - `setcontext`), `name_to_handle_at`, `open_by_handle_at`, `sendmmsg`, + `setcontext`), `name_to_handle_at`, `open_by_handle_at`, + `process_vm_readv`, `process_vm_writev`, `sendmmsg`, `setns`, `sync_file_range` * `syncfs` @@ -194,6 +200,14 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 Need changes equivalent to c55fbd1ea768f9fdef34a01377702c0d72cbc213 + 14d96785125abee5e9a49a1c3037f35a581750bd. + * `madvise`, `MADV_DONTNEED` + + [[glibc_madvise_vs_static_linking]]. + + * `msync` + + Then define `_POSIX_MAPPED_FILES`, `_POSIX_SYNCHRONIZED_IO`. + * Create `t/cleanup_kernel-features.h`. * Add tests from Linux kernel commit messages for `t/dup3` et al. @@ -366,12 +380,23 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 * a7c8e6a1478de9f990b11e5e853318ccbe4330f2 (`Fix invalid conversion in __cmsg_nxthdr`). Probably just a C++ thing and not relevant for us; see [[message-id "87r52nk1kx.fsf@kepler.schwinge.homeip.net"]]. + * [high] `__ctype_init`, fd5bdc0924e0cfd1688b632068c1b26f3b0c88da. + Probably need to mirror `init-first.c` change. + * [high] `__attribute__ ((__leaf__))`, `BZ #13344`, + aa78043a4aafe5db1a1a76d544a833b63b4c5f5c + + 49a43d80ec5c97cf6136b1ee2687414773b2d5aa + + 3871f58f065dac3917eb18220a479e9591769c8c + + 9beb2334930db81ceada5aa6051fe5ac0554db32 + + 0ffc4f3ebaace42cd545db55a2ac50b6e0cc7d89 + + edc5984d4d18296d7aa3d8f4ed8f7336a743170e + + 57769839788e2c62b68d9dfbf4b35052321278ba. + <http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Function-Attributes.html>. # Build Here's a log of a glibc build run; this is from our [[Git repository's -0abded0927c62f02399658395837917548d5e281 (2011-10-14) +d740cf9d201dc9ecb0335b0a585828dea9cce793 (2011-10-25) sources|source_repositories/glibc]], run on coulomb.SCHWINGE. $ export LC_ALL=C @@ -383,7 +408,7 @@ sources|source_repositories/glibc]], run on coulomb.SCHWINGE. This takes up around 400 MiB and needs roughly 120 min on coulomb.SCHWINGE. <!-- - $ (make install_root=/INVALID && touch .go-check) 2>&1 | tee log_build_ && test -f .go-check && make -k install_root=/INVALID check TIMEOUTFACTOR=100 2>&1 | tee log_check + $ (make install_root=/INVALID && touch .go-install) 2>&1 | tee log_build_ && test -f .go-install && (make install_root="$PWD".install install && touch .go-check) 2>&1 | tee log_install && test -f .go-check && make -k install_root=/INVALID check TIMEOUTFACTOR=100 2>&1 | tee log_check $ find ./ -name \*.o -o -name \*.os -o -name \*.oS | while read f; do ~/tmp/gcc/git/contrib/compare-debug --preserve ../Roger_Whittaker.build-gcc-4.4-486.O/"$f" "$f"; done 2>&1 | less $ while read f; do (readelf -a "$f" && objdump -xDrtw "$f") > N && (cd ../Roger_Whittaker.build-gcc-4.4-486.O/ && readelf -a "$f" && objdump -xDrtw "$f") > O && diff -u O N | less; done $ find ./ -name \*.o -o -name \*.os -o -name \*.oS | while read f; do readelf -h "$f" | grep OS/ABI | (read a b && [ x"$b" != x'UNIX - System V' ] && echo "### $f: $b"); done @@ -401,7 +426,7 @@ TODO. TODO. <!-- - $ make install 2>&1 | tee log_install + $ make install_root="$PWD".install install 2>&1 | tee log_install [...] This takes up around 50 MiB, and needs roughly 1 min on kepler.SCHWINGE and 3 @@ -539,6 +564,8 @@ There is quite a baseline of failures. * `elf/tst-unique3lib.so`, `elf/tst-unique3lib2.so`, `elf/tst-unique4lib.so` + Only with GCC 4.4; no longer with 4.5 or 4.6: + /home/thomas/tmp/glibc/tschwinge/Roger_Whittaker.build-gcc-4.4-486/elf/tst-unique3lib.os:(.data.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0' diff --git a/open_issues/glibc/debian.mdwn b/open_issues/glibc/debian.mdwn index ce6ba9d2..331632f3 100644 --- a/open_issues/glibc/debian.mdwn +++ b/open_issues/glibc/debian.mdwn @@ -37,10 +37,28 @@ fail. (Same for `libc6-dev_extra_pkg_install` in `debian/sysdeps/i386.mk`, for example.) Why is this special handling only done for `xen`, but not for `i686`? +> Samuel: Historically because it's done that way in linux-i386. I don't know +> the real reason. + Do `export LC_ALL=C` before building, otherwise the testsuite/make error messages will be different from those stored in the -`debian/testsuite-checking/expected-results-*` files. +`debian/testsuite-checking/expected-results-*` files, resulting in a spurious +build failure. + +Run `debian/rules build-arch DEB_BUILD_OPTIONS=parallel=2 [EGLIBC_PASSES=...]` +to build (or `build` instead of `build-arch` to build the arch-independent +stuff, too). Can interrupt with `C-c` during locale stuff or testsuite if only +interested in the build tree. + +Run `fakeroot debian/rules binary DEB_BUILD_OPTIONS=parallel=2 +[EGLIBC_PASSES=...]` to build Debian packages or `binary-arch` for just the +architecture-dependent ones. + +The latter two steps can also be combined as `dpkg-buildpackage -R'debian/rules +EGLIBC_PASSES=libc' -nc -b -uc`. `-nc` will prevent the *clean step* which +would first try to un-patch, which may conflict if you have done any edits +apter applying patches. -Run `debian/rules build-arch DEB_BUILD_OPTIONS=parallel=2 [EGLIBC_PASSES=...]`. -Can stop with `C-c` during locale stuff or testsuite if only interested in the -build tree. +If the Debian symbol versioning file is not up to date and the build of Debian +packages fails due to this, putting `DPKG_GENSYMBOLS_CHECK_LEVEL=0` in the +environment \`\`helps''; see `man dpkg-gensymbols`. diff --git a/open_issues/glibc/t/tls.mdwn b/open_issues/glibc/t/tls.mdwn new file mode 100644 index 00000000..14ef36e4 --- /dev/null +++ b/open_issues/glibc/t/tls.mdwn @@ -0,0 +1,66 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_glibc open_issue_libpthread]] + +# To Do + + * Discuss d2431f633e6139a62e1575ec18830f7e81160cf0 with Samuel. + + * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed. + + +# Documentation + +[[!taglink open_issue_documentation]] + + * IRC, freenode, #hurd, 2011-11-26 + + <tschwinge> In glibc multiarch support (strcasecmp for i686 SSE3, etc.) + there is access to memory via gs: -- this will need to be changed for + us, right? + <youpi> depends on the access + <tschwinge> * `optimized strcasecmp and strncasecmp for x86-32` + (multiarch), + <tschwinge> 76e3966e9efc3808a9e7ad09121c5dfc1211c20b + + <tschwinge> 6abf346582ba678f4850a88b4a5950593841df1d + + <tschwinge> 5583a0862cf94f71cbcde91c4043a20af65facca. `gs` + access. + <youpi> + movl __libc_tsd_LOCALE@GOTNTPOFF(%ebx), %eax + <youpi> that's handled by the linker fine + <youpi> it's only the things held in the tcb_t structure which can pose + problem + <tschwinge> tcbhead_t? + <tschwinge> I'm looking at this. + <tschwinge> So, at gs:0, there is the TCB. + <tschwinge> And we have the same layout as NPTL/Linux, just that we + don't have as much data there as they have. + <tschwinge> We're missing multiple_threads, sysinfo, sttack_guard, + pointer_guard, gscope_flag, private_futex, __private_tm[5]. + <tschwinge> So, if one of these is referenced (be it my name or by + numeric offset), this is invalid for us. + <tschwinge> Anything else should work equivalently. + <youpi> yes + <youpi> usually the only numeric offset being used is 0 + <youpi> so it would simply not build + <tschwinge> And the other offsers are generated via tcb-offsets.sym. + <tschwinge> glibc's elf/stackguard-macros.h is wrong for us (but not + used anywhere apart from elf/tst-stackguard1.c, I think). + <tschwinge> __thread __locale_t __libc_tsd_LOCALE = &_nl_global_locale; + -- this means that a __libc_tsd_LOCALE values will be in the TLS + segment, and this is what is being accessed from the assembler code + with %gs:__libc_tsd_LOCALE@NTPOFF, and the linker will resolve this. + <youpi> yes + <youpi> see in the nm output, the libc_tsd symbols + <youpi> these provide the offsets + <tschwinge> youpi: Thank you, I'm now understanding this part of TLS + much better. + <youpi> have you had a look at the tls.pdf from Uli ? + <youpi> all the gory details are there :) diff --git a/open_issues/libpthread_pthread_key_create_reuse.mdwn b/open_issues/libpthread_pthread_key_create_reuse.mdwn deleted file mode 100644 index ca2da2f5..00000000 --- a/open_issues/libpthread_pthread_key_create_reuse.mdwn +++ /dev/null @@ -1,82 +0,0 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!meta title="libpthread: pthread_key_create, reuse"]] - -[[!tag open_issue_libpthread]] - -IRC, FreeNode, #hurd, 2011-07-02: - - < pinotree> hm, maybe i found a libpthread bug - * pinotree tries a testcase - < pinotree> yesssss, found the bug :) - < pinotree> youpi: it's a problem of the key reuse in pthread_key_create() - < youpi> it doesn't reset it? - < youpi> were you looking at the licq issue? - < pinotree> no, gtest - < youpi> k - < youpi> licq has a failing threadspecific issue - < youpi> [ FAILED ] ThreadSpecificData.dataDeletedWhenThreadExits - < pinotree> basically, pthread_key_delete() does not delete the key values - from the "thread_specifics" ihash - < pinotree> but those were new keys, so i'm not sure it is allowed to - return values of previous keys? - < pinotree> after all, the actual key value is an implementation detail, - applications shouldn't care about it being reused - < pinotree> (imho) - < youpi> Upon key creation, the value NULL shall be associated with the new - key in all active threads. - < youpi> ok, so we have to clear it in all threads - < youpi> that's a pity - < pinotree> or just remove the entry from the hash on key removal - < youpi> pinotree: from all the hashes, you mean? - < pinotree> youpi: from how i see it, adding a snippet like - http://paste.debian.net/121690/ in pthread_key_delete() should do the job - < youpi> that only drops from the current thread - < pinotree> ah hm, other threads - < youpi> we need to drop from all threads - < youpi> that's the pity part - < pinotree> youpi: the licq case could look like a similar issue, at a - veeery quick glance - -Test program: [[pthread_key_create_reuse.c]] - - -2011-11-01: - - <pinotree> youpi: about the bug with pthread keys (reuse): would be an - acceptable solution having a mutex for the thread_specifics of each - thread? - <youpi> you mean one per thread, one global, or one per key, or ? - <youpi> what is it supposed to protect? - <pinotree> the thread_specifics of each thread - <youpi> pinotree: but against what? - <pinotree> the idea would be: when destroying a key, iterate over all the - exiting threads and remove the key data from the thread_specifics of each - thread - <youpi> one of the issue is getting to browse through the whole list of - threads - <youpi> the other is concurrency between that, and a thread dying - <pinotree> there's the __pthread_threads_lock rwlock - <youpi> it should be enough to keep it locked during the iteration - <pinotree> but that wouldn't be enough when one thread is destroying a key, - and another one is doing {get,set}specific() on that key - <youpi> that's not supposed to happen - <pinotree> mmm - <youpi> “The effect of calling pthread_getspecific() or - pthread_setspecific() with a key value not obtained from - pthread_key_create() or after key has been deleted with - pthread_key_delete() is undefined.” - <youpi> undefined -> you are allowed to just blow up - <pinotree> but it's not been deleted yet... :) - <youpi> it could be, just a matter of time - <youpi> you're not supposed to rely on time-luckyness :) - <pinotree> mmm - <pinotree> bah, you've convinced me ( :) ) diff --git a/open_issues/libpthread_pthread_key_create_reuse/pthread_key_create_reuse.c b/open_issues/libpthread_pthread_key_create_reuse/pthread_key_create_reuse.c deleted file mode 100644 index f7f5874e..00000000 --- a/open_issues/libpthread_pthread_key_create_reuse/pthread_key_create_reuse.c +++ /dev/null @@ -1,48 +0,0 @@ -#include <pthread.h> -#include <stdio.h> -#include <assert.h> - -#define DEBUG - -void del(void *x __attribute__((unused))) -{ -} - -void work(int val) -{ - pthread_key_t key1; - pthread_key_t key2; - -#ifdef DEBUG - printf("work/%d: start\n", val); -#endif - assert(pthread_key_create(&key1, &del) == 0); - assert(pthread_key_create(&key2, &del) == 0); -#ifdef DEBUG - printf("work/%d: pre-setspecific: %p,%p\n", val, pthread_getspecific(key1), pthread_getspecific(key2)); -#else - assert(pthread_getspecific(key1) == NULL); - assert(pthread_getspecific(key2) == NULL); -#endif - assert(pthread_setspecific(key1, (void *)(0x100 + val)) == 0); - assert(pthread_setspecific(key2, (void *)(0x200 + val)) == 0); -#ifdef DEBUG - printf("work/%d: post-setspecific: %p,%p\n", val, pthread_getspecific(key1), pthread_getspecific(key2)); -#else - assert(pthread_getspecific(key1) == (void *)(0x100 + val)); - assert(pthread_getspecific(key2) == (void *)(0x200 + val)); -#endif - assert(pthread_key_delete(key1) == 0); - assert(pthread_key_delete(key2) == 0); -} - -int main() -{ - int i; - - for (i = 0; i < 8; ++i) { - work(i + 1); - } - - return 0; -} diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn index 1fc2c318..0f6b9f19 100644 --- a/open_issues/multithreading.mdwn +++ b/open_issues/multithreading.mdwn @@ -24,7 +24,7 @@ Hurd servers / VFS libraries are multithreaded. # Design -Roughly using one thread per +See [[hurd/libports]]: roughly using one thread per incoming request. This is not the best approach: it doesn't really make sense to scale the number of worker threads with the number of incoming requests, but instead they should be scaled according to the backends' characteristics. diff --git a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn index 37fe9c53..931fd0ee 100644 --- a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn +++ b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn @@ -27,10 +27,6 @@ extracted from cdf7c161ebd4a934c9e705d33f5247fd52975612 sources, 2010-10-24. On the idle grubber, this one repeatedly takes a few minutes wall time to complete successfully, contrary to a few seconds on a GNU/Linux system. -> On order of slowness may in fact be due to a Xen-specific issue, see -> [[xen_lseek]]. (But there are probably still one or two orders left, even -> without Xen.) - While processing the object files, there is heavy interaction with the relevant [[hurd/translator/ext2fs]] process. Running [[hurd/debugging/rpctrace]] on the testee shows that (primarily) an ever-repeating series of `io_seek` and @@ -38,17 +34,6 @@ the testee shows that (primarily) an ever-repeating series of `io_seek` and shows the equivalent thing (`_llseek`, `read`) -- but Linux' I/O system isn't as slow as the Hurd's. ---- - -IRC, freenode, #hurd, 2011-09-01: - - <youpi> hum, f951 does myriads of 71->io_seek_request (32768 0) = 0 32768 - <youpi> no wonder it's slow - <youpi> unfortunately that's also what it does on linux, the system call is - just less costly - <youpi> apparently gfortran calls io_seek for, like, every token of the - sourced file - <youpi> (fgetpos actually, but that's the same) - <youpi> and it is indeed about 10 times slower under Xen for some reason - -[[!tag open_issue_xen]] +As Samuel figured out later, this slowness may in fact be due to a Xen-specific +issue, see [[Xen_lseek]]. After the latter has been addressed, we can +re-evaluate this issue here. diff --git a/open_issues/posix_fadv_volatile.mdwn b/open_issues/posix_fadv_volatile.mdwn new file mode 100644 index 00000000..6bd25e3e --- /dev/null +++ b/open_issues/posix_fadv_volatile.mdwn @@ -0,0 +1,16 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="POSIX_FADV_VOLATILE"]] + +[[!tag open_issue_gnumach open_issue_hurd]] + +May be interesting to watch what happens: LWN, Jonathan Corbet, +[`POSIX_FADV_VOLATILE`](http://lwn.net/Articles/468896/), 2011-11-22. diff --git a/open_issues/ti-rpc_then_nfs.mdwn b/open_issues/ti-rpc_then_nfs.mdwn new file mode 100644 index 00000000..aa36e020 --- /dev/null +++ b/open_issues/ti-rpc_then_nfs.mdwn @@ -0,0 +1,20 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd open_issue_porting]] + +TI-RPC replaces glibc's Sun RPC implementation, [[!message-id +"4D0632C5.1040107@RedHat.com"]]. + +It needs some work on our side, [[!message-id +"20101214213212.GU1095@kepler.schwinge.homeip.net"]]. + +Then, the Hurd's [[hurd/translator/nfs]] translator and [[hurd/nfsd]] can be +re-enabled, [[!message-id "87hb2j7ha7.fsf@gnu.org"]]. diff --git a/open_issues/translators_set_up_by_untrusted_users.mdwn b/open_issues/translators_set_up_by_untrusted_users.mdwn index 97f48bba..044d5411 100644 --- a/open_issues/translators_set_up_by_untrusted_users.mdwn +++ b/open_issues/translators_set_up_by_untrusted_users.mdwn @@ -282,6 +282,9 @@ and [Hardlink Protection](https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#Hardlink_Protection) do bear some similarity with the issue we're discussing here. +Likewise, Kees Cook, [fs: symlink restrictions on sticky +directories](http://lwn.net/Articles/468215/), 2011-11-18. + # IRC, freenode, #hurd, 2011-08-31 diff --git a/open_issues/xen_lseek.mdwn b/open_issues/xen_lseek.mdwn index accc7c8f..756abf5e 100644 --- a/open_issues/xen_lseek.mdwn +++ b/open_issues/xen_lseek.mdwn @@ -10,6 +10,17 @@ License|/fdl]]."]]"""]] [[!tag open_issue_xen]] +IRC, freenode, #hurd, 2011-09-01: + + <youpi> hum, f951 does myriads of 71->io_seek_request (32768 0) = 0 32768 + <youpi> no wonder it's slow + <youpi> unfortunately that's also what it does on linux, the system call is + just less costly + <youpi> apparently gfortran calls io_seek for, like, every token of the + sourced file + <youpi> (fgetpos actually, but that's the same) + <youpi> and it is indeed about 10 times slower under Xen for some reason + IRC, freenode, #hurd, 2011-11-02: <youpi> btw, we have a performance issue with xen @@ -33,3 +44,14 @@ IRC, freenode, #hurd, 2011-11-02: http://www.gnu.org/software/hurd/open_issues/performance/io_system/binutils_ld_64ksec.html [[performance/io_system/binutils_ld_64ksec]]. + +Also see the simple testcases [[test-lseek.c]] and [[test-mach.c]]. + +IRC, freenode, #hurd, 2011-11-05: + + <youpi> [test-mach.c is] mostly as a reference for the trap overhead + <youpi> 0.56µs (xen) vs 0.48µs(kvm) on test-mach + <youpi> 455µs(xen) vs 16µs(kvm) on test-lseek + <youpi> that might simply be an issue in the RPC mechanism, which behaves + badly with the xen memory management + <youpi> yes, about 0.5ms for an lseek, that's quite high :) diff --git a/open_issues/xen_lseek/test-lseek.c b/open_issues/xen_lseek/test-lseek.c new file mode 100644 index 00000000..667dce66 --- /dev/null +++ b/open_issues/xen_lseek/test-lseek.c @@ -0,0 +1,17 @@ +#include <stdio.h> +#include <math.h> +#include <fcntl.h> +#include <unistd.h> +#include <sys/time.h> +#define N 100000 +int main(void) { + int fd = open("test.c", O_RDONLY); + struct timeval tv1, tv2; + int i; + gettimeofday(&tv1, NULL); + for (i = 0; i < N; i++) + lseek(fd, 0, SEEK_CUR); + gettimeofday(&tv2, NULL); + printf("%fµs\n", (float)((tv2.tv_sec-tv1.tv_sec) * 1000000 + tv2.tv_usec - tv1.tv_usec)/N); + return 0; +} diff --git a/open_issues/xen_lseek/test-mach.c b/open_issues/xen_lseek/test-mach.c new file mode 100644 index 00000000..90337346 --- /dev/null +++ b/open_issues/xen_lseek/test-mach.c @@ -0,0 +1,19 @@ +#define _GNU_SOURCE +#include <stdio.h> +#include <fcntl.h> +#include <mach/mach.h> +#define N 1000000 +int main(void) { + struct timeval tv1, tv2; + int i; + task_t task; + task = mach_task_self(); + mach_port_urefs_t refs; + gettimeofday(&tv1, NULL); + for (i = 0; i < N; i++) { + mach_port_get_refs(task, task, MACH_PORT_RIGHT_RECEIVE, &refs); + } + gettimeofday(&tv2, NULL); + printf("%fµs\n", (float)((tv2.tv_sec-tv1.tv_sec) * 1000000 + tv2.tv_usec - tv1.tv_usec)/N); + return 0; +} diff --git a/public_hurd_boxen.mdwn b/public_hurd_boxen.mdwn index 5a281368..ac53630e 100644 --- a/public_hurd_boxen.mdwn +++ b/public_hurd_boxen.mdwn @@ -28,27 +28,32 @@ image|hurd/running/qemu]]. "[[bddebian]]","goober","Debian GNU/Hurd","?" "[[bddebian]]","grubber","Debian GNU/Hurd","Celeron 2.2 GHz; 554 MiB","Xen domU on [[zenhost]]; for experimental stuff" "[[bddebian]]","[[zenhost]]","Debian GNU/Linux","Celeron 2.2 GHz","Xen dom0 for several hosts" -"Debian","[strauss.debian.net](http://strauss.debian.net/ssh)","Debian GNU/Hurd",,"all Debian Developers have access" +"[[sceen]]","darnassus","Debian GNU/Hurd","Core i5 3.1 GHz, 1.8 GiB","KVM guest on shattrath; public Hurd box" +"[[sceen]]","ironforge","Debian GNU/Hurd","Core i5 3.1 GHz, 1.8 GiB","KVM guest on shattrath; Debian buildd" +"[[sceen]]","exodar","Debian GNU/Hurd","Core i5 3.1 GHz, 1.8 GiB","KVM guest on shattrath; Debian porterbox, all Debian Developers have access" +"[[sceen]]","shattrath","Debian GNU/Linux","Core i5 3.1 GHz","KVM host" +"Debian","strauss","Debian GNU/Hurd","Sempron 2800+","all Debian Developers have access" """]] -To request an account on the *[[bddebian]]* machines either contact +If you are a Debian Developer, you can log into the exodar or strauss machines +with your DD account, your public ssh keys are already there. To request a +non-DD account on them, please contact admin@{exodar,strauss}.debian.net, with +your Alioth account name and a public ssh key. + +To request an account on the *[[bddebian]]* or *[[sceen]]* machines, either contact *tschwinge* (other people might also be able to help) in [[IRC]] or send email to <hurd-shell-account@gnu.org> (please include your desired user name and public SSH key). Also use these contact addresses for requesting support with respect to software installations, etc. - For easy access, you should append your public SSH key(s) to `~/.ssh/authorized_keys` on the remote machine. -Also, add the [[!toggle id=bddebian_ssh_config text="following stanza (click -here to toggle display)"]] to the `~/.ssh/config` file of the machine you're -connecting from. - -[[!toggleable id=bddebian_ssh_config text=""" +Also, add the following stanza to the `~/.ssh/config` file of the machine +you're connecting from. # Stanza from <http://www.gnu.org/software/hurd/public_hurd_boxen.html>, - # 2010-12-15. + # 2011-11-06. Host blubber.bddebian.com blubber HostName blubber.bddebian.com @@ -80,13 +85,16 @@ connecting from. Port 2260 Host blubber.bddebian.com blubber grubber.bddebian.com grubber snubber.bddebian.com snubber - ProxyCommand ssh zenhost socat - TCP4:%h:%p + # Tunnel through zenhost. + ProxyCommand ssh zenhost exec socat - TCP4:%h:%p Host *.bddebian.com blubber clubber flubber gnubber goober grubber snubber zenhost + # Don't worry about the IP address (dial-up connection). CheckHostIP no User [user name] - -The `CheckHostIP` statement is for not having to worry about the machines's IP -addresses changing (due to being behind a dial-up connection). - -"""]] + + Host darnassus.sceen.net darnassus + HostName darnassus.sceen.net + + Host *.sceen.net darnassus + User [user name] diff --git a/public_hurd_boxen/sceen.mdwn b/public_hurd_boxen/sceen.mdwn new file mode 100644 index 00000000..25416857 --- /dev/null +++ b/public_hurd_boxen/sceen.mdwn @@ -0,0 +1,11 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +<http://www.sceen.net/> diff --git a/shortcuts.mdwn b/shortcuts.mdwn index b62b2981..fd4d4dee 100644 --- a/shortcuts.mdwn +++ b/shortcuts.mdwn @@ -87,6 +87,13 @@ ikiwiki will include your shortcut in the standard underlay. * [[!shortcut name=FF_project url="http://www.fossfactory.org/project/p%s" desc="FOSS Factory bounty (p%s)"]] +## Savannah + + * [[!shortcut name=GNU_Savannah_Git_hurd_hurd + url="http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=%s" + desc="hurd/hurd.git commit %s"]] + + ## Notmuch'n'Gmane. * [[!shortcut name=message-id url="http://thread.gmane.org/%s" desc="""`id:"%s"`"""]] diff --git a/unsorted/FunnyHurd.mdwn b/unsorted/FunnyHurd.mdwn index 1653ec77..1a70fb8a 100644 --- a/unsorted/FunnyHurd.mdwn +++ b/unsorted/FunnyHurd.mdwn @@ -1,39 +1,23 @@ -## <a name="Fun_stuff_ripped_from_the_Intern"> Fun stuff ripped from the Internet </a> - -<table border="1" cellpadding="1" cellspacing="0"> - <tr> - <td> %ATTACHURL%/hurd-windows.gif <br /> Hurd Windows, availble from <a href="http://www.hurd.com" target="_top">http://www.hurd.com</a></td> - <td> %ATTACHURL%/HurdExchange.gif <br /> Exchange your Hurd at <a href="http://www.thunderinghurd.com" target="_top">http://www.thunderinghurd.com</a></td> - </tr> - <tr> - <td> %ATTACHURL%/HurdCarDeal.jpg <br /> ... and we can of course also offer you a great deal on this -91 Chevy! :-) </td> - <td> %ATTACHURL%/HurdLodge.jpg <br /> The many perks of being a Hurd user also includes our own ski lodge! <br /><font size="+2">Hurd House</font><br /> - <ul> - <li>Knotty pine kitchen</li> - <li>Spacious kitchen &amp; living room with loft</li> - <li>Leather couch and love seat with a TV &amp; VCR</li> - <li>Outdoor Jacuzzi</li> - <li>Spacious master bedroom/bath upstairs</li> - <li>Twin beds in one room / queen bed in another</li> - </ul> - </td> - </tr> - <tr> - <td> %ATTACHURL%/HurdMagician.jpg <br /> From <a href="http://www.magicposters.com/buy/h-k.html" target="_top">http://www.magicposters.com/buy/h-k.html</a></td> - <td> %ATTACHURL%/CrystalAwards.jpg <br /> "Wow dude, I saw the Debian Swirl logo on last nights <a href="http://www.wif.org/events/crystals.html" target="_top">Crystal Awards</a>!" </td> - </tr> -</table> - ----- - -These images and links are only here to serve as a comic relief to this site. It is **not** the intention to humiliate the people, corporations or organizations behind these factual sites. - -If your [company] name or organization is listed here and you do not approve you can remove yourself simply by clicking on the "Edit" button. In the login window that appears you enter _TWikiGuest_ as username and _guest_ as password. - ----- - -### <a name="Comments"> Comments </a> - -Created the page. - --- [[Main/JoachimNilsson]] - 09 Nov 2002 +Fun stuff ripped from the Internet. + + * [*Martine termine + GNU/Hurd*](https://plus.google.com/105752945023653836002/posts/VLDp1HfXLZ6). + <img + src="https://lh6.googleusercontent.com/-rzRt1aMSWrE/TrBJvkDboRI/AAAAAAAABx8/3KhzIvPTZ1w/h301/martine-termine-gnu-hurd.jpg"> + + * Hurd Windows, availble from <http://www.hurd.com>. [[hurd-windows.gif]] + + * Exchange your Hurd at <http://www.thunderinghurd.com>. + [[HurdExchange.gif]] + + * ... and we can of course also offer you a great deal on this -91 Chevy! + :-) [[HurdCarDeal.jpg]] + + * From <http://www.magicposters.com/buy/h-k.html>. [[HurdMagician.jpg]] + + +--- + +These images and links are only here to serve as a comic relief to this site. +It is **not** the intention to humiliate the people, corporations or +organizations behind these factual sites. diff --git a/unsorted/FunnyHurd/CrystalAwards.jpg b/unsorted/FunnyHurd/CrystalAwards.jpg Binary files differdeleted file mode 100644 index 2daac850..00000000 --- a/unsorted/FunnyHurd/CrystalAwards.jpg +++ /dev/null diff --git a/unsorted/FunnyHurd/HurdLodge.jpg b/unsorted/FunnyHurd/HurdLodge.jpg Binary files differdeleted file mode 100644 index d13562f5..00000000 --- a/unsorted/FunnyHurd/HurdLodge.jpg +++ /dev/null diff --git a/user/Maksym_Planeta.mdwn b/user/Maksym_Planeta.mdwn new file mode 100644 index 00000000..4f8ababd --- /dev/null +++ b/user/Maksym_Planeta.mdwn @@ -0,0 +1,312 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] +#Notes on tmpfs + +## Current state + +26.01.12 + +Infinite fsx on ext2fs: + +Test 1. After about hour of work ext2fs breaks with such message: ext2fs.static: /home/buildd/build/chroot-sid/home/buildd/byhand/hurd/./ext2fs/pager.c:399: file_pager_write_page: Assertion 'block' failed. Fsx ended with Resource lost. There is no 'vmstat 1' log. + +Test 2. Left for night. Finally got this message: ext2fs.static: /home/buildd/build/chroot-sid/home/buildd/byhand/hurd/./ext2fs/pager.c:399: file_pager_write_page: Assertion 'block' failed. vm_fault: memory_object_data_unlock failed. Fsx ended with Bus error. + +Test 3. Finally got this message: ext2fs.static: /home/buildd/build/chroot-sid/home/buildd/byhand/hurd/./ext2fs/pager.c:399: file_pager_write_page: Assertion 'block' failed. memory_object_data_request(0x0, 0x0, 0x38000, 0x1000, 0x3) failed, 10000003. Fsx ended with Bus error. vmstat didn't show any leek. + +The same test for tmpfs: + +Test 1. The test continued about 5 hours and made 18314766 operations. Then this error message appeared: + +READ BAD DATA: offset = 0x3317, size = 0x7012 +OFFSET GOOD BAD RANGE +0x 8a93 0x80f8 0x82f8 0x 1 +operation# (mod 256) for the bad data may be 248 +LOG DUMP (18314753 total operations): +18314754(2 mod 256): WRITE 0x38c1a thru 0x3ffff (0x73e6 bytes) +18314755(3 mod 256): READ 0x92c3 thru 0x181b3 (0xeef1 bytes) +18314756(4 mod 256): WRITE 0x138c1 thru 0x1a06b (0x67ab bytes) +18314757(5 mod 256): MAPWRITE 0x2564 thru 0xbc75 (0x9712 bytes) ******WWWW +18314758(6 mod 256): MAPREAD 0x3f4e8 thru 0x3ffff (0xb18 bytes) +18314759(7 mod 256): READ 0x2946a thru 0x30213 (0x6daa bytes) +18314760(8 mod 256): MAPWRITE 0x31fe6 thru 0x3ffff (0xe01a bytes) +18314761(9 mod 256): TRUNCATE DOWN from 0x40000 to 0x145e8 +18314762(10 mod 256): MAPWRITE 0xca89 thru 0x1ba74 (0xefec bytes) +18314763(11 mod 256): MAPWRITE 0xb421 thru 0x11a37 (0x6617 bytes) +18314764(12 mod 256): READ 0x9495 thru 0xaa45 (0x15b1 bytes) +18314765(13 mod 256): TRUNCATE DOWN from 0x1ba75 to 0x66cb ******WWWW +18314766(14 mod 256): READ 0x5fa5 thru 0x66ca (0x726 bytes) + +vmstat has broken after about 2 hours, so no leak has been detected. + +5.01.12 + +UPD 26.01: All these bugs are fixed. + +There left 2 bugs, I found: + +1. Passive translator doesn't work. UPD: Seems I confused something, but now it works: +$ showtrans foo +/usr/bin/env LD_LIBRARY_PATH=/home/mcsim/git/hurd-build/lib tmpfs/tmpfs 10M + +3. Writing by freed address somewhere. As workaround I set kalloc_max in mach-defpager/kalloc.c to 3, so vm_allocate is always used. UPD: Now I use another workaround (see below). + +4. Fsx still breaks. With 11 from 12 seeds, I tested the problem was in first 4 Kb. In 12th case problem was in range between 4 Kb and 8 Kb. I find this quite odd. Usual amount of operations before fsx breaks passes 100 000. UPD: The problem seems to be fixed. + +## mach-defpager + +[[defpager|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#defpager81111]] + +[[http://www.mail-archive.com/bug-hurd@gnu.org/msg18859.html]] + +[[My patch that fixes tmpfs and defpager|http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00086.html]] + +My vision of the problem in short: when user tries to access to the memory backed by tmpfs first time, kernel asks defpager to initialize memory object and sets it's request port to some value. But sometimes user directly accesses to defpager and supplies it own request port. So defpager should handle different request port, what causes errors. + +TODO: pager_extend always frees old pagemap and allocates new. Consider if it is necessary. + +## Steps + +### Find out what causes crashes in tmpfs with defpager + +[[http://www.gnu.org/s/hurd/hurd/translator/tmpfs/notes_various.html]] + +TODO: Consider deleting of parameter "port" in function mach-defpager/default_pager.c:pager_port_list_insert +since this parameter is unused + +Probably pager_request shouldn't be stored because request may arrive from different kernels (or from kernel and translator), so this parameter doesn't have any sense. + +22.11.11 Reading/writing for any size works, [[this|http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00127.html]] works, but fsx test fails ([[see|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#fsx_fail2211]]). + +24.11.11 The problem with fsx. + +Here are follow operations: + +1. Write some data at address 0x100 with size 0x20 +2. Truncate file to size 0x80. First written data should be lost. +3. Write some data at address 0x200 size of 0x20. By this operation file size is increased up to 0x220. +4. Read data at address 0x110. Fsx expects here zeros, but in fact here is data, that was written at step 1. + +When fsx tries to read data kernel calls pager with seqno_memory_object_data_request, and pager returns on step 4 zeros either with memory_object_data_provided or memory_object_data_unavailable. Before this, in default_pager_set_size memory_object_lock_request called to flush any kernel caches, that could hold data to be truncated. When I set offset to 0 and size to limit in memory_object_lock_request it appeared another error ([[see|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#fsx_fail2411]]). Both these behaviors appear to be quite strange for me. It is quite late now, so i put these notes to not forget this and went sleep. Continue tomorrow. + +5.12.11 Here is a problem with writing by address, which was freed already. It happens in function dealloc_direct in macros invalidate_block. This function is called from pager_truncate in branch when condition "if (!INDIRECT_PAGEMAP(old_size))" is true. But I didn't find why reference to freed object is kept. As workaround we can reduce kalloc_max in hurd/mach-defpager/kalloc.c to 3 to make allocator use vm_allocate always. The drawback is that allocator will allocate only multiple of vm_page_size, but this is temporary tradeoff. Till now fsx reaches operation number 14277. + +6.12.11 fsx works quite long and doesn't interrupt. I've stopped at 124784. Continued. It broke at 181091. + +4.01.12 I've localized the problem. It is in these lines (starting approx. at hurd/mach-defpager/default_pager.c:1177): + + /* Now reduce the size of the direct map itself. We don't bother + with kalloc/kfree if it's not shrinking enough that kalloc.c + would actually use less. */ + if (PAGEMAP_SIZE (new_size) <= PAGEMAP_SIZE (old_size) / 2) + { + const dp_map_t old_mapptr = pager->map; + pager->map = (dp_map_t) kalloc (PAGEMAP_SIZE (new_size)); + memcpy (pager->map, old_mapptr, PAGEMAP_SIZE (old_size)); + my_tag |= 20; + kfree ((char *) old_mapptr, PAGEMAP_SIZE (old_size)); + } + +I didn't find out yet what is wrong here exactly, but when I exclude this code memory spoiling disappears. Still breaks at 181091. + +### Write own pager + + 6.11.11 Reading/writing for files that fit in vm_page_size works + + 7.11.11 Works for any size. + + TODO: During execution tmpfs hangs in random places. The most possible is variant is deadlocks, + because nothing was undertaken for thread safety. + + TODO: Make tmpfs use not more space than it was allowed. + +### Make links work + +Symlinks behavior: [[links|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#links81111]] + +8.11.11 Symlinks work. + +[[Patch by Ben Asselstine.|http://thread.gmane.org/gmane.os.hurd.bugs/11829/focus=12098]] + +### After sometime of inactivity tmpfs exits. + + TODO: Find out why and correct this. + +> This may perhaps be the standard translator +> shutdown-after-a-period-of-inactivity functionality? This is of course not +> valid in the tmpfs case, as all its state (file system content) is not stored +> on any non-volatile backend. Can this auto-shutdown functionality be +> disabled (in [[hurd/libdiskfs]], perhaps? See `libdiskfs/init-first.c`, the +> call to `ports_manage_port_operations_multithread`, the `global_timeout` +> paramenter (see the [[hurd/reference_manual]]) (here: `server_timeout`). +> Probably this should be set to `0` in tmpfs to disable timing out? +> --[[tschwinge]] + +### Passive translator doesn't work + +> Must be some bug: `diskfs_set_translator` is implemented in `node.c`, so this +> is supposed to work. --[[tschwinge]] + +#Challenges + +Translators vs FUSE: + +[[What can a translator do that FUSE can't?|http://lists.gnu.org/archive/html/bug-hurd/2010-07/msg00061.html]] + +[[Re: Hurd translators on FUSE|http://lists.gnu.org/archive/html/l4-hurd/2009-09/msg00146.html]] + +[[Example of sane utilization of filesystem stored in RAM|http://habrahabr.ru/blogs/gdev/131043/]] (Russian). Author of this article copied some resources of game "World of Tanks" to RAM-drive and game started load much faster. Although he used Windows in this article, this could be good example of benefits, which filesystem, stored in RAM, could give. + +#Debugging + +To debug tmpfs, using libraries from "$PWD"/lib and trace rpc: + + settrans -ca foo /usr/bin/env LD_LIBRARY_PATH="$PWD"/lib utils/rpctrace -I /usr/share/msgids/ tmpfs/tmpfs 1M + LD_LIBRARY_PATH="$PWD"/lib gdb tmpfs/tmpfs `pidof tmpfs` + +For debugging ext2fs: + + settrans --create --active ramdisk0 /hurd/storeio -T copy zero:32M && \ + /sbin/mkfs.ext2 -F -b 4096 ramdisk0 && \ + settrans --active --orphan ramdisk0 /usr/bin/env LD_LIBRARY_PATH="$PWD"/lib utils/rpctrace -I /usr/share/msgids/ \ + ext2fs/ext2fs.static ramdisk0 + +#Questions + +1. What are sequence numbers? What are they used for? + +> See [[microkernel/mach/ipc/sequence_numbering]]. --[[tschwinge]] + +2. Is there any way to debug mach-defpager? When I set breakpoint to any function in it, pager never breaks. + +> Is that still an unresolved problem? If not, what was the problem? --[[tschwinge]] + +3. Is it normal that defpager panics and breaks on any wrong data? + +> No server must ever panic (or reach any other invalid state), whichever input +> data it receives from untrusted sources (which may include every possible +> source). All input data must be checked for validity before use; anything +> else is a bug. --[[tschwinge]] + +#Links + +1. [[Cthreads manuals|http://www.cc.gatech.edu/classes/cs6432_99_winter/threads_man/]] + +#Conversations + +## 8.11.11 + +### links<a id="links81111"/> + (10:29:11) braunr: mcsim: ln -s foo/bar foo/baz means the link name is baz in the foo directory, + and its target (relative to its directory) is foo/bar (which would mean /tmp/foo/foo/bar in canonical form) + (10:29:42) braunr: youpi: tschwinge: what did ludovic achieve ? + (10:30:06) tschwinge: mcsim: As Richard says, symlink targets are always relative to the directory they're contained in. + (10:31:26) braunr: oh ok + (10:31:27) mcsim: so, if I want to create link in cd, first I need to cd there? + (10:31:36) mcsim: in foo* + (10:31:36) braunr: mcsim: just provide the right paths + (10:32:11) braunr: $ touch foo/bar + (10:32:14) braunr: $ ln -s bar foo/baz + (10:32:32) braunr: bar + (10:32:35) braunr: baz -> bar + +### defpager<a id="defpager81111"/> + + earlier: + <tschwinge>: 1. On every system there is a ``default pager'' (mach-defpager). That one is responsible + for all ``anonymous memory''. For example, when you do malloc(10 MiB), and then there is memory pressure, + this 10 MiB memory region is backed by the default pager, whose job then is it to provide the backing store for this. + <tschwinge>: This is what commonly would be known as a swap partition. + <tschwinge>: And this is also the way tmpfs works (as I understand it). + <tschwinge>: malloc(10 MiB) can also be mmap(MAP_ANONYMOUS, 10 MIB); that's the same, essentially. + <tschwinge>: Now, for ext2fs or any other disk-based file system, this is different: + <tschwinge>: The ext2fs translator implements its own backing store, namely it accesses the disk for storing + changed file content, or to read in data from disk if a new file is opened. + + ... + + (10:36:14) mcsim: who else uses defpager besides tmpfs and kernel? + (10:36:27) braunr: normally, nothing directly + (10:37:04) mcsim: than why tmpfs should use defpager? + (10:37:22) braunr: it's its backend + (10:37:28) braunr: backign store rather + (10:37:38) braunr: the backing store of most file systems are partitions + (10:37:44) braunr: tmpfs has none, it uses the swap space + (10:39:31) mcsim: if we allocate memory for tmpfs using vm_allocate, will it be able to use swap partition? + (10:39:56) braunr: it should + (10:40:27) braunr: vm_allocate just maps anonymous memory + (10:41:27) braunr: anonymous memory uses swap space as its backing store too + (10:43:47) braunr: but be aware that this part of the vm system is known to have deficiencies + (10:44:14) braunr: which is why all mach based implementations have rewritten their default pager + (10:45:11) mcsim: what kind of deficiencies? + (10:45:16) braunr: bugs + (10:45:39) braunr: and design issues, making anonymous memory fragmentation horrible + + ... + + (15:23:33) antrik: mcsim: vm_allocate doesn't return a memory object; so it can't be passed to clients for mmap() + (15:50:37) mcsim: antrik: I use vm_allocate in pager_read_page + (15:54:43) antrik: mcsim: well, that means that you have to actually implement a pager yourself + (15:56:10) antrik: also, when the kernel asks the pager to write back some pages, it expects the memory to become free. + if you are "paging" to ordinary anonymous memory, this doesn't happen; so I expect it to have a very bad effect + on system performance + (15:56:54) antrik: both can be avoided by just passing a real anonymous memory object, i.e. one provided by the defpager + (15:57:07) antrik: only problem is that the current defpager implementation can't really handle that... + +#Pastes + +## fsx test on 22.11.11 <a id="fsx_fail2211"/> + $ ~/src/fsx/fsx -W -R -t 4096 -w 4096 -r 4096 bar + mapped writes DISABLED + truncating to largest ever: 0x32000 + truncating to largest ever: 0x39000 + READ BAD DATA: offset = 0x16000, size = 0xd9a0 + OFFSET GOOD BAD RANGE + 0x1f000 0x0000 0x01a0 0x 2d14 + operation# (mod 256) for the bad data may be 1 + LOG DUMP (16 total operations): + 1(1 mod 256): WRITE 0x1f000 thru 0x21d28 (0x2d29 bytes) HOLE ***WWWW + 2(2 mod 256): WRITE 0x10000 thru 0x15bfe (0x5bff bytes) + 3(3 mod 256): READ 0x1d000 thru 0x21668 (0x4669 bytes) + 4(4 mod 256): READ 0xa000 thru 0x16a59 (0xca5a bytes) + 5(5 mod 256): READ 0x8000 thru 0x8f2c (0xf2d bytes) + 6(6 mod 256): READ 0xa000 thru 0x17fe8 (0xdfe9 bytes) + 7(7 mod 256): READ 0x1b000 thru 0x20f33 (0x5f34 bytes) + 8(8 mod 256): READ 0x15000 thru 0x1c05b (0x705c bytes) + 9(9 mod 256): TRUNCATE UP from 0x21d29 to 0x32000 + 10(10 mod 256): READ 0x3000 thru 0x5431 (0x2432 bytes) + 11(11 mod 256): WRITE 0x29000 thru 0x34745 (0xb746 bytes) EXTEND + 12(12 mod 256): TRUNCATE DOWN from 0x34746 to 0x19000 ******WWWW + 13(13 mod 256): READ 0x14000 thru 0x186d8 (0x46d9 bytes) + 14(14 mod 256): TRUNCATE UP from 0x19000 to 0x39000 ******WWWW + 15(15 mod 256): WRITE 0x28000 thru 0x3548c (0xd48d bytes) + 16(16 mod 256): READ 0x16000 thru 0x2399f (0xd9a0 bytes) ***RRRR*** + Correct content saved for comparison + (maybe hexdump "bar" vs "bar.fsxgood") + +## fsx test on 24.11.11 <a id="fsx_fail2411"/> + + $ ~/src/fsx/fsx bar + truncating to largest ever: 0x13e76 + READ BAD DATA: offset = 0x1f62e, size = 0x2152 + OFFSET GOOD BAD RANGE + 0x1f62e 0x0206 0x0000 0x 213e + operation# (mod 256) for the bad data unknown, check HOLE and EXTEND ops + LOG DUMP (6 total operations): + 1(1 mod 256): TRUNCATE UP from 0x0 to 0x13e76 + 2(2 mod 256): WRITE 0x17098 thru 0x26857 (0xf7c0 bytes) HOLE ***WWWW + 3(3 mod 256): READ 0xc73e thru 0x1b801 (0xf0c4 bytes) + 4(4 mod 256): MAPWRITE 0x32e00 thru 0x331fc (0x3fd bytes) + 5(5 mod 256): MAPWRITE 0x7ac1 thru 0x11029 (0x9569 bytes) + 6(6 mod 256): READ 0x1f62e thru 0x2177f (0x2152 bytes) ***RRRR*** + Correct content saved for comparison + (maybe hexdump "bar" vs "bar.fsxgood") diff --git a/user/Sergio_Lopez.mdwn b/user/Sergio_Lopez.mdwn new file mode 100644 index 00000000..b514982e --- /dev/null +++ b/user/Sergio_Lopez.mdwn @@ -0,0 +1,70 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +## About Myself +- Name: Sergio Lopez +- Email: <slp@sinrega.org> +- Blog: <http://blogs.nologin.es/slopez> +- Languages: Spanish, English + +## Stuff I'm working on + +### Advisory Pageout + +#### Rationale +This work has two objectives: + +- Put a limit on the number of dirty (or better said, potentially dirty) pages that can be at one given time on Hurd. Having lots of dirty pages stresses the translators when syncing its memory objects, since they could receive a huge amount of m_o_data_return messages in a short amount of time. This is the primary cause for thread explosions. + +- When memory is scarce due to work load, try to avoid flushing anonymous memory. This is done by trying to satisfy the target of free pages releasing clean external pages (which can be considered as cache). + +#### Trying it +I've commited this work to this branch: + +<http://git.savannah.gnu.org/cgit/hurd/gnumach.git/log/?h=k0ro/advisory_pageout/master> + +You'll also need the counterpart for user space: + +<http://blogs.nologin.es/slopez/files/hurd-advisory_pageout.patch> + +If you try it, please let me know your experience with it. + +### Reduce task map fragmentation +After some hours of workload over an ext2fs translator, the number of entries in its memory map (which uses to be something around 60 when clean) is increased up to a few thousand. This increases the cost of each vm_map_lookup_entry, and this is perceived from a user perspective as a significant slowdown on the services provided by that translator. + +The causes for the fragmentation of the task map are: + +#### The problem + +- When a big chunk of memory is reserved, and the protection level is changed in small fractions of it, each of those generates a new entry in the map. This happens in glibc's malloc and sysdeps/mach/hurd/brk.c. + +- Allocating memory for permanent data structures in small chunks, without putting them at an specific address. GNU Mach has the ability of coalescing similar memory entries, but they must be contiguous. As every Hurd server needs to allocate small, temporal buffers for sending and receiving data, these ones will get in the middle of the permanent chunks, preventing them for being coalesced. This is the case of glibc's hurd/hurdmalloc.c, libpager's pagemap and libthread's per thread stack (and possibly some others). + +#### The solution + +- GNU Mach should be able of coalescing entries with similar properties after a call to vm_protect. I'm working in this direction. + +- The use of vm_allocate/mmap in translators should be avoided, except for temporal buffers for IPC operations. + +#### Workaround +While trying to find the causes of the task map fragmentation, I developed some changes which don't provide a complete, clean solution for this problem, but they minimize it (at least, for ext2fs) without significant drawbacks. You can find them here: + +- <http://blogs.nologin.es/slopez/files/glibc-memory_fragmentation.patch> +- <http://blogs.nologin.es/slopez/files/libpager-memory_fragmentation.patch> +- <http://blogs.nologin.es/slopez/files/libthreads-memory_fragmentation.patch> + +### memfs (Memory FS translator) + +### Other miscellaneous changes + +- ext2fs: write multiple pages at once to the backend storage. <http://blogs.nologin.es/slopez/files/ext2fs-multipage_data_return.patch> + +- GNU Mach: avoid calling vm_object_deactivate_pages every time an object enters the cache. <http://blogs.nologin.es/slopez/files/gnumach-optimize_page_deactivation.patch> + diff --git a/user/musial.mdwn b/user/musial.mdwn new file mode 100644 index 00000000..dee4588a --- /dev/null +++ b/user/musial.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Robert Musial - Cleveland, OH + +musial/at/gnu/dot/org + +http://musial.sollux.net |
