diff options
| -rw-r--r-- | faq/which_microkernel.mdwn (renamed from open_issues/sudo_date_crash.mdwn) | 11 | ||||
| -rw-r--r-- | history.mdwn | 10 | ||||
| -rw-r--r-- | history/port_to_another_microkernel.mdwn | 171 | ||||
| -rw-r--r-- | history/port_to_another_microkernel/discussion.mdwn (renamed from hurd/faq/which_microkernel.mdwn) | 45 | ||||
| -rw-r--r-- | history/port_to_l4.mdwn | 105 | ||||
| -rw-r--r-- | hurd-l4.mdwn | 8 | ||||
| -rw-r--r-- | hurd.mdwn | 8 | ||||
| -rw-r--r-- | hurd/ng.mdwn | 2 | ||||
| -rw-r--r-- | microkernel/coyotos.mdwn | 7 | ||||
| -rw-r--r-- | microkernel/l4.mdwn | 5 | ||||
| -rw-r--r-- | open_issues/anatomy_of_a_hurd_system.mdwn | 73 | ||||
| -rw-r--r-- | open_issues/ext2fs_page_cache_swapping_leak.mdwn | 23 | ||||
| -rw-r--r-- | open_issues/gdb_noninvasive_mode_new_threads.mdwn | 15 | ||||
| -rw-r--r-- | open_issues/gdb_thread_ids.mdwn | 12 | ||||
| -rw-r--r-- | open_issues/pfinet_vs_system_time_changes.mdwn | 42 | ||||
| -rw-r--r-- | open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn | 163 | ||||
| -rw-r--r-- | open_issues/unit_testing.mdwn | 20 |
17 files changed, 548 insertions, 172 deletions
diff --git a/open_issues/sudo_date_crash.mdwn b/faq/which_microkernel.mdwn index 53303abc..608e6b3f 100644 --- a/open_issues/sudo_date_crash.mdwn +++ b/faq/which_microkernel.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,9 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_gnumach]] +[[!meta title="What happened with the ports to the L4 / Coyotos / Viengoos +microkernels?"]] -IRC, unknown channel, unknown date. - - <grey_gandalf> I did a sudo date... - <grey_gandalf> and the machine hangs +This story is told on the page about the +[[history/port_to_another_microkernel]]. diff --git a/history.mdwn b/history.mdwn index 8f155b54..0abcbd52 100644 --- a/history.mdwn +++ b/history.mdwn @@ -1,13 +1,13 @@ -[[!meta copyright="Copyright © 1998, 1999, 2001, 2002, 2007, 2008, 2009 Free -Software Foundation, Inc."]] +[[!meta copyright="Copyright © 1998, 1999, 2001, 2002, 2007, 2008, 2009, 2011 +Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!tag stable_URL]] @@ -91,4 +91,4 @@ mailing lists. --- - * [[Port_to_L4]] + * [[Port_to_another_microkernel]] diff --git a/history/port_to_another_microkernel.mdwn b/history/port_to_another_microkernel.mdwn new file mode 100644 index 00000000..b347cf38 --- /dev/null +++ b/history/port_to_another_microkernel.mdwn @@ -0,0 +1,171 @@ +[[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, +2009, 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Porting the Hurd to another microkernel"]] + +At first, there was an effort to port the Hurd from the [[Mach +microkernel|microkernel/mach]] to the [[L4_microkernel_family|microkernel/l4]]. +Then the story continued... + +[[!toc levels=2]] + + +# L4 + +## Initial Idea + +Encountering a number of fundamental design issues with the [[Mach +microkernel|microkernel/mach]] (mostly regarding [[resource +management|open_issues/resource_management_problems]]), some of the Hurd +developers began experimenting with using other microkernels for the Hurd +around the turn of the millenium. + +The idea of using L4 as a [[microkernel]] for a Hurd system was initially +voiced in the [[community]] by Okuji Yoshinori, who, for discussing this +purpose, created the [[mailing_lists/l4-hurd]] mailing list in November 2000. + +Over the years, a lot of discussion have been held on this mailing list, which +today is still the right place for [[next-generation Hurd|hurd/ng]] +discussions. + + +## Why? + +Even though that said resource management issues constitute a broad research +topic, there was no hope that the original Mach project would work on these: +[[microkernel/Mach]] wasn't maintained by its original authors anymore. Mach +had served its purpose as a research vehicle, and has been retired by its +stakeholders. + +Thus, switching to a well-maintained current [[microkernel]] was expected to +yield a more solid foundation for a Hurd system than the [[decaying +Mach|microkernel/mach/history]] design and implementation was able to. + +At that time, the [[L4 microkernel family|microkernel/L4]] was one obvious +choice. Being a second-generation microkernel, it was deemed to provide for a +faster system kernel implementation, especially in the time-critical [[IPC]] +paths. Also, as L4 was already implemented for a bunch of different +architectures (x86, Alpha, MIPS; also including SMP support), and the Hurd +itself being rather archtecture-agnostic, it was expected to be able to easily +support more platforms than with the existing system. + + +## Steps and Goals + +At the same time, the idea was -- while mucking with the system's core anyway +-- to improve on some fundamental design issues, too -- like the resource +management problems, for example. + +One goal of porting the Hurd to L4 was to make the Hurd independent of +[[microkernel/Mach]] interfaces, to make it somewhat microkernel-agnostic. + +One idea was to first introduce a Mach-on-L4 emulation layer, to easily get a +usable (though slow) Hurd-using-Mach-interfaces-on-L4 system, and then +gradually move the Hurd servers to use L4 intefaces rather than Mach ones. + +A design upon the lean L4 kernel would finally have made it feasible to move +devices drivers out of the kernel's [[TCB]]. + + +# Implementation + +The project itself then was mostly lead by Marcus Brinkmann and Neal Walfield. +Neal started the original Hurd/L4 port while visiting Karlsruhe university in +2002. He explains: + +> My intention was to adapt the Hurd to exploit L4's concepts and intended +> [[design_pattern]]s; it was not to simply provide a Mach +> [[compatibility_layer]] on top of L4. When I left Karlsruhe, I no longer had +> access to [[microkernel/l4/Pistachio]] as I was unwilling to sign an NDA. +> Although the specification was available, the Karlsruhe group only [released +> their code in May +> 2003](https://lists.ira.uni-karlsruhe.de/pipermail/l4ka/2003-May/000345.html). +> Around this time, Marcus began hacking on Pistachio. He created a relatively +> complete run-time. I didn't really become involved again until the second +> half of 2004, after I complete by Bachelors degree. + +Development of Hurd/L4 was done in the [CVS module +`hurd-l4`](http://savannah.gnu.org/cgi-bin/viewcvs/hurd/hurd-l4/). The `doc` +directory contains a design document that is worth reading for anyone who +wishes to learn more about Hurd/L4. + +Even though there was progress -- see, for example, the [[QEMU image for +L4|hurd/running/qemu/image_for_l4]] -- this port never reached a releasable +state. Simple POSIX programs, such as `banner` could run, but for more complex +system interfaces, a lot more work was needed. + +Eventually, a straight-forward port of the original Hurd's design wasn't deemed +feasible anymore by the developers, partly due to them not cosidering L4 +suitable for implementing a general-purpose operating system on top of it, and +because of deficiencies in the original Hurd's design, which they discovered +along their way. Neal goes on: + +> Before Marcus and I considered [[microkernel/Coyotos]], we had already +> rejected some parts of the Hurd's design. The +> [[open_issues/resource_management_problems]] were +> what prompted me to look at L4. Also, some of the problems with +> [[hurd/translator]]s were already well-known to us. (For a more detailed +> description of the problems we have identified, see our [[hurd/critique]] in the +> 2007 July's SIGOPS OSR. We have also written a forward-looking +> [[hurd/ng/position_paper]].) + +> We visited Jonathan Shapiro at Hopkins in January 2006. This resulted in a +> number of discussions, some quite influential, and not always in a way which +> aligned our position with that of Jonathan's. This was particularly true of +> a number of security issues. + +A lange number of discussion threads can be found in the archives of the +[[mailing_lists/l4-hurd]] mailing list. + +> Hurd-NG, as we originally called it, was an attempt to articulate the system +> that we had come to envision in terms of interfaces and description of the +> system's structure. The new name was selected, if I recall correctly, as it +> clearly wasn't the Hurd nor the Hurd based on L4. + + +## Termination + +As of 2005, development of Hurd/L4 has stopped. + + +# Coyotos + +Following that, an attempt was started to use the kernel of the +[[microkernel/Coyotos]] system. As Coyotos is an object capability system +througout, the microkernel would obviously be more suitable for this purpose; +and it looked pretty promising in the beginning. However, further +investigations found that there are some very fundamental philosophical +differences between the Coyotos and Hurd designs; and thus this this attempt +was also abandonned, around 2006 / 2007. (This time before producing any +actual code.) + + +# Viengoos + +By now (that is, after 2006), there were some new [[microkernel/L4]] variants +available, which added protected [[IPC]] paths and other features necessary for +object capability systems; so it might be possible to implement the Hurd on top +of these. However, by that time the developers concluded that microkernel +design and system design are interconnected in very intricate ways, and thus +trying to use a third-party microkernel will always result in trouble. So Neal +Walfield created the experimental [[microkernel/Viengoos]] kernel instead -- +based on the experience from the previous experiments with L4 and Coyotos -- +for his [[research on resource +management|open_issues/resource_management_problems]]. Currently he works in +another research area though, and thus Viengoos is on hold. + + +# Intermediate Results + +Note that while none of the microkernel work is active now, the previous +experiments already yielded a lot of experience, which will be very useful in +the further development / improvement of the mainline (Mach-based) Hurd +implementation. diff --git a/hurd/faq/which_microkernel.mdwn b/history/port_to_another_microkernel/discussion.mdwn index f6225188..f2161195 100644 --- a/hurd/faq/which_microkernel.mdwn +++ b/history/port_to_another_microkernel/discussion.mdwn @@ -8,49 +8,6 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!meta title="What happened to the L4 / Coyotos / Viengoos microkernels?"]] - -Encountering a number of fundamental design issues with the Mach microkernel -(mostly with resource management), -some of the Hurd developers began experimenting with using other microkernels for the Hurd. - -The first attempt was reimplementing the Hurd on the L4 (Pistachio) microkernel. -This got going around 2003/2004, -and got pretty far (running some simple POSIX programs, such as "banner"); -however over time some lingering design issues turned out to be fundamental problems: -the original L4 is not suitable for building object capability systems like the Hurd. -Thus development was aborted in 2005. - -Following that, an attempt was started to use the kernel of the Coyotos system. -As Coyotos is an object-capability system througout, -the microkernel would obviously be more suitable for this purpose; -and it looked pretty promising in the beginning. -However, further investigations found -that there are some very fundamental philosophical differences -between the Coyotos and Hurd designs; -and thus this this attempt was also abandonned, around 2006/2007. -(This time before producing any actual code.) - -By now there were some new L4 variants available, -which added protected IPC paths and other features necessary for object capability systems; -so it might be possible to implement the Hurd on top of these. -However, by that time the developers concluded that microkernel design and system design -are interconnected in very intricate ways, -and thus trying to use a third-party microkernel will always result in trouble. -So Neal Walfield created the experimental [[Viengoos|microkernel/viengoos]] kernel instead -- -based on the experience from the previous experiments with L4 and Coyotos -- -for his research on resource management. -Currently he works in another research area though, and thus Viengoos is on hold. - -Note that while none of the microkernel work is active now, -the previous experiments already yielded a lot of experience, -which will be very useful in the further development/improvement -of the mainline (Mach-based) Hurd implementation. - -<!-- - ---- - IRC, #hurd, 2011-01-12. [[!taglink open_issue_documentation]] @@ -110,5 +67,3 @@ IRC, #hurd, 2011-01-12. <antrik> manpower is not something that comes from nowhere. again, having something working is crucial in a volunteer project like this <antrik> there are no fixed plans - ---> diff --git a/history/port_to_l4.mdwn b/history/port_to_l4.mdwn index b58c0d91..3f951a64 100644 --- a/history/port_to_l4.mdwn +++ b/history/port_to_l4.mdwn @@ -1,108 +1,13 @@ -[[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, -2009, 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="Porting the Hurd to L4: Hurd/L4"]] +[[!tag stable_URL]] -There was an effort to port the Hurd from [[microkernel/Mach]] to the -[[L4_microkernel_family|microkernel/L4]]. - -The idea of using L4 as a [[microkernel]] for a [[Hurd_system|hurd]] was -initially voiced in the [[Hurd_community|community]] by Okuji Yoshinori, who, -for discussing this purpose, created the [[mailing lists/l4-hurd]] mailing list -in November 2000. - -The project itself then was mostly lead by Marcus Brinkmann and Neal Walfield. -Even though there was progress -- see, for example, the -[[QEMU image for L4|hurd/running/qemu/image for l4]] -- this port never reached a -releasable state. Eventually, a straight-forward port of the original Hurd's -design wasn't deemed feasible anymore by the developers, partly due to them not -cosidering L4 suitable for implementing a general-purpose operating system on -top of it, and because of deficiencies in the original Hurd's design, which -they discovered along their way. Read the [[hurd/critique]] and a -[[hurd/ng/position paper]]. - -By now, the development of Hurd/L4 has stopped. However, Neal Walfield moved -on to working on a newly designed kernel called [[microkernel/viengoos]]. - -Over the years, a lot of discussion have been held on the -[[mailing lists/l4-hurd]] mailing list, which today is still the right place -for [[next-generation Hurd|hurd/ng]] discussions. - -Development of Hurd/L4 was done in the `hurd-l4` module of the Hurd CVS -repository. The `doc` directory contains a design document that is worth -reading for anyone who wishes to learn more about Hurd/L4. - - -One goal of porting the Hurd to L4 was to make the Hurd independend of Mach -interfaces, to make it somewhat microkernel-agnostic. - -Mach wasn't maintained by its original authors anymore, so switching to a -well-maintained current [[microkernel]] was expected to yield a more solid -foundation for a Hurd system than the decaying Mach design and implementation -was able to. - -L4 being a second-generation [[microkernel]] was deemed to provide for a faster -system kernel implementation, especially in the time-critical [[IPC]] paths. -Also, as L4 was already implemented for a bunch of different architectures -(IA32, Alpha, MIPS; SMP), and the Hurd itself being rather archtecture-unaware, -it was expected to be able to easily support more platforms than with the -existing system. - -A design upon the lean L4 kernel would finally have moved devices drivers out -of the kernel's [[TCB]]. - - -One idea was to first introduce a Mach-on-L4 emulation layer, to easily get a -usable (though slow) Hurd-using-Mach-interfaces-on-L4 system, and then -gradually move the Hurd servers to use L4 intefaces rather than Mach ones. - - -Neal Walfield started the original Hurd/L4 port while at Karlsruhe in 2002. He -explains: - -> My intention was to adapt the Hurd to exploit L4's concepts and intended -> [[design_pattern]]s; it was not to simply provide a Mach -> [[compatibility_layer]] on top of L4. When I left Karlsruhe, I no longer had -> access to [[microkernel/l4/Pistachio]] as I was unwilling to sign an NDA. -> Although the specification was available, the Karlsruhe group only [released -> their code in May -> 2003](https://lists.ira.uni-karlsruhe.de/pipermail/l4ka/2003-May/000345.html). -> Around this time, Marcus began hacking on Pistachio. He created a relatively -> complete run-time. I didn't really become involved again until the second -> half of 2004, after I complete by Bachelors degree. - -> Before Marcus and I considered [[microkernel/Coyotos]], we had already -> rejected some parts of the Hurd's design. The -> [[open issues/resource management problems]] were -> what prompted me to look at L4. Also, some of the problems with -> [[hurd/translator]]s were already well-known to us. (For a more detailed -> description of the problems we have identified, see our [[hurd/critique]] in the -> 2007 July's SIGOPS OSR. We have also written a forward-looking -> [[hurd/ng/position paper]].) - -> We visited Jonathan Shapiro at Hopkins in January 2006. This resulted in a -> number of discussions, some quite influential, and not always in a way which -> aligned our position with that of Jonathan's. This was particularly true of -> a number of security issues. - -A lange number of discussion threads can be found in the archives of the -[[mailing lists/l4-hurd]] mailing list. - -> Hurd-NG, as we originally called it, was an attempt to articulate the system -> that we had come to envision in terms of interfaces and description of the -> system's structure. The new name was selected, if I recall correctly, as it -> clearly wasn't the Hurd nor the Hurd based on L4. - - -The source code is still available in [CVS module -`hurd-l4`](http://savannah.gnu.org/cgi-bin/viewcvs/hurd/hurd-l4/) (note that -this repository has in the beginning also been used for Neal's -[[microkernel/Viengoos]]). +[[!meta redir=port_to_another_microkernel]] diff --git a/hurd-l4.mdwn b/hurd-l4.mdwn index 579c1190..afc8f8f3 100644 --- a/hurd-l4.mdwn +++ b/hurd-l4.mdwn @@ -1,13 +1,13 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!tag stable_URL]] -[[!meta redir=history/port_to_l4]] +[[!meta redir=history/port_to_another_microkernel]] @@ -1,13 +1,13 @@ [[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, -2009, 2010 Free Software Foundation, Inc."]] +2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] The GNU Hurd is under active development. Because of that, there is no *stable* version. We distribute the Hurd sources only through [[Git|source_repositories]] at present. @@ -31,7 +31,7 @@ in the *unstable* branch of the Debian archive. * [[What_Is_the_GNU_Hurd]] - A Brief Description * [[Advantages]]. And [[challenges]]. * [[History]] - * [[history/Port_to_L4]] + * [[history/Port_to_another_microkernel]] * [[Logo]] * [[Status]] * [[KnownHurdLimits]] diff --git a/hurd/ng.mdwn b/hurd/ng.mdwn index de33949d..481386a4 100644 --- a/hurd/ng.mdwn +++ b/hurd/ng.mdwn @@ -13,7 +13,7 @@ This section explains the motivations behind the new design: * [[Issues_with_L4_Pistachio]] * [[Limitations_of_the_original_Hurd_design]] - * History of the [[history/port_to_L4]] + * History of the [[history/port_to_another_microkernel]] # Work already done diff --git a/microkernel/coyotos.mdwn b/microkernel/coyotos.mdwn index 5ecea688..fec023ba 100644 --- a/microkernel/coyotos.mdwn +++ b/microkernel/coyotos.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2006, 2007, 2008, 2010 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2006, 2007, 2008, 2010, 2011 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -28,3 +28,6 @@ design. The coyotos microkernel specification can be found [here](http://www.coyotos.org/docs/ukernel/spec.html). + +There once was the idea of a GNU/Hurd [[port using the Coyotos +microkernel|history/port_to_another_microkernel]], but this didn't come live. diff --git a/microkernel/l4.mdwn b/microkernel/l4.mdwn index 45929842..7af5e6fc 100644 --- a/microkernel/l4.mdwn +++ b/microkernel/l4.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2004, 2006, 2007, 2008, 2010 Free Software +[[!meta copyright="Copyright © 2004, 2006, 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -20,7 +20,8 @@ on formally verifying an L4 microkernel. * {{$sel4}} -There was a GNU/Hurd [[history/port_to_L4]], which is now stalled. +There was a GNU/Hurd [[port to L4|history/port_to_another_microkernel]], which +is now stalled. [[!ymlfront data=""" diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn new file mode 100644 index 00000000..e1d5c9d8 --- /dev/null +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -0,0 +1,73 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!taglink open_issue_documentation]] + +A bunch of this should also be covered in other (introductionary) material, +like Bushnell's Hurd paper. All this should be unfied and streamlined. + +IRC, freenode, #hurd, 2011-03-08 + + <foocraft> I've a question on what are the "units" in the hurd project, if + you were to divide them into units if they aren't, and what are the + dependency relations between those units(roughly, nothing too pedantic + for now) + <antrik> there is GNU Mach (the microkernel); there are the server + libraries in the Hurd package; there are the actual servers in the same; + and there is the POSIX implementation layer in glibc + <antrik> relations are a bit tricky + <antrik> Mach is the base layer which implements IPC and memory management + <foocraft> hmm I'll probably allocate time for dependency graph generation, + in the worst case + <antrik> on top of this, the Hurd servers, using the server libraries, + implement various aspects of the system functionality + <antrik> client programs use libc calls to use the servers + <antrik> (servers also use libc to communicate with other servers and/or + Mach though) + <foocraft> so every server depends solely on mach, and no other server? + <foocraft> s/mach/mach and/or libc/ + <antrik> I think these things should be pretty clear one you are somewhat + familiar with the Hurd architecture... nothing really tricky there + <antrik> no + <antrik> servers often depend on other servers for certain functionality + +--- + +IRC, freenode, #hurd, 2011-03-12 + + <dEhiN> when mach first starts up, does it have some basic i/o or fs + functionality built into it to start up the initial hurd translators? + <antrik> I/O is presently completely in Mach + <antrik> filesystems are in userspace + <antrik> the root filesystem and exec server are loaded by grub + <dEhiN> o I see + <dEhiN> so in order to start hurd, you would have to start mach and + simultaneously start the root filesystem and exec server? + <antrik> not exactly + <antrik> GRUB loads all three, and then starts Mach. Mach in turn starts + the servers according to the multiboot information passed from GRUB + <dEhiN> ok, so does GRUB load them into ram? + <dEhiN> I'm trying to figure out in my mind how hurd is initially started + up from a low-level pov + <antrik> yes, as I said, GRUB loads them + <dEhiN> ok, thanks antrik...I'm new to the idea of microkernels, but a + veteran of monolithic kernels + <dEhiN> although I just learned that windows nt is a hybrid kernel which I + never knew! + <rm> note there's a /hurd/ext2fs.static + <rm> I belive that's what is used initially... right? + <antrik> yes + <antrik> loading the shared libraries in addition to the actual server + would be unweildy + <antrik> so the root FS server is linked statically instead + <dEhiN> what does the root FS server do? + <antrik> well, it serves the root FS ;-) + <antrik> it also does some bootstrapping work during startup, to bring the + rest of the system up diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn new file mode 100644 index 00000000..0ace5cd3 --- /dev/null +++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn @@ -0,0 +1,23 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +IRC, OFTC, #debian-hurd, 2011-03-24 + + <youpi> I still believe we have an ext2fs page cache swapping leak, however + <youpi> as the 1.8GiB swap was full, yet the ld process was only 1.5GiB big + <pinotree> a leak at swapping time, you mean? + <youpi> I mean the ext2fs page cache being swapped out instead of simply + dropped + <pinotree> ah + <pinotree> so the swap tends to accumulate unuseful stuff, i see + <youpi> yes + <youpi> the disk content, basicallyt :) diff --git a/open_issues/gdb_noninvasive_mode_new_threads.mdwn b/open_issues/gdb_noninvasive_mode_new_threads.mdwn new file mode 100644 index 00000000..9b3992f4 --- /dev/null +++ b/open_issues/gdb_noninvasive_mode_new_threads.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb]] + +Debugging a translator. `gdb binary`. `set noninvasive on`. `attach [PID]`. +Translator does some work. GDB doesn't notice new threads. `detach`. `attach +[PID]` -- now new threads are visible. diff --git a/open_issues/gdb_thread_ids.mdwn b/open_issues/gdb_thread_ids.mdwn index c31a9967..c04a10ee 100644 --- a/open_issues/gdb_thread_ids.mdwn +++ b/open_issues/gdb_thread_ids.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="GDB: thread ids"]] @@ -23,3 +23,9 @@ GNU GDB's Pedro Alves: Also see [[thread numbering of ps and GDB]]. + +--- + +`attach` to a multi-threaded process. See threads 1 to 5. `detach`. `attach` +again -- thread numbers continue where they stopped last time: now they're +threads 6 to 10. diff --git a/open_issues/pfinet_vs_system_time_changes.mdwn b/open_issues/pfinet_vs_system_time_changes.mdwn new file mode 100644 index 00000000..a9e1e242 --- /dev/null +++ b/open_issues/pfinet_vs_system_time_changes.mdwn @@ -0,0 +1,42 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +IRC, unknown channel, unknown date. + + <grey_gandalf> I did a sudo date... + <grey_gandalf> and the machine hangs + +This was very likely as misdiagnosis: + +IRC, freenode, #hurd, 2011-03-25 + + <tschwinge> antrik: I suspect it'S some timing stuff in pfinet that perhaps + uses absolute time, and somehow wildely gets confused? + <antrik> tschwinge: BTW, pfinet doesn't actually die I think -- it just + drops open connections... + <antrik> perhaps it thinks they timed out + <tschwinge> antrik: Isn't the translator restarted instead? + <antrik> don't think so + <antrik> when pfinet actually dies, I also loose the NFS mounts, which + doesn't happen in this case + <antrik> hehe "... and the machine hangs" + <antrik> he didn't bother to check that the machine is perfectly fine, only + the SSH connection got dropped + <tschwinge> Ah, I see. So it'S perhaps indeed simply closes TCP + connections that have been without data for ``too long''? + <antrik> yeah, that's my guess + <antrik> my clock is speeding, so ntpdate sets it in the past + <antrik> perhaps there is some math that concludes the connection have been + inactive for -200 seconds, which (unsigned) is more than any timeout :-) + <tschwinge> (The other way round, you might likely get some integer + wrap-around, and thus the same result.) + <tschwinge> Yes. diff --git a/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn b/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn new file mode 100644 index 00000000..9db92250 --- /dev/null +++ b/open_issues/rpc_to_self_with_rendez-vous_leading_to_duplicate_port_destroy.mdwn @@ -0,0 +1,163 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[RPC to self with rendez-vous leading to duplicate port +destroy](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00045.html) + +IRC, freenode, #hurd, 2011-03-14 + + <antrik> youpi: I wonder, why does the root FS call diskfs_S_dir_lookup() + at all?... + <youpi> errr, because a client asked for it? + <youpi> (problem with RPCs is you can't easily know where they come from :) + ) + <youpi> (especially when it's the root fs...) + <antrik> ah, it's about a client request... didn't see that + <youpi> well, I just said "is called", yes + <antrik> I do not really understand though why it tries to reauthenticate + against itself... + <antrik> I fear my memory of the lookup mechanism grew a bit dim + <youpi> see the source + <youpi> it's about a translated entry + <antrik> (and I never fully understood some aspects anyways...) + <youpi> it needs to start the translated entry as another user, possibly + <antrik> yes, but a translated entry normally would be served by *another* + process?... + <youpi> sure, but ext2fs has to prepare it + <youpi> thus reauthenticate to prepare the correct set of rights + <antrik> prepare what? + <youpi> rights + <youpi> so the process is not root, doesn't have / opened as root, etc. + <antrik> rights for what? + <youpi> err, about everything + <antrik> IIRC the reauthentication is done by the parent FS on the port to + the *translated* node + <antrik> and the translated node should be a different process?... + <youpi> that's not what I read in the source + <youpi> fshelp_fetch_root + <youpi> ports[INIT_PORT_CRDIR] = reauth (getcrdir ()); + <youpi> here, getcrdir() returns ext2fs itself + <antrik> well, perhaps the issue is that I have no idea what + fshelp_fetch_root() does, nor why it is called here... + <youpi> it notably starts the translator that dir_lookup is looking at, if + needed + <youpi> possibly as a different user, thus reauthentication of CRDIR + <antrik> so this is about a port that is passed to the translator being + started? + <youpi> no + <youpi> well, depends on what you mean by "port" + <youpi> it's about reauthenticating a port to be passed to the translator + being started + <youpi> and for that a rendez-vous port is needed for the reauthentication + <youpi> and that's the one at stake + <antrik> yeah, I meant the port that is reauthenticated + <antrik> what is CRDIR? + <youpi> current root dir ... + <antrik> so the parent translator passes it's own root dir to the child + translator; and the issue is that for the root FS the root dir points to + the root FS itself... + <youpi> yes + <antrik> OK, that makes sense + <youpi> (but that's only one example, rgrep mach_port_destroy hurd/ show + other potential issues) + <antrik> well, that's actually what I wanted to mention next... why is the + rendez-vous port destroyed, instead of just deallocating the port right + and letting reference counting to it's thing?... + <antrik> do its thing + <youpi> "just to make sure" I guess + <antrik> it's pretty obvious that this will cause trouble for any RPC + referencing itself... + <youpi> well, follow-up with that on the list + <youpi> with roland/tb in CC + <youpi> only they would know any real reason for destroy + <youpi> btw, if you knew how we could make _hurd_select()'s raw __mach_msg + call be interruptible by signals, that'll permit to fix sudo + <youpi> (damn, I need sleep, my tenses are all wrong) + <antrik> BTW, does this cause any actual trouble?... + <antrik> I don't know much about interruption... cfhammer might have a + better idea, he look into that stuff quite a bit AIUI + <antrik> looked + <antrik> (hehe, it's not only your tenses... guess there's something in the + ether ;-) ) + <youpi> it makes sudo, mailq, etc. fail sometimes + <antrik> I mean the rendez-vous thing + <youpi> that's it, yes + <youpi> sudo etc. fail at least due to this + <antrik> so these are two different problems that both affect sudo? + <antrik> (rendez-vous and interruption I mean) + <youpi> yes + <youpi> with my patch the buildds have much fewer issues, but still some + <youpi> (my interrupt-related patch) + <youpi> I'm installing a s/destroy/deallocate/ version of ext2fs on the + buildds, we'll see how it behaves + <youpi> (it fixes my testcase at least) + <antrik> interrupt-related patch? + <antrik> only thing interrupt-related I remember was the reauthentication + race... + <youpi> that's what I mean + <antrik> well, cfhammer investigated this is quite some depth, explaining + quite well why the race is only mitigated but still exists... problem is + that we didn't know how to fix it properly + <antrik> because nobody seems to understand the cancellation code, except + perhaps for Roland and Thomas + <antrik> (and I'm not even entirely sure about them :-) ) + <antrik> I think his findings and our conclusions are documented on the + ML... + <youpi> by "much fewer issues", I mean that some of the symptoms have + disappeared, others haven't + <antrik> BTW, couldn't the rendez-vous thing be worked around by simply + ignoring the errors from the failing deallocate?... + <youpi> no, failing deallocate are actually dangerous + <antrik> why? + <youpi> since the name might have been reused for something else in the + meanwhile + <youpi> that's the whole point of the warning I had added in the kernel + itself + <antrik> I see + <youpi> such things really deserve tracking, since they can have any kind + of consequence + <antrik> does Mach try to reuse names quickly, rather than only after + wrapping around?... + <youpi> it seems to + <antrik> OK, then this is a serious problem indeed + <youpi> (note: I rarely divine issues when there aren't actual frequent + symptoms :) ) + <antrik> well, the problem with the warning is that it only shows in the + cases that do *not* cause a problem... so it's hard to associate them + with any specific issues + <youpi> well, most of the time the port is not reused quickly enough + <youpi> so in most case it shows up more often than causing problem + +IRC, freenode, #hurd, 2011-03-14 + + <youpi> ok, mach_port_deallocate actually can't be used + <youpi> since mach_reply_port() returns a receive right, not a send right + * youpi guesses he will really have to manage to understand all that port + stuff completely + <antrik> oh, right + <antrik> youpi: hm... now I'm confused though. if one client holds a + receive right, the other client (or in this case the same process) should + have a send or send-once right -- these should *not* share the same name + in my understanding + <antrik> destroying the receive right should turn the send right into a + dead name + <antrik> so unless I'm missing something, the destroy shouldn't be a + problem, and there must be something else going wrong + <antrik> hm... actually I'm probably wrong + <antrik> yeah, definitely wrong. receive rights and "ordinary" send rights + share the name. only send-once rights are special + <antrik> I wonder whether the problem could be worked around by using a + send-once right... + <antrik> mach_port_mod_refs(mach_task_self(), name, + MACH_PORT_RIGHT_RECEIVE, -1) can be used to deallocate only the receive + right + <antrik> oh, you already figured that out :-) diff --git a/open_issues/unit_testing.mdwn b/open_issues/unit_testing.mdwn index a5ffe19d..feda3be4 100644 --- a/open_issues/unit_testing.mdwn +++ b/open_issues/unit_testing.mdwn @@ -320,3 +320,23 @@ freenode, #hurd channel, 2011-03-07: this, and just generally though that some sort of automated testing is needed, and thus started collecting ideas. <tschwinge> antrik: You're of course invited to fix that. + +IRC, freenode, #hurd, 2011-03-08 + +(After discussing the [[anatomy_of_a_hurd_system]].) + + <antrik> so that's what your question is actually about? + <foocraft> so what I would imagine is a set of only-this-server tests for + each server, and then we can have fun adding composite tests + <foocraft> thus making debugging the composite scenarios a bit less tricky + <antrik> indeed + <foocraft> and if you were trying to pass a composite test, it would also + help knowing that you still didn't break the server-only test + <antrik> there are so many different things that can be tested... the + summer will only suffice to dip into this really :-) + <foocraft> yeah, I'm designing my proposal to focus on 1) make/use a + testing framework that fits the Hurd case very well 2) write some tests + and docs on how to write good tests + <antrik> well, doesn't have to be *one* framework... unit testing and + regression testing are quite different things, which can be covered by + different frameworks |