diff options
73 files changed, 3710 insertions, 354 deletions
diff --git a/community/gsoc.mdwn b/community/gsoc.mdwn index 7e355ec4..e6f07822 100644 --- a/community/gsoc.mdwn +++ b/community/gsoc.mdwn @@ -11,20 +11,20 @@ License|/fdl]]."]]"""]] [[!meta title="Google Summer of Code"]] +The Google Summer of Code 2013 is over. Chances are that we will again be +participating in 2014, stay tuned. + +<!-- We're in! The GNU Hurd project is again participating in the [Google Summer of Code](http://www.google-melange.com/) under the [GNU umbrella](http://www.gnu.org/software/soc-projects/). -<!-- - As of Monday, 2013-04-22 it's the *student application period*. This will last until [Friday, 2013-05-03](http://www.google-melange.com/gsoc/events/google/gsoc2013), which is plenty of time for preparing and discussing your applications -- but please don't wait to the last minute! ---> - This year's *student application period* is over. Thanks for sending in your applications! We're now reviewing and discussing these, so please pay attention to any questions posted on your proposal's page. The Google site's @@ -37,17 +37,24 @@ with us, be it by answering the evaluators' questions on your proposal's page, or by talking to us on the [[mailing_lists]] or on [[IRC]]. At this time, it is important for us to get a good impression about the seriousness you're showing with your application. +--> +If you intend to apply for any such projects in the future, it's a good idea to +already start perparing for it now: the sooner, the better. It is a good idea to get familiar with the GNU Hurd, by reading some of our [[documentation]], and by using a GNU/Hurd system. It is also a good idea to -send in some basic patches (this has already been mentioned in our -[[student_application_form]]), or discuss with us the principal steps you're -planning on doing in your intended work area. Of course, we don't expect you +send in some basic patches (as mentioned in our +[[student_application_form]]), and talk to us on the [[mailing_lists]] or +on [[IRC]], for example about the principal steps you're +planning on doing in your intended work area. +<!-- +Of course, we don't expect you to already start working seriously on your project, but any input you're giving us will make it easier for us to justify selectiong your specific proposal. At this time, it is not quantity that matters, and it also is not *the perfect patch* we're waiting for, but it is rather that we see how you're generally able to work with the code. +--> If you have any questions, don't be shy: please ask! Nobody expects you to know everything. Even for the long-term Hurd contributors it is common to @@ -56,16 +63,14 @@ how to do `X`, can someone please help me?* And, as we're not working next to each other in a conventional office or university setup, we'll need to establish and get used to different communication channels. -[Timeline](http://www.google-melange.com/gsoc/events/google/gsoc2013). - <!-- +[Timeline](http://www.google-melange.com/gsoc/events/google/gsoc2013). As boring as it is, but the next step is waiting: we will have to wait for Google to announce the number of slots that the whole GNU project gets, and we'll be discussing with our GNU peers about how to split these up among all the GNU subprojects. - --> @@ -75,8 +80,10 @@ We have a list of [[project_ideas]], and students are likewise encouraged to submit their own project proposals. Please follow our [[student_application_form]]. +<!-- Then, don't forget to visit <http://www.google-melange.com/>, and press the button for submitting your proposal. +--> Please read up about [[contributing]] in general, and please ask any questions you might have, on the [[mailing_lists]], or on [[IRC]], for example at one of diff --git a/community/gsoc/2013/hacklu.mdwn b/community/gsoc/2013/hacklu.mdwn index b7de141b..d3e43dd6 100644 --- a/community/gsoc/2013/hacklu.mdwn +++ b/community/gsoc/2013/hacklu.mdwn @@ -1177,23 +1177,6 @@ In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: <hacklu> tschwinge: ok , this doesn't affect me now. If I have time I will figure out it. - <teythoon> btw, what about the copyright assignment process? - <tschwinge> teythoon, hacklu: You still haven't heard from the FSF about - your copyright assignments? What's the latest you have heard? - <hacklu> tschwinge: I have wrote a emali to ask for that, but no reply. - <teythoon> tschwinge: last and only response I got was on July 1st, the - last ping with explicit request for confirmation was on July the 12th - <tschwinge> hacklu: When did you send this email? - <hacklu> tschwinge: last week. - <tschwinge> teythoon: I suggest you send another inquiry, and please put me - in CC. And if there'S no answer within a couple days (well, I'm away - until Monday...), I'll follow up. - <tschwinge> hacklu: Likewise for you; depending on when exactly ;-) you - sent the last email. (Always allow for a few days until you exect an - answer, but if nothing happend within a week for such rather simple - administrative tasks, better ask again, unfrotunately.) - <hacklu> tschwinge:ok , I will email more - <hacklu> how to understand the asyn RPC? <braunr> hacklu: hm ? <hacklu> for instance, [hurd]/proc/main.c proc_server is loop in listening @@ -1619,25 +1602,6 @@ In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: <youpi> I'd rather let tschwinge comment on this <hacklu> youpi: ok :) - <youpi> how about the copyright assignments? did hacklu or teythoon receive - any answer? - <teythoon> youpi: I did, the copyright clerk told me that he finally got my - papers and that everything is in order now - <youpi> few! - <youpi> s/f/ph - <youpi> teythoon: you mean all steps are supposed to be done now, or is he - doing the last steps? I don't see your name in the copyright folder yet - <teythoon> youpi: well, he said that he had the papers and they are about - to be signed - <youpi> teythoon: ok, so it's not finished, that's why your name is not on - the list yet - <youpi> this paper stuff is really a pain - <hacklu> youpi: I haven't got any answer from FSF now. - <youpi> did you ping them recently? - <hacklu> I have pinged 2 week ago. - <hacklu> what you mean of ping? I just write an email to him. Is it enough? - <youpi> yes - # IRC, freenode, #hurd, 2013-08-12 @@ -1733,9 +1697,6 @@ In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: < hacklu> hello everyone, this is my week report. http://hacklu.com/blog/gsoc-weekly-report10-174/ - < hacklu> btw, my FSF copyright assignment has been concepted. They guy - said, they have recived my mail for a while but forget to handle it. - < hacklu> but now I face a new problem, when I typed the first continue command, gdb will continue all the breakpoint, and the inferior will run until normally exit. diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn index a9176f51..87250c62 100644 --- a/community/gsoc/2013/nlightnfotis.mdwn +++ b/community/gsoc/2013/nlightnfotis.mdwn @@ -429,25 +429,6 @@ License|/fdl]]."]]"""]] <tschwinge> nlightnfotis: But before doing that, please do the diff first, so that we know (hopefully) where the erroneous build results were coming from. - <nlightnfotis> considering the Copyright assignment files, I have sent them - from day 1 (that is the 20th of June). I have not heard anything about - those documents to date (sadly) - <nlightnfotis> what's worst is that although I have a reference number to - track those documents, their (greek postal office) tracking service sucks - so badly, that one day it's offline, the next it suggests it can't find - the object in their database, the next it says it is still in the local - post office - <nlightnfotis> let me check it out now - <nlightnfotis> still nothing from their online service - <nlightnfotis> let me call them - <nlightnfotis> tschwinge: I called the post office regarding the copyright - papers. They told me that the same day (the 20th of June) it left from - Herakleion, Crete to Athens and the same day it must have left the - country heading towards the US. They also told me it takes about 1 week - for it to arrive. - <tschwinge> nlightnfotis: OK, so probably waiting at the FSF office to be - processed. Let's allow for some more time. After all, this is not - critical for your progress. # IRC, freenode, #hurd, 2013-07-10 @@ -826,25 +807,6 @@ License|/fdl]]."]]"""]] worker threads are actually never destroyed on debian (because of a debian specific patch) - <teythoon> youpi, nlightnfotis, hacklu_: btw, what about the copyright - assignment process - <tschwinge> nlightnfotis just got his on file, so there is progress. - <tschwinge> I have email from Donald R Robertson III - <copyright-clerk@fsf.org> about that -- but it is not yet present in the - FSF copyright.list file... - <tschwinge> I think I received that email because I was CCed on - nlightnfotis' submission. - <nlightnfotis> tschwinge: I have got the papers, and they were signed by - the FSF. They stated delivery date 11 of July, but the documents were - signed on the 10th of July :P - <tschwinge> Ah, no, I received it via hurd-maintainers@gnu.org -- and the - strange thing is that not all assignments that got processed got sent - there... - <tschwinge> At the recent GNU Tools Cauldron we also discussed this in the - GCC context; and their experience was the very same. Emails get lost, - and/or take ages to be processed, etc. - <tschwinge> It seems the FSF is undermanned. - # IRC, freenode, #hurd, 2013-07-27 @@ -3035,3 +2997,18 @@ But not the [[open_issues/libpthread_dlopen]] issue? <nlightnfotis> and we wanna prove that go violates this rule right? That the stack pointer is not pointing at the initial stack <braunr> yes + + +# IRC, freenode, #hurd, 2013-10-09 + + <gnu_srs> braunr: The crash is not in the assembly code, but in the called + function from it: + <gnu_srs> pthread_sigmask (how=2, set=0xf9cac <server_block_set>, + oset=oset@entry=0x0) at ./pthread/pt-sigmask.c:29 + <gnu_srs> 29 struct __pthread *self = _pthread_self (); + <gnu_srs> Program received signal SIGSEGV, Segmentation fault. + <braunr> gnu_srs: ok so, same problem as in gcc go + <braunr> changing the stack pointer prevents libpthread from correctly + fetching thread-specific data (including _pthread_self()) correctly + <braunr> this will be fixed when threadvards are finally replaced with true + tls diff --git a/community/gsoc/project_ideas/object_lookups.mdwn b/community/gsoc/project_ideas/object_lookups.mdwn index 88ffc633..ca586dea 100644 --- a/community/gsoc/project_ideas/object_lookups.mdwn +++ b/community/gsoc/project_ideas/object_lookups.mdwn @@ -69,3 +69,66 @@ In context of [[!message-id "20130918081345.GA13789@dalaran.sceen.net"]]. <neal> braunr: That's called protected payload. <neal> braunr: The idea is that the kernel appends data to the message in flight. + + +## IRC, freenode, #hurd, 2013-10-24 + + <teythoon> and, with some effort, getting rid of the hash table lookup by + letting the kernel provide the address of the object (iirc neil knew the + proper term for that) + <braunr> teythoon: that is a big interface change + <teythoon> how so + <braunr> optimizing libihash and libpthread should already be a good start + <braunr> well how do you intend to add this information ? + <braunr> ok, "big" is overstatement, but still, it's a low level interface + change that would probably break a lot of things + <teythoon> store a pointer in the port structure in gnumach, make that + accessible somehow + <braunr> yes but how ? + <teythoon> interesting question indeed + <braunr> my plan for x15 is to make this "label" part of received messages + <braunr> which means you need to change the format of messages + <braunr> that is what i call a big change + <teythoon> ok, so we need to provide an update path + <teythoon> but once done, the change to hurd will be minimal, patching + libports should cover most of that + <braunr> normally yes + <teythoon> so this amounts to messing with gnumach and mig and designing a + clever way to make the update process safe + + <braunr> libihash is known to show high collision rates + <teythoon> right, libihash + <teythoon> it could use an integer hash function on the keys to distribute + them better + <braunr> i think that's already what it tries to do + <braunr> so merely using a better hash algorithm such as murmur should do + the job + <braunr> or use another data structure altogether + <teythoon> no, it does no hashing of its own on the keys + <braunr> are you sure ? + <teythoon> well, it uses only prime numbers as sizes, and computes key % + size + <braunr> well that's hashing .. :) + <teythoon> but this is not really a good hash + <braunr> yes + <braunr> isn't that what i said ? + <teythoon> right + <teythoon> ok, I didn't get that ;) + <teythoon> also, the sizes start quite small, 3, 7, 19... + <teythoon> and each time the hash table is grown, all items will have to be + updated + <braunr> which is why we could consider another data structure + <teythoon> or, for starters, to thin out that list of sizes + <braunr> my personal preference being radix trees + <teythoon> I assume you have an implementation handy? + <braunr> yes + <teythoon> cool :D + <braunr> but good hashing is excellent too + <braunr> radix trees have their own issues + <teythoon> braunr: http://burtleburtle.net/bob/hash/integer.html + <braunr> i use thomas wang's hashing function in x15 + <braunr> or rather, my own personal c utility library, since x15 doesn't + hash anything currently + <braunr> but murmur is better + <braunr> we prefer distribution over hashing performances + <braunr> https://131002.net/siphash/ diff --git a/community/meetings.mdwn b/community/meetings.mdwn index 0c680ddb..1d8199ad 100644 --- a/community/meetings.mdwn +++ b/community/meetings.mdwn @@ -13,16 +13,13 @@ License|/fdl]]."]]"""]] # Upcoming - -## In the Future - - * [[FOSDEM_2013]] - * [[Self-organised]] + * [[FOSDEM_2014]] # Past * [[GNU Hackers Meeting, 2013, Paris|ghm2013]] + * [[FOSDEM_2013]] * [[GNU Hackers Meeting, 2012, Düsseldorf|ghm2012]] * [[FOSDEM_2012]] * [[FrOSCon_2011]] @@ -39,4 +36,5 @@ License|/fdl]]."]]"""]] * [[FOSDEM_2006]] * [[RMLL_2005]] * [[FOSDEM_2005]] + * [[OSDI_1994]] * ... diff --git a/community/meetings/fosdem_2013.mdwn b/community/meetings/fosdem_2013.mdwn index 30860cb6..0ac36838 100644 --- a/community/meetings/fosdem_2013.mdwn +++ b/community/meetings/fosdem_2013.mdwn @@ -12,7 +12,7 @@ License|/fdl]]."]]"""]] <http://fosdem.org/2013> -FOSDEM will take place on February 2th/3th at the Université Libre de +FOSDEM will take place on February 2nd/3rd at the Université Libre de Bruxelles. diff --git a/open_issues/gdb_qemu_debugging_gnumach.mdwn b/community/meetings/fosdem_2014.mdwn index d3105f50..1379968e 100644 --- a/open_issues/gdb_qemu_debugging_gnumach.mdwn +++ b/community/meetings/fosdem_2014.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,12 +8,22 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_gdb open_issue_gnumach]] +[[!meta title="FOSDEM 2014"]] -\#hurd, freenode, June (?) 2010 +<http://fosdem.org/2014> - <jkoenig> is there a way to get gdb to map addresses as required when debugging mach with qemu ? - <jkoenig> I can examine the data if I manually map the addresses th 0xc0000000 but maybe there's an easier way... - <youpi> jkoenig: I haven't found a way - <youpi> I'm mostly using the internal kdb +FOSDEM will take place on February 1st/2nd at the Université Libre de +Bruxelles. + +# Who and When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return" +"""]] + + +# Devroom: Microkernel and component-based operating systems + +[[!message-id desc="Announcement and CfP" +"5255453A.70302@os.inf.tu-dresden.de"]]. diff --git a/community/meetings/osdi_1994.mdwn b/community/meetings/osdi_1994.mdwn new file mode 100644 index 00000000..f0c4b33c --- /dev/null +++ b/community/meetings/osdi_1994.mdwn @@ -0,0 +1,26 @@ +[[!meta copyright="Copyright © 2013 Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="First Symposium on Operating Systems Design and Implementation +(OSDI '94)"]] + +<http://www.cs.utah.edu/~lepreau/osdi94/> + + * {{$bushnell_hurd}} + + +[[!ymlfront data=""" + +bushnell_hurd: + + "tutorial by Michael I. Bushnell: [*The Architecture of the GNU Hurd*](http://www.cs.utah.edu/~lepreau/osdi94/tutorials/tutorials.html#HURD) ([slides](http://www.cs.utah.edu/~lepreau/osdi94/tutorials/hurdslides.ps.gz))" + +"""]] + diff --git a/community/meetings/self-organised.mdwn b/community/meetings/self-organised.mdwn deleted file mode 100644 index 1403c115..00000000 --- a/community/meetings/self-organised.mdwn +++ /dev/null @@ -1,54 +0,0 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] - -[[!meta title="Self-organised meeting"]] - -This meeting will be held in a to-be-determined place, at a to-be-determined time. This page hopes to help finding a good time and place, and finding out who wants to come. - -# Who wants to come? - -Please add yourself here. - -* [[Bas_Wijnen|baswijnen]] (no preference for specific times) -* [[Thomas_Schwinge|tschwinge]] -* [[Tom_Bachmann|tombachmann]] (weekend in the middle of germany would be preferred) -* [[Gianluca_Guida|GianlucaGuida]] (wherever, whenever) -* [[Samuel_Thibault|SamuelThibault]] (wherever, whenever) - -# Who will come? - -* (to be filled in when the date is set) - -# When is a good time? - -Please add any suggestions here, and add to your name above if that time is good for you. - -# Where is a good place? - -## Somewhere in Germany - -This likely has the benefit of being relatively close to most people - -<http://www.linuxhotel.de/community.html> might be a suitable venue at very -reasonable pricing. - -## Somewhere in Italy - -This likely has the benefit of better weather. ;-) - -## Venice (Italy) - -This certainly has the benefit of being in an awesome place. :-) Perhaps we shouldn't care too much about that, since we're mostly busy with ourselves anyway. Or perhaps we should: beauty helps creativity (wow, I should use this as my next catch-phrase to convince a girl to stay with me: I will fail again, but with style! Gianluca). - -# What will we do? - -There will be talks with discussions: - -* Bas will give a talk about Capability-microkernel-based operating systems, with an emphasis on how this can be useful for the Hurd. The talk hopes to get people enthousiastic for the concept, and it will be tried to keep it interesting for people who are not yet familiar with the concepts. diff --git a/contributing.mdwn b/contributing.mdwn index 4e9512e9..2e461819 100644 --- a/contributing.mdwn +++ b/contributing.mdwn @@ -123,13 +123,13 @@ For guidelines, please have a look at the dedicated [[porting_page|hurd/porting] ### Debian GNU/Hurd -[[!template id=note text="""#### Goal: Debian Wheezy Release Canditate - +Along with the official Debian "wheezy" release (but not as an official Debian +release), in May 2013 the Debian GNU/Hurd team released [[Debian GNU/Hurd +2013|news/2013-05-debian_gnu_hurd_2013]]. There is a goal of getting Debian GNU/Hurd into shape for a technology -preview/release candidate with Debian Wheezy (expected towards the end of 2012 -or beginning of 2013). +preview for integration as a proper Debian release candidate. -The *to do* list is on <http://wiki.debian.org/Debian_GNU/Hurd>."""]] +The *to do* list is on <http://wiki.debian.org/Debian_GNU/Hurd>. The following missing packages/missing functionality block a lot of other packages, and are thus good candidates for porting, in order to increase diff --git a/faq/how_many_developers.mdwn b/faq/how_many_developers.mdwn index da457d96..9e079852 100644 --- a/faq/how_many_developers.mdwn +++ b/faq/how_many_developers.mdwn @@ -67,3 +67,10 @@ We're an open project: any interested party (*you*!) are very welcome to start Likewise, for reaching out to new developers, we're participating in [[Google's Summer of Code program|community/gsoc]]. + +As *el_presidente* commented on +<http://lwn.net/Articles/568745/#CommAnchor568780>: + +> Developers, developers, developers, developers. +> +> They are the people that matter at this point in time. diff --git a/faq/old_faq.txt b/faq/old_faq.txt index 98c605eb..e370277e 100644 --- a/faq/old_faq.txt +++ b/faq/old_faq.txt @@ -18,8 +18,8 @@ users when they are talking about, installing, using, compiling and developing the GNU Hurd as well as its binary distribution Debian GNU/Hurd. Be sure to read this before asking for help. -The GNU Hurd is under active development and a stable version has not -yet been released. Be aware that: there is a lot of work yet to be +The GNU Hurd is under active development. +Be aware that there is a lot of work yet to be completed; you will find bugs; your system will crash. That said, there is a lot of room for contributions at all levels: development of the Hurd and Mach proper, porting applications, writing documentation and, @@ -406,7 +406,7 @@ at: ?? What is OSKit-Mach? -{NHW,FH} There are two versions of GNU Mac that are in use: GNU Mach +{NHW,FH} There have once been two versions of GNU Mach developed: GNU Mach 1.x and GNU Mach 2.x, formerly known as OSKit-Mach. The former uses the drivers from Linux 2.0.x while the latter uses the University of Utah's OSKit library for drivers. You can find out more about the @@ -414,9 +414,7 @@ OSKit library at: http://www.cs.utah.edu/flux/oskit -GNU Mach 2.x is usable, but some things are still missing or not -working, like the serial port and scsi drivers. This is why GNU Mach -2.0 hasn't released yet and the two versions coexist. +The project didn't work out too well, so development has again been halted. ?? Where is the documentation? diff --git a/faq/release.mdwn b/faq/release.mdwn index e782a9fb..6ec26925 100644 --- a/faq/release.mdwn +++ b/faq/release.mdwn @@ -12,6 +12,12 @@ License|/fdl]]."]]"""]] [[!meta title="When will the Hurd be released?"]] -Next year. +0.5 has been [[released on 2013-09-27|news/2013-09-27]]. Read about the +[[Hurd's status|hurd/status]]. -Save that, read about the Hurd's [[hurd/status]]. +> OK, but when will it be finished? + +Well, is the Linux kernel considered to be really "finished"? Hurd 0.5 does +work, but of course it can still become better -- beginning to +[[contribute|contributing]] and [[joining us|how_many_developers]] is the best +way for you to help achieve that. :-) diff --git a/faq/sata_disk_drives/discussion.mdwn b/faq/sata_disk_drives/discussion.mdwn index e9da8560..d05566b6 100644 --- a/faq/sata_disk_drives/discussion.mdwn +++ b/faq/sata_disk_drives/discussion.mdwn @@ -238,3 +238,21 @@ License|/fdl]]."]]"""]] <braunr> i'll stick with ide for now, but at least setting sata with libvirt was quite easy to do <braunr> so we can easily switch later + + +## IRC, freenode, #hurd, 2013-10-22 + + <teythoon> youpi: do I need to do anything to enable the ahci driver? + gnumach 1.4 should include it, right? + <youpi> it should, yes + <youpi> make sure to put your board in ahci mode, not raid mode + <youpi> (and not ata mode) + <teythoon> youpi: hm, I will try to do so + <teythoon> youpi: does the driver print anything to the console? + <youpi> teythoon: yes, AHCI SATA 00:04.0 BAR 0xfebf1000 IRQ 11 + <teythoon> youpi: well, the bios has two modes of operation, 'raid' and + 'ide', I selected 'ide' + <youpi> ergl + <teythoon> youpi: hm, I think my board has no ahci controller, linux uses + the sata_via module to talk to it :/ + <youpi> ah :/ diff --git a/glibc/process.mdwn b/glibc/process.mdwn index ded2e1f7..c8a1ce79 100644 --- a/glibc/process.mdwn +++ b/glibc/process.mdwn @@ -13,7 +13,7 @@ The GNU Hurd uses a similar concept to [[UNIX processes|unix/process]]. As a [[Mach task|microkernel/mach/task]] only implements a part of a UNIX process, there is additional work to be done, for example for [[signal]]s, -[[environment_variable]]s, [[file_descriptor]]s. +[[environment_variable]]s, [[file_descriptor]]s, [[hurd/authentication]]. # Startup diff --git a/history.mdwn b/history.mdwn index c2add410..7cf5cd6e 100644 --- a/history.mdwn +++ b/history.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!tag stable_URL]] +[[!toc]] + Richard Stallman (RMS) started GNU in 1983, as a project to create a complete free operating system. In the text of the [GNU Manifesto](http://www.gnu.org/gnu/manifesto.html), he @@ -56,7 +58,6 @@ Linux|hurd-and-linux]], where he mentions that the FSF started developing the Hurd in 1990. As of [Gnusletter, Nov. 1991], the Hurd (running on Mach) is GNU's official kernel. ---- # Announcements @@ -64,41 +65,62 @@ These are all the announcements made over the years. Most of them were either sent to the <A HREF="news:gnu.announce">gnu.announce</A> news group or Hurd interest mailing lists. - * [[hurd-flash15]] -- Release 0.2 announcement (complete GNU system) - * [[hurd-flash14]] -- Release 0.2 announcement (Hurd) - * [[hurd-flash13]] -- Test release announcement (Aug 96) - * [[hurd-flash12]] -- Test release status (Jul 96) + * [[hurd-announce]] -- GNU Hurd announcement, May 91 + * [[hurd-announce2]] -- GNU Hurd announcement, Nov 93 + * [[hurd-flash]] -- News flash, Apr 94 -- it boots! + * [[hurd-flash2]] -- News flash, May 94 + * [[hurd-flash3]] -- News flash, Jul 94 -- emacs runs! + * [[hurd-flash4]] -- News flash, Aug 94 + * [[hurd-flash5]] -- News flash, Sep 94 -- gcc runs! + * [[hurd-flash6]] -- News flash, Nov 94 + * [[hurd-flash7]] -- New Snapshot, Apr 95 + * [[hurd-flash8]] -- New Snapshot, Jul 95 -- ext2fs support + * [[hurd-flash9]] -- News Flash, Nov 95 -- ftp works! + * [[hurd-flash10]] -- New Snapshot, Apr 96 -- NFS and lots else works! * [[hurd-flash11]] -- Binary image available, Apr 96 This and [NetBSD](http://www.netbsd.org/) boot flopies should be enough to get a working GNU/Hurd system! - * [[hurd-flash10]] -- New Snapshot, Apr 96 -- NFS and lots else works! - * [[hurd-flash9]] -- News Flash, Nov 95 -- ftp works! - * [[hurd-flash8]] -- New Snapshot, Jul 95 -- ext2fs support - * [[hurd-flash7]] -- New Snapshot, Apr 95 - * [[hurd-flash6]] -- News flash, Nov 94 - * [[hurd-flash5]] -- News flash, Sep 94 -- gcc runs! - * [[hurd-flash4]] -- News flash, Aug 94 - * [[hurd-flash3]] -- News flash, Jul 94 -- emacs runs! - * [[hurd-flash2]] -- News flash, May 94 - * [[hurd-flash]] -- News flash, Apr 94 -- it boots! - * [[hurd-announce2]] -- GNU Hurd announcement, Nov 93 - * [[hurd-announce]] -- GNU Hurd announcement, May 91 - ---- - - * [History - 1997-2003](http://lists.gnu.org/archive/html/l4-hurd/2005-10/msg00718.html) - -- personal view of Marcus Brinkmann about Hurd development in 1997-2003. - ---- - - * [[Port_to_another_microkernel]] + * [[hurd-flash12]] -- Test release status (Jul 96) + * [[hurd-flash13]] -- Test release announcement (Aug 96) + * [[!message-id desc="GNU Hurd 0.1" + "199609070249.WAA24297@geech.gnu.ai.mit.edu"]] (1996-09-06) + * [[!message-id desc="GNU Mach 1.0" + "199704142043.QAA01894@churchy.gnu.ai.mit.edu"]] (1997-04-14) + * [[!message-id desc="GNU Mach 1.1" + "199705091753.NAA15557@sugar-bombs.gnu.ai.mit.edu"]] (1997-05-09) + * [[!message-id desc="GNU Mach 1.1.1" + "199705121633.MAA26409@sugar-bombs.gnu.ai.mit.edu"]] (1997-05-12) + * [[!message-id desc="GNU Mach 1.1.2" + "199706102159.RAA13943@churchy.gnu.ai.mit.edu"]] (1997-06-10?) + * [[hurd-flash14]] -- Release 0.2 announcement (Hurd) + * [[hurd-flash15]] -- Release 0.2 announcement (complete GNU system) + * [[!message-id desc="GNU 0.2" + "199706162021.QAA18370@sugar-bombs.gnu.ai.mit.edu"]] (1997-06-16) + * [[!message-id desc="GNU MIG 1.0.1" + "199812040849.DAA07660@baalperazim.frob.com"]] (1998-12-04) + * [[!message-id desc="GNU Mach 1.2" + "199906211918.PAA26822@x15-cruise-basselope.mit.edu"]] (1999-06-21) + * [[!message-id desc="GNU MIG 1.1" "199906231741.NAA46739@pusey.mit.edu"]] + (1999-06-23) + * [[!message-id desc="GNU MIG 1.2" "20010622215446.I6130@212.23.136.22"]] + (2001-06-22) + * [[!message-id desc="GNU MIG 1.3" + "20020308220603.A90E61B9C4@perdition.linnaean.org"]] (2002-03-08) + * [[!message-id desc="GNU Mach 1.3" + "20020527203155.88DBE1BA15@perdition.linnaean.org"]] (2002-05-27) + * [[!message-id desc="GNU MIG 1.3.1" + "20020830194413.GA6819@outpost.dnsalias.org"]] (2002-08-30) + * [[GNU Hurd 0.5, GNU Mach 1.4, GNU MIG 1.4 released|news/2013-09-27]] + (2013-09-27) ---- # An Incomplete GNU Hurd Timeline * 1997: GNU Hurd 0.2. + * First attempts to [[port_to_another_microkernel]] + * Personal view of Marcus Brinkmann about [[!message-id desc="Hurd + development in 1998-2003" + "87mzkvx7u8.wl%marcus.brinkmann@ruhr-uni-bochum.de"]] * 2002: GNU MIG 1.3, libio-based glibc, GNU Mach 1.3, Hurd L4 starts, work on the transition from cthreads to pthreads starts, Hurd installation party in Heidelberg, Toronto Hurd User Group meeting, Presentation at EpX in Paris diff --git a/hurd/authentication.mdwn b/hurd/authentication.mdwn index 2d6084bf..09dfb821 100644 --- a/hurd/authentication.mdwn +++ b/hurd/authentication.mdwn @@ -1,18 +1,22 @@ -[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -UIDs on the Hurd are separate from processes. A process has +UIDs on the Hurd are separate from [[glibc/process]]es. A process has [[capabilities|capability]] designating so-called UID vectors that are implemented by an [[translator/auth]] server. This makes them easily [[virtualizable|virtualization]]. +The standard POSIX interfaces to a [[glibc/process]]' UID management are +implemented in [[glibc]]. + When a process wishes to gain access to a resource provided by a third party (e.g., a file system) and that party wishes to authenticate the client so as to implement some identity-based access control ([[IBAC]]) policy, @@ -25,3 +29,237 @@ naming a newly authenticated session with the server and the server is delivered the client's designated UID vector. For more details, see section 2.3 of the [[critique]]. + + +# Open Issues + +[[!tag open_issue_hurd]] + + +## IRC, freenode, #hurd, 2013-09-26 + + <braunr> teythoon: hum, i just saw something disturbing + <braunr> teythoon: to isolate the leak, i created my own proc directory + <braunr> and the mtab translators it spawns seem to be owned by root oO + <teythoon> braunr: but how is that possible? are you sure? have you checked + with 'ids'? + <braunr> no i'm not sure + <braunr> also, ext2fs seems to ignore --writable when started as a passive + translator + <braunr> < teythoon> braunr: but how is that possible? + <braunr> messup with passive translators i guess + <braunr> teythoon: actually, it looks like it has effective/available id + <braunr> it has no* + <braunr> this feature doesn't map well in unix + <teythoon> braunr: ah yes, htop doesn't handle this well and shows root + indeed, our ps shows - as username + <braunr> yes + + +## IRC, freenode, #hurd, 2013-09-28 + + <braunr> mhmm + <braunr> this process has no uid + <braunr> isn't it a security issue that processes can remove their identity + ? + <braunr> i really don't like that we allow processes to loose their + identity ... + <teythoon> braunr: y not? I think that's a killer feature + <teythoon> one that is notoriously absent in unices + <braunr> not exactly + <braunr> gaining rights to switch your identity is ok + <braunr> since you have proven that you are allowed to do it + <braunr> now, if you can remove your identity, you can create "ghost" + processes + <braunr> processes that can spend their day causing denial of services + without the possibility for the administrator to know who is responsible + <braunr> the unix "way" of dealing with DoS is to warn and ban users after + they violated the rules + <braunr> we need to have at least that possibility + <youpi> perhaps we need to add an "initial" uid + <teythoon> otoh the unix way of dropping privileges is hardly being able to + do so at all ;) + <braunr> teythoon: ? + <braunr> on unix, you need privileges to drop your identity :) + <braunr> i understand it involves security risks, but that's understandable + <braunr> the thing is, we actually don't care about dropping privileges + <braunr> we care about gaining them + <teythoon> you cannot drop your identity, you can just use another one + <braunr> exactly + <braunr> that's what i want + <braunr> and the way the hurd does it is superior + <braunr> let's keep that + <braunr> processes that should run with least privileges can simply have + their own user/group as it's done on unix + <teythoon> then how do you obtain such a uid/gid? + <braunr> teythoon: you gain the right, use it to prove who you can be, and + ask an identity switch + <braunr> identities would then be managed at server side (in proc for + example) + <teythoon> I know how it's done on the Hurd, but who creates them for you? + <braunr> the password server + <braunr> well no + <braunr> the password server gives you the right you need to prove who you + can be + <braunr> then i'd assume you'd ask the proc server for the switch + <teythoon> but who creates the uid for you in the first place, who sets up + a passwd entry + <braunr> the administrator ? + <braunr> what bothers me is that it goes directly against the main goal of + the hurd + <teythoon> indeed + <braunr> but i think it's a better compromise of freedom/order + <teythoon> I always thought that the ability to drop the unix-like + credentials is really nice + + +### IRC, freenode, #hurd, 2013-09-29 + + <antrik> braunr: dropping privileges doesn't unregister a process from the + proc server... so it shouldn't be too hard to find out who is + responsible. however, there are more effective ways to create ghost + processes -- a special executable skipping the ordinary bootstrap won't + be registered with proc at all... + <antrik> however, that would be a non-issue if we had proper resource + accounting + <teythoon> antrik: I do not believe that this is correct. every mach task + will eventually be picked up by the proc server + <teythoon> eventually being next time someone fork(2)s or so + <teythoon> but if noone uses proc_child to claim the new process ones + child, it will not be presented by the proc server as unix process (aiui) + <antrik> teythoon: not sure what you mean by "pick up" + <antrik> of course proc will see the process when listing all tasks on the + system; but it will have no additional knowlegde about it + <antrik> (which is the whole purpose of proc) + + +### IRC, freenode, #hurd, 2013-09-30 + + <braunr> antrik: proc should be redesigned to fix these issues + <braunr> in particular, the way that proc lists mach tasks to show them to + the rest of the system is something i find deeply disturbing + <braunr> hurd processes should be forced to go through proc + <antrik> braunr: well, if we are talking about fundamentally fixing things, + I believe the central proc server is not a good idea in the first place + :-) + <antrik> I believe hierarchical management of resource management and + information flow -- cf. nghurd and genode -- is a much better approach + <braunr> antrik: i agree with hierarchical management of resources, but i + don't see why this prevents a central proc server + <braunr> i.e. one proc server per hurd instance + + +### IRC, freenode, #hurd, 2013-10-06 + + <antrik> braunr: hierarchical management of resources doesn't prevent a + central proc server of course; but a central proc server would fit really + ill with hierarchical management of permissions... + + +### IRC, freenode, #hurd, 2013-10-07 + + <braunr> antrik: does proc manage permissions ? + <antrik> braunr: well, it manages some permissions... like who is allowed + to send signals + <antrik> hm... or perhaps proc is not involved in signal delivery itself? + don't remember. but at any rate, proc decides whether it hands out + privileged task ports + <braunr> antrik: yes, it decides whether or not a client is allowed to + obtain the message port of another task + <braunr> antrik: but i don't see why this is a problem + <braunr> what we have now is one proc server per hurd instance + <braunr> how is that not both central (inside the hurd instance) and + hierarchical with regard to resource management ? + <antrik> braunr: we are probably talking past each other + <antrik> all I'm saying is that in an ideal world, there should not be a + central server deciding who is allowed to see and/or control other + processes + <braunr> antrik: how should it be structured then ? + <braunr> i mean, how would you see it ? + <antrik> child processes should be fully controlled by their parent -- + including outside communication + <antrik> (like in genode AIUI) + <braunr> isn't that conflicting with the unix design ? + <kilobug> antrik: maybe I'm saying silly stuff since I don't have all the + background, but seems problematic to me with SUID/SGID programs + <kilobug> antrik: in which a child can be more privilegied than the parent + <braunr> kilobug: that's part of my question too + <kilobug> and it's even "worse" with Hurd's addauth in which any process + can be given additional rights in runtime, but not its parent + <antrik> braunr: one of the conclusions from ngHurd was that suid as such + is problematic. it makes more sense to have privileged services actually + run by the privileged user, and only requested by the unprivileged one + using RPC + <antrik> admittedly, I'm not sure how this interacts with UNIX + compatibility ;-) + <antrik> kilobug: in the genode approach, the parent would control that as + well + <braunr> in unix, the idea of parent processes doesn't imply much + <braunr> parents merely exist to reap resources from their children + <braunr> and as templates when forking + <antrik> yeah, but that's one of the major shortcomings of UNIX in my + book... + <braunr> sure + <braunr> i'm just thinking out loud + <braunr> if we want to maintain posix compatibility, it seems reasonable to + keep it that way + <braunr> despite the shortcomings + <braunr> does that imply a centralized proc server anyway ? + <antrik> I suspect we could similate suid in the hierarchical design, only + that the resources would be accounted to the user under whose ID the + process runs, rather than to the one who invoked it + <braunr> i also have a hard time seeing what the completely hierarchical + model brings compared to what the hurd does (isolating system instances) + <antrik> and I don't think we need a central proc server. it could probably + be done similar to the VFS: each process implements the interfaces, + passing on to the children as appropriate + <antrik> braunr: it's much easier to isolate parts of the system for + security and/or customisation + <antrik> that's actually one of the things discussed in the "critique" + IIRC... + <braunr> i'm not sure + <braunr> anyway, processes implementing the interface looks bad to me + <braunr> that's already a problem with the current hurd + <braunr> using normal client processes as servers means we rely on them to + behave nicely + <antrik> you have a point there: while untrusted filesystems can be ignored + easily, ignoring untrusted proc providers would be problematic... + <antrik> (I don't think it's strictly necessary to know about other user's + processes; but we could hardly keep a UNIX feel without it...) + <antrik> other users' + <braunr> i feel the hierarchical model may imply some unnecessary burden + <braunr> capabilities along with resource containers look much more + flexible + <braunr> and not less secure + <braunr> children would share the same container as their parent by + default, unless they obtain the right to use another or create their own + <antrik> braunr: decoupling control from resource ownership is *always* + problematic. that's pretty much the major takeaway from ngHurd + discussions (and the major reason why Coyotos was abandoned as a base for + ngHurd) + <antrik> if a process runs on your resources, you should have full control + over it. anything else faciliates DRM & Co. + <braunr> antrik: i see + <braunr> nonetheless, i don't see the point of that restriction, since the + simplest way to avoid drms in the first place is not using them + <braunr> and that restriction makes posix compatibility hard to provide + <antrik> I'm not sure it's really all that hard... + <antrik> IIRC genode is aiming at POSIX compatibility + <antrik> I'm not sure it's any harder than with the current Hurd + architecture + <braunr> i didn't see anything like that + <braunr> they provide posix compatibility by running legacy systems on top + of it + <braunr> well, namely linux + <antrik> hm... they have some UNIX compatibility at least... perhaps not + aiming at full POSIX. don't remember the details + <antrik> Linux on genode? that's news to me... I know they do run genode on + Linux + <braunr> anyway, i'll probably stick with the close unix approach for x15 + <braunr> http://genode.org/documentation/general-overview/ + <braunr> In an preliminary study, a user-level version of the Linux kernel + (L4Linux) was successfully ported to the Genode OS Framework running on a + L4 kernel. This study suggests that Genode combined with an appropriate + kernel or hypervisor is suited as a virtual-machine hosting platform. + <antrik> hm... that's boring though ;-) + <braunr> isn't it :) diff --git a/hurd/console.mdwn b/hurd/console.mdwn index 55581870..10c74bf9 100644 --- a/hurd/console.mdwn +++ b/hurd/console.mdwn @@ -1,5 +1,5 @@ [[!meta copyright="Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2011, -2012 Free Software Foundation, Inc."]] +2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -37,8 +37,8 @@ where the numbered nodes represent virtual consoles and their contents are all alike. As the following graph shows, the console, input and display nodes are the -interfaces used by the terminal server, input driver and display drivers -respectively. +interfaces used by the [[terminal server|translator/term]], input driver and +display drivers respectively. +------------------+ +-----------------+ | Input driver | | Terminal Server | @@ -67,7 +67,8 @@ respectively. +----------------+ +-----------------+ The input driver takes scancodes from the in-kernel kbd queue, translates them -into characters and writes them to the input node. Then the terminal server +into characters and writes them to the input node. Then the [[terminal +server|translator/term]] reads the console node taking the characters out of the console server. Each of theese actions is actually an RPC handled by the translator on @@ -110,7 +111,8 @@ Additional information about the console can be found in the [Hurd Console Tutor **_The new Hurd console features:_** -**A console server**, which provides a number of virtual consoles to term servers, with a full set of terminal capabilities. +**A console server**, which provides a number of virtual consoles to [[term +servers|translator/term]], with a full set of terminal capabilities. The console server supports any encoding supported by iconv, but uses Unicode internally. The default encoding is ISO8859-1, another useful variant is UTF-8. @@ -307,7 +309,13 @@ If you use mutt, install `mutt-utf8` package. For lynx, edit `/etc/lynx.cfg`, ma If you use other applications, try to search with google for "application-name utf8" or "application-name unicode". Often you find what you need. The issues are the same for the GNU/Hurd and GNU/Linux systems, so most of the information can be shared, except how to setup the system console to support Unicode, of course. -The `console-server` watches for new hurdio terms (devices translated with `/hurd/term`) and adds them to `/dev/vcs` automatically. What this means is, if you create a new tty with `MAKEDEV`, and then attach something to it, it will now appear in `/dev/vcs`. When a term is disconnected from, it disappears from `/dev/vcs`. `/libexec/getty` is what is usually attached to a term. You can see this automatic adding and removing of terms from the `console-server` by typing the following: +The `console-server` watches for new [[hurdio terms (devices translated with +`/hurd/term`)|translator/term]] and adds them to `/dev/vcs` automatically. +What this means is, if you create a new tty with `MAKEDEV`, and then attach +something to it, it will now appear in `/dev/vcs`. When a term is disconnected +from, it disappears from `/dev/vcs`. `/libexec/getty` is what is usually +attached to a term. You can see this automatic adding and removing of terms +from the `console-server` by typing the following: # cd /dev # ls vcs/ diff --git a/hurd/console/discussion.mdwn b/hurd/console/discussion.mdwn index 0022ec23..73d605ed 100644 --- a/hurd/console/discussion.mdwn +++ b/hurd/console/discussion.mdwn @@ -48,3 +48,50 @@ License|/fdl]]."]]"""]] <pinotree> http://xkbcommon.org/ ¡û sounds interesting for our console translator + + +# IRC, freenode, #hurd, 2013-10-01 + +[[!tag open_issue_hurd]] + + <pinotree> teythoon_: df: `/dev/cons': Operation not supported + <pinotree> missing/stub implementation in the console translator? + + +## IRC, freenode, #hurd, 2013-10-02 + + <teythoon_> pinotree: yes, df does file_statfs which fails + + +# IRC, freenode, #hurd, 2013-10-22 + + <C-Keen> hello hurders! I happened to watch samuel's gnu hackers talk and + wanted to start to use the hurd more regularily. However I noticed that + when I use the preinstalled image, there seems to be some issue with the + console driver + <C-Keen> when I start emacs the mode line is drawn 3 times above the bottom + of the screen + <C-Keen> is this know or did I miss a step in setting it up? Or should I + use the debian installer and start from scratch again? + <youpi> C-Keen: it's probably unknown, and not an issue on your side. Did + you try to upgrade to the latest packages? + <C-Keen> youpi: doing that now + <C-Keen> my base image is debian-hurd-20130504.img + <youpi> still an issue with the latest packages indeed + <youpi> it seems emacs and the hurd console don't agree on the number of + lines... + <youpi> C-Keen: you can set TERM=vt100 to work around the issue + <C-Keen> ah alright. + <youpi> or TERM=linux + <C-Keen> youpi: can you start the emacs in X? I get an empty window here + <youpi> I never tried + <youpi> I never use emacs :) + <C-Keen> I see ;) + <youpi> it seems there's a bug in cud1 indeed + <C-Keen> what's cud1? + <youpi> see man 5 terminfo + <braunr> yes it's a terminfo problem + <braunr> the hurd console isn't well defined there + <youpi> braunr: actually it seems like a bug in emacs + <youpi> cud may or may not scroll the screen, depending on the + implementation diff --git a/hurd/libfuse.mdwn b/hurd/libfuse.mdwn index 78e96022..28125dd9 100644 --- a/hurd/libfuse.mdwn +++ b/hurd/libfuse.mdwn @@ -49,6 +49,25 @@ etc. <braunr> and they could almost readily use our libfuse version +## IRC, freenode, #hurd, 2013-10-01 + + <pinotree> our libfuse implementation is still basic atm (there's a wiki + page about it) + <alsuren> okay... talk to me about FUSE + <pinotree> even with the improvements i have in my public branch, it still + cannot do real-world fs'es + <alsuren> okay, so you're the person to ask about FUSE + <alsuren> it strikes me that HURD not having FUSE support is a bit of an + architectural oversight + <pinotree> i'm not sure what's your point about fuse, since what fuse on + linux (and not only) does is done *natively* by the hurd + <alsuren> exactly + <pinotree> all of the hurd filesystems (which are just a type of servers) + run in userspace already + <alsuren> so FUSE should Just Work + <pinotree> well no + + # Source [[source_repositories/incubator]], libfuse/master. diff --git a/hurd/porting/guidelines.mdwn b/hurd/porting/guidelines.mdwn index d28a777e..a9acd9f9 100644 --- a/hurd/porting/guidelines.mdwn +++ b/hurd/porting/guidelines.mdwn @@ -1,5 +1,5 @@ [[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2011, -2012 Free Software Foundation, Inc."]] +2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -83,6 +83,24 @@ because else *-gnu* would catch i386-pc-linux-gnu for instance... Note: some of such statements are not from the source package itself, but from aclocal.m4 which is actually from libtool. In such case, the package simply needs to be re-libtoolize-d. + +## Preprocessor Define + +### IRC, freenode, #hurd, 2013-10-23 + + <C-Keen> Is there a preprocessor define gcc sets for hurd which I can check + in my code? + <braunr> __GNU__ + <braunr> glibc sets it if i'm right + <C-Keen> I also see that __MACH__ gets set + <azeem> that's also set on Mac OS X + <C-Keen> right, which uncovered a bug in the code + <braunr> the microkernel doesn't always implies what operating system runs + on top of it + <C-Keen> braunr: but __GNU__ is the correct define for hurd specific code? + <braunr> yes + + ## <a name="Undefined_bits_confname_h_tt_mac"> Undefined `bits/confname.h` macros (`PIPE_BUF`, ...) </a> If macro `XXX` is undefined, but macro `_SC_XXX` or `_PC_XXX` is defined in `bits/confname.h`, you probably need to use `sysconf`, `pathconf` or `fpathconf` to obtain it dynamicaly. diff --git a/hurd/running/virtualbox.mdwn b/hurd/running/virtualbox.mdwn index 822f771d..23a0b156 100644 --- a/hurd/running/virtualbox.mdwn +++ b/hurd/running/virtualbox.mdwn @@ -40,7 +40,44 @@ To convert the image you need the VirtualBox package properly installed with a V If [[QEMU with KVM|qemu]] is not available, VirtualBox reportedly has better performance. -IRC, freenode, #hurd, 2011-10-31: + +# Open Issues + +## IRC, freenode, #hurd, 2011-10-31 <youpi> I don't know what virtualbox does with hardware emulation, but gnumach is awfully slow to probe things there + + +## IRC, freenode, #hurd, 2013-09-28 + + <snadge> the problem is if i giveit more than 1855 it says truncating to + that + <snadge> so i give it that.. then it has kmem alloc error + <snadge> 1536mb same.. 1024 isok + <braunr> hum + <braunr> that's weird + <braunr> virtual box ? + <snadge> yeah + <snadge> i wonder what cpu features i should enable/disable + <snadge> pae ? + <braunr> make sure vbox doesn't count on the so called memory balloon + <braunr> pae isn't used except on xen + <braunr> disable apic + <braunr> enable host io cache in disk controllers + <youpi> do we have these written on the wiki? + <braunr> no because i didn't run into these problems + <braunr> but since i know the system well enough to avoid them in the first + place .. + <braunr> we need real users to report them + <braunr> i'm not sure we have anything about vbox in the wiki actually + <youpi> ./hurd/running/virtualbox.mdwn + <youpi> we seem to have a page at least + <snadge> it seems to be okay with 1024MiB + <braunr> still weird + <braunr> looks more random than buggy with more memory + <braunr> do you have the exact error message you got during your previous + attempts ? + <snadge> no.. i should have taken a screenshot.. its easy enough to + reproduce though + <snadge> i'll wait until after its installed diff --git a/hurd/status.mdwn b/hurd/status.mdwn index cca8fc15..6aa319d1 100644 --- a/hurd/status.mdwn +++ b/hurd/status.mdwn @@ -38,15 +38,10 @@ Although the [[POSIX interface|faq/posix_compatibility]] is provided, some additional interfaces like POSIX shared memory or semaphores are still under development. -All this applies to the current development version, and not to the -last release (0.2). We encourage everybody who is interested to try +We encourage everybody who is interested to try out the current development version, and send feedback to the Hurd developers. -The Hurd team doesn't create Hurd-only releases, but instead relies on -the distributions done by folks from *Debian*, *Arch* (since 2010), and *Nix* -(since 2012). - [[!img hurd-iceweasel-screenshot-2012-03-21.png size=300x alt="Iceweasel running on GNU/Hurd" title="Iceweasel running on GNU/Hurd" @@ -63,17 +58,6 @@ GNU/Hurd 2013|news/2013-05-debian_gnu_hurd_2013]]. [[hurd/running/Nix]] provides QEMU images. -That said, the last official release of the Hurd -without the Debian parts was 0.2 done in 1997 ([[history]]). - -New official releases will be done, as soon as -the Hurd is sufficently stable and feature -complete. If an official release of the Hurd were made -now, people would try the Hurd and be disappointed. People -already expect delays; to disappoint them in this way as -well would be unfortunate. Moreover, it would lessen the -possibility that they would want to try the Hurd again in the future. - ## Usability Reports diff --git a/hurd/subhurd/discussion.mdwn b/hurd/subhurd/discussion.mdwn index fac93625..892387ef 100644 --- a/hurd/subhurd/discussion.mdwn +++ b/hurd/subhurd/discussion.mdwn @@ -180,3 +180,37 @@ License|/fdl]]."]]"""]] <braunr> safer <braunr> perhaps more powerful too, but that entirely depends on the features you want inside + + +# IRC, freenode, #hurd, 2013-10-04 + + <braunr> hm, looks like we broke subhurds again + <braunr> freezes after starting exec + <teythoon> o_O + <braunr> looks like some translator refuses to start + <braunr> teythoon: we need better error reporting first :) + +[[open_issues/subhurd_error_messages]]. + + <braunr> and better visibility in general + <braunr> teythoon: it may be that the subhurd i'm using is a bit od + <braunr> old + <braunr> one weird thing about subhurds is that they actually use the + ext2fs and linker from the host + <braunr> so it's better if the subhurd and the host uses the same bootstrap + protocol :) + <teythoon> braunr: isn't boot --boot-root=DIR there to specify which root + translator and linker to use? + <braunr> teythoon: yes but you don't want your root file system mounted + from the host when starting your subhurd + <teythoon> you can mount it r/o just fine, no? + <braunr> ideally, we'd have a userspace version of grub reading the files + from the disk, as it's done when booting + <braunr> hm + <braunr> right + + +## IRC, freenode, #hurd, 2013-10-07 + + <teythoon> braunr: btw, did you straighten out your subhurd issue? + <braunr> teythoon: no i haven't diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index 52cd09f7..32562a8b 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -106,6 +106,7 @@ The [[concept|concepts]] of translators creates its own problems, too: * [[symlink]] * [[firmlink]] * [[fifo]] +* [[term]] * ... diff --git a/hurd/translator/auth.mdwn b/hurd/translator/auth.mdwn index 7fd4832c..10cfb3aa 100644 --- a/hurd/translator/auth.mdwn +++ b/hurd/translator/auth.mdwn @@ -8,7 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -The *auth server* (or, *authentification server*). +The *auth server* (or, *authentification server*) is a key component managing +[[authentication]] in a Hurd system. It is stated by `/hurd/init`. diff --git a/hurd/translator/discussion.mdwn b/hurd/translator/discussion.mdwn index e038ba84..95f5ab0c 100644 --- a/hurd/translator/discussion.mdwn +++ b/hurd/translator/discussion.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_hurd]] -IRC, freenode, #hurd, 2011-08-25: + +# IRC, freenode, #hurd, 2011-08-25 < frhodes> how can I replace an existing running server with a new one without rebooting? @@ -23,3 +24,24 @@ IRC, freenode, #hurd, 2011-08-25: nature < antrik> in some cases, you might even be able simply to remove the old translator... but obviously only for non-critical stuff :-) + + +# IRC, freenode, #hurd, 2013-10-21 + + <braunr> mhmm, there is a problem with thread destruction + +[[open_issues/libpthread/t/fix_have_kernel_resources]]. + + <braunr> actually, translator self destruction + <braunr> if a request arrives after the last thread servicing a port set + returns from mach_msg because of a timeout, but before the translator is + detached from its parent, the client will get an error + <braunr> it should very rarely happen, but if it does, we could face the + same kind of issues we have when a server crashes + <braunr> e.g. sshd looping over select() returning EBADF, consuming all cpu + <braunr> not sure we want to introduce such new issues + + <braunr> i don't think i'll be able to make translators disappear reliably + .. + <braunr> but at least, thread consumption will correctly decrease with + inactivity diff --git a/hurd/translator/ext2fs.mdwn b/hurd/translator/ext2fs.mdwn index e2f6b044..cfd09502 100644 --- a/hurd/translator/ext2fs.mdwn +++ b/hurd/translator/ext2fs.mdwn @@ -163,6 +163,11 @@ small backend stores, like floppy devices. <youpi> ok +#### IRC, freenode, #hurd, 2013-10-08 + + <braunr> ogi: your ext2fs patches were finally merged upstream :) + + ## Sync Interval [[!tag open_issue_hurd]] @@ -209,39 +214,6 @@ That would be a nice improvement, but only after writeback throttling is impleme <teythoon> tschwinge: well, thanks anyway ;) -## Increased Memory Consumption - -### IRC, freenode, #hurd, 2013-09-18 - - <braunr> ext2fs is using a ginormous amount of memory on darnassus since i - last updated the hurd package :/ - <braunr> i wonder if my ext2fs large store patches rework have introduced a - regression - <braunr> the order of magnitude here is around 1.5G virtual space :/ - <braunr> it used to take up to 3 times less before that - <braunr> looks like my patches didn't make it into the latest hurd package - <braunr> teythoon: looks like there definitely is a new leak in ext2fs - <teythoon> :/ - <braunr> memory only - <braunr> the number of ports looks stable relative to file system usage - <teythoon> braunr: I tested my patches on my development machine, it's up - for 14 days (yay libvirt :) and never encountered problems like this - <braunr> i've been building glibc to reach that state - <teythoon> hm, that's a heavy load indeed - <teythoon> could be the file name tracking stuff, I tried to make sure that - everything is freed, but I might have missed something - <braunr> teythoon: simply running htop run shows a slight, regular increase - in physical memory usage in ext2fs - <pinotree> old procfs stikes again? :) - <teythoon> braunr: I see that as well... curious... - <braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried - to make sure that everything is freed, but I might have missed something - <braunr> how knows, maybe completely unrelated - <teythoon> the tracking patch isn't that big, I've gone over it twice today - and it still seems reasonable to me - <braunr> hm - - # Documentation * <http://e2fsprogs.sourceforge.net/ext2.html> diff --git a/hurd/translator/fifo.mdwn b/hurd/translator/fifo.mdwn index 857922fc..4132e94a 100644 --- a/hurd/translator/fifo.mdwn +++ b/hurd/translator/fifo.mdwn @@ -46,3 +46,9 @@ The *fifo* translator implements named pipes (FIFOs). <pochu> gg0: got an example? <gg0> http://bugs.debian.org/629184 <gg0> i didn't close it myself + + +## IRC, OFTC, #debian-hurd, 2013-10-04 + + <braunr> there is new-fifo, which you can try + <braunr> i guess none of us know what it was really meant for diff --git a/hurd/translator/magic.mdwn b/hurd/translator/magic.mdwn index 84bacdfb..2b0d1bf7 100644 --- a/hurd/translator/magic.mdwn +++ b/hurd/translator/magic.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2006, 2007, 2008, 2010 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2006, 2007, 2008, 2010, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -9,7 +9,13 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -The magic translator provides `/dev/fd`. +The `magic` translator returns magic retry results, which are then resolved by +[[glibc]]'s *name lookup* routines. + +[[!toc]] + + +# `/dev/fd`. $ showtrans /dev/fd /hurd/magic --directory fd @@ -20,3 +26,253 @@ individually like this: $ ls -l /dev/fd/0 crw--w---- 1 bing tty 0, 0 Nov 19 18:00 /dev/fd/0 + + +# `/dev/tty` + + $ showtrans /dev/tty + /hurd/magic tty + + +## Open Issues + +### IRC, OFTC, #debian-hurd, 2013-06-18 + + <XTaran> http://www.zsh.org/mla/workers/2013/msg00547.html + + +#### IRC, OFTC, #debian-hurd, 2013-06-19 + + <XTaran> youpi: http://www.zsh.org/mla/workers/2013/msg00548.html -- Is + that realistic? If yes, can someone of you test it? I though would expect + that if /dev/tty exists everywhere, it's a chardev everywhere, too. + <youpi> that's not impossible indeed + <youpi> I've noted it on my TODO list + + +#### IRC, OFTC, #debian-hurd, 2013-06-20 + + <pinotree> youpi: wrt the /dev/tty existance, + https://buildd.debian.org/status/fetch.php?pkg=mksh&arch=hurd-i386&ver=46-2&stamp=1371553966 + <pinotree> For the build logs, demonstrate that /dev/null and /dev/tty + exist: + <pinotree> ls: cannot access /dev/tty: No such device or address + <youpi> uh?! + <youpi> ah, ENODEV + <youpi> so that's what we was thinking, no tty -> no /dev/tty + + +#### IRC, OFTC, #debian-hurd, 2013-09-20 + + <XTaran> Hi. zsh still FTBFS on Hurd due to some test failure: + https://buildd.debian.org/status/package.php?p=zsh -- IIRC I checked last + time on some porterbox and couldn't reproduce the failure there. Any + insight if /dev/tty is not accessible on the buildds inside the chroot? + Or is it no character device there? I checked on strauss and there it is + a character device. + <XTaran> My only other option to debug this (didn't think of that yesterday + before the upload unfortunately) would be to override dh_auto_test with + "ls -l /dev/tty; dh_auto_test". Do you think that would be helpful? + <pinotree> i see /dev/tty on exodar, in the root system and in the chroot + <XTaran> pinotree: And it is a character device? + <XTaran> ... in both cases? + <pinotree> crw--w---- 1 pino tty 0, 0 Sep 20 10:20 /dev/tty + <pinotree> yes + <XTaran> pinotree: Hrm. + <pinotree> (/dev in the chroot is a firmlink to the system /dev, iirc) + <XTaran> pinotree: What is a firmlink? :) + <XTaran> pinotree: /dev/tty belongs to your user in the example above. + <pinotree> something between a (sym)link and an union mount + <XTaran> pinotree: Is it possible that /dev/tty is not visible if the + buildd runs without a connected terminal? + <pinotree> that i'm not sure + <XTaran> I see. + <pinotree> wouldn't it be possible to skip only that check, instead of the + whole test suite? + <pinotree> maybe something like + <pinotree> tty=$(find /dev/ -name 'tty*' -type c -print) + <pinotree> if [[ -n $tty ]]; then / [[ -c $tty[(f)1] && ! -c $zerolength ]] + / else / print -u$ZTST_fd 'Warning: Not testing [[ -c tty ]] (no tty + found)' / [[ ! -c $zerolength ]] / fi + <pinotree> (never used zsh, so please excuse me if i wrote something silly + above) + <XTaran> re + <XTaran> pinotree: Yeah, sure. That would be one way to get the thing + building again, if that's really the cause. + <pinotree> i guess it would find any of the available tty* devices + <pinotree> it does that for block devices, why not with tty devices, after + all? :) + <XTaran> pinotree: I just wonder if the failing test is because the test + doesn't work properly on that architecture or because it indicates that + there is a bug in zsh which only is present on hurd. + <pinotree> wouldn't the change proposed above help in determining it? + <XTaran> If I'm sure that it's a broken test, I'll try to disable that + one. If not I'd report (more details) to upstream. :) + <XTaran> pinotree: Oh, indeed. + <pinotree> if you get no warning, then a tty device was found with find + (using its -type c option), so the failing condition would be a zsh (or + maybe something in the stack below) bug + <pinotree> with the warning, somehow there were no tty devices available, + hence nothing to test -c with + <XTaran> So basically doing a check with dash to see if we should run the + zsh test. + <pinotree> dash? + <XTaran> Well, whatever /bin/sh points to. :) + <pinotree> ah, do you mean because of $(find ...)? + <XTaran> Ah, right, -type c is from find not /bin/sh + <XTaran> pinotree: That's my try: + http://anonscm.debian.org/gitweb/?p=collab-maint/zsh.git;a=commitdiff;h=ba5c7320d4876deb14dba60584fcdf5d5774e13b + <pinotree> o_O + <pinotree> isn't that a bit... overcomplicated? + <XTaran> pinotree: Yeah, it's a little bit more complicated as the tests + itself are not pure shell code but some format on their own. + <pinotree> why not the "thing" i wrote earlier? + <XTaran> pinotree: Actually it is what I understand you wanted to do, just + with more debug output. Or I dunderstood + <XTaran> pinotree: Actually it is what I understand you wanted to do, just + with more debug output. Or I understood your thing wrongly. + <pinotree> <pinotree> tty=$(find /dev/ -name 'tty*' -type c -print) + <pinotree> <pinotree> if [[ -n $tty ]]; then / [[ -c $tty[(f)1] && ! -c + $zerolength ]] / else / print -u$ZTST_fd 'Warning: Not testing [[ -c tty + ]] (no tty found)' / [[ ! -c $zerolength ]] / fi + <XTaran> pinotree: Yeah, I know. + <pinotree> that is, putting these lines instead of the current two + tty=/dev/tty + following + <pinotree> imho that should be fit for upstream + <XTaran> pinotree: You mean inside C02cond.ztst? + <pinotree> yep + <XTaran> pinotree: No, IMHO that's a bad idea. + <pinotree> why? + <XTaran> pinotree: That file is to test the freshly compiled zsh. I can't + rely on their code if I'm testing it. + <pinotree> uh? + <pinotree> the test above for -b is basically doing the same + <XTaran> pinotree: Indeed. Hrm. + <pinotree> that's where i did c&p most of it :) + <XTaran> So upstream relies on -n in the testsuite before it has tested it? + Ugly. + <pinotree> if upstream does it, why cannot i too? :D + <XTaran> pinotree: You've got a point there. + <XTaran> Ok, rethinking. :) + <pinotree> otoh you could just move the testcase for -n up to that file, so + after that you know it works already + <XTaran> pinotree: Well, if so, upstream should do that, not me. :) + <pinotree> you could suggest them to, given the -n usage in the -b testcase + <XTaran> pinotree: Looks alphabetically sorted, so I guess that's at least + not accidentially. + <XTaran> pinotree: Ok, you've convinced me. :) + <pinotree> :D + <XTaran> Especially because this is upstream-suitable once it proved to fix + the Hurd FTBFS. :) + <XTaran> pinotree: The previous upstream code (laast change 2001) instead + of the hardcoded /dev/tty was btw "char=(/dev/tty*([1]))", so I suspect + that the find may work on Cygwin, too. + <XTaran> s/aa/a/ + <pinotree> ah, so that's that comment about globbing on cygwin was + referring to + <XTaran> Yep + <pinotree> cool, so incidentally i've solved also that small issue :9 + <pinotree> :) + <XTaran> pinotree: I hope so. :) + <XTaran> Then again, I hope, external commands like find are fine for + upstream. + <pinotree> then they should rework the already existing testcases ;) + <XTaran> pinotree: Ah, I fall again for the same assumptions. :) + <XTaran> Seems as I would really build test suites with a different + approach. :) + <pinotree> nothing bad in that, i'd say + <XTaran> I'd try to make the tests as far as possible independent from + other tools or features to be sure to test only the stuff I want to test. + <XTaran> Warning: Not testing [[ -c tty ]] (no tty found) + <XTaran> Interesting. I didn't expect that outside a chroot. :) + <pinotree> where's that? + <XTaran> pinotree: A plain "debuild on my Sid VM. + <pinotree> ah + <XTaran> Linux, amd64 + <XTaran> (and Debian of course ;-) + <XTaran> pinotree: Ah, my fault, I kept upstreams char= but didn't change + it in your code. :) + <pinotree> hehe + <XTaran> pinotree: Will be included in the next zsh upload. But I don't + want to upload a new package before the current one moved to testing (or + got an RC bug report to fix :-) + <pinotree> oh sure, that's fine + <XTaran> pinotree: + http://anonscm.debian.org/gitweb/?p=collab-maint/zsh.git;a=commitdiff;h=22bc9278997a8172766538a2ec6613524df03742 + <XTaran> (I've reverted my previous commit) + <pinotree> \o/ + + +#### IRC, OFTC, #debian-hurd, 2013-09-30 + + <XTaran> Anyone knows why the building of zsh on ironforge restarted? It + was at something like "building 4h20m" when I looked last and it now is + at "building 1h17m" but there's no old or last log, so it does still look + like the first build. + <pinotree> most probably got stuck + <XTaran> Oh, ok. + <XTaran> pinotree: So there are cases where the log is not kept? + <pinotree> looks so + <youpi> when the machine crashes, yes :) + <XTaran> youpi: Ooops. Was that me? + <youpi> no, I just rebooted the box + <youpi> I didn't easily find which process to kill + <XTaran> Ok. Then I'll check back tomorrow morning if pinotree's fix for + zsh's test suite on hurd worked. :) + <youpi> it seems to be hung on + /build/buildd-zsh_5.0.2-5-hurd-i386-vO9pnz/zsh-5.0.2/obj/Test/../Src/zsh + <youpi> ../Src/zsh ../../Test/ztst.zsh ../../Test/Y02compmatch.ztst + <XTaran> :( + <XTaran> At least pinotree's patch worked as it then likely passed + C02cond.ztst. :) + <XTaran> youpi: For how long? There are multiple tests which take at least + 3 seconds per subtest. + <youpi> one hour already + <XTaran> Ok. + <XTaran> That's far too long + + +#### IRC, OFTC, #debian-hurd, 2013-10-01 + + <XTaran> pinotree: I've just checked + https://buildd.debian.org/status/fetch.php?pkg=zsh&arch=hurd-i386&ver=5.0.2-5&stamp=1380608100 + manually: Your fix unfortunately seemed not to help, but another test + failed, too, and that one came later and was hence suspected as primary + failing issue. + <XTaran> pinotree: But "+ find: `/dev/tty': No such device or address" + gives some hint. I just have no idea, why find issues that message. + * XTaran really wonders how that message can be caused. + <XTaran> So find sees /dev/tty, but gets an error if it tries to access + (maybe only stat) it while not being connected to a terminal. + <XTaran> Bingo: This reproduces the issue (note the missing -t option to + ssh): ssh exodar.debian.net "find /dev/ -nowarn -maxdepth 1 -name 'tty*' + -type c -ls" + <XTaran> Even clearer: $ ssh exodar.debian.net "ls -l /dev/" | grep 'tty$' + <XTaran> ls: cannot access /dev/tty: No such device or address + <XTaran> ?????????? ? ? ? ? ? tty + <XTaran> I'd say this is a bug somewhere deep down, either in libc or the + kernel. + <pinotree> or in the console translator + <XTaran> pinotree: Never heard of that so far. :) + <XTaran> pinotree: Someone from zsh upstream suggests to use /dev/null or + /dev/zero instead of /dev/tty* -- will try that for the next upload. + <pinotree> ah right, /dev/null should be standard POSIX + <XTaran> I hope so. :) + <pinotree> http://pubs.opengroup.org/onlinepubs/9699919799/ check in POSIX + <pinotree> in any case, sorry for the troubles it is giving you... + <XTaran> pinotree: I'm more concerned about the hanging second test. I + think I can get that test working with using /dev/null. + <XTaran> Now that I've understood why the original test is failing. + <XTaran> pinotree: Shall I write a bug report for that issue? If so, + against which package? + <pinotree> XTaran: not sure it is worth at this stage, having a clearer + situation on what happens could be useful + <pinotree> it is something that can happen sporadically, though + <XTaran> pinotree: Well, it seems a definitely unwanted inconsistency + between what the directory listing shows and which (pseudo) files are + accessible. Independently of where the bug resides, this needs to be + fixed IMHO. + <pinotree> sure, nobody denies that + <XTaran> pinotree: I'd call it easily reproducible. :) + <pinotree> not really + <XTaran> ... once you know where to look for. diff --git a/hurd/translator/mtab/discussion.mdwn b/hurd/translator/mtab/discussion.mdwn index 0734e1e6..961711f3 100644 --- a/hurd/translator/mtab/discussion.mdwn +++ b/hurd/translator/mtab/discussion.mdwn @@ -2103,7 +2103,173 @@ In context of [[open_issues/mig_portable_rpc_declarations]]. <youpi> anyway, got to run -## IRC, freenode, #hurd, 2013-09-20 +## Memory Leak + +Fixed in 2013-09-28 Hurd commit a81c0c28ea606b0d0a2ad5eeb74071c746b7cdeb +(libdiskfs), and 2013-10-04 Hurd commit +98b6f846b628e858acbae9258bac78cf54126d27 (libnetfs). + +### IRC, freenode, #hurd, 2013-09-18 + + <braunr> ext2fs is using a ginormous amount of memory on darnassus since i + last updated the hurd package :/ + <braunr> i wonder if my ext2fs large store patches rework have introduced a + regression + <braunr> the order of magnitude here is around 1.5G virtual space :/ + <braunr> it used to take up to 3 times less before that + <braunr> looks like my patches didn't make it into the latest hurd package + <braunr> teythoon: looks like there definitely is a new leak in ext2fs + <teythoon> :/ + <braunr> memory only + <braunr> the number of ports looks stable relative to file system usage + <teythoon> braunr: I tested my patches on my development machine, it's up + for 14 days (yay libvirt :) and never encountered problems like this + <braunr> i've been building glibc to reach that state + <teythoon> hm, that's a heavy load indeed + <teythoon> could be the file name tracking stuff, I tried to make sure that + everything is freed, but I might have missed something + <braunr> teythoon: simply running htop run shows a slight, regular increase + in physical memory usage in ext2fs + <pinotree> old procfs stikes again? :) + <teythoon> braunr: I see that as well... curious... + <braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried + to make sure that everything is freed, but I might have missed something + <braunr> how knows, maybe completely unrelated + <teythoon> the tracking patch isn't that big, I've gone over it twice today + and it still seems reasonable to me + <braunr> hm + + +### IRC, freenode, #hurd, 2013-09-25 + + <braunr> seems like a small leak per file access + <braunr> but htop makes it obvious because it makes lots of them + <braunr> shouldn't be too hard to find + <braunr> since it might also come from the large store patch, i'll take a + look at it + + +### IRC, freenode, #hurd, 2013-09-27 + + <braunr> teythoon: found the leak :) + <braunr> although its origin is weird + <teythoon> braunr: where is it? + <braunr> i'm still building packages to make sure that's it + <braunr> see + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/blob/HEAD:/libdiskfs/dir-lookup.c + <braunr> which you changed in + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/commit/06d49cdadd9e96361f3fe49b9c940b88bb869284 + <braunr> line 306 is "return error" instead of "goto out" + <braunr> has been so since 1994 + <braunr> what is unclear is why this code path is now run + <braunr> patch is here: + http://darnassus.sceen.net/~rbraun/0001-Fix-memory-leak-in-libdiskfs.patch + <teythoon> I see, weird indeed + <braunr> teythoon: the system also feels slower somehow + <braunr> such errors might have introduced unexpected retries + <teythoon> i think it's possible to write a coccinelle patch to find such + errors + + +### IRC, freenode, #hurd, 2013-09-28 + + <youpi> braunr: bah, I havent noticed the leak on my box, even after + building eglibc & hurd several times + <braunr> that's weird + <braunr> are you sure it's up to date ? + <braunr> also, is procfs correctly attached to /proc ? + <braunr> that's what seems to trigger it + <youpi> yes, 20130924-2, with procfs on /proc + + <teythoon> braunr: that turned out to be the leak indeed? and somehow my + changes triggered it? did you discover why? + <braunr> teythoon: yes, yes, no + <braunr> but youpi didn't see the leak on his system + <teythoon> ^^ cool that you found it + <teythoon> I did + <braunr> oh yes you mean you saw the leak + <teythoon> yes + + +### IRC, freenode, #hurd, 2013-10-01 + + <braunr> the fix i did in libdiskfs might have fixed other issues + <braunr> apparently, it's the code path taken when error isn't ENOENT, + including no error (translator started) + <pinotree> the memory leak fix, you mean? + <braunr> yes + <braunr> it might haved fixed reference counting too + <braunr> although i'm not sure if we actually ever run into that issue in + the past + <braunr> the weird thing is, that path is taken when starting a passive + translator + <braunr> (i think) + <braunr> (it might be any kind of translator, and just doing nothing if + alcready active) + <braunr> already* + <braunr> anyway, the fact that the leak was so visible means this code was + run very often + <braunr> which doesn't make sense + <braunr> hm ok, it seems that code was run every time actually + <braunr> but the leak became visible when it concerned memory + <pinotree> which side-effects did the old code produce? + <braunr> teythoon added a dynamically allocated path that wasn't freed + <braunr> reference leaks + <braunr> which might explain the assertion on reference we sometimes see + with ext2fs + <braunr> when a counter overflows and becomes 0 + +[[open_issues/ext2fs_libports_reference_counting_assertion]]? + + <pinotree> hmm + <braunr> which is why i'm mentioning it + <braunr> :) + <braunr> i'll try to reproduce the assertion + <braunr> pinotree: actually, after a more in-depth look, reference counting + looks valid before the fix too + <pinotree> ok, thanks for checking + <braunr> pinotree: the assertion affects the root translator, and is + triggered by a test that stresses memory + <pinotree> memory as in ram, or as in disk storage? + <braunr> malloc + <pinotree> ok + <braunr> i suspect the code doesn't handle memory failure well + + +### IRC, freenode, #hurd, 2013-10-02 + + <teythoon_> braunr: btw, did you fix the leak? + <braunr> yes + <braunr> + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/commit/a81c0c28ea606b0d0a2ad5eeb74071c746b7cdeb + <braunr> 1h after tagging 0.5 ( + <braunr> :( + <teythoon> ah yes, I've seen that commit + <teythoon> I just wanted to know whether this settled the issue + <braunr> it does :) + <teythoon> good + <braunr> i still can't figure out why youpi didn't had it + <braunr> the code path is run when no error (actually error != ENOENT) + <braunr> which explains why the leak was so visible + <teythoon> so my patch exposed this b/c of the allocation I added, makes + sense + <teythoon> it's funny actually, b/c this wasn't an issue for me as well, I + had my development vm running on that patches for two weeks + + +### IRC, freenode, #hurd, 2013-10-02 + + <braunr> libnetfs suffers from the same leak as libdiskfs when looking up a + translator + <braunr> i'll fix it too + + +## Multiple mtab Translators Spawned + +Fixed in 2013-10-05 procfs commit c87688a05a8b49479ee10127470cc60acebead4a? + + +### IRC, freenode, #hurd, 2013-09-20 <braunr> teythoon: how come i see three mtab translators running ? <braunr> 6 now oO @@ -2113,10 +2279,231 @@ In context of [[open_issues/mig_portable_rpc_declarations]]. <braunr> teythoon: more bug fixing for you :) -## IRC, freenode, #hurd, 2013-09-23 +### IRC, freenode, #hurd, 2013-09-23 <teythoon> so it might be a problem with either libnetfs (which afaics has never supported passive translator records before) or procfs, but tbh I haven't investigated this yet [[open_issues/libnetfs_passive_translators]]. + + +### [[!debbug 724868]] + + +### IRC, freenode, #hurd, 2013-10-03 + + <braunr> i can't manage to find out where the hurd stores information about + active translators ... + <braunr> there is this transbox per node + <braunr> but where are nodes stored ? + <braunr> what if they are are dropped ? + <pinotree> braunr: iirc, see libfshelp + <braunr> well i have + <braunr> i still can't find it + <braunr> i fear that it works for ext2fs because that particular translator + implements a cache of open nodes + <braunr> whereas things like procfs drop and recreate nodes per open + <braunr> which would be the root cause for the multiple mtab bug + <pinotree> doesn't tmpfs support translators? + <braunr> good idea + <braunr> although it's still a libdiskfs based one + <braunr> no problem for tmpfs, so it would be a netfs/procfs issue + <braunr> better than what i feared :) + <braunr> now, how is libdiskfs able to find active translators .. + <braunr> ah, there is a name cache in libdiskfs .. + <braunr> nope, looks fine + + +### IRC, freenode, #hurd, 2013-10-04 + + <braunr> nodes with a translator seem to keep a reference in libdiskfs and + not in libnetfs + <braunr> mhmmpf + <braunr> oh great .. + <braunr> each libdiskfs that "works" seems to implement its own + diskfs_cached_lookup function + <braunr> so both ext2fs and tmpfs actually maintain a list of nodes, + keeping a reference on those with a translator + <braunr> while procfs simply doesn't + <braunr> teythoon: ^ + <braunr> *sigh* + <teythoon> braunr: ok, thanks, I'll look into that + <braunr> i'm not sure how to fix it + <braunr> we can either fix node destruction to cleanly shut down + translators + <braunr> but this would mean starting mtab on each access + <braunr> or we could implement a custom cache in procfs + <braunr> or perhaps a very custom change in the lookup callback for mounts + <braunr> i'll try the latter + <teythoon> err, shouldn't we try to fix this in lib*fs? + <braunr> unless you really want to work on it + <braunr> i dont' know + <teythoon> ah, so the node is destroyed but the translator is kept running? + that's what you mean by the above? + <teythoon> and ext2fs makes an effort of killing it in its node cleanup + code? + <braunr> yes + <braunr> grmbl, i'm lagging a lot + <braunr> i'm not sure + <braunr> ext2fs maintains it + <braunr> with ext2fs, translators can only be explicitely removed + <braunr> i mean, ext2fs keeps all node descriptors alive once accessed + <braunr> while procfs doesn't + <braunr> teythoon: ok, looks like i have a working patch that merely caches + the node for mounts + + <braunr> i installed my fixed procfs on darnassus, only one mtab :) + <teythoon> nice :) + + <braunr> i have a fix for the multiple mtab issue, will send a patch + tonight + + +## `/home` Missing + +### IRC, freenode, #hurd, 2013-10-04 + + <braunr> now, why is there no /home in df output ? + <teythoon> not sure + <teythoon> note how /dev/tty* end up in /proc/mounts, those are passive + translators too, no? + <braunr> yes + <braunr> but that's a good thing i guess + <braunr> or was mounts intended for file systems only ? + <braunr> well, in the unix traditional meaning + <teythoon> I think its nice too, yes + <teythoon> but why are they fine and your /home is not... + <braunr> that's weirder + <braunr> also, mounts actually doesn't show passive translators + <braunr> teythoon: does your code perform any kind of comparison ? + <braunr> i see /servers/socket/26 but not /servers/socket/2 + <braunr> s/comparison/filter/g + <teythoon> hmm + <teythoon> well, yes, try /hurd/mtab --insecure / + <teythoon> (I cannot connect to darnassus from here...) + <braunr> ok but that looks unrelated + <braunr> both /servers/socket/26 and /servers/socket/2 refer to the same + translator + <braunr> i was wondering if mtab was filtering similar entries based on + that + <teythoon> no + <braunr> that's weird too then, isn't it ? + <teythoon> yes ;) + <braunr> ok + <teythoon> btw, how is that done with the same traanslator being bound to + two nodes? settrans cannot do that, can it? + <braunr> no it can't + <braunr> the translator does it when started + <teythoon> ah + <braunr> (which means there is a race if both are started simulatneously, + although it's very rare and not hard to solve) + <teythoon> a weird beaving translator then :) + + <braunr> teythoon: if ext2fs is set active, mtab output reports it + + <braunr> teythoon: looks like this bug is what allows mtab not to deadlock + <braunr> teythoon: when i attach it as an active translator, cat freezes + + <braunr> teythoon: if (control && control->pi.port_right == fsys) + <braunr> that's the filtering i was previously talking about + + <braunr> oh please don't name global variables "path" ... + + +### IRC, freenode, #hurd, 2013-10-06 + + <antrik> teythoon: pty-s also bind to two nodes, not only pfinet + + +### IRC, freenode, #hurd, 2013-10-07 + + <braunr> teythoon: please tell us when you're available, we need to work + out the last mtab issues + <teythoon> braunr: I'm available now :) + <teythoon> I'm sorry, I've been very busy the last two weeks, but I've + plenty of time now + <braunr> great :) + <braunr> did you see youpi's mail ? + <braunr> i have the exact same question + <teythoon> I did + <braunr> it seems your code registers active translators + <braunr> but parent translators don't seem to register them when they're + created from passive translators + <braunr> or am i mistaken ? + <teythoon> I'll need a moment to get my hurd machine and myself up to + speed... + <teythoon> braunr: I concur with youpi, hooking into fshelp_fetch_root + should do just fine + <teythoon> I'll just try that + <braunr> ok + <braunr> how do you deal with mtab reporting itself ? + <teythoon> o_O does it do that? + <braunr> no, but it should + <braunr> when i set it as an active translator, i get a deadlock + <braunr> hm + <braunr> teythoon: before you change libfshelp, i'd like you to try + something else + <braunr> use more appropriate names for global variables in mtab.c + <braunr> in particular, the variable path clashes with local names + <teythoon> noted + <braunr> teythoon: as a side note (i'm not asking to rewrite anything) + <braunr> i strongly recommend a very explicit object oriented style of + coding + <braunr> (or data-oriented as it's sometimes called) + <braunr> use prefixes for all your interfaces so they can be made public if + needed (which acts as a namespace and avoids lots of collisions + naturally) + <braunr> use "constructors" and "destructors" (functions that both allocate + and initialize) + <braunr> this helps avoiding leaks a lot too + <teythoon> hm, I thought I did that, could you be more specific? + <braunr> ok didn't see the comment + <braunr> /* XXX split up */ error_t mtab_populate (... + <braunr> :) + <braunr> as a better example, see your code in libfshelp/translator-list.c + <braunr> struct translator should have been treated as an object + <braunr> this would probably have completely avoided any leaks in the first + place + <teythoon> braunr: right, I deviated from that style there + <braunr> teythoon: these are minor details, don't mind them too much, i + just find it helps me a lot + <teythoon> braunr: sure, I appreciate the feedback :) + + +### IRC, freenode, #hurd, 2013-10-08 + + <teythoon> braunr: I'm on to the passive translator not getting registered + issue + <teythoon> however, removing them from the list if the active translator is + killed does not work as expected... I still need to fiddle with the + notifications to get this right + <braunr> ok + + +### IRC, freenode, #hurd, 2013-10-16 + + <teythoon> braunr: btw, I fixed the 'passive translator not showing up in + proc/mounts'-issue + <teythoon> but 4 ports do leak each time a translator is killed and + reinstalled + <teythoon> this happens with passive ones as well as active ones + <braunr> teythoon: is that issue tied to your changed ? + <braunr> changes* + <teythoon> I'm not sure tbh, testing that is on my list of things to do + <braunr> ok + <braunr> first thing to know i guess + <teythoon> yes + + +## Memory Leak in `translator_ihash_cleanup` + +### IRC, freenode, #hurd, 2013-10-04 + + <braunr> teythoon: isn't there a leak in translator_ihash_cleanup ? + <teythoon> braunr: looks like, yes + <teythoon> braunr: I probably forgot to add the free (element->name) when I + added the name field + <braunr> teythoon: ok + <braunr> teythoon: i let you fix that :p + <teythoon> braunr: sure ;) diff --git a/hurd/translator/proc.mdwn b/hurd/translator/proc.mdwn index d5e0960c..b3b5e703 100644 --- a/hurd/translator/proc.mdwn +++ b/hurd/translator/proc.mdwn @@ -63,6 +63,11 @@ It is stated by `/hurd/init`. something special +## IRC, freenode, #hurd, 2013-09-25 + + <braunr> so nice to finally see proc in top :) + + # Process Discovery ## IRC, freenode, #hurd, 2013-08-26 diff --git a/hurd/translator/procfs/jkoenig/discussion.mdwn b/hurd/translator/procfs/jkoenig/discussion.mdwn index fc071337..87ff0248 100644 --- a/hurd/translator/procfs/jkoenig/discussion.mdwn +++ b/hurd/translator/procfs/jkoenig/discussion.mdwn @@ -436,6 +436,72 @@ Also used in `[GCC]/intl/relocatable.c`:`find_shared_library_fullname` for `#ifdef __linux__`. +### IRC, freenode, #hurd, 2013-10-03 + + <camm`> what's the equivalent of cat /proc/self/maps on hurd? + <braunr> camm`: for now, /proc/self doesn't work as expected + <camm`> thanks, I just want to get a list of maps and protection status for + a running process -- how? + <braunr> vminfo + <camm`> thanks so much! I'm trying to debug an unexec failure on hurd when + a linker script is present. All works with the default script, but when + the text address is changed, unexec fails, running into a page with no + access in the middle of the executable: 0xc4b000[0x1000] (prot=0, + max_prot=RWX, offs=0xb55000) + <camm`> I get a segfault when trying to read from this page. + <braunr> unexec ? + <camm`> emacs/gcl/maxima/acl2/hol88/axiom use unexec to dump a running + image into a saved executable elf file. + <braunr> what is unexec ? + <braunr> ok looks like a dirty tool + <braunr> camm`: what is segfaulting, unexec or the resulting executable ? + <camm`> unexec opens the file from which the running program was originally + executed, finds its section start addresses, then writes a new file + replacing any data in the old file with possibly modified versions in + running memory. The reverse of 'exec'. + <camm`> the read from running memory delimited by the addresses in the + executable file is hitting a page which has been protected with *no* + access, and is segfaulting. Somehow, when the binary file is loaded, + hurd turning off all rights to this page. + <camm`> let me check the stack location ... + <camm`> ok I think I've got it -- hurd moves the sbrk(0) address away from + the end of .data (as reported by readelf) if the addresses are low, + presumably to avoid running into the stack. + <camm`> starting sbrk(0)!=.data+data_size on hurd + <braunr> i'm not sure there is anything like the heap on the hurd + <braunr> sbrk is probably implemented on top of mmap + <braunr> camm`: hm no, i'm wrong, glibc implements brk and sbrk mostly as + expected, but remapping the area isn't atomic + <braunr> "Now reallocate it with no access allowed" + <braunr> then, there is a call to vm_protect + <braunr> and no error checking + <braunr> ... + <camm`> ok, that's fine, but need to know -- in general there is no + relationship between the address returned by sbrk(0) and the .data + addresses reported by readelf on the file, (hurd only) yes? + <braunr> i don't know about that + <braunr> there should be .. + <camm`> Specific example: readelf -a -> [24] .data PROGBITS + 000f5580 0c4580 000328 00 WA 0 0 32 + <camm`> + <camm`> sbrk(0)->(void *) 0x8021000 + <braunr> camm`: is that on an executable or a shared object ? + <camm`> executable + <braunr> 000f5580 looks very low + <camm`> This is using a linker script. The default setup works just fine. + <camm`> I think it (might) make sense for hurd to silently do this give the + placement of the C stack, but the assumptions behind my algorithm need + changing (perhaps). + <camm`> (I probe in configure the allowable range of __executable_start, + and then choose a value to either ensure a large free signed range around + NULL, or a low data start to maximize heap) + <camm`> braunr: are there any guarantees of sbrk(0)==.data+size without a + linker script? + <braunr> camm`: i'm not sure at all + <braunr> sbrk isn't even posix + <camm`> thanks + + # `/proc/[PID]/mem` Needed by glibc's `pldd` tool (commit @@ -471,3 +537,56 @@ Needed by glibc's `pldd` tool (commit <braunr> both htop and top seem to have problems report the cpu time <braunr> so i expect the problem to be in procfs + + +# IRC, freenode, #hurd, 2013-10-03 + + <braunr> teythoon: any reason the static variable translator_exists isn't + protected by a lock in procfs/rootdir.c ? + + +## IRC, freenode, #hurd, 2013-10-04 + + <braunr> teythoon: can you tell me why translator_exists isn't protected + from shared access in rootdir_mounts_exists ? + <teythoon> braunr: hm, dunno tbh, I probably thought the race was harmless + enough + <braunr> it probably is + <braunr> settrans -Rg doesn't work on procfs :( + + +# Kernel PID + +## IRC, freenode, #hurd, 2013-09-25 + + <braunr> hm cute, htop layout has become buggy, top just won't start + <teythoon> braunr: make sure your procfs knows the correct kernel pid + <teythoon> # showtrans /proc + <teythoon> /hurd/procfs -c -k 3 + <teythoon> we could have handled this nicer if procfs were integrated + upstream + <teythoon> we should probably just update the default + <braunr> teythoon: mhm + <braunr> $ fsysopts /proc + <braunr> /hurd/procfs --stat-mode=444 --fake-self=1 + <braunr> $ showtrans /proc + <braunr> /hurd/procfs -c + <pinotree> -c == --stat-mode=444 --fake-self=1 + <braunr> better indeed + <braunr> teythoon: thanks + + +## IRC, freenode, #hurd, 2013-10-04 + + <braunr> youpi: i fixed procfs on ironforge and exodar to be started as + procfs -c -k 3 + <braunr> without -k 3, many things as simple as top and uptime won't work + + +## IRC, freenode, #hurd, 2013-10-24 + + <gg0> braunr: i'm using your repo and i can't see cpu percentage in htop + anymore, all zeroes, confirmed? + <braunr> gg0: no + <braunr> gg0: you probably need to reset procfs + <braunr> gg0: settrans /proc /hurd/procfs -c -k 3 diff --git a/hurd/translator/term.mdwn b/hurd/translator/term.mdwn new file mode 100644 index 00000000..78c83276 --- /dev/null +++ b/hurd/translator/term.mdwn @@ -0,0 +1,214 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The *term* translator implements POSIX termios discipline. + + +# Open Issues + +## [[open_issues/Term_Blocking]] + +## Leaks/Not Re-used/Not Terminating + +[[!tag open_issue_hurd]] + + +### IRC, freenode, #hurd, 2013-10-14 + + <braunr> good news + <braunr> the terminal leak is related to privilege separation + +[[tschwinge]] confirming (using `ssh -t [machine] tty`) that with privilege +separation enabled there is this problem, and once disabled it goes away. + + <atheia> I love how, as an unknowing by-stander, that is somehow good news + :-) + <braunr> :) + <braunr> it's a good news because 1/ we have more knowledge about the issue + <braunr> and 2/ it may not even be a hurd bug + <braunr> but rather an openssh-on-hurd bug + <braunr> this explains why i didn't see the issue on anything else + (mach/hurd consoles, x terminals) + <braunr> and this will also indirectly solve the screen lockup issue + <teythoon> braunr: good catch :) + <braunr> s/a good news/good news/ + <atheia> ah, yes, both definitely good news. Congrats on the progress. + <braunr> i remember we used to disable privilege separation in the past + <braunr> i'll have to dig what made us use it + +[[news/2010-09]]. + + <braunr> interesting, screen seems to be affected nonetheless + <braunr> so it's something common to both screen and ssh privsep + <braunr> apparently, what sshd+privse and screen have in common is a fifo + <braunr> so it's probably a tricky hurd bug actually + + +### IRC, freenode, #hurd, 2013-10-16 + + <braunr> pflocal is leaking ports .. + <braunr> this might be what blocks terminals + * pinotree gives braunr a stick of glue + <braunr> thanks + + <braunr> pflocal leaks struct sock .. + <braunr> grmbl + + <braunr> hm nice, pflocal leaks each time a socket is bound and/or accepted + on + <braunr> looks like a simple ref mess + <pinotree> braunr: really? + <braunr> yes + <pinotree> a leak in pflocal feels strange, never noticed it taking lots of + memory (and it's used a lot) + <braunr> it's a port leak + <braunr> well + <braunr> no it's both a memory and port leak + <braunr> not sure which one is the root cause yet + <braunr> i guess server sockets aren't automatically unbound + <braunr> if you want to see the leak, just disable priv separation in ssh + (to avoid the terminal leak ....) and write a shell loop to start ssh + your_server echo hello + <braunr> google shows mails about the leak in the past + <braunr> i also hope it fixes the terminal leak, although i'm really not + sure :( + + +### IRC, freenode, #hurd, 2013-10-17 + + <braunr> hm nice, apparently, there is no pflocal leak + <braunr> but a libdiskfs one ! + <braunr> since ext2fs enables the ifsock shortcut + <braunr> seems like it leaks a reference on sock node deletion + <teythoon> braunr: have you looked at libdiskfs/dead-name.c? + <teythoon> braunr: I think I'm hunting a very similar problem + <braunr> i'm doing it now + <teythoon> I had the problem of dead name notifications not being delivered + <braunr> wow + <teythoon> b/c I held no reference to the ports_info thing, so the dead + name handler in libports could no longer find the pi struct, so the + notification was silently dropped + <braunr> i see + <braunr> but it looks like dropping a node makes sure the associated + sockaddr has been deleted if any + <teythoon> are you sure the node is dropped in the first place? + <braunr> no + <braunr> well + <braunr> i see something happenning at the pflocal side when removing the + node + <braunr> but there is still a send right lingering somewhere + <braunr> (see why we need a global lsof :p) + <teythoon> indeed + <braunr> i'll try portinfo with that option we talked about + <teythoon> yes + <braunr> 121 => 1682: send (refs: 1) + <braunr> yep, ext2fs still has it + <teythoon> (I wonder how portinfo does that...) + <braunr> i guess it imports rights from the target task + <braunr> and see if it gets the same name as a local right + <teythoon> makes sense + <braunr> easy to check + <teythoon> well, no, it cannot do that for receive rights + <braunr> it creates an empty task just for that purpose + <braunr> and uses mach_port_extract_right + <teythoon> but it works as you described, yes + <braunr> so yes it does work for receive rights too + <teythoon> yes + <teythoon> cool :) + <braunr> so it assumes identical port names are part of the ipc interface + <braunr> something neal said we shouldn't rely on + <braunr> iirc + <teythoon> yes, I remember something like that too + <braunr> here is the strange thing + <braunr> node->sockaddr is deallocated on a dead name notification + <braunr> drop_node checks that sockaddr is null + <braunr> so how can the dead name notification occur before the node is + dropped ? + <braunr> so maybe the node is still around indeed + <braunr> apparently, libdiskfs considers the address holds a reference on + the node + <braunr> on the other hand, the server socket won't get released unless the + address gets a no-sender notification ... + <braunr> this should probably be turned into a weak reference + <braunr> teythoon: indeed, the node is leaked + + <braunr> pflocal crashes when removing correctly deallocating addresses and + removing server sockets :/ + + <braunr> ok, pflocal bug fixed + <braunr> still have to fix the libdiskfs leak + <braunr> and libdiskfs leak fixed too + <braunr> :) + <braunr> i'll build hurd packages with my changes to make sure i don't + break something before comitting + <braunr> and see if this fixes the term issue + + <braunr> looks like my patches work just fine :) + <braunr> it doesn't solve the term issue though + + <braunr> so, according to portinfo, pflocal has send rights to terminals oO + + <braunr> mhhhmmmmmm + <braunr> openssh seems to pass terminal file descriptors through unix + sockets when using privilege separation + <pinotree> braunr: i a write(sock, &pid, sizeof int) (or the like)? + <pinotree> *ie + <braunr> not pid, file descriptors + <braunr> SCM_RIGHTS + <pinotree> ah ok + <braunr> the socket send/recv interface does support passing mach ports + <braunr> and the leaked ports do turn into dead names when i kill terminals + <pinotree> yes, we support with a patch pochu did few years ago + <braunr> so it seems the leak is related to libpipe this time + <braunr> ok got it :) + <braunr> pflocal used copy_send instead of move_send + <braunr> \o/ + <braunr> that bug was such a pain + * braunr happy + <teythoon> :) + <pinotree> speaking of it, in pflocal' S_socket_recv is it correct the + "out_flags = 0;"? + <braunr> nice catch + <braunr> although i wonder why flags are returned + <braunr> it may have been set to null to tell us that we don't want to + return flags + <braunr> pfinet seems to use it + <pinotree> but you change a local variable anyway + <braunr> yes it's not useful + <braunr> hmm + <braunr> out_flags is what gets in struct msghdr -> msg_flags + <braunr> so i guess it makes sense to fix it to *out_flags = 0, just to be + safe + <braunr> pinotree: do you want me to push it tonight along with the others + ? + <pinotree> yes please + <braunr> ok + <pinotree> thanks! + <braunr> pflocal seems to not leak any memory or ports at all + <braunr> great :> + + <braunr> there, patches pushed :) + + +## `screen` Logout Hang + +[[!tag open_issue_hurd]] + + +### IRC, freenode, #hurd, 2013-10-14 + + <braunr> i fixed term so that screen can shutdown properly + <braunr> read() wouldn't return EIO after terminal hangup + + +### IRC, freenode, #hurd, 2013-10-17 + + <braunr> and the missing EOI prevented screen from correctly shutting down + windows diff --git a/hurd/translator/tmpfs/discussion.mdwn b/hurd/translator/tmpfs/discussion.mdwn index 20aba837..8c332d84 100644 --- a/hurd/translator/tmpfs/discussion.mdwn +++ b/hurd/translator/tmpfs/discussion.mdwn @@ -430,3 +430,40 @@ License|/fdl]]."]]"""]] <youpi> ok <youpi> but that indeed means writeback of ext2fs works, which is a good sign :) + + +# IRC, freenode, #hurd, 2013-10-04 + + <teythoon> btw, I noticed that fifos do not work on tmpfs + <braunr> teythoon: tmpfs seems limited, yes + <teythoon> that's annoying b/c /run is a tmpfs on Debian and sysvinit + creates a crontrol fifo there + <teythoon> I wonder why I didn't notice that before + <braunr> also, fifos, like symlinks, can be shortcircuited in libdiskfs + <braunr> i wonder if that has anything to do with the problem at hand + +[[mtab/discussion]], *Multiple mtab Translators Spawned*. + + <teythoon> b/c this breaks reboot & friends + <teythoon> I do too + <teythoon> b/c I cannot find any shortcut related code in tmpfs + <braunr> well, it's optional normally + <braunr> so that's ok + <braunr> but has it really been tested when the option wasn't there ? :) + <teythoon> yes, but the tmpfs requests this by setting diskfs_shortcut_fifo + = 1; + <pinotree> hm i remember tmpfs was said to be working with + sockets/fifos/etc, back then when it was fixed + <braunr> teythoon: oh + + +## IRC, freenode, #hurd, 2013-10-11 + + <teythoon> this will have to wait for the next hurd pkg unfortunately, b/c + I broke tmpfs by accident :-/ + <pinotree> how so? + <teythoon> the dropping of privileges broke passive translators and mkfifo + <braunr> there actually is a reason why those are run as root or with the + privilege of their owner + <braunr> privileges should be decoupled from identity + <teythoon> yes @@ -1,5 +1,5 @@ [[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, -2009, 2010, 2011 Free Software Foundation, Inc."]] +2009, 2010, 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -98,8 +98,9 @@ are rendered on a [[Debian_GNU/Hurd|hurd/running/debian]] system. # Current Status -There has not yet been an official 1.0 release. The Hurd is developed by a few -volunteers in their spare time. The project welcomes any assistance you can provide. +The last release is [[GNU Hurd 0.5|news/2013-09-27]], 2013-09-27. The Hurd is +developed by [[a few volunteers in their spare time|faq/how_many_developers]]. +The project welcomes any assistance [[you can provide|contributing]]. Porting and development expertise is still badly needed in many key areas. Functional systems are installable in a dual-boot configuration. Development diff --git a/media_appearances.mdwn b/media_appearances.mdwn index 3d1ae7c6..2f8df76c 100644 --- a/media_appearances.mdwn +++ b/media_appearances.mdwn @@ -16,20 +16,108 @@ A lot of stuff is missing here. [[!toc levels=2]] - # 2013 +## September + + * In context of the [[GNU Hurd 0.5, GNU Mach 1.4, GNU MIG 1.4 + releases|news/2013-09-27]]: + + * *[New GNU Hurd, Mach, and MIG + releases](http://lwn.net/Articles/568745/)*, LWN, Jonathan Corbet, + 2013-09-28 + + * *[GNU Hurd 0.5, GNU Mach 1.4 + Released](http://www.phoronix.com/scan.php?page=news_item&px=MTQ3MTk)*, + Phoronix, Michael Larabel, 2013-09-28 + + * *[GNU Hurd 0.5 erschienen + \[de\]](http://www.pro-linux.de/news/1/20302/gnu-hurd-05-erschienen.html)*, + Pro-Linux, Hans-Joachim Baader, 2013-09-29 + + * *[Neue Hurd-Version zum Jubiläum + \[de\]](http://www.heise.de/newsticker/meldung/Neue-Hurd-Version-zum-Jubilaeum-1969877.html)*, + heise online, Mirko Dölle, 2013-09-30 + + * *[GNU Hurd 0.5 + released](http://www.osnews.com/story/27356/GNU_Hurd_0_5_released)*, OS + News, Thom Holwerda, 2013-10-01 + + * *[GNU Hurd 0.5, GNU Mach 1.4, GNU MIG 1.4 + Released](http://tech.slashdot.org/story/13/09/27/2342234/gnu-hurd-05-gnu-mach-14-gnu-mig-14-released)*, + Slashdot, 2013-09-27 + + ## August * GNU Hackers Meeting in Paris: {{$community/meetings/ghm2013#thibault_hurd}} +## May + + * In context of the [[Debian GNU/Hurd 2013 + release|news/2013-05-debian_gnu_hurd_2013]]: + + * *[A Debian GNU/Hurd snapshot](http://lwn.net/Articles/554992/)*, + LWN, Nathan Willis, 2013-06-19 + + * *[Debian GNU/Hurd 2013: Debian mit Mach-Microkernel + \[de\]](http://www.pro-linux.de/news/1/19818/debian-gnuhurd-2013-debian-mit-mach-microkernel.html)*, + Pro-Linux, Falko Benthin, 2013-05-23 + + * *[Debian GNU/Hurd 2013 released](http://lwn.net/Articles/551329/)*, + LWN, Jake Edge, 2013-05-22 + + * *[Debian GNU/Hurd 2013 Release Brings New + Packages](http://www.phoronix.com/scan.php?page=news_item&px=MTM3NzA)*, + Phoronix, Michael Larabel, 2013-05-22 + + * *[Debian GNU/Hurd 2013 rounded + up](http://www.h-online.com/open/news/item/Debian-GNU-Hurd-2013-rounded-up-1867328.html), + The H Open, Fabian Scherschel, 2013-05-22 + + * *[Debian GNU/Hurd 2013 veröffentlicht + \[de\]](http://www.admin-magazin.de/News/Debian-GNU-Hurd-2013-veroeffentlicht)*, + Admin-Magazin, Oliver Frommel, 2013-05-22 + + * *[Debian GNU/Hurd 2013 + released](http://www.osnews.com/story/27059/Debian_GNU_Hurd_2013_released)*, + OS News, Thom Holwerda, 2013-05-22 + +## February + + * *[GNU/Hurd Plans For A Future With USB, SATA, + 64-Bit](http://www.phoronix.com/scan.php?page=news_item&px=MTI5ODM)*, + Phoronix, Michael Larabel, 2013-02-10 + + +## January + + * *[GNU Hurd Is Still Moving, Albeit + Slowly](http://www.phoronix.com/scan.php?page=news_item&px=MTI3OTE)*, + Phoronix, Michael Larabel on 2013-01-20 + + * *[LLVM/Clang Compiler Being Ported To + GNU/Hurd](http://www.phoronix.com/scan.php?page=news_item&px=MTI3MjI)*, + Phoronix, Michael Larabel on 2013-01-11 + # 2012 +## December + + * *[Whatever happened to the Hurd? - The story of the GNU + OS](http://www.linuxuser.co.uk/features/whatever-happened-to-the-hurd-the-story-of-the-gnu-os), + Linux User, Richard Hillesley, 2012-12-23 + + Follow-up discussion: + + * <http://www.osnews.com/comments/26673> + + ## April * *[People behind Debian: Samuel Thibault, working on accessibility and the @@ -37,6 +125,15 @@ A lot of stuff is missing here. Raphaël Hertzog, 2012-04-19 +## March + + * After publishing [[news/2011-q4]]: + + * *[Hurd-Quartalsbericht: Fortschritte bei der Anwendungskompatibilität + \[de\]](http://www.pro-linux.de/news/1/18183/hurd-quartalsbericht-fortschritte-bei-der-anwendungskompatibilitaet.html)*, + Pro-Linux, Hans-Joachim Baader, 2012-03-22 + + # 2011 @@ -50,36 +147,34 @@ A lot of stuff is missing here. ### After Publishing [[news/2011-q2]] - * {{$news/2011-q2#lwn}} - - * {{$news/2011-q2#phoronix-1}} - - * {{$news/2011-q2#phoronix-2}} - - * {{$news/2011-q2#phoronix-3}} - * {{$news/2011-q2#golem}} * {{$news/2011-q2#h-online}} * {{$news/2011-q2#innocenthacker}} - * {{$news/2011-q2#netzwelt}} + * {{$news/2011-q2#lwn}} - * {{$news/2011-q2#operation-tunnelbau}} + * {{$news/2011-q2#netzwelt}} * {{$news/2011-q2#osnews}} + * {{$news/2011-q2#phoronix-1}} + + * {{$news/2011-q2#phoronix-2}} + + * {{$news/2011-q2#phoronix-3}} + * {{$news/2011-q2#pro-linux}} * {{$news/2011-q2#reddit-1}} * {{$news/2011-q2#reddit-2}} - * {{$news/2011-q2#slashdot}} - * {{$news/2011-q2#schmehl}} + * {{$news/2011-q2#slashdot}} + * ... @@ -100,17 +195,22 @@ A lot of stuff is missing here. * GNU Hackers Meeting in the Hague: {{$community/meetings/ghm2010#walfield_hurd}} - * Koen Vervloesem: [*The Hurd: GNU's quest for the perfect - kernel*](http://lwn.net/Articles/395150/) + * *[The Hurd: GNU's quest for the perfect + kernel](http://lwn.net/Articles/395150/)*, LWN, Koen Vervloesem Follow-up discussion: * <http://www.osnews.com/comments/23827> - * Richard Hillesley: [*GNU HURD: Altered visions and lost - promise*](http://www.h-online.com/open/features/GNU-HURD-Altered-visions-and-lost-promise-1030942.html) - ([German - translation](http://www.heise.de/open/artikel/GNU-HURD-Veraenderte-Visionen-und-verworfene-Versprechen-1046753.html)) + +## June + + * *[GNU HURD: Altered visions and lost + promise](http://www.h-online.com/open/features/GNU-HURD-Altered-visions-and-lost-promise-1030942.html)*, + Richard Hillesley, 2010-06-30 (translation: *[GNU HURD: Veränderte Visionen + und verworfene Versprechen + \[de\]](http://www.heise.de/open/artikel/GNU-HURD-Veraenderte-Visionen-und-verworfene-Versprechen-1046753.html)*, + 2010-07-28) Follow-up discussion, some of it valid and constructive criticism, some of it less so: @@ -124,7 +224,16 @@ A lot of stuff is missing here. # 2004 - * Marcus Brinkmann, *The GNU Hurd - Lessons and Perspective*, from the + * *The GNU Hurd - Lessons and Perspective*, Marcus Brinkmann, from the [OpenWeekend](http://openweekend.cz/) 2004 conference organized by the Silicon Hill club. <http://video.google.com/videoplay?docid=-7449462856350014702> + + +# 1994 + +## November + + * [[First Symposium on Operating Systems Design and Implementation (OSDI + '94)|community/meetings/osdi_1994]]: + {{$community/meetings/osdi_1994#bushnell_hurd}}. diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index 8f47f61f..2e205a9a 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -2384,3 +2384,310 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. concurrently <braunr> (which is another contention issue when using mach-like ipc, which often do need to allocate/release virtual memory) + + +## IRC, freenode, #hurd, 2013-09-28 + + <rah> braunr: http://git.sceen.net/rbraun/x15.git/blob/HEAD:/README + <rah> "X15 is a free microkernel." + <rah> braunr: what distinguishes it from existing microkernels? + + +## IRC, freenode, #hurd, 2013-09-29 + + <braunr> rah: the next part maybe ? + <braunr> "Its purpose is to provide a foundation for a Hurd-like operating + system." + <rah> braunr: there are already microkernels that canbe used as the + foundatin for Hurd=like operating systems; why are you creating another + one? + <rah> braunr: what distinguishes your microkernel from existing + microkernels? + <tschwinge> rah: + http://www.gnu.org/software/hurd/microkernel/mach/deficiencies.html + <braunr> rah: it's better :) + <braunr> rah: and please, cite one suitable kernel for the hurd + <rah> tschwinge: those are deficiencies in Mach; I'm asking about x15 + <rah> braunr: in what way is it better exactly? + <braunr> rah: more performant, more scalable + <rah> braunr: how? + <braunr> better algorithms, better interfaces + <braunr> for example, it supports smp + <rah> ah + <rah> it supports SMP + <rah> ok + <rah> that's one thing + <braunr> it implements lockless synchronization à la rcu + <rah> are there any others? + <rah> ok + <rah> lockless sync + <rah> anything else? + <braunr> it can scale from 4MB of physical memory up to several hundreds + GiB + <braunr> ipc is completely different, leading to simpler code, less data + involved, faster context switches + <braunr> (although there is no code for that yet) + <rah> how can it support larger memory while other microkernels can't? + <rah> how is the ipc "different"? + <braunr> others can + <braunr> gnumach doesn't + <rah> how can it support larger memory while gnumach can't? + <azeem_> because it's not the same code base? + <braunr> gnumach doesn't support temporary kernel mapping + <rah> ok, so x15 supports temporary kernel mapping + <braunr> not exactly + <braunr> virtual memory is completely different + <rah> how so? + <braunr> gnumach does the same as linux, physical memory is mapped in + kernel space + <braunr> so you can't have more physical memory than you have kernel space + <braunr> which is why gnumach can't handle more than 1.8G right now + <braunr> it's a 2/2 split + <braunr> in x15, the kernel maps what it needs + <braunr> and can map it from anywhere in physical memory + <tschwinge> rah: I think basically all this has already been discussed + before and captured on that page? + <braunr> it already supports i386/pae/amd64 + <rah> I see + <braunr> the drawback is that it needs to update kernel page tables more + often + <braunr> on linux, a small part of the kernel space is reserved for + temporary mappings, which need page table updates too + <braunr> but most allocations don't use that + <braunr> it's complicated + <braunr> also, i plan to make virtual memory operations completely + concurrent on x15, similar to what is described in radixvm + <rah> ok + <braunr> which means mapping operations on non overlapping regions won't be + serialized + <braunr> a big advantage for microkernels which base their messaging + optimizations on mapping + <braunr> so simply put, better performance because of simpler ipc and data + structures, and better scalability because of improved data structure + algorithms and concurrency + <rah> tschwinge: yes but that page is no use to someone who wants a summary + of what distinguishes x15 + <braunr> x15 is still far from complete, which is why i don't advertise it + other than here + <rah> "release early, release often"? + <braunr> give it a few more years :p + <braunr> release what ? + <braunr> something that doesn't work ? + <rah> software + <rah> yes + <braunr> this release early practice applies to maintenance + <rah> release something that doesn't work so that others can help make it + work + <braunr> not big developments + <braunr> i don't want that for now + <braunr> i have a specific idea of what i want, and both explaining and + defending it would take time, better spent in development itself + <braunr> just wait for a first prototype + <braunr> and then you'll see if you want to help or not + * rah does not count himself as one of the "others" who might help make it + work + <braunr> one big difference with other microkernels is that x15 is + specifically intended to run a unix like system + <braunr> a hurd like system providing a psoix interface more accurately + <braunr> and efficiently + <braunr> so for example, while many microkernels provide only sync ipc, x15 + provides both sync ipc and signals + <braunr> and then, there are a lot of small optimizations, like port names + which will transparently identify as file descriptors + <braunr> light reference counting + <braunr> a restriction on ipc that only allows reliable transfers across + network to machines with same arch and endianness + <braunr> etc.. + <rah> http://darnassus.sceen.net/~hurd-web/microkernel/x15/ + <rah> please take note of the fact that this newly created page is not just + a dump of IRC logs + + +## IRC, freenode, #hurd, 2013-09-30 + + <braunr> rah: i'm uncomfortable with a page about x15 on the wiki ... + <braunr> there is a reason i don't want to advertise it for now + <braunr> and you're just completely ignoring it + <rah> braunr: detailed information about x15 is already included elsewhere + in the wiki + <braunr> rah: really ? + <rah> braunr: there is a section named "X15" on this page: + http://www.gnu.org/software/hurd/microkernel/mach/deficiencies.html + <braunr> rah: oh ok, but it's still small and hard to find ;p + <rah> braunr: "small"?! + <rah> braunr: the X15 section starts at about 10% down the page and + finishes at the bottom of the page + <rah> braunr: and the page is huge + <braunr> rah: hm ok, but that's still listed as mach deficiencies, not as + x15 itself + <rah> braunr: I heard about x15 + <rah> braunr: I wanted to learn about it + <rah> braunr: there was no easily accessible information for doing so + <rah> braunr: it's not unreasonable to want to learn about it, having heard + about it + <rah> braunr: others will want to learn about it + <azeem_> rah: please respect the developer's policy of how to advertise + their project + <rah> braunr: having learned about it myself, I've helped those who will + follow me by giving them the summary that I wanted + <rah> azeem_: I'm not disrespecting the developer's policy of how to + advertise their project; I'm not advertising their project + <azeem_> rah: maybe replace the wiki page by "If you would like to know + about X15, please contact <your email>" + <rah> azeem_: that's ridiculous + <braunr> rah: then ask me directly + <braunr> rah: don't make wiki pages + <rah> braunr: I don't understand what you mean + <rah> braunr: I have already asked you directly + <rah> braunr: I needed to ask you directly in order to make the wiki page + <azeem> rah: braunr does not like your wiki page, how hard is it to + understand? + <rah> azeem: my discussion is with braunr, not you + <braunr> rah: if someone wants to know more about x15, he can me directly, + no need for a wiki page + + +## IRC, freenode, #hurd, 2013-10-01 + + <rah> braunr: that's hyperbole; there's no "need" for a wiki, or for x15 or + even for the Hurd + <rah> braunr: a wiki page is helpful + <rah> useful, even + <braunr> rah: as azeem said, i'm just not willing to advertise outside this + channel for now + <braunr> it makes sense to mention it in the defficiencies page, since this + page talks about what's lacking in gnumach + <braunr> and the wiki is about the hurd, including gnumach + <rah> braunr: why does it make sense to mention it in the deficiencies page + but not in a dedicated page? + <braunr> rah: because gnumach is a hurd project, x15 isn't + <rah> braunr: what do you mean by "a hurd project"? + <rah> braunr: you're saying that x15 differs from gnumach in some way, and + that this difference is the reason it doesn't make sense to have a wiki + page devoted to x15; the phrase you've used to descibe that difference is + "a hurd project" but it's not clear what, exactly, you mean by that + <rah> braunr: could you explain what you mean by that? + <azeem> rah: this is getting off-topic, please take this conversation + elsewhere + <rah> azeem: that's a very tenuous statement + <rah> azeem: I think this is the appropriate place to discuss the matter + <azeem> I leave that to braunr to decide + <rah> azeem: I think *you* don't want the conversation to be had at all and + are attempting to censor it using a tenuous excuse + <azeem> no no, I'm not censoring it - I am just saying you should take it + elsewhere + <braunr> let's take it elsewhere + + +## IRC, freenode, #hurd, 2013-10-12 + + <zacts> braunr: are you still working on x15/propel? + * zacts checks the git logs + <braunr> zacts: taking a break for now, will be back on it when i have a + clearer view of the new vm system + + +## IRC, freenode, #hurd, 2013-10-15 + + <gnufreex> braunr, few questions about x15. I was reading IRC logs on hurd + site, and in the latest part, you say (or I misunderstood) that x15 is + now hybrid kernel. So what made you change design... or did you? + <braunr> gnufreex: i always intended to go for a hybrid + + +## IRC, freenode, #hurd, 2013-10-19 + + <zacts> braunr: when do you plan to start on x15/propel again? + <braunr> zacts: after i'm done with thread destruction on the hurd + +[[open_issues/libpthread/t/fix_have_kernel_resources]]. + + <zacts> and do you plan to actually run hurd on top of x15, or are you + still going to reimplement hurd as propel? + <braunr> and no, i don't intend to run the hurd on top of x15 + + +## IRC, freenode, #hurd, 2013-10-24 + + <neal> braunr: What is your Mach replacement doing? + <braunr> "what" ? :) + <braunr> you mean how i guess + <neal> Sure. + <braunr> well it's not a mach replacement any more + <braunr> and for now it's stalled while i'm working on the hurd + <neal> that could be positive :) + <braunr> it's in good shape + <neal> how did it diverge? + <braunr> sync ipc, with unix-like signals + <braunr> and qnx-like bare data messages + <neal> hmm, like okl5? + <braunr> (with scatter gather) + <neal> okl4 + <braunr> yes + <braunr> btw, if you can find a document that explains this property of + okl4, i'm interested, since i can't find it again on my own :/ + <braunr> basically, x15 has a much lighter ipc interface + <neal> capabilities? + <braunr> mach ports are mostly retained + <braunr> but reference counting will be simplified + <neal> hmm + <neal> I don't like the reference counting part + <braunr> port names will be plain integers, to directly be usable as file + descriptors and avoid a useless translation layer + <braunr> (same as in qnx) + <neal> this sounds like future tense + <braunr> there is no ipc code yet + <neal> so I guess this stuff is not implemented + <neal> ok. + <braunr> next step is virtual memory + <braunr> and i'm taking my time because i want it to be a killer feature + <neal> so if you don't IPC and you don't have VM, what do you have? :) + <braunr> i have multiprocessor multithreading + <neal> I see. + <braunr> mutexes, condition variables, rcu-like lockless synchronization, + work queues + <braunr> basic bsd-like virtual memory + <braunr> which i want to rework + <neal> I ignored all of that in Viengoos :) + <braunr> and since ipc will still depend on virtual memory for zero-copy, i + want the vm system to be right + <braunr> well, i'm more interested in the implementation than the + architecture + <braunr> for example, i have unpublished code that features a lockless + radix tree for vm_object lookups + <braunr> that's quite new for a microkernel based system, but the ipc + interface itself is very similar to what already exists + <braunr> your half-sync ipc are original :) + <neal> I'm considering getting back in the OS game. + <braunr> oh + <neal> But, I'm not going to write a kernel this time. + <braunr> did anyone here consider starting a company for such things, like + genode did ? + <neal> I was considering using genode as a base. + <braunr> neal: why genode ? + <neal> I want to build a secure system. + <neal> I think the best way to do that is using capabilities. + <neal> Genode runs on Fiasco.OC, for instance + <neal> and it provides a lot of infrastructure + <braunr> neal: why not l4re for example ? + <braunr> neal: how important is the ability to revoke capabilities ? + +In the discussion on [[community/gsoc/project_ideas/object_lookups]], *IRC, +freenode, #hurd, 2013-10-24*: + + <teythoon> and, with some effort, getting rid of the hash table lookup by + letting the kernel provide the address of the object (iirc neil knew the + proper term for that) + <braunr> teythoon: that is a big interface change + <teythoon> how so + <braunr> optimizing libihash and libpthread should already be a good start + <braunr> well how do you intend to add this information ? + <braunr> ok, "big" is overstatement, but still, it's a low level interface + change that would probably break a lot of things + <teythoon> store a pointer in the port structure in gnumach, make that + accessible somehow + <braunr> yes but how ? + <teythoon> interesting question indeed + <braunr> my plan for x15 is to make this "label" part of received messages + <braunr> which means you need to change the format of messages + <braunr> that is what i call a big change diff --git a/microkernel/mach/gnumach/boot_trace.mdwn b/microkernel/mach/gnumach/boot_trace.mdwn index 7b729c23..ea999a9b 100644 --- a/microkernel/mach/gnumach/boot_trace.mdwn +++ b/microkernel/mach/gnumach/boot_trace.mdwn @@ -227,3 +227,25 @@ License|/fdl]]."]]"""]] >> vm\_pageout >> Does not return. + + +# IRC, freenode, #hurd, 2013-10-07 + + <cureOS> look, where should i dig or where from should i start from, if i + have desire to know how the kernel was written from baremetal? Can it be + ever done nowadays? + <youpi> cureOS: the boot entry of the kernel is i386/i386at/boothdr.S , + boot_entry + <youpi> that's what grub jumps to + <youpi> then that jumps to c_boot_entry + <youpi> and everything else is C + <cureOS> grub loads it somehow. how does it prepare cpu and memoty, cpu + cache control if any... segments for stack.. + <youpi> see the grub documentation + <youpi> basically it's all flat linear space + <cureOS> does kernel transform it after that? + <youpi> see the ldt/gdt initialization + <youpi> from i386at_init and children + <youpi> nothing much fancy, a kernel cs/ds, and user cs/ds + <braunr> and paging, naturally + <youpi> sure diff --git a/microkernel/mach/message/msgh_id.mdwn b/microkernel/mach/message/msgh_id.mdwn index 799ed5cc..b20f1fe0 100644 --- a/microkernel/mach/message/msgh_id.mdwn +++ b/microkernel/mach/message/msgh_id.mdwn @@ -15,6 +15,8 @@ files. [[!toc]] +See also [[open_issues/versioning]]. + # IRC, freenode, #hurd, 2012-07-12 diff --git a/news/2011-q2-ps.mdwn b/news/2011-q2-ps.mdwn index 28d2bbb0..f62a23ae 100644 --- a/news/2011-q2-ps.mdwn +++ b/news/2011-q2-ps.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -29,7 +29,6 @@ have been running all over the net: * {{$news/2011-q2#h-online}}; * {{$news/2011-q2#innocenthacker}}; * {{$news/2011-q2#netzwelt}}; - * {{$news/2011-q2#operation-tunnelbau}}; * {{$news/2011-q2#osnews}}; * {{$news/2011-q2#pro-linux}}; * {{$news/2011-q2#reddit-1}}; diff --git a/news/2011-q2.mdwn b/news/2011-q2.mdwn index f4dba68c..1c677670 100644 --- a/news/2011-q2.mdwn +++ b/news/2011-q2.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -71,83 +71,87 @@ his Java/Hurd Google Summer of Code work. golem: - "Sebastian Grüner (golem): [Debian 7 kommt offiziell mit Hurd als - Kernel \[de\]](http://www.golem.de/1107/84947.html)" + "*[Debian 7 kommt offiziell mit Hurd als Kernel + \[de\]](http://www.golem.de/1107/84947.html)*, Golem.de, Sebastian Grüner, + 2011-07-14" h-online: - "Dj Walker-Morgan (The H Open): [Hurd Progresses - Debian GNU/Hurd by end of - 2012?](http://www.h-online.com/open/news/item/Hurd-Progresses-Debian-GNU-Hurd-by-end-of-2012-1279253.html)" + "*[Hurd Progresses - Debian GNU/Hurd by end of + 2012?](http://www.h-online.com/open/news/item/Hurd-Progresses-Debian-GNU-Hurd-by-end-of-2012-1279253.html)*, + The H Open, Dj Walker-Morgan, 2011-07-14" innocenthacker: - "Amit Khajuria (Innocent Hacker): [Debian 7 might come in a GNU Hurd - version](http://www.innocenthacker.com/2011/07/debian-7-might-come-in-gnu-hurd-version.html)" + "*[Debian 7 might come in a GNU Hurd + version](http://www.innocenthacker.com/2011/07/debian-7-might-come-in-gnu-hurd-version.html)*, + Innocent Hacker, Amit Khajuria, 2011-07-17" lwn: - "Joe 'Zonker' Brockmeier (LWN): [Signs of life from GNU - Hurd](http://lwn.net/Articles/452296/)" + "*[Signs of life from GNU Hurd](http://lwn.net/Articles/452296/)*, LWN, Joe + 'Zonker' Brockmeier, 2011-07-20" netzwelt: - "Markus Franz (netzwelt) [Debian 7.0 Wheezy: Erste Pläne für Hurd statt - Linux-Kernel - \[de\]](http://www.netzwelt.de/news/87551-debian-7-0-wheezy-erste-plaene-hurd-statt-linux-kernel.html)" - -operation-tunnelbau: - - "Jens Reil: [Hurd kommt zusammen mit Duke Nukem - Forever. Fast. \[de\]](http://blog.operation-tunnelbau.de/archives/66-Hurd-kommt-zusammen-mit-Duke-Nukem-Forever.-Fast..html)" + "*[Debian 7.0 Wheezy: Erste Pläne für Hurd statt Linux-Kernel + \[de\]](http://www.netzwelt.de/news/87551-debian-7-0-wheezy-erste-plaene-hurd-statt-linux-kernel.html)*, + Netzwelt, Markus Franz, 2011-07-15" osnews: - "Thom Holwerda (OS News): [GNU Hurd Quarterly Status - Report](http://www.osnews.com/story/24942/GNU_Hurd_Quarterly_Status_Teport)" + "*[GNU Hurd Quarterly Status + Report](http://www.osnews.com/story/24942/GNU_Hurd_Quarterly_Status_Teport)*, + OS News, Thom Holwerda, 2011-07-13" phoronix-1: - "Michael Larabel (Phoronix): [A Status Update On GNU Hurd: Java, Debian, - Money](http://www.phoronix.com/scan.php?page=news_item&px=OTY2Nw)" + "*[A Status Update On GNU Hurd: Java, Debian, + Money](http://www.phoronix.com/scan.php?page=news_item&px=OTY2Nw)*, Phoronix, + Michael Larabel, 2011-07-13" phoronix-2: - "Michael Larabel (Phoronix): [Coming Up: Benchmarks Of GNU - Hurd](http://www.phoronix.com/scan.php?page=news_item&px=OTY3NQ)" + "*[Coming Up: Benchmarks Of GNU + Hurd](http://www.phoronix.com/scan.php?page=news_item&px=OTY3NQ)*, Phoronix, + Michael Larabel, 2011-07-15" phoronix-3: - "Michael Larabel (Phoronix): [Test Driving GNU Hurd, With Benchmarks Against - Linux](http://www.phoronix.com/scan.php?page=article&item=debian_gnu_hurd)" + "*[Test Driving GNU Hurd, With Benchmarks Against + Linux](http://www.phoronix.com/scan.php?page=article&item=debian_gnu_hurd)*, + Phoronix, Michael Larabel, 2011-07-18" pro-linux: - "Hans-Joachim Baader (Pro Linux): [GNU Hurd will offiziell in nächste - Debian-Version - \[de\]](http://www.pro-linux.de/news/1/17261/gnu-hurd-will-offiziell-in-naechste-debian-version.html)" + "*[GNU Hurd will offiziell in nächste Debian-Version + \[de\]](http://www.pro-linux.de/news/1/17261/gnu-hurd-will-offiziell-in-naechste-debian-version.html)*, + Pro-Linux, Hans-Joachim Baader, 2011-07-13" reddit-1: - "TheSilentNumber (reddit): [RMS may finally shout \"It's alive!\" at GNU+HURD - thanks to - Debian](http://www.reddit.com/r/linux/comments/j2ztr/rms_mlayout_finally_shout_its_alive_at_gnuhurd_thanks/)" + "*[Debian now has concrete plans to bring GNU Hurd to the larger + community. It is expected to be released with the release of Debian 7.0 + Wheezy towards the end of 2012 or beginning of + 2013](http://www.reddit.com/r/linux/comments/ipxxt/debian_now_has_concrete_plans_to_bring_gnu_hurd/)*, + reddit, mepper, 2011-07-14" reddit-2: - "mepper (reddit): [Debian now has concrete plans to bring GNU Hurd to the - larger community. It is expected to be released with the release of Debian - 7.0 Wheezy towards the end of 2012 or beginning of - 2013](http://www.reddit.com/r/linux/comments/ipxxt/debian_now_has_concrete_plans_to_bring_gnu_hurd/)" + "*[RMS may finally shout \"It's alive!\" at GNU+HURD thanks to + Debian](http://www.reddit.com/r/linux/comments/j2ztr/rms_mlayout_finally_shout_its_alive_at_gnuhurd_thanks/)*, + reddit, TheSilentNumber, 2011-07-28" schmehl: - "Alexander \"Tolimar\" Reichle-Schmehl: [About Debian, The Hurd and Linux or - in short: Yes, we will still have a Linux - kernel](http://blog.schmehl.info/Debian/hurd-not-default)" + "*[About Debian, The Hurd and Linux or in short: Yes, we will still have a + Linux kernel](http://blog.schmehl.info/Debian/hurd-not-default)*, Alexander + \"Tolimar\" Reichle-Schmehl, 2011-07-15" slashdot: - "timothy (Slashdot): [Watch Out Linux, GNU Hurd - Coming](http://news.slashdot.org/story/11/07/14/2141229/watch-out-linux-gnu-hurd-coming)" + "*[Watch Out Linux, GNU Hurd + Coming](http://news.slashdot.org/story/11/07/14/2141229/watch-out-linux-gnu-hurd-coming)*, + Slashdot, timothy, 2011-07-14" """]] diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn index b0c95612..edb2dccd 100644 --- a/open_issues/64-bit_port.mdwn +++ b/open_issues/64-bit_port.mdwn @@ -155,3 +155,10 @@ In context of [[mondriaan_memory_protection]]. <braunr> the problem is the interfaces themselves <braunr> type widths <braunr> as passed between userspace and kernel + + +# IRC, OFTC, #debian-hurd, 2013-10-05 + + <dharc> and what about 64 bit support, almost done? + <youpi> kernel part is done + <youpi> MIG 32/64 trnaslation missing diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index ba72b00f..a3c55063 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -803,3 +803,11 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <braunr> and hoping it didn't corrupt something important like file system caches before being flushed <giuscri> kilobug, braunr : mhn, ook + + +# IRC, freenode, #hurd, 2013-10-13 + + <ahungry> ahh, ^c isn't working to cancel a ping - is there alternative? + <braunr> ahungry: ctrl-c does work, you just missed something somewhere and + are running a shell directly on a console, without a terminal to handle + signals diff --git a/open_issues/boehm_gc.mdwn b/open_issues/boehm_gc.mdwn index 623dcb83..0a476d71 100644 --- a/open_issues/boehm_gc.mdwn +++ b/open_issues/boehm_gc.mdwn @@ -523,3 +523,22 @@ restults of GNU/Linux and GNU/Hurd look very similar. <congzhang> hi, I am dotgnu work on hurd, and even winforms app <congzhang> s/am/make <congzhang> and maybe c# hello world translate another day :) + + +## Leak Detection + +### IRC, freenode, #hurd, 2013-10-17 + + <teythoon> I spent the last two days integrating libgc - the boehm + conservative garbage collector - into hurd + <teythoon> it can be used in leak detection mode + <azeem> whoa, cool + <teythoon> and it actually kind of works, finds malloc leaks in translators + <braunr> i think there were problems with signal handling in libgc + <braunr> i'm not sure we support nested signal handling well + <teythoon> yes, I read about them + <teythoon> libgc uses SIGUSR1/2, so any program installing handlers on them + will break + <azeem> (which is not a problem on Linux, cause there some RT-signals or so + are used) + <teythoon> yes diff --git a/open_issues/code_analysis/discussion.mdwn b/open_issues/code_analysis/discussion.mdwn index 7ac3beb1..4cb03293 100644 --- a/open_issues/code_analysis/discussion.mdwn +++ b/open_issues/code_analysis/discussion.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -42,6 +43,8 @@ License|/fdl]]."]]"""]] <braunr> i tried duma, and it crashes, probably because of cthreads :) +# Static Analysis + ## IRC, freenode, #hurd, 2012-09-08 <mcsim> hello. What static analyzer would you suggest (probably you have @@ -49,3 +52,54 @@ License|/fdl]]."]]"""]] <braunr> mcsim: if you find some good free static analyzer, let me know :) <pinotree> a simple one is cppcheck <mcsim> braunr: I'm choosing now between splint and adlint + + +## IRC, freenode, #hurd, 2013-10-17 + + <teythoon> whoa, llvm kinda works, enough to make scan-build work :) + <braunr> teythoon: what is scan-build ? + <teythoon> braunr: clangs static analyzer + <braunr> ok + <teythoon> I'm doing a full build of the hurd using it, I will post the + report once it is finished + <teythoon> this will help spot many problems + <teythoon> well, here are the scan-build reports I got so far: + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/ + <teythoon> I noticed it finds problems in mig generated code, so there are + probably lot's of duplictaes for those kind of problems + <pinotree> what's a... better one to look at? + <teythoon> it's also good at spotting error handling errors, and can spot + leaks sometimes + <teythoon> hm + <teythoon> + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-yVBHO1.html + <braunr> that's minor, the device always exist + <braunr> but that's still ugly + <teythoon> + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-MtgWSa.html + <teythoon> + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-QdsZIm.html + <teythoon> this could be important: + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-PDMEbk.html + <teythoon> this is the issue it finds in mig generated server stubs: + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-iU3soc.html + <braunr> this one is #if TypeCheck1 + <braunr> the libports one looks weird indeed + <teythoon> but TypeCheck is 1 (the tooltip shows macro expansion) + <teythoon> it is defined in line 23 + <braunr> oh + <teythoon> hmmm... clang does not support nested functions, that will limit + its usefulness for us :/ + <braunr> yes + <braunr> one more reason not to use them + + +### IRC, freenode, #hurd, 2013-10-18 + + <teythoon> more complete, now with index: + https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build-2/ + + +# Leak Detection + +See *Leak Detection* on [[boehm_gc]]. diff --git a/open_issues/dbus.mdwn b/open_issues/dbus.mdwn index a41515a1..4473fba0 100644 --- a/open_issues/dbus.mdwn +++ b/open_issues/dbus.mdwn @@ -253,3 +253,115 @@ See [[glibc]], *Missing interfaces, amongst many more*, *`SOCK_CLOEXEC`*. to know how to find this sendmsg.c file? <pinotree> (it's in glibc, but otherwise the remark is valid) <pinotree> s/otherwise/anyway/ + + +# Emails + +# IRC, freenode, #hurd, 2013-10-16 + + <braunr> gnu_srs: how could you fail to understand credentials need to be + checked ? + <gnu_srs> braunr: If data is sent via sendmsg, no problem, right? + <braunr> gnu_srs: that's irrelevant + <gnu_srs> It's just to move the check to the receive side. + <braunr> and that is the whole problem + <braunr> it's not "just" doing it + <braunr> first, do you know what the receive side is ? + <braunr> do you know what it can be ? + <braunr> do you know where the corresponding source code is to be found ? + <gnu_srs> please, describe a scenario where receiving faulty ancillary data + could be a problem instead + <braunr> dbus + <braunr> a user starting privileged stuff although he's not part of a + privileged group of users for example + <braunr> gnome, kde and others use dbus to pass user ids around + <braunr> if you can't rely on these ids being correct, you can compromise + the whole system + <braunr> because dbus runs as root and can give root privileges + <braunr> or maybe not root, i don't remember but a system user probably + <pinotree> "messagebus" + <gnu_srs> k! + <braunr> see http://www.gnu.org/software/hurd/open_issues/dbus.html + <braunr> IRC, freenode, #hurd, 2013-07-17 + <braunr> <teythoon> and the proper fix is to patch pflocal to query the + auth server and add the credentials? + <braunr> <pinotree> possibly + <braunr> <teythoon> that doesn't sound to bad, did you give it a try? + + +# IRC, freenode, #hurd, 2013-10-22 + + <gnu_srs> I think I have a solution on the receive side for SCM_CREDS :) + + <gnu_srs> A question related to SCM_CREDS: dbus use a zero data byte to get + credentials sent. + <gnu_srs> however, kfreebsd does not care which data (and credentials) is + sent, they report the credentials anyway + <gnu_srs> should the hurd implementation do the same as kfreebsd? + <youpi> gnu_srs: I'm not sure to understand: what happens on linux then? + <youpi> does it see zero data byte as being bogus, and refuse to send the + creds? + <gnu_srs> linux is also transparent, it sends the credentials independent + of the data (but data has to be non-null) + <youpi> ok + <youpi> anyway, what the sending application writes does not matter indeed + <youpi> so we can just ignore that + <youpi> and have creds sent anyway + <braunr> i think the interface normally requires at least a byte of data + for ancilliary data + <youpi> possibly, yes + <braunr> To pass file descriptors or credentials over a SOCK_STREAM, + you need to send or + <braunr> receive at least one byte of non-ancillary data in + the same sendmsg(2) or + <braunr> recvmsg(2) call. + <braunr> but that may simply be linux specific + <braunr> gnu_srs: how do you plan on implementing right checking ? + <gnu_srs> Yes, data has to be sent, at least one byte, I was asking about + e.g. sending an integer + <braunr> just send a zero + <braunr> well + <braunr> dbus already does that + <braunr> just don't change anything + <braunr> let applications pass the data they want + <braunr> the socket interface already deals with port rights correctly + <braunr> what you need to do is make sure the rights received match the + credentials + <gnu_srs> The question is to special case on a zero byte, and forbid + anything else, or allow any data. + <braunr> why would you forbid + <braunr> ? + <gnu_srs> linux and kfreebsd does not special case on a received zero byte + <braunr> same question, why would you want to do that ? + <gnu_srs> linux sends credentials data even if no SCM_CREDENTIALS structure + is created, kfreebsd don't + <braunr> i doubt that + <gnu_srs> To be specific:msgh.msg_control = NULL; msgh.msg_controllen = 0; + <braunr> bbl + <gnu_srs> see the test code: + http://lists.debian.org/debian-hurd/2013/08/msg00091.html + <braunr> back + <braunr> why would the hurd include groups when sending a zero byte, but + only uid when not ? + <gnu_srs> ? + <braunr> 1) Sent credentials are correct: + <braunr> no flags: Hurd: OK, only sent ids + <braunr> -z Hurd: OK, sent IDs + groups + <braunr> and how can it send more than one uid and gid ? + <braunr> "sent credentials are not honoured, received ones are created" + <gnu_srs> Sorry, the implementation is changed by now. And I don't special + case on a zero byte. + <braunr> what does this mean ? + <braunr> then why give me that link ? + <gnu_srs> The code still applies for Linux and kFreeBSD. + <gnu_srs> It means that whatever you send, the kernel emits does not read + that data: see + <gnu_srs> socket.h: before struct cmsgcred: the sender's structure is + ignored ... + <braunr> do you mean receiving on a socket can succeed with gaining + credentials, although the sender sent wrong ones ? + <gnu_srs> Looks like it. I don't have a kfreebsd image available right now. + <gnu_srs> linux returns EPERM + <braunr> anyway + <braunr> how do you plan to implement credential checking ? + <gnu_srs> I'll mail patches RSN diff --git a/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn b/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn index e3a6b648..3faa56fc 100644 --- a/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn +++ b/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,8 +13,22 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gdb open_issue_gnumach]] +[[!toc]] -# IRC, freenode, #hurd, 2011-07-14 + +# Memory Map + +## IRC, freenode, #hurd, 2010-06 (?) + + <jkoenig> is there a way to get gdb to map addresses as required when + debugging mach with qemu ? + <jkoenig> I can examine the data if I manually map the addresses th + 0xc0000000 but maybe there's an easier way... + <youpi> jkoenig: I haven't found a way + <youpi> I'm mostly using the internal kdb + + +## IRC, freenode, #hurd, 2011-07-14 <mcsim> Hello. I have problem with debugging gnumach. I set 2 brakepoints in file i386/i386at/model_dep.c on functions gdt_init and idt_init. Then @@ -114,3 +129,18 @@ License|/fdl]]."]]"""]] <antrik> oh, right, without GDB... <antrik> though if that's what he meant, his statement was very misleading at least + + +# Multiboot + +See also discussion about *multiboot* on [[arm_port]]. + + +## IRC, freenode, #hurd, 2013-10-09 + + <matlea01> I was just wondering - once gnumach is compiled and I have the + gnumach elf, is that bootable? I.e. can I use something like + "qemu-system-i386 -kernel gnumach"? + <kilobug> matlea01: you need something with multiboot support (like grub) + to provide the various bootstrap modules to the kernel + <matlea01> Ah, I see diff --git a/open_issues/emacs.mdwn b/open_issues/emacs.mdwn index cdd1b10d..749649be 100644 --- a/open_issues/emacs.mdwn +++ b/open_issues/emacs.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -1525,3 +1525,18 @@ perhaps prepared (I did not yet have a look), and re-tries again and again? Why doesn't Mach page out some pages to make memory available? This is stock GNU Mach from Git, no patches, configured for Xen domU usage. + + +# IRC, freenode, #hurd, 2013-10-04 + + <pinotree> given you are an emacs user: could you please pick the build + patch from deb#725099, recompile emacs24 and test it with your daily + work? + + +## IRC, freenode, #hurd, 2013-10-07 + + <gnu_srs> Wow! emacs24 runs in X:-D + <gnu_srs> pinotree: I've now built and installed emacs 24.3. So far so good + ^ + <pinotree> good, keep testing and stressing diff --git a/open_issues/exec_memory_leaks.mdwn b/open_issues/exec_memory_leaks.mdwn index 67281bdc..1fc5a928 100644 --- a/open_issues/exec_memory_leaks.mdwn +++ b/open_issues/exec_memory_leaks.mdwn @@ -94,3 +94,28 @@ After running the libtool testsuite for some time: 8 39.5 0:15.60 28:48.57 9 0.0 0:04.49 10:24.12 10 12.8 0:08.84 19:34.45 + + +# IRC, freenode, #hurd, 2013-10-08 + + * braunr hunting the exec leak + <braunr> and i think i found it + <braunr> yes :> + <braunr> testing a bit more and committing the fix later tonight + <braunr> pinotree: i've been building glibc for 40 mins and exec is still + consuming around 1m memory + <pinotree> wow nice + <pinotree> i've been noticing exec leaking quite some time ago, then forgot + to pay more attention to that + <braunr> it's been more annoying since darnassus provides web access to + cgis + <braunr> automated tools make requests every seconds + <braunr> the leak occurred when starting a shell script or using system() + <braunr> youpi: not sure you saw it, i fixed the exec leak + + +## IRC, freenode, #hurd, 2013-10-10 + + <gg0> braunr: http://postimg.org/image/jd764wfpp/ + <braunr> exec 797M + <braunr> this should be fixed with the release of the next hurd packages diff --git a/open_issues/ext2fs_libports_reference_counting_assertion.mdwn b/open_issues/ext2fs_libports_reference_counting_assertion.mdwn index ff1c4c38..9ff43afa 100644 --- a/open_issues/ext2fs_libports_reference_counting_assertion.mdwn +++ b/open_issues/ext2fs_libports_reference_counting_assertion.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -91,3 +91,14 @@ With that patch in place, the assertion failure is seen more often. sure we can get that easily lol [[automatic_backtraces_when_assertions_hit]]. + + +# IRC, freenode, #hurd, 2013-10-09 + + <braunr> mhmm, i may have an explanation for the weird assertions we + sometimes see in ext2fs + <braunr> glibc uses alloca to reserve memory for one reply port per thread + in abort_all_rpcs + <braunr> if this erases the thread-specific area, we can expect all kinds + of wreckage + <braunr> i'm not sure how to fix this though diff --git a/open_issues/gdb_signal_handler.mdwn b/open_issues/gdb_signal_handler.mdwn index 3084f7e3..5e27a099 100644 --- a/open_issues/gdb_signal_handler.mdwn +++ b/open_issues/gdb_signal_handler.mdwn @@ -401,3 +401,74 @@ License|/fdl]]."]]"""]] <zyg> braunr: are you sure? there is minimal user-code run before the signal is going into the handler. <braunr> you "step out of the handler" + + +# IRC, freenode, #hurd, 2013-10-24 + + <gnu_srs> how come some executables are not debuggable with gdb, e.g Cannot + access memory at address xxx. -fPIC flag? + <braunr> no + <braunr> i'm not sure but it's certainly not -fPIC + <gnu_srs> Another example is localedef: ./debian/tmp-libc/usr/bin/localedef + -i en_GB -c -f UTF-8 -A /usr/share/locale/locale.alias en_GB.UTF-8 + segfailts + <gnu_srs> and in gdb hangs after creating a thread., after C-c no useful + info: stack ends with: Cannot access memory at address 0x8382c385 + <braunr> if it's on the stack, it's probably a stack corruption + <nalaginrut> gnu_srs: are u using 'x' command or 'print' in GDB? IIRC + print may throw such message, but x may not + <gnu_srs> bt + <braunr> x may too + <braunr> what you're showing looks like an utf-8 string + <braunr> c385 is Å + <braunr> 83 is a special f + <braunr> 82 is a comma + <gnu_srs> so the stack is corrupted:-( + <braunr> probably + <braunr> well, certainly + <braunr> but gdb should show you where the program counter is + <gnu_srs> is that: ECX: the count register + <braunr> no + <braunr> eip + <braunr> program counter == instruction pointer + <gnu_srs> k!, the program counter is at first entry in bt: #0 0x01082612 + in _hurd_intr_rpc_msg_in_trap () at intr-msg.c:133 + <braunr> this is the hurd interruptible version of mach_msg + <braunr> so it probably means the corruption was made by a signal handler + <braunr> which is one of the reasons why gdb can't handle Ctrl-c + <gnu_srs> what to do in such a case, follow the source code + single-stepping? + <braunr> single stepping also uses signals + <braunr> and using printf will probably create an infinite recursion + <braunr> in those cases, i use mach_print + <braunr> as a first step, you could make sure a signal is actually received + <braunr> and which one + <braunr> hmm + <braunr> also, before rushing into conclusions, make sure you're looking at + the right thread + <braunr> i don't expect localedef to be multithreaded + <braunr> but gdb sometimes just doesn't get the thread where the segfault + actually occurred + <gnu_srs> two threads: 1095.4 and 1095.5 (created when starting localedef + in gdb) + <braunr> no, at the time of the crash + <braunr> the second thread is always the signal thread + <gnu_srs> OK,in gdb the program hangs, interrupted by C-c, outside it + segfaults + <braunr> when you use bt to get the corrupted stack, you can also use info + threads and thread apply all bt + <gnu_srs> I did: http://paste.debian.net/61170/ + <braunr> ok so it confirms there is only one real application thread, the + main one + <braunr> and that the corruption probably occurs during signal handling + <gnu_srs> rpctrace (edited out non-printable characters): + http://paste.debian.net/61178/ + <gnu_srs> Ah, have to do it again as root;-) + <braunr> yes .. :p + <gnu_srs> new last part: http://paste.debian.net/61181/ + <braunr> so, there is a seek, then a stat, then a close perhaps (port + deallocation) and then a signal received (probably sigsegv) + <braunr> gnu_srs: when you try running it in gdb, do you get a sigkill ? + <braunr> damn, gdb on darnassus is bugged :-( + <gnu_srs> It hangs, interrupted with C-c. + <braunr> ok diff --git a/open_issues/git-core-2.mdwn b/open_issues/git-core-2.mdwn index cbf47bd2..a92b3ebb 100644 --- a/open_issues/git-core-2.mdwn +++ b/open_issues/git-core-2.mdwn @@ -61,6 +61,113 @@ Fixing this situation is easy enough: Still seen. +## IRC, freenode, #hurd, 2013-10-10 + + <sea`> Huh? I've cloned the 'hurd' repository and I'm attempting to compile + it, but the 'rtnetlink.h' header in + 'hurd/pfinet/linux-src/include/linux/' is just blank. (Which leads to an + error later down when a macro that's supposed to be defined in there is + first used) + <sea`> So I'm just wondering, is that file really blank? Or is this some + unexpected error of decompression? + <braunr> clone again and see + <braunr> the file is definitely not empty + <sea`> I cloned it twice, both have that file blank. BUT, I want to point + out that both clones do have some decompression errors. (Some files are + missing chunks in /both/ cloned repositories). + <braunr> where did you clone it from ? + <sea`> git.sv.gnu.org/hurd/hurd.git + <braunr> hum decompression errors ? + <braunr> can you paste them please ? + <sea`> Hmm, I can clone again and show you an example if I find one + <sea`> This was on the hurd. When I run: git clone $repo;, it seems to fail + almost randomly with "incorrect header check", but when it does succeed, + occasionally some files are missing chunks + <sea`> and apparently entire files can be blank + <braunr> http or git ? + <sea`> git. + <braunr> that's really weird + <braunr> actually i don't even have problems with http any more nowadays .. + <sea`> This is using the hurd image from sthibault + <sea`> So once I get it recompiled and shuffle in the new binaries, the + problem should probably go away + <braunr> no + <braunr> well maybe but + <braunr> don't recompile + <braunr> upgrade packages instead + <sea`> Alright, I'll do an upgrade instead. Why that path specifically? + <braunr> rebuilding is long + <braunr> i wonder if the image you got is corrupted + <braunr> compute the checksum + <braunr> we've had weird reports in the past about the images he provides + <braunr> well not the images themselves, but differences after dowloading + .. + <braunr> downloading* + <sea`> The MD5SUMS file on his site isn't including the values for the most + recent images. + <sea`> It stops at 2012-12-28 + <braunr> hummm + <sea`> Anyway, let's see. git clone failed again: + <sea`> Receiving objects: 100% (50955/50955), 15.48 MiB | 42 KiB/s, done. + <sea`> error: inflate: date stream error (incorrect header check) <- This + is the interesting part + <sea`> fatal: serious inflate inconsistency + <sea`> fatal: index-pack failed + <braunr> not intereseting enough unfortunately + <braunr> but it might come from savannah too + <braunr> try the mirrors at + http://darnassus.sceen.net/gitweb/?a=project_list;pf=savannah_mirror + <sea`> Let's see..if I try: 'git clone + git://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git', I get: + 'fatal: remote error: access denied or repository not exported: + /gitweb/savannah_mirror/hurd.git' + <braunr> my bad + <braunr> that's weird, it should work .. + <braunr> oh, stupid translation error + <sea`> translation? From one human language to another? + <braunr> not translation actually + <braunr> typo :) + <braunr> it's either + <braunr> git://darnassus.sceen.net/savannah_mirror/hurd.git + <braunr> or + <braunr> http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git + <braunr> copy paste the url exactly please + <braunr> /gitweb/ is only present in the http url + <sea`> Ah, right. Okay, I'll paste it exactly + <sea`> Ehm. The whole thing locked up badly. I'll reboot it and try again. + <braunr> are you sure it locked oO ? + <braunr> the hurd can easily become unresponsive when performing io + operations + <braunr> but you need more than such a git repository to reach that state + <sea`> Yeah, that happens occasionally. It's not related to git, but rather + it happens when I cancel some command. + <braunr> your image must be corrupted + <braunr> have you enabled host io caching btw ? + <sea`> By now it's corrupted for sure..everytime it crashes the filesystem + gets into a weird state. + <sea`> I'll unpack a fresh image, then update the packages, and then try + cloning this git repository. + <braunr> i'll get the image too so we can compare sums + <sea`> 957bb0768c9558564f0c3e0adb9b317e ./debian-hurd.img.tar.gz + <sea`> Which unpacks to: debian-hurd-20130504.img + <azeem_> the NSA might backdoor the Hurd, in anticipation of our scheduled + world-dominance + <braunr> for now they're doing it passively : + <braunr> :p + <braunr> sea`: same thing here + <braunr> sea`: if you still have problems, the image itself might be wrong + <braunr> in which case you should try with the debian network installer + <sea`> Ah, so if problems persist, try with the network installer. Okay + <sea`> Is there some recipe for constructing a hurd/mach minimal + environment? + <sea`> A system with only just enough tools and libraries to compile and + poke at things. + <braunr> not currently + <braunr> we all work in debian environments + <braunr> the reason being that a lot of patches are queued for integration + upstream + + # 2010-11-17 A very similar issue. The working tree had a lot of diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index b453b44f..f2dc3159 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -18,6 +18,8 @@ Here's what's to be done for maintaining glibc. # [[General information|/glibc]] +## [[Versioning]] + # [[Sources|source_repositories/glibc]] @@ -330,6 +332,33 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 clearly not a priority <nalaginrut> ok + IRC, freenode, #hurd, 2013-09-26: + + <nalaginrut> if I want to have epoll/kqueue like things, where + should it dwell? kernel or some libs? + <braunr> libs + <pinotree> userland + <braunr> that would be a good project to work on, something i + intended to do (so i can help) but it requires a lot of work + <braunr> you basically need to add a way to explicitely install and + remove polling requests (instead of the currently way that + implicitely remove polling requests when select/poll returns) + <braunr> while keeping the existing way working for some time + <braunr> glibc implements select + <braunr> the hurd io interface shows the select interface + <braunr> servers such as pfinet/pflocal implement it + <braunr> glibc implements the client-side of the call + <nalaginrut> where's poll? since epoll just added edge-trigger in + poll + <braunr> both select and poll are implemented on top of the hurd io + select call (which isn't exactly select) + <braunr> + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/blob/HEAD:/hurd/io.defs + <braunr> this is the io interface + <braunr> + http://darnassus.sceen.net/gitweb/savannah_mirror/glibc.git/blob/refs/heads/tschwinge/Roger_Whittaker:/hurd/hurdselect.c + <braunr> this is the client side implementation + * `sys/eventfd.h` * `sys/inotify.h` @@ -854,6 +883,298 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 <braunr> to check where those locks are held and determine the right order + IRC, OFTC, #debian-hurd, 2013-09-28: + + <gg0_> now we'd just need tls + <gg0_> http://bugs.ruby-lang.org/issues/8937 + <gg0_> well, it would pass makecheck at least. makecheckall would + keep hanging on threads/pipes tests i guess, unless tls/thread + destruction patches fix them + + IRC, OFTC, #debian-hurd, 2013-10-05: + + <youpi> so what is missing for ruby2.0, only disabling use of + context for now, no? + <pinotree> i'm not tracking it closely, gg0_ is + <gg0_> maybe terceiro would accept a patch which only disables + *context, "maybe" because he rightly said changes must go + upstream + <gg0_> anyway with or without *context, many many tests in + makecheckall fail by making it hang, first with and without + assertion you removed, now they all simply hang + <gg0_> youpi: what do we want to do? if you're about finishing tls + migration (as i thought a couple of weeks ago), i won't propose + anything upstream. otherwise i could but that will have to be + reverted upstream once you finish + <gg0_> about tests, current ruby2.0 doesn't run makecheckall, only + makecheck which succeeds on hurd (w/o context) + <gg0_> if anyone wants to give it a try: + http://paste.debian.net/plain/51089 + <gg0_> first hunk makes makecheck (not makecheckall) succeed and + has been upstreamed, not packaged yet + <pinotree> what about makecheckall for ruby2.0? + <gg0_> 16:58 < gg0_> anyway with or without *context, many many + tests in makecheckall fail by making it hang, first with and + without assertion you removed, now they all simply hang + <pinotree> i for a moment thought it as for 1.9.1, ok + <pinotree> these hangs should be debugged, yes + <gg0_> nope, tests behavior doesn't change between 1.9 and 2.0. i + started suppressing tests onebyone on 2.0 as well and as happened + on 1.9, i gave up cause there were too many + <gg0_> yep a smart mind could start debugging them, starting from + patch above pasted by a lazy one owner + <gg0_> one problem is that one can't reproduce them by isolate + them, they don't fail. start makecheckall then wait for one fail + <gg0_> now after my stupid report, someone like pinotree could take + it over, play with it for half an hour/an hour (which equals to + half a gg0's year/a gg0's year + <gg0_> ) + <gg0_> and fix them all + + <gg0_> 17:05 < gg0_> youpi: what do we want to do? if you're about + finishing tls migration (as i thought a couple of weeks ago), i + won't propose anything upstream. otherwise i could but that will + have to be reverted upstream once you finish + <youpi> gg0_: I don't really know what to answer + <youpi> that's why I didn't answer :) + <gg0_> youpi: well then we could upstream context disable and keep + it disabled even if you fix tls. ruby won't be as fast as it + would be with context but i don't think anyone will complain + about that. then once packaged, if terceiro doesn't enable + makecheckall, we will have ruby2.0 in main + <youpi> that can be a plan yes + <gg0_> btw reverting it upstream should not be a problem eventually + <youpi> sure, the thing is remembering to do it + <gg0_> filed http://bugs.ruby-lang.org/issues/8990 + <gg0_> please don't fix tls too soon :) + <gg0_> s/makecheck/maketest/g + + IRC, OFTC, #debian-hurd, 2013-10-08: + + <gg0_> ok. *context disabled http://bugs.ruby-lang.org/issues/8990 + + <gg0> bt full of an attached stuck ruby test + http://paste.debian.net/plain/53788/ + <gg0> anything useful? + <youpi> uh, is that really all? + <youpi> there's not much interesting unfortunately + <youpi> did you run thread apply all bt full ? + <youpi> (not just bt full) + <gg0> no just bt full + <gg0> http://paste.debian.net/plain/53790/ + <gg0> wait, there's a child + <gg0> damn ctrl-c'ing while it was loading symbols made it crash :/ + <gg0> restarted testsuite + <gg0> isn't it interesting that failed tests fail only if testsuite + runs from beginning, whereas if run singularly, they succeed? + <gg0> as it got out of whatever resources + <gg0> youpi: http://paste.debian.net/plain/53798/ + <youpi> the interesting part is actually right at the top + <youpi> it's indeed stuck in the critical section spinlock + <youpi> question being what is keeping it + <youpi> iirc I had already checked in the whole glibc code that all + paths which lock critical_section_lock actually release it in + all cases, but maybe I have missed some + <youpi> (I did find some missing paths, which I fixed) + <gg0> i guess the same check you and braunr talk about in + discussion just before this anchor + http://darnassus.sceen.net/~hurd-web/open_issues/glibc/#recvmmsg + <youpi> yes, but the issue we were discussing there is not what + happens here + <youpi> we would see another thread stuck in the other way roudn, + otherwise + <gg0> no way to get what is locking? + <youpi> no, that's not recorded + <gg0> and what about writing it somewhere right after getting the + lock? + <youpi> one will have to do that in all spots taking that lock + <youpi> but yes, that's the usual approach + <gg0> i would give it try but eglibc rebuild takes too much time, + that conflicts with my laziness + <gg0> i read even making locks timed would help + + IRC, OFTC, #debian-hurd, 2013-10-09: + + <gg0> so correct order would be: + <gg0> __spin_lock (&ss->lock); // locks sigstate + <gg0> __spin_lock (&ss->critical_section_lock); + <gg0> [do critical stuff] + <gg0> __spin_unlock (&ss->critical_section_lock); + <gg0> __spin_unlock (&ss->lock); // unlocks sigstate + <gg0> ? + + <gg0> 21:44 < gg0> terceiro: backported to 2.0 (backport to 1.9 is + waiting) https://bugs.ruby-lang.org/issues/9000 + <gg0> 21:46 < gg0> that means that if you take a 2.0 snapshot, + it'll build fine on hurd (unless you introduce maketestall as in + 1.9, that would make it get stuck like 1.9) + <gg0> 21:48 < terceiro> gg0: nice + <gg0> 21:48 < terceiro> I will try to upload a snapshot as soon as + I can + <gg0> 21:52 < gg0> no problem. you might break my "conditional + satisfaction" by adding maketestall. better if you do that on + next+1 upload so we'll have at least one 2.0 built :) + + <gg0> would it be a problem granting me access to a porter box to + rebuild eglibc+ruby2.0? + <gg0> i'm already doing it on another vm but host often loses power + <pinotree> you cannot install random stuff on a porterbox though + <gg0> i know i'd just need build-deps of eglibc+ruby2.0 i guess + <gg0> (already accessed to porter machines in the past, account + lele, mips iirc) + <gg0> ldap should remember that + <gg0> don't want to disturb anyone else work btw. if it's not a + problem, nice. otherwise no problem + <pinotree> please send a request to admin@exodar.debian.net so it + is not forgotten + <gg0> following this one would be too "official"? + http://dsa.debian.org/doc/guest-account/ + <pinotree> hurd is not a release architecture, so hurd machines are + not managed by DSA + <gg0> ok + <pinotree> the general procedure outlines is ok though, just need + to be sent to the address above + <gg0> sent + <gg0> (1st signed mail with mutt, in the worst case i've attached + passphrase :)) + <youpi> gg0: could you send me an ssh key? + <pinotree> no alioth account? + <youpi> yes, but EPERM + <gg0> youpi: sent to youpi@ + <youpi> youpi@ ? + <gg0> (... which doesn't exist :/) + <gg0> sthibault@ + <youpi> please test gg0-guest@exodar.debian.net ? + <youpi> (I'd rather not adduser the ldap name, who knows what might + happen when you get your DD account) + <gg0> i'm in. thanks + <youpi> you're welcome + <gg0> ldap users need to be adduser'ed? + <youpi> I'm not getting your ldap user account from ud-replicate, + at least + <gg0> (btw i never planned to apply nm, i'd be honoured but i + simply think not to deserve it) + <youpi> never say never ;) + <gg0> bah i like failing. that would be a success. i can't :) + <gg0> gg0-guest@exodar:~$ dchroot + <gg0> E: Access not authorised + <gg0> I: You do not have permission to access the schroot service. + <gg0> I: This failure will be reported. + <youpi> ah, right, iirc I need to add you somewhere + <youpi> gg0: please retry? + <gg0> works + <youpi> good + <gg0> are there already eglibc+ruby2.0 build-deps? + <youpi> yes + <gg0> oh that means i should do something myself now :) + <youpi> yep, that had to happen at some point :) + <gg0> my laziness thanks: "at some point" is better than "now" :) + + IRC, freenode, #hurd, 2013-10-10: + + <gg0> ok just reproduced the + former. ../sysdeps/mach/hurd/jmp-unwind.c:53 waits + <braunr> 20:37 < braunr> gg0: does ruby create and destroy threads + ? + <gg0> no idea + <gg0> braunr: days ago you and youpi talked about locking order + (just before this anchor + http://darnassus.sceen.net/~hurd-web/open_issues/glibc/#recvmmsg) + <braunr> oh right + <gg0> <youpi> could you submit the fix for jmp-unwind.c to + upstream? + <braunr> it didn't made it in the todo list + <gg0> so correct order is in hurd_thread_cancel, right? + <braunr> sorry about that + <braunr> we need to make a pass to make sure it is + <gg0> that means locking first ss->critical_section_lock _then_ + ss->lock + <gg0> correct? + <braunr> but considering how critical hurd_thread_cancel is, i + expect so + + <gg0> i get the same deadlock by swapping locks + <gg0> braunr: youpi: fyi ^ + <gg0> 20:51 < braunr> 20:37 < braunr> gg0: does ruby create and + destroy threads ? + <gg0> how could i check it? + <braunr> gg0: ps -eflw + <youpi> gg0: that's not surprising, since in the b acktrace you + posted there isn't another thread locked in the other order + <youpi> so it's really that somehow the thread is already in + critical sesction + <braunr> youpi: you mean there is ? + <braunr> ah, it's not the same bug + <youpi> no, in what he posted, no other thread is stuck + <youpi> so it's not a locking order + <youpi> just that the critical section is actually busy + <gg0> youpi: ack + <gg0> braunr: what's the other bug? ext2fs one? + <braunr> gg0: idk + <gg0> braunr: thanks. doesn't show threads (found -T for that) but + at least doesn't limit columns number if piped (thanks to -w) + <braunr> it does + <braunr> there is a TH column + <gg0> ok thread count. -T gives more info + + IRC, freenode, #hurd, 2013-10-24: + + <youpi> ruby2.0 builds fine with the to-be-uploaded libc btw + <gg0> youpi: without d-ports patches? surprise me :) + <youpi> gg0: plain main archive source + <gg0> you did it. surprised + <gg0> ah ok you just pushed your tls. great! + <braunr> tls will fix a lot of things + + * `sigaltstack` + + IRC, freenode, #hurd, 2013-10-09: + + <gnu_srs1> Hi, is sigaltstack() really supported, even if it is + defined as well as SA_ONSTACK? + <braunr> probably not + <braunr> well, + <braunr> i don't know actually, mistaking with something else + <braunr> it may be supported + <pinotree> iirc no + <gnu_srs1> pinotree: are you sure? + <pinotree> this is what i remember + <pinotree> if you want to be sure that $foo works, just do the + usual way: test it yourself + <gnu_srs1> found it: hurd/TODO: *** does sigaltstack/sigstack + really work? -- NO + <pinotree> well TODO is old and there were signal-related patches + by jk in the meanwhile, although i don't think they would have + changed this lack + <pinotree> in any case, test it + <gnu_srs1> anybody fluent in assembly? Looks like this code + destroys the stack: http://paste.debian.net/54331/ + <braunr> gnu_srs1: why would it ? + <braunr> it does something special with the stack pointer but it + just looks like it aligns it to 16 bytes, maybe because of sse2 + restrictions (recent gcc align the stack already anyway) + <gnu_srs1> Well, in that case it is the called function: + http://paste.debian.net/54341/ + <braunr> how do you know there is a problem with the stack in the + first place ? + <gnu_srs1> tracing up to here, everything is OK. then esp and ebp + are destroyed. + <gnu_srs1> and single stepping goes backward until it segfaults + <braunr> "destroyed" ? + <gnu_srs1> zero if I remember correctly now. the x86 version built + for is i586, should that be changed to i486? + <braunr> this shouldn't change anything + <braunr> and they shouldn't get to 0 + <braunr> use gdb to determine exactly which instruction resets the + stack pointer + <gnu_srs1> how to step into the assembly part? using 's' steps + through the function since no line information: + <gnu_srs1> Single stepping until exit from function + wine_call_on_stack, + <gnu_srs1> which has no line number information. + <braunr> gnu_srs1: use break on the address + <gnu_srs1> how do i get the address of where the assembly starts? + * `recvmmsg`/`sendmmsg` (`t/sendmmsg`) From [[!message-id "20120625233206.C000A2C06F@topped-with-meat.com"]], @@ -1543,6 +1864,70 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 type for some flock structure field) <youpi> s/by/be/ + * `truncate`/`ftruncate` + + Hurd fixed with 2013-10-03 commit 6c3825f2b750bf9b913c6ea986739e648c470603, + glibc still to be done? + + IRC, freenode, #hurd, 2013-10-01: + + <pinotree> libdiskfs/node-drop.c: assert (np->dn_stat.st_size == 0); ← + this one? + <pinotree> iirc you constantly get that when building ustr + <braunr> is ustr a package ? + <pinotree> yes + <pinotree> iirc the ustr tests are mostly disk-intensive + + IRC, freenode, #hurd, 2013-10-02: + + <braunr> i've traced the problem up to truncate + <braunr> which gets a negative size + <braunr> shouldn't take long to find out where it comes from now + <pinotree> it seems our truncate doesn't handle negative values well though + <braunr> EINVAL The argument length is negative or larger than the + maximum file size. + <braunr> i still have to see whether it comes from the user (unlikely) or + if it's an internal inconsistency + <braunr> i suspect some code wrongly handles vm_map failures + <braunr> leading to that inconsistency + <braunr> pinotree: looks like glibc doesn't check for length >= 0 + <pinotree> yeah + <braunr> servers should do it nonetheless + <pinotree> should we fix glibc, libdiskfs/libnetfs/libtrivfs/etc, or both? + <braunr> it appears a client does the truncate + <braunr> i'd say both + <braunr> can you take the glibc part ? :) + <pinotree> i was going to do the hurd part... :p + <pinotree> ok, i'll pick libc + <braunr> well i'm doing it already + <braunr> i want to write a test case first + <braunr> to make sure that's the problem + <pinotree> already on the hurd part, you mean? + <braunr> yes + <pinotree> ok + <braunr> ok looks like it + <braunr> i can't reproduce the assertion but it does make ext2fs freeze + <braunr> pinotree: http://darnassus.sceen.net/~rbraun/test_ftruncate.c + <pinotree> merci + <braunr> pinotree: ustr builds + <pinotree> wow + <braunr> the client code (ustr) seems to perform a ftruncate with size + ((size_t)-1) whereas lengths are signed .. + <braunr> i'll check other libraries and send a patch soon + + IRC, freenode, #hurd, 2013-10-03: + + <braunr> youpi: i've committed a fix to hurd that checks for negative sizes + when truncating files + <braunr> this allows building the ustr package without making ext2fs choke + on an assertion + <braunr> pinotree is preparing a patch for glibc + <braunr> see truncate/ftruncate + <braunr> with an off_t size parameter, which can be negative + <braunr> EINVAL The argument length is negative or larger than the + maximum file size. + <braunr> hurd servers were not conforming to that before my change + * Verify baseline changes, if we need any follow-up changes: * a11ec63713ea3903c482dc907a108be404191a02 diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn index 7ce36f41..40d1463e 100644 --- a/open_issues/glibc/t/tls-threadvar.mdwn +++ b/open_issues/glibc/t/tls-threadvar.mdwn @@ -116,3 +116,40 @@ dropped altogether, and `__thread` directly be used in glibc. ## IRC, OFTC, #debian-hurd, 2013-09-23 <youpi> yay, errno threadvar conversion success + + +## IRC, OFTC, #debian-hurd, 2013-10-05 + + <gg0_> youpi: any ETA for tls? + <youpi> gg0_: one can't have an ETA for bugfixing + <gg0_> i don't call them bugs if there's something missing to implement btw + <youpi> no, here it's bugs + <youpi> the implementation is already in the glibc branches in our + repository + <youpi> it just makes some important regressions + + +## IRC, OFTC, #debian-hurd, 2013-10-07 + + <youpi> about tls, I've made some "progress": now I'm wondering how raise() + has ever been working before :) + + +## IRC, OFTC, #debian-hurd, 2013-10-15 + + <youpi> good, reply_port tls is now ok + <youpi> last but not least, sigstate + + +## IRC, OFTC, #debian-hurd, 2013-10-21 + + <youpi> started testsuite with threadvars dropped completely + <youpi> so far so good + + +## IRC, OFTC, #debian-hurd, 2013-10-24 + + <youpi> ok, hurd boots with full-tls libc, no threadvars at all any more + <gg0> \o/ + <gg0> good bye threadvars bugs, welcome tls ones ;) + <youpi> now I need to check that threads can really use another stack :) diff --git a/open_issues/gnumach_page_cache_policy.mdwn b/open_issues/gnumach_page_cache_policy.mdwn index 5e93887e..77e52ddb 100644 --- a/open_issues/gnumach_page_cache_policy.mdwn +++ b/open_issues/gnumach_page_cache_policy.mdwn @@ -811,3 +811,63 @@ License|/fdl]]."]]"""]] <braunr> have* <braunr> and even if laggy, it doesn't feel much more than the usual lag of a network (ssh) based session + + +# IRC, freenode, #hurd, 2013-10-08 + + <braunr> hmm i have to change what gnumach reports as being cached memory + + +## IRC, freenode, #hurd, 2013-10-09 + + <braunr> mhmm, i'm able to copy files as big as 256M while building debian + packages, using a gnumach kernel patched for maximum memory usage in the + page cache + <braunr> just because i used --sync=30 in ext2fs + <braunr> a bit of swapping (around 40M), no deadlock yet + <braunr> gitweb is a bit slow but that's about it + <braunr> that's quite impressive + <braunr> i suspect thread storms might not even be the cataclysmic event + that we thought it was + <braunr> the true problem might simply be parallel fs synces + + +## IRC, freenode, #hurd, 2013-10-10 + + <braunr> even with the page cache patch, memory filled, swap used, and lots + of cached objects (over 200k), darnassus is impressively resilient + <braunr> i really wonder whether we fixed ext2fs deadlock + + <braunr> youpi: fyi, darnassus is currently running a patched gnumach with + the vm cache changes, in hope of reproducing the assertion errors we had + in the past + <braunr> i increased the sync interval of ext2fs to 30s like we discussed a + few months back + <braunr> and for now, it has been very resilient, failing only because of + the lack of kernel map entries after several heavy package builds + <gg0> wait the latter wasn't a deadlock it resumed after 1363.06 s + <braunr> gg0: thread storms can sometimes (rarely) fade and let the system + resume "normally" + <braunr> which is why i increased the sync interval to 30s, this leaves + time between two intervals for normal operations + <braunr> otherwise writebacks are queued one after the other, and never + processed fast enough for that queue to become empty again (except + rarely) + <braunr> youpi: i think we should consider applying at least the sync + interval to exodar, since many DDs are just unaware of the potential + problems with large IOs + <youpi> sure + + <braunr> 222k cached objects (1G of cached memory) and darnassus is still + kicking :) + <braunr> youpi: those lock fixing patches your colleague sent last year + must have helped somewhere + <youpi> :) + + +## IRC, freenode, #hurd, 2013-10-13 + + <youpi> braunr: how are your tests going with the object cache? + <braunr> youpi: not so good + <braunr> youpi: it failed after 2 days of straight building without a + single error output :/ diff --git a/open_issues/hurd_101.mdwn b/open_issues/hurd_101.mdwn index 574a03ec..25822512 100644 --- a/open_issues/hurd_101.mdwn +++ b/open_issues/hurd_101.mdwn @@ -60,3 +60,41 @@ Not the first time that something like this is proposed... <neal> how ipc works <neal> and understand exactly what state is stored where <zacts> ok + + +# IRC, freenode, #hurd, 2013-10-12 + + <ahungry> Hi all, can anyone expand on + https://www.gnu.org/software/hurd/contributing.html - if I proceed with + the quick start and have the system running in a virtual image, how do I + go from there to being able to start tweaking the source (and recompiling + ) in a meaningful way? + <ahungry> Would I modify the source, compile within the VM and then what + would be the next step to actually test my new changes? + <braunr> ahungry: we use debian + <braunr> i suggest formatting your changes into patches, importing them + into debian packages, rebuilding those packages, and installing them over + the upstream ones + <ahungry> what about modifications to mach itself? or say I wanted to try + to work on the wifi drives - I would build the translator or module or + whatever and just add to the running instance of hurd? + <ahungry> s/drives/drivers + <braunr> same thing + <braunr> although + <braunr> during development, it's obviously a bit too expensive to rebuild + complete packages each time + <braunr> you can use the hurd on top of a gnumach kernel built completely + from upstream sources + <braunr> you need a few debian patches for the hurd itself + <braunr> a lot of them for glibc + <braunr> i usually create a temporary local branch with the debian patches + i need to make my code run + <braunr> and then create the true development branch itself from that one + <braunr> drivers are a a dark corner of the hurd + <braunr> i wouldn't recommend starting there + <braunr> but if you did, yes, you'd write a server to run drivers, and + start it + <braunr> you'd probably write a translator (which is a special kind of + server), yes + <ahungry> braunr: thanks for all the info, hittin the sack now but ill have + to set up a box and try to contribute diff --git a/open_issues/hurd_init.mdwn b/open_issues/hurd_init.mdwn index b0b58a70..cc06935c 100644 --- a/open_issues/hurd_init.mdwn +++ b/open_issues/hurd_init.mdwn @@ -214,3 +214,11 @@ License|/fdl]]."]]"""]] <teythoon> I've been hacking on init/startup, I've looked into cleaning it up + + +## IRC, freenode, #hurd, 2013-10-07 + + <teythoon> braunr: btw, what do you think of my /hurd/startup proposal? + <braunr> i haven't read it in detail yet + <braunr> it's about separating init right ? + <teythoon> yes diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn index 6f09ea0d..feea7c0d 100644 --- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn +++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn @@ -413,3 +413,67 @@ Address problem mentioned in [[/libpthread]], *Threads' Death*. <braunr> oh, git is multithreaded <braunr> great <braunr> so i've actually tested my libpthread patch quite a lot + + +## IRC, freenode, #hurd, 2013-09-25 + + <braunr> on a side note, i was able to build gnumach/libc/hurd packages + with thread destruction + <teythoon> nice :) + <braunr> they boot and work mostly fine, although they add their own issues + <braunr> e.g. the comm field of the root ext2fs is empty + <braunr> ps crashes when trying to display threads + <braunr> but thread destruction actually works, i.e. servers (those that + are configured that away at least) go away after some time, and even + heavily used servers such as ext2fs dynamically scale over time :) + + +## IRC, freenode, #hurd, 2013-10-10 + + <braunr> concerning threads, i think i figured out the last bugs i had with + thread destruction + <braunr> it should be well on its way to be merged by the end of the year + + +## IRC, freenode, #hurd, 2013-10-11 + + <gg0> braunr: is your thread destruction patch ready for testing? + <braunr> gg0: there are packages at my repository, yes + <braunr> but i still have hurd fixes to do before i polish it + <braunr> in particular, posix says returning from main() stops the entire + process and all other threads + <braunr> i didn't check that during the switch to pthreads, and ext2fs (and + maybe others) actually return from main but expect other threads to live + on + <braunr> this creates problems when the main thread is actually destroyed, + but not the process + <teythoon> braunr: tmpfs does something like that, but calls pthread_exit + at the end of main + <braunr> same effect + <braunr> this was fine with cthreads, but must be changed with pthreads + <braunr> and libpthread must be fixed to enforce it + <braunr> (or libc) + + <braunr> diskfs_startup_diskfs should probably be changed to reuse the main + thread instead of returning + + +## IRC, freenode, #hurd, 2013-10-19 + + <zacts> I know what threads are, but what is 'thread destruction'? + <braunr> the hurd currently never destroys individual threads + <braunr> they're destroyed when tasks are destroyed + <braunr> if the number of threads in a task peaks at a high number, say + thousands of them, they'll remain until the task is terminated + <braunr> such tasks are usually file systems, normally never restarted (and + in the case of the root file system, not restartable) + <braunr> this results in a form of leak + <braunr> another effect of this leak is that servers which should go away + because of inactivity still remain + <braunr> since thread destruction doesn't actually work, the debian package + uses a patch to prevent worker threads from timeouting + <braunr> and to finish with, since thread destruction actually doesn't + work, normal (unpatched) applications that destroy threads are certainly + failing bad + <braunr> i just need to polish a few things, wait for youpi to finish his + work on TLS to resolve conflicts, and that will be all diff --git a/open_issues/lsof.mdwn b/open_issues/lsof.mdwn index 2cbf2302..2651932d 100644 --- a/open_issues/lsof.mdwn +++ b/open_issues/lsof.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -11,3 +11,41 @@ License|/fdl]]."]]"""]] We don't have a `lsof` tool. Perhaps we could cook something with having a look at which ports are open at the moment (as [[`portinfo`|hurd/portinfo]] does, for example)? + + +# IRC, freenode, #hurd, 2013-10-16 + + <teythoon> braunr: there's something I've been working on, it's not yet + finished but usable + <teythoon> http://paste.debian.net/58266/ + <teythoon> it graphs port usage + <teythoon> it's a bit heavy on the dependency-side though... + <braunr> but + <braunr> is it able to link rights from different ipc spaces ? + <teythoon> no + <teythoon> what do you mean exactly? + <braunr> know that send right 123 in task 1 refers to receive right 321 in + task 2 + <braunr> basically, lsof + <braunr> i'm not sure it's possible right now, and that's what we'd really + need + <teythoon> does the kernel hand out this information? + <braunr> ^ + <teythoon> right, I'm not sure it's possible either + <braunr> but a graph maker in less than 300 is cute :) + <braunr> 300 lines* + <teythoon> well, it leverages pymatplotlib or something, it needs half of + the pythonverse ;) + <braunr> lsof and pmap and two tools we really lack on the hurd + <teythoon> what does portinfo --translate=PID do? + <braunr> i guess it asks proc so that ports that refer to task actually + give useful info + <braunr> hml + <braunr> no + <braunr> doesn't make sense to give a pid in this case + <braunr> teythoon: looks like it does what we talked about + <teythoon> :) + <braunr> teythoon: the output looks a bit weird anyway, i think we need to + look at the code to be sure + <teythoon> braunr: this is what aptitude update looks like: + https://teythoon.cryptobitch.de/portmonitor/aptitude_portmonitor.svg diff --git a/open_issues/mach-defpager_swap.mdwn b/open_issues/mach-defpager_swap.mdwn index 7d3b001c..6e4dc088 100644 --- a/open_issues/mach-defpager_swap.mdwn +++ b/open_issues/mach-defpager_swap.mdwn @@ -18,3 +18,24 @@ License|/fdl]]."]]"""]] <lifeng> I allocated a 5GB partition as swap, but hurd only found 1GB <youpi> use 2GiB swaps only, >2Gib are not supported <youpi> (and apparently it just truncates the size, to be investigated) + +## IRC, freenode, #hurd, 2013-10-25 + + <C-Keen> mkswap truncated the swap partiton to 2GB + <teythoon> :/ + <teythoon> have you checked with 'free' ? + <teythoon> I have a 4gb swap partition on one of my boxes + <C-Keen> how did you create it? + <C-Keen> 2gig swap alright + <C-Keen> according to free + + +# Swap Files + +## IRC, freenode, #hurd, 2013-10-25 + + <braunr> C-Keen: swapfiles are not to work very badly on the hurd + <braunr> swapfiles cause recursion and reservation problems on every system + but on the hurd, we just never took the time to fix the swap code + +Same issues as we generally would have with `hurd-defpager`? diff --git a/open_issues/multiprocessing.mdwn b/open_issues/multiprocessing.mdwn index 0ac7f195..eaaa2289 100644 --- a/open_issues/multiprocessing.mdwn +++ b/open_issues/multiprocessing.mdwn @@ -17,7 +17,7 @@ for applying multiprocessing. That is, however, only true from a first and inexperienced point of view: there are many difficulties. -IRC, freenode, #hurd, August / September 2010 +# IRC, freenode, #hurd, August / September 2010 <marcusb> silver_hook: because multi-server systems depend on inter-process communication, and inter-process communication is many times more @@ -32,7 +32,7 @@ IRC, freenode, #hurd, August / September 2010 serious research challenges -IRC, freenode, #hurd, 2011-07-26 +# IRC, freenode, #hurd, 2011-07-26 < braunr> 12:03 < CTKArcher> and does the hurd take more advantages in a multicore architecture than linux ? @@ -57,7 +57,7 @@ IRC, freenode, #hurd, 2011-07-26 < braunr> (here, thread migration means being dispatched on another cpu) -debian-hurd list +# debian-hurd list On Thu, Jan 02, 2003 at 05:40:00PM -0800, Thomas Bushnell, BSG wrote: > Georg Lehner writes: diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn index ae05e128..772fd865 100644 --- a/open_issues/performance.mdwn +++ b/open_issues/performance.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -44,6 +44,8 @@ call|/glibc/fork]]'s case. * [[metadata_caching]] + * [[community/gsoc/project_ideas/object_lookups]] + --- diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index cd39328f..05a58f2e 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -3031,3 +3031,13 @@ License|/fdl]]."]]"""]] <mcsim> so, add? <braunr> if that's what you want to do, ok <braunr> i'll think about your initial question tomorrow + + +## IRC, freenode, #hurd, 2013-09-30 + + <antrik> talking about which... did the clustered I/O work ever get + concluded? + <braunr> antrik: yes, mcsim was able to finish clustered pageins, and it's + still on my TODO list + <braunr> it will get merged eventually, now that the large store patch has + also been applied diff --git a/open_issues/performance/microkernel_multi-server.mdwn b/open_issues/performance/microkernel_multi-server.mdwn index 111d2b88..0382c835 100644 --- a/open_issues/performance/microkernel_multi-server.mdwn +++ b/open_issues/performance/microkernel_multi-server.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,7 +12,8 @@ License|/fdl]]."]]"""]] Performance issues due to the microkernel/multi-server system architecture? -IRC, freenode, #hurd, 2011-07-26 + +# IRC, freenode, #hurd, 2011-07-26 < CTKArcher> I read that, because of its microkernel+servers design, the hurd was slower than a monolithic kernel, is that confirmed ? @@ -45,3 +46,181 @@ IRC, freenode, #hurd, 2011-07-26 < braunr> but in 95, processors weren't that fast compared to other components as they are now < youpi> while disk/mem haven't evovled so fast + + +# IRC, freenode, #hurd, 2013-09-30 + + <snadge> ok.. i noticed when installing debian packages in X, the mouse + lagged a little bit + <snadge> that takes me back to classic linux days + <snadge> it could be a side effect of running under virtualisation who + knows + <braunr> no + <braunr> it's because of the difference of priorities between server and + client tasks + <snadge> is it simple enough to increase the priority of the X server? + <snadge> it does remind me of the early linux days.. people were more + interested in making things work, and making things not crash.. than + improving the desktop interactivity or responsiveness + <snadge> very low priority :P + <braunr> snadge: actually it's not the difference in priority, it's the + fact that some asynchronous processing is done at server side + <braunr> the priority difference just gives more time overall to servers + for that processing + <braunr> snadge: when i talk about servers, i mean system (hurd) servers, + no x + <snadge> yeah.. linux is the same.. in the sense that, that was its + priority and focus + <braunr> snadge: ? + <snadge> servers + <braunr> what are you talking about ? + <snadge> going back 10 years or so.. linux had very poor desktop + performance + <braunr> i'm not talking about priorities for developers + <snadge> it has obviously improved significantly + <braunr> i'm talking about things like nice values + <snadge> right.. and some of the modifications that have been done to + improve interactivity of an X desktop, are not relevant to servers + <braunr> not relevant at all since it's a hurd problem, not an x problem + <snadge> yeah.. that was more of a linux problem too, some time ago was the + only real point i was making.. a redundant one :p + <snadge> where i was going with that.. was desktop interactivity is not a + focus for hurd at this time + <braunr> it's not "desktop interactivity" + <braunr> it's just correct scheduling + <snadge> is it "correct" though.. the scheduler in linux is configurable, + and selectable + <snadge> depending on the type of workload you expect to be doing + <braunr> not really + <snadge> it can be interactive, for desktop loads.. or more batched, for + server type loads.. is my basic understanding + <braunr> no + <braunr> that's the scheduling policy + <braunr> the scheduler is cfs currently + <braunr> and that's the main difference + <braunr> cfs means completely fair + <braunr> whereas back in 2.4 and before, it was a multilevel feedback + scheduler + <braunr> i.e. a scheduler with a lot of heuristics + <braunr> the gnumach scheduler is similar, since it was the standard + practice from unix v6 at the time + <braunr> (gnumach code base comes from bsd) + <braunr> so 1/ we would need a completely fair scheduler too + <braunr> and 2/ we need to remove asynchronous processing by using mostly + synchronous rpc + <snadge> im just trying to appreciate the difference between async and sync + event processing + <braunr> on unix, the only thing asynchronous is signals + <braunr> on the hurd, simply cancelling select() can cause many + asynchronous notifications at the server to remove now unneeded resources + <braunr> when i say cancelling select, i mean one or more fds now have + pending events, and the others must be cleaned + <snadge> yep.. thats a pretty fundamental change though isnt it? .. if im + following you, you're talking about every X event.. so mouse move, + keyboard press etc etc etc + <snadge> instead of being handled async.. you're polling for them at some + sort of timing interval? + <snadge> never mind.. i just read about async and sync with regards to rpc, + and feel like a bit of a noob + <snadge> async provides a callback, sync waits for the result.. got it :p + <snadge> async is resource intensive on hurd for the above mentioned + reasons.. makes sense now + <snadge> how about optimising the situation where a select is cancelled, + and deferring the signal to the server to clean up resources until a + later time? + <snadge> so like java.. dont clean up, just make a mess + <snadge> then spend lots of time later trying to clean it up.. sounds like + my life ;) + <snadge> reuse stale objects instead of destroying and recreating them, and + all the problems associated with that + <snadge> but if you're going to all these lengths to avoid sending messages + between processes + <snadge> then you may as well just use linux? :P + <snadge> im still trying to wrap my head around how converting X to use + synchronous rpc calls will improve responsiveness + <pinotree> what has X to do with it? + <snadge> nothing wrong with X.. braunr just mentioned that hurd doesnt + really handle the async calls so well + <snadge> there is more overhead.. that it would be more efficient on hurd, + if it uses sync rpc instead + <snadge> and perhaps a different task scheduler would help also + <snadge> ala cfs + <snadge> but i dont think anyone is terribly motivated in turning hurd into + a desktop operating system just yet.. but i could be wrong ;) + <braunr> i didn't say that + <snadge> i misinterpreted what you said then .. im not surprised, im a + linux sysadmin by trade.. and have basic university OS understanding (ie + crap all) at a hobbyist level + <braunr> i said there is asynchronous processing (i.e. server still have + work to do even when there is no client) + <braunr> that processing mostly comes from select requests cancelling what + they installed + <braunr> ie.e. you select fd 1 2 3, even on 2, you cancel on 1 and 3 + <braunr> those cancellations aren't synchronous + <braunr> the client deletes ports, and the server asynchronously receives + dead name notifications + <braunr> since servers have a greater priority, these notifications are + processed before the client can continue + <braunr> which is what makes you feel lag + <braunr> X is actually a client here + <braunr> when i say server, i mean hurd servers + <braunr> the stuff implementing sockets and files + <braunr> also, you don't need to turn the hurd into a desktop os + <braunr> any correct way to do fair scheduling will do + <snadge> can the X client be made to have a higher priority than the hurd + servers? + <snadge> or perhaps something can be added to hurd to interface with X + <azeem_> well, the future is wayland + <snadge> ufs .. unfair scheduling.. give priority to X over everything else + <snadge> hurd almost seams ideal for that idea.. since the majority of the + system is seperated from the kernel + <snadge> im likely very wrong though :p + <braunr> snadge: the reason we elevated the priority of servers is to avoid + delaying the processing of notifications + <braunr> because each notification can spawn a server thread + <braunr> and this lead to cases where processing notifications was so slow + that spawning threads would occur more frequently, leading to the server + exhausting its address space because of thread stacks + <snadge> cant it wait for X though? .. or does it lead to that situation + you just described + <braunr> we should never need such special cases + <braunr> we should remove async notifications + <snadge> my logic is this.. if you're not running X then it doesnt + matter.. if you are, then it might.. its sort of up to you whether you + want priority over your desktop interface or whether it can wait for more + important things, which creates perceptible lag + <braunr> snadge: no it doesn't + <braunr> X is clearly not the only process involved + <braunr> the whole chain should act synchronously + <braunr> from the client through the server through the drivers, including + the file system and sockets, and everything that is required + <braunr> it's a general problem, not specific to X + <snadge> right.. from googling around, it looks like people get very + excited about asyncronous + <snadge> there was a move to that for some reason.. it sounds great in + theory + <snadge> continue processing something else whilst you wait for a + potentially time consuming process.. and continue processing that when + you get the result + <snadge> its also the only way to improve performance with parallelism? + <snadge> which is of no concern to hurd at this time + <braunr> snadge: please don't much such statements when you don't know what + you're talking about + <braunr> it is a concern + <braunr> and yes, async processing is a way to improve performance + <braunr> but don't mistake async rpc and async processing + <braunr> async rpc simply means you can send and receive at any time + <braunr> sync means you need to recv right after send, blocking until a + reply arrives + <braunr> the key word here is *blocking*ù + <snadge> okay sure.. that makes sense + <snadge> what is the disadvantage to doing it that way? + <snadge> you potentially have more processes that are blocking? + <braunr> a system implementing posix such as the hurd needs signals + <braunr> and some event handling facility like select + <braunr> implementing them synchronously means a thread ready to service + these events + <braunr> the hurd currently has such a message thread + <braunr> but it's complicated and also a scalability concern + <braunr> e.g. you have at least two thread per process + <braunr> bbl diff --git a/open_issues/pthread_atfork.mdwn b/open_issues/pthread_atfork.mdwn index 1b656f05..06b9d6c6 100644 --- a/open_issues/pthread_atfork.mdwn +++ b/open_issues/pthread_atfork.mdwn @@ -18,3 +18,89 @@ can probably be borrowed from `nptl/sysdeps/unix/sysv/linux/register-atfork.c`. <pinotree> SRCDIR/opal/mca/memory/linux/arena.c:387: warning: warning: pthread_atfork is not implemented and will always fail + + +# Samuel's implementation + +TODO. + + +## IRC, OFTC, #debian-hurd, 2013-10-08 + + <pinotree> youpi: if you need/want to test your pthread_atfork + implementation, you can check libposix-atfork-perl and its test suite + (whose test 004 hangs now, with eglibc -93) + <youpi> while it failed previously indeed + <youpi> we might simply need to rebuild perl against it + <youpi> (I see ifdef pthread_atfork in perl) + + +## IRC, freenode, #hurd, 2013-10-16 + + <teythoon> tschwinge: I'd love to try your cross-gnu tool, the wiki page + suggests that the list of required source packages is outdated. can you + give me some hints? + <teythoon> tschwinge: I got this error running cross-gnu: + http://paste.debian.net/58303/ + make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/setjmp' + make subdir=string -C ../string ..=../ objdir=/home/teythoon/repos/hurd/cross/obj/glibc -f Makefile -f ../elf/rtld-Rules rtld-all rtld-modules='rtld-strchr.os rtld-strcmp.os rtld-strcpy.os rtld-strlen.os rtld-strnlen.os rtld-memchr.os rtld-memcmp.os rtld-memmove.os rtld-memset.os rtld-mempcpy.os rtld-stpcpy.os rtld-memcpy.os rtld-rawmemchr.os rtld-argz-count.os rtld-argz-extract.os rtld-stpncpy.os' + make[4]: Entering directory `/home/teythoon/repos/hurd/cross/src/glibc/string' + make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/string' + make[4]: Entering directory `/home/teythoon/repos/hurd/cross/src/glibc/string' + make[4]: Nothing to be done for `rtld-all'. + make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/string' + make[3]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/elf' + i686-pc-gnu-gcc -shared -static-libgcc -Wl,-O1 -Wl,-z,defs -Wl,-dynamic-linker=/lib/ld.so.1 -B/home/teythoon/repos/hurd/cross/obj/glibc/csu/ -Wl,--version-script=/home/teythoon/repos/hurd/cross/obj/glibc/libc.map -Wl,-soname=libc.so.0.3 -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -nostdlib -nostartfiles -e __libc_main -L/home/teythoon/repos/hurd/cross/obj/glibc -L/home/teythoon/repos/hurd/cross/obj/glibc/math -L/home/teythoon/repos/hurd/cross/obj/glibc/elf -L/home/teythoon/repos/hurd/cross/obj/glibc/dlfcn -L/home/teythoon/repos/hurd/cross/obj/glibc/nss -L/home/teythoon/repos/hurd/cross/obj/glibc/nis -L/home/teythoon/repos/hurd/cross/obj/glibc/rt -L/home/teythoon/repos/hurd/cross/obj/glibc/resolv -L/home/teythoon/repos/hurd/cross/obj/glibc/crypt -L/home/teythoon/repos/hurd/cross/obj/glibc/mach -L/home/teythoon/repos/hurd/cross/obj/glibc/hurd -Wl,-rpath-link=/home/teythoon/repos/hurd/cross/obj/glibc:/home/teythoon/repos/hurd/cross/obj/glibc/math:/home/teythoon/repos/hurd/cross/obj/glibc/elf:/home/teythoon/repos/hurd/cross/obj/glibc/dlfcn:/home/teythoon/repos/hurd/cross/obj/glibc/nss:/home/teythoon/repos/hurd/cross/obj/glibc/nis:/home/teythoon/repos/hurd/cross/obj/glibc/rt:/home/teythoon/repos/hurd/cross/obj/glibc/resolv:/home/teythoon/repos/hurd/cross/obj/glibc/crypt:/home/teythoon/repos/hurd/cross/obj/glibc/mach:/home/teythoon/repos/hurd/cross/obj/glibc/hurd -o /home/teythoon/repos/hurd/cross/obj/glibc/libc.so -T /home/teythoon/repos/hurd/cross/obj/glibc/shlib.lds /home/teythoon/repos/hurd/cross/obj/glibc/csu/abi-note.o /home/teythoon/repos/hurd/cross/obj/glibc/elf/soinit.os /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/sofini.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/interp.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/ld.so /home/teythoon/repos/hurd/cross/obj/glibc/mach/libmachuser-link.so /home/teythoon/repos/hurd/cross/obj/glibc/hurd/libhurduser-link.so -lgcc + /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os: In function `__fork': + /home/teythoon/repos/hurd/cross/src/glibc/posix/../sysdeps/mach/hurd/fork.c:70: undefined reference to `__start__hurd_atfork_prepare_hook' + /home/teythoon/repos/hurd/cross/lib/gcc/i686-pc-gnu/4.8.1/../../../../i686-pc-gnu/bin/ld: /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os: relocation R_386_GOTOFF against undefined hidden symbol `__start__hurd_atfork_prepare_hook' can not be used when making a shared object + /home/teythoon/repos/hurd/cross/lib/gcc/i686-pc-gnu/4.8.1/../../../../i686-pc-gnu/bin/ld: final link failed: Bad value + collect2: error: ld returned 1 exit status + make[2]: *** [/home/teythoon/repos/hurd/cross/obj/glibc/libc.so] Error 1 + make[2]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/elf' + make[1]: *** [elf/subdir_lib] Error 2 + make[1]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc' + make: *** [all] Error 2 + + rm -f /home/teythoon/repos/hurd/cross/sys_root/lib/ld.so + + exit 100 + + binutils-2.23.2, + gcc-4.8.1, + everything else is from git as specified in the wiki. + + +## IRC, freenode, #hurd, 2013-10-24 + + <AliciaC> in recent glibc commits (tschwinge/Roger_Whittaker branch) there + are references to _hurd_atfork_* symbols in sysdeps/mach/hurd/fork.c, and + some _hurd_fork_* symbols, some of the _hurd_fork_* symbols seem to be + defined in Hurd's boot/frankemul.ld (mostly guessing by their names being + mentioned, I don't know linker script syntax), but those _hurd_atfork_* + symbols don't seem to be defined there, are they supposed to be defined + elsewhere or is th + <AliciaC> does anyone know where the _hurd_atfork_* group of symbols + referenced in glibc are defined (if anywhere)? + <youpi> AliciaC: it's the DEFINE_HOOK (_hurd_atfork_prepare_hook, (void)); + in glibc/sysdeps/mach/hurd/fork.c + <AliciaC> hm, is that not just a declaration? + <youpi> no, it's a definition, as its name suggests : + <AliciaC> (despite the macro name) + <youpi> :) + <AliciaC> ok + <AliciaC> I should look into it more, I could have sworn I was getting + undefined references, but maybe the symbol names used are different from + those defined, but that'd be odd as well, in the same file and all + <AliciaC> I mean, I do get undefined references, but question is if it's to + things that should have been defined or not + <youpi> what undefined references do you gaT? + <youpi> s/gaT/get + <AliciaC> I'll get back to you once I have that system up again + <AliciaC> youpi: sysdeps/mach/hurd/fork.c:70: undefined reference to + `__start__hurd_atfork_prepare_hook' + <AliciaC> fork.c:70: 'RUN_HOOK (_hurd_atfork_prepare_hook, ());' + <AliciaC> DEFINE_HOOK (_hurd_atfork_prepare_hook, (void)); is higher up in + the file + <AliciaC> though there is also this message: build/libc_pic.os: relocation + R_386_GOTOFF against undefined hidden symbol + `__start__hurd_atfork_prepare_hook' can not be used when making a shared + object diff --git a/open_issues/smp.mdwn b/open_issues/smp.mdwn index a45a1e22..89474d25 100644 --- a/open_issues/smp.mdwn +++ b/open_issues/smp.mdwn @@ -37,3 +37,11 @@ See also the [[FAQ entry|faq/smp]]. ## Richard, 2013-03-20 This task actually looks too big for a GSoC project. + + +## IRC, freenode, #hurd, 2013-09-30 + + <braunr> also, while the problem with hurd is about I/O, it's actually a + lot more about caching, and even with more data cached in, the true + problem is contention, in which case having several processors would + actually slow things down even more diff --git a/open_issues/strict_aliasing.mdwn b/open_issues/strict_aliasing.mdwn index b7d39805..0e59f796 100644 --- a/open_issues/strict_aliasing.mdwn +++ b/open_issues/strict_aliasing.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -29,3 +29,16 @@ License|/fdl]]."]]"""]] issues (if gcc catches them all) <tschwinge> The strict aliasing things should be fixed, yes. Some might be from MIG. + + +# IRC, freenode, #hurd, 2013-10-17 + + <braunr> we should build gnumach and the hurd with -fno-strict-aliasing + <pinotree> aren't the mig-generated stubs the only issues related to that? + <braunr> no + <teythoon> b/c we often have pointers of different type pointing to the + same address? for example code using libports? + <braunr> the old linux code, including pfinet, and even the hurd libraries, + use techniques that assume aliasing + <braunr> exactly + <teythoon> right, I agree diff --git a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn index 7159551d..f40e0455 100644 --- a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn +++ b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn @@ -50,3 +50,5 @@ IRC, unknown channel, unknown date: result in others trying to take it... <youpi> nope: look at the code :) <youpi> or maybe the cancel_hook, but I really doubt it + +See discussion about *`critical_section_lock`* on [[glibc]]. diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn index 367db872..d9f1fa1d 100644 --- a/open_issues/time.mdwn +++ b/open_issues/time.mdwn @@ -837,3 +837,17 @@ not get a define for `HZ`, which is then defined with a fallback value of 60. <nalaginrut> braunr: Guile2 works smoothly now, let me try something cool with it <braunr> nalaginrut: nice + + +### IRC, OFTC, #debian-hurd, 2013-09-29 + + <pinotree> youpi: is the latest glibc carrying the changes related to + timing? what about gb guile-2.0 with it? + <youpi> it does + <youpi> so that was the only issue with guile? + <youpi> well at least we'll see + <pinotree> iirc yes + <pinotree> according to nalaginrut and the latest build log, it'd seem so + <youpi> started + <youpi> yay, guile-2.0 :) + <pinotree> yay diff --git a/open_issues/glibc/0.4.mdwn b/open_issues/versioning.mdwn index f864469d..69a6cb20 100644 --- a/open_issues/glibc/0.4.mdwn +++ b/open_issues/versioning.mdwn @@ -8,12 +8,47 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_glibc open_issue_libpthread]] +Things to consider regarding *versioning*. + +The provider and user of any interface need to agree about how to interpret the +data being exchanged. Internal-only interfaces can be changed easily, because +you can change the provider and user at the same time. Interfaces that are +exposed externally require more attention, for obvious reasons. To *change* +interfaces means to either remove, or add, or modify an existing interface. +Modify basically means to remove and then re-add a variant, re-using the former +name/identifier. + + +# [[RPC]]s + +## [[microkernel/mach/message/msgh_id]] + + +# Shared Libraries + + * [[!wikipedia soname]] + * ELF symbol versioning + * [[!wikipedia "GNU Libtool"]] -Things to consider doing when bumping the glibc SONAME. +## Hurd -# IRC, freenode, #hurd, 2012-12-14 +Transition to "normal" ELF symbol versioning/libtool? + +For all libraries, the SONAME is currently set to *0.3*. [[!message-id +desc="Not changed" "87ob7cxbu6.fsf@kepler.schwinge.homeip.net"]] when doing the +[[Hurd 0.5 release|news/2013-09-27]]. + + +## glibc + +Bump the glibc SONAME to some point, or can do everything with symbol +versioning? + + +### IRC, freenode, #hurd, 2012-12-14 + +[[!tag open_issue_glibc open_issue_libpthread]] In context of [[packaging_libpthread]]/[[libpthread]]. diff --git a/open_issues/wine.mdwn b/open_issues/wine.mdwn index 65e6c584..f8bb469b 100644 --- a/open_issues/wine.mdwn +++ b/open_issues/wine.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -21,7 +22,7 @@ requirements Wine has: only libc / POSIX / etc., or if there are allocation. There is kernel support for this,* however. -IRC, freenode, #hurd, 2011-08-11 +# IRC, freenode, #hurd, 2011-08-11 < arethusa> I've been trying to make Wine work inside a Debian GNU/Hurd VM, and to that end, I've successfully compiled the latest sources from Git @@ -67,3 +68,13 @@ IRC, freenode, #hurd, 2011-08-11 < youpi> yes < pinotree> (but that patch is lame) + + +# IRC, freenode, #hurd, 2013-10-02 + + <gnu_srs> youpi: I've come a little further with wine, see debian bug + #724681 (same problem). + <gnu_srs> Now the problem is probably due to the specific address space + and stack issues to be + <gnu_srs> fixed for wine to run as braunr pointed out some months ago + (IRC?) when we discussed wine. diff --git a/unix/process.mdwn b/unix/process.mdwn index 21fbfc69..82034751 100644 --- a/unix/process.mdwn +++ b/unix/process.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,13 +8,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -A *UNIX process* is TODO. +A *UNIX process* is a program in execution, that is, an instance running in an +execution context. Generally, especially in [[microkernel]]-based systems, the [[kernel]]'s idea of a task is not as encompassing as a UNIX process, and will use additional effort to enhance the kernel's primitive to a full-fledged UNIX model. - -A [[Mach task|microkernel/mach/task]] implements a part of a UNIX process. - In the GNU/Hurd, processes are based on [[Mach task|microkernel/mach/task]]s, but are [[enhanced by the glibc|glibc/process]]. |